Inspect and load models

Inspection is separate from accelerator allocation. Use it to reject an unsupported checkpoint or present its capabilities before committing device memory.

use libmir::{GenerationOverrides, ModelDescriptor};

let descriptor = ModelDescriptor::inspect(
    "/models/qwen",
    GenerationOverrides::default(),
)?;
let manifest = descriptor.manifest()?;

println!("id={}", manifest.id);
println!("context={}", manifest.context_len);
println!("task={:?}", descriptor.task());
println!("quantization={:?}", manifest.quantization);
println!("generation={:?}", descriptor.generation());
Ok::<(), libmir::Error>(())

Discovery is based on configuration values, tokenizer assets, tensor names and shapes, and the resulting execution contract. A recognizable repository or architecture name alone does not admit a checkpoint.

Estimate memory

use libmir::{GenerationOverrides, ModelDescriptor, RuntimeConfig};

let descriptor = ModelDescriptor::inspect(
    "/models/qwen",
    GenerationOverrides::default(),
)?;
let estimate = descriptor.memory_estimate(&RuntimeConfig::default());

println!("weights={} bytes", estimate.weight_bytes);
println!("kv={} bytes", estimate.kv_cache_bytes);
println!("workspace={} bytes", estimate.workspace_bytes);
println!("required={} bytes", estimate.required_bytes);
Ok::<(), libmir::Error>(())

This is a conservative planning value, not a reservation. Compare it with Library::memory_snapshot() on the backend that will load the model.

Load and unload

use libmir::{GenerationOverrides, Library, RuntimeConfig};

let library = Library::new(RuntimeConfig::default());
let model = library.load(
    "/models/qwen",
    GenerationOverrides::default(),
    &mut |event| eprintln!("{event:?}"),
)?;

println!("backend={:?}", model.engine().target());
model.unload()?;
Ok::<(), libmir::Error>(())

Model is cheaply cloneable. Model::unload succeeds only when the value is the sole remaining owner; active clones or sessions produce Error::ModelInUse. Dropping a Session releases its backend and K/V resources automatically.