Embed libmir
libmir is the model-neutral Rust inference library used by MiRMiR. It
discovers Hugging Face-style checkpoints, loads a native accelerator backend,
creates independent inference sessions, and exposes generation, tool calling,
embeddings, reranking, progress, cancellation, metrics, and memory information.
It deliberately does not contain:
- CLI argument or environment parsing;
- an HTTP, gRPC, or WebSocket server;
- a terminal or browser interface;
- Hugging Face search and download policy;
- application configuration files.
Those responsibilities belong to an embedding application such as MiRMiR.
Public lifecycle
ModelDescriptor::inspect(path)
|
v
Library::new(RuntimeConfig)
|
v
Library::load(path) -> Model
|
+-- Model::generate(&GenerationRequest, ...)
+-- Model::embed(...)
+-- Model::rerank(...)
+-- Model::session() -> prefill / decode
+-- Model::unload()
Applications should depend on the top-level libmir package and import from
its root. The internal workspace crates, including the CUDA and Metal adapters
and their configuration types, are implementation details and are not
re-exported; backend settings are adjusted through RuntimeConfig fields (see
Metal and CUDA).
Choose an entry point
| Need | API |
|---|---|
| Inspect without allocating accelerator weights | ModelDescriptor::inspect |
| Load a checkpoint | Library::load |
| High-level chat generation | Model::generate |
| Tool calls and schema-constrained arguments | Conversation::tools, GenerationRequest::tool_constraints |
| JSON content | GenerationRequest::response_format |
| Laya decisions | decision::DecisionModel on CPU, Metal, or CUDA |
| Cooperative cancellation | Model::generate_cancellable |
| Single-image generation | Model::generate_image_cancellable |
| Dense vectors | Model::embed |
| Document scoring (cross-encoder or causal) | Model::rerank |
| Manual token loop | Model::prepare, Model::session, Session::prefill, Session::decode |
| Capacity planning | ModelDescriptor::memory_estimate, Library::memory_snapshot |
Start with Add libmir to a project.