Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Embed libmir

libmir is the model-neutral Rust inference library used by MiRMiR. It discovers Hugging Face-style checkpoints, loads a native accelerator backend, creates independent inference sessions, and exposes generation, tool calling, embeddings, reranking, progress, cancellation, metrics, and memory information.

It deliberately does not contain:

  • CLI argument or environment parsing;
  • an HTTP, gRPC, or WebSocket server;
  • a terminal or browser interface;
  • Hugging Face search and download policy;
  • application configuration files.

Those responsibilities belong to an embedding application such as MiRMiR.

Public lifecycle

ModelDescriptor::inspect(path)
        |
        v
Library::new(RuntimeConfig)
        |
        v
Library::load(path) -> Model
        |
        +-- Model::generate(&GenerationRequest, ...)
        +-- Model::embed(...)
        +-- Model::rerank(...)
        +-- Model::session() -> prefill / decode
        +-- Model::unload()

Applications should depend on the top-level libmir package and import from its root. The internal workspace crates, including the CUDA and Metal adapters and their configuration types, are implementation details and are not re-exported; backend settings are adjusted through RuntimeConfig fields (see Metal and CUDA).

Choose an entry point

NeedAPI
Inspect without allocating accelerator weightsModelDescriptor::inspect
Load a checkpointLibrary::load
High-level chat generationModel::generate
Tool calls and schema-constrained argumentsConversation::tools, GenerationRequest::tool_constraints
JSON contentGenerationRequest::response_format
Laya decisionsdecision::DecisionModel on CPU, Metal, or CUDA
Cooperative cancellationModel::generate_cancellable
Single-image generationModel::generate_image_cancellable
Dense vectorsModel::embed
Document scoring (cross-encoder or causal)Model::rerank
Manual token loopModel::prepare, Model::session, Session::prefill, Session::decode
Capacity planningModelDescriptor::memory_estimate, Library::memory_snapshot

Start with Add libmir to a project.