Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Native inference, from app to library

MiRMiR 0.2.0 is a native local model runtime with a terminal dashboard, an optional browser dashboard, a command-line interface, and an OpenAI-compatible HTTP server. libmir is the embeddable Rust inference library underneath it.

Use these docs to operate MiRMiR or build your own native application on the same model discovery, loading, generation, K/V cache, Metal, and CUDA layers.

Run MiRMiRInstall the 0.2.0 binary, download a compatible checkpoint, and start a local chat. Use the HTTP APIConnect existing OpenAI clients to chat, embeddings, reranking, and streaming endpoints. Embed libmirLoad a checkpoint and stream native inference directly from a Rust program. See the runtimeManage models and inspect throughput, latency, memory, K/V cache, and activity.

Choose a surface

SurfaceUse it when
TUIYou want an interactive terminal workflow with no browser.
Web dashboardYou want the same local runtime controls and telemetry in a browser.
CLIYou need scripts, machine-readable output, or repeatable benchmarks.
HTTP APIAn existing application already understands OpenAI-style requests.
libmirA Rust application needs direct ownership of models and sessions.

MiRMiR and libmir are pre-1.0. These pages describe version 0.2.0 and call out current limitations rather than treating planned functionality as available.

The stack

your application
      |
      +-- HTTP / SSE --> mirmir server --+
      +-- TUI / CLI --> private gRPC ----+--> libmir --> Metal / CUDA
      +-- Rust API -----------------------+

MiRMiR owns the application, configuration, transport, dashboards, model catalog, and telemetry. libmir owns checkpoint discovery, tokenization, generation semantics, sessions, scheduling, cache policy, and backend adapters.