Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Native inference, from app to library

MiRMiR 0.6.2 is a native local model runtime with a terminal dashboard, an optional browser dashboard, a command-line interface, and an OpenAI-compatible HTTP server. libmir is the embeddable Rust inference library underneath it.

Use these docs to operate MiRMiR or build your own native application on the same model discovery, loading, generation, K/V cache, Metal, and CUDA layers.

See what changes in 0.6.2: faster Metal and more models on it. In 0.6.1 each model’s context is chosen before it loads, so several models fit side by side. 0.6.0 added remote access, Mirmir for Mac and images for Nemotron Omni on CUDA.

Choose a surface

SurfaceUse it when
TUIYou want an interactive terminal workflow with no browser.
Web dashboardYou want the same local runtime controls and telemetry in a browser.
CLIYou need scripts, machine-readable output, or repeatable benchmarks.
HTTP APIAn existing application already understands OpenAI-style requests.
libmirA Rust application needs direct ownership of models and sessions.

MiRMiR and libmir are pre-1.0. These pages describe version 0.6.2. Support claims require complete checkpoint and backend validation; planned formats are not described as available.

The stack

your application
      |
      +-- HTTP / SSE --> mirmir server --+
      +-- TUI / CLI --> private gRPC ----+--> libmir --> Metal / CUDA
      +-- Rust API -----------------------+

MiRMiR owns the application, configuration, transport, dashboards, model catalog, and telemetry. Interactive and scripted clients attach to the same local runtime or start a temporary one when needed. libmir owns checkpoint discovery, tokenization, generation semantics, sessions, scheduling, cache policy, and backend adapters. Laya decisions use a separate typed API backed by Metal, CUDA, or the native CPU module mircup.