Upgrade to 0.6.1
MiRMiR 0.6.1 lets you choose each model’s context before it loads, so several models fit in memory side by side. All native crates and Mirmir for Mac move to 0.6.1 together.
cargo install mirmir --version 0.6.1 --locked
mirmir --version
Restart the runtime after upgrading.
What changes
- Context per model. The TUI’s load dialog, the web dashboard and Mirmir
for Mac show a Context control before Load: automatic, or a token count
(the K/V cache shared by the model’s concurrent requests), with the memory
it needs. The choice is
saved for the model and used by every load. From the CLI:
mirmir model load <model> --context 32768. See Configuration. - Loaded models report real memory. Inspecting a loaded model no longer shows a 16-token cache; it estimates a reload with the model’s own memory counted as free.
- Bounded generation settings. Max tokens, temperature, top P, top K and repetition penalty are sliders with fixed ranges in every client; max tokens is limited by the chosen context.
- Saved context survives a new download. Pulling a newer snapshot of a model keeps its saved generation settings and context.
- Mirmir for Mac. A download in progress shows its progress instead of an error. Removing a model from disk and selecting a model on a remote server no longer crash the app.
gRPC clients
The protocol stays version 2. ModelContextChoice and ModelContext are new;
InspectModelRequest, InspectModelResponse, LoadModelRequest and
UpdateModelGenerationRequest gain an optional context. A 0.6.0 server
sends no context in its inspection; clients hide the control then.
Rust consumers
libmir::Library::preview_load resolves what a load would use without
changing the library, and libmir::LoadPreview holds the result.
Library::model_config_with_options no longer replaces the library’s cached
configuration.