Upgrade from 0.3
MiRMiR 0.4.0 keeps the 0.3 control surfaces, CLI, configuration keys, and
private protocol. It changes how the runtime is organized internally, makes the
OpenAI-compatible API stricter and more explicit, adds tool calling and
per-request reasoning control, and narrows the libmir facade. Version 0.3.1
was never published; upgrade directly from 0.3.0.
HTTP clients
Review clients that relied on silently ignored fields or on the 0.3 status codes:
- A non-zero
presence_penaltyorfrequency_penalty, and any non-nullreasoning_budgetorreasoning_effort, now return400. Omit them, or userepetition_penaltyand the MiRMiRreasoningextension instead. tool_choicemust beauto,none,required, or a named function.- A prompt that exceeds the model context, or can never fit the K/V cache, is
now
400instead of500. A request that could fit later remains429. - A cancelled operation returns
408 cancelled. - A tool call that violates
toolsortool_choice, or is malformed or truncated, returns422 invalid_model_output. - Inference endpoints return
503until startup model restoration completes. Point readiness probes at the newGET /ready; keepGET /healthfor liveness. - Closing an SSE connection now cancels the generation on the server.
New request capabilities are opt-in: tools with validated tool_calls
responses, tool_constraints: "schema" on Metal and CUDA, reasoning and
chat_template_kwargs.enable_thinking, thinking_token_budget,
return_token_ids for streams, and top_n for reranking. See
Chat completions and
Embeddings and reranking.
Configuration
Existing config.toml files remain valid; no key was renamed. New optional
runtime keys tune scheduling: prefill_batch_wait_us,
cached_prefill_policy, prefill_decode_policy, prefill_refill_policy, and,
on macOS, metal_decode_reservation. Omitting them keeps the backend defaults.
See Configuration.
Models
Decoder rerankers that ship a sentence-transformers LogitScore module are now
admitted as causal rerankers on Metal and CUDA. Qwen XML tool calls are parsed
natively. Re-run mirmir model inspect after upgrading, because admission
reports can change between releases.
libmir embedders
- Depend on
libmir = "0.4.0". The facade is now the only public entry point: thefoundation,models, andruntimemodules and the backend-specific CUDA and Metal types are no longer re-exported. ChatCompletionRequestand theChat*message types are replaced byGenerationRequest,Conversation,Message,Tool,ToolCall, and theToolChoiceenum.Model::generate*takes&GenerationRequest, andModel::preparetakes&Conversation.ProgressEventis an enum withstage(),unit(),count(), anddetail()accessors.SchedulerConfigandGenerationTokengained fields; construct them with..Default::default()or update struct literals.
See Generate and stream for the new request shape.
Upgrade checklist
- Install 0.4.1 with
cargo install mirmir --version 0.4.1 --lockedand runmirmir --version. - Run
mirmir config validateagainst the existing configuration. - Remove unsupported penalty and reasoning-effort fields from API clients and
handle
408,422, and503responses. - Switch readiness checks to
GET /ready. - Inspect each checkpoint on its target Metal or CUDA machine.