Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Upgrade from 0.3

MiRMiR 0.4.0 keeps the 0.3 control surfaces, CLI, configuration keys, and private protocol. It changes how the runtime is organized internally, makes the OpenAI-compatible API stricter and more explicit, adds tool calling and per-request reasoning control, and narrows the libmir facade. Version 0.3.1 was never published; upgrade directly from 0.3.0.

HTTP clients

Review clients that relied on silently ignored fields or on the 0.3 status codes:

  • A non-zero presence_penalty or frequency_penalty, and any non-null reasoning_budget or reasoning_effort, now return 400. Omit them, or use repetition_penalty and the MiRMiR reasoning extension instead.
  • tool_choice must be auto, none, required, or a named function.
  • A prompt that exceeds the model context, or can never fit the K/V cache, is now 400 instead of 500. A request that could fit later remains 429.
  • A cancelled operation returns 408 cancelled.
  • A tool call that violates tools or tool_choice, or is malformed or truncated, returns 422 invalid_model_output.
  • Inference endpoints return 503 until startup model restoration completes. Point readiness probes at the new GET /ready; keep GET /health for liveness.
  • Closing an SSE connection now cancels the generation on the server.

New request capabilities are opt-in: tools with validated tool_calls responses, tool_constraints: "schema" on Metal and CUDA, reasoning and chat_template_kwargs.enable_thinking, thinking_token_budget, return_token_ids for streams, and top_n for reranking. See Chat completions and Embeddings and reranking.

Configuration

Existing config.toml files remain valid; no key was renamed. New optional runtime keys tune scheduling: prefill_batch_wait_us, cached_prefill_policy, prefill_decode_policy, prefill_refill_policy, and, on macOS, metal_decode_reservation. Omitting them keeps the backend defaults. See Configuration.

Models

Decoder rerankers that ship a sentence-transformers LogitScore module are now admitted as causal rerankers on Metal and CUDA. Qwen XML tool calls are parsed natively. Re-run mirmir model inspect after upgrading, because admission reports can change between releases.

libmir embedders

  • Depend on libmir = "0.4.0". The facade is now the only public entry point: the foundation, models, and runtime modules and the backend-specific CUDA and Metal types are no longer re-exported.
  • ChatCompletionRequest and the Chat* message types are replaced by GenerationRequest, Conversation, Message, Tool, ToolCall, and the ToolChoice enum. Model::generate* takes &GenerationRequest, and Model::prepare takes &Conversation.
  • ProgressEvent is an enum with stage(), unit(), count(), and detail() accessors.
  • SchedulerConfig and GenerationToken gained fields; construct them with ..Default::default() or update struct literals.

See Generate and stream for the new request shape.

Upgrade checklist

  1. Install 0.4.1 with cargo install mirmir --version 0.4.1 --locked and run mirmir --version.
  2. Run mirmir config validate against the existing configuration.
  3. Remove unsupported penalty and reasoning-effort fields from API clients and handle 408, 422, and 503 responses.
  4. Switch readiness checks to GET /ready.
  5. Inspect each checkpoint on its target Metal or CUDA machine.