Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Upgrade to 0.5.5

The 0.5.5 release aligns mirtal, mircuda, mircup, libmir, their support crates, and mirmir at one version. The website and bundled dashboard carry the same version. Existing 0.4.x and 0.5.2 installations should review these changes before replacing their binary.

cargo install mirmir --version 0.5.5 --locked
mirmir --version
mirmir config validate

Stop and restart the runtime after upgrading; attaching a new client does not replace an already running server. Keep your configuration and model files. For service installations, retain the service account and socket permissions.

Available capabilities

  • Nemotron-H on CUDA: text generation for the validated Nemotron-3.5-Super NVFP4 checkpoint, with Mamba-2, attention without RoPE, latent MoE, and optional multi-token prediction (MTP). This does not add Nemotron-H execution to Metal or enable the checkpoint’s vision tower.
  • Gemma 4 on CUDA: sliding-window attention and 512-wide global heads use accelerated prefill paths. This is separate from the existing Metal Gemma path; benchmark results remain tied to their recorded builds.
  • Structured content: response_format accepts JSON objects and strict JSON schemas on Metal and CUDA. Reasoning is separate from JSON content; literal protocol markers inside JSON strings remain ordinary text.
  • Laya decisions: libmir::decision runs on CPU, Metal, or CUDA. The manual-testing /v1/decide endpoint exposes CPU and macOS Metal only.
  • Memory and scheduling: per-model K/V block limits, opt-in MTP, and the existing explicit cached-refill and short-prompt admission policies.
  • Metal foundation: the native bridge targets MLX 0.32.3. Build against the matching native installation; the Python mlx package is not required.

Request compatibility

JSON content cannot be combined with schema-constrained tools or a required or named tool_choice. Such requests return 400 before generation. Use JSON content for a final structured answer, or a named schema-constrained tool for a tool invocation. strict: false is not supported.

thinking_token_budget now applies to both schema tools and JSON content, with enabled reasoning and an explicit total completion limit. It reserves space inside that limit; it does not increase the limit. Incomplete or invalid structured output remains an error, including an SSE error after streaming has started.

Rust consumers

Update your libmir requirement to 0.5.5 and select metal, cuda, or cpu as needed. The default feature set enables no backend. CPU currently provides Laya decisions, not the general generation/embedding runtime. Use GenerationRequest { ..Default::default() } to retain defaults for new request fields. Cross-repository dependencies remain crates.io versions.

This release changes Rust source compatibility with 0.5.2:

Public APIRequired adaptation
Error::InvalidResponseFormatUpdate exhaustive error matches for invalid or incomplete structured content.
ArchitectureCapabilityHandle StateSpaceMixer, LatentRoutedExperts, ReluSquared, and SigmoidBiasedTopKRouter in exhaustive matches.
GenerationRequestNew reasoning_effort, reasoning_history, and response_format fields; prefer ..Default::default() where appropriate.
Runtime DecodeRequest / DecodeSequenceInitialize speculation (None for ordinary decoding).
Runtime DecodeOutputInitialize accepted and drafts (empty for ordinary decoding).
DecoderConfig / Metal MixerLowering::SoftmaxInitialize the new typed Q/K normalization field when constructing these structs directly. Prefer parsing checkpoint configuration and lowering the resolved spec.
New MTP configurationConstruct the bounded window with DraftTokens::try_from(count); values must be 1–8.

These additions are not hidden by adding #[non_exhaustive] retroactively, which would itself change construction and matching rules. Rebuild and test Rust consumers when upgrading; binary configuration compatibility is separate.

See JSON output, Laya decisions, and configuration for complete examples.