Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Model compatibility

MiRMiR 0.6.2 supports a checkpoint only when its complete execution contract is valid for the selected backend. A repository name, architecture label, or SafeTensors container is useful context, but none is a compatibility guarantee by itself.

What MiRMiR inspects

Before admission, MiRMiR combines:

  • the task: generation, vision-language generation, embeddings, or reranking with an encoder or a causal decoder;
  • semantic architecture: attention, normalization, positional encoding, feed-forward composition, routing, pooling, and activation;
  • tokenizer, processor, chat template, and protocol tokens;
  • tensor names, roles, shapes, dtypes, packing, and companion scales;
  • physical weight encoding and quantization geometry;
  • Metal or CUDA capabilities and the available accelerator memory.

Remote search and pull use bounded header preflight. Local inspection evaluates the complete checkpoint. Compatibility is reported independently as supported, partial, unsupported, or unknown for each backend.

Validated model systems

The 0.6.2 execution contracts cover these systems when every required component is present:

SystemCovered composition
Dense decodersGrouped-query attention, RoPE variants, tied weights, and projection biases.
Routed MoETop-k routing, shared experts, dense-plus-routed blocks, and clamped routed experts.
Hybrid attentionFull and sliding softmax attention combined with linear or recurrent mixer layers.
Nemotron-H (CUDA only)Mamba-2, attention without RoPE, latent routed experts, and optional MTP for checkpoints with a prediction head. Nemotron Omni checkpoints with the RADIO vision encoder also accept images on CUDA.
Laya decisionsModernBERT decision encoder on CPU, Metal, or CUDA through the Rust API. HTTP exposes CPU and macOS Metal.
EmbeddingsEncoder checkpoints with declared pooling and optional output normalization.
Sequence scoringEncoder classification and reranking with typed pooling and output boundaries.
Causal scoringDecoder checkpoints with a sentence-transformers LogitScore module, reranked from final answer-token logits on Metal and CUDA.
Vision-languageVision tower, projector, image processor, protocol tokens, and text decoder admitted together.

These are structural categories, not a closed family allowlist. Inspect the exact checkpoint before promising support.

Gemma 4 retains its Metal path and gains accelerated CUDA prefill for sliding attention and 512-wide global heads. Support claims do not imply fresh performance measurements on every release. Historical matrices identify their model revision, backend, and measured software version.

Gemma 4’s omitted normalization and scaling settings have built-in defaults: Q/K/V RMS normalization, attention scale 1.0, and embedding multiplier sqrt(hidden_size). A standard supported checkpoint needs no manually created sidecar. Explicit configuration values override the defaults; a complete mir-model-spec.toml overrides the resolved semantic contract. Metal rejects normalization or scaling overrides its current execution path cannot implement, with an error before weights are loaded. CUDA executes its supported declared semantics. The dense Gemma 4 12B composition remains a CUDA path; this change does not add a new dense GELU runner to Metal.

The validated Nemotron-3.5-Super 120B-A12B NVFP4 text weights occupy about 66.6 GiB in the CUDA implementation, with higher runtime peaks; the vision encoder adds about 1.9 GB of resident weights. Active parameter count does not describe resident weight memory: this checkpoint is not a 64 GB Mac workload, and its state-space path is not implemented on Metal.

Checkpoint ecosystems

MiRMiR 0.6.2 reads Hugging Face and MLX Community repositories stored as SafeTensors. It interprets Transformers configuration, tokenizer assets, shard indexes, and chat templates natively; Transformers, PyTorch, bitsandbytes, and Python are not runtime dependencies.

GGUF files are not admitted by the 0.6.2 runtime.

Weight encodings

SafeTensors encodingMetalCUDANotes
F32, F16, BF16SupportedSupportedNative dense execution.
MLX grouped affine 2/3/4/5/6/8-bitSupportedSupportedPacked weights remain accelerator-resident.
Compressed Tensors INT8 and INT4SupportedSupportedTyped packing, grouping, scale, and zero-point contracts.
AWQSupportedSupportedBackend-specific direct execution or persistent device repacking.
GPTQSupportedSupportedIncludes typed format and activation-order contracts.
FP8 E4M3 and E5M2SupportedSupportedScale direction and geometry are validated explicitly.
MXFP4 and MXFP8SupportedSupportedBlock values and scale representation remain distinct.
NVFP4Supported (role-specific)Supported (native)Metal converts ordinary matrices once and executes routed banks directly.
bitsandbytes NF4 and FP4SupportedSupportedCodebook, block scales, nested quantization, and compute dtype are preserved.

Supported means the format has a complete contract, backend operation path, real-checkpoint validation, deterministic generation coverage where applicable, and bounded accelerator-memory behavior. It does not mean that every model architecture using that encoding is implemented.

Device conversion policy

MiRMiR may execute a checkpoint’s packed representation directly or convert it once into a persistent backend-native representation. Conversion is admitted only when it is bounded, runs on the accelerator, is included in memory preflight, and does not leave a full dense host-side shadow of the weights.

Check a checkpoint

mirmir model inspect OWNER/REPOSITORY
mirmir model load OWNER/REPOSITORY

Inspection explains the discovered task, format, backend admission, memory estimate, and safe context capacity. A failed structural or backend check must be resolved with a compatible checkpoint; --force cannot bypass it.