Model compatibility
MiRMiR 0.6.2 supports a checkpoint only when its complete execution contract is valid for the selected backend. A repository name, architecture label, or SafeTensors container is useful context, but none is a compatibility guarantee by itself.
What MiRMiR inspects
Before admission, MiRMiR combines:
- the task: generation, vision-language generation, embeddings, or reranking with an encoder or a causal decoder;
- semantic architecture: attention, normalization, positional encoding, feed-forward composition, routing, pooling, and activation;
- tokenizer, processor, chat template, and protocol tokens;
- tensor names, roles, shapes, dtypes, packing, and companion scales;
- physical weight encoding and quantization geometry;
- Metal or CUDA capabilities and the available accelerator memory.
Remote search and pull use bounded header preflight. Local inspection evaluates
the complete checkpoint. Compatibility is reported independently as
supported, partial, unsupported, or unknown for each backend.
Validated model systems
The 0.6.2 execution contracts cover these systems when every required component is present:
| System | Covered composition |
|---|---|
| Dense decoders | Grouped-query attention, RoPE variants, tied weights, and projection biases. |
| Routed MoE | Top-k routing, shared experts, dense-plus-routed blocks, and clamped routed experts. |
| Hybrid attention | Full and sliding softmax attention combined with linear or recurrent mixer layers. |
| Nemotron-H (CUDA only) | Mamba-2, attention without RoPE, latent routed experts, and optional MTP for checkpoints with a prediction head. Nemotron Omni checkpoints with the RADIO vision encoder also accept images on CUDA. |
| Laya decisions | ModernBERT decision encoder on CPU, Metal, or CUDA through the Rust API. HTTP exposes CPU and macOS Metal. |
| Embeddings | Encoder checkpoints with declared pooling and optional output normalization. |
| Sequence scoring | Encoder classification and reranking with typed pooling and output boundaries. |
| Causal scoring | Decoder checkpoints with a sentence-transformers LogitScore module, reranked from final answer-token logits on Metal and CUDA. |
| Vision-language | Vision tower, projector, image processor, protocol tokens, and text decoder admitted together. |
These are structural categories, not a closed family allowlist. Inspect the exact checkpoint before promising support.
Gemma 4 retains its Metal path and gains accelerated CUDA prefill for sliding attention and 512-wide global heads. Support claims do not imply fresh performance measurements on every release. Historical matrices identify their model revision, backend, and measured software version.
Gemma 4’s omitted normalization and scaling settings have built-in defaults:
Q/K/V RMS normalization, attention scale 1.0, and embedding multiplier
sqrt(hidden_size). A standard supported checkpoint needs no manually created
sidecar. Explicit configuration values override the defaults; a complete
mir-model-spec.toml overrides the resolved semantic contract. Metal rejects
normalization or scaling overrides its current execution path cannot implement,
with an error before weights are loaded. CUDA executes its supported declared
semantics. The dense Gemma 4 12B composition remains a CUDA path; this change
does not add a new dense GELU runner to Metal.
The validated Nemotron-3.5-Super 120B-A12B NVFP4 text weights occupy about 66.6 GiB in the CUDA implementation, with higher runtime peaks; the vision encoder adds about 1.9 GB of resident weights. Active parameter count does not describe resident weight memory: this checkpoint is not a 64 GB Mac workload, and its state-space path is not implemented on Metal.
Checkpoint ecosystems
MiRMiR 0.6.2 reads Hugging Face and MLX Community repositories stored as SafeTensors. It interprets Transformers configuration, tokenizer assets, shard indexes, and chat templates natively; Transformers, PyTorch, bitsandbytes, and Python are not runtime dependencies.
GGUF files are not admitted by the 0.6.2 runtime.
Weight encodings
| SafeTensors encoding | Metal | CUDA | Notes |
|---|---|---|---|
| F32, F16, BF16 | Supported | Supported | Native dense execution. |
| MLX grouped affine 2/3/4/5/6/8-bit | Supported | Supported | Packed weights remain accelerator-resident. |
| Compressed Tensors INT8 and INT4 | Supported | Supported | Typed packing, grouping, scale, and zero-point contracts. |
| AWQ | Supported | Supported | Backend-specific direct execution or persistent device repacking. |
| GPTQ | Supported | Supported | Includes typed format and activation-order contracts. |
| FP8 E4M3 and E5M2 | Supported | Supported | Scale direction and geometry are validated explicitly. |
| MXFP4 and MXFP8 | Supported | Supported | Block values and scale representation remain distinct. |
| NVFP4 | Supported (role-specific) | Supported (native) | Metal converts ordinary matrices once and executes routed banks directly. |
| bitsandbytes NF4 and FP4 | Supported | Supported | Codebook, block scales, nested quantization, and compute dtype are preserved. |
Supported means the format has a complete contract, backend operation path,
real-checkpoint validation, deterministic generation coverage where applicable,
and bounded accelerator-memory behavior. It does not mean that every model
architecture using that encoding is implemented.
Device conversion policy
MiRMiR may execute a checkpoint’s packed representation directly or convert it once into a persistent backend-native representation. Conversion is admitted only when it is bounded, runs on the accelerator, is included in memory preflight, and does not leave a full dense host-side shadow of the weights.
Check a checkpoint
mirmir model inspect OWNER/REPOSITORY
mirmir model load OWNER/REPOSITORY
Inspection explains the discovered task, format, backend admission, memory
estimate, and safe context capacity. A failed structural or backend check must
be resolved with a compatible checkpoint; --force cannot bypass it.