Limits and compatibility
These are the 0.6.2 product boundaries, not commitments for later releases.
HTTP and media
- chat completions support
n = 1; - embedding output supports
encoding_format = "float"; - one image may be attached to a generation request;
- supported image containers are PNG, JPEG, WebP, and GIF;
- decoded application image input is limited to 20 MiB;
- the default HTTP body limit is 29,360,128 bytes;
- default request timeout is 300 seconds;
- default HTTP concurrency is 16 requests;
- CORS is disabled until exact origins are configured;
/v1inference endpoints return503until startup model restoration completes;presence_penalty,frequency_penalty,reasoning_budget, andreasoning_effortare rejected rather than ignored;response_formatsupports text, JSON objects, and strict JSON schemas; JSON content rejects required/named tool choices and schema-tool mode;- schema conformance is checked on completion; streamed content is provisional until final validation succeeds.
Tool calling
- tool calls are validated against
toolsandtool_choice; violations return422; - schema-constrained tool decoding runs on Metal and CUDA and requires a native XML tool template such as Qwen’s;
- constrained root schemas are objects with at most 64 top-level properties
and 512 KB; remote and file
$refretrieval is disabled; thinking_token_budgetrequires enabled reasoning, text input, schema mode, and an explicitmax_tokens.
The same budget control supports JSON content. See Structured JSON output.
Web dashboard
- the dashboard is opt-in;
- it is restricted to loopback;
- its local session lasts eight hours;
- the OpenAI bearer token cannot authorize dashboard administration.
Models
- compatibility is determined from configuration, tokenizer, tensor schema, shapes, quantization, and backend capabilities;
- the 0.6.2 container boundary is SafeTensors from supported Hugging Face, Transformers, MLX, and quantization-toolchain layouts;
- a family or repository name is not a compatibility guarantee;
- Metal and CUDA have different admitted checkpoint coverage;
- Nemotron-H generation and MTP require CUDA; Metal rejects its state-space mixer;
- CPU supports Laya decisions, not general chat generation;
/v1/decideexposes CPU/Metal only; CUDA decisions use the Rust API;- a forced load bypasses memory-fit policy, not structural validation;
- model load cancellation waits for a future safe backend interruption boundary.
The complete encoding matrix is documented under Model compatibility.
libmir stability
libmir is pre-1.0. Pin 0.6.2 and review release notes before upgrading.
Since 0.4.0 the top-level facade is the only supported entry point: the
internal workspace crates are no longer re-exported.