Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Limits and compatibility

These are the 0.6.2 product boundaries, not commitments for later releases.

HTTP and media

  • chat completions support n = 1;
  • embedding output supports encoding_format = "float";
  • one image may be attached to a generation request;
  • supported image containers are PNG, JPEG, WebP, and GIF;
  • decoded application image input is limited to 20 MiB;
  • the default HTTP body limit is 29,360,128 bytes;
  • default request timeout is 300 seconds;
  • default HTTP concurrency is 16 requests;
  • CORS is disabled until exact origins are configured;
  • /v1 inference endpoints return 503 until startup model restoration completes;
  • presence_penalty, frequency_penalty, reasoning_budget, and reasoning_effort are rejected rather than ignored;
  • response_format supports text, JSON objects, and strict JSON schemas; JSON content rejects required/named tool choices and schema-tool mode;
  • schema conformance is checked on completion; streamed content is provisional until final validation succeeds.

Tool calling

  • tool calls are validated against tools and tool_choice; violations return 422;
  • schema-constrained tool decoding runs on Metal and CUDA and requires a native XML tool template such as Qwen’s;
  • constrained root schemas are objects with at most 64 top-level properties and 512 KB; remote and file $ref retrieval is disabled;
  • thinking_token_budget requires enabled reasoning, text input, schema mode, and an explicit max_tokens.

The same budget control supports JSON content. See Structured JSON output.

Web dashboard

  • the dashboard is opt-in;
  • it is restricted to loopback;
  • its local session lasts eight hours;
  • the OpenAI bearer token cannot authorize dashboard administration.

Models

  • compatibility is determined from configuration, tokenizer, tensor schema, shapes, quantization, and backend capabilities;
  • the 0.6.2 container boundary is SafeTensors from supported Hugging Face, Transformers, MLX, and quantization-toolchain layouts;
  • a family or repository name is not a compatibility guarantee;
  • Metal and CUDA have different admitted checkpoint coverage;
  • Nemotron-H generation and MTP require CUDA; Metal rejects its state-space mixer;
  • CPU supports Laya decisions, not general chat generation;
  • /v1/decide exposes CPU/Metal only; CUDA decisions use the Rust API;
  • a forced load bypasses memory-fit policy, not structural validation;
  • model load cancellation waits for a future safe backend interruption boundary.

The complete encoding matrix is documented under Model compatibility.

libmir stability

libmir is pre-1.0. Pin 0.6.2 and review release notes before upgrading. Since 0.4.0 the top-level facade is the only supported entry point: the internal workspace crates are no longer re-exported.