Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Structured JSON output

POST /v1/chat/completions accepts response_format on Metal and CUDA:

ValueResult
{"type":"text"} or omittedOrdinary generation.
{"type":"json_object"}A JSON object in message.content.
{"type":"json_schema","json_schema":{...}}JSON matching the supplied schema.
curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Authorization: Bearer $MIRMIR_HTTP_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "qwen",
    "messages": [{"role":"user","content":"Classify: the parcel arrived damaged."}],
    "reasoning": "disabled",
    "repetition_penalty": 1,
    "max_completion_tokens": 256,
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "classification",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {"category":{"enum":["damage","delay","other"]}},
          "required": ["category"],
          "additionalProperties": false
        }
      }
    }
  }'

Use a model whose template supports the requested reasoning switch, or omit the switch to retain model defaults. A JSON schema needs a nonblank name; omit strict or set it to true. strict: false returns 400.

Validation and limits

  • Sampling requires repetition_penalty: 1, min_tokens: 0, and ignore_eos: false. Omitted controls use these neutral values, overriding checkpoint defaults.
  • JSON content and tool_constraints: "schema" are exclusive. Required or named tool choices are also rejected. auto/none remain accepted; this mode constrains the final content rather than producing tool calls.
  • Unsupported schemas are rejected before generation. Remote/file schema reference retrieval is disabled. The complete output is validated against the original schema; constraints such as uniqueItems are checked at that final stage, not guaranteed by the token mask.
  • Truncated, invalid, or incomplete JSON produces 422 invalid_model_output for a non-streaming request. This intentionally differs from OpenAI’s finish_reason: "length" with truncated content: MiRMiR does not return an incomplete structured document as a successful completion. Streaming clients receive an error after any partial chunks and must discard that document. Schema validity does not verify factual truth.

JSON response schemas and explicit incompatible sampling settings are rejected before generation or SSE starts. When sampling controls are omitted in schema mode, repetition penalty is 1, minimum tokens is 0, and EOS is honored, even if the checkpoint has different defaults. Valid schemas continue to constrain token selection during decode; completed content is also checked against the original schema. Model-specific tokenizer/backend failures can still occur during setup.

Reasoning and streaming

With supported native reasoning delimiters, reasoning can precede the JSON answer and is returned separately in reasoning_content. Once JSON begins, strings containing text such as <think> remain literal content.

With reasoning: "enabled" and an explicit completion limit, thinking_token_budget bounds reasoning within the total. It must leave at least two tokens for the delimiter and answer. If omitted, the HTTP adapter reserves one quarter of the total (at least two tokens) for structured output. Reserve more for large schemas; this does not guarantee completion.

For stream: true, accumulate delta.content before parsing JSON. Individual chunks need not be valid JSON. The final validation can fail after bytes have already been emitted; handle the stream’s error event rather than treating a partial response as a validated document.