Chat completions
Use POST /v1/chat/completions with a model already loaded into the shared
runtime.
curl http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $MIRMIR_HTTP_API_KEY" \
-d '{
"model": "qwen",
"messages": [{"role": "user", "content": "Hello"}],
"temperature": 0.2,
"max_tokens": 256
}'
import os
from openai import OpenAI
client = OpenAI(
base_url="http://127.0.0.1:8080/v1",
api_key=os.environ["MIRMIR_HTTP_API_KEY"],
)
reply = client.chat.completions.create(
model="qwen",
messages=[{"role": "user", "content": "Hello"}],
temperature=0.2,
max_tokens=256,
)
print(reply.choices[0].message.content)
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "http://127.0.0.1:8080/v1",
apiKey: process.env.MIRMIR_HTTP_API_KEY,
});
const reply = await client.chat.completions.create({
model: "qwen",
messages: [{ role: "user", content: "Hello" }],
temperature: 0.2,
max_tokens: 256,
});
console.log(reply.choices[0].message.content);
Parameters
| Field | Type | Notes |
|---|---|---|
model | string | Required loaded model selector. |
messages | array | Required non-empty chat history. Assistant tool_calls arguments may be a JSON object or a JSON-encoded string. |
stream | boolean | Returns server-sent events when true. |
max_tokens | integer | Total generated-token limit, including reasoning and final content. |
max_completion_tokens | integer | Alias; must agree with max_tokens if both appear. |
min_tokens | integer | Suppress end-of-sequence before this many tokens. |
ignore_eos | boolean | Continue past end-of-sequence until max_tokens. |
temperature | number | Sampling temperature. |
top_p | number | Nucleus cutoff from 0 to 1. |
top_k | integer | Candidate count; zero leaves it unbounded. |
repetition_penalty | number | 1 disables the penalty. |
seed | integer | Reproducible sampler seed. |
n | integer | Only 1 is supported. |
tools | array | OpenAI function tool declarations. |
tool_choice | string or object | auto (default), none, required, or {"type":"function","function":{"name":…}}. |
tool_constraints | string | MiRMiR extension: none (default) or schema. See Tool calling. |
response_format | object | text, json_object, or strict json_schema. See Structured JSON output. |
reasoning | string | MiRMiR extension: model_default (default), enabled, or disabled. |
chat_template_kwargs.enable_thinking | boolean | vLLM-compatible alias for reasoning; must agree with it when both appear. |
chat_template_kwargs.medium_effort | boolean | Uses shorter reasoning when true and the template declares this control. |
chat_template_kwargs.truncate_history_thinking | boolean | Keeps or drops earlier reasoning when the template declares this control. |
thinking_token_budget | integer | Reasoning allowance inside the total limit for schema tools or JSON content. |
return_token_ids | boolean | Streaming only: adds generated token ids to each chunk. |
stream_options.include_usage | boolean | Adds usage to the final SSE chunk. |
MiRMiR rejects controls it cannot honor instead of silently ignoring them. A
non-zero presence_penalty or frequency_penalty and any non-null
reasoning_budget or reasoning_effort return 400.
Unknown chat_template_kwargs keys are rejected. Explicit effort/history
controls are validated against the selected template; they are not universal
switches for every model.
Streaming
curl -N http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "qwen",
"messages": [{"role": "user", "content": "Hello"}],
"stream": true,
"stream_options": {"include_usage": true}
}'
Each event contains a chat.completion.chunk. The stream ends with
data: [DONE]. Reasoning-capable models emit delta.reasoning_content
separately from delta.content. Tool calls arrive as one delta.tool_calls
chunk before the final chunk, whose finish_reason is tool_calls. With
return_token_ids, each chunk carries choices[0].token_ids, and the final
chunk carries any remaining ids.
Closing the connection cancels the generation on the server.
Reasoning
reasoning selects the thinking switch for one text request without changing
other requests:
{"model": "qwen", "messages": [{"role": "user", "content": "Return 2+2."}],
"reasoning": "disabled", "max_completion_tokens": 256}
Explicit enabled and disabled modes require a model chat template that
reads enable_thinking, or a supported built-in Qwen or Gemma template. Other
templates and image requests reject explicit modes. model_default preserves
the template’s own behavior. The max_tokens limit covers reasoning and final
content together; reaching it does not guarantee a final answer.
Tool calling
Declare OpenAI function tools and select how the model may use them:
{
"model": "qwen",
"messages": [{"role": "user", "content": "What's the weather in Warsaw?"}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}
}],
"tool_choice": "auto"
}
Native Qwen XML (<tool_call><function=…>) and Mistral [TOOL_CALLS] output
is normalized into OpenAI message.tool_calls with finish_reason: "tool_calls". MiRMiR validates every call against the request and returns
422 invalid_model_output when the model omits a call that required or a
named function demands, calls a tool while tool_choice is none, calls a
different or undeclared tool, or emits malformed or truncated arguments.
Schema-constrained tools
On Metal and CUDA, "tool_constraints": "schema" constrains decoding to the
named tool’s parameter schema, so the model can only produce a well-formed
call. It requires
a named tool_choice, repetition_penalty: 1, min_tokens: 0, and
ignore_eos: false; other combinations return 400.
- Root schemas must be objects with declared properties, at most 64 top-level
properties and 512 KB. Remote and file
$refretrieval is disabled. - Types, required fields, enums, and array cardinality are enforced while
decoding;
uniqueItemsis checked on the completed call (422on violation). Parameter order follows the declaration. - With
reasoning: "enabled", reasoning stays unconstrained until the model’s</think>marker, then the tool grammar takes over.thinking_token_budgetcaps reasoning insidemax_tokensand must leave at least two tokens for the delimiter and tool. When omitted, MiRMiR reserves a quarter ofmax_tokens(at least two tokens) for the tool. - A limit that truncates the tool call still returns
422.
One image
The OpenAI-style image_url content part accepts a data URL. Version 0.6.2
supports one PNG, JPEG, WebP, or GIF image per request; an animated GIF contributes its first frame.
Image requests cannot set an explicit reasoning mode, effort or history; leave
chat_template_kwargs.enable_thinking unset.
IMAGE_DATA=$(base64 < image.jpg | tr -d '\n')
curl http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d "{\"model\":\"vision-model\",\"messages\":[{\"role\":\"user\",\"content\":[{\"type\":\"image_url\",\"image_url\":{\"url\":\"data:image/jpeg;base64,$IMAGE_DATA\"}},{\"type\":\"text\",\"text\":\"Describe this image\"}]}]}"
The application limit is 20 MiB per decoded image. The server body limit must also accommodate the larger base64 request.