Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Embeddings and reranking

MiRMiR discovers the task from the checkpoint structure. A generation model cannot be used as an embedding model, and it serves reranking only when the checkpoint declares a causal scoring head (see below).

Embeddings

curl http://127.0.0.1:8080/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "embedding",
    "input": ["first text", "second text"],
    "encoding_format": "float"
  }'
FieldTypeNotes
modelstringLoaded embedding model.
inputstring or string[]One input or an ordered batch.
dimensionsintegerOptional prefix dimension, capped by the native size.
prompt_namestringOptional prompt preset declared by the checkpoint.
encoding_formatstringOnly float is supported.

The output follows the OpenAI list shape and returns normalized f32 vectors plus prompt token usage.

Reranking

curl http://127.0.0.1:8080/v1/rerank \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "reranker",
    "query": "native inference",
    "documents": ["Rust and Metal", "a hosted Python service"],
    "return_documents": true
  }'
FieldTypeNotes
modelstringLoaded sequence-scoring or causal-scoring model.
querystringRequired non-empty query.
documentsstring[]Required non-empty candidate list.
max_lengthintegerPair token limit, capped by model context.
top_nintegerReturn only the best n results after sorting.
raw_scoresbooleanReturn logits instead of logistic relevance scores.
return_documentsbooleanInclude each original document in results.

Results are sorted from highest to lowest relevance while retaining the original document index.

Causal rerankers

Since 0.4.0, a decoder checkpoint that ships a sentence-transformers LogitScore module is admitted as a causal reranker on Metal and CUDA. The module declares the “true” and “false” answer tokens and an optional instruction, which is rendered as a system message. MiRMiR renders each query and document through the model’s chat template and scores the pair as the difference between the two answer-token logits at the final position. raw_scores: true returns that difference; otherwise it passes through the logistic function like a cross-encoder score.