Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Embeddings and reranking

The same loaded Model type exposes task-specific methods. Calling the wrong method for a checkpoint returns Error::TaskMismatch.

Embeddings

use libmir::{EmbeddingRequest, GenerationOverrides, Library, RuntimeConfig};

let model = Library::new(RuntimeConfig::default()).load(
    "/models/embedding",
    GenerationOverrides::default(),
    &mut |_| {},
)?;

let output = model.embed(EmbeddingRequest {
    inputs: vec!["first text".into(), "second text".into()],
    dimensions: None,
    prompt_name: None,
})?;

println!("vectors={}", output.embeddings.len());
println!("tokens={}", output.prompt_tokens);
Ok::<(), libmir::Error>(())

dimensions may retain a prefix no larger than the checkpoint’s native dimension. prompt_name must name a preset declared by the checkpoint.

Reranking

use libmir::{GenerationOverrides, Library, RerankRequest, RuntimeConfig};

let model = Library::new(RuntimeConfig::default()).load(
    "/models/reranker",
    GenerationOverrides::default(),
    &mut |_| {},
)?;

let output = model.rerank(RerankRequest {
    query: "native inference".into(),
    documents: vec!["Rust and Metal".into(), "a hosted service".into()],
    max_length: None,
    raw_scores: false,
})?;

for result in output.results {
    println!("{} {:.4} {}", result.index, result.score, result.document);
}
Ok::<(), libmir::Error>(())

Results are relevance-sorted but preserve each candidate’s original index. Set raw_scores to return the raw score instead of its logistic transform.

Two checkpoint contracts are served by Model::rerank, reported by ModelDescriptor::task_plan():

PlanCheckpointRaw score
TaskExecutionPlan::SequenceScoringSequence-classification cross-encoderClassifier logit for the query/document pair.
TaskExecutionPlan::CausalScoringCausal decoder with a sentence-transformers LogitScore moduleFinal-position logit of the true token minus that of the false token.

Causal scoring runs on Metal and CUDA. It renders the checkpoint chat template with query and document turns, preceded by the checkpoint’s default prompt as a system instruction when one is declared. The rendered prompt is never truncated; a candidate longer than max_length returns Error::Context.