Add libmir to a project

Version 0.2.0 enables no accelerator by default. Select the backend explicitly.

Apple Metal

[dependencies]
libmir = { version = "0.2.0", default-features = false, features = ["metal"] }

The Metal feature currently requires Apple Silicon, a full Xcode installation with the separately installed Metal toolchain, and native MLX.

NVIDIA CUDA

[dependencies]
libmir = { version = "0.2.0", default-features = false, features = ["cuda"] }

CUDA currently targets Linux with the NVIDIA driver, CUDA Toolkit 13.x, NVRTC, CCCL/CUB headers, and CUTLASS 4.4.2 headers.

Minimal generation

use std::io::{self, Write};

use libmir::{
    ChatCompletionRequest, ChatMessage, GenerationOverrides, Library,
    RuntimeConfig,
};

fn main() -> libmir::Result<()> {
    let model_path = std::env::args_os().nth(1).expect("model path");
    let library = Library::new(RuntimeConfig::default());
    let model = library.load(
        model_path,
        GenerationOverrides::default(),
        &mut |progress| eprintln!("{progress:?}"),
    )?;

    let request = ChatCompletionRequest {
        model: model.handle().id.clone(),
        messages: vec![ChatMessage {
            role: "user".into(),
            content: "Explain paged K/V cache briefly.".into(),
            reasoning_content: None,
        }],
        stream: true,
        max_tokens: Some(256),
        temperature: Some(0.0),
        top_p: Some(1.0),
        top_k: Some(0),
        repetition_penalty: Some(1.0),
        seed: Some(7),
    };

    let output = model.generate(&request, &mut |_| {}, &mut |token| {
        print!("{}", token.text);
        let _ = io::stdout().flush();
    })?;

    println!("\nfinish_reason={}", output.finish_reason);
    Ok(())
}

Library initializes its native backend lazily. Loading inspects the checkpoint first, resolves the execution contract, then reports real loading progress through the callback.

The same lifecycle is available as the generate example in the libmir repository.