Runtime configuration
RuntimeConfig is an explicit Rust value. libmir never reads command-line
arguments, dotenv files, or environment variables.
use libmir::{KvCacheDType, Library, RuntimeConfig};
let mut config = RuntimeConfig::default();
config.kv_cache.block_size = 32;
config.kv_cache.block_count = 2_048;
config.kv_cache.dtype = KvCacheDType::Auto;
config.scheduler.max_batch_requests = 8;
config.scheduler.max_batch_tokens = 4_096;
config.vision.max_pixels = Some(1_048_576);
config.vision.memory_percent = 70;
let library = Library::new(config);
drop(library);Shared configuration
| Field | Default | Purpose |
|---|---|---|
kv_cache.block_size | 16 | Tokens per page-backed cache block. |
kv_cache.block_count | 4096 | Physical block count. |
kv_cache.dtype | Auto | Backend-resolved cache storage. |
scheduler.max_batch_requests | 16 | Maximum requests in a decode batch. |
scheduler.max_batch_tokens | 8192 | Prompt plus output token budget. |
scheduler.decode_batch_wait_us | 200 | Short batching window. |
scheduler.decode_priority_burst | 8 | Decode fairness control. |
vision.max_pixels | checkpoint cap | Maximum resized image area. |
vision.attention_budget_bytes | automatic | Vision attention workspace. |
vision.memory_percent | 80 | Available memory used by automatic budgeting. |
Backend-specific fields exist only when their Cargo feature is enabled. A Metal-only consumer cannot accidentally configure CUDA, and a backend-neutral build has neither field.