Telemetry and benchmarking
MiRMiR publishes the same runtime state to the TUI, web dashboard, CLI status, and private gRPC clients.
Metrics
| Metric | Meaning |
|---|---|
| TTFT | Time from request start to the first generated token. |
| Prefill | Prompt tokens processed per second. |
| Decode | Generated tokens processed per second after prefill. |
| E2E throughput | Completion tokens divided by complete request time. |
| Memory | Active and cached accelerator or unified-memory occupancy. |
| K/V occupancy | Used blocks, free capacity, and prefix-cache effectiveness. |
The server samples runtime metrics every second, keeps 900 samples, and
persists telemetry atomically under ~/.local/state/mirmir/telemetry.toml.
The web dashboard renders the latest 60 samples and restores them after a
restart.
Inspect current state
mirmir status
The command prints server health, ready models, and the latest telemetry snapshot as JSON.
Benchmark a prompt
mirmir prompt \
--model qwen \
--prompt "Explain continuous batching." \
--warmup 1 \
--samples 5 \
--seed 7 \
--json \
--csv benchmark.csv
The report contains individual samples and mean, median, p50, p95, population standard deviation, minimum, and maximum for E2E, TTFT, prefill, decode, and their durations.
Logging
mirmir serve emits structured lifecycle and request logs at INFO. Narrow or
expand application targets explicitly:
RUST_LOG=mirmir=trace,libmir=debug mirmir serve
RUST_LOG=mirmir=warn,libmir=warn mirmir serve