Systems design · interactive estimator
Inference
Capacity Planner
Explore how token demand, replicas and batching interact—then see the cost and latency implications of your assumptions.
Synthetic planning model · no network calls
Workload assumptions
Change inputs to explore a steady-state scenario.
requests / minute
input tokens / request
output tokens / request
tokens / second / replica
serving replicas
requests per batch
simultaneous requests
illustrative USD / hour
seconds
All values are editable assumptions.
Capacity & trade-offs
A transparent estimate, not a live benchmark.
Token demand
tokens / secondNominal capacity
tokens / secondEstimated p95
seconds · rough heuristicEstimated spend
USD / hourThe queue/p95 proxy is intentionally simple. It does not model actual hardware, model-specific prefill/decode, bursty traffic, cache hits or scheduler behavior. Measure representative traffic before committing to capacity or SLOs.