Systems design · interactive estimator

Inference
Capacity Planner

Explore how token demand, replicas and batching interact—then see the cost and latency implications of your assumptions.

Synthetic planning model · no network calls

Workload assumptions

Change inputs to explore a steady-state scenario.

requests / minute
input tokens / request
output tokens / request
tokens / second / replica
serving replicas
requests per batch
simultaneous requests
illustrative USD / hour
seconds
All values are editable assumptions.

Capacity & trade-offs

A transparent estimate, not a live benchmark.

Estimated demand / nominal capacity
Token demand
tokens / second
Nominal capacity
tokens / second
Estimated p95
seconds · rough heuristic
Estimated spend
USD / hour

The queue/p95 proxy is intentionally simple. It does not model actual hardware, model-specific prefill/decode, bursty traffic, cache hits or scheduler behavior. Measure representative traffic before committing to capacity or SLOs.