Rate Limits
LLM Rate Limit & Throughput Calculator
Work out max sustainable throughput, time to process N requests, required concurrency, and whether TPM or RPM is your binding limit.
0 = no TPM limit
0 = no RPM limit
Input + output tokens
For the concurrency estimate
TPM is your bottleneck
TPM allows 50 req/min; RPM allows 1,000 req/min.
50 req/min
200,000
33h 20m
9
Concurrency ≈ throughput/sec × average request duration — the number of requests to keep in flight to saturate your binding limit. If TPM binds, larger requests hurt; if RPM binds, batching more work per request helps.
TL;DR
Your real throughput is min(RPM, TPM ÷ tokens-per-request) — TPM binds on large requests, RPM binds on small frequent ones. Enter your tier's TPM and RPM plus your request size to see the binding limit, max sustainable throughput, time to process a batch, and the concurrency needed to saturate it. The "which limit binds" table is below.
Which limit binds you
Updated Jul 5, 2026Providers cap you on two axes at once: TPM (tokens per minute) and RPM (requests per minute). Your real ceiling is whichever runs out first. TPM binds when requests are large; RPM binds when requests are small and frequent. Convert each to requests/min and take the smaller:
| TPM | RPM | Tokens/req | Max req/min | Binds |
|---|---|---|---|---|
| 30,000 | 500 | 1,000 | 30 | TPM |
| 200,000 | 1,000 | 4,000 | 50 | TPM |
| 2,000,000 | 10,000 | 8,000 | 250 | TPM |
| 450,000 | 5,000 | 500 | 900 | TPM |
Max req/min = min(RPM, TPM ÷ tokens-per-request). If TPM binds, split or shrink prompts; if RPM binds, do more per call or request a higher RPM tier.
How it works
reqPerMinByRPM = RPM; reqPerMinByTPM = TPM ÷ tokensPerRequest; maxThroughput = min(...). Time to process N requests = N ÷ maxThroughput. Suggested concurrency = ⌈maxThroughput ÷ 60 × avgDuration⌉.
All arithmetic — enter your own tier limits (they vary by provider, model, and usage tier, and don't assume a specific one). Tokens per request should include both input and output, since both count against TPM.
FAQ
- How do I calculate LLM throughput from TPM and RPM limits?
- Convert both limits to requests per minute and take the smaller. RPM is already requests/min. TPM ÷ tokens-per-request gives the requests/min the token limit allows. Your sustainable throughput is min(RPM, TPM ÷ tokens-per-request); that's your binding limit.
- Is TPM or RPM the binding limit?
- It depends on request size. Large requests exhaust TPM first (few requests use all your tokens), so TPM binds. Small, frequent requests exhaust RPM first (you hit the request cap while tokens are plentiful), so RPM binds. The break-even is when TPM ÷ tokens-per-request equals RPM.
- How much concurrency do I need to hit my rate limit?
- Roughly throughput-per-second × average request duration. If your binding limit is 50 requests/min (≈0.83/sec) and each request takes 10 seconds, you need about 9 requests in flight at once. Fewer and you leave throughput on the table; more and you'll hit the rate limit and get 429s.
- How do I increase my effective throughput?
- If TPM binds, reduce tokens per request (shorter prompts, trimmed context, smaller max output) or request a higher TPM tier. If RPM binds, batch more work into each request or raise your RPM tier. Also add retry-with-backoff so transient 429s don't stall the queue.