Rate limits & concurrency

The platform enforces multiple limits for fairness and stability. Hitting one returns 429 rate_limit_exceeded.

Limit types

Type Description
Rate (RPM) Requests per minute cap
Throughput (TPM) Tokens per minute cap
Concurrency Concurrent in-flight requests cap
Quota Optional monthly spend cap per key

Hitting a limit

  • Returns 429 rate_limit_exceeded; the retry-after header suggests how long to wait.
  • Not billed: rate-limited requests incur no charge.

Pre-authorization

To prevent overspending, the platform conservatively reserves an amount before each request, settles against actual usage afterward, and releases the difference. Reservations are not final balance — only actual usage counts.

Recommendations

  • Implement exponential backoff and honor retry-after.
  • Contact support to raise limits for high-concurrency workloads.