Rate Limits

Overview

Rate limits keep the platform fast and fair for everyone. The primary limit is requests per minute (RPM), enforced in a sliding window and shared across all servers. Limits are resolved in this order, most specific first:

1. Per-API-key limit — a custom limit you set on an individual key (Dashboard → API Keys).
2. Tier limit — your account tier's RPM, applied to AI-generation endpoints.
3. Per-endpoint limit — protective limits on specific routes (see below).
4. Global default — a baseline fallback.

Tier limits

Endpoint-specific limits

Some routes carry their own protective limits regardless of tier, for example:

Health checks, the model list, and batch/collection management endpoints are excluded from rate limiting entirely.

Rate-limit headers

Responses include headers describing your current limit state:

X-RateLimit-Limit: 60
X-RateLimit-Remaining: 57
X-RateLimit-Reset: 1709251200
X-RateLimit-Resource: api_key
X-RateLimit-Used: 3
X-Request-Priority: 2
X-Concurrency-Remaining: 8

When you're throttled

Exceeding a limit returns 429 Too Many Requests with a Retry-After header and this body:

{
  "error": {
    "code": "rate_limit_exceeded",
    "message": "Rate limit exceeded: 60 requests per minute allowed. Please retry in 12 seconds.",
    "retry_after": 12
  }
}

Retry strategy

Honor Retry-After (or error.retry_after) when present; otherwise back off exponentially with jitter:

import time, random

def call_with_retry(fn, max_retries=5):
    for attempt in range(max_retries):
        try:
            return fn()
        except Exception as e:
            if "rate_limit" not in str(e).lower():
                raise
            wait = (2 ** attempt) + random.uniform(0, 1)
            time.sleep(wait)
    raise RuntimeError("Max retries exceeded")

Staying under the limit

• Respect the headers. Watch X-RateLimit-Remaining and slow down before you hit zero.
• Use streaming. A streamed response is a single request regardless of output length.
• Batch offline work. For large non-interactive jobs, use the Batch API (also ~50% cheaper).
• Cache repeated prompts. Avoid re-requesting identical completions.
• Set per-key limits. Cap noisy services with a custom per-key limit so one client can't starve the rest.
• Monitor consumption. Track usage from the Dashboard or the Usage & Monitoring endpoints.