Rate limits

Per-tier daily request limits

Each plan has separate daily caps per operation type, reset at 00:00 UTC. Upstream providers may apply additional per-second or burst limits, which InferAll handles via automatic failover.

A new key without a card is not governed by this table. Those keys have no daily cap of their own; what limits them is the no-card trial allowance, plus a short-burst cap. Add a card and the plan you are on below starts applying.

Values are requests per day per operation. Counts reset at 00:00 UTC. The gateway is bursty-tolerant within the daily cap — there's no per-second sublimit. Each plan also includes a monthly $-amount of premium-provider usage ($5 free, $20 Pro, $100 Team) before prepaid balance is drawn down; see /pricing for the full structure.

Activating NIM access (new accounts)

New accounts get 25 free NVIDIA NIM open-source model calls before any payment, so your first request works as soon as you have a key. When those run out, the $5 starter pack unlocks ongoing NIM access ($0 input/$0 output on 40+ models). Visit /billing and complete checkout. The $5 is not a fee, it becomes spendable balance you can then use on any premium provider (OpenAI, Anthropic, Google) at zero markup.

Once the trial calls are used and before the charge settles, NIM requests return HTTP 402 with an upgrade URL pointing at /billing. After it settles you graduate to the free-plan daily caps in the table above, and premium-provider usage bills against your prepaid balance.

Response headers

Rate-limited responses (HTTP 429 or 529) include Retry-After. Quota-exhausted responses use HTTP 402 with a JSON body that includes the remaining balance and the credits-required link.

Exponential backoff

When InferAll returns 429 or 529, retry with exponential backoff and jitter. Prefer Retry-After when present.

# 429 / 529 response from InferAll
# Honor Retry-After when present, otherwise use exponential backoff.

attempt = 0
while attempt < MAX_RETRIES:
    response = http.post(url, headers=h, json=body)
    if response.status_code not in (429, 529):
        return response

    # Prefer server-supplied delay when available.
    delay_seconds = (
        int(response.headers.get("Retry-After", "0"))
        or (2 ** attempt) + jitter()
    )
    sleep(delay_seconds)
    attempt += 1

Recommended defaults: max 5 retries, base delay 1 second, multiplier 2, jitter up to 1 second. The OpenAI and Anthropic SDKs implement compatible behavior by default — using them as drop-in clients against InferAll Just Works.

Upstream provider limits

Individual upstream providers (Anthropic, OpenAI, Google, NVIDIA, Replicate, Runway) apply their own rate limits. InferAll's gateway watches for 429, 529, 5xx, and 30 second timeouts and transparently retries the same request against the next provider that can serve the requested model class. From your client's perspective, upstream rate limits typically do not surface as failures — see the SLA page for the full failover behavior.