Correction, 2026-09-02 — pricing. An earlier version of this post said premium provider tokens are billed at the provider's rate with zero markup. That was not accurate: InferAll bills premium tokens at its own published per-token rates, which sit above the underlying provider cost. Current rates are on /pricing. The post has been corrected; nothing else about the models or the setup has changed.
Most AI apps are locked to one provider. When OpenAI has an outage, you're down. When Anthropic raises prices, you rebuild. When a better model launches, you rewrite your integration.
InferAll gives you one AI inference API key that routes to any provider — OpenAI, Anthropic, Google, NVIDIA, Replicate, and Runway. Switching is a parameter change, not a rewrite. Sign-up is a $5 starter pack that becomes usage credit you can spend on any model — open or premium — at our published per-token rates (inferall.ai/models).
One prompt, four providers
from openai import OpenAI
client = OpenAI(
base_url="https://api.inferall.ai/v1",
api_key="ifu_your_key_here", # get one at inferall.ai/keys
)
prompt = "What are the tradeoffs between SQL and NoSQL databases?"
# Route to any provider by changing just `model`
for model in [
"nvidia/nemotron-3-ultra-550b-a55b", # NVIDIA NIM — open model
"openai/gpt-oss-20b", # OpenAI gpt-oss — open-weight model
"z-ai/glm-5.3", # Z.ai GLM 5.3, open model
"anthropic/claude-sonnet-4-6", # Anthropic Claude — premium
]:
response = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}],
max_tokens=200,
)
print(f"\n=== {model.split('/')[-1]} ===")
print(response.choices[0].message.content)
The first three route to NVIDIA NIM open models at our open-model rate. The last bills at Anthropic's published per-token rate at our published rates. All four come off the same ifu_ key, on the same invoice.
Automatic failover
InferAll falls back to the next provider automatically when one fails. If the primary returns a 500, rate limit, or timeout, the gateway retries on the configured fallback chain — without any code in your application.
# This call retries on NVIDIA if Anthropic fails:
response = client.chat.completions.create(
model="anthropic/claude-sonnet-4-6", # primary
messages=[{"role": "user", "content": "Explain neural networks."}],
)
# provider=anthropic attempted first, nvidia fallback on failure
No retries in your application code, no provider-specific error handling.
Compare providers on the same task
import asyncio
async def compare(prompt: str, models: list[str]):
import httpx
results = []
async with httpx.AsyncClient() as http:
tasks = [
http.post(
"https://api.inferall.ai/v1/chat/completions",
headers={"Authorization": "Bearer ifu_your_key"},
json={
"model": m,
"messages": [{"role": "user", "content": prompt}],
"max_tokens": 150,
},
timeout=30,
)
for m in models
]
responses = await asyncio.gather(*tasks, return_exceptions=True)
for model, resp in zip(models, responses):
if isinstance(resp, Exception):
print(f"{model}: error")
else:
data = resp.json()
text = data["choices"][0]["message"]["content"]
print(f"\n{model.split('/')[-1]}:\n{text[:300]}")
asyncio.run(compare(
"Write a haiku about distributed systems.",
["nvidia/nemotron-3-ultra-550b-a55b", "z-ai/glm-5.3-flash", "openai/gpt-oss-20b"]
))
Route by task type
Different providers excel at different tasks. InferAll lets you route at the application layer:
def get_model(task: str) -> str:
if task == "code":
return "z-ai/glm-5.3" # general open model, good at code
elif task == "reasoning":
return "nvidia/nemotron-3-ultra-550b-a55b" # largest open model
elif task == "fast":
return "nvidia/nemotron-3-ultra-550b-a55b" # low-latency open model
else:
return "openai/gpt-oss-20b" # fast open default
These are all open models on NVIDIA NIM — billed at our open-model rate against your starter balance. Swap in gpt-4o or claude-sonnet-4-6 when a task earns premium spend.
task = "code"
model = get_model(task)
response = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": "Write a binary search in Python."}],
)
Open models available today
All of these route through NVIDIA NIM at our open-model rate:
curl https://api.inferall.ai/ai/v1/models \
| python3 -c "
import sys, json
models = json.load(sys.stdin)
nim = [(k, v) for k, v in models.items() if v.get('provider') == 'nvidia' and v.get('type') == 'token']
print(f'{len(nim)} open token models')
for k, _ in sorted(nim)[:10]:
print(f' {k}')
"
Get started
Sign up at inferall.ai/keys and fund a key with the $5 starter pack. That $5 becomes usage credit you can spend on any model — open or premium — at our published per-token rates (inferall.ai/models). Then point your existing OpenAI SDK at https://api.inferall.ai/v1 and pass any model ID in this post.