Use case
Choose a free open-source model for a small first task, then compare it with Claude when the task needs more reasoning. InferAll gives Claude Code one base URL for both. You choose the model; the gateway does not decide which coding tasks need Claude. The setup below also disables alternative routes so you can evaluate the model you selected.
Claude Code reads two environment variables to decide where to send requests: ANTHROPIC_API_KEY and ANTHROPIC_BASE_URL. Point those at InferAll, then choose a supported model. InferAll handles protocol translation for non-Anthropic providers. Tool and coding workflows still need validation with your client and chosen model.
Trial-eligible accounts can make 25 free NIM calls before payment. After the trial, the $5 starter pack becomes spendable balance and unlocks ongoing access. Free token rates still have daily request quotas. Sign in at /keys, copy the key (prefix ifu_), export the variables below, and run Claude Code. The custom-header setting requires Claude Code 2.1.227 or newer. If your client cannot send that header, selecting a model alone leaves automatic fallback enabled; see routing docs.
Claude Code — first read-only task with exact routing
# Create a key at https://inferall.ai/keys and keep it private.
# Trial-eligible accounts can try 25 free NIM calls before payment.
# After the trial, use /billing for the $5 starter pack.
export ANTHROPIC_API_KEY=ifu_...
export ANTHROPIC_BASE_URL=https://api.inferall.ai
# Claude Code 2.1.227+: keep this route, with no alternative-model fallback.
# Preserve other custom headers. If X-Inferall-Routing is already present,
# replace that line with "X-Inferall-Routing: exact" instead of appending it.
export ANTHROPIC_CUSTOM_HEADERS="${ANTHROPIC_CUSTOM_HEADERS:+${ANTHROPIC_CUSTOM_HEADERS}
}X-Inferall-Routing: exact"
# Start with a read-only task in a small repository. Review the answer.
claude --model nvidia/nemotron-3-ultra-550b-a55b -p "Read README.md and summarize what this project does. Do not edit files."
# An error is a failed first attempt, not a completed workload.
# Check its message for model, access, quota or provider availability issues.The honest framing is that a free OSS model and Claude aren't interchangeable; they have different sweet spots. The job is to route by task class, not to pick once and hope.
Try the free NIM route on a small read-and-summarize task, then inspect whether the result is useful. Test tool use and single-file edits separately before expanding to a larger coding workflow. A model listing or one successful response is not evidence that it will handle your whole project.
For a comparison with paid Claude, select anthropic/claude-sonnet-4-6 or anthropic/claude-opus-4-8. Use the same task and review the output before deciding which route fits your quality, latency and cost requirements.
Keep the exact-routing header configured when switching between models if you want failures returned instead of an answer from an alternative route. Claude Code's in-session /model command makes this a one-line switch.
A free-model request only falls back to other eligible free models by default. To keep one specific model, send X-Inferall-Routing: exact as in the setup above. Default auto mode can change models; paid requests can also change providers. Exact mode returns an error when the selected route fails. It does not bypass access, billing or context limits.
Pin and switch
# With the exact-routing header from setup still configured:
claude --model nvidia/nemotron-3-ultra-550b-a55b
# In the session, choose paid Claude explicitly when you need it:
# /model anthropic/claude-sonnet-4-6
# This requires paid access and is billed at InferAll's published model rate.
# Return to the free route with:
# /model nvidia/nemotron-3-ultra-550b-a55b
# The model selection alone does not disable automatic fallback.
# Exact mode keeps the selected route or returns an error.InferAll catalogs 10+ open-source models hosted on NVIDIA NIM. Check the catalog for current availability and $0 input and output rates before choosing a route. Examples to evaluate:
The full current list lives in the model catalog. Listings can change and do not guarantee provider uptime, latency or successful tool use. Use exact routing for model comparisons so a fallback cannot change the result.
The shape of the savings depends on your mix of cheap vs complex turns. We won't invent specific dollar figures because they shift with provider pricing and your own usage pattern. The structural argument:
If 60% of your Claude Code tokens are cheap turns that an OSS model handles well, routing those turns to a NIM open model (billed at $0 input/$0 output) while keeping the complex 40% on Claude (at our published per-token rate) eliminates the cost of the cheap 60% entirely. The exact savings depend on your turn-shape mix; the structural point is that you stop paying premium prices for cheap turns.
The math below is a worked example. Substitute your own token volumes and your own current Anthropic billing rate to get a number that matches your situation.
Worked example
# Illustrative pricing math — not a guarantee, not a promise.
# Replace these with your own numbers from /docs and /#pricing.
# Hypothetical month of Claude Code usage:
# Total prompt + completion tokens: 5,000,000
# Mix: 60% cheap turns (file reads, summaries, classification)
# 40% complex turns (multi-file edits, planning, reasoning)
# Option A: every turn routes to Claude.
# 5,000,000 tokens at Anthropic's published rate = $X total
# (where $X depends on the Claude model and the
# per-million-token rate at the time you do this math.)
# Option B: through InferAll, cheap turns route to free OSS.
# 3,000,000 tokens to NIM-hosted open models:
# $0 input / $0 output (daily request limits apply, see /#pricing)
# 2,000,000 tokens to Claude at InferAll's Claude rate:
# use the current input and output rates shown on /models.
#
# Compare input and output tokens separately at each model's current rate.
# Include retries and any extra calls needed to get a usable result.
# Use exact routing if the comparison must keep the selected models.Free OSS first; Claude only for files where the OSS review came back vague. The cheap turns — “is this file consistent with the rest of the module” — are what OSS handles well. The expensive turns — “does this change break invariants in three other files” — are what Claude is worth paying for.
Try a free model on a small diff and verify that the message describes the change accurately. Keep exact routing enabled when comparing models, then choose the result that meets your needs.
Claude. Multi-file consistency is where premium models earn their cost. Free OSS can do single-function renames; multi-file refactors with type changes or contract changes are Claude territory.
Mix. Pin a free model for the question loop (“what does this library do, what's the pattern here, give me a starter example”) and bump to Claude when you're ready to write the production version. Most of the tokens land on the cheap side; the expensive ones are short and worth their cost.
Be careful. Agent loops can burn the daily request quota quickly because they call the model repeatedly on every step. Cap loop turns and client retries, and check the dashboard and your current plan limits. Free token pricing does not remove request quotas. The cost story for long-running agents is fundamentally different than for an interactive Claude Code session.
We built InferAll because we wanted exactly this workflow for ourselves — Claude Code routed through a gateway with a free open-source allowance, premium models on tap when we needed them, one base URL and one bill. The market had either OSS gateways we'd have to operate, observability platforms that didn't solve the routing problem, or single-provider keys that locked us into one upstream. So this page is biased: we're describing the workflow we're selling. The technical claims should still hold up if you check them, and the corrections email at the bottom is real. If we got something wrong, tell us.
Will my code be sent to NVIDIA when I route Claude Code to an open NIM model?
Yes. The prompt, including code you include, is forwarded to NVIDIA for a NIM-hosted model. Choose a provider approved for your code and data. A provider prefix selects the initial route; default automatic fallback can change models, and for paid requests it can change providers. Set X-Inferall-Routing: exact to disable alternative routes. Free-model requests only fall back to other eligible free models.
Can I disable fallback so Claude Code stays on my selected model?
Yes. In Claude Code 2.1.227 or newer, send X-Inferall-Routing: exact through ANTHROPIC_CUSTOM_HEADERS, as shown below. Keep any other required custom headers. A model ID alone selects the initial route; exact disables alternative-model/provider fallback and parallel backup requests. If the selected route fails, the request fails. It does not bypass trial access, billing, quotas or model context limits. A provider-owned alias may still change upstream; exact does not guarantee a historical model version.
Which Claude model gets used when I switch a turn?
Use an explicit supported ID such as anthropic/claude-sonnet-4-6 or anthropic/claude-opus-4-8. In exact mode, InferAll keeps that route or returns an error; in auto mode, fallback may answer with another model. On a trial account, a bare Claude ID such as claude-sonnet-4-6 is refused with HTTP 400, not answered by a free NIM substitute. Accounts that have paid can also route supported bare Claude IDs, subject to access and balance. Exact mode refuses aliases that InferAll would rewrite to a different model ID; choose the canonical ID instead.
How much does it cost to use Claude Code through InferAll?
Trial-eligible new accounts get 25 free NIM calls before payment. After the trial, the $5 starter pack unlocks ongoing access and becomes spendable balance. Models listed at $0 input and $0 output do not draw from that balance; daily request quotas still apply. Anthropic and other premium text models use InferAll's published per-token rates. Check /models for model prices and /pricing for current plan limits; an agent session can make several requests per task.
Does Claude Code know it is hitting NVIDIA instead of Anthropic?
Claude Code sends Anthropic-format requests to ANTHROPIC_BASE_URL. InferAll translates them for the selected upstream where needed. Set a model explicitly: nvidia/nemotron-3-ultra-550b-a55b selects a NIM model, while anthropic/claude-sonnet-4-6 selects paid Anthropic. A bare Claude ID on a trial account is refused. For HTTP-level checks, inspect X-Inferall-Actual-Provider, X-Inferall-Actual-Model and X-Inferall-Fallback-Used; a client may not display those headers.
What about tool calls and file edits — do free OSS models handle them?
The gateway accepts Anthropic-format tool requests and translates them for supported routes, but model and client behavior still matter. Start with a read-only task on a small repository, inspect the answer and any proposed tool calls, then test file edits separately. A successful text response does not prove a complete coding workflow. Compare models on your own tasks before relying on unattended edits.
How do request and context limits work?
Calls count toward the request quota for your plan and operation, which resets each UTC day. A Claude Code task can require multiple calls. Free token pricing does not remove quotas or context limits. If a prompt is too large, shorten it or explicitly select a supported model with a suitable context window; do not rely on an automatic move to Gemini. See /pricing and /models for current limits and model details.
InferAll home — the gateway, the $5 starter, the failover story.
Free LLM API for AI agents — run agent loops on free OSS models, pin a frontier model for the hard steps.
InferAll for VS Code — Cline-based agent with the gateway pre-wired, free first run.
Security — data handling, trust boundaries, what leaves your machine.
Rate limits — free-tier caps, per-key behavior, what happens on 429.
Pricing — current plans and request quotas.
AI inference API — endpoint surface, supported providers, code examples.
Unified AI API — one key, one bill, every provider.
Last updated: 2026-10-07.
InferAll facts on this page are drawn from this site and the gateway running at api.inferall.ai. Model availability on NVIDIA NIM may shift as upstream hosts change their roster. Specifics change. Have a correction? Email contact@kindly.fyi.