Correction, 2026-09-02: pricing. An earlier version of this post said premium provider tokens are billed at the provider's rate with zero markup. That was not accurate: InferAll bills premium tokens at its own published per-token rates, which sit above the underlying provider cost. Current rates are on /pricing. The post has been corrected; nothing else about the models or the setup has changed.
If you're building with LangChain or LlamaIndex, you probably have OpenAI's API key hardcoded somewhere and an eye on your usage bill. You can route the same code to open-source models — Nemotron 3 Ultra, gpt-oss-20b, GLM 5.3, and more — for a fraction of the per-token cost, with two environment variables and no code changes. One ifu_ key reaches both open NVIDIA NIM endpoints and every major premium provider at the provider's published rate (published-rate).
LangChain
LangChain's ChatOpenAI accepts a custom base_url. Point it at InferAll:
from langchain_openai import ChatOpenAI
# Before: ChatOpenAI(model="gpt-4o", openai_api_key="sk-...")
# After: open-source model at NIM rate, same code
llm = ChatOpenAI(
model="nvidia/nemotron-3-ultra-550b-a55b", # NVIDIA NIM open model
base_url="https://api.inferall.ai/v1",
api_key="ifu_your_key_here", # get one at inferall.ai/keys
)
response = llm.invoke("What are the SOLID principles in software design?")
print(response.content)
Or use environment variables so your code stays unchanged:
export OPENAI_BASE_URL=https://api.inferall.ai/v1
export OPENAI_API_KEY=ifu_your_key_here
# then pass model="nvidia/nemotron-3-ultra-550b-a55b"
from langchain_openai import ChatOpenAI
# No changes to your existing code needed
llm = ChatOpenAI(model="nvidia/nemotron-3-ultra-550b-a55b")
LangChain with chains and agents
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
llm = ChatOpenAI(
model="nvidia/nemotron-3-ultra-550b-a55b",
base_url="https://api.inferall.ai/v1",
api_key="ifu_your_key_here",
)
# Standard LangChain chains work unchanged
prompt = ChatPromptTemplate.from_messages([
("system", "You are a helpful code assistant."),
("user", "{question}")
])
chain = prompt | llm | StrOutputParser()
result = chain.invoke({"question": "How do I implement a binary search tree in Python?"})
print(result)
LangChain with streaming
for chunk in llm.stream("Explain gradient descent step by step."):
print(chunk.content, end="", flush=True)
LlamaIndex
Install llama-index-llms-openai-like and use OpenAILike for third-party model IDs. The standard OpenAI wrapper validates names against OpenAI's own models.
from llama_index.llms.openai_like import OpenAILike
from llama_index.core import Settings
# Set InferAll as the LLM backend
Settings.llm = OpenAILike(
model="nvidia/nemotron-3-ultra-550b-a55b",
api_base="https://api.inferall.ai/v1",
is_chat_model=True,
api_key="ifu_your_key_here",
)
from llama_index.core.llms import ChatMessage
response = Settings.llm.chat([
ChatMessage(role="user", content="Explain a binary search tree in one paragraph.")
])
print(response)
This configures text generation. For vector retrieval, configure Settings.embed_model separately before building an index; Settings.llm does not configure embeddings. See the LlamaIndex OpenAILike reference for its model settings.
Which open model for LangChain development?
| Model | Use case |
|---|---|
nvidia/nemotron-3-ultra-550b-a55b |
Complex reasoning, longer context |
z-ai/glm-5.3-flash |
General purpose, instruction following |
z-ai/glm-5.3 |
General, handles code well |
openai/gpt-oss-20b |
Fast agent steps, OpenAI's open-weight model |
All on NVIDIA NIM at our open-model rate, the cheapest tier in the gateway.
Switch to premium models when production demands it
# Development / high-volume inner loop: open model on NIM
llm = ChatOpenAI(model="nvidia/nemotron-3-ultra-550b-a55b", ...)
# Production hard task: swap to Claude Sonnet at our published per-token rate
# Just change the model string — same base_url, same key
llm = ChatOpenAI(model="anthropic/claude-sonnet-4-6", ...) # or openai/gpt-4o
Get started
Sign up at inferall.ai/keys and fund a key with the $5 starter pack — that $5 becomes usage credit you can spend on any model (open or premium) at our published per-token rates (inferall.ai/models). See the LLM API aggregator overview for full details on supported providers and models.