Correction, 2026-09-02: pricing. An earlier version of this post said premium provider tokens are billed at the provider's rate with zero markup. That was not accurate: InferAll bills premium tokens at its own published per-token rates, which sit above the underlying provider cost. Current rates are on /pricing. The post has been corrected; nothing else about the models or the setup has changed.

Update, 2026-09-12: this model is not answering. google/gemma-4-31b-it is still listed in NVIDIA's catalogue and still accepts requests, but it stopped returning responses. We measured it six times across two hours (at 45s, 30s and 60s timeouts, sequential and spaced so the probe was not hitting our own rate cap), and every attempt timed out. It is not retired or removed upstream; it simply never replies, which is worse than an error because your request hangs instead of failing fast.

We have removed it from the InferAll catalogue so it cannot be handed to you as a suggestion. The snippets below now use openai/gpt-oss-20b, which is also $0 in / $0 out and answered short prompts in a median 1.3s over the week to 2026-09-29. (They used GLM 5.3 Flash until then; its median for short replies had grown to about 28s, so we switched.) If Gemma 4 starts responding again we will re-add it and update this post.

The free NVIDIA NIM tier through InferAll still gives you $0 input/output models. No credit card, no billing setup. Create a key and call one now.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.inferall.ai/v1",
    api_key="ifu_your_key_here",  # get one at inferall.ai/keys
)

response = client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[{"role": "user", "content": "What are Gemma 4's key improvements over Gemma 3?"}],
    max_tokens=512,
)

print(response.choices[0].message.content)

What is Gemma 4?

Gemma 4 is Google's fourth generation of open-weight foundation models. The 31B instruction-tuned variant (gemma-4-31b-it) offers strong performance on reasoning, coding, and instruction following, significantly more capable than the Gemma 3 family while remaining fully open-weight and free to run via NVIDIA NIM.

Like all Gemma models, it's fully open-weight under Google's Gemma Terms of Use, available for commercial use, and hosted without charge on NVIDIA's DGX Cloud infrastructure.

TypeScript / Node.js

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inferall.ai/v1",
  apiKey: process.env.INFERALL_API_KEY,
});

const response = await client.chat.completions.create({
  model: "openai/gpt-oss-20b",
  messages: [{ role: "user", content: "Write a Python function to parse JSON." }],
});

console.log(response.choices[0].message.content);

Streaming

with client.chat.completions.create(
    model="openai/gpt-oss-20b",
    messages=[{"role": "user", "content": "Explain transformer attention in plain English."}],
    stream=True,
) as stream:
    for chunk in stream:
        print(chunk.choices[0].delta.content or "", end="")

Claude Code / Cline / Cursor

export ANTHROPIC_BASE_URL=https://api.inferall.ai
export ANTHROPIC_API_KEY=ifu_your_key_here
export ANTHROPIC_MODEL=openai/gpt-oss-20b

Free alternative on InferAll

Model	Size	Notes
`openai/gpt-oss-20b`	20B	OpenAI's open-weight model; median 1.3s for short replies, week to 2026-09-29

All are free on NVIDIA NIM. The full model list is always live at the API.

Compare with other free models

# gpt-oss-20b vs Nemotron — one prompt, two free models
models = [
    "openai/gpt-oss-20b",
    "nvidia/nemotron-3-ultra-550b-a55b",
]

for model in models:
    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": "What year was the transformer paper published?"}],
        max_tokens=50,
    )
    print(f"{model.split('/')[-1]}: {resp.choices[0].message.content.strip()}")

Get started

inferall.ai/keys — sign up free and start calling (25 free NIM calls before any payment; the $5 starter pack unlocks ongoing use) at /billing. The $5 becomes spendable balance: 10+ open NIM models stay $0 in/out against it (within the free-plan daily request caps); premium providers (OpenAI, Anthropic, Google) bill at our published per-token rates.

Google Gemma 4 31B free API — not responding, and what to use instead