Meta Llama 3.3 70B — OpenAI-compatible API

Update, 2026-08-27 — meta/llama-3.3-70b-instruct has been retired. NVIDIA end-of-lifed the Llama 3.1/3.2/3.3 family on its DGX Cloud tier at 2026-08-26T09:00:00Z; calls to this id now return 410 Gone. The historical discussion below remains as context, while runnable snippets have been updated away from the retired ID.

To run the same examples today, substitute a model that is currently live — as of this update nvidia/nemotron-3-super-120b-a12b, which is $0 in/out on the same free tier and works on the OpenAI-compatible snippets below. The free roster changes on NVIDIA's schedule, not ours, so the durable move is to fetch GET /ai/v1/models and pick from what it returns rather than hardcoding any id from a blog post — including this one.

This article originally covered Meta's Llama 3.3 70B, the refined final iteration of the Llama 3.x 70B line. That exact route is no longer served. The runnable examples below now use nvidia/nemotron-3-super-120b-a12b, a current $0 NVIDIA NIM model on the same OpenAI-compatible path.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.inferall.ai/v1",
    api_key="ifu_your_key_here",  # get one at inferall.ai/keys
)

response = client.chat.completions.create(
    model="nvidia/nemotron-3-super-120b-a12b",
    messages=[{"role": "user", "content": "Explain the difference between Llama 3.1, 3.3, and 4."}],
    max_tokens=400,
)

print(response.choices[0].message.content)

---

Llama 3.3 70B vs 3.1 70B vs Llama 4

Before the NVIDIA retirement, the Llama family choices looked like this:

Llama 3.1 70B (meta/llama-3.1-70b-instruct) — the original stable 70B model. Widely tested, reliable baseline while it was served.

Llama 3.3 70B (meta/llama-3.3-70b-instruct) — the refined version. Better instruction following, improved math and reasoning, same 70B architecture.

Llama 4 Maverick (meta/llama-4-maverick-17b-128e-instruct) — a later MoE-family model that is also not served through this free route today.

Do not copy those retired IDs into production code. Use GET /ai/v1/models for the current roster.

---

TypeScript / Node.js

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inferall.ai/v1",
  apiKey: process.env.INFERALL_API_KEY,
});

const response = await client.chat.completions.create({
  model: "nvidia/nemotron-3-super-120b-a12b",
  messages: [{ role: "user", content: "Summarize the key differences between REST and GraphQL." }],
});

Streaming

with client.chat.completions.create(
    model="nvidia/nemotron-3-super-120b-a12b",
    messages=[{"role": "user", "content": "Write a guide to async programming in Python."}],
    stream=True,
) as stream:
    for chunk in stream:
        print(chunk.choices[0].delta.content or "", end="")

Claude Code / Cline / Cursor

export ANTHROPIC_BASE_URL=https://api.inferall.ai
export ANTHROPIC_API_KEY=ifu_your_key_here

For Anthropic-compatible clients, pin a current free model explicitly when you want the $0 route, or use an anthropic/ prefix when you want real Claude.

---

Current free replacements

Model	Size	Notes
nvidia/nemotron-3-super-120b-a12b	120B	Current hot-route-probed free default
google/gemma-4-31b-it	31B	General-purpose open model
mistralai/mistral-nemotron	MoE	Mistral-family free route
minimaxai/minimax-m3	V4 Flash	Lower-latency general/coding route

All are on the current free roster as of this update. Fetch the live catalog before hardcoding one.

---

Get started

Sign up at inferall.ai/keys and fund a key with the $5 starter pack — that $5 becomes usage credit you can spend on any model (Llama, GPT, Claude, Gemini, NIM open models) at the provider's published rate with zero markup.