Use free models in the llm command-line tool with InferAll

llm is a command-line tool for running prompts against many models. It can use any OpenAI-compatible API, so it works with InferAll's free models ($0 input, $0 output).

1. Save your key in llm

Get a key at inferall.ai/keys, then:

llm keys set inferall
# paste your key when asked

2. Add the models

llm reads extra OpenAI-compatible models from extra-openai-models.yaml in its config folder. Find the folder with dirname "$(llm logs path)", then create the file there:

- model_id: inferall-ultra
  model_name: nvidia/nemotron-3-ultra-550b-a55b
  api_base: "https://api.inferall.ai/v1"
  api_key_name: inferall
  supports_tools: true
- model_id: inferall-gpt-oss
  model_name: openai/gpt-oss-20b
  api_base: "https://api.inferall.ai/v1"
  api_key_name: inferall
  supports_tools: true

model_id is the short name you type; model_name is the id InferAll receives. Check with llm models | grep inferall.

3. Run a prompt

llm -m inferall-ultra "In one sentence, what is an API?"

Tool calling works too:

llm -m inferall-ultra --functions 'def multiply(a: int, b: int) -> int:
    "Multiply two integers"
    return a * b' "What is 1234 times 5678? Use the multiply tool." --td

What we tested (2026-10-21, llm 0.36)

Model	Plain prompt	Tool call (1234 × 5678)
inferall-ultra (Nemotron 3 Ultra)	answered, 10s	called multiply, answered 7,006,652, 3s
inferall-gpt-oss (gpt-oss-20b)	answered, 3s	called multiply, answered 7,006,652, 5s

Times vary with load on the shared free models. If one is busy, InferAll answers from another free model and says so in the response.

Paid models work the same way with a funded key: add an entry with, for example, model_name: anthropic/claude-sonnet-4-6.

InferAll is an LLM aggregator: one API key for free open models and paid OpenAI, Anthropic, Google and xAI models. New accounts get 25 free calls on open models, no card needed.