Model IDs Go Stale: How InferAll Keeps Your LLM Calls From Silently 404ing

There's a failure mode that almost every LLM app ships with and nobody plans for: the model ID you hardcoded months ago quietly stops existing. It doesn't throw at deploy time. It doesn't show up in tests. It just sits there until the day the provider retires that ID — and then every request pinned to it starts coming back `404 model_not_found`. If that model was on a hot path, you find out from your users. This isn't hypothetical, and it isn't rare. It's a structural property of building on top of fast-moving providers. ### Model IDs drift constantly Pick any provider and the catalog is moving under your feet: - **OpenAI** retired the **DALL·E** image models in favor of the newer `gpt-image` family. Apps that hardcoded `dall-e-3` for image generation now hit an invalid-model error. - **Google** sunset the **Gemini 1.5** line as 2.x became the default. The old `1.5` IDs stopped resolving. - **Anthropic** publishes **dated Claude snapshots** (the `claude-…-YYYYMMDD` form) and deprecates older ones over time. Pin a snapshot, and it has an expiry date you didn't write down. - **NVIDIA NIM** rotates open-model IDs as new versions land — a model you called last quarter may be served under a new ID today. Now multiply that by every provider you use. The whole point of a multi-provider setup is access to all of them — but it also means you've signed up for *all* of their deprecation schedules at once. Hardcode IDs across four providers and you're effectively running four countdown timers, none of which page you when they hit zero. ### Why this is a gateway's job, not yours A neutral gateway sits between your code and every provider. That's exactly the right place to absorb drift, because it can do two things your application can't easily do on its own. **1. Keep deprecated IDs working with transparent aliases.** When a provider retires a dated snapshot, we don't pass the 404 through to you. If you're still pinned to a deprecated Claude snapshot, the gateway routes it to the current equivalent — and labels the response with the model that actually served the request, so there's no fiction about what ran. Your code keeps working while you migrate on your own schedule, instead of during an incident. **2. Detect dead models automatically — before a user does.** You can't fix drift you can't see. So we run an automated health check across the entire catalog (190+ models) on a schedule. The interesting part is *how* — because the naive approach is a trap. ### The $0 way to health-check a model catalog The obvious idea is to ping every model with a tiny request and see what fails. Don't. For a catalog with ~186 free open models, that's hundreds of inference calls per run — it costs money on paid models, and it hammers the very free tier you depend on for live traffic. You'd be degrading production to monitor it. There's a cleaner signal that costs nothing: **every major provider publishes a live list of the models it currently serves** (`GET /v1/models` on OpenAI- and Anthropic-style APIs; the equivalent on NVIDIA and Google). A model that the provider no longer lists is, by definition, a model that will 404 when you call it. So the check is a set-membership test, not an inference sweep: ```text for each model in our catalog: provider_live_ids = GET {provider}/v1/models # one cheap call per provider, cached if model not in provider_live_ids: flag as DEAD # the provider stopped serving it elif model is a known-deprecated alias → current: flag as ALIASED # still works via the gateway, but the ID is stale else: OK ``` One list fetch per provider, cached — not one call per model. It's free, it's fast, and it catches the exact thing that breaks apps: an advertised ID the provider has dropped. We also do a real, minimal liveness probe of just the handful of free models we route to *by default*, since those are load-bearing — but that's four calls, not several hundred. We expose the result at a public endpoint: ```bash curl https://api.inferall.ai/ai/v1/models/health ``` ```json { "ok": true, "summary": { "total": 199, "alive": 189, "aliased": 1, "dead": 0, "unchecked": 9 }, "hotRoutes": [ { "id": "meta/llama-3.1-70b-instruct", "status": "alive" }, "…" ] } ``` When `dead` goes above zero, it fails a daily job and alerts us — so a provider deprecation becomes a fix we ship quietly, not an outage you debug. ### The takeaway: target a stable interface The durable lesson isn't "memorize the deprecation calendars." It's *don't pin to a moving target in the first place.* Two habits make drift a non-event: 1. **Read the catalog at runtime, not from memory.** The current, live list is always one call away — don't trust a hardcoded table (including one in a blog post) that can go stale: ```bash curl https://api.inferall.ai/ai/v1/models ``` 2. **Let the layer between you and the providers absorb the churn.** That's the whole value of a gateway: you target one stable interface and one key, and provider-side deprecations, renames, and rotations get handled below you. You're going to keep shipping. The providers are going to keep deprecating. Those two facts don't have to collide in production. If you want the stable-interface version of this — one key, every provider, and a catalog that's actively monitored for drift — you can [start on the free tier](https://inferall.ai): 100,000 tokens/month across 110+ open models, using the OpenAI or Anthropic SDK you already have.