Sep 2026
Same Name, Different Model: The Hidden Risk Behind Every LLM Deployment
TL;DR
- The model behind an LLM API can change overnight, same name, same endpoint, different behaviour, because vendors quietly retrain or re-tune it. A Stanford and UC Berkeley study found GPT-4's accuracy on a simple prime-number task fall from 84% to 51% in just three months, with no warning to anyone using it.1
- This has already caused real harm: Air Canada was held legally liable for its chatbot inventing a discount policy that didn't exist.2 New York City's official business chatbot told owners they could legally take staff tips and turn away housing voucher tenants, both illegal.3
- Most organisations can't see this happening. Gartner estimates only 15% of generative AI deployments today have any real behavioural monitoring in place, the rest can only confirm the system responded, not whether the response was right.4
- Financial services carries the sharpest exposure: core systems are deterministic, a balance has one correct answer, while an LLM is probabilistic. When OpenAI quietly retired several models overnight at the launch of a new one, it wasn't hypothetical, customers felt it first, real users losing tools they depended on, with zero notice.5
The model you tested isn't always the model you're running
Here's the part of LLM risk that catches most teams off guard. You validate a model once, sign off on it, and start using it in production. But the version behind that API call can change without you doing anything at all. Same name. Same endpoint. Different model.
This isn't a hypothetical edge case. Researchers at Stanford and UC Berkeley tested GPT-4 and GPT-3.5 on the same tasks, three months apart, using nothing but the standard public API.
On one task, telling prime numbers from non-prime numbers, GPT-4's accuracy dropped from 84% to 51% in 3 months. On a stricter version of the same test, it fell from 97.6% to 2.4%. Nobody using the API was told this had happened. The model's name hadn't changed. Its behaviour had.1
It's a bit like ordering the same dish at a restaurant every week and one day it tastes completely different, no new item on the menu, no announcement, just a different result from the same order.
There's a genuine debate about what caused this. Some researchers argue the model didn't get worse at reasoning, it just stopped formatting its answers the way it used to, which broke how the test was scored.
That's a fair point, but it doesn't make the risk smaller. If a small, invisible tuning change can break how a model formats its output, it can just as easily break whatever your system depends on.
The lesson isn't "GPT-4 got dumber." It's that the same input can produce a materially different output after a change nobody told you about, and there was no error message when it happened.
Traditional software gives you the same output for the same input, every time, because someone wrote an explicit rule for it. An LLM gives you the most likely output based on what it learned, and that likelihood shifts the moment the underlying weights shift, even slightly. A single word added or removed from a prompt, or a small adjustment made during a routine fine-tuning pass, can be enough to nudge the model toward a different answer, a different format, a different tone.
What this looks like when it goes wrong
This isn't theoretical. A few real cases show exactly how it plays out.
Air Canada lost a legal case after its website chatbot told a grieving customer he could claim a bereavement fare discount retroactively, a policy that never existed. Air Canada actually argued in court that the chatbot was responsible for its own words, not the company. The tribunal rejected that outright, stating "if you deploy it, you own what it says". The payout was small, a few hundred dollars, but the precedent is the real story.2
New York City's official small-business chatbot told users they could legally take a share of their employees' tips and could refuse tenants using housing vouchers, both illegal under New York law. Different users asking the identical question sometimes got different, contradictory answers. The city left it running with a disclaimer for months before shutting it down.3
A UK delivery company's chatbot swore at a customer and wrote a poem calling its own employer "useless," triggered by a routine system update. The company's own explanation was blunt: "something changed after an update, and the bot's behaviour changed with it".6
And when OpenAI launched a new flagship model in 2025, it quietly removed close to nine older models from its consumer product overnight, with no warning. Users who had built habits, prompts, and workflows around those specific models lost them without notice. After enough of an uproar, OpenAI restored one of them within days and publicly admitted it should have handled the transition differently.5
None of these are exotic failures. They're the direct, visible cost of a system whose behaviour can change on someone else's schedule, without your knowledge.
Why heavily regulated firms feel this hardest
Most software is deterministic in a way, ask the same question, get the same answer, every time. However, an LLM isn't. Ask it the same question twice and you might get two different answers. And if the underlying model quietly changes behind the scenes, you might get a third answer next month that looks nothing like the first two.
For a regulated firm, that's a genuinely different category of risk than a normal IT system failure. A hallucinated interest rate, an invented compliance exemption, or a fabricated transaction detail isn't a glitch you patch and move on from.
It could lead to potential customer harm and a regulatory finding waiting to happen. It's part of why several major firms have deliberately kept public-facing generative AI on a short leash rather than let it run unsupervised in customer conversations.
The bigger problem that almost nobody is actually watching
Here's the uncomfortable part. Most organisations running these systems in production have no real way of knowing when this kind of change happens. Gartner estimates that only around 15% of generative AI deployments today have genuine behavioural monitoring in place. The rest can confirm the system is technically running, but nothing about whether what it's saying is still correct, on-policy, or consistent with what was tested at launch.4
Separate industry research on AI agents in production found that observability, the ability to actually see what the system is doing, is one of the biggest gaps between a pilot and a reliable production deployment.
This is the practical shape of the risk. A system that can change without notice, running inside an organisation that has no way of noticing.
What should actually be in place
None of this means LLMs can't be used safely, it means the controls have to match how these systems actually fail, not how traditional software fails.
A few things matter most. Pin to a specific, dated version of the model rather than a generic name that the vendor can silently redirect. This alone closes most of the "same name, different model" problem. Track vendor deprecation notices as seriously as a patch calendar, because models get retired on the vendor's timeline, not yours.
Build a small set of known-answer test questions specific to your use case, and re-run them regularly and immediately after any suspected model update.
And treat this as a governance question, not just an engineering one. Someone needs to own watching for this, with an audit trail showing what version was live, when, and what its outputs looked like, because a regulator or a court will eventually ask.
The core insight is a simple one, the model you approved and the model you're running are not guaranteed to be the same thing, indefinitely.
References
- Chen, Zaharia, Zou, "How Is ChatGPT's Behavior Changing over Time?" — arxiv.org/abs/2307.09009
- Moffatt v. Air Canada, 2024 BCCRT 149 — canlii.org
- The Markup, "NYC's AI Chatbot Tells Businesses to Break the Law" — themarkup.org
- Gartner, "Gartner Predicts By 2028, Explainable AI Will Drive LLM Observability Investments to 50% for Secure GenAI Deployment" — gartner.com
- Marketing AI Institute, on the GPT-5 launch and model deprecations — marketingaiinstitute.com
- AI Incident Database, entry 631 — incidentdatabase.ai