A thought paper for the operator who trusted the model, watched the accuracy hold, and still isn’t sure anything moved.
You Did This Properly
You didn’t rush it. You picked a real problem, not a vanity one. You checked the data before you committed, not after. You watched the pilot before you scaled it. The accuracy held up in production, not just on the vendor’s slide.
Nobody in the business can say you moved carelessly.
And yet.
There’s a feeling. Not that the model is wrong — the model is fine, by every measure anyone thought to build. Something else. The dashboard keeps being green. The number you actually watch — the one that made you sign off on this in the first place — hasn’t moved the way the business case said it would. You keep waiting for the result to catch up to the story you were told it would tell. It hasn’t, quite.
The Hunter With the Better Scope
Give a hunter a better scope and something real changes that same night. He can see the animal four hundred metres out, in bad light, at a distance nobody in the camp could manage a year ago. He is, honestly, a better hunter than he was.
What the scope cannot do is tell him when the animal has changed its trail, stopped coming through that valley at all, or been replaced by something the camp has never hunted before. A rifle only gets better at hitting what’s already lined up in front of it. It says nothing about whether that’s still the right thing to be aiming at.
The rifle got better. Knowing whether the jungle was still the same jungle did not.
This is the part nobody mentions when they sell you the forecast, the model, the AI. They sell you the scope. Nobody mentions that the jungle keeps moving after the sale closes.
Sometimes the animal never moves at all, and the ground still shifts underneath the hunt anyway. A season arrives early. A trail nobody used becomes the only way through. Nothing about the shot changed — the world the shot was aimed into did, and a hunter who only ever re-checks his aim will never notice.
The jungle does not reward that kind of stillness for long.
It is where forecast models go to die — not because they missed, but because they kept hitting the same mark, accurately, while the ground underneath it had already moved.
A model that scores perfectly against yesterday’s jungle has not learned the jungle. It has only memorised where the animal used to stand.
A Different Distinction
Here is the gap in plain language: a forecast and a target are not the same instrument, even though they usually arrive wearing the same number.
The forecast reads the world as it is. The target is what leadership has decided the business needs it to be. One is a measurement. The other is a decision. The trouble starts the moment the two get blended into a single figure and both get called “the forecast” — because once that happens, nobody downstream can tell which part they’re looking at, and the reading stops being testable against reality at all.
A forecast and a target are different instruments doing different jobs. Most businesses have never cleanly separated them.
The model can read fifty factors. It cannot read the difference between a business that wants an honest number and a business that wants a number it can defend at the next review — and reading fifty factors instead of five does not change which one it’s handed.
Three Questions
Not diagnostic questions from a framework. The questions an operator asks themselves quietly, on the walk back, when the model’s dashboard is green and the business result still feels like it’s a step behind.
One.
The number in this quarter’s plan — is it what the model actually predicted, or the model’s number with leadership’s ambition already folded in?
If you can’t answer that cleanly, you don’t have a forecast anymore. You have a target wearing one.
Two.
The last time someone standing in the actual market saw something change — a competitor’s move, a road that closed, a season arriving early — how did that reach the model?
If the honest answer is ‘it didn’t, until the numbers proved it months later’ — nobody equipped that person to tell you, and the model is still hunting last season’s map.
Three.
The last time the model was wrong — and it was wrong somewhere — did that finding change the model, or did it just get absorbed into next quarter’s explanation?
Most organisations can tell you their model’s accuracy to two decimal places. Far fewer can tell you what happened the last time it missed, and whether the model is any different for it.
The old hunters never trusted the scope alone. They asked what the season was doing before they ever left camp — and it’s worth noticing what that habit would have caught, in each of the two that follow.
Two Ways the Jungle Bites Back
We have watched a model draw the same clean line to the same shortage, in the same spot, and be right — in daylight. Weeks later, same shelf, same product, the light had changed: a festival two streets over, a competitor’s push, a road that wasn’t open anymore. The model kept aiming exactly where it had aimed before, because nothing in it knew the light had moved. The jungle wasn’t testing the model’s accuracy that week. It was testing whether the model knew it was standing in a different jungle. It didn’t.
And we have watched the jungle empty out entirely. A drought year, when the game doesn’t move to a different trail — it simply isn’t there. No scope finds an animal that has left the region. No trail connects a camp to meat that doesn’t exist. In that season the only wrong move is to keep aiming as if the problem is precision, when the problem is that nothing is within range of anyone’s rifle. The hunters who get through a drought are not the ones who fire more accurately. They are the ones who recognise, early and together, that this is a different kind of season — and stop hunting the way they hunted last month.
Neither of those is a model failure. Both are jungle failures — the kind that only show up once you’ve walked the same ground, in every season, more than once.
What the Camp Tells Itself
There’s a way this fails that has nothing to do with the jungle changing at all — and it’s older than any model the camp has ever bought.
The chief does the same thing at a larger scale, and has for longer than anyone remembers. Long before a dashboard existed, the chief announced how much meat the camp would eat this moon, and everyone’s job became making that number true — not checking whether it had ever been the right one. A model that reads fifty factors instead of one hunter’s gut doesn’t retire that habit. It just hands the camp a more convincing number to chase.
The people who could tell you the truth about the jungle are the same people whose job is to make the number look right. That was true before the model arrived. Reading fifty factors correctly does not change it.
And the distortion reaches back further than the forecast itself. Last summer, when the take was thin, the camp put twenty more hunters on the trail and doubled what it offered traders for an early share. A model can learn that pattern — more hunters, more trade, more meat. What it cannot learn is how much of that summer was calm planning and how much was panic, because nobody ever wrote that difference down. A model trained on last summer’s ground has partly learned to predict panic, and calls it demand.
Nor does the truth reliably reach the model on the way in. The hunter who first notices a competitor’s camp two ridges over, or a trail that’s closed, or a season arriving early — that hunter usually has nowhere to put what he’s seen, and nobody ever told him it mattered. The model doesn’t fail to learn the jungle changed. It’s never told.
A hunter comes back and says the animal bolted, when the truth is the shot missed. Once that story works, it gets told again — not to hurt the camp, just because it costs the hunter nothing to tell it. It costs the camp everything, quietly, because the fire ends up planning around a version of the hunt that never happened.
A camp that only listens to its own hunters, and never checks the story against the ground, will believe it’s well fed right up until it isn’t.
The Quieter Fact
The gap between a forecast that reads well and a business that trusts it does not behave like a technology problem. It behaves like a role that was never defined — what the forecast is for, who it answers to, and what it is not being asked to do.
A forecast was never built to move the business. That’s a different muscle — leadership, incentives, the parts of the organisation whose entire job is making numbers move. The forecast’s job is to bring a calm, honest reading into a room that has, for generations, defaulted to panic the moment a number looked bad. Reading fifty factors instead of five does not change that job. It only makes the reading harder to argue with, if the business gives it a place to do that job at all.
This is an old mistake wearing a new instrument. Something meant to read the world gets asked to move it instead, and the two jobs get called by the same name until nobody can tell which one they’re actually holding.
The hunters who eat well are rarely the ones with the best scope. They’re the ones whose camp had already decided what the forecast was for, before the season tested it.
A prepared business has done exactly that — given the forecast a clearly defined role, tuned to how the business actually behaves and where it actually wants to go, so the data has somewhere real to land instead of a review it has to survive.
If you can’t say cleanly, right now, whether this quarter’s number is what the model predicted or what leadership needed it to say — that’s worth a conversation.
The Prepared Business
We help organizations give the forecast a clearly defined role in the business — so a forecast reads the world, a target motivates the people, and nobody downstream has to guess which one they’re looking at.
Leave a Reply