On the Limits of Prediction and Forecasting

Consider how rarely anyone keeps score. The airwaves and feeds are thick with prediction — about markets, elections, technologies, conflicts, the weather of next decade and the price of next week — and almost none of it comes attached to the only thing that would let you evaluate it: a record of how the forecaster has done before. We consume confidence and call it insight. The two are not the same.

This is a long argument about a short idea: that the value of a forecast lies almost entirely in information we are usually not given, and that learning to ask for that information would change how we read nearly everything. It is not an argument against forecasting. It is an argument for doing it honestly, and for treating the limits of prediction as facts to be respected rather than embarrassments to be hidden.

The two kinds of hard

It helps to separate two reasons a prediction can be difficult, because they call for different responses.

The first is ignorance that can, in principle, be reduced. If I do not know how many people live in a city, I can go and count. More data, better instruments and more careful analysis genuinely help. A great deal of useful forecasting lives here — actuarial tables, epidemiological models, the steady improvement of weather prediction over recent decades to the point where a few days out is now remarkably reliable.

The second kind of hard is different in nature. Some systems are so sensitive to their starting conditions that tiny, unmeasurable differences grow into large divergences over time. This is the insight at the heart of chaos theory, and it is why even an excellent weather model degrades as it reaches into the future: not because the science is poor, but because the atmosphere itself amplifies the smallest uncertainty. No amount of additional data fully tames such a system. The limit is structural.

The mistake we make constantly is treating the second kind of problem as if it were the first — assuming that a confident-enough expert with enough data can predict what is, in fact, intrinsically unpredictable beyond a certain horizon. The forecaster who admits “beyond here, I genuinely cannot say” is often more trustworthy than the one who projects a crisp number into the fog.

Calibration: the skill no one is graded on

There is, however, a measurable standard for forecasts, and its general neglect is one of the quiet scandals of public discourse. It is called calibration.

A forecaster is well calibrated if, across all the times they say something is 70 percent likely, that thing actually happens about 70 percent of the time. Notice what this does not require: it does not require always being right. A perfectly calibrated forecaster who says “30 percent” will be “wrong” most of the time on those particular calls, and that is exactly correct. Calibration grades not the bet but the honesty of the odds.

The statistical machinery for this is well understood; bodies such as the Royal Statistical Society have long promoted scoring rules that reward accuracy and penalise false confidence. The remarkable thing is how seldom it is applied to the predictions that shape public life. Pundits are invited back regardless of their record; strategists who were spectacularly wrong are quoted as readily as those who were right, because no one keeps the ledger that would tell them apart.

Imagine a world in which every public forecaster carried a visible, audited track record — a calibration score, the way a fund discloses past returns. The effect would not be to silence prediction but to redistribute attention toward those who have actually earned it. Our business and economy coverage tries, where it can, to flag when a confident market call rests on no track record at all.

Why the important forecasts are the worst ones

There is a cruel pattern in all of this. The forecasts we care about most tend to be the ones we can make least well.

The questions that matter — the trajectory of a new technology, the tipping points of an economy, the path of a political movement — are almost always questions about complex adaptive systems, full of feedback loops and human behaviour reacting to the forecasts themselves. These are precisely the systems where small errors compound and where the act of predicting can change the outcome. The easy, reliable forecasts (the tide tables, the orbital mechanics) are easy precisely because nothing in them is watching and adjusting.

Climate science offers an instructive middle case. The Intergovernmental Panel on Climate Change does not issue a single confident number for the future; it publishes ranges, scenarios and explicit confidence levels, distinguishing what is well established from what remains uncertain. This is sometimes mistaken for weakness — “they can’t even agree on a figure.” It is the opposite: it is what honest forecasting of a complex system looks like, a distribution rather than a point, with its uncertainty stated rather than smuggled away. Our science coverage leans on exactly this kind of probabilistic framing.

How to read a prediction

If forecasts are unavoidable — and they are, because every decision is a bet on the future — then the practical question is how to consume them well. Distrust the single number offered without a range; a forecast with no stated uncertainty is not more confident, only less honest. Ask what would prove it wrong, and when — a prediction that can never be checked is not a prediction but a mood. Weight forecasters by their record rather than their fluency, since the persuasive voice and the accurate one are only occasionally the same person. And treat “I don’t know, and here is why” as a sign of expertise rather than its absence. The genuine forecaster knows where the map runs out.

What’s at stake

The deeper risk in our forecast-saturated culture is not that individual predictions fail — they always have and always will. It is that the steady diet of confident, unscored prediction trains us to expect the future to be knowable, and so to feel betrayed, or to blame the wrong people, when it refuses to be.

A society that understood the limits of prediction would not be paralysed by them. It would plan for ranges instead of points, build resilience against the scenarios it could not rule out, and reserve its trust for the forecasters humble enough to show their working — an approach we say more about on our about page. The future will keep arriving unannounced. The least we can do is stop pretending we were promised otherwise.

Opinion. The views expressed in this article are the author’s own.

Sources

• Royal Statistical Society
• National Weather Service — about forecasting
• Intergovernmental Panel on Climate Change
• Nature

Related from Opinion & Analysis