Skip to content
writing

A Number That Never Moved Isn't Stable. It's Unwatched.

A forecast that hasn't changed in years isn't stable. Nobody is checking it against what actually happened.

GovernanceEvaluationHypothesisLast tended · 2026-07-30

Confidence: hypothesis. The updating failures below draw on published forecasting research. The four-year flat number is a stylized composite used as a test case, not a firsthand report. The pattern at the end is an argument, not a shipped system. Argue with it.

There is a kind of number that looks reassuring and should worry you. It is the estimate that has not changed in years. A program's odds of success, a project's risk rating, a system's health score, sitting at the same value review after review. People read that stability as confidence. Often it is the opposite. The number did not hold still because the world held still. It held still because nobody was watching it.

The clinical why

Consider a stylized composite, useful because it is extreme and still recognizable. A program's probability of success sits at 65% for four straight years. Over those same four years, the program misses a secondary endpoint, loses a development partner, and watches a competitor win approval in the same disease. Every one of those events should have moved the number. None of them does.

The number was not tracking the evidence. It was tracking something else: the funding threshold, the level below which the program would face a kill review. It stayed at 65% because 65% was safe. Everyone in the room half knew this, and nobody wrote it down. Meanwhile the portfolio math, the expected value, the resource plans, all of it was built on that number, and inherited its fiction with full precision.

A forecast that cannot tell you what evidence it survived is not current. It is just unrevised. Those are not the same thing, and the difference is the whole problem.


This pattern is one piece of a longer treatment. The full essay is issue 4 of Stage × AI, a series walking the entire clinical-trial lifecycle stage by stage — what each stage really does, where AI helps, where it must not go, and one buildable pattern per stage:

full essay
Probability of Success: The Number That Moves the Money

Evidence in, evidence out. Corrections welcome.