The useful question about a reporting system is not what it measures, but how long the organisation would take to find out it was wrong.
The program has been running for eleven months. The monthly report has shown green for nine of them, amber for two. Spend is within tolerance. Milestones are largely met. The steering committee has been attentive, well-briefed and appropriately challenging.
And then, somewhere between month eleven and month thirteen, the position changes completely. Not because something dramatic happened, but because a set of conditions that had been drifting for most of a year finally became impossible to present as anything other than what they were.
Every executive with program experience recognises this. The instinctive diagnosis is that someone was managing the message. Sometimes that is true. More often the reporting was accurate every month, and the problem is structural: the organisation was measuring things that could not move until the situation had already deteriorated.
This is the third part of a series on deciding under uncertainty. The first argued that a plan's function is detection rather than prediction. The second argued that risk is deviation in either direction. This one addresses the practical consequence of both — if the plan exists to reveal that reality has diverged from it, what should actually be watched, and how quickly can divergence be seen?
The Strategic Context
Portny's account of managing work under uncertainty ends where most treatments of the subject begin to matter. Having identified what may not go as anticipated and assessed the consequences, the response is to select what will be monitored closely, develop contingencies, establish indicators of how the work is progressing, and watch continuously so that deviation is identified as soon as it occurs — revising the plan when it does.
The phrase carrying the weight is as soon as it occurs. It implies a property that most enterprise reporting does not possess and is rarely designed for: detection latency, the interval between a situation changing and the organisation becoming aware that it has.
Latency is not the same as reporting frequency. A program reported weekly can have a four-month detection latency if everything on the report is a lagging measure. Conversely, a program reviewed quarterly can have a two-week latency if the right things are instrumented and escalation is unblocked between reviews. Frequency is how often you look. Latency is how old the information is when you look at it.
Most enterprises manage the first and never measure the second.
What Leaders Commonly Misread
That more measurement reduces latency. It usually does not. Adding measures to a report increases the volume of information without changing the age of it. A dashboard with sixty indicators, all of which are lagging, has exactly the same detection latency as one with six — and worse signal-to-noise, because genuine movement is harder to see against a busier background.
That confidence ratings are information. A traffic-light status is a judgement about a situation, produced by a person who has incentives. It is a useful summary and a poor detector. When the underlying data has not moved and the rating changes, what has changed is someone's willingness to say so. That is worth knowing, but it is not early warning.
That the absence of bad news is evidence of good position. Under genuine uncertainty, a stable report is at least as likely to indicate a measurement system insensitive to the things that matter as it is to indicate a stable program. The silence has to be interpreted, not accepted.
That measuring the plan measures the outcome. Schedule adherence, spend against budget and milestone completion measure conformance to a set of assumptions. Where those assumptions are sound, conformance is a reasonable proxy for progress. Where they are not, a program can conform perfectly to a plan that stopped being relevant in month four, and the reporting will show green throughout.
Reframing the Issue
The reframe: a measurement system is an engineered detection instrument, and its most important specification is response time.
Engineers think about instruments this way as a matter of course. A sensor is specified not only by what it detects but by how fast it responds, how much noise it tolerates, and what it is blind to. No competent engineer would install a temperature sensor with a four-hour lag in a process that can fail in twenty minutes, then conclude from a stable reading that the process was under control.
Enterprise reporting is rarely specified this way. It is assembled from what is available, what governance requires, and what previous programs reported — none of which are design criteria. The question how long would this system take to tell us we are wrong? is almost never asked at the point the reporting pack is designed, which is the only point at which the answer can be changed cheaply.
Once asked, it reorders priorities immediately. A small number of measures that can move early is worth more than a comprehensive set that cannot.
Designing for Early Movement
A hypothetical health service consolidating clinical systems across several hospitals offers a clear illustration. The measures that will appear on the program report are predictable: implementation milestones, spend, sites migrated, training sessions delivered, defects raised and closed.
Every one of them is a measure of activity, and every one can look healthy while the thing that determines whether the program creates value — whether clinicians actually change how they work — moves in the wrong direction. That divergence will eventually appear, but it will appear in adoption data, in workaround prevalence, in overtime, in complaint volumes and in staff turnover, months after it became true and long after the cheapest window to respond has closed.
The measures with genuinely low latency here are different in kind and mostly unglamorous. How long, in the first weeks after a site goes live, does a common clinical task take compared with before? What proportion of users are using the intended pathway rather than a parallel one? How quickly are locally-invented workarounds appearing, and are they spreading between wards?
These share three properties worth generalising:
They can move before the money does. Anything that can only change after cost is incurred is a lagging measure by construction.
They measure behaviour rather than delivery. Delivery measures tell you whether the organisation did what it planned. Behaviour measures tell you whether it is working — which is the only thing that determines whether the assumptions still hold.
They are cheap enough to collect continuously. An indicator that requires a special exercise will be collected when someone is worried, which is precisely when its early-warning value has already been spent.
Decision Framework
For each material initiative, test the detection instrument rather than the report.
| Test | Question | Adequate answer |
|---|---|---|
| Latency | If our central assumption became false today, how long until we knew? | Weeks, and stated as a number |
| Precedence | Can any measure move before spend does? | At least two can |
| Behaviour | Do we measure what people do, or only what we delivered? | Both, and separately |
| Falsifiability | What observation would show the approach is wrong? | Specified in advance |
| Ownership | Who is obliged to raise a deviation, and to whom? | Named, with a route that bypasses the monthly cycle |
| Cost of looking | Does obtaining the measure require an exercise? | No — it is continuously available |
Where an initiative cannot produce a latency figure at all, that is the finding. It means no one has established how the organisation would learn it was wrong, and the reporting is documenting activity rather than detecting divergence.
From Strategy to Execution
Immediate. For the two or three most consequential programs in flight, ask the sponsor a single question: what would we see first, and how long after? Do not accept a list of measures in response. The answer required is a sequence and an interval.
Medium term. Separate the detection pack from the assurance pack. They serve different readers and different purposes, and fusing them corrupts both — the assurance pack drives out early indicators because they look alarming, while the detection pack becomes bloated with material that exists to demonstrate diligence. Two documents, honestly labelled, work better than one that tries to do both.
Long term. Build the discipline of revising plans on evidence without treating revision as failure. A detection system is only worth its cost if the organisation acts on what it detects, and an enterprise that punishes re-planning will keep receiving reassuring reports regardless of what its instruments say. The instrument and the response culture are one capability, not two. [Related article: Uncertainty Is the Case for Planning, Not the Excuse Against It]
Signals to Monitor
- Reports that change status abruptly. A jump from green to red without an amber period is evidence of latency, not of sudden events.
- Indicators that have never moved. A measure that has been stable for a year is either measuring something genuinely stable or measuring nothing.
- Escalation that only travels through the monthly cycle. If nothing reaches leadership between reports, the reporting rhythm is also the detection ceiling.
- Growing distance between delivery measures and behaviour measures. Milestones met while adoption falls is the classic signature of a plan that has outlived its assumptions.
- Requests for special exercises to establish position. If understanding the current state requires a review, the instrument is not working.
Questions for the Leadership Team
- For our largest program, what is the detection latency in weeks — and does anyone here know?
- Which of the measures on our current report could move before money is spent?
- When did we last change a plan because an indicator moved, rather than because a deadline was missed?
- Do we measure whether people are behaving differently, or only whether we delivered what we said we would?
- What would have to be observed for us to conclude that our current approach is wrong, and is that written down anywhere?
Closing Perspective
Programs rarely fail suddenly. They fail slowly, and are discovered suddenly — and the gap between those two events is a property the organisation designed, whether or not it realised it was designing anything.
Reducing that gap is not a reporting improvement. It is the difference between governing an initiative and being informed about one after the decisions that mattered have already been made by default.
Related article: Uncertainty Is the Case for Planning, Not the Excuse Against It
Related article: Risk Is Not the Chance That Things Go Wrong
Related article: What Must Be True: The Assumptions Register as a Strategy Instrument
About the author
Kevin Jogin is Founder & Principal Advisor at EraNorth. Meet the Founder.