Nine Alerts, One Incident

Nine alarms, one minute

On 1 September 2026, between 10:10:31 and 10:10:32 UTC, our own Observer sent nine alarms from a single property. One per metric:

• sessions
• newUsers
• screenPageViews
• itemViews
• addToCarts
• checkouts
• ecommercePurchases
• totalRevenue
• eventCount

These were our emails, from our tool. That is the reason this piece exists.

That is one incident, not nine

Nothing in any of the nine messages says so. Each one arrived as a complete, self-contained statement about a single metric crossing a single threshold, with no indication that eight others had done the same thing in the same second.

The diagnostic rule, up front

One metric moving alone usually means the metric broke. Every metric moving together means collection broke upstream of all of them.

Different diagnosis, different owner, different urgency. We argued the diagnosis side of this at length in seven alerts, one problem, and that piece is the better read if what you want is how to triage a correlated morning.

This piece is about something narrower and more embarrassing: the alert could have told you which case you were in, and did not.

Why the alert cannot know

Each monitor is an independent predicate. It has a metric, a threshold and a schedule. It evaluates, it decides, it sends.

It has no idea the other eight exist, because nothing in the design gives it a reason to. That is correct engineering. Independent evaluation is simpler, more testable and less likely to produce weird coupled failures of its own.

It also produces an output shape that is wrong for the reader. The system knew nine monitors had breached on one property in one second. That knowledge existed, at send time, in the sending process, and it was thrown away before the message went out.

The cost is paid every morning

Triage cost is identical whether you are looking at one real problem or nine. You open the first message, form a hypothesis, open the second, notice it is the same property, open the third, and somewhere around the fifth you work out that this is one thing.

You have just reconstructed by hand, over several minutes, a fact the system had for free before it sent anything.

Not just monitoring

Any system that evaluates conditions independently reports a correlated failure as N independent failures.

• A log stream showing forty errors that are one retry loop.
• A test suite reporting thirty failures from one broken fixture.
• Uptime checks on twelve endpoints behind one load balancer.

In each case the shape of the wall is the answer, and the shape is only visible to someone who looks at all of it at once. Nothing in the tooling is arranged to make that the default view.

The fix, and its smallness is the point

One line in the message: "8 other monitors on this property breached in the same window."

That is it. No new monitors, no correlation engine, no incident model. A count computed at send time from information the sender already has, printed in the body.

To be clear about our own position: we have not shipped this. It is a proposal sitting in our own backlog, written up here because the nine emails were ours and the argument is more useful than the embarrassment is costly.

Two checks this week

1. Group last week's alerts by property rather than by time, and count how many incidents were really one. Most teams have never looked at their alert stream this way and the ratio is usually worse than expected.
2. Ask whether anything in your stack can say "and eight others". Not whether it could be built. Whether any message you currently receive carries that context.

Who owns the message

Alert coverage has an owner almost everywhere. Alert design usually does not, which is why messages accumulate thresholds and never accumulate context. The related failure of a monitor that nobody watches is in nobody monitors the monitor.