Anomaly Alerts That Do Not Spam: Static Thresholds vs Dynamic Baselines

"CAC is up 40%!" — because yesterday's conversions have not arrived yet. "Revenue dropped to zero!" — because the pipeline credential expired. "ROAS crashed!" — because Diwali was in the comparison window last year. Every alert that cries wolf trains its readers to ignore the next one, and the next one is eventually real.

Anomaly alerts fail in predictable ways: static thresholds on moving metrics, no respect for data maturity, no memory of seasonality, no severity ladder. This post designs alerts that stay trusted — thresholds where they belong, dynamic baselines where they are needed, and the operating discipline that keeps the channel signal-rich.

The short version: alert on the unexpected, not on the clock

A dynamic baseline is the expected value of a metric for a given moment — same weekday last 4 weeks, trend-adjusted, season-aware — plus a band around it. The alert fires on deviation from expectation, not on crossing an absolute line. Diwali week does not page anyone when the baseline knows it is Diwali week.

The four failure modes, each with its fix

1. Alerting on immature windows

Yesterday's ROAS is provisional (conversion lag), yesterday's spend may restate, COD revenue settles on delivery. Any alert evaluated on unsettled data is a random-number generator with a Slack integration.

Fix: evaluate alerts on matured windows only — trailing 7-day ending 3 days ago for lagged metrics, with the evaluation date printed in the alert. Intraday alerts are legitimate only for spend pacing (money leaving now), never for efficiency (returns arriving later).

2. Static thresholds on growing metrics

"Revenue below ₹5L" was a crisis bound in January and a normal Tuesday by July. Static lines on trending metrics guarantee either silence during real decay or noise during normal growth.

Fix: baseline as trailing median with trend. The simplest robust version:

3. No seasonality memory

Festive spikes, payday cycles, weekend B2C troughs — a baseline with a 7-day memory treats every annual event as an anomaly. Tag known events (sales, festivals, site outages, budget changes) in a calendar table and exempt-or-adjust expectations around them. An alert during a known sale is not insight; it is the system announcing it cannot read a calendar.

4. Flat severity — everything pages, nothing prioritises

Operating discipline: alerts are a product, not a config

Frequently Asked Questions

Should early-stage startups bother with dynamic baselines?

The discipline, yes; the infrastructure, minimally. Three scheduled checks with trailing-median logic (like the SQL above) cover the failure modes that kill young companies: overspend with no conversions, broken checkout, stale pipeline. Sophistication can wait; the severity ladder cannot.

How do scheduled AI narratives fit with deterministic alerts?

Deterministic rules detect; narratives explain. The alert fires on the deviation, and the scheduled narrative (or an on-demand AI question) decomposes it — CPM vs CVR, channel vs creative, weekday vs trend. Detection must be reproducible; explanation can be intelligent.

What about alerting on positive anomalies?

Yes — a 40% efficiency windfall deserves investigation too (broken tracking inflating conversions is a "positive" anomaly with real consequences, and genuine wins deserve scale-up). Same machinery, same severity thinking, happier mornings.

Summary & Next Steps

Trusted alerts respect data maturity, expect seasonality, fire on deviation-from-baseline rather than fixed lines, and arrive with severity, owner and decomposition attached.

• Use dynamic baselines for every efficiency metric worth monitoring.
• Use conversion lag awareness to keep immature windows out of the alert path.
• Use freshness checks as the foundational alert everything else assumes.