Anomaly Alerts That Do Not Spam: Static Thresholds vs Dynamic Baselines (fire on surprise, not on schedule)
Chinmay Raibagkar · has muted an alert channel and rebuilt it quieter🔔 with a severity ladder, not a lecturePublished September 6, 2026
The 60-second version
- Every false alarm trains readers to ignore the real one — tune to a few alerts a month, most of them real.
- Fire on deviation from a living baseline, evaluated on matured windows only — never judge efficiency on yesterday.
- Attach severity, owner, and decomposition — detection must be reproducible; explanation can be intelligent.
"CAC is up 40%!" — because yesterday's conversions have not arrived yet. "Revenue dropped to zero!" — because the pipeline credential expired. "ROAS crashed!" — because Diwali was in the comparison window last year. Every alert that cries wolf trains its readers to ignore the next one, and the next one is eventually real. then nobody listens
Anomaly alerts fail in predictable ways: static thresholds on moving metrics, no respect for data maturity, no memory of seasonality, no severity ladder. This post designs alerts that stay trusted — thresholds where they belong, dynamic baselines where they are needed, and the operating discipline that keeps the channel signal-rich.
Alert on the unexpected
Why Alerts Spam
Data JourneyStatic threshold
'Alert if CAC > ₹400.' Growth, seasonality and scale make every fixed line wrong within a quarter — then it fires weekly.
Immature data
Alerting on windows that have not settled — conversion lag, restatements, RTO outcomes — manufactures emergencies from incompleteness.
Dynamic baseline
'Alert if today deviates from its expected range given weekday, trend and season.' Fires on genuine surprises, silent on everything else.
📏 The static threshold
Alert if CAC crosses a fixed line. Growth and seasonality make every fixed line wrong within a quarter.
- Silent during real decay
- Noisy during normal growth
- Muted within 2 months
🌊 The dynamic baseline
Alert if today breaks its expected range. Same weekday, trend-adjusted, season-aware.
- Fires on genuine surprises
- Quiet on festive weeks
- 2–3 alerts a month, each real
A dynamic baseline is the expected value of a metric for a given moment — same weekday last 4 weeks, trend-adjusted, season-aware — plus a band around it. The alert fires on deviation from expectation, not on crossing an absolute line. Diwali week does not page anyone when the baseline knows it is Diwali week.
The target rate for a trusted channel: a few alerts a month, most of them real. More than one a week and humans develop banner blindness; fewer than one a quarter and the thresholds are decoration. Tune to the rate, not to the threshold value.
Four failure modes and fixes
1. Alerting on immature windows
Yesterday's ROAS is provisional (conversion lag), yesterday's spend may restate, COD revenue settles on delivery. Any alert evaluated on unsettled data is a random-number generator with a Slack integration.
Fix: evaluate alerts on matured windows only — trailing 7-day ending 3 days ago for lagged metrics, with the evaluation date printed in the alert. Intraday alerts are legitimate only for spend pacing (money leaving now), never for efficiency (returns arriving later).
2. Static thresholds on growing metrics
"Revenue below ₹5L" was a crisis bound in January and a normal Tuesday by July. Static lines on trending metrics guarantee either silence during real decay or noise during normal growth.
Fix: baseline as trailing median with trend. The simplest robust version:
3. No seasonality memory
Festive spikes, payday cycles, weekend B2C troughs — a baseline with a 7-day memory treats every annual event as an anomaly. Tag known events (sales, festivals, site outages, budget changes) in a calendar table and exempt-or-adjust expectations around them. An alert during a known sale is not insight; it is the system announcing it cannot read a calendar.
4. Flat severity — everything pages, nothing prioritises
A Severity Ladder That Gets Read
Reporting HierarchyFYI (digest)
Notable but explainable: 15% deviation inside a known cause. Batched into the morning narrative, never paged.
Investigate (channel alert)
Unexplained 25%+ deviation on a matured window. Someone owns the next step by end of day.
Act now (page)
Money-burning states: spend pacing 2x with zero conversions, pipeline stale over SLA, checkout conversion halved.
The alert that earned its keep
Run alerts like a product
Keeping the Channel Trusted
Process FlowStart with three alerts, not thirty
Spend pacing (money now), CAC deviation (efficiency, matured), pipeline freshness (truth of everything else). Add a fourth only when all three are quiet and trusted.
Attach the decomposition, not just the deviation
'CAC +38% (CPM flat, CVR −27%, UPI flow)' beats 'CAC anomaly detected' by the entire investigation. The alert should start the diagnosis.
Review false positives monthly
Every false alarm is a threshold or calendar bug with a fix. A channel whose false positives are never reviewed is a channel being trained to be ignored.
Let the AI narrate, not decide severity
Scheduled narratives summarise; deterministic rules set severity. An LLM choosing what pages whom is unaccountable automation with a friendly tone.
Keep the channel trusted fewer, better alerts →
- Start with three alerts — spend pacing, CAC deviation, pipeline freshness. Add a fourth only when all three are quiet.
- Evaluate matured windows only — trailing 7-day ending 3 days ago for anything lagged.
- Attach the decomposition — the alert should start the diagnosis, not just the panic.
- Review false positives monthly — every false alarm is a threshold or calendar bug with a fix.
Never alert a human about something no human can act on. "iOS modeled share rose 2%" has no owner and no action — it belongs in a quarterly methods note, not a Slack channel. Every alert must imply an owner and a first step, or it is noise by construction.
Quick gut-check
One question on Monday-morning panic. check the window first!
A CAC alert fires every single Monday. What is most likely?
Frequently Asked Questions
Should early-stage startups bother with dynamic baselines?
The discipline, yes; the infrastructure, minimally. Three scheduled checks with trailing-median logic (like the SQL above) cover the failure modes that kill young companies: overspend with no conversions, broken checkout, stale pipeline. Sophistication can wait; the severity ladder cannot.
How do scheduled AI narratives fit with deterministic alerts?
Deterministic rules detect; narratives explain. The alert fires on the deviation, and the scheduled narrative (or an on-demand AI question) decomposes it — CPM vs CVR, channel vs creative, weekday vs trend. Detection must be reproducible; explanation can be intelligent.
What about alerting on positive anomalies?
Yes — a 40% efficiency windfall deserves investigation too (broken tracking inflating conversions is a "positive" anomaly with real consequences, and genuine wins deserve scale-up). Same machinery, same severity thinking, happier mornings.
Summary & Next Steps
Trusted alerts respect data maturity, expect seasonality, fire on deviation-from-baseline rather than fixed lines, and arrive with severity, owner and decomposition attached.
- Use dynamic baselines for every efficiency metric worth monitoring.
- Use conversion lag awareness to keep immature windows out of the alert path.
- Use freshness checks as the foundational alert everything else assumes.
Takeaways for your next report
- Fire on deviation from a living baseline, not on crossing a fixed line.
- Judge matured windows only — yesterday is always “down”.
- Carry severity, owner, and decomposition in every alert.
- Start with three alerts; review false positives monthly.
Budget Pacing Calculator
Project whether a campaign will land under, on, or over its monthly budget, based on spend-to-date and days elapsed.
Chinmay Raibagkar
About author →Founder of DataLens AI. He helps non-technical teams read their ad and database numbers with confidence — which number to trust, what to do next, and what to ignore.