Why your AI report always has a reason (even when there isn’t one)
Chinmay Raibagkar · has chased an AI-invented cause for a whole morning📉 with a news-or-noise checker, not a lecturePublished October 4, 2026
The 60-second version
- AI narrators complete the pattern “metric moved → because”, inventing plausible causes for moves that are just normal variation.
- Only moves beyond ~2 normal swings, on matured data, deserve a reason — immature days create fake dips every week.
- The fix: feed the narrator normal ranges, maturity flags and the change log, require evidence for every cause, and allow “no meaningful change”.
Monday, 9 AM, the AI-written weekly report lands in Slack: "ROAS dipped 6% this week, driven by weaker creative performance on Meta and rising competition in Google Search."
It reads like an analyst wrote it. The team spends the morning on it — pulling creative reports, checking auction insights, drafting a refresh brief. By lunch they've found nothing. No creative got weaker. No competitor showed up. ROAS was 3.10x against a usual 3.30x, which is the kind of wobble this account does most weeks. Two of the seven days hadn't even finished counting conversions.
The AI didn't lie about the number. It invented the reason. a story for every wiggle Language models are built to give helpful, complete answers — and a helpful-sounding answer to "what happened?" always has a cause. Unless you teach the narrator what normal looks like, it will explain noise with total confidence.
A 6% dip that needed no explanation
The narrator's job is to sound helpful
An AI writing a report isn't measuring anything. It's completing a pattern: metric moved → here's why. Three habits make that pattern dangerous.
It always answers
Models are trained to be helpful, and 'I don't know why' rarely looks helpful. Faced with a dip, they reach for the most plausible-sounding cause.
It knows the usual suspects
Creative fatigue, competition, seasonality — the model has read thousands of reports blaming these. Plausible is not the same as true.
It can't see noise
Without a normal range in front of it, a 6% move looks like news. It has no way to know this account swings 5% most weeks.
The fix isn't a smarter model. It's giving the narrator the three things a good analyst checks before writing a single word: is the move bigger than normal, is the data finished, and did anything actually change?
Is the move even real?
A move only deserves an explanation if it's larger than the account's usual week-to-week swing — measured on data that has finished arriving. Try it with your own numbers:
From your conversion-lag curve for the last few days
Inside the usual range once the data matures. The honest report line is 'no meaningful change' — no causes needed.
Immature data creates fake dips. The most recent days are always missing conversions that haven't arrived yet, so the latest week almost always looks worse than it will. A narrator that reads raw numbers will "explain" that dip every single week. Account for conversion lag before anything gets a cause.
Ground the narrator
Here is the same week, written twice — once by a narrator given only the numbers, once by a narrator given the numbers plus a normal range, maturity flags and the change log.
🎭 The confident storyteller
"ROAS dipped 6% this week, driven by weaker creative performance on Meta and rising competition in Google Search."
- Every move gets a cause
- Causes come from general knowledge, not your data
- Sends the team chasing ghosts
🧾 The careful analyst
"ROAS is 3.10x, within its normal range (3.30x ± 0.15x) once the last two days mature. No changes in the account log. Nothing needs action this week."
- States whether the move is unusual
- Flags unfinished days
- Names a cause only with evidence
The second version is less exciting and far more useful. A report that says "nothing happened" on quiet weeks is the reason people believe it on loud ones.
What to feed the narrator
Reporting HierarchyA normal range for every metric
The trailing average and typical swing, so the model can tell news from wobble. A dynamic baseline that respects weekdays and seasons is even better.
Maturity flags
Which days are still collecting conversions, and roughly how much is missing. The narrator must not explain unfinished data.
The change log
Budget edits, launches, pauses, site deploys. Causes should come from here, not from the model's general knowledge.
An evidence rule
Every cause named in the report must cite a breakdown, a query or a log entry. No citation, no cause.
Permission to say nothing
Tell the narrator explicitly that 'no meaningful change' is a correct and welcome answer.
This query prepares the first two inputs: each week's ROAS, its 8-week normal range, how unusual the move is, and whether the week has finished maturing. Feed its output to the narrator instead of raw totals.
The narrator_instruction column does the heavy lifting: the model no longer decides whether something is news. The data tells it. For the thresholds themselves, see anomaly alerts that don't spam.
Narrator rules for your AI reports paste these into the report prompt →
- Explain only what's unusual — moves under two normal swings get 'within normal range', nothing more.
- Never explain unfinished days — flag them as still maturing and leave them out of any cause.
- Cite or stay quiet — every cause must point to a breakdown, a query or a change-log entry; otherwise write 'cause unclear'.
Quick gut-check
One question. If you get it, the whole post clicks. no stats degree needed
Your AI report says 'CPA rose 8% due to audience saturation.' The account's CPA usually swings ±10% week to week, and nothing changed in the account. What's the right reading?
Frequently asked questions
Why do AI-generated reports give reasons for every change?
Language models are trained to give complete, helpful-sounding answers. When a metric moves, the most helpful-looking answer includes a cause, so the model supplies a plausible one from general knowledge — even when the move is just normal variation.
How do I stop AI reports from explaining noise?
Give the AI each metric's normal range, flag days that are still collecting conversions, provide the change log, and require evidence for any cause it names. Tell it explicitly that "no meaningful change" is an acceptable answer.
What counts as a meaningful change in a weekly metric?
A useful rule of thumb is a move larger than two typical weekly swings (two standard deviations of the last eight weeks), measured on matured data. Smaller moves are usually noise; very large ones deserve a cause backed by evidence.
The summary
- AI narrators complete the pattern "metric moved → because", even when the move is noise.
- Without a normal range, a 6% wobble looks like news to a model.
- Recent days are always missing conversions, so unmatured data creates fake dips every week.
- Feed the narrator a normal range, maturity flags, the change log and an evidence rule.
- Let the report say "nothing meaningful changed" — it's what makes the loud weeks believable.
Takeaways for your next report
- An AI report will invent a cause for every move unless it knows what normal looks like.
- Only moves larger than about two normal swings, on matured data, deserve an explanation.
- Causes must come from your change log or a breakdown — never from the model's general knowledge.
- Flag unfinished days so the narrator never explains a conversion-lag dip.
- 'No meaningful change' is a correct answer; make sure your narrator is allowed to give it.
ROAS Drop Doctor
ROAS fell and nobody agrees why. Answer 5 questions about what moved — CPM, CTR, CVR, AOV, frequency — and get a ranked diagnosis with a fix checklist.
Chinmay Raibagkar
About author →Founder of DataLens AI. He helps non-technical teams read their ad and database numbers with confidence — which number to trust, what to do next, and what to ignore.