How to Audit an AI-Generated Query in 60 Seconds

You do not need to be fluent in SQL to catch a wrong AI answer. You need sixty seconds and three places to look: the tables it chose, the filters it applied, and the grain it aggregated at. Almost every confidently wrong number fails at least one of those three checks.

This is the practical companion to the seven checks — the same scepticism, compressed into a routine you can run on any generated query before its number leaves your screen.

1. Check the FROM and JOIN first, not the SELECT

Beginners read a query top to bottom, starting with SELECT. Auditors read it inside out, starting with FROM. The reasoning is simple: the SELECT clause is cosmetics — which columns are displayed and what they are named. The FROM and JOIN clauses decide what universe of rows the answer is computed over, and a wrong universe cannot be rescued by a correct-looking select list.

Three questions, in order:

Do I recognise these tables? A query answering "revenue last month" from orders_v1 when your team migrated to orders_v2 last year is stale before it runs. A query reaching past your curated semantic views into raw staging tables is improvising. If a table name surprises you, stop — that surprise is the audit working.

Are the join keys unique on at least one side? This is the single most expensive failure in AI-generated SQL. Joining orders to campaign spend on date multiplies every order by the number of campaigns that ran that day:

You do not need to parse the whole query to check this. Find each JOIN, look at the ON condition, and ask: could one row on the left match several rows on the right? If yes — dates, channel names, regions — the totals are multiplied and the number is fiction.

Does the join path match how your business actually connects these things? Orders connect to campaigns through attributed line items, not through calendar dates. Spend connects to dates directly. A query joining the wrong two things on a plausible-sounding key is the model's most characteristic error, because the plausible key is what its training data suggests.

2. Check every WHERE filter against your actual question

If FROM decides the universe, WHERE decides which slice of it you measured. Read each filter as an English sentence and compare it to what you asked.

The date column. Warehouses hold several timestamps per order — created, paid, shipped, delivered — and they answer different questions. "Revenue last month" measured on shipped_at during a fulfilment backlog is really "revenue from two months ago, shipped late." Confirm the filter uses the timestamp your question meant, and that the range covers what you think: trailing 30 days is not the same as calendar last month.

The status filters. Does the query exclude cancelled and refunded orders? The classic silent failure:

-- ⚠️ Looks careful, drops every row whose status is NULL:
WHERE order_status != 'cancelled';

In SQL's three-valued logic, NULL != 'cancelled' is unknown, and unknown rows are filtered out. Newly created orders with no status yet vanish from your report. The safe form names what to include (status IN ('paid', 'partially_refunded')) rather than what to exclude.

Timezone and edge cases. If your database stores UTC and your business reports in local time, a day boundary drawn in the wrong zone misattributes the highest-traffic hours of every day. And glance at division: SUM(revenue) / SUM(spend) crashes on zero-spend days unless guarded by NULLIF or SAFE_DIVIDE — a query without the guard was never reviewed for reality.

3. Check the GROUP BY grain against what you expect

The grain — the entity each row of the result represents — is where "revenue by campaign" quietly becomes something else. GROUP BY campaign_id, order_date returns one row per campaign per day; summing that column mentally as "revenue by campaign" double-counts nothing but misleads anyone who treats daily rows as campaign totals without a second aggregation.

Two checks:

Does the GROUP BY match the noun in your question? "Customers" means grouping by customer_id or customer_key. A query grouping by order_id while calling the result "customers" counts repeat buyers once per order — and inflates every downstream ratio built on it, from CAC to repeat rate.

Are the DISTINCTs distincting the right thing? COUNT(DISTINCT order_id) and COUNT(DISTINCT customer_id) differ by your repeat-purchase rate, which for a healthy store is a large multiple. When the audited query counts something, verify the thing inside the COUNT DISTINCT is the entity you asked about, not its neighbour.

Grain errors share a tell: the number is plausible, round-ish, and wrong by a factor that looks like a business multiple — 1.3x, 2x, 5x. Exact multiples smell like fan-out joins; plausible-but-off numbers smell like grain.

The 60-second checklist, summarized

If all four pass, trust the number provisionally — the way you would trust a junior analyst's checked work. If any step raises a question you cannot resolve in the minute, escalate rather than ship: paste the SQL to your analyst with the step that failed, or ask the tool to regenerate with the specific correction ("use net_total, exclude full refunds, group by customer_key"). A precise correction request produces a far better second attempt than "try again."

Frequently asked questions

What if I can't read SQL at all?

Read the nouns and the verbs and skip the rest. Table names after FROM and JOIN, column names inside WHERE, and the list after GROUP BY carry 90% of the audit value, and all of them are English words you already know. Ask the tool to explain the query in one sentence per clause — then audit the sentences.

What if the query references tables I've never heard of?

Treat that as a finding, not a gap in your knowledge. A generated query should draw on tables your team recognises as sources of truth. An unfamiliar table is either deprecated, raw, or staging — in all three cases the correct response is to ask why it was chosen before trusting anything built on it.

Should I also check the numbers, not just the query?

Yes — the five-second smell test in step 4. Compare against last month, against the dashboard, against your felt sense of the business. Humans carry a prior over plausible numbers that no validator has. A 5x move that the query cannot explain is a data problem wearing a query costume.

Does this catch every failure mode?

No. It catches translation failures — the query answering a different question than asked. It does not catch stale pipelines, incomplete source coverage, or definitions your whole team disagrees about. Those need freshness monitoring, source audits, and a semantic layer, respectively.

Where does cost fit into the audit?

Step 4's byte estimate is a correctness signal disguised as a cost control. Routine questions scan gigabytes; a terabyte estimate means a missing partition filter or an accidental cross join — something semantically wrong that also happens to be expensive. Estimate before executing, every time.

The summary

• Audit inside out: FROM and JOINs first, WHERE second, GROUP BY third. The SELECT is cosmetics.
• On every JOIN, ask whether one left row can match several right rows. Dates and names multiply; keys should be unique on at least one side.
• Translate each WHERE filter to English and compare the sentence to your question — date column, status inclusion, NULLs, timezone.
• Confirm the GROUP BY equals the noun in your question, and the COUNT DISTINCT targets that same entity.
• Finish with the byte estimate and the smell test. Escalate with the specific failed step, not a generic retry.