Skip to content

MODULE 7 · LESSON 40 · AI-assisted analysis

Detecting invented findings

Identify claims that require fields, sources or study designs absent from the supplied evidence.

By Laurence Svekis · Self-paced · All worked data is synthetic unless you supply your own source.

Try the practice ↓

PredictInspectCalculateCheckExplainTransfer

Your starting point

Prerequisites: Use the ideas and vocabulary from lesson 39; revisit its worked examples if needed.

Definition: An invented or unsupported finding is a claim presented as established even though the supplied evidence does not substantiate it. It may involve fabricated numbers, nonexistent sources, unmeasured variables, causal language or explanations that were never tested.

Map every claim to evidence

Break a generated paragraph into individual statements. Mark each as directly observed, calculated, inferred or unsupported. A statement about customer satisfaction cannot be verified from a table containing only visits and dates. A causal statement requires more than a time sequence. This claim-by-claim audit prevents a correct number from lending false credibility to an unrelated assertion.

Check citations and specificity

A precise percentage or formal-sounding source title can make a claim feel trustworthy. Verify that the source exists, contains the relevant evidence and matches the population and period. A link to a general topic page is not necessarily support for a specific effect size. When you cannot verify a source, remove or qualify the claim rather than repeating it with stronger wording.

Distinguish a hypothesis from a finding

An assistant can suggest plausible explanations worth investigating. Label them as hypotheses and state what evidence would test them. Do not discard useful ideas merely because they are unproven, but do not promote them into observed results. A careful report can say what was found, what might explain it and what remains unknown in separate sentences.

Two worked examples

An absent variable

The supplied table contains week and visits. AI writes that satisfaction improved by 18% because a redesign made navigation easier. Neither satisfaction nor redesign exposure is recorded. Even if visits increased, the satisfaction number and causal explanation are unsupported by the supplied evidence.

A valid calculation beside an invalid claim

A report correctly calculates 20 conversions from 100 sessions as 20%, then concludes that the channel is preferred by younger customers. Age and preference are absent. The correct rate does not validate the demographic statement; audit each claim independently.

Where this becomes useful

A manager receives a polished executive report generated from a small export. A claim-evidence table can expose missing variables before the report influences a decision or circulates as an authoritative source.

Make an attempt

Use the verification lab’s case with only week and visits. Write a claim-evidence list before revealing the notes. Identify the missing fields and propose a non-sensitive way to collect relevant evidence without inventing the answer.

Open the accompanying interactive lab →

Misconception check

Claim: Specific numbers are more trustworthy than cautious statements.

Inspect the correction

Precision can be fabricated. Trace each number and claim to a verifiable source and calculation.

Independent application

Create a short synthetic AI-style report containing one correct calculation, one plausible hypothesis and one unsupported claim. Label each statement, supply evidence for the calculation and rewrite the report so uncertainty and missing information are explicit.

Completion records practice and self-review. It is not certification of statistical competence.