To ground truth data with reporting, you trace every number back to the document that produced it, redo the calculation yourself, and confirm the result against a second independent source or your own on-the-ground observation. Most of the work is documentary rather than technical, and a small representative sample is enough to get through an editor’s review.
The phrase comes out of remote sensing and machine learning, where a labelled reference set is held apart from the data it tests. In a newsroom the idea is narrower and more useful: a number you can defend line by line when an editor asks where it came from, a lawyer asks what you relied on, or a reader emails with a different figure.
On a story with a handful of numbers, this takes an afternoon. On a story built from a scraped or third-party dataset, it is the difference between a finding and a rumour. Below is the workflow I would hand a reporter starting from scratch, including the parts most guides skip: tracing a number end to end, grounding data you can no longer observe, and writing down what you could not check.
What You Need
Five things, and you probably already have most of them. What you do not have yet is usually the fifth one.
A claim written precisely enough to fail. “Unemployment fell” is not testable. “Unemployment in one county fell from 6.8 percent to 5.9 percent between March and September” is, because it names a place, a period and a measure.
The primary source document. Not the summary, not the news coverage of the release, not the chart someone posted. Ask for the table, the appendix and the underlying file, because summaries drop footnotes and change denominators without saying so.
A second source that did not produce the first. Two outlets citing the same agency release is one source with two bylines. Independence is what makes a cross-check meaningful, and independence is a test you have to apply deliberately rather than assume.
A method chosen before you look. Spot-check, cell-level audit or full recomputation. Deciding which one you used after seeing the result is how people quietly talk themselves into a pass they did not earn.
A log. One row per check: the claim, the source, the method, the result, the date and who ran it. A spreadsheet is fine. So is a plain document, as long as somebody else could follow it six months later.
Step-by-Step
Seven steps, in order. The order matters because each one narrows what the next one has to check.
Step 1: Define the Claim Being Tested
Take the sentence in your story that carries the most weight and rewrite it as a standalone statement. Then add the three things it silently assumes: which geography, which time window, and which definition of the category.
This is where most verification projects quietly go wrong. An agency counted “evictions” as completed judge-ordered removals in one court district, while your story implies a citywide count including sheriff sales. Both numbers are defensible. They are not the same number.
Split the claim too. The verifiable part might be “filings rose 18 percent year on year”. The interpretive part is “which suggests landlords are displacing tenants faster”. Verify the first with reporting. The second you defend with argument, not data.
Step 2: Build a Source Checklist
Work down the reliability ladder. Official records and the underlying data file sit at the top. Then direct observation you make yourself. Then credible third-party datasets whose provenance you can trace. Then interviews with the people who produced the number. Then independent reporting from another outlet.
When the official source publishes aggregates but not the underlying records, the records request is your ground truth acquisition method. Ask for the raw file, the code or worksheet that produced the published figure, and the revision history. That third item is the one people forget, and it is the one that explains why two years of the same series do not match.
Records requests are slow, and agencies answer in their own vocabulary. Before you file, read the methodology page until you can repeat their definitions back to them. A request written in the wrong terminology costs you weeks.
Step 3: Sample the Data Systematically
Do not check the rows that are easiest to reach. Convenience sampling is the most common methodological failure in this kind of work because the results look reassuring and cannot be defended by anyone else.
Use three kinds of check. Random checks test the typical case. Stratified checks spread the sample across regions, time periods or categories so that no single group dominates. Targeted checks go after the outliers, the largest values and the zeros, because that is where systematic errors hide.
A workable rule of thumb: for a chart with twenty bars, verify five spread across the range including the largest and the smallest. For a dataset of several thousand rows, a stratified sample in the range of sixty to a hundred rows usually surfaces every systematic problem that matters.
Historical and archival data cannot be re-observed, and this is a real gap rather than a shortcut. For a past period, build the reference from contemporaneous sources: archived releases, reporting filed at the time, snapshots you stored, and secondary datasets built from the same underlying records. Label it in your notes as documentary reconstruction, and keep the confidence label honest.
Step 4: Compare Data with Reporting Evidence
Now put the two side by side, record by record. For each sampled row, note what the dataset says, what the source document or on-the-ground evidence says, and whether they agree.
Keep the mismatches visible instead of discarding them. A mismatch is information about the dataset, not a nuisance to be cleaned up before the editor sees it. Group them as you go: does this one row fail because of a date, a boundary, a unit, or because the underlying record is wrong?
One honest discrepancy is worth more than a story that never encounters one. It tells you where the dataset’s edges are, and readers care about that more than they care about your percentage agreement rate.
Step 5: Investigate and Resolve Discrepancies
Work through the causes in this order, because the early ones explain most cases:
Definition. Do the two sources count the same thing under the same label?
Date window. Is the period inclusive, and is it the same period?
Geographic boundary. Are city limits, districts and service areas drawn the same way on both sides?
Units and rounding. Are you comparing thousands to units, or a rounded figure to an exact one?
Revision. Has the source restated history since the release you downloaded?
Transcription. Does the number survive re-keying it from the source by hand?
What is left after those six is a genuine conflict of accounts. You do not resolve that by picking a winner. You report it, attribute both sides, and tell the reader what you could not establish.
Step 6: Document Confidence and Limitations
Classify each claim with a plain label: confirmed, partly confirmed, unresolved or contradicted. These four labels are more useful to an editor than any accuracy percentage, because they say what you can publish.
If you do want numbers, report them with their sampling design attached. Overall agreement across your sample is fine. Per-category agreement tells you the dataset is weak for one group and strong for another, which is the part a reader needs. Margin of error applies when the reference is a sample rather than a complete count.
The same habit applies when you are validating something a model or a sensor produced. Overall accuracy, per-class accuracy, root mean squared error and kappa all come from one rule: accuracy reported without its sampling design cannot be interpreted.
| Confidence label | What supports it | What you can write |
|---|---|---|
| Confirmed | Two independent sources, or one source document reproduced by hand | State it as fact, with attribution |
| Partly confirmed | Source document reproduced but the sample covered one category or period only | State it with the scope of your check attached |
| Unresolved | Sources disagree and the cause could not be identified | Report the conflict and attribute both sides |
| Contradicted | Primary source contradicts the dataset on the point checked | Do not publish the dataset figure; publish the correction if relevant |
Step 7: Publish the Verification Trail
The verification is only half finished until a reader can see it. Publish a short methodology note with the story and a data appendix where it makes sense.
The note should state where the data came from, the sample size and how it was chosen, which figures you reproduced, which you did not, and what you could not resolve. Link to a versioned snapshot of the files you downloaded rather than a live page that will change under you.
Say out loud what you did not verify. Readers trust a newsroom that names its gaps more than one that implies total certainty, and the gaps are often the most useful part of the story for other reporters.
Then set the correction policy before you need it. When the source revises its data and your story changes, publish a dated correction, keep the snapshot of what you originally published, and explain what moved. Numbers change between releases for reasons that are usually innocent and always worth a sentence.
Common Mistakes
These are the errors that actually get stories corrected, and each one has a cheap fix.
Treating confirmation as proof. If one official figure and your own reading of the same release agree, you have checked nothing. Agreement between a number and its own summary is not independent verification.
Checking only easy examples. Verify the awkward rows, the boundary cases and the ones nobody would ever pick. Errors concentrate where the data is messy, and the messy rows are rarely the ones you check first.
Trusting copied figures. A number in a chart, a tweet or a competitor’s story has lost its provenance. Trace it to whoever produced it before you build on it, or treat it as a lead rather than evidence.
Ignoring definitions. Two official numbers can both be right and still conflict, because the programmes define the same word differently. Check the definitions before you check the arithmetic, or you will spend a day reconciling two correct figures that never described the same thing.
Sampling what is convenient. If you cannot explain why you picked the rows you picked, the result is not defensible to an editor or a critic. Write the selection rule down first.
Publishing unresolved discrepancies silently. A mismatch you quietly dropped becomes an error someone else finds. Report it, attribute it and let the reader see it.
Two smaller habits help more than they sound like they will. Have a second reporter reproduce your arithmetic from your notes without talking to you, since anything you explained verbally was something you should have written down. And keep the raw files timestamped the day you downloaded them.
Frequently Asked Questions
What is ground truth in data?
Ground truth is independently verified information about what is actually true at a known place and time, used as the reference against which a dataset, measurement or claim is checked. In reporting practice it means tracing a number to the document that produced it, redoing the calculation, and confirming the result against a second source or direct observation. The dataset is the claim; ground truth is the check.
What is meant by ground truthing?
Ground truthing is the act of comparing a dataset or model output against an independent reference measure and recording where the two disagree. The reference has to be collected separately from the data it validates, otherwise you are checking a source against itself. In a newsroom that reference is usually a source document, a records request, or your own observation on the ground.
What is the difference between ground truth and ground-truth data?
Only the punctuation differs, not the meaning. As an open compound noun, write ground truth: the ground truth we found in the filings. As a modifier placed before another noun, hyphenate it: ground-truth data. Most style guides treat the hyphen as a matter of clarity rather than a strict rule, so either form is accepted in body copy.
How do I verify a dataset before publishing a story?
Rewrite your main claim so it names a place, a period and a definition. Find the primary source document and the underlying data file, then pick a method before you look: spot-check, cell-level audit or full recomputation. Sample systematically across categories and time periods, record every agreement and mismatch, and log each result. Publish the method with the story.
What should go in a methodology note for a data story?
Five items carry most of the weight: where the data came from, how the sample was chosen and how large it was, which figures you reproduced by hand, which figures you did not check, and what you could not resolve. Link to a versioned snapshot of the files rather than a live page. Readers can then repeat your check, and a rival outlet can try it too.
Conclusion
How to ground truth data with reporting comes down to four habits: define one claim tightly enough to fail, sample it systematically rather than conveniently, compare it against a source that did not produce it, and write down every result including the ones you could not explain.
Start with one number. Pick the single figure your story would collapse without, pull the source document, redo the arithmetic, and log the result. That single pass usually teaches you more about the dataset than a week of reading about validation frameworks, and it gives you a template you can repeat on the next figure.
