Fact-checkers, and all journalists, rely on data. But before you use the data to fact-check a claim or include in a story, it is essential to fact-check the data set itself.
Treat data sets like all other sources. No source should be trusted blindly. Just as you verify statements made by people you interview, check the track record of the organization and verify the data set.
Read the metadata. Before delving into the data set itself, read those small lines that no one reads. Metadata and methodologies can indicate what, if any, data points are missing or what has been estimated. Some estimates may rest on faulty assumptions.
Reconstruct the data collection. After reading the methodology, assess the reliability of the data-gathering process in light of the social and political context for the specific indicator. For cross-country data sets, check whether a single organization gathered the data or whether different offices shared information.
Test the spreadsheets themselves. The following tips are entirely adapted from Giannina Segnini’s chapter of the Verification Handbook: Check the lowest and highest entries to see whether they make sense or look suspicious. Assess what is missing: Are there empty rows that should not be empty? If you only see sample data, is it clear what was excluded and why? Randomly verify individual entries: Choose one or two records, and verify them autonomously through a search external to the data set.
Arm yourself with relevant tools. Fact-checkers often deal with data uploaded in highly user-unfriendly formats — for example, spreadsheets uploaded as PDFs rather than as CSV/Excel files, or databases that can be consulted only one query at a time. Not only is consulting these data sets more time-consuming, it also adds another layer of potential error as the fact-checker transcribes the entries manually. Learn to use online scraping tools such as import.io or Chrome’s Scraper.
Taken from Fact-checking: How to Improve Your Skills in Accountability Journalism, a self-directed course by Alexios Mantzarlis and Jane Elizabeth at Poynter NewsU.
Have you missed a Coffee Break Course? Here’s our complete lineup. Or follow along at #coffeebreakcourse.
