The death of facts may be exaggerated, but you might forgive fact-checkers for mourning.
Fact-checking started as an attempt to practice journalism the way readers say they want it: by comparing politicians’ claims against reality, rather than against each other. Combine rigorous reporting and analysis with deep research, the theory went, and readers will reward you with their eyeballs, loyalty and trust.
Yet as Howard Kurtz recently pointed out on his Fox News show, “MediaBuzz”, not everyone is buying it: “It shows you how much the media have lost the confidence of the American people that so many of you say, well, I don’t believe your fact checks because I don’t believe the media.”
This is hardly a new phenomenon, and it’s not restricted to the U.S. In reporting this story, we spoke to fact-checkers from Canada, Australia, Turkey, Italy and South Africa, all of whom said they battle accusations of bias. “We’ve gotten charges of bias pretty much from the very beginning, and in my discussions with other fact-checkers, they get it too,” says Angie Drobnic Holan, editor of the U.S.’s PolitiFact.
Africa Check, a nonprofit fact-checking organization based in Johannesburg, South Africa, also faces accusations of bias from multiple sides of the political spectrum, said Africa Check executive director Peter Cunliffe-Jones.
“We are accused of both being at the beck and call of the African National Congress (ANC) and being anti-ANC, being part of a white colonialist establishment and part of an anti-white liberal elite,” Cunliffe-Jones said.
Fact-checkers understandably find this a key challenge. How can they address reader concerns about bias? We talked to practitioners and researchers to find out.
Avoid bias in the first place. This one may seem obvious, but it’s easy to get complacent about it. Research shows that like all humans, journalists have a tendency to underestimate their own biases, says Tim Groeling, professor and chair of communications studies at the University of California, Los Angeles. Good training helps journalists recognize and adjust for bias, but fact-checkers have to start by acknowledging that they have blind spots.And their process has to be as objective as possible — which Groeling thinks isn’t always the case right now. For one thing, Groeling says, fact-checkers too often rule on what they think the politician meant, rather than what the person actually said. “Those drive me crazy,” Groeling says.But the fact-checkers we spoke to pointed to a variety of systems that help keep their work honest. At PolitiFact, three editors debate every rating awarded. At FactCheck.org, every article gets reviewed by four people who were uninvolved in the reporting and writing. Similarly, at ABC Fact Check, run out of the Australian Broadcasting Corporation, each fact check goes through a “chief fact-checker” uninvolved in the research up to that point — and then on to a panel of experts outside of ABC.
Be transparent. Not only does fact-checking methodology have to be rigorous — but to earn readers’ trust, that methodology must be readily available and understandable. The transparency principle extends to sources and corrections policies, too. Italy’s Pagella Politica has an “oops page,” a live post that tracks every time the outlet has corrected one of its fact checks or changed a verdict. Africa Check’s website has a section called “How to fact check,” and the outlet encourages readers to use this to scrutinize Africa Check itself.In Canada, FactsCan tries to play offense by engaging with critical readers on Facebook and Twitter. The organization will often link to earlier fact checks to demonstrate its balanced coverage. It will ask readers to get very specific about criticisms so it can see if it made an error. And, co-founder Jacob Schroeder says, “Even if you’re not changing that person’s mind, you’re still demonstrating that bias is being taken seriously.”
Think carefully about your claim selection process. Transparency takes different forms at different organizations, and best practices are not universally agreed upon. One key point of disagreement: should the outlet try to do a certain proportion of fact checks for each party? Or should it choose solely based on what claims are most newsworthy, interesting, and checkable?Turkey’s Doğruluk Payı tries to write fact checks in proportion to the number of seats each party has in parliament. “They don’t have the exact proportions but they’re close,” project coordinator Batuhan Ersun says. ABC Fact Check doesn’t have a choice: it must give parties equal time, in keeping with ABC’s charter as a public corporation. FactsCan decides what to fact-check based on the party’s presence in parliament, modified either by the party’s standing in polls (if an election is coming up) or the percentage of popular vote received. The organization displays the party breakdown of its fact checks as a pie chart on its homepage.But, Holan argues, “I think picking [claims] based on news value is a greater reader service.”Still, even if they’re not fact-checking parties equally, outlets can take steps to assure readers of their objectivity. “It’s plausible that if you give readers a reason for why you’re fact-checking one side more than the other… then that’s going to make them a little bit less upset,” says Emily Thorson, a political science professor at Boston College. Fact checks can also link back to checks of the opposing side, as Africa Check and other outlets do.
Know how people think and acknowledge difference. Social science predicts that when people disagree with your finding, they’ll often call you biased. In a study for the American Press Institute (now being peer-reviewed), Michelle Amazeen of Rider University, Thorson, Ashley Muddiman of the University of Kansas and Lucas Graves of the University of Wisconsin found that people who read fact checks with a finding of “false” found the fact-checker less credible if the politician in question was from the reader’s own party.That’s backed by a substantial body of literature demonstrating a “hostile media effect,” one that goes well beyond fact-checking to other types of journalism, Graves says. These experiments have found that two people from different parties can look at the same story, and each find it as biased against his or her ideology.The upshot in the U.S. seems to be that Democrats trust fact-checkers more than Republicans do. That finding was reinforced by research from Brendan Nyhan of Dartmouth College and Jason Reifler of the University of Exeter, which revealed that Democrats were slightly more persuaded by fact checks.At the same time, some research has found that different political opinions expressed by Democrats and Republicans may be more for show — a sort of “partisan cheerleading” — than a true expression of belief.
Research suggests ways you can turn these human tendencies to your advantage. One is to use sources your readers will trust (but that still provide accurate information). Holan explains: “If a gun rights group says, ‘Yeah, that claim isn’t right,’ gun rights supporters would be more likely to accept the finding.”
Fact-checkers must also show that they understand, and respect, the different beliefs and base assumptions that readers start out with, Amazeen argues. Otherwise you risk alienating people from the start.
Get creative. Fact-checking is an evolving practice, with new innovations cropping up all the time. There’s likely more that can be done to both to ensure the rigor of fact checks and to earn readers’ trust.At the recent Tech & Check conference, some participants suggested that fact-checkers share their “pipelines” — the list of claims they initially considered checking. Some or all of the claims could be suggested by readers.News organizations could go a step further, as the U.K.’s Full Fact recently started doing, with Reddit-style up and down votes to allow the public to choose what should be checked. Or, Groeling says, members of the audience could be chosen on a random, rotating basis to help select claims — a fraud-prevention approach similar to the system that tech news site Slashdot uses for comment moderation.And to address latent journalist bias, Groeling suggests that an outlet often perceived as left-leaning — like The New York Times — could partner with an organization accused of leaning right — like The Wall Street Journal — to create joint fact checks.
As intractable as people’s beliefs can be, and as much as this colors perceptions of fact-checkers’ work, there is at least one bright spot for this brand of journalism: Research* has found that readers prefer it. “Fact-checking in itself is part of the prescription for journalism to try to build trust,” Graves says.
Perhaps the best mindset for that trust-building: A balance of optimism and realism.
“Perfection is impossible,” says Pagella Politica managing director Daniele De Bernardin. “So credibility is what we want to achieve.”
Disclaimer: The author performs paid consulting work for the American Press Institute. This has included blog posts on the research mentioned above by Thorson, Amazeen, Graves, Muddiman, Nyhan and Reifler.
*An abstract of Thorson’s paper on the topic is available here. Search “Thorson” and “2013” and look for “Effects of the Misinformation Economy on Citizens, the Media, and Politicians.
