Journalist-sourced reliability ratings of news outlets could actually affect readers’ perceptions of their accuracy, according to a new report from the Knight Foundation and Gallup. The ratings helped audiences across the political spectrum identify trustworthy news sources compared to misinformation.
The survey asked 2,000 adults in the U.S. to rate the accuracy of 12 headlines. Six came from reliable news sources (some national, some local; see the list at the bottom of this article) and six came from unreliable sources known to spread misinformation. Experienced journalists determined the reliability ratings based on evaluations of sources’ work, funding and other factors.
A control group of participants viewed the headlines with no source rating, another group viewed them with a red flag of “unreliable” or a green flag of “reliable,” and a third group viewed the headlines with a red or green flag and an added option to click to see more about the source.
The ratings worked — outlets marked as green were perceived as more accurate, and red outlets were seen as less so. Participants were less likely to read, share and like headlines marked as red.
“This survey experiment offers compelling evidence that the use of an online tool to indicate news organization reliability increases healthy skepticism when individuals consume news online,” the report reads.
The green flag increased the probability that respondents would read reliable sources by 5.7 percent, while the red flag decreased the probability of reading misinformation by 6.2 percent.
The ratings worked across the political spectrum — the perceived accuracy of sources marked as red declined significantly for both Republicans and Democrats surveyed. These source ratings likely caused partisans to think critically and question the reliability of headlines even if they aligned with their political beliefs, according to the report.
However, while Democrats were less likely to read, share or like stories with a red flag, Republicans’ behavior didn’t change after seeing the red flag.
In the control group that didn’t see ratings with the headlines, 44 percent of participants said they would share at least one false story. Even with visible ratings, 34 percent still said they would share a false story, primarily to spread the story’s message to a wider audience.
There's more research to be done, and other studies have shown that credibility labels don't strongly influence how audiences perceive news stories.
Reliability ratings or fact-checks to combat misinformation are often associated with the “backfire effect,” where correcting false beliefs (such as with a fact-check) actually strengthens them. This didn’t apply in the survey, possibly because the flags gave general reliability ratings to a site rather than refuting specific facts or claims, according to the report.
Sources marked "reliable" included the National Review, Vox, Portland Press Herald, Baltimore Sun, Axios and The Marshall Project. Sources marked "unreliable" included Mediaconservative, Boston Tribune, Truthfeed, TD Alliance, National Report and Seattle Tribune.
