This article was originally published on FactCheckingDay.com for International Fact-Checking Day on April 2.
If you’ve never received a chain email from a relative linking to another viral study about chocolate, you’re lucky.
But for the rest of us, weeding through the most recent studies about health, science and other topics can be arduous. Bad studies can fool lots of people — even when they’re conducted by real researchers. Junk science is such a problem that it’s even become a popular listicle format for mainstream news organizations.
To address the tide of misinformation about science and academic studies, a few specialist fact-checking organizations like Climate Feedback and HealthNewsReview have emerged. A podcast run by professors at Boston University exclusively examines how studies are conducted and whether or not they’re trustworthy, which is similar to the Gimlet-run podcast Science Vs.
But fact-checkers can’t cover everything. Here’s a list of tips on how you can become a more discerning reader of fad studies and avoid spreading misinformation.
1. Be skeptical — the more fantastic or absurd the finding, the less likely it is to be true. Counterintuitive findings should take a lot more evidence to prove.
2. Try searching for the study on Google. Has it been covered by mainstream media organizations or fact-checkers? Oftentimes they will write about a study’s solidity.
3. Check the expert’s expertise. Media reports will sometimes cite or quote specific authors in a story, but they’re also available by searching for the study title. Look for their academic profile on Google Scholar, ResearchGate or individual institution and university websites. From Emmanuel Vincent, founder and project lead at Climate Feedback:
“The profile should link to a list of publications in peer-reviewed scientific journals. If an expert does not have recent publications on the topic at hand, it is much much less likely s/he is a scientific authority on the topic.”
4. At the same time, keep in mind that all journals aren’t created equal — some are more credible than others. Check the reputation of scientific journals by seeing if they’re included on this list of “predatory publishers,” compiled by University of Colorado librarian Jeffrey Beall, that only require authors to pay to publish. Scimago Journal & Country Rank is another good way to gauge a publication’s standing in the academic community. “Fake experts might try to publish in low ranking journals to appear as if they have an academic background,” Vincent said.
5. Watch out for conflicts of interest that aren’t immediately apparent from initial searches. For health studies, check HealthNewsReview.org’s list of independent experts. (For more tips on fact-checking health studies specifically, check out this past tip sheet.)
6. Think critically about what the numbers are actually telling you — not just how they’re being reported. Beware of causal language, because there are plenty of junk studies that conflate correlation with causation.
7. Find out what the study’s subjects were. Search for the study’s title on Google Scholar and read the abstract. Just because some findings hold true for test animals like mice doesn’t mean the same effects will be observable in humans.
8. Look for some of the study’s limitations. Oftentimes for peer-reviewed studies, these will be listed high up in an article, and are an important way to contextualize reporting on the subject. Think about what the study doesn’t tell you in addition to what it does.
For more tips on fact-checking scientific studies, read this past tip sheet. Have a tip that didn’t make either list? Send it to us at factchecknet@poynter.org.
