Donald Trump was about 20 minutes into his address when he touted job creation.
“Since the election, we have created 2.4 million new jobs,” he said during the State of the Union on Tuesday night.
Not so fast, Eugene Kiely said.
“In Trump’s first 11 months, employment increased by 1.84 million — 12 percent lower than the 2 million jobs that were created in the previous 11 months,” the director of Factcheck.org wrote — only seconds after Trump made the claim.
That quick take was the first thing posted to FactStream, an app developed by the Duke Reporters’ Lab to compile live coverage from fact-checking organizations into one place. PolitiFact, The Washington Post Fact Checker and Factcheck.org all contributed to the beta, which launched specifically to test during the State of the Union, both in the form of links to prior fact checks and quick comments on what Trump was saying.
During the State of the Union, Poynter tested FactStream alongside Twitter lists composed of journalists and, more specifically, fact-checkers. We found that the app surfaced important context and fact checks that often got buried under a mountain of quote regurgitations, GIFs, quick takes and photos that dominated Twitter. While feeds that constituted only fact-checkers fared better, they still didn’t lend a lot of breathing room to fact checks.
Quick thoughts on FactStream so far:
• It’s super fast — new entries appear before my YouTube livestream even gets to the claim
• The difference between quick takes vs. related fact checks is nice for adding instant context
• It helps cut through the noise on Twitter#SOTU— Daniel Funke (@dpfunke) January 31, 2018
And Kiely said, from a fact-checkers’ perspective, the process was a natural one.
“On my end, it went very smoothly,” he told Poynter in an email. “I had created a long list of possible claims that I thought Trump might make and then had pre-written responses and links to past stories ready. That way, I could easily tweak the language based on his exact words and then post quickly.”
Usually, live coverage of speeches like the State of the Union primarily takes place on Twitter. But amid the noise of commentary from pundits, jokes from social media personalities and hot takes from friends, it can be hard to find work from fact-checkers in the general feed — especially since they all publish differently.
Bill Adair said that’s where FactStream adds value.
“When you do a live event like this you’ve got to be quick,” the Reporters’ Lab director and Knight Professor of Journalism & Public Policy told Poynter prior to the State of the Union. “The app is based on the phenomenon that politicians repeat their talking points and often we have fact checks based on the new things they’re saying.”
“The whole purpose of the State of the Union exercise is to learn lessons.”
Over the course of the address, about 3,100 people opened and engaged with FactStream, Adair told Poynter in an email Wednesday, with more than 1,000 users using the app concurrently at one point. All told, the three fact-checking partners published 32 updates to the app — the majority of which were partially or wholly false, by Poynter’s count.
There were some hiccups, namely a couple of mismatched URLs for The Fact Checker. A few users also requested notifications for new fact checks and didn’t understand the different levels of detail in the app, which Adair said will lead to more user testing and tweaks.
Still, he said he’s heard mostly good things from users who used the app during Trump’s address.
“Users got real-time fact-checking in a single app. The fact checks were timely and valuable,” he said. “Our feedback from users was overwhelmingly positive.”
Staci D. Kramer, a veteran journalist interested in fact-checking, used FactStream to follow along during Tuesday’s address. She told Poynter in a Twitter message that the app performed surprisingly well, but that she’d like to see one key interface issue resolved before its next test.
“The timely fact checks didn’t cover every claim I wanted to track, but it collected enough in one place to make it a tool I’ll use again — if another issue is resolved,” she said. “Most fact checks repeated the claim on the first screen without a clue about validity … As the speech wore on, I stopped looking. Instead, I went back later to scroll through and follow some links to details.”
FactStream is an outgrowth of the Duke Tech & Check Cooperative, which received funding in September to build automated apps and tools for fact-checkers. While the app isn’t automated, instead fueled by manually published updates from fact-checkers, the Cooperative has also built an alert service that automatically scans CNN transcripts for checkable claims, then relaying them via email to fact-checking partners.
Beyond FactStream, fact-checking took front and center Tuesday night. PolitiFact’s site crashed for a few minutes after a surge in traffic, and at least two of its Trump-related Facebook posts went viral within an hour after they were published. The New York Times, CBS, CNN, The Boston Globe and other mainstream outlets published their own fact-checking roundups from the speech.
Overall, it seems the ratio of false to true ratings in FactStream was a good indicator for how factual Trump was during his address.
“President Donald Trump offered a rosy assessment of American life in his first State of the Union address — but several of his points were factually flawed,” PolitiFact wrote in its review last night.
