Today an Argentinian fact-checking outlet is taking a big step toward automating more of its workflow.
Chequeado has announced the release of Chequeabot, a tool that automatically identifies claims in the media and matches them with existing fact checks. The bot draws upon natural language processing and machine learning to help perform everyday tasks for fact-checkers — who are already using it in the newsroom.
Over the past year, several Chequeado fact checks began with Chequeabot, which automatically scans text from 25 media outlets around the country and identifies claims from politicians. One fact check about the electricity grid was unearthed from coverage of a politician’s press conference, while two more came from newspaper and radio interviews.
That feature, called “¿Qué se chequea hoy?” (What should we check today?), now leads off each Monday meeting at Chequeado.
“We like to think that Chequeabot is part of the meeting because he or she can tell us what we should fact-check that week, usually very accurately,” said Editorial Innovation Director Pablo Martín Fernández in an interview with Poynter. “Chequeabot is finding claims in the whole country, and that is something that’s important for us — not being biased and only focused on (Buenos Aires).”
After Chequeabot finds some dubitable claims, it matches them against a growing database of existing fact checks, which Fernández said is about 1,000-strong. Then, the bot provides a raw text file with links so that fact-checkers can easily tweet it or post it on Facebook.
“That is also great because right now we have to trust the memory of the journalists in the room to say we already fact-checked that,” he said.
Building something like Chequeabot has long been a goal for Chequeado, which has been thinking about automation since before the inaugural Tech & Check conference in 2016 at the Duke Reporters’ Lab (The International Fact-Checking Network co-hosted the event). Later that year, the organization received a fellowship from the IFCN to travel to the United Kingdom and work with Full Fact, a fact-checking charity that has also been developing automated tools.
Among the lessons from that visit — such as the need for common standards, collaboration and better structured fact checks — was the realization that developing automated technology in Spanish would have to be done mostly from scratch, since most existing tools are focused on English speakers.
So that’s what Chequeado did. It helped that Mariano Falcón, one of its developers, happened to have some natural language processing experience.
“That’s a technique that is really important for this,” Fernández said. “It’s much harder in Spanish because we don’t have some tools that already exist in English.”
Falcón uses tools like Freeling from the Polytechnic University of Catalonia, Nltk and Gensim to extract text, parse through sentences and build the predictive machine learning that powers Chequeabot — a name that Chequeado’s Twitter followers chose. And automation experts Poynter spoke with were impressed to learn about it.
Giovanni Luca Ciampaglia, an assistant research scientist at the Indiana University Network Science Institute, told Poynter that the greatest asset of automated tools like Chequeabot and Full Fact’s Live platform — aside from the obvious streamlining of newsroom workflows — is the fact that they’re being developed by fact-checkers themselves.
“Projects like Chequeado and Full Fact are best-positioned to apply this kind of technology at this moment because their focus is specific to a particular domain, country or language,” he said. “That simplifies the task greatly because once you know the sources you’re following, that makes things easier.”
And James T. Hamilton, the Hearst Professor of Communication and director of the journalism program at Stanford University, told Poynter in an email that Chequeabot’s methodology — automatically scanning media outlets for false or misleading claims, then applying existing fact checks to them — is a good one.
“This approach works in part because claims are recycled by politicians using talking points and are recycled by journalists copying points made in prior media reports. Even after claims are fact-checked and publicized as unfounded, they may be repeated by politicians and other reporters,” he said.
That said, Chequeabot does have limitations.
While British public records laws are fairly open and TV networks like the BBC freely share their captions, allowing Full Fact to create automated feeds based on both, access to raw data in Argentina is harder to come by. Fernández said Argentine TV stations don’t provide closed captions because it’s rather expensive, and the government’s databases are still mostly closed to the public.
In both cases, Chequeado is advocating for change and looking at potential collaborations with the government in order to make Chequeabot more comprehensive. Accessing official databases, through APIs, on, say, the inflation rate — a hot-button issue in Argentina — could help Chequeado check more politically relevant stories. Fernández said the organization is also looking at adding data feeds from universities, unions and think tanks to Chequeabot’s sources.
In either case, Fernández said the fact-checking outlet will look to speech-to-text translation technology like Google’s speech API to transcribe TV and radio interviews, which doesn’t function as well in Spanish as English.
In the future, Fernández said they plan to continue collaborating with Full Fact so as to not duplicate efforts. But while Chequeabot’s capabilities have already surpassed what many fact-checking outlets have been able to accomplish, he said Chequeado has no plans to replace its journalists with robots.
“We’re always having in mind that we want the journalist or fact-checker to be the last barrier,” he said. “Right now, it’s difficult to have an automation tool that creates content as humans do.”
Within the year, Fernández said Chequeado would like to start developing automated products that are more audience-facing. They also want to share the technology behind Chequeabot with other Spanish-speaking fact-checkers, in much the same way that Full Fact plans to partner with Africa Check.
“We are trying to make this useful to every Spanish-speaking newsroom in the whole world,” he said. “(Fact-checking) is something that, with a little help, we can do faster.”
