Stop a journalist on the street and ask her to list the fundamentals of the job and you’re almost certain to hear mention of accuracy.
In “The Elements of Journalism,” Bill Kovach and Tom Rosenstiel wrote that journalism’s “essence is a discipline of verification.”
But how do journalists actually go about verifying information in their everyday work? What does it look like in practice, and how does it vary from one reporter to the next?
Fundamental questions, and yet there’s little academic research to answer them.
“While there is a long tradition of measuring news reports’ accuracy post hoc … substantially less work has examined the processes by which journalists seek to attain accuracy,” write Canadian journalism researchers Ivor Shapiro, Colette Brin, Isabelle Bédard-Brûlé and Kasia Mychajlowycz in their newly published paper, “Verification As A Strategic Ritual: How journalists retrospectively describe processes for ensuring accuracy.”
It’s perhaps the first paper to offer a look at how working journalists view and practice verification.
The researchers found that verification is widely seen as essential and core to a journalist’s work. But at the same time, the methods for achieving accuracy vary from one journalist to the next. There is no single standard for verification, and not every fact is treated the same.
“A small, easily checkable fact needs to be checked; a larger but greyer assertion, not so much — unless it is defamatory,” they write. “Thus, verification for a journalist is a rather different animal from verification in scientific method, which would hold every piece of data subject to a consistent standard of observation and replication.”
The method
To gather data, the researchers interviewed 28 Canadian journalists (men and women; French and English), half of whom had recently won an award for a piece of work; the other half were selected after the authors chose “14 stories semi-randomly from a constructed population of texts commensurate in length with the set of award-winning stories.”
They met with the journalists and spoke about the verification practices used to produce the stories.
Like good researchers, they noted one weakness to this approach.
“We were relying entirely on the subject journalists’ own accounts of their work, with no available means to verify (!) the truth of those accounts,” they write. “Apart from the possibility of the subjects varnishing their verification efforts, we also were limited by the capacity of their memories.”
Variations in verification
One theme in the paper is that different journalists practice verification in different ways — though they all agree it’s hugely important.
‘‘There’s no point in being a journalist if you’re not going to relay accurate, correct factual information to the public,” one interviewee said.
Shapiro told me by email that, “while journalists see the norm of verification as quite pivotal to their professional identity, the recognition of this norm is not quite matched by the kind of methodological discipline that Kovach and Rosenstiel speak of.”
Rosenstiel, a member of Poynter’s National Advisory Board, contributor to Poynter.org, and the executive director of the American Press Institute, disagrees with Shapiro’s assessment of how the findings relate to the treatment of verification in “The Elements of Journalism.”
“Far from challenging what we found in Elements, the study reinforces it,” he told me. “We conclude, as they do from their 28 interviews, that journalists aspire to accuracy and being truth tellers but lack standard routines or sufficient intellectual training for seriously doing it. We note that these routines are highly individualized and idiosyncratic. We also outlined some of those individual routines as a way to suggest how to make this discipline more conscious and more serious. That, indeed, is the point. Elements is a call for journalists to live up to their aspirations with more rigor, not a celebration of current practice.”
So why are journalists unable to systematize their aspirations of accuracy? In that regard, the paper shared an important fact: there’s little specific guidance for verification offered in journalism textbooks. They write:
Many journalism textbooks are devoid of references to verification or fact-checking (e.g. Frost 2002; Gaines 1998; Harcup 2004; Harris and Spark 1997; Spark 1999) or confine themselves to only the briefest references to the importance of double-checking basic facts such as names, ages and locations, and the necessity for more than one source where a charge of misconduct is made (Lanson and Stephens 2008).
I asked Shapiro, who is chair of the School of Journalism at Toronto’s Ryerson University, why something so important has not been standardized and incorporated into core teaching material.
“Columbia [University’s journalism school] has a course called ‘evidence and inference’; Ryerson has a course called ‘Exactly so’, and I am sure many schools try to teach verification strategies and critical thinking…. but textbooks are another matter,” he said. “There are probably some great ones out there that do a good job of tackling this question, but I haven’t seen them yet.”
This partly explains the varying ways journalists practice verification: They come into the profession having been taught in different ways, or perhaps not much at all.
And yet, we place accuracy on a pedestal.
“We actually found the interviews journalists’ passion for accuracy often inspiring,” Shapiro told me. Journalists “have to juggle competing priorities in delivering products that their employers can sell. Not one of our 28 interviewees gave us any reason to think that s/he does not take the responsibility for accuracy very seriously.”
“They are professionals, but they are artists, not scientists – and, mostly, artists on a deadline,” Shapiro added.
One positive trend I’ve seen in recent years, which wasn’t part of this specific research, is that the increasing use of user-generated content by newsrooms has resulted in organizations creating a defined verification process. (Read more about the processes in place at Storyful, the BBC, AP and CNN’s iReport.)
This is encouraging, particularly as new technologies and media continue to transform newsgathering and fact finding at a rapid pace.
How journalists verify
“There’s no hard and fast rule about any of this stuff,” one interviewee told the authors. “You have to exercise your judgment all the time.”
‘‘[T]o me, verification is much more rooted in the actual reporting process, step by step and looping back in upon itself,” said another.
Among the interviewees there were those who applied a disciplined approach to verification.
“Some arrived for the interview armed with indexed binders full of source materials; some had clearly refreshed their memories of the reporting by reviewing their notes, and related articles, before their meetings with us; one checked additional facts and followed-up via email,” the researchers write.
A commitment to verification may be shared across journalists, but this research suggests any shared norms are combined with variations in practice.
“Methods for ensuring accuracy varied greatly, with some factual statements relayed, with or without attribution, based on a single subject’s word, while others were rigorously triangulated,” the authors write. “Strongly idealistic statements about the need for verification were often made during the course of the same interview as were indications of methodological ambiguity.”
In terms of specifics, here are a few findings from the research:
- On checking names: “An almost universal practice among participants is asking sources to spell their own name to ensure correct spelling, either at the beginning or the end of the interview.” Some journalists also check names against official sources.
- On offering sources pre-publication review: “Despite some evidence in the literature that partial pre-publication review is not the taboo it used to be (Stoltzfus 2006; Carr 2012), our subjects displayed a strong sense that it was a discouraged practice.”
- On quotes: “…methods for checking the accuracy of quotes vary greatly. Some reporters routinely record and transcribe interviews, while some record but rarely transcribe and others rarely use recorders at all. Some check quotes against tapes only if there is a specific concern, such as difficulty hearing, or the threat of libel litigation.”
- On a source’s personal history: “Facts relating to a source’s personal history are considered not to require verification … or are simply not verified because there is no practical way to do so.”
A ‘Strategic Ritual’
In the end, the researchers likened the journalists’ commitment to verification to a doctor’s oath to “do no harm”:
Put another way, using the language of professional identity rather than professional ethics, verification might be seen as a ‘strategic ritual’, as Tuchman (1972) said of the aforementioned (and perhaps now rather old-fashioned) idea of ‘objectivity’ — something that legitimizes a journalist’s social role as being demonstrably different from other communicators.
Now let’s work on giving them better guidance and practices to make the ritual even more real, and reliable.
As Rosenstiel told me, “the aspiration to vet the news is an essential goal of most journalists, but … the processes for living up to that goal are not well-defined and not rigorous enough. And for journalism to survive, much more needs to be done to give the process of verification more throw weight.”
Comments