By Dr. Pegie Stark Adam, Sara Quinn and Rick Edmonds
We’ve described how deep into a story participants read, whether they were viewing broadsheets, tabloids or Web sites as methodical readers or scanners. But how much do they learn and recall? Eyetracking can capture precise eye movements, but it cannot measure comprehension or how much information a reader retains.
In order to study that, we created prototypes to be read by each respondent. After each person was eyetracked while reading the newspaper or Web site, she was given one of six prototypes to view (three in print, three online). She received a print prototype if she’d read a broadsheet or tab; she viewed an online prototype if she’d read a newspaper Web site. Prototypes were distributed to readers sequentially after they completed the eyetracking portion of the research.
All prototypes contained the same information on bird flu, including many facts about the disease’s origins, spread and the possibility of a bird flu epidemic. The information was edited and packaged in three different ways.
Prototype 1 was conventional, with headline, narrative and photograph. Prototype 2 contained a narrative story with some of the information broken out in a map and some in a fact box. Prototype 3 was very visual with no traditional narrative. It featured alternative story forms — a map, a Q&A, a numbers chart and other graphic storytelling.
The print prototypes were in single-page broadsheet form on newsprint, and each contained a column of unrelated briefs. The Web site prototypes also contained unrelated briefs.
Each person read one of the prototypes and was not shown the other prototypes. The readers were not told they would be tested on the information in the prototypes, nor were they given a time limit. After five minutes, we asked them to stop reading. Then, readers were asked to answer nine questions on a computer. The questions focused on facts about bird flu.
Our hypothesis was that the more graphic presentation of a news story in Prototypes 2 and 3 –- relying on text blocks and visual storytelling — might lead to better comprehension and information retention. What we found supported that hypothesis.
Readers of Prototype 3 answered the most questions correctly.
Readers of the prototype that displayed no narrative, with the story told through an explanatory map, a Q&A, a numbers chart and more, answered the most questions correctly when asked about facts on bird flu.
There were 156 respondents who answered at least 7 of 9 questions correctly; Forty-six percent of them read Prototype 3, 28 percent read Prototype 2, and 26 percent read Prototype 1.
Both print and online readers of Prototype 3 showed greater recall, but it was more pronounced among print readers; Forty-three percent of people who read Prototype 3 in print answered 7 or more questions correctly while 25 percent of those who read Prototype 3 online answered the same number correctly.
Ten subjects answered all nine questions correctly. All but one were Prototype 3 readers. Seven were print readers and all read Prototype 3. The other three were online readers; two read Prototype 3 and one read Prototype 2.
Regardless of which prototype was read, broadsheet readers had the highest recall rate, while online readers had the lowest.
Why did print readers recall information better than online readers? While information about bird flu was available on a single broadsheet page, it appeared on as many as six screens within the prototype Web sites. In addition, the broadsheet prototypes looked like conventional news pages that readers were used to seeing. The prototype Web sites looked different from the daily Web sites most online readers routinely see.
In addition to learning which prototype would generate the most accurate responses to factual questions, we wanted to gauge reader reaction to the story and its presentation.
We were curious about whether the more graphic presentations of the bird flu story in Prototypes 2 and 3 would generate more interest, and compel readers to want more information on the topic.
So we asked participants for additional information in the computer questionnaire. We gave readers a list of six statements related to the bird flu story. They were asked to indicate their level of agreement or disagreement with the statements, using a scale of 1 to 4 where 4 indicated “agree completely” and 1 indicated “disagree completely”:
• I found the story so compelling I couldn’t stop reading it.
• After reading about bird flu, I want to talk to my doctor to get more information.
• I learned something new about bird flu.
• I feel that the topic of bird flu was covered thoroughly.
• I want to find out more about bird flu.
• I’m worried about getting bird flu.
Readers of Prototype 3 generally showed a higher level of agreement with the statements, especially the statement about the topic being covered thoroughly. A higher percentage of print readers than online readers agreed with the statements.
Readers said the story was well covered and that they learned something. Two statements generated high agreement (more than 50 percent for all readers). They were:
• I feel that the topic of bird flu was covered thoroughly. (79 percent agreed.)
• I learned something about bird flu. (65 percent agreed.)
Two statements generated little agreement (under 20 percent):
• After reading about bird flu, I want to talk to my doctor to get more information. (13 percent agreed)
• I’m worried about getting bird flu. (17 percent agreed)
Commentary by Dr. Pegie Stark Adam: Our findings echo the results of a study I conducted in 1991 with about 400 college students at the University of Florida and the Indiana University. In that research, I created four prototypes with the same story displayed in different ways — one with story text only, one with story text and photo, one with story text and graphic, and one with story text, photo and graphic. Students who read the prototype with story text, photo and graphic answered more questions accurately than the other readers. Like the bird flu story in EyeTrack07 prototypes, the story used in the 1991 study was loaded with facts — numbers, a diagram, step-by-step information on how a plane crash happened. This suggests we need to be cautious and avoid displaying all stories in alternative ways. When deciding on story form, let the content be your guide.
