Crowdsourcing during a crisis has its drawbacks

With all the talk of database journalism and mapping data, one would think crowdmapping would be taking off. But, it’s unclear how useful the practice is for journalists, especially with data collected in dangerous humanitarian crises, like the one in Syria.

There are success stories. Crowdmapping software Ushahidi immediately comes to mind. It gained popularity in 2008 when the software was used to map the fallout of the post-election violence in Kenya.

The Ushahidi software was celebrated for its ability to synthesize and geotag user-generated content in a simple way. It’s been used to map casualties in South Sudan, provide critical humanitarian information during the earthquake in Haiti and, most recently, to map out casualties in the Syrian war.

But data reporters and social scientists are still experimenting with how to verify the data on crowdsourced maps, to present clean, reliable information. Susan McGregor, assistant director of Columbia University’s Tow Center for Digital Journalism, thinks journalists are still adapting to the concept of using, visualizing and verifying data, let alone crowdsourcing it. “We are better at reading physical cues. We can judge if someone is trustworthy face-to-face,” she said. “The bandwidth of text is very narrow.”

That may be one reason why – despite their cachet among data journalists – early crowdsourced maps have not drawn much of an audience, according to a report from the nonprofit, Internews. Journalists are also grappling with how to minimize the risks to the people putting sensitive information into such a public space.

The Challenges of Mapping Syria

Taha Kass-Hout encountered these challenges when he created Humanitarian Tracker over three years ago. Kass-Hout, who is trained in biostatistics and also has an MD, hoped to create a platform for witnesses of crises to report on them directly through a secure Internet portal. He works with a team of volunteer social scientists to visualize data and verify user-generated content on crisis in the world.

For one project in particular, called Syria Tracker, this team has used tweets, videos and photos from sources on the ground to crowdmap the extensive casualties of Syria’s civil war.

But the dangerous nature of the conflict makes it difficult to verify reliable sources, said Katie Lee, a data scientist who works with Humanitarian Tracker. When transparent governments collect reliable information and when multiple journalists are present, it’s far easier to verify information.

in Syria, though, there are few ways to double-check information on the ground. “When you have things like sniper attacks, artillery and gunfire which are the top leading cause of death of women in Syria, I think that’s where the line gets a little blurry,” Lee said.

It’s taught the team to trust few citizen journalists in the country. “Out of the 600-plus reporters [who have posted] over the past few years, we consider about a dozen of those to be credible,” Kass-Hout said. As a result, they’ve only published about 5,000 of the more than 80,000 reports they’ve received on anything from casualties, to chemical weapons use, to sexual assault.

But, these trusted sources place themselves at a significant risk to transmit data to Kass-Hout. Governments around the world monitor the Internet and could take reprisals against a citizen who puts sensitive information on a crisis crowdmap. “Along the way, we have lost reporters. We get reports from them for months and months and then we stop getting reports from them,” Kass-Hout said. He rarely finds out why.

Syria Tracker encourages reporters to not give their names and to use encryption software such as Tor to protect their identity online. Still, some slip through the cracks. “At least we know that we did our part to make sure that they’re protected,” Kass-Hout said.

Lara Setrakian, a former foreign correspondent and founder of the news aggregator Syria Deeply, thinks that the verification process should be even more stringent for both citizen and mainstream media journalists compared to that of some of the advocacy-based crowdmaps.

The data presented on Syria Deeply’s crowdmap comes only from well-vetted sources – the Violations Documentation Center, an activist-run group that collects data on human rights abuses based in Syria, for casualties, and the UN High Commissioner for Refugees. The VDC has a network of their own reporters on the ground to report the casualty figures. The data then gets crosschecked with other reports from Syria documenting similar cases. The UNHCR also has its own strict verification process and functions largely in the refugee camps outside of Syria.

Setrakian is skeptical of the value of crowdsourced data that does not go through as rigorous a verification process, especially in a conflict as polarized as the one in Syria. “Do I trust someone who hates Bashar Al-Assad to report in an unbiased way? Not really,” she said.

Thus the crowdmap at Syria Deeply remains more of an experiment than a core part of its reporting. It doesn’t have the same crowdsourced roots that many other Ushahidi maps have, presenting less total information. “I think it would be most effective in those quantitative measures, in those spaces where you have numbers that need to be collected,” Setrakian said. “Can we visualize the location of bakeries, how about the price of bread? Things that are depoliticized.”

The crowdmap designed by Women Under Siege, a group that works to document rape and sexual assault in conflict situations, has a more concrete purpose. The map aggregates reports of sexual assault from citizen journalists, mainstream media and non-governmental organizations. Lauren Wolfe, director of Women Under Siege, said she hopes that there will be a war tribunal to try the atrocities taking place in Syria once the civil war is over – and that the group’s crowdmap could be used as evidence to contribute to justice.

Most of the reports are listed as unverified, though, in contrast with journalistic standards. Still, Wolfe considers the work she does to be journalism. “We are reporting every case we come across, impartially, whether the perpetrators are allegedly government or rebel forces,” she said. “I believe in journalism for the public good and if some people want to call that advocacy, fine. It’s not what I call it,” she said.

From a legal standpoint, gathering evidence and testimony through crowdmapping is still a relatively new and untested method said Dinah PoKempner, the general counsel for Human Rights Watch.

The anonymous nature of the reports would make it particularly hard to go back and gather testimony, PoKempner said. “Rape is an inordinately difficult crime to get any kind of report on,” PoKempner said, due to the difficulty of gathering biological proof of the crime.

That doesn’t mean she thinks the endeavor has no validity. A critical mass of reports may in fact be enough to convince a judge that targeted sexual assault did occur in Syria. “There’d be a probability that at least some of [the reporting] was done in good faith,” PoKempner said. In the meantime, it’s up to the readers to decide whether they view the unverified testimonies as sufficient evidence.

The Future of Crowdmapping

Anahi Ayala has been studying crowdmapping since its inception as a part of the Internews Center for Innovation and Learning. Their research culminated in a report, called “Mapping the Maps,” [when] where they looked at who actually used the information available on crowdsourced maps and the most effective kinds of data that can be presented.

In their study, Internews found some of the same issues as the Syria crowdmappers – data was difficult to verify, or the projects simply weren’t able to stand on their own. Fewer than a third of the people who put information on a crisis crowdmap felt that it actually delivered a message heard by policymakers.

It also served little purpose to Syrians on the ground. “If I was a Syrian in Syria, it’s not really going to help me out,” she said about many of the Syria mapping projects that exist. “If I have to go online and I have 30 minutes to spare, I’ll probably check a lot of other information than a crowdmap on Syrian casualties.”

The Tow Center’s Susan McGregor disagrees. “It has the potential to explore a greater diversity of voices,” McGregor said, explaining that crowdmapping can be another tool for empowerment to citizen journalists. “The act of [reporting] is actually the objective [in some cases].”

But, she thinks it’s only a matter of time before crowdmapping and crowdsourcing in general becomes a more accepted reporting tool in North American journalism. While there are some superficial differences, she says the challenges of verification faced by crowdmappers aren’t that different from the challenges most reporters face. “It’s a lot of the same strategies you would use [in reporting] in-person.”