This article was co-published with Faked Up, a newsletter about digital deception, dreck and disinformation.
Just ahead of the election, X “re-architected” its Community Notes feature to make crowdsourced fact checks appear more rapidly on potentially misleading tweets. A lot was riding on the rare moderation instrument to survive Elon Musk’s takeover of the platform, especially given that election misinformation often comes from inside the house.
To get a sense of whether Community Notes delivered on its promise to “create a better-informed world,” we downloaded all of the 3,637 notes made on Nov. 5, Election Day in the United States.
We restricted our analysis to the tweets that had at least one note with more than 100 ratings as a proxy for popularity. We focused on tweets about the U.S. election (about two-thirds of the total; floods in Spain were another recurring topic) and annotated them based on whether they addressed claims that could be fact-checked (as opposed to opinions or predictions).
We were left with 166 unique tweets and 849 notes. Of these, only 55 were actually fact-checkable. Only 24 of the notes in our sample were rated helpful, fewer than 5% of the total.
At the broadest level, it’s hard to call the program a success.
Only 29% of fact-checkable tweets in our sample carry a note rated helpful. And of the helpful notes, only 67% are assigned to a fact-checkable tweet. This is not the kind of precision and recall figures that typically get a product shipped at a Big Tech platform.
The three tweets with the most notes were a Musk claim about record turnout among men, one of many tweets purporting to show presidential candidate Kamala Harris pretending to speak with a voter on the phone, and a dog pic by vice presidential candidate JD Vance. The first two tweets are somewhat fact-checkable, and only the first one is vaguely fact check-worthy.
(Screenshots/X)
The most viewed Election Day tweets to get Community Notes were a sorrier bunch still. Former President Barack Obama’s prediction that the election result would not be known overnight was marginally Community Notes-worthy, but neither this tweet asking men who voted for Harris for their pictures nor this Musk repost of a satirical video about “dropping illegal immigrants off at a rich liberal’s house” get anywhere close to deserving a fact check. Most of the related notes are best described as banal bickering.
(Screenshots/X)
The only glimmer of usefulness for Community Notes comes if you look at the three tweets whose notes received the most “helpful” ratings. Incidentally, all three were pro-Trump posts. Two originated from the Musk-aligned @DogeDesigner, who claimed tech platforms Google Search and ChatGPT were biased against Trump, while the other one fabricated a claim by Maryland Democrat Jamie Raskin.
(Screenshots/X)
Looking at the sample as a whole, the tweets with the most notes were about celebrity endorsements, which are easy to fact-check and relatively uncontroversial. These were followed by false claims about tech bias, a topic that is important but not as sensitive as reported issues at voting locations. Fully 41 notes were on spammy posts about Twitter changing its like button to a ballot box icon.
(Alexios Mantzarlis)
And then, there were plenty of checkable claims that didn’t receive any notes, including hundreds shared on X’s Election Integrity Community (see Faked Up No. 25), some of which racked up millions of views.
Zooming out, the picture doesn’t look any brighter. In the three days leading up to the election, fewer than 6% of the roughly 15,000 notes reached helpful status.
We estimate that only about 13% of all notes during this crucial period were even about the election. Top-rated public notes addressed instead whether French has a word for “toes,” football star Jason Kelce’s phone-smashing incident and former U.S. President Bill Clinton’s relationship with Monica Lewinsky.
If Community Notes had an impact on election information quality on X, it was marginal at best.
Comments