Fact-checking works to undercut misinformation in many countries

In the wake of the flood of misinformation that’s drowning the US, lots of organizations have turned to fact-checks. Many newsrooms set up dedicated fact-check groups, and some independent organizations were formed to provide the service. We get live fact-checking of political debates, and Facebook will now tag material it deems misinformation with links to a fact-check.

Obviously, given how many people are still afraid of COVID-19 vaccines, there are limits to how much fact-checking can accomplish. But might it be effective outside the overheated misinformation environment in the US? A new study tests out the efficacy of fact-checking in a set of countries that are both geographically and culturally diverse, and it finds that fact-checking is generally more effective at shaping public understanding than misinformation is.

Checking in with different countries

The two researchers behind the new work, Ethan Porter and Thomas Wood, identified three countries that are outside the usual group of rich, industrialized nations where most population surveys occur. These were Argentina, Nigeria, and South Africa. As a bit of a control for the typical surveys, they also ran their study in the UK. All four of these countries have professional fact-checking organizations that assisted with the work and were able to recruit 2,000 citizens for the study.

The design of the experiment was simple. The researchers created a set of misinformation. Two of the items in the set (on climate change and the pandemic) were globally relevant. But they also developed items that were tailored to the misinformation environment in each country. These include examples like the percentage of the South African’s budget that goes to salary, the public debt-to-GDP ratio in Argentina, or the levels of youth unemployment in Nigeria.

The participants were randomly assigned one of three conditions. The control group simply got an unrelated piece of misinformation. Another group got the misinformation that is widely circulated in their country as a simple, factual statement. The third group got the misinformation plus a more detailed fact-check of it.

Afterwards, participants were asked to rate their belief in the misinformation on a five-point scale, from strongly think it’s true to strongly think it’s false. In all countries but Nigeria, the same participants were contacted two weeks later to see whether the fact-checking had stuck. They were also asked answer a 10-question survey that placed people on a conservative to liberal ideological spectrum.

All told, this allowed the researchers to get at whether the fact-checking worked and whether ideological tendencies influence its effectiveness.

It works, within limits

Overall, on the researchers’ five-point scale, misinformation barely registered. That’s not much of a surprise, as it was simply presented in the form of a figure. In the real world, most misinformation comes packaged in cultural and ideological signals that increase its effectiveness. Still, fact-checking turned out to be highly effective across the different countries, wiping out the effect of misinformation and then boosting the acceptance of accurate information by a half-point on the five-point scale.

Its effectiveness varied considerably based on topic, though. It was highly effective at correcting misapprehensions about the total malaria deaths in Nigeria, for example, but far less so at ensuring Argentinians knew their public debt-to-GDP ratio. But every single fact-check was effective to one degree or another. And, when checked two weeks after, nine of the 15 fact-checks had stuck with people.

Boris who?

Ideology did seem to play a role everywhere, eliminating the effectiveness of fact-checks on a few specific issues, like how much funding Boris Johnson has pledged for education in the UK. But these instances averaged out, and fact-checks overall improved the accuracy of people’s beliefs no matter where they were on the ideological spectrum.

Given the vast differences in economy, culture, and politics among these countries, Porter and Wood conclude that fact-checking is generally effective. That doesn’t mean it will work on everyone or be equally effective for all topics. But at least, on some level, people are willing to accept the details of a fact-check.

That said, the researchers view their work mostly as an indication that this is something that is worth studying in different countries. They acknowledge that their work had some limitations, like its simplistic presentation of the misinformation to the participants. It also doesn’t get at the social factors that alter the effectiveness of fact-checking on different issues. For example, they note that studies in the US have identified demographic groups and cognitive habits that make people more susceptible to misinformation and therefore potentially more resistant to fact-checking. Those sorts of things will be important to study in other cultures.

But of course, before doing those studies, it’s critical we know there’s something to study there. And this work suggests there is.

PNAS, 2021. DOI: 10.1073/pnas.2104235118  (About DOIs).