How scientists debunked one of conservation’s most influential statistics

The statistic seemed to crop up everywhere. Versions were cited at UN negotiations, on protest banners, in 186 peer-reviewed scientific papers – even by the film-maker James Cameron, while promoting his Avatar films. Exact wording varied, but the claim was this: that 80% of the world’s remaining biodiversity is protected by Indigenous peoples.

When scientists investigated its origins, however, they found nothing. In September, the scientific journal Nature reported that the much-cited claim was “a baseless statistic”, not supported by any real data, and could jeopardise the very Indigenous-led conservation efforts it was cited in support of. Indigenous communities play “essential roles” in conserving biodiversity, the comment says, but the 80% claim is simply “wrong” and risks undermining their credibility.

The carefully worded article, written by 13 authors including three Indigenous scientists, had been about five years in the making. But it raised other questions: including how a foundationless factoid got so much traction – and what other inaccuracies were circulating.

“There were policy reports using it. There were scientific reports. It was cited in more than 180 scientific publications,” says Álvaro Fernández-Llamazares, an ethnobiologist at the Universitat Autònoma de Barcelona and one of the authors of the article. It was checked as “true” by a dedicated factchecking organisation, and quoted by numerous news organisations (including the Guardian). Fernández-Llamazares stressed they did not blame those who used the figure. Instead, he said: “What we are questioning is: how can it be that this figure has gone unchallenged for so many years?”

To check the claim, the scientists searched decades of literature and citations. They did not find anything resembling an actual calculation. Instead, reports by the UN and the World Bank from the early 2000s seem to have popularised it. They in turn cited an encyclopedia article on eco-regions occupied by Indigenous peoples, and research that found some ​​Indigenous tribes in the Philippines were “maintaining over 80% of the original high-biodiversity forest cover”.

Perhaps, however, the statistic should have raised eyebrows from the outset. Despite recent advances in measurability, biodiversity as a concept is still hard to define, let alone quantify and count. Millions of species aren’t even described or their status as a species is debated. “The 80% claim is based on two assumptions: that biodiversity can be divided into countable units, and that these can be mapped spatially at the global level. Neither feat is possible,” the Nature authors wrote.

On the face of it, the biodiversity field is very numbers driven. But the look of mathematical precision can be misleading, in a field that deals with measuring under-studied species, changing ecosystems and data black spots.

“We are not honest with ourselves within our own ranks,” says Matthias Glaubrecht, a professor at the Leibniz Institute for the Analysis of Biodiversity Change in Hamburg. “Biology is a dirty science, so to speak: numbers here are an auxiliary construction to prove a case, but always accompanied by a big question mark.”

Elephants in Africa, for example, are often used as a symbol of mass extinction. Discourse around African elephants often focuses on a dramatic decline in the 20th century. Popular data platform Our World in Data reported that there were once 26 million elephants in Africa, which declined to 10 million in 1900, to half a million today. The same figures are widely used by NGOs and the press.

But 26 million elephants would mean almost one elephant for each square kilometre across the entire African continent, with its huge variations in habitat – a figure that strains plausibility.

The number originated from a PhD thesis in the early 1990s by Oxford biologist Eleanor Jane Milner-Gulland. Debates around a ban on the ivory trade were running high at the time, and Milner-Gulland tried to estimate the influence of poaching on population sizes. Because there were no robust elephant counts until well into the 1900s, she built a statistical model, taking recent counts from areas populated by elephants and multiplying them out across the continent to areas where elephants could have lived. She arrived at an estimate of between 13.5 million and 26.9 million elephants for the early 19th century.

“The assumption of the study is wrong,” says Chris Thouless, research director for Save the Elephants in Kenya: “It was written with the idea that hardly any people lived in Africa.”

Thouless says an unsurprising range would be “a few million – rather than tens of millions”. There is no doubt that elephant populations have suffered. But their decline is a more complicated story than the sudden apocalypse sometimes painted. After being approached by the Guardian about the veracity of historical elephant data, Our World in Data removed the numbers.

Related: ‘A biodiversity catastrophe’: how the world could look in 2050 – unless we act now

Statistical modelling of a world we might have lost is common in the field. But it’s tricky to do. “Historical land use reconstruction is a very messy business, especially at the global scale,” says Erle Ellis from the University of Maryland. Ellis works with these kinds of models, dating back 12,000 years. A single parameter based on an archaeological find can change an entire region. “There are lots of models – for example on habitat loss and what it does to a given species. But is there a good model that does that? I don’t think so,” Ellis says.

Despite the importance of robust data in environmental crises, calling out bad statistics is sometimes seen as an attack on conservation itself. The Nature article about the 80% was in the making for five years, one of the authors says, because the topic is so sensitive and could be abused politically. In the article, they write that “the 80% claim could undermine [more] rigorous studies – as well as effective efforts to conserve biodiversity by Indigenous peoples on the ground”. After its publication, however, the authors faced some intense criticism.

“The feedback here in Mexico is strong … is rude. Someone told me this is a call for war,” says Yesenia H Márquez, a co-author of the article and member of the expert group on Indigenous and local knowledge at the UN’s Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services (Ipbes). “But I think it’s not a problem to promote the paper,” she says. “We know our territories. We know all the biodiversity that we have.”

Tin Fischer is a data journalist based in Berlin, and author of a book on how political allegiances can change perception of data.