Ananish Chaudhuri: Claims about New Zealand saving 20,000 Covid lives are inaccurate and meaningless

In a recent article in the New Zealand Medical Journal, Michael Baker and his co-authors claim that New Zealand saved at least 20,000 lives from Covid death. This article has received a fair amount of media exposure. 

Given that this is a “viewpoint” article, I am not sure whether the paper went through peer review or not.

The article provides an overview of the progression of the Covid pandemic and various associated issues. Most parts of the article are uncontroversial as it presents a series of facts. There are bits of opinion thrown in that might raise some eyebrows such as:

As COVID-19 transitions to becoming endemic, some argue that it should be treated more like other infectious diseases. We propose the converse approach of treating other serious respiratory infections such as influenza and respiratory syncytial virus (RSV) more like COVID-19. 

It is not clear exactly what this means but it seems like Baker et al. are suggesting that we should implement lockdowns to deal with other respiratory diseases. At this point, it has become clear that Baker and his colleagues are either unable or unwilling to acknowledge the massive collateral damages of the policies they espouse. It is well-established that lockdowns have no effect on mortality. But equally, one could take the charitable view that as medical professionals it is their job to focus on diseases and advocate for stringent responses. It is up to the policy makers to figure out the optimal response bearing in mind the various trade-offs.

But where did the headline “20,000 lives saved” come from? 

As far as I can see this comes from a section in the paper’s Appendix, which deals with excess mortality around the world. The authors reproduce a chart from Our World in Data that shows excess mortality in different countries in July 2023. The authors write:

If New Zealand (resident population 5.185 million in 2022) had experienced the cumulative excess mortality of the US (3,739.3 per million) then we would have had around 19,390 excess deaths up to the end of June 2023. With the United Kingdom (UK) excess mortality (3,164.8 per million), we would have had around 16,410 excess deaths, or using the experience of Sweden (1,436.3 per million) we would have had 7,450 excess deaths. New Zealand’s excess was varying around zero in mid-2023 (122 at the time of writing). 

The claim is deeply problematic. 

First, how do we know that this excess mortality is the result of Covid? Second, what is an appropriate benchmark or baseline that this mortality is being compared against? Third, what about the age at death, knowing that the age profile of Covid deaths is heavily skewed toward the elderly? Fourth, what about the other social, economic and psychological opportunity costs of some of those policies? Fifth, it is well-known that the definition of a Covid-19 death, in many countries, did not imply that someone actually died from Covid. 

Finally, it is well-known in the public health literature that “lives saved” is not a meaningful measure. The usual measure is “quality adjusted life years” (QALY) saved. This is the measure used, for instance, by Pharmac in deciding which drugs to fund and which not. New Zealand’s current life expectancy is 82 years. This means that if you save the life of a healthy 80-year-old then you have possibly saved two years of quality-adjusted life while preventing the death of a 5-year-old implies potentially saving 77 years of quality-adjusted life. And yes, this is the exactly the type of calculation carried out by bodies like Pharmac. 

But suppose, Baker and his colleagues are correct. Even if we take their arguments at face value then based on what they write, the excess mortality is between 7,450 excess deaths (Sweden) and 19,390 deaths in the US. So rightfully the headline should be: “New Zealand may have saved between 8000 and 20,000 lives.” 

But are these the right comparisons? Should we compare New Zealand with 5 million people to the US (330 million) or the UK (68 million) with both having far greater population density (and other problems) than New Zealand? 

What if we compare with Denmark, Norway and Finland, each of which has around 5.5 million people. On July 2, 2023, Norway had 1315 excess deaths per million, Denmark 716, Finland 2164. What does this imply for excess deaths? Given that the populations are roughly equal to New Zealand, we could multiple each of these numbers by 5. At best, we can say that we have saved somewhere between 3580 (Denmark) and 10,820 (Finland) lives, a far cry from 20,000. I could only find data for Australia from May 28 when its excess death stood at 1094 per million. This would imply approximately 5,470 deaths. 

So, the “20,000 lives saves” message is numerically inaccurate and practically meaningless. 

It is possible that the message got distorted in reporting. It is possible that Baker said that we may have saved as many as 20,000 lives but this was written up as “20,000 lives saved”. But Baker should have noted that “lives saved” is a meaningless measure. It is also true that the fallacies in the argument would be obvious if journalists engaged in double-sourcing, which they are expected to do, by seeking a second opinion from an objective third-party.

Ananish Chaudhuri is Professor of Experimental Economics at the University of Auckland. Besides Auckland, he has taught at Harvard Kennedy School, Rutgers University, Washington State University and Wellesley College. This article was first published HERE