Cite as:

Chen Shen, Derrick Van Gennep, and Yaneer Bar-Yam, The IFR of the Diamond Princess has been misreported, best current value is 2.0%, New England Complex Systems Institute (August 5, 2020).

The COVID-19 cases on the Diamond Princess have served as an important case study for analysis of the outbreak. Here we show that the reported IFR and its subsequent use in the widely quoted study and meta-analysis that incorporates it have fundamental errors. This includes not updating for subsequent deaths and use of an extrapolated correction of Chinese data rather than the actual Diamond Princess data. Instead of a value of 0.6% these corrections would lead to an IFR of 2.0%, though even this number is unjustified by multiple manifestly invalid assumptions.

Recently, the US CDC updated its guidelines on epidemiological parameters of COVID-19 [1]. In this new version, the CDC replaced CFR (fatality rate among all confirmed cases) with IFR (fatality rate among all infections) as the dimension to measure severity. The CDC gave the current estimate of 0.65% IFR, based on the work of Gideon Meyerowitz-Katz and Lea Merone [2], referred to as MKM in the following.

MKM examined publications on the topic up to 16/06/2020. 24 IFR estimations are included. We have found multiple issues with these estimates. This is the first of a series of reports to clarify the IFR calculations. In this report we focus on one of the 24: Estimating the infection and case fatality ratio for COVID-19 using age-adjusted data from the outbreak on the Diamond Princess cruise ship [3], referred to as DP in the following. There are three major problems in how MKM used this study and the study itself.

  1. The 1.2% IFR from Diamond Princess data was calculated based on early data. The deaths doubled after the publication of the study.

  2. MKM did not actually use the 1.2% IFR calculated for Diamond Princess, as they claimed in their primary table on prior results, instead using an extrapolated 0.6% IFR for China based upon multiple questionable logic steps. These steps invalidate the inference and violate meta analysis principles of sampling experimental results.

  3. The Diamond Princess paper adjusted the China CFR with the Diamond Princess CFR obtained from a very small statistical sample of only 7 deaths, and has many unjustified assumptions unaddressed.

In summary, rather than a 0.6% IFR value, a value that might be used in a meta analysis from DP would be 2.0% based upon the data from the Diamond Princess. Even this value is suspect given remarkably poor statistics, and other inconsistencies in data interpretation to be detailed below. 

We map the logic steps of DP in Fig. 1. The study collected the infection data on Diamond Princess up to Feb. 20 [4,5] and the infection outcome by Mar 3. It calculated a "corrected CFR (cCFR)" of the cruise. The term "corrected CFR" as contrasted to a "naive CFR (nCFR)" indicates it is not the direct ratio of deaths to confirmed cases at a particular time point, but considers the delay in "confirmation-to-death." This adjustment is justified as nCFR tends to underestimate the severity of an epidemic (especially at the beginning) because some of the current confirmed cases will lead to deaths that are unaccounted for in the calculation of nCFR. We summarize various relevant values in MKM and DP in Table 1.

Table. 1: Values used in MKM and DP, official reports are as of Aug. 4, 2020.

DP also calculated the ratio of symptomatic to infected individuals α as 49% on the cruise based on a Japanese government report [5], However, the government report uses the term "asymptomatic" differently from the literature, and indeed differently from what would be needed to calculate an IFR. It includes in its definition of asymptomatic "(A) an individual who developed mild or unnoticeable symptoms and self-identified as asymptomatic" and "(C) an individual with recent infection who has not yet developed symptoms and was identified by test before symptom onset" [5]. (A) depends on self-reporting and includes mild symptoms, and (C) should be classified as "presymptomatic." Both mild and presymptomatic cases should be included in symptomatic cases. This means asymptomatic case counts are overestimated, hence α is underestimated. Significantly, there is no conclusion that can be drawn about α from this data except that it is the lower bound of α for the Diamond Princess, whereas the true value of alpha from this population is unknown.

DP then cites the data from [6] which indicates that the China nCFR (up to Feb.11) is 2.3%, with the 70-79 y/o group at 8.0%, and the 80+ y/o group at 14.8%. DP applied the age-stratified nCFR to the Diamond Princess population, calculating that 15.15 deaths are expected. Comparing to the 7 deaths observed by Mar 3, the authors concluded 

DiamondPrincessIFR-2.jpg

(step A1 and A2 in in Fig. 1).

In order for such a correction to be justified: (1) the proportion of over 80 y/o in the Chinese population must be equal to the proportion in the Diamond Cruise, (2) there must not be other factors that lead to differences in the age-stratified fatality rates between the two populations. 

The China study has 3% of patients ≥ 80 y/o [6], while the Diamond Princess has 9.6% of symptomatic cases (8.7% of all infections) ≥ 80 y/o [3]. The much larger proportion combined with an increasing CFR with age indicates that any assumption of corresponding fatality rates is unjustified. Given this breakdown of the assumptions, it is apparent that the cruise members over 80 y/o do not have the same vulnerability as members of the Chinese population over 80 y/o, reasonably including infirmities and disabilities that would prevent an individual from going on a cruise. An 80 y/o on a cruise, unsurprisingly, may not be representative of an 80 y/o in the general population. The discrepancy may also be due to the small sample of individuals on the cruise. So multiplying the China nCFR by 46% to obtain a 1.1% "cCFR" is incorrect. Moreover, this observation of population distinctions also invalidates the study use as a means of estimating the mortality rate of the population as a whole. 

The study then multiplies this cCFR by α to obtain the cIFR of China (step B1 and B2 in Fig. 1). Once again, this assumes the China patient cohort has a similar α to that of the cruise.  While DP failed to provide evidence for this assumption, the study it cites [6] noted 889 asymptomatic cases and 44672 confirmed symptomatic cases, indicating an α of 98%, doubling that of the cruise. The discrepancy provides evidence against the calculation in DP. Regardless of the actual value, given that the α of the cruise is underestimated by incorrect reporting of the asymptomatic cases, it is also incorrect as an estimate of α in China. As a result, the 0.5% cIFR obtained is an underestimate because both cCFR and α of China are underestimated.

Moreover, the two main quantities providing empirical evidence from which the authors constructed the calculation are both out of date. For the Diamond Princess, by Apr 14, 14 of the 696 cases died [7], doubling the deaths observed by Mar 3. For China, by July 21, 4634 of the 83707 symptomatic cases died [8], resulting in a 5.5% nCFR, more than doubling the nCFR observed by Feb. 11. We also note, the current China nCFR is very close to the cCFR, because the remaining active cases (the upper limit of potential uncounted deaths) are relatively small compared to cumulative deaths. 

In addition, DP claimed 6 out of the first 7 death cases were patients in their 70s and 1 in their 80s (Table 2 [3]), while the Japanese government reported 4 in their 80s, 2 in their 70s, and one unspecified [9]. This manifest discrepancy has no available explanation.

In Fig. 2, we plot the value of the CFR measured in three ways. The denominators are labeled (a) nCFR: all symptomatic cases, (b) CFR by outcome: all symptomatic cases that have a clinical outcome, and (c) CFR by delay: all symptomatic cases confirmed by 14 days prior. We see the three CFR lines converge after Apr. 17 to around 5.5%. Therefore, both CFRs cited in the cruise study are gross underestimations due to the use of values from early time estimates. 

According to our analysis, [3] has flaws in the empirical data it relies upon, whose definitions do not correspond to the way they are being used in calculations, and in its assumptions and conclusions. The inclusion of this study in MKM is questionable at best. Yet more strikingly, MKM uses this study not for its calculated 1.2% cIFR for the Diamond Princess (which would be appropriate for a meta-analysis of multiple empirical data sources), but rather for the extrapolated 0.6% cIFR for China. This is true even though all of the calculations in DP are based upon data for Diamond Princess. MKM actually annotated this study as "IFR was 1.3%" in Table 1 [2], but incorporated it as 0.6% (Figure 1-4). We find it all the more remarkable that despite all of these errors, the CDC quotes the 0.65% IFR solely based on this meta-analysis. We would expect a higher standard for science in general, and for an ongoing pandemic analysis in particular. We have also found there are problems in other studies included in MKM, which will be addressed in additional reports.

References

  1. COVID-19 Pandemic Planning Scenarios

  2. Meyerowitz-Katz, G., Merone, L. (2020). A systematic review and meta-analysis of published research data on COVID-19 infection-fatality rates. MedRxiv, 2020.05.03.20089854. https://doi.org/10.1101/2020.05.03.20089854

  3. Russell, T., Hellewell, J., Jarvis, C., van-Zandvoort, K., Abbott, S., Ratnayake, R., Kucharski, A. (2020). Estimating the infection and case fatality ratio for COVID-19 using age-adjusted data from the outbreak on the Diamond Princess cruise ship. Eurosurveillance, 2020.03.05.20031773. https://doi.org/10.1101/2020.03.05.20031773

  4. National Institute of Infectious Diseases. Field Briefing: Diamond Princess COVID-19 Cases.

  5. National Institute of Infectious Diseases. Field Briefing: Diamond Princess COVID-19 Cases.

  6. Wu, Z., McGoogan, J. M. (2020, April 7). Characteristics of and Important Lessons from the Coronavirus Disease 2019 (COVID-19) Outbreak in China: Summary of a Report of 72314 Cases from the Chinese Center for Disease Control and Prevention. JAMA - Journal of the American Medical Association. American Medical Association. https://doi.org/10.1001/jama.2020.2648

  7. https://www.mhlw.go.jp/stf/newpage_09997.html

  8. http://www.nhc.gov.cn/xcs/yqtb/202007/13f951072dfa4c3a8ebf396e308847d7.shtml

  9. https://www.wikiwand.com/en/COVID-19_pandemic_on_Diamond_Princess#/Demographics

Fig. 1: Logical flow chart of DP [3]. Blue and turquoise shaded squares are two empirical data values the study cites. Rounded squares are logic steps in the paper connecting from the empirical values to the conclusions. The red arrows are problematic logic steps.

Fig. 2: New cases, fatalities, and CFRs of China (7 day averages). Three ways to calculate: solid line (nCFR) is death cases divided by all symptomatic cases. Dashed line is death cases divided by all symptomatic cases that have a clinical outcome (recover or death). Dot-dashed line is death cases by all symptomatic cases confirmed 14 days ago.

NECSI on the Coronavirus

Updates:

Position Statements:

Guides to Action:

Analyses

Critiques

Innovation Ideas

Background