Uncertainty in computerised analysis of maternal deaths
Authors: Sarika Chaturvedi, Bharat Randive
Journal: The Lancet Global Health (2021)

Maternal mortality continues to be a major concern for global public health despite commendable progress in reducing maternal deaths during the Millennium Development Goal era. Progress in reducing maternal mortality is largely dependent on the pace of achievements in the low-resource settings where the burden of maternal mortality is concentrated. To test the effectiveness of corrective actions, precise measurement of the appropriate indicator is essential. Vital registration systems in most low-resource settings are poor.1Mikkelsen L Phillips DE AbouZahr C et al.A global assessment of civil registration and vital statistics systems: monitoring data quality and progress.Lancet. 2015; 386: 1395-1406Summary Full Text Full Text PDF PubMed Scopus (246) Google Scholar Alternatives to overcome the inadequacy in vital registration of deaths are physician review and verbal autopsy. Physician review can be costly, time consuming, inconsistent across contexts and practices, and unfeasible in settings with shortages in human resources for health. Verbal autopsies are relatively rapid and cost effective. Verbal autopsies involve complex skip patterns (ie, question series associated with conditional responses), and hence are suitable for automatisation and portable data collection (eg, by mobile phones). Such automatisation can also improve consistency. Conversely, an advantage of physician review is the addition of context-specific insights, which can aid quality improvement and provide input for developing models for verbal autopsies. In this context, the analysis by Annet M Aukes and colleagues, assessing agreement between physician review and software analysis of verbal autopsies (with the WHO interpreting verbal autopsies [InterVA] model, versions 4 and 5) for assigning cause of maternal deaths, is highly relevant.2Aukes AM Arion K Bone J et al.Causes and circumstances of maternal death: a secondary analysis of the Community-Level Interventions for Pre-eclampsia (CLIP) trials cohort.Lancet Glob Health. 2021; (published online July 29.)https://doi.org/10.1016/S2214-109X(21)00263-1Summary Full Text Full Text PDF PubMed Scopus (1) Google Scholar They use secondary data from the Community-Level Interventions for Pre-eclampsia (CLIP) cluster randomised trials in India, Pakistan, and Mozambique, allowing generalisability to settings where automatised verbal autopsies are required. Notably, InterVA software is open source and high speed—InterVA-5 can process 100 verbal autopsy records in 1 min. In addition to causes of death, the paper also compares physician's review with InterVA-5 for agreement on circumstances of mortality categories (COMCAT). The importance of distinguishing the circumstances around a death that might not be reflected in medical causes has long been recognised for policy action; by adding this element to the comparison the paper makes an important contribution to generating policy-relevant evidence. Overall, the study offers confidence in the InterVA tools, particularly InterVA-4, for public health use. It reports moderate concurrence between physician review and InterVA (higher with InterVA-4 than InterVA-5) for broad classification of cause of maternal death (direct maternal cause, indirect maternal cause, or undetermined). In terms of specific causes of death, the top three causes were the same by all three methods—obstetric haemorrhage, non-obstetric complications, and hypertensive disorders. However, while the contribution of hypertensive disorders as causes of death did not change by the method used, there were considerable differences between methods in attribution to obstetric haemorrhage. This outcome is noteworthy, given that obstetric haemorrhage is among the top causes of maternal death in many of the low-resource settings that might use InterVA software.3Nathan LM An overview of obstetric hemorrhage.Semin Perinatol. 2019; 43: 2-4Crossref PubMed Scopus (9) Google Scholar As such, the authors suggest the need for improvement in the next iteration of InterVA. Aukes and colleagues' paper is a case of focusing on the detail in an attempt to provide a broader view; they apply the tools at the individual level and generalise its use to the population level. Difficulties in evaluating the assignment of cause of death are well known and owe to the absence of a precise comparator. InterVA has been validated against two standards—physician review and hospital medical records—however, both are not true gold standards. In 2020, Menéndez and colleagues4Menéndez C Quintó L Castillo P et al.Limitations to current methods to estimate cause of death: a validation study of a verbal autopsy model.Gates Open Res. 2020; 4: 55Crossref PubMed Scopus (3) Google Scholar evaluated InterVA-4-coded cause of death against complete diagnostic autopsy, which can be considered the gold standard. They found that, of 26 maternal deaths attributed to eclampsia by InterVA, only four were assigned as eclampsia-related deaths on complete diagnostic autopsy. Such considerable overestimation of eclampsia as a cause of death by InterVA compared with complete diagnostic autopsy has also been reported in the past, with this type of misdiagnosis common for deaths from infectious diseases.5Ordi J Ismail MR Carrilho C et al.Clinico-pathological discrepancies in the diagnosis of causes of maternal death in sub-Saharan Africa: retrospective analysis.PLoS Med. 2009; 6e1000036Crossref PubMed Scopus (51) Google Scholar This inaccurate attribution has important implications for eclampsia prevention programmes (including the CLIP trial), which might appear to fail due to misdiagnosis of eclampsia mortality. Aukes and colleagues' finding that InterVA-5 is not better than its precursor InterVA-4 highlights the importance of documenting experience of use for modifications in the software life cycle. However, this body of research is growing, and early evaluations of InterVA-5 report it to be highly concordant with InterVA-4 (concordance correlation coefficient 0·91 [95% CI 0·86–0·96]).6Byass P Hussain-Alkhateeb L D'Ambruoso L et al.An integrated approach to processing WHO-2016 verbal autopsy data: the InterVA-5 model.BMC Med. 2019; 17: 102Crossref PubMed Scopus (16) Google Scholar Although Aukes and colleagues' findings highlight the potential use of InterVA-4 to process data in civil registration systems cheaply, speedily, and consistently, the usefulness of InterVA for global maternal health depends largely on the quality of input data and appropriate interpretation. While InterVA can be used for tracking regional mortality patterns and locally tailored decision making, the search for improved tools should be continued. We declare no competing interests. Causes and circumstances of maternal death: a secondary analysis of the Community-Level Interventions for Pre-eclampsia (CLIP) trials cohortOur findings indicate that InterVA-5 is less accurate than InterVA-4 at ascertaining causes and circumstances of maternal death, when compared with physician review. Our results suggest a need to improve the next iteration of InterVA, and for researchers and clinicians to preferentially use InterVA-4 when recording maternal deaths. Full-Text PDF Open Access