# Accuracy of pulse CO-oximetry to evaluate blood carboxyhemoglobin level: a systematic review and meta-analysis of diagnostic test accuracy studies

Mathilde Papin, Chloé Latour, Brice Leclère, François Javaudin

► To cite this version:

Mathilde Papin, Chloé Latour, Brice Leclère, François Javaudin. Accuracy of pulse CO-oximetry to evaluate blood carboxyhemoglobin level: a systematic review and meta-analysis of diagnostic test accuracy studies. European Journal of Emergency Medicine, 2023, 30 (4), pp.233-243.
10.1097/MEJ.0000000000001043. hal-04809073

HAL Id: hal-04809073

https://nantes-universite.hal.science/hal-04809073v1

Submitted on 28 Nov 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Accuracy of pulse CO-oximetry to evaluate blood carboxyhemoglobin level: a systematic review and meta-analysis of diagnostic test accuracy studies

Mathilde Papinª, Chloé Latourᵇ, Brice Leclèreᶜ,ᵈ and François Javaudinª,ᵈ

Carbon monoxide (CO) poisoning is one of the most common causes of poisoning death and its diagnosis requires an elevated carboxyhemoglobin (COHb) level. Noninvasive CO saturation by pulse oximetry (SpCO) has been available since 2005 and has the advantage of being portable and easy to use, but its accuracy in determining blood COHb level is controversial. To evaluate the accuracy of SpCO (index test) to estimate COHb (reference test). Systematic review and meta-analysis of diagnostic test accuracy (DTA) studies. Four electronic databases were searched (Medline, Embase, Cochrane Central Register of Controlled Trials, and OpenGrey) on 2 August 2022. All studies of all designs published since the 2000s evaluating the accuracy and reliability of SpCO measurement compared to blood COHb levels in human volunteers or ill patients, including children, were included. The primary outcome was to assess the diagnostic accuracy of SpCO for estimating COHb by blood sampling by modeling receiver operating characteristic (ROC) curves and calculating sensitivity and specificity (primary measures). The secondary measures were to calculate the limits of agreement (LOA) and the mean bias. This systematic review was conducted according to the Preferred Reporting Items for a Systematic Review and Meta-analysis-DTA 2018 guidelines and has been registered on International Prospective Register of Systematic Reviews (PROSPERO, CRD42020177940). The risk of bias was evaluated using the Quality Assessment of Diagnostic Accuracy Studies-2 tool. Twenty-one studies were eligible for the systematic

review; 11 could be included for the quantitative analysis of the primary measures and 18 for the secondary measures. No publication bias was found. The area under the summary ROC curve was equal to 86%. The mean sensitivity and specificity were 0.77, 95% confidence interval (CI, 0.66–0.85) and 0.83, 95% CI (0.74–0.89), respectively (2089 subjects and 3381 observations). The mean bias was 0.75% and the LOA was –7.08% to 8.57%, 95% CI (–8.89 to 10.38) (2794 subjects and 4646 observations). Noninvasive measurement of COHb (SpCO) using current pulse CO oximeters do not seem to be highly accurate to estimate blood COHb (moderate sensitivity and specificity, large LOA). They should probably not be used to confirm (rule-in) or exclude (rule-out) CO poisoning with certainty. *European Journal of Emergency Medicine* 30: 233–243 Copyright © 2023 The Author(s). Published by Wolters Kluwer Health, Inc.

European Journal of Emergency Medicine 2023, 30:233–243

Keywords: carbon monoxide poisoning, carboxyhemoglobin, carbon monoxide saturation, CO oximeters, pulse oximetry

ªEmergency Department, Nantes University Hospital, Nantes, ᵇDepartment of Medicine, Hospital of Pontivy, Pontivy, ᵈDepartment of Medical Evaluation and Epidemiology, Nantes University Hospital and ᵈCibles et Médicaments des Infections et de l'Immunité, UR1155 IICiMed, Nantes University, Nantes, France

Correspondence to François Javaudin, MD, PhD, Cibles et Médicaments des Infections et de l'Immunité, IICiMed, 10 UR1155, Nantes Université, Institut de Recherche en Santé 2, 22 Boulevard Bénoni Goullin, 44200 Nantes, France
Tel: +33 675 691 630; e-mail: francois.javaudin@chu-nantes.fr

Received 15 March 2023 Accepted 24 April 2023.

## Introduction

Carbon monoxide (CO) poisoning is one of the most common causes of poisoning death and is responsible for approximately 50 000 emergency department (ED) visits per year in the USA [1]. In 2014 in the USA, 1319 deaths from accidental or intentional CO poisoning were reported, and a decline in annual cases has been seen since 1999 [2]. Worldwide, the estimated incidence of

CO poisoning is 137 cases per million and 4.6 deaths per million [3]. However, the numbers of intoxication cases are most likely underestimated [4,5]. Carbon monoxide is an odorless and colorless gas, and the symptoms of CO poisoning are nonspecific: headache, dizziness, nausea, vomiting, fatigue, and loss of consciousness. The main sources of intoxication are fires, heating defects, and cars [6]. The diagnosis of CO poisoning requires a history of recent CO exposure, the presence of symptoms, and an elevated carboxyhemoglobin (COHb) level [7]. Either arterial or venous blood may be used to measure the COHb level [7–9]. Since 2005, noninvasive pulse CO oximeters have been available and approved by the Food and Drug Administration (FDA). They can quickly estimate COHb using readings at eight wavelengths of light at the fingertip (SpCO, CO saturation by pulse oximetry)

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's website (www.euro-emergencymed.com).

This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial-No Derivatives License 4.0 (CCBY-NC-ND), where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially without permission from the journal.
[10,11]. As early as 2006, protocols were published on the use of noninvasive pulse CO oximeters for the triage of the victims of suspected CO poisoning directly on the scene by first responders [12]. Actually, CO oximeters are used daily by thousands of fire and emergency medical services around the world [13]. Nevertheless, the results of studies evaluating the accuracy of SpCO in assessing COHb were heterogeneous and brought into question its use in clinical practice [7,11]. Some experts do not recommend its use for the diagnosis of patients suspected of having CO poisoning in the ED [10].

The aim of this systematic review and meta-analysis was to evaluate the accuracy of SpCO (index test) to estimate COHb in any population.

## Methods

This systematic review was conducted according to the Preferred Reporting Items for a Systematic Review and Meta-analysis-DTA 2018 guidelines [14] and has been registered on International Prospective Register of Systematic Reviews (PROSPERO, CRD42020177940).

## Population

We included studies evaluating the accuracy and reliability of multiple wavelength pulse CO-oximeter (SpCO) measurements (index test) in human volunteers or ill patients, including children, compared to blood COHb levels (reference test). Studies of all designs published after 2000 were eligible, without language restrictions. Case reports were not included.

## Search strategy, information sources, and study selection

Two authors (F.J. and C.L.) independently screened titles, abstracts, and full texts, following the inclusion criteria. The discrepancies (n = 6) were resolved after a discussion between the two authors (and if necessary, the intervention of a third author, M.P.; n = 0). We performed searches of the following databases on 2 August 2022: Medline (via Pubmed), Embase, and Cochrane Central Register of Controlled Trials (CENTRAL) on the Cochrane Library. We also searched the grey literature via OpenGrey. Our search algorithms for different databases included the following terms: ('oximetry', 'pulse oximetry', 'CO oximeter', 'CO oximetry', 'pulse CO oximetry', 'noninvasive') and ('carboxyhemoglobin', 'COHb'). Quantitative data from the studies were independently extracted by two authors (M.P. and F.J.).

## Quality assessment

The risk of bias was independently evaluated by two authors (M.P. and C.L.) using the Quality Assessment of Diagnostic Accuracy Studies-2 tool [15]. The assessed risks of bias were selection bias, studied test use, reference test use, and timing between the tests. Each of these items allowed the study to be classified into different risk

categories (low, high, or unclear). In cases of discordance, the classification was discussed between the authors. All discrepancies were resolved by consensus (n = 2).

## Study outcomes

The primary outcome was to estimate the diagnostic accuracy by modeling receiver operating characteristic (ROC) curves and measuring sensitivity (Se) and specificity (Sp) of SpCO for estimating COHb by blood sampling (primary measures). The secondary measures were to report the limits of agreement (LOA), the mean bias, and to perform subgroup analyses: according to the SpCO device used, and among subjects with suspected CO poisoning.

## Data analysis

The analyses were performed by a biostatistician (B.L.) using R software version 4.0.3 (The R Foundation, Boston, Massachusetts, USA) (2020-10-10). The meta-analysis was performed using the mada package version 0.5.10 (mada: Meta-Analysis of Diagnostic Accuracy, https://cran.r-project.org/web/packages/mada), focusing on Se and Sp measures. The mean Se and Sp, and their 95% confidence regions were estimated using a bivariate linear mixed model that was proposed in 2005 by Reitsma et al. [16]. In the absence of a definition of the threshold of positivity, it was set at ≥10% (SpCO and COHb) when individual data were available. Summary ROC curves (SROC) were estimated using three methods: Rutter and Gatsonis [17], Moses et al. [18], and Rücker and Schumacher [19]. LOA calculations were based on the method described by Tipton and Shuster [20]. The analysis of publication bias was performed using a Deeks' funnel plot and its significance was tested using the associated test. This method is recommended for the evaluation of diagnostic studies [21].

## Results

### Study selection

Of the 293 reports identified from databases and 6 from citation searching and websites, 21 studies were included in the systematic review and 11 in the meta-analysis for the primary measures (absence of exploitable quantitative data, n = 10), and 18 for the secondary measures (absence of exploitable quantitative data, n = 3) (Fig. 1).

### Study characteristics

Of the 21 articles included [22–42], 5 studies were experimental and 16 observational (Table 1). The experimental studies consisted of breathing CO in healthy volunteers and performing several measurements of SpCO and COHb. Among the observational studies, 10 were conducted in the ED and 2 in a hyperbaric center. Three different devices were used to measure SpCO: Rad-57 (Masimo, Irvine, California, USA) in 14 studies, Radical-7 (Masimo, Irvine) in 6 studies, and V-Spec Monitoring
<figure><img src="image_2.png" /><figcaption>Fig. 1</figcaption></figure>

Flow diagram of study selection. COHb, carboxyhemoglobin; SpCO, pulse carbon monoxide oximetry.

System (Senspec, Rostock, Germany) used once, in an experimental study.

## Risk of bias and concerns

The patient selection was the principal source of bias and concerns regarding applicability (Table 2). Indeed, the index test was mostly well documented such as the reference one. Besides, the concerns regarding applicability were low because the threshold was predefined, and the biological result was not subject to interpretation. However, the patient selection was less clear in some of the studies. We found that in 33% of the studies, patient selection could have introduced a high risk of bias, and in 33%, the risk of bias induced by patient selection is doubtful. Obviously, this is leading to an increase in concerns regarding the applicability of the patient selection. To a lesser extent, the flow and timing were also unclear, with 28% of studies with a high risk of bias induced by the flow and timing design.

Masimo was involved in the funding or loan of materials in 11 of 20 of the studies evaluating their devices (RAD-57 and Radical-7), and Senspec also provided materials for the study evaluating the V-Spec Monitoring System.

## Outcomes of individual studies

The observed sensitivity values were very different between studies, ranging from 48% for Touger *et al.* [38] to 100% for Piatkowski *et al.* [33], Roth *et al.* in non-smokers subgroup [34], and Sebbanne *et al.* in smokers subgroup [36]. The same finding was observed for specificity ranging from 51% for Feiner *et al.* when the COHb threshold was ≥5% [27] to 99% for Touger *et al.* (threshold ≥15%) [38]. The highest area under the ROC curve was 99% (Feiner *et al.*) when a COHb threshold ≥5% was used to define positive cases [27]. The LOA varied from ±3% for Kulcke *et al.* [31] to ±15% (calculated on individual data) for Zorbalar *et al* [42].

## Primary measures

A total of 11 studies were included to estimate SROC and mean Se and Sp (10 studies without exploitable data for the primary measures). Overall, 2089 subjects and 3381 observations were analyzed (i.e. coupled measurement of SpCO and COHb). The mean Se and Sp were 0.77, 95% CI (0.66–0.85) and 0.83, 95% CI (0.74–0.89), respectively (Fig. 2). The area under the SROC curve, estimated using the Rutter and Gatsonis method, was equal to 86%.
<table><caption>Table 1 Main characteristics of the studies included in the systematic review (n=21)</caption><thead><tr><th>Study</th><th>Design</th><th>Clinical setting</th><th>Number of subjects</th><th>Number of observations</th><th>Participant characteristics</th><th>SpCO device (index test)</th><th>COP SpCO threshold</th><th>COHB measurement (reference test), sample size, and devices</th><th>COP COHb threshold</th><th>Funding and COI</th><th>Inclusion in the meta-analysis for primary (SROC, Se, Sp) or secondary (LOA, mean bias) measures</th></tr></thead><tbody><tr><td>Barker et al. (2006) [22]</td><td>Experimental</td><td>NA</td><td>10</td><td>530</td><td>Healthy adults</td><td>RAD-57 (Masimo)</td><td>NA</td><td>Arterial blood ABL730 (Radiometer, København, Denmark) and OSM-3s (Radiometer)</td><td>NA</td><td>Masimo and Department of Anesthesiology, University of Arizona College of Medicine</td><td>Secondary</td></tr><tr><td>Bidstrup et al. (2021) [23]</td><td>Observational</td><td>Hyperbaric center</td><td>10</td><td>50</td><td>COP adults treated with HBO<sub>2</sub></td><td>Radical-7 (Masimo)</td><td>NA</td><td>Arterial or venous blood ABL800 FLEX (Radiometer)</td><td>NA</td><td>Loan of equipment (device, sensors, computer) by Masimo</td><td>Secondary</td></tr><tr><td>Caboot et al. (2012) [24]</td><td>Experimental</td><td>Comprehensive sickle cell center</td><td>50</td><td>50</td><td>Children with sickle cell disease</td><td>Radical-7 (Masimo)</td><td>NA</td><td>Arterial or venous blood RapidLab 1265 (Siemens, Huntsville, North Carolina, USA)</td><td>NA</td><td>Supported by the National Heart, Lung, and Blood Institute, and the National Center for Research Resources</td><td>Secondary</td></tr><tr><td>Cardwell et al. (2012) [25]</td><td>Experimental</td><td>Operating room</td><td>203</td><td>203</td><td>Preanesthetic children</td><td>Radical-7 (Masimo)</td><td>NA</td><td>Venous blood RapidLab C1265 (Siemens)</td><td>NA</td><td>Supported by NIH/NCRR Colorado CTSI</td><td>Secondary</td></tr><tr><td>Coulange et al. (2008) [26]</td><td>Experimental</td><td>ED</td><td>12</td><td>12</td><td>Suspected COP non-smoking patients</td><td>RAD-57 (Masimo)</td><td>≥10% (individual data)</td><td>Venous blood IL 682 CO-oximeter (Instrumentation Laboratory, Barcelona, Spain)</td><td>≥10% (individual data)</td><td>None</td><td>Primary Secondary</td></tr><tr><td>Feiner et al. (2013) [27]</td><td>Experimental</td><td>NA</td><td>12</td><td>283</td><td>Healthy nonsmoking adults breathing CO gas to produce a target %COHb level of ≥10% to 12%</td><td>Radical-7 (Masimo)</td><td>≥6.6%</td><td>Arterial blood ABL800 FLEX (Radiometer)</td><td>≥10%</td><td>Masimo</td><td>Primary Secondary</td></tr><tr><td>Kaya et al. (2018) [28]</td><td>Experimental</td><td>ED</td><td>213</td><td>213</td><td>Suspected COP adults</td><td>RAD-57 (Masimo)</td><td>≥23%</td><td>Arterial or venous blood ABL700 (Radiometer)</td><td>≥23%</td><td>None</td><td>Secondary</td></tr><tr><td>Kot et al. (2008) [29]</td><td>Experimental</td><td>Hyperbaric center</td><td>49</td><td>49</td><td>Suspected COP patients</td><td>RAD-57 (Masimo)</td><td>≥20%</td><td>Capillary blood CO-xyliite AVL 9.2 (AVL Biomedical Instrument, Basle, Switzerland)</td><td>≥20%</td><td>None</td><td>Primary Secondary</td></tr><tr><td>Koyuncu et al. (2020) [30]</td><td>Experimental</td><td>ED</td><td>106</td><td>106</td><td>Suspected COP adults</td><td>RAD-57 (Masimo)</td><td>NA</td><td>Arterial blood</td><td>>5% for non-smokers</td><td>None</td><td>Secondary</td></tr></tbody></table>

(Continued)
<table><caption>**Table 1**
(Continued)</caption><thead><tr><th>Study</th><th>Design</th><th>Clinical setting</th><th>Number of subjects</th><th>Number of observations</th><th>Participant characteristics</th><th>SpCO device (index test)</th><th>COP SpCO threshold</th><th>COHb measurement (reference test), sample size, and devices</th><th>COP COHb threshold</th><th>Funding and COI</th><th>Inclusion in the meta-analysis for primary (SROC, Se, Sp) or secondary (LOA, mean bias) measures</th></tr></thead><tbody><tr><td>Kulcke et al. (2016) [31]</td><td>Experimental</td><td>NA</td><td>6</td><td>888</td><td>Healthy nonsmoking adults</td><td>V-Spec (Senspec)</td><td>≥8.9%</td><td>Arterial blood ABL90 FLEX (Radiometer) or OSM3, (Radiometer)</td><td>≥10%</td><td>University of California, San Francisco, Department of Anesthesia and Perioperative Care. Oximeter and probes were supplied by Senspec</td><td>Primary Secondary</td></tr><tr><td>Nagano et al. (2008) [32]</td><td>Experimental</td><td>NA</td><td>10</td><td>10</td><td>Healthy smoking adults</td><td>RAD-57 (Masimo)</td><td>NA</td><td>Sample blood (not specified)</td><td>NA</td><td>None</td><td>Absence of exploitable data for the meta-analysis</td></tr><tr><td>Piatkowski et al. (2009) [33]</td><td>Observational</td><td>Burn center</td><td>20</td><td>20</td><td>Suspected COP patients</td><td>RAD-57 (Masimo)</td><td>≥10% (individual data)</td><td>Venous blood ABL700 (Radiometer)</td><td>≥10% (individual data)</td><td>Pulse CO-oximeter device was supplied by Masimo</td><td>Primary Secondary</td></tr><tr><td>Roth et al. (2011) [34]</td><td>Observational</td><td>ED</td><td>1578</td><td>1578</td><td>All ED patients</td><td>Radical-7 (Masimo)</td><td>≥6.6%</td><td>Arterial or venous blood ABL700 (Radiometer)</td><td>>6.6%</td><td>Masimo</td><td>Primary Secondary</td></tr><tr><td>Ruppel et al. (2011) [35]</td><td>Experimental</td><td>Pulmonary function laboratory</td><td>139</td><td>139</td><td>Patients referred for DLCO and arterial blood gas measurement.</td><td>RAD-57 (Masimo)</td><td>NA</td><td>Arterial blood RapidPoint 405 (Siemens)</td><td>NA</td><td>None</td><td>Secondary</td></tr><tr><td>Sebbane et al. (2013) [36]</td><td>Experimental</td><td>ED</td><td>93</td><td>93</td><td>Suspected COP adults</td><td>RAD-57 (Masimo)</td><td>≥9%</td><td>Venous blood IL 682 CO-oximeter (Instrumentation Laboratory, Milan, Italy)</td><td>>5% for non-smokers</td><td>None</td><td>Primary Secondary</td></tr><tr><td>Suner et al. (2008) [37]</td><td>Experimental</td><td>ED</td><td>64</td><td>64</td><td>All ED patients ≥17 years old</td><td>RAD-57 (Masimo)</td><td>≥9% for non-smokers</td><td>Venous blood</td><td>≥9% for non-smokers</td><td>Masimo loaned the devices and supported the data collection by providing funds for research</td><td>Primary</td></tr><tr><td>Touger et al. (2010) [38]</td><td>Experimental</td><td>ED</td><td>120</td><td>120</td><td>Suspected COP patients</td><td>RAD-57 (Masimo)</td><td>≥15%</td><td>Arterial or venous blood RapidLab 1200 (Siemens)</td><td>≥13% for smokers</td><td>assistants</td><td>Primary Secondary</td></tr><tr><td>Villalba et al. (2019) [39]</td><td>Experimental</td><td>ED</td><td>126</td><td>126</td><td>Suspected COP patients with SpCO ≥10%</td><td>Radical-7 (Masimo)</td><td>≥10%</td><td>Venous blood XN9000 hematology analyzer (Sysmex America, Lincolnshire, Illinois, USA)</td><td>≥15%</td><td>Devices and training were supplied by Masimo</td><td>Primary Secondary</td></tr></tbody></table>

(Continued)
<table><caption>**Table 1**
(Continued)</caption><thead><tr><th>Study</th><th>Design</th><th>Clinical setting</th><th>Number of subjects</th><th>Number of observations</th><th>Participant characteristics</th><th>SpCO device (index test)</th><th>COP SpCO threshold</th><th>COHb measurement (reference test), sample size, and devices</th><th>COP COHb threshold</th><th>Funding and COI</th><th>Inclusion in the meta-analysis for primary (SROC, Se, Sp) or secondary (LOA, mean bias) measures</th></tr></thead><tbody><tr><td>Weaver et al. (2013) [40]</td><td>Observational</td><td>ED</td><td>1363</td><td>1363</td><td>All ED patients</td><td>RAD-57 (Masimo)</td><td>>6%</td><td>Arterial or venous blood ABL825 (Radiometer)</td><td>>6%</td><td>Centers for Disease Control and SciMetrika. Masimo provided the oximeters and educated the research staff</td><td>Absence of exploitable data for the meta-analysis</td></tr><tr><td>Zaouter et al. (2012) [41]</td><td>Experimental</td><td>NA</td><td>9</td><td>148</td><td>Healthy nonsmoking adults</td><td>RAD-57 (Masimo)</td><td>≥10%</td><td>Venous blood ABL80 FLEX (Radiometer)</td><td>≥10%</td><td>COI: the authors have disclosed relationships with SciMetrika and Masimo</td><td>Primary Secondary</td></tr><tr><td>Zorbalar et al. (2014) [42]</td><td>Observational</td><td>ED</td><td>38</td><td>38</td><td>Headache patients with SpCO ≥10% and ≥14 years old</td><td>RAD-57 (Masimo)</td><td>>10%</td><td>Venous blood GEM Premier 3000-GEM OPL CO-Oximeter (Instrumentation Laboratory)</td><td>>10%</td><td>Masimo loaned the devices and sensors</td><td>Secondary</td></tr></tbody></table>

COHb, carboxyhemoglobin; COI, conflicts of interest; COP, carbon monoxide poisoning; DLCO, diffusing capacity of the lungs for carbon monoxide; ED, emergency department; HBO₂, Hyperbaric hyperbaric oxygen; LOA, limits of agreement; NA, not applicable; ROC, receiver operating characteristic; Se, sensitivity; Sp, specificity; SpCO, pulse carbon monoxide oximetry; SROC, summary ROC curves.
<table><caption>Table 2. Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2)</caption><thead><tr><th rowspan="2">Study</th><th colspan="4">Risk of bias</th><th colspan="3">Concerns regarding applicability</th></tr><tr><th>Patient selection</th><th>Index test</th><th>Reference standard</th><th>Flow and timing</th><th>Patient selection</th><th>Index test</th><th>Reference standard</th></tr></thead><tbody><tr><td>Barker et al. 2006 [22]</td><td>?</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>Bidstrup et al. 2021 [23]</td><td>-</td><td>+</td><td>+</td><td>+</td><td>-</td><td>+</td><td>+</td></tr><tr><td>Caboot et al. 2012 [24]</td><td>-</td><td>+</td><td>+</td><td>+</td><td>-</td><td>+</td><td>+</td></tr><tr><td>Cardwell et al. 2012 [25]</td><td>?</td><td>+</td><td>?</td><td>-</td><td>-</td><td>-</td><td>+</td></tr><tr><td>Coulange et al. 2008 [26]</td><td>?</td><td>+</td><td>+</td><td>?</td><td>?</td><td>+</td><td>+</td></tr><tr><td>Feiner et al. 2013 [27]</td><td>-</td><td>+</td><td>+</td><td>+</td><td>?</td><td>+</td><td>+</td></tr><tr><td>Kaya et al. 2018 [28]</td><td>+</td><td>?</td><td>+</td><td>-</td><td>+</td><td>?</td><td>+</td></tr><tr><td>Kot et al. 2008 [29]</td><td>?</td><td>?</td><td>+</td><td>?</td><td>?</td><td>?</td><td>+</td></tr><tr><td>Koyuncu et al. 2020 [30]</td><td>-</td><td>?</td><td>+</td><td>-</td><td>+</td><td>+</td><td>+</td></tr><tr><td>Kulcke et al. 2016 [31]</td><td>-</td><td>+</td><td>+</td><td>+</td><td>?</td><td>-</td><td>+</td></tr><tr><td>Nagano et al. 2008 [32]</td><td>-</td><td>?</td><td>+</td><td>+</td><td>-</td><td>?</td><td>?</td></tr><tr><td>Piatkowski et al. 2009 [33]</td><td>?</td><td>?</td><td>+</td><td>?</td><td>?</td><td>+</td><td>+</td></tr><tr><td>Roth et al. 2011 [34]</td><td>+</td><td>+</td><td>+</td><td>-</td><td>+</td><td>+</td><td>+</td></tr><tr><td>Ruppel et al. 2011 [35]</td><td>+</td><td>?</td><td>+</td><td>+</td><td>?</td><td>+</td><td>+</td></tr><tr><td>Sebbane et al. 2013 [36]</td><td>+</td><td>+</td><td>+</td><td>?</td><td>+</td><td>+</td><td>+</td></tr><tr><td>Suner et al. 2008 [37]</td><td>?</td><td>+</td><td>?</td><td>-</td><td>+</td><td>+</td><td>+</td></tr><tr><td>Touger et al. 2010 [38]</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>Villalba et al. 2019 [39]</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td><td>+</td></tr><tr><td>Weaver et al. 2013 [40]</td><td>?</td><td>+</td><td>+</td><td>+</td><td>?</td><td>+</td><td>+</td></tr><tr><td>Zaouter et al. 2012 [41]</td><td>-</td><td>+</td><td>+</td><td>+</td><td>-</td><td>+</td><td>+</td></tr><tr><td>Zorbalar et al. 2014 [42]</td><td>+</td><td>+</td><td>+</td><td>-</td><td>+</td><td>+</td><td>+</td></tr></tbody></table>

Risks and concerns: Low (+), High (-), Unclear (?)

To achieve a Se of 95% the Sp was between 45 and 50%, depending on the method used. Conversely, for a Sp of 95%, the Se was between 42 and 55% (Fig. 3).

Although no publication bias was statistically found, with an estimated bias of -6.37 (not significantly different from zero, P = 0.22), there was an asymmetry on the funnel plot concerning only studies with small sample sizes (Fig. 4).

## Secondary measures

A total of 18 studies were included to estimate the LOA (3 studies without exploitable data for the LOA). Overall, 2794 subjects and 4646 observations were analyzed. The mean bias was 0.75%, 95% CI (-6.26 to 7.75) and the LOA was -7.08% to 8.57%, 95% CI (-8.89 to 10.38) (Fig. 5).

When the device used was the Rad-57, the mean Se and Sp were 0.70, 95% CI (0.56-0.81) and 0.88, 95% CI
<figure><img src="image_3.png" /><figcaption>Fig. 2</figcaption></figure>

Sensitivities and specificities of the 11 included studies.

<figure><img src="image_4.png" /><figcaption>Fig. 3</figcaption></figure>

Summary ROC curves from the 11 included studies. ROC, receiver operating characteristic.

(0.77–0.94) (Figure E1 in the online data supplement, Supplemental Digital Content 1, http://links.lww.com/EJEM/A377), respectively; and the LOA was –7.83% to 8.73%, 95% CI (–10.09 to 10.99) (mean bias, 0.45%; Figure E2 in the online data supplement, Supplemental Digital Content 1, http://links.lww.com/EJEM/A377). The mean Se and Sp were 0.74, 95% CI (0.59–0.85) and 0.79, 95% CI (0.73–0.85) when using the Radical-7 device (Figure E3 in the online data supplement, Supplemental Digital Content 1, http://links.lww.com/EJEM/A377), respectively; and the LOA was –5.89% to 8.45%, 95% CI (–9.74 to 12.30) (mean bias, 1.28%; Figure E4 in the online data supplement, Supplemental Digital Content 1, http://links.lww.com/EJEM/A377). Among patients with suspected CO poisoning the mean Se and Sp was 0.70, 95% CI (0.56–0.81) and 0.88, 95% CI (0.77–0.94) (Figure E5 in the online data supplement, Supplemental Digital Content 1, http://links.lww.com/EJEM/A377), respectively; and the LOA was –8.11% to 8.59%, 95% CI (–10.74 to

12.23) (mean bias, 0.74%; Figure E6 in the online data supplement, Supplemental Digital Content 1, http://links.lww.com/EJEM/A377).

## Discussion

In this meta-analysis, we found that the SpCO level had a mean Se of 77% and a mean Sp of 83% to determine CO poisoning using COHb as reference. The LOA of SpCO level was ±8% (mean bias +1%). Some experts have suggested by consensus that acceptable LOA values in this context are ±5% [6,38]. Nevertheless, the FDA 510(k) accuracy specification of the RAD-57 is 3% (SD) corresponding to LOA of ±5.9% [13]. Therefore, the noninvasive measurement of COHb is clearly less accurate than advertised by the manufacturers. Indeed, here we found LOA of ±8.3% when the Rad-57 was used.

The use of CO oximeters for prehospital triage can save time for admission to the hyperbaric center and limit the number of hospital transfers [43]. In clinical practice, SpCO level has a major influence on the decision to transfer, or not, suspected CO poisoning victims to hospitals [44]. However, to achieve a safety rule-out the test used must have a high Se [45]. Here, we found a false-negative rate (1 – Se) of 23% with the SpCO measurement, considering the blood COHb level as reference. By selecting only studies with subjects with suspected CO poisoning, up to 30% of false-negatives were found. There were even negative SpCO levels (0%) while COHb was very high (>35%) [38]. The use of CO oximeters for the triage of potential CO poisoning victims is already discouraged by several experts because of their lack of accuracy, and the results from our meta-analysis also suggest this [6,10,11].

The CO oximeters are proposed as a screening tool in the ED in populations with nonspecific symptoms, to detect occult CO poisoning [10]. Among the three studies included in our systematic review evaluating this population (unselected cohorts in the ED), the false-positive rates were 9%, 23%, and 54% [34,37,40]. A study started in an urban ED on all admitted patients
was prematurely stopped because all patients with positive SpCO (>8% in nonsmokers and >13% in smokers, *n* = 5) had negative blood COHb levels. Emergency staff became skeptical about the clinical usefulness of these devices and decided to stop this research [46]. In

addition, an incidental finding of elevated COHb does not always correlate with CO poisoning [47]. Indeed, it can be due to endogenous production of CO, as found in hemolytic anemia [48]. The use of CO oximeters in an unselected population in the ED would likely be responsible for misdiagnoses as well as an increase in unnecessary COHb blood tests.

In our meta-analysis, the positive likelihood ratios (LR+) was 4.5 and the negative LR (LR−) was 0.3. Usually, LR+ >10 and LR− < 0.1 are considered to provide strong evidence to admit (rule-in) or exclude (rule-out) diagnoses, respectively [49,50]. With LR+ between 2 and 5 and LR− between 0.2 and 0.5, the contribution of SpCO in determining blood COHb level is considered low [51].

A possible alternative to CO oximeters is the measurement of exhaled CO, but the correlation with blood COHb level is uncertain and the measurement can be biased, especially in patients with severe airflow obstruction [52–55]. The correlation between capillary and venous COHb seems to be very high and could be assayed as part of triage using point-of-care analyzers [56].

**Limitations**

First, we included heterogeneous populations in our analysis, which may have introduced noise into our model. Nevertheless, the whole population is concerned by CO poisoning, and for this reason, we did not exclude healthy subjects, sick subjects, or children.

<figure><img src="image_5.png" /><figcaption>Fig. 4</figcaption></figure>

Funnel plot of the 11 included studies.

<figure><img src="image_6.png" /><figcaption>Fig. 5</figcaption></figure>

Limits of agreement of the 18 included studies for the secondary measures.
Second, the studies included in the meta-analysis did not use the same COHb thresholds to define CO poisoning (from >5% to >20%); sometimes without distinction between smokers and nonsmokers. The same is true for the SpCO thresholds used, which also varied from >5% to >20%. It is conventionally accepted that COHb values >3-4% in nonsmokers and >10% in smokers are considered pathological [8]. The use of these different thresholds has an influence on the calculated Se and Sp values. In addition, in most studies, SpCO measurements were performed in optimal settings, and subjects with nail pathologies or nail polish were excluded, which is a deviation from current practice and tended to overestimate the Se and Sp.

Third, we were unable to obtain the individual patient-level data from all the studies and create a global Bland-Altman plot. Furthermore, some studies could not be included in the quantitative analysis due to the lack of usable results. Indeed, diagnostic studies should clearly report the 2x2 diagnostic contingency matrix to be properly exploitable [57]. Therefore, there are only a limited number of studies included in this systematic review, and also a small number of subjects.

## Conclusion

Noninvasive measurement of COHb (SpCO) using current pulse CO-oximeters does not seem to be highly accurate to estimate blood COHb. In clinical practice, it does not appear possible to replace them with blood tests either for safely ruling in or ruling out CO poisoning. Future studies are needed to assess whether SpCO measurement has an impact on outcomes or provides any benefit in the management of subjects with suspected CO poisoning.

## Acknowledgements

The authors thank Dr Oussama Gasmi (Department of Paediatric Emergency Medicine, Nantes University Hospital, France) for his help in structuring the database.

F.J., C.L., and B.L. conceived and designed the study. F.J. and C.L. conducted the literature search and selected the studies. M.P. and C.L. analyzed the quality of the studies. M.P. and F.J. collected the data. B.L. analyzed the data, B.L. and F.J. interpreted the data. M.P. and F.J. wrote the first draft of the article, with all other authors making important critical revisions. All authors have read and approved the final version of the article.

## Conflicts of interest

F.J. is a member of the editorial board of this journal. For the remaining authors, there are no conflicts of interest.

## References

1. Hampson NB, Weaver LK. Carbon monoxide poisoning: a new incidence for an old disease. *Undersea Hyperb Med* 2007; **34**:163-168.

2. Hampson NB. U.S. mortality due to carbon monoxide poisoning, 1999-2014. Accidental and intentional deaths. *Ann Am Thorac Soc* 2016; **13**:1768-1774.

3. Mattiuzzi C, Lippi G. Worldwide epidemiology of carbon monoxide poisoning. *Hum Exp Toxicol* 2020; **39**:387-392.

4. Barret L, Danel V, Faure J. Carbon monoxide poisoning, a diagnosis frequently overlooked. *J Toxicol Clin Toxicol* 1985; **23**:309-313.

5. Hampson NB, Dunn SL; UHMCS/CDC CO Poisoning Surveillance Group. Symptoms of carbon monoxide poisoning do not correlate with the initial carboxyhemoglobin level. *Undersea Hyperb Med* 2012; **39**:657-665.

6. Chiew AL, Buckley NA. Carbon monoxide poisoning in the 21st century. *Crit Care* 2014; **18**:221.

7. Hampson NB, Piantadosi CA, Thom SR, Weaver LK. Practice recommendations in the diagnosis, management, and prevention of carbon monoxide poisoning. *Am J Respir Crit Care Med* 2012; **186**:1095-1101.

8. Lopez DM, Weingarten-Arams JS, Singer LP, Conway EE. Relationship between arterial, mixed venous, and internal jugular carboxyhemoglobin concentrations at low, medium, and high concentrations in a piglet model of carbon monoxide toxicity. *Crit Care Med* 2000; **28**:1998-2001.

9. Touger M, Gallagher EJ, Tyrell J. Relationship between venous and arterial carboxyhemoglobin levels in patients with suspected carbon monoxide poisoning. *Ann Emerg Med* 1995; **25**:481-483.

10. Wolf SJ, Maloney GE, Shih RD, Shy BD, Brown MD; American College of Emergency Physicians Clinical Policies Subcommittee (Writing Committee) on Carbon Monoxide Poisoning. Clinical policy: critical issues in the evaluation and management of adult patients presenting to the emergency department with acute carbon monoxide poisoning. *Ann Emerg Med* 2017; **69**:98-107.e6.

11. Rose JJ, Wang L, Xu Q, McTiernan CF, Shiva S, Tejero J, et al. Carbon monoxide poisoning: pathogenesis, management, and future directions of therapy. *Am J Respir Crit Care Med* 2017; **195**:596-606.

12. Hampson NB, Weaver LK. Noninvasive CO measurement by first responders. A suggested management algorithm. *JEMS* 2006; **31**:S10-S12.

13. O'Reilly M. Performance of the Rad-57 pulse CO-oximeter compared with standard laboratory carboxyhemoglobin measurement. *Ann Emerg Med* 2010; **56**:442-4; author reply 444.

14. McInnes MDF, Moher D, Thombs BD, McGrath TA, Bossuyt PM, Clifford T, et al.; and the PRISMA-DTA Group. Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies: the PRISMA-DTA statement. *JAMA* 2018; **319**:388-396.

15. Whiting PF, Rutjes AWS, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al.; QUADAS-2 Group. QUADAS-2: a revised tool for the Quality Assessment of Diagnostic Accuracy Studies. *Ann Intern Med* 2011; **155**:529-536.

16. Reitsma JB, Glas AS, Rutjes AWS, Scholten RJPM, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. *J Clin Epidemiol* 2005; **58**:982-990.

17. Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. *Stat Med* 2001; **20**:2865-2884.

18. Moses LE, Shapiro D, Littenberg B. Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. *Stat Med* 1993; **12**:1293-1316.

19. Rücker G, Schumacher M. Summary ROC curve based on a weighted Youden index for selecting an optimal cutpoint in meta-analysis of diagnostic accuracy. *Stat Med* 2010; **29**:3069-3078.

20. Tipton E, Shuster J. A framework for the meta-analysis of Bland-Altman studies based on a limits of agreement approach. *Stat Med* 2017; **36**:3621-3635.

21. van Enst WA, Ochodo E, Scholten RJPM, Hooft L, Leeflang MM. Investigation of publication bias in meta-analyses of diagnostic test accuracy: a meta-epidemiological study. *BMC Med Res Methodol* 2014; **14**:70.

22. Barker SJ, Curry J, Redford D, Morgan S. Measurement of carboxyhemoglobin and methemoglobin by pulse oximetry: a human volunteer study. *Anesthesiology* 2006; **105**:892-897.

23. Bidstrup D, Ravn F, Smidt-Nielsen IG, Wahl AM, Jansen EC, Hyldegaard O. Non-invasive monitoring of carboxyhemoglobin during hyperbaric oxygen therapy. *Undersea Hyperb Med* 2021; **48**:33-42.

24. Caboot JB, Jawad AF, McDonough JM, Bowdre CY, Arens R, Marcus CL, et al. Non-invasive measurements of carboxyhemoglobin and methemoglobin in children with sickle cell disease. *Pediatr Pulmonol* 2012; **47**:808-815.
25 Cardwell K, Pan Z, Boucher R, Zuk J, Friesen RH. Screening by pulse CO-oximetry for environmental tobacco smoke exposure in preanesthetic children. *Paediatr Anaesth* 2012; **22**:859–864.

26 Coulange M, Barthelemy A, Hug F, Thierry AL, De Haro I. Reliability of new pulse CO-oximeter in victims of carbon monoxide poisoning. *Undersea Hyperb Med* 2008; **35**:107–111.

27 Feiner JR, Rollins MD, Sall JW, Eilers H, Au P, Bickler PE. Accuracy of carboxyhemoglobin detection by pulse CO-oximetry during hypoxemia. *Anesth Analg* 2013; **117**:847–858.

28 Kaya S, Bulut M, Varsıli B, Kati Y, Karaoğlu U. Comparing noninvasive pulse CO-oximeter vs blood gas analysis in emergency department patients with carbon monoxide poisoning. *Anatolian J Emerg Med* 2018; **1**:1–4.

29 Kot J, Sićko Z, Górczyk P. Carbon monoxide pulse oximetry vs direct spectrophotometry for early detection of CO poisoning. *Anestezjol Intens Ter* 2008; **40**:75–78.

30 Koyuncu S, Bol O, Ertan T, Günay N, Akdogan HI. The detection of occult CO poisoning through noninvasive measurement of carboxyhemoglobin: a cross-sectional study. *Am J Emerg Med* 2020; **38**:1110–1114.

31 Kulcke A, Feiner J, Menn I, Holmer A, Hayoz J, Bickler P. The accuracy of pulse spectroscopy for detecting hypoxemia and coexisting methemoglobin or carboxyhemoglobin. *Anesth Analg* 2016; **122**:1856–1865.

32 Nagano T, Iseki K, Niki T, Suzuki A, Kawamae K. Experience in employing pulse CO-oximetry. *Masui* 2008; **57**:621–623.

33 Piatkowski A, Ulrich D, Grieb G, Pallua N. A new tool for the early diagnosis of carbon monoxide intoxication. *Inhal Toxicol* 2009; **21**:1144–1147.

34 Roth D, Herkner H, Schreiber W, Hubmann N, Gamper G, Laggner AN, et al. Accuracy of noninvasive multiwave pulse oximetry compared with carboxyhemoglobin from blood gas analysis in unselected emergency department patients. *Ann Emerg Med* 2011; **58**:74–79.

35 Ruppel GL, Wilson HA, Gall VK, Hempkens JA. Multi-wavelength pulse oximeter is not suitable for adjusting DLCO measurements. *Respir Care* 2011; **56**:1115–1121.

36 Sebbane M, Claret P-G, Mercier G, Lefebvre S, Théry R, Dumont R, et al. Emergency department management of suspected carbon monoxide poisoning: role of pulse CO-oximetry. *Respir Care* 2013; **58**:1614–1620.

37 Suner S, Partridge R, Sucov A, Valente J, Chee K, Hughes A, et al. Non-invasive pulse CO-oximetry screening in the emergency department identifies occult carbon monoxide toxicity. *J Emerg Med* 2008; **34**:441–450.

38 Touger M, Birnbaum A, Wang J, Chou K, Pearson D, Bijur P. Performance of the RAD-57 pulse CO-oximeter compared with standard laboratory carboxyhemoglobin measurement. *Ann Emerg Med* 2010; **56**:382–388.

39 Villalba N, Osborn ZT, Derickson PR, Manning CT, Herrington RR, Kaminsky DA, et al. Diagnostic performance of carbon monoxide testing by pulse oximetry in the emergency department. *Respir Care* 2019; **64**:1351–1357.

40 Weaver LK, Churchill SK, Deru K, Cooney D. False positive rate of carbon monoxide saturation by pulse oximetry of emergency department patients. *Respir Care* 2013; **58**:232–240.

41 Zaouter C, Zavorsky GS. The measurement of carboxyhemoglobin and methemoglobin using a non-invasive pulse CO-oximeter. *Respir Physiol Neurobiol* 2012; **182**:88–92.

42 Zorbalar N, Yesilaras M, Aksay E. Carbon monoxide poisoning in patients presenting to the emergency department with a headache in winter months. *Emerg Med J* 2014; **31**:e66–e70.

43 Hampson NB. Noninvasive pulse CO-oximetry expedites evaluation and management of patients with carbon monoxide poisoning. *Am J Emerg Med* 2012; **30**:2021–2024.

44 Ferrés-Padró V, Sequera VG, Vilajeliu A, Vidal M, Trilla A. Experience with pulse carbon monoxide oximetry for prehospital assessment of patients exposed to smoke from fires in closed spaces in Catalonia. *Emergencies* 2015; **27**:23–26.

45 van Stralen KJ, Stel VS, Reitsma JB, Dekker FW, Zoccali C, Jager KJ. Diagnostic methods I: sensitivity, specificity, and other measures of accuracy. *Kidney Int* 2009; **75**:1257–1263.

46 O'Malley GF. Non-invasive carbon monoxide measurement is not accurate. *Ann Emerg Med* 2006; **48**:477–478.

47 Hampson NB. Non-invasive pulse CO-oximetry: what is measured? *J Emerg Med* 2009; **37**:310.

48 Hampson NB. Carboxyhemoglobin elevation due to hemolytic anemia. *J Emerg Med* 2007; **33**:17–19.

49 Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. *BMJ* 2004; **329**:168–169.

50 Furukawa TA, Strauss SE, Bucher HC, Thomas A, Guyatt G. Diagnostic tests. In: Guyatt G, Rennie D, Meade MO, Cook DJ, editors. *Users' guides to the medical literature: a manual for evidence-based clinical practice*. 3rd ed. McGraw Hill; 2015. pp. 345–358. Chapter 18.

51 Rubinstein ML, Kraft CS, Parrott JS. Determining qualitative effect size ratings using a likelihood ratio scatter matrix in diagnostic test accuracy systematic reviews. *Diagnosis (Berl)* 2018; **5**:205–214.

52 Hullin T, Aboab J, Desseaux K, Chevret S, Annane D. Correlation between clinical severity and different non-invasive measurements of carbon monoxide concentration: a population study. *PLoS One* 2017; **12:e0174672**.

53 Cunningham AJ, Hornbrey P. Breath analysis to detect recent exposure to carbon monoxide. *Postgrad Med J* 2002; **78**:233–237.

54 Lapostolle F, Raynaud PJ, Le Toumelin P, Benaissa A, Agostinucci JM, Adnet F, et al. Measurement of carbon monoxide in expired breath in prehospital management of carbon monoxide intoxication. *Ann Fr Anesth Reanim* 2001; **20**:10–15.

55 Togores B, Bosch M, Agustí AG. The measurement of exhaled carbon monoxide is influenced by airflow obstruction. *Eur Respir J* 2000; **15**:177–180.

56 Garvican LA, Burge CM, Cox AJ, Clark SA, Martin DT, Gore CJ. Carbon monoxide uptake kinetics of arterial, venous and capillary blood during CO rebreathing. *Exp Physiol* 2010; **95**:1156–1166.

57 Behringer W, Freund Y. Clinical translation of diagnostic studies: pitfalls of the usual reported characteristics. *Eur J Emerg Med* 2021; **28**:165–166.