Evaluation and Modelling of the Performance of an
Automated SARS-CoV-2 Antigen Assay According to
Sample Type, Target Population and Epidemic Trends

Nicolas Yin, Cyril Debuysschere, Valery Daubie, Marc Hildebrand, Charlotte

Martin, Sonja Curac, Fanny Ponthieux, Marie-Christine Payen, Olivier

Vandenberg, Marie Hallin

► To cite this version:

Nicolas Yin, Cyril Debuysschere, Valery Daubie, Marc Hildebrand, Charlotte Martin, et al.. Evaluation and Modelling of the Performance of an Automated SARS-CoV-2 Antigen Assay According to Sample Type, Target Population and Epidemic Trends. Diagnostics, 2023, Diagnostics, 12, pp.447. 10.3390/diagnostics12020447. [hal-04454349](https://hal-univ-lille.fr/hal-04454349)

HAL Id: hal-04454349

https://hal.univ-lille.fr/hal-04454349

Submitted on 15 Feb 2024

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Article

# Evaluation and Modelling of the Performance of an Automated SARS-CoV-2 Antigen Assay According to Sample Type, Target Population and Epidemic Trends

Nicolas Yin <fnref n="1" />,* ¹D, Cyril Debuysschere <fnref n="1" />, Valery Daubie ¹D, Marc Hildebrand <fnref n="2" />, Charlotte Martin <fnref n="3" />, Sonja Curac <fnref n="4" />, Fanny Ponthieux <fnref n="1" />, Marie-Christine Payen <fnref n="3" />, Olivier Vandenberg <fnref n="5" />,<fnref n="6" />,<fnref n="7" /> and Marie Hallin <fnref n="1" />,<fnref n="5" />

<figure><img src="image_1.png" /></figure>

check for
updates

**Citation:** Yin, N.; Debuysschere, C.; Daubie, V.; Hildebrand, M.; Martin, C.; Curac, S.; Ponthieux, F.; Payen, M.-C.; Vandenberg, O.; Hallin, M. Evaluation and Modelling of the Performance of an Automated SARS-CoV-2 Antigen Assay According to Sample Type, Target Population and Epidemic Trends.
*Diagnostics* **2022**, *12*, 447.
https://doi.org/10.3390/diagnostics12020447

Academic Editor: Anna Baraniak

Received: 5 January 2022

Accepted: 7 February 2022

Published: 9 February 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_2.png" /></figure>

**Copyright:** © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

**Abstract:** The Lumipulse® G SARS-CoV-2 Ag assay performance was evaluated on prospectively collected saliva and nasopharyngeal swabs (NPS) of recently ill in- and outpatients and according to the estimated viral load. Performances were calculated using RT-PCR positive NPS from patients with symptoms ≤ 7 days and RT-PCR negative NPS as gold standard. In addition, non-selected positive NPS were analyzed to assess the performances on various viral loads. This assay yielded a sensitivity of 93.1% on NPS and 71.4% on saliva for recently ill patients. For NPS with a viral load > 10³ RNA copies/mL, sensitivity was 96.4%. A model established on our daily routine showed fluctuations of the performances depending on the epidemic trends but an overall good negative predictive value. Lumipulse® G SARS-CoV-2 assay yielded good performance for an automated antigen detection assay on NPS. Using it for the detection of recently ill patients or to screen high-risk patients could be an interesting alternative to the more expensive RT-PCR.

**Keywords:** SARS-CoV-2; COVID-19; model; diagnostic; test; assay

## 1. Introduction

The Lumipulse® G SARS-CoV-2 Ag assay (Fujirebio, Tokyo, Japan) is an automated chemiluminescence enzyme immunoassay (CLEIA) allowing SARS-CoV-2 antigen quantification. Its throughput is between 60 to 120 samples per hour depending on the instrument used (Lumipulse® G600II or G1200) and it can be performed on both universal transport medium (UTM) preserved nasopharyngeal swabs (NPS) and saliva. Saliva sample use could indeed improve the comfort of patients, and decrease the needs in terms of swabs and transport media [1,2]. The World Health Organization (WHO) recommends a ≥80% sensitivity and ≥97% specificity for antigen detection test but without providing any specific

<footnote n="1">¹ Department of Microbiology, Laboratoire Hospitalier Universitaire de Bruxelles-Universitair Laboratorium Brussel (LHUB-ULB), Université Libre de Bruxelles (ULB), 1000 Brussels, Belgium; cyril.debuysschere@lhub-ulb.be (C.D.); valery.daubie@lhub-ulb.be (V.D.); fanny.ponthieux@lhub-ulb.be (F.P.); marie.hallin@ulb.be (M.H.)</footnote>

<footnote n="2">² Department of Internal Medicine, Erasme University Hospital, Université Libre de Bruxelles (ULB), 1070 Brussels, Belgium; marc.hildebrand@erasme.ulb.ac.be</footnote>

<footnote n="3">³ Department of Infectious Diseases, Saint-Pierre University Hospital, Université Libre de Bruxelles (ULB), 1000 Brussels, Belgium; charlotte.martin@stpierre-bru.be (C.M.); marie-christine.payen@stpierre-bru.be (M.-C.P.)</footnote>

<footnote n="4">⁴ Emergency Department, Erasme University Hospital, Université Libre de Bruxelles (ULB), 1070 Brussels, Belgium; sonja.curac@erasme.ulb.ac.be</footnote>

<footnote n="5">⁵ Centre for Environmental Health and Occupational Health, School of Public Health, Université Libre de Bruxelles (ULB), 1050 Brussels, Belgium; olivier.vandenberg@lhub-ulb.be</footnote>

<footnote n="6">⁶ Clinical Research and Innovation Unit, Laboratoire Hospitalier Universitaire de Bruxelles-Universitair Laboratorium Brussel (LHUB-ULB), Université Libre de Bruxelles (ULB), 1000 Brussels, Belgium</footnote>

<footnote n="7">⁷ Division of Infection and Immunity, Faculty of Medical Sciences, University College London, London WC1E 6BT, UK</footnote>

<footnote>* Correspondence: nicolas.yin@lhub-ulb.be</footnote>
Article

# Evaluation and Modelling of the Performance of an Automated SARS-CoV-2 Antigen Assay According to Sample Type, Target Population and Epidemic Trends

Nicolas Yin <fnref n="1" />,* ¹D, Cyril Debuysschere <fnref n="1" />, Valery Daubie ¹D, Marc Hildebrand <fnref n="2" />, Charlotte Martin <fnref n="3" />, Sonja Curac <fnref n="4" />, Fanny Ponthieux <fnref n="1" />, Marie-Christine Payen <fnref n="3" />, Olivier Vandenberg <fnref n="5" />,<fnref n="6" />,<fnref n="7" /> and Marie Hallin <fnref n="1" />,<fnref n="5" />

<figure><img src="image_2.png" /></figure>

check for
updates

**Citation:** Yin, N.; Debuysschere, C.; Daubie, V.; Hildebrand, M.; Martin, C.; Curac, S.; Ponthieux, F.; Payen, M.-C.; Vandenberg, O.; Hallin, M. Evaluation and Modelling of the Performance of an Automated SARS-CoV-2 Antigen Assay According to Sample Type, Target Population and Epidemic Trends.
*Diagnostics* **2022**, *12*, 447.
https://doi.org/10.3390/diagnostics12020447

Academic Editor: Anna Baraniak

Received: 5 January 2022

Accepted: 7 February 2022

Published: 9 February 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

<figure><img src="image_3.png" /></figure>

**Copyright:** © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

**Abstract:** The Lumipulse® G SARS-CoV-2 Ag assay performance was evaluated on prospectively collected saliva and nasopharyngeal swabs (NPS) of recently ill in- and outpatients and according to the estimated viral load. Performances were calculated using RT-PCR positive NPS from patients with symptoms ≤ 7 days and RT-PCR negative NPS as gold standard. In addition, non-selected positive NPS were analyzed to assess the performances on various viral loads. This assay yielded a sensitivity of 93.1% on NPS and 71.4% on saliva for recently ill patients. For NPS with a viral load > 10³ RNA copies/mL, sensitivity was 96.4%. A model established on our daily routine showed fluctuations of the performances depending on the epidemic trends but an overall good negative predictive value. Lumipulse® G SARS-CoV-2 assay yielded good performance for an automated antigen detection assay on NPS. Using it for the detection of recently ill patients or to screen high-risk patients could be an interesting alternative to the more expensive RT-PCR.

**Keywords:** SARS-CoV-2; COVID-19; model; diagnostic; test; assay

## 1. Introduction

The Lumipulse® G SARS-CoV-2 Ag assay (Fujirebio, Tokyo, Japan) is an automated chemiluminescence enzyme immunoassay (CLEIA) allowing SARS-CoV-2 antigen quantification. Its throughput is between 60 to 120 samples per hour depending on the instrument used (Lumipulse® G600II or G1200) and it can be performed on both universal transport medium (UTM) preserved nasopharyngeal swabs (NPS) and saliva. Saliva sample use could indeed improve the comfort of patients, and decrease the needs in terms of swabs and transport media [1,2]. The World Health Organization (WHO) recommends a ≥80% sensitivity and ≥97% specificity for antigen detection test but without providing any specific

<footnote n="1">¹ Department of Microbiology, Laboratoire Hospitalier Universitaire de Bruxelles-Universitair Laboratorium Brussel (LHUB-ULB), Université Libre de Bruxelles (ULB), 1000 Brussels, Belgium; cyril.debuysschere@lhub-ulb.be (C.D.); valery.daubie@lhub-ulb.be (V.D.); fanny.ponthieux@lhub-ulb.be (F.P.); marie.hallin@ulb.be (M.H.)</footnote>

<footnote n="2">² Department of Internal Medicine, Erasme University Hospital, Université Libre de Bruxelles (ULB), 1070 Brussels, Belgium; marc.hildebrand@erasme.ulb.ac.be</footnote>

<footnote n="3">³ Department of Infectious Diseases, Saint-Pierre University Hospital, Université Libre de Bruxelles (ULB), 1000 Brussels, Belgium; charlotte.martin@stpierre-bru.be (C.M.); marie-christine.payen@stpierre-bru.be (M.-C.P.)</footnote>

<footnote n="4">⁴ Emergency Department, Erasme University Hospital, Université Libre de Bruxelles (ULB), 1070 Brussels, Belgium; sonja.curac@erasme.ulb.ac.be</footnote>

<footnote n="5">⁵ Centre for Environmental Health and Occupational Health, School of Public Health, Université Libre de Bruxelles (ULB), 1050 Brussels, Belgium; olivier.vandenberg@lhub-ulb.be</footnote>

<footnote n="6">⁶ Clinical Research and Innovation Unit, Laboratoire Hospitalier Universitaire de Bruxelles-Universitair Laboratorium Brussel (LHUB-ULB), Université Libre de Bruxelles (ULB), 1000 Brussels, Belgium</footnote>

<footnote n="7">⁷ Division of Infection and Immunity, Faculty of Medical Sciences, University College London, London WC1E 6BT, UK</footnote>

<footnote>* Correspondence: nicolas.yin@lhub-ulb.be</footnote>
setting [3]. The European Centre for Disease Control agrees with the WHO but advocates for higher performance (≥90% sensitivity and >98% specificity) for samples taken within 5 days from symptom onset (DSO) or seven days from exposure [4]. While global performance of the Lumipulse® G SARS-CoV-2 Ag assay was assessed previously [5–8], performance data are still missing regarding saliva samples for this latter specific population. Furthermore, existing studies [5–8] did not use the pretreatment step recommended now by the manufacturer for viral deactivation and sample fluidification. Adding an in-house inactivation step does improve safety but can also decrease the test sensitivity [9]. Here, we evaluated the performance of the Lumipulse® G SARS-CoV-2 assay using the manufacturer's sample extraction solution (SES) pretreatment on both NPS and saliva and assessed its performance with regards to the target population and the viral load. In a second step, we used the semi-quantitative PCR results from our daily routine to model the influence of the epidemic curve on the overall performance of this assay.

## 2. Materials and Methods

### 2.1. Study Design, Population and Sample Collection

Residual UTMs of patients' NPS were selected on the basis of SARS-CoV-2 daily routine RT-PCR results. In particular, and as required for the CE-IVD certification, a minimum of 300 SARS-CoV-2 negative RT-PCR samples from outpatients, 100 SARS-CoV-2 negative RT-PCR samples from inpatients and 100 SARS-CoV-2 positive RT-PCR samples from patients with maximum 7 DSO were included. In addition, residuals of consecutive available positive samples that did not meet the above selection criteria (i.e., either taken more than 7 DSO or for which DSO was not known) were included to better assess the limit of detection by increasing the number of positive samples and broadening the range of antigen concentration tested. All these samples were stored at 4 °C and analyzed within 2 days after the RT-PCR was performed.

Saliva samples were collected from patients prospectively enrolled in two settings: outpatients at the consultation or at the sampling center with a prescription of a SARS-CoV-2 PCR test on NPS as well as inpatients with a positive SARS-CoV-2 RT-PCR test on NPS for less than 2 days. The NPS was paired with the saliva sample when available.

### 2.2. SARS-CoV-2 RT-PCR

SARS-CoV-2 RT-PCR on NPS was considered as the gold standard. If the NPS was not available anymore to perform the assay, the RT-PCR performed on saliva was considered as a proxy of the gold standard.

The RT-PCR was performed using the Alinity *m* SARS-CoV-2 assay (Abbott Molecular, Des Plaines, IL, USA). The two target sequences (in the RdRp and the N genes) are detected using the same fluorophore. A 4-fold dilution was performed on saliva samples before extraction to reach the volume of 500 µL needed for this assay. Cycle threshold (Ct) values were plotted with standards provided by the Belgian national reference center following recommendations by Sciensano to provide semi-quantitative results [10]. The correspondence table between Ct values and viral load estimation is summarized in Table 1.

<table><caption>**Table 1.** Semi-quantification of SARS-CoV-2 RT-PCR results using the Alinity *m* SARS-CoV-2 assay (Abbott Molecular, Des Plaines, IL, USA).</caption><thead><tr><td>Semi-Quantification</td><td>Ct Values</td><td>Estimated Viral Load (RNA Copies/mL)</td></tr></thead><tbody><tr><td>Weak</td><td>&gt;29.9</td><td>&lt;10<sup>3</sup></td></tr><tr><td>Mild</td><td>&gt;23.3–29.9</td><td>10<sup>3</sup>–&lt;10<sup>5</sup></td></tr><tr><td>Strong</td><td>&gt;16.7–23.3</td><td>10<sup>5</sup>–&lt;10<sup>7</sup></td></tr><tr><td>Very strong</td><td>&le;16.7</td><td>&ge;10<sup>7</sup></td></tr></tbody></table>
setting [3]. The European Centre for Disease Control agrees with the WHO but advocates for higher performance (≥90% sensitivity and >98% specificity) for samples taken within 5 days from symptom onset (DSO) or seven days from exposure [4]. While global performance of the Lumipulse® G SARS-CoV-2 Ag assay was assessed previously [5–8], performance data are still missing regarding saliva samples for this latter specific population. Furthermore, existing studies [5–8] did not use the pretreatment step recommended now by the manufacturer for viral deactivation and sample fluidification. Adding an in-house inactivation step does improve safety but can also decrease the test sensitivity [9]. Here, we evaluated the performance of the Lumipulse® G SARS-CoV-2 assay using the manufacturer’s sample extraction solution (SES) pretreatment on both NPS and saliva and assessed its performance with regards to the target population and the viral load. In a second step, we used the semi-quantitative PCR results from our daily routine to model the influence of the epidemic curve on the overall performance of this assay.

## 2. Materials and Methods

### 2.1. Study Design, Population and Sample Collection

Residual UTMs of patients' NPS were selected on the basis of SARS-CoV-2 daily routine RT-PCR results. In particular, and as required for the CE-IVD certification, a minimum of 300 SARS-CoV-2 negative RT-PCR samples from outpatients, 100 SARS-CoV-2 negative RT-PCR samples from inpatients and 100 SARS-CoV-2 positive RT-PCR samples from patients with maximum 7 DSO were included. In addition, residuals of consecutive available positive samples that did not meet the above selection criteria (i.e., either taken more than 7 DSO or for which DSO was not known) were included to better assess the limit of detection by increasing the number of positive samples and broadening the range of antigen concentration tested. All these samples were stored at 4 °C and analyzed within 2 days after the RT-PCR was performed.

Saliva samples were collected from patients prospectively enrolled in two settings: outpatients at the consultation or at the sampling center with a prescription of a SARS-CoV-2 PCR test on NPS as well as inpatients with a positive SARS-CoV-2 RT-PCR test on NPS for less than 2 days. The NPS was paired with the saliva sample when available.

### 2.2. SARS-CoV-2 RT-PCR

SARS-CoV-2 RT-PCR on NPS was considered as the gold standard. If the NPS was not available anymore to perform the assay, the RT-PCR performed on saliva was considered as a proxy of the gold standard.

The RT-PCR was performed using the Alinity *m* SARS-CoV-2 assay (Abbott Molecular, Des Plaines, IL, USA). The two target sequences (in the RdRp and the N genes) are detected using the same fluorophore. A 4-fold dilution was performed on saliva samples before extraction to reach the volume of 500 µL needed for this assay. Cycle threshold (Ct) values were plotted with standards provided by the Belgian national reference center following recommendations by Sciensano to provide semi-quantitative results [10]. The correspondence table between Ct values and viral load estimation is summarized in Table 1.

<table><caption>**Table 1.** Semi-quantification of SARS-CoV-2 RT-PCR results using the Alinity *m* SARS-CoV-2 assay (Abbott Molecular, Des Plaines, IL, USA).</caption><thead><tr><td>Semi-Quantification</td><td>Ct Values</td><td>Estimated Viral Load (RNA Copies/mL)</td></tr></thead><tbody><tr><td>Weak</td><td>&gt;29.9</td><td>&lt;10<sup>3</sup></td></tr><tr><td>Mild</td><td>&gt;23.3–29.9</td><td>10<sup>3</sup>–&lt;10<sup>5</sup></td></tr><tr><td>Strong</td><td>&gt;16.7–23.3</td><td>10<sup>5</sup>–&lt;10<sup>7</sup></td></tr><tr><td>Very strong</td><td>&le;16.7</td><td>&ge;10<sup>7</sup></td></tr></tbody></table>
## 2.3. SARS-CoV-2 Antigen Quantification

Antigen quantification was performed using the Lumipulse® G SARS-CoV-2 Ag assay, expressing the dosage in pg/mL. Viral inactivation was performed using the sample extraction solution (SES) by diluting 4 parts of UTM with 1 part of SES or 1 part of saliva with 1 part of SES. A 30 min incubation at room temperature was performed before centrifugation at 2000 × g for 5 min. Lumipulse® SARS-CoV-2 antigen testing was performed on a Lumipulse® G600II instrument according to the manufacturer's instruction. To obtain a binary qualitative result (negative/positive), a positivity threshold was determined using the positive samples from patients with ≤7 DSO and all the negative samples using the following criteria: highest possible Youden's index, specificity >98% and a minimum of 0.6 pg/mL (instrument quantification limit). Samples presenting a dosage below this threshold were considered as negative.

## 2.4. SARS-CoV-2 Variant Determination

As a part of the Belgian national surveillance program, all the strong and very strong available positive samples (see Table 1) were prospectively sequenced at the time of the experimentation using the COVIDSeq kit on NextSeq (Illumina, San Diego, CA, USA). Hence, they represent the variants circulating at the time of the study in Belgium.

## 2.5. Model

All the available positive RT-PCR results of our clinical laboratory were semi-quantified according to their Ct values regardless of their indication (symptoms, screening ...). Daily overall sensitivity (Se), positive predictive value (PPV) and negative predictive value (NPV) were estimated according to the performance of the automated antigen quantification for each semi-quantitative RT-PCR category. To minimize day-to-day and holiday-related fluctuations, data were computed from 1 May 2020 to 30 October 2021 using a moving average of 14 days (hereafter referred as "14-day Se", "14-day PPV" and "14-day NPV").

## 2.6. Statistical Analyses

Statistical analyses and receiver operating characteristic (ROC) curves were performed using Analyse-it® for Microsoft Excel v5.30.4.

## 2.7. Ethical Approval

The Erasme University Hospital Ethics Committee approved and reviewed the comparative performance evaluation study on saliva and NPS. The Ethics Committee of the Saint-Pierre Hospital waived ethical approval for the use of residual human body material for evaluation purpose.

# 3. Results

## 3.1. Population

In total, 632 patients were included with 605 NPS and 144 saliva of which 117 were paired samples. The details of the repartition between in- and outpatients for negative samples and DSO for positive samples are summarized in Table 2.

## 3.2. Threshold Determination

### 3.2.1. NPS

A total of 502 samples were selected: 102 positive samples with ≤7 DSO and 400 negative samples. ROC curve analysis yielded an area under the curve (AUC) at 0.973 ± 0.023 (Figure 1). The highest Youden index was at a threshold of 2.47 pg/mL (sensitivity 93.1%, specificity 99.0%). Analytical performance and their confidence interval at this threshold are summarized in Table 3.
## 2.3. SARS-CoV-2 Antigen Quantification

Antigen quantification was performed using the Lumipulse® G SARS-CoV-2 Ag assay, expressing the dosage in pg/mL. Viral inactivation was performed using the sample extraction solution (SES) by diluting 4 parts of UTM with 1 part of SES or 1 part of saliva with 1 part of SES. A 30 min incubation at room temperature was performed before centrifugation at 2000 × g for 5 min. Lumipulse® SARS-CoV-2 antigen testing was performed on a Lumipulse® G600II instrument according to the manufacturer's instruction. To obtain a binary qualitative result (negative/positive), a positivity threshold was determined using the positive samples from patients with ≤7 DSO and all the negative samples using the following criteria: highest possible Youden's index, specificity >98% and a minimum of 0.6 pg/mL (instrument quantification limit). Samples presenting a dosage below this threshold were considered as negative.

## 2.4. SARS-CoV-2 Variant Determination

As a part of the Belgian national surveillance program, all the strong and very strong available positive samples (see Table 1) were prospectively sequenced at the time of the experimentation using the COVIDSeq kit on NextSeq (Illumina, San Diego, CA, USA). Hence, they represent the variants circulating at the time of the study in Belgium.

## 2.5. Model

All the available positive RT-PCR results of our clinical laboratory were semi-quantified according to their Ct values regardless of their indication (symptoms, screening ... ). Daily overall sensitivity (Se), positive predictive value (PPV) and negative predictive value (NPV) were estimated according to the performance of the automated antigen quantification for each semi-quantitative RT-PCR category. To minimize day-to-day and holiday-related fluctuations, data were computed from 1 May 2020 to 30 October 2021 using a moving average of 14 days (hereafter referred as "14-day Se", "14-day PPV" and "14-day NPV").

## 2.6. Statistical Analyses

Statistical analyses and receiver operating characteristic (ROC) curves were performed using Analyse-it® for Microsoft Excel v5.30.4.

## 2.7. Ethical Approval

The Erasme University Hospital Ethics Committee approved and reviewed the comparative performance evaluation study on saliva and NPS. The Ethics Committee of the Saint-Pierre Hospital waived ethical approval for the use of residual human body material for evaluation purpose.

# 3. Results

## 3.1. Population

In total, 632 patients were included with 605 NPS and 144 saliva of which 117 were paired samples. The details of the repartition between in- and outpatients for negative samples and DSO for positive samples are summarized in Table 2.

## 3.2. Threshold Determination

### 3.2.1. NPS

A total of 502 samples were selected: 102 positive samples with ≤7 DSO and 400 negative samples. ROC curve analysis yielded an area under the curve (AUC) at 0.973 ± 0.023 (Figure 1). The highest Youden index was at a threshold of 2.47 pg/mL (sensitivity 93.1%, specificity 99.0%). Analytical performance and their confidence interval at this threshold are summarized in Table 3.
<table><caption>**Table 2.** Study population (N: total number of samples, Ag: antigen quantification, NPS: nasopharyngeal swabs, DSO: days since symptom onset).</caption><thead><tr><th>Samples Collected</th><th>N</th><th>Ag (NPS)</th><th>Ag (Saliva)</th></tr></thead><tbody><tr><td>Overall</td><td>632</td><td>605</td><td>144</td></tr><tr><td>Negative RT-PCR</td><td>408</td><td>400</td><td>83</td></tr><tr><td>Outpatients</td><td>304</td><td>300</td><td>76</td></tr><tr><td>Inpatients</td><td>104</td><td>100</td><td>7</td></tr><tr><td>Positive RT-PCR</td><td>224</td><td>205</td><td>61</td></tr><tr><td>≤7 DSO</td><td>116</td><td>102</td><td>35</td></tr><tr><td>>7 DSO or unknown</td><td>108</td><td>103</td><td>26</td></tr></tbody></table>

<figure><img src="image_3.png" /><figcaption>**Figure 1.** Receiver Operating Curve (ROC) analysis of antigen quantification (Ag) on nasopharyngeal swabs (NPS) vs. RT-PCR on NPS as gold standard.</figcaption></figure>
<table><caption>**Table 2.** Study population (N: total number of samples, Ag: antigen quantification, NPS: nasopharyngeal swabs, DSO: days since symptom onset).</caption><thead><tr><th>Samples Collected</th><th>N</th><th>Ag (NPS)</th><th>Ag (Saliva)</th></tr></thead><tbody><tr><td>Overall</td><td>632</td><td>605</td><td>144</td></tr><tr><td>Negative RT-PCR</td><td>408</td><td>400</td><td>83</td></tr><tr><td>Outpatients</td><td>304</td><td>300</td><td>76</td></tr><tr><td>Inpatients</td><td>104</td><td>100</td><td>7</td></tr><tr><td>Positive RT-PCR</td><td>224</td><td>205</td><td>61</td></tr><tr><td>≤7 DSO</td><td>116</td><td>102</td><td>35</td></tr><tr><td>>7 DSO or unknown</td><td>108</td><td>103</td><td>26</td></tr></tbody></table>

<figure><img src="image_4.png" /><figcaption>**Figure 1.** Receiver Operating Curve (ROC) analysis of antigen quantification (Ag) on nasopharyngeal swabs (NPS) vs. RT-PCR on NPS as gold standard.</figcaption></figure>
<table><caption>**Table 3.** Analytical performance of the Lumipulse® G SARS-CoV-2 assay on nasopharyngeal swabs (NPS) and saliva vs. RT-PCR on NPS. T: positivity threshold, CI: confidence interval, DSO: days since symptom onset.</caption><thead><tr><th rowspan="2">Samples Collected</th><th colspan="2">Ag (NPS, T = 2.47 pg/mL)</th><th colspan="2">Ag (Saliva, T = 0.60 pg/mL)</th></tr><tr><th>N</th><th>% (Wilson 95% CI)</th><th>N</th><th>% (Wilson 95% CI)</th></tr></thead><tbody><tr><td>Sensitivity (DSO ≤ 7)</td><td>95/102</td><td>93.1 (86.5–96.6)</td><td>25/35</td><td>71.4 (54.9–83.7)</td></tr><tr><td>Specificity (overall)</td><td>396/400</td><td>99.0 (97.5–99.6)</td><td>82/83</td><td>98.8 (93.5–99.8)</td></tr><tr><td>Outpatients</td><td>298/300</td><td>99.3 (97.6–99.8)</td><td>75/76</td><td>98.7 (92.9–99.8)</td></tr><tr><td>Inpatients</td><td>98/100</td><td>98.0 (93.0–99.4)</td><td>7/7</td><td>100 (64.6–100)</td></tr></tbody></table>

### 3.2.2. Saliva

In total, 118 samples were selected: 35 positive samples with ≤7 DSO and 83 negative samples. ROC curve analysis yielded an area under the curve (AUC) at 0.973 ± 0.023 (Figure 2). The highest Youden index was at a threshold of 0.55 pg/mL (sensitivity 71.4%, specificity 98.8%). To respect the quantification limit, the positivity threshold was set at 0.6 pg/mL. Analytical performance and their confidence interval at this threshold are summarized in Table 3.

<figure><img src="image_4.png" /><figcaption>**Figure 2.** Receiver Operating Curve (ROC) analysis of antigen quantification (Ag) on saliva vs. RT-PCR on nasopharyngeal swabs (NPS) as gold standard.</figcaption></figure>

### 3.3. Detection Limit Assessment

The sensitivity of antigen quantification was better for higher estimated viral loads with an overall sensitivity of 96.4% on NPS and of 69.6% on saliva when excluding the low positive samples (estimated viral load < $10^3$ RNA copies/mL, Ct > 29.9), whereas the
<table><caption>**Table 3.** Analytical performance of the Lumipulse® G SARS-CoV-2 assay on nasopharyngeal swabs (NPS) and saliva vs. RT-PCR on NPS. T: positivity threshold, CI: confidence interval, DSO: days since symptom onset.</caption><thead><tr><th rowspan="2">Samples Collected</th><th colspan="2">Ag (NPS, T = 2.47 pg/mL)</th><th colspan="2">Ag (Saliva, T = 0.60 pg/mL)</th></tr><tr><th>N</th><th>% (Wilson 95% CI)</th><th>N</th><th>% (Wilson 95% CI)</th></tr></thead><tbody><tr><td>Sensitivity (DSO ≤ 7)</td><td>95/102</td><td>93.1 (86.5–96.6)</td><td>25/35</td><td>71.4 (54.9–83.7)</td></tr><tr><td>Specificity (overall)</td><td>396/400</td><td>99.0 (97.5–99.6)</td><td>82/83</td><td>98.8 (93.5–99.8)</td></tr><tr><td>Outpatients</td><td>298/300</td><td>99.3 (97.6–99.8)</td><td>75/76</td><td>98.7 (92.9–99.8)</td></tr><tr><td>Inpatients</td><td>98/100</td><td>98.0 (93.0–99.4)</td><td>7/7</td><td>100 (64.6–100)</td></tr></tbody></table>

### 3.2.2. Saliva

In total, 118 samples were selected: 35 positive samples with ≤7 DSO and 83 negative samples. ROC curve analysis yielded an area under the curve (AUC) at 0.973 ± 0.023 (Figure 2). The highest Youden index was at a threshold of 0.55 pg/mL (sensitivity 71.4%, specificity 98.8%). To respect the quantification limit, the positivity threshold was set at 0.6 pg/mL. Analytical performance and their confidence interval at this threshold are summarized in Table 3.

<figure><img src="image_5.png" /><figcaption>**Figure 2.** Receiver Operating Curve (ROC) analysis of antigen quantification (Ag) on saliva vs. RT-PCR on nasopharyngeal swabs (NPS) as gold standard.</figcaption></figure>

### 3.3. Detection Limit Assessment

The sensitivity of antigen quantification was better for higher estimated viral loads with an overall sensitivity of 96.4% on NPS and of 69.6% on saliva when excluding the low positive samples (estimated viral load < $10^3$ RNA copies/mL, Ct > 29.9), whereas the
sensitivity for the low positive samples was 10.5% on NPS. The detailed performance at four different levels of RT-PCR semi-quantification on NPS is summarized in Table 4.

<table><caption>**Table 4.** Analytical performance of the Lumipulse® G SARS-CoV-2 assay on nasopharyngeal swabs (NPS) and saliva vs. RT-PCR on NPS at different levels of viral loads (T: positivity threshold, CI: confidence interval, Ct: cycle threshold value).</caption><thead><tr><th rowspan="2">Estimated Viral Load (RNA Copies/mL)</th><th colspan="2">Ag (NPS, T = 2.47 pg/mL)</th><th colspan="2">Ag (Saliva, T = 0.60 pg/mL)</th></tr><tr><th>N</th><th>% (Wilson 95% CI)</th><th>N</th><th>% (Wilson 95% CI)</th></tr></thead><tbody><tr><td>&lt;10<sup>3</sup> (Ct &gt; 29.9)</td><td>4/38</td><td>10.5 (4.2-24.1)</td><td>5/8</td><td>62.5 (30.6-86.3)</td></tr><tr><td>≥10<sup>3</sup> (Ct ≤ 29.9)</td><td>141/167</td><td>96.4 (92.4-98.3)</td><td>32/46</td><td>69.6 (55.2-80.9)</td></tr><tr><td>10<sup>3</sup>-&lt;10<sup>5</sup> (Ct &gt; 23.3-29.9)</td><td>36/41</td><td>87.8 (74.5-94.7)</td><td>8/19</td><td>42.1 (23.1-63.7)</td></tr><tr><td>10<sup>5</sup>-&lt;10<sup>7</sup> (Ct &gt; 16.7-23.3)</td><td>70/71</td><td>98.6 (92.4-99.8)</td><td>12/15</td><td>80.0 (54.8-93.0)</td></tr><tr><td>≥10<sup>7</sup> (Ct ≤ 16.7)</td><td>55/55</td><td>100 (93.5-100)</td><td>12/12</td><td>100 (75.8-100)</td></tr></tbody></table>

### 3.4. Variants of Interest

A total of 93 (strong and very strong) positive samples were successfully sequenced including 61 (65.6%) alpha variants, 8 (8.6%) gamma variants, 22 (23.7%) delta variants and 2 (2.2%) others (not variants of concern). All these samples yielded a positive antigen result on NPS at the previously determined threshold; no variant effect was highlighted.

### 3.5. Modelling the Influence of Epidemic Trends on the Overall Performance of the Assay

The model showed that using the automated SARS-CoV-2 antigen quantification on our day-to-day non-selected routine samples would exhibit important variation of sensitivity and predictive positive value according to the dynamic of the epidemic at that time (Figure 3). Indeed, during low circulation phases, sensitivity decreases sharply due to a very high proportion of weak positive samples. Likewise, a higher proportion of false positive penalizes the positive predictive value. Conversely, during epidemic peaks, the positive predictive value increased above 90% in November 2020 while the negative predictive value decreased slightly below 95% after the end of the first epidemic wave in Belgium in May 2020.

<figure><img src="image_5.png" /><figcaption>**Figure 3.** Modelling SARS-CoV-2 automated antigen detection performance on epidemic trends. Data computed from 1 May 2020 to 30 October 2021 using a backward sliding window of 14 days (14-day Se: 14-day sensitivity, 14-day PPV: 14-day Positive Predictive Value, 14-day NPV: 14-day Negative Predictive Value).</figcaption></figure>
sensitivity for the low positive samples was 10.5% on NPS. The detailed performance at four different levels of RT-PCR semi-quantification on NPS is summarized in Table 4.

<table><caption>**Table 4.** Analytical performance of the Lumipulse® G SARS-CoV-2 assay on nasopharyngeal swabs (NPS) and saliva vs. RT-PCR on NPS at different levels of viral loads (T: positivity threshold, CI: confidence interval, Ct: cycle threshold value).</caption><thead><tr><th rowspan="2">Estimated Viral Load (RNA Copies/mL)</th><th colspan="2">Ag (NPS, T = 2.47 pg/mL)</th><th colspan="2">Ag (Saliva, T = 0.60 pg/mL)</th></tr><tr><th>N</th><th>% (Wilson 95% CI)</th><th>N</th><th>% (Wilson 95% CI)</th></tr></thead><tbody><tr><td>&lt;10<sup>3</sup> (Ct &gt; 29.9)</td><td>4/38</td><td>10.5 (4.2-24.1)</td><td>5/8</td><td>62.5 (30.6-86.3)</td></tr><tr><td>≥10<sup>3</sup> (Ct ≤ 29.9)</td><td>141/167</td><td>96.4 (92.4-98.3)</td><td>32/46</td><td>69.6 (55.2-80.9)</td></tr><tr><td>10<sup>3</sup>-&lt;10<sup>5</sup> (Ct &gt; 23.3-29.9)</td><td>36/41</td><td>87.8 (74.5-94.7)</td><td>8/19</td><td>42.1 (23.1-63.7)</td></tr><tr><td>10<sup>5</sup>-&lt;10<sup>7</sup> (Ct &gt; 16.7-23.3)</td><td>70/71</td><td>98.6 (92.4-99.8)</td><td>12/15</td><td>80.0 (54.8-93.0)</td></tr><tr><td>≥10<sup>7</sup> (Ct ≤ 16.7)</td><td>55/55</td><td>100 (93.5-100)</td><td>12/12</td><td>100 (75.8-100)</td></tr></tbody></table>

### 3.4. Variants of Interest

A total of 93 (strong and very strong) positive samples were successfully sequenced including 61 (65.6%) alpha variants, 8 (8.6%) gamma variants, 22 (23.7%) delta variants and 2 (2.2%) others (not variants of concern). All these samples yielded a positive antigen result on NPS at the previously determined threshold; no variant effect was highlighted.

### 3.5. Modelling the Influence of Epidemic Trends on the Overall Performance of the Assay

The model showed that using the automated SARS-CoV-2 antigen quantification on our day-to-day non-selected routine samples would exhibit important variation of sensitivity and predictive positive value according to the dynamic of the epidemic at that time (Figure 3). Indeed, during low circulation phases, sensitivity decreases sharply due to a very high proportion of weak positive samples. Likewise, a higher proportion of false positive penalizes the positive predictive value. Conversely, during epidemic peaks, the positive predictive value increased above 90% in November 2020 while the negative predictive value decreased slightly below 95% after the end of the first epidemic wave in Belgium in May 2020.

<figure><img src="image_6.png" /><figcaption>**Figure 3.** Modelling SARS-CoV-2 automated antigen detection performance on epidemic trends. Data computed from 1 May 2020 to 30 October 2021 using a backward sliding window of 14 days (14-day Se: 14-day sensitivity, 14-day PPV: 14-day Positive Predictive Value, 14-day NPV: 14-day Negative Predictive Value).</figcaption></figure>
## 4. Discussion

In a previous study [6], we proposed to position the different available assays for the direct diagnostic of COVID-19 in a structured algorithm. Rapid antigen tests were dedicated to the fast diagnostic of symptomatic patients and automated antigen quantification assays were used as a screening method. This strategy allows the provision of a faster and cheaper but reliable alternative to large molecular platforms. In the present study, the Lumipulse® G SARS-CoV-2 assay yielded excellent analytical performance on NPS for recently symptomatic patients with a sensitivity of 93.1% for patients with DSO ≤ 7 and a specificity of 99.0%. Compared to our previous study [6], the AUC of the ROC curve improved, which can be attributed to the use of SES instead of an in-house inactivation method but also to the selection of positive samples from patients with DSO ≤ 7. Performances of the Lumipulse® G SARS-CoV-2 on NPS were even closer to those of RT-PCR when excluding low positive samples (Se = 96.4%), demonstrating its particular interest to (i) diagnose recently ill patients and (ii) efficiently detect infectious people in a wider screening strategy. As expected, saliva yielded less of a good performance and should be restricted to settings with limited access to swabs or patients presenting contraindication for nasopharyngeal sampling. Comparable results were previously obtained on saliva and on other instruments [11–13].

Additionally, we modelled the estimated performance of this automated quantification assay on our day-to-day routine (“all comers” non-selected samples). As expected, in our hospital laboratory an important proportion (46.8%) of samples were weak positive samples, which impaired in this model the overall sensitivity and positive predictive value of the test as compared with results obtained during the analytical evaluation. However, using this extrapolation could be of use to choose the best technique on the best target population in their own setting. Despite this non-selection of samples, the negative predictive value stayed over 95% during the epidemic peaks. This good negative predictive value and the good sensitivity on samples with significant viral load confirm that the use of this technique is interesting for the fast and large screening of asymptomatic patients in contexts such as hospital admission, participation to a social event and travel. Indeed, this technique was implemented at several airports in Germany to screen travelers using oropharyngeal swabs [14], and in Italy in schools using saliva [15]. Only 9.1% (10/110) of the positive samples from patients with DSO ≤ 7 had a weak positive RT-PCR result, whereas it was the case for 29.9% (32/107) of patients with DSO > 7 or unknown. This difference underlines the lower viral concentration at the late infection stage where the infectiousness is lower. The lower performance of automated antigen quantification in this setting should consequently have a limited impact on virus circulation. However, during periods of low virus circulation, this technique should be used in a two-step strategy, with positive samples checked by a reflex RT-PCR, to balance the lower positive predictive value observed during these periods. From a public health perspective, such strategy would dramatically decrease the costs of the screening for a limited risk and free the PCR instruments for more serious cases and other uses.

Automation allows a higher throughput than manual antigen rapid testing (60–120 tests per hour per instrument) and provides an automated, objective reading of the test. Furthermore, the use of UTM also allows secondary PCR checks and sequencing on positive samples.

Our results provide evidence that the Lumipulse® G SARS-CoV-2 Ag assay is a robust antigen quantification assay for the detection of SARS-CoV-2 on UTM preserved NPS especially for recently ill patients and people with high viral loads. The use of this technique could offer a fast and cost-efficient solution for diverse situations such as systematic or targeted screening in specific situations (travelers, social events, nightlife etc.).
## 4. Discussion

In a previous study [6], we proposed to position the different available assays for the direct diagnostic of COVID-19 in a structured algorithm. Rapid antigen tests were dedicated to the fast diagnostic of symptomatic patients and automated antigen quantification assays were used as a screening method. This strategy allows the provision of a faster and cheaper but reliable alternative to large molecular platforms. In the present study, the Lumipulse® G SARS-CoV-2 assay yielded excellent analytical performance on NPS for recently symptomatic patients with a sensitivity of 93.1% for patients with DSO ≤ 7 and a specificity of 99.0%. Compared to our previous study [6], the AUC of the ROC curve improved, which can be attributed to the use of SES instead of an in-house inactivation method but also to the selection of positive samples from patients with DSO ≤ 7. Performances of the Lumipulse® G SARS-CoV-2 on NPS were even closer to those of RT-PCR when excluding low positive samples (Se = 96.4%), demonstrating its particular interest to (i) diagnose recently ill patients and (ii) efficiently detect infectious people in a wider screening strategy. As expected, saliva yielded less of a good performance and should be restricted to settings with limited access to swabs or patients presenting contraindication for nasopharyngeal sampling. Comparable results were previously obtained on saliva and on other instruments [11–13].

Additionally, we modelled the estimated performance of this automated quantification assay on our day-to-day routine (“all comers” non-selected samples). As expected, in our hospital laboratory an important proportion (46.8%) of samples were weak positive samples, which impaired in this model the overall sensitivity and positive predictive value of the test as compared with results obtained during the analytical evaluation. However, using this extrapolation could be of use to choose the best technique on the best target population in their own setting. Despite this non-selection of samples, the negative predictive value stayed over 95% during the epidemic peaks. This good negative predictive value and the good sensitivity on samples with significant viral load confirm that the use of this technique is interesting for the fast and large screening of asymptomatic patients in contexts such as hospital admission, participation to a social event and travel. Indeed, this technique was implemented at several airports in Germany to screen travelers using oropharyngeal swabs [14], and in Italy in schools using saliva [15]. Only 9.1% (10/110) of the positive samples from patients with DSO ≤ 7 had a weak positive RT-PCR result, whereas it was the case for 29.9% (32/107) of patients with DSO > 7 or unknown. This difference underlines the lower viral concentration at the late infection stage where the infectiousness is lower. The lower performance of automated antigen quantification in this setting should consequently have a limited impact on virus circulation. However, during periods of low virus circulation, this technique should be used in a two-step strategy, with positive samples checked by a reflex RT-PCR, to balance the lower positive predictive value observed during these periods. From a public health perspective, such strategy would dramatically decrease the costs of the screening for a limited risk and free the PCR instruments for more serious cases and other uses.

Automation allows a higher throughput than manual antigen rapid testing (60–120 tests per hour per instrument) and provides an automated, objective reading of the test. Furthermore, the use of UTM also allows secondary PCR checks and sequencing on positive samples.

Our results provide evidence that the Lumipulse® G SARS-CoV-2 Ag assay is a robust antigen quantification assay for the detection of SARS-CoV-2 on UTM preserved NPS especially for recently ill patients and people with high viral loads. The use of this technique could offer a fast and cost-efficient solution for diverse situations such as systematic or targeted screening in specific situations (travelers, social events, nightlife etc.).
**Supplementary Materials:** The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/diagnostics12020447/s1, Table S1: Dataset.

**Author Contributions:** Conceptualization, N.Y., O.V. and M.H. (Marie Hallin); Data curation, N.Y.; Formal analysis, N.Y.; Funding acquisition, O.V.; Investigation, N.Y., C.D., V.D., M.H. (Marc Hildebrand), C.M., S.C., F.P. and M.-C.P.; Methodology, N.Y.; Resources, O.V. and M.H. (Marie Hallin); Supervision, M.H. (Marie Hallin); Validation, N.Y., O.V. and M.H. (Marie Hallin); Writing—original draft, N.Y.; Writing—review and editing, O.V. and M.H. (Marie Hallin). All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by Fujirebio, which provided reagents and instruments and compensated staff costs. The authors did not perceive any personal grants or funding.

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of Erasme Hospital (protocol code P2021/104/B4062021000050, approved 2 March 2021).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the performance evaluation study on saliva. No written consent was required for the use of residual human body material for evaluation purpose.

**Data Availability Statement:** The data presented in this study are available in the Supplementary Material.

**Acknowledgments:** We wish to thank the personnel of the LHUB-ULB for their daily technical assistance. This work is dedicated to the healthcare workers, the patients, and families affected by SARS-CoV-2.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

## References

1. Williams, E.; Bond, K.; Zhang, B.; Putland, M.; Williamson, D.A. Saliva as a Noninvasive Specimen for Detection of SARS-CoV-2. *J. Clin. Microbiol.* **2020**, *58*, e00776-20. [CrossRef]

2. Han, P.; Ivanovski, S. Saliva—Friend and Foe in the COVID-19 Outbreak. *Diagnostics* **2020**, *10*, 290. [CrossRef] [PubMed]

3. World Health Organization. *Antigen-Detection in the Diagnosis of SARS-CoV-2 Infection—Interim Guidance*; World Health Organization: Geneva, Switzerland, 2021.

4. European Centre for Disease Prevention and Control. *Options for the Use of Rapid Antigen Tests for COVID-19 in the EU/EEA and the UK*; European Centre for Disease Prevention and Control: Solna, Sweden, 2020.

5. Hirotsu, Y.; Maejima, M.; Shibusawa, M.; Nagakubo, Y.; Hosaka, K.; Amemiya, K.; Sueki, H.; Hayakawa, M.; Mochizuki, H.; Tsutsui, T.; et al. Comparison of Automated SARS-CoV-2 Antigen Test for COVID-19 Infection with Quantitative RT-PCR Using 313 Nasopharyngeal Swabs, Including from Seven Serially Followed Patients. *Int. J. Infect. Dis.* **2020**, *99*, 397–402. [CrossRef] [PubMed]

6. Yin, N.; Debuyschere, C.; Decroly, M.; Bouazza, F.-Z.; Collot, V.; Martin, C.; Ponthieux, F.; Dahma, H.; Gilbert, M.; Wautier, M.; et al. SARS-CoV-2 Diagnostic Tests: Algorithm and Field Evaluation From the Near Patient Testing to the Automated Diagnostic Platform. *Front. Med.* **2021**, *8*, 650581. [CrossRef] [PubMed]

7. Yokota, I.; Shane, P.Y.; Okada, K.; Unoki, Y.; Yang, Y.; Iwasaki, S.; Fujisawa, S.; Nishida, M.; Teshima, T. A Novel Strategy for SARS-CoV-2 Mass Screening with Quantitative Antigen Testing of Saliva: A Diagnostic Accuracy Study. *Lancet Microbe* **2021**, *2*, e397–e404. [CrossRef]

8. Basso, D.; Aita, A.; Padoan, A.; Cosma, C.; Navaglia, F.; Moz, S.; Contran, N.; Zambon, C.-F.; Maria Cattelan, A.; Plebani, M. Salivary SARS-CoV-2 Antigen Rapid Detection: A Prospective Cohort Study. *Clin. Chim. Acta Int. J. Clin. Chem.* **2021**, *517*, 54–59. [CrossRef] [PubMed]

9. Burton, J.; Love, H.; Richards, K.; Burton, C.; Summers, S.; Pitman, J.; Easterbrook, L.; Davies, K.; Spencer, P.; Killip, M.; et al. The Effect of Heat-Treatment on SARS-CoV-2 Viability and Detection. *J. Virol. Methods* **2021**, *290*, 114087. [CrossRef] [PubMed]

10. Risk Assessment Group RAG. *Interpretation and Reporting of SARS-CoV-2 PCR Results*; Risk Assessment Group RAG: Brussels, Belgium, 2020.

11. Lefever, S.; Indevuyst, C.; Cuypers, L.; Dewaele, K.; Yin, N.; Cotton, F.; Padalko, E.; Oyaert, M.; Descy, J.; Cavalier, E.; et al. Comparison of the Quantitative DiaSorin Liaison Antigen Test to Reverse Transcription-PCR for the Diagnosis of COVID-19 in Symptomatic and Asymptomatic Outpatients. *J. Clin. Microbiol.* **2021**, *59*, e00374-21. [CrossRef] [PubMed]
**Supplementary Materials:** The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/diagnostics12020447/s1, Table S1: Dataset.

**Author Contributions:** Conceptualization, N.Y., O.V. and M.H. (Marie Hallin); Data curation, N.Y.; Formal analysis, N.Y.; Funding acquisition, O.V.; Investigation, N.Y., C.D., V.D., M.H. (Marc Hildebrand), C.M., S.C., F.P. and M.-C.P.; Methodology, N.Y.; Resources, O.V. and M.H. (Marie Hallin); Supervision, M.H. (Marie Hallin); Validation, N.Y., O.V. and M.H. (Marie Hallin); Writing—original draft, N.Y.; Writing—review and editing, O.V. and M.H. (Marie Hallin). All authors have read and agreed to the published version of the manuscript.

**Funding:** This research was funded by Fujirebio, which provided reagents and instruments and compensated staff costs. The authors did not perceive any personal grants or funding.

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki and was approved by the Ethics Committee of Erasme Hospital (protocol code P2021/104/B4062021000050, approved 2 March 2021).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the performance evaluation study on saliva. No written consent was required for the use of residual human body material for evaluation purpose.

**Data Availability Statement:** The data presented in this study are available in the Supplementary Material.

**Acknowledgments:** We wish to thank the personnel of the LHUB-ULB for their daily technical assistance. This work is dedicated to the healthcare workers, the patients, and families affected by SARS-CoV-2.

**Conflicts of Interest:** The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

## References

1. Williams, E.; Bond, K.; Zhang, B.; Putland, M.; Williamson, D.A. Saliva as a Noninvasive Specimen for Detection of SARS-CoV-2. *J. Clin. Microbiol.* **2020**, *58*, e00776-20. [CrossRef]

2. Han, P.; Ivanovski, S. Saliva—Friend and Foe in the COVID-19 Outbreak. *Diagnostics* **2020**, *10*, 290. [CrossRef] [PubMed]

3. World Health Organization. *Antigen-Detection in the Diagnosis of SARS-CoV-2 Infection—Interim Guidance*; World Health Organization: Geneva, Switzerland, 2021.

4. European Centre for Disease Prevention and Control. *Options for the Use of Rapid Antigen Tests for COVID-19 in the EU/EEA and the UK*; European Centre for Disease Prevention and Control: Solna, Sweden, 2020.

5. Hirotsu, Y.; Maejima, M.; Shibusawa, M.; Nagakubo, Y.; Hosaka, K.; Amemiya, K.; Sueki, H.; Hayakawa, M.; Mochizuki, H.; Tsutsui, T.; et al. Comparison of Automated SARS-CoV-2 Antigen Test for COVID-19 Infection with Quantitative RT-PCR Using 313 Nasopharyngeal Swabs, Including from Seven Serially Followed Patients. *Int. J. Infect. Dis.* **2020**, *99*, 397–402. [CrossRef] [PubMed]

6. Yin, N.; Debuyschere, C.; Decroly, M.; Bouazza, F.-Z.; Collot, V.; Martin, C.; Ponthieux, F.; Dahma, H.; Gilbert, M.; Wautier, M.; et al. SARS-CoV-2 Diagnostic Tests: Algorithm and Field Evaluation From the Near Patient Testing to the Automated Diagnostic Platform. *Front. Med.* **2021**, *8*, 650581. [CrossRef] [PubMed]

7. Yokota, I.; Shane, P.Y.; Okada, K.; Unoki, Y.; Yang, Y.; Iwasaki, S.; Fujisawa, S.; Nishida, M.; Teshima, T. A Novel Strategy for SARS-CoV-2 Mass Screening with Quantitative Antigen Testing of Saliva: A Diagnostic Accuracy Study. *Lancet Microbe* **2021**, *2*, e397–e404. [CrossRef]

8. Basso, D.; Aita, A.; Padoan, A.; Cosma, C.; Navaglia, F.; Moz, S.; Contran, N.; Zambon, C.-F.; Maria Cattelan, A.; Plebani, M. Salivary SARS-CoV-2 Antigen Rapid Detection: A Prospective Cohort Study. *Clin. Chim. Acta Int. J. Clin. Chem.* **2021**, *517*, 54–59. [CrossRef] [PubMed]

9. Burton, J.; Love, H.; Richards, K.; Burton, C.; Summers, S.; Pitman, J.; Easterbrook, L.; Davies, K.; Spencer, P.; Killip, M.; et al. The Effect of Heat-Treatment on SARS-CoV-2 Viability and Detection. *J. Virol. Methods* **2021**, *290*, 114087. [CrossRef] [PubMed]

10. Risk Assessment Group RAG. *Interpretation and Reporting of SARS-CoV-2 PCR Results*; Risk Assessment Group RAG: Brussels, Belgium, 2020.

11. Lefever, S.; Indevuyst, C.; Cuypers, L.; Dewaele, K.; Yin, N.; Cotton, F.; Padalko, E.; Oyaert, M.; Descy, J.; Cavalier, E.; et al. Comparison of the Quantitative DiaSorin Liaison Antigen Test to Reverse Transcription-PCR for the Diagnosis of COVID-19 in Symptomatic and Asymptomatic Outpatients. *J. Clin. Microbiol.* **2021**, *59*, e00374-21. [CrossRef] [PubMed]
12. Favresse, J.; Gillot, C.; Oliveira, M.; Cadrobbi, J.; Elsen, M.; Eucher, C.; Laffineur, K.; Rosseels, C.; Van Eeckhoudt, S.; Nicolas, J.-B.; et al. Head-to-Head Comparison of Rapid and Automated Antigen Detection Tests for the Diagnosis of SARS-CoV-2 Infection. *J. Clin. Med.* **2021**, *10*, 265. [CrossRef]

13. Amendola, A.; Sberna, G.; Lalle, E.; Colavita, F.; Castilletti, C.; Menchinelli, G.; Posteraro, B.; Sanguinetti, M.; Ippolito, G.; Bordi, L.; et al. Saliva Is a Valid Alternative to Nasopharyngeal Swab in Chemiluminescence-Based Assay for Detection of SARS-CoV-2 Antigen. *J. Clin. Med.* **2021**, *10*, 1471. [CrossRef]

14. Centogene to Offer Fujirebio SARS-CoV-2 Antigen Test at German Airports. 360Dx. 2020. Available online: https://www.360dx.com/infectious-disease/centogene-offer-fujirebio-sars-cov-2-antigen-test-german-airports#.X99e_NJKhhE (accessed on 20 December 2020).

15. Bordi, L.; Parisi, G.; Sberna, G.; Amendola, A.; Mariani, B.; Meoni, G.; Orazi, D.; Bartoletti, P.; Lombardozzi, L.; Barca, A.; et al. Effective Screening Strategy against SARS-CoV-2 on Self-Collected Saliva Samples in Primary School Setting: A Pilot Project. *J. Infect.* **2021**, *83*, e8–e10. [CrossRef]
12. Favresse, J.; Gillot, C.; Oliveira, M.; Cadrobbi, J.; Elsen, M.; Eucher, C.; Laffineur, K.; Rosseels, C.; Van Eeckhoudt, S.; Nicolas, J.-B.; et al. Head-to-Head Comparison of Rapid and Automated Antigen Detection Tests for the Diagnosis of SARS-CoV-2 Infection. *J. Clin. Med.* **2021**, *10*, 265. [CrossRef]

13. Amendola, A.; Sberna, G.; Lalle, E.; Colavita, F.; Castilletti, C.; Menchinelli, G.; Posteraro, B.; Sanguinetti, M.; Ippolito, G.; Bordi, L.; et al. Saliva Is a Valid Alternative to Nasopharyngeal Swab in Chemiluminescence-Based Assay for Detection of SARS-CoV-2 Antigen. *J. Clin. Med.* **2021**, *10*, 1471. [CrossRef]

14. Centogene to Offer Fujirebio SARS-CoV-2 Antigen Test at German Airports. 360Dx. 2020. Available online: https://www.360dx.com/infectious-disease/centogene-offer-fujirebio-sars-cov-2-antigen-test-german-airports#.X99e_NJKhhE (accessed on 20 December 2020).

15. Bordi, L.; Parisi, G.; Sberna, G.; Amendola, A.; Mariani, B.; Meoni, G.; Orazi, D.; Bartoletti, P.; Lombardozzi, L.; Barca, A.; et al. Effective Screening Strategy against SARS-CoV-2 on Self-Collected Saliva Samples in Primary School Setting: A Pilot Project. *J. Infect.* **2021**, *83*, e8–e10. [CrossRef]