# Evaluation of commercial Anti-SARS-CoV-2 neutralizing antibody assays in seropositive subjects

Kahina Saker, Bruno Pozzetto, Vanessa Escuret, Virginie Pitiot, Amélie Massardier-Pilonchéry, Bouchra Mokdad, Carole Langlois-Jacques, Muriel Rabilloud, Dulce Alfaiate, Nicolas Guibert, et al.

<figure><img src="image_1.png" /></figure>

► To cite this version:

Kahina Saker, Bruno Pozzetto, Vanessa Escuret, Virginie Pitiot, Amélie Massardier-Pilonchéry, et al. Evaluation of commercial Anti-SARS-CoV-2 neutralizing antibody assays in seropositive subjects. Journal of Clinical Virology, 2022, 152, pp.105169. 10.1016/j.jcv.2022.105169. hal-03699350

HAL Id: hal-03699350

https://hal.science/hal-03699350

Submitted on 18 Mar 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
<figure><img src="image_1.png" /></figure>

# Evaluation of commercial Anti-SARS-CoV-2 neutralizing antibody assays in seropositive subjects

<figure><img src="image_2.png" /></figure>

Check for
updates

Kahina Saker<sup>a, *</sup>, Bruno Pozzetto<sup>b</sup>, Vanessa Escuret<sup>a, b</sup>, Virginie Pitiot<sup>c</sup>,
Amélie Massardier-Pilonchéry<sup>c</sup>, Bouchra Mokdad<sup>a</sup>, Carole Langlois-Jacques<sup>d</sup>,
Muriel Rabilloud<sup>d</sup>, Dulce Alfaiate<sup>c</sup>, Nicolas Guibert<sup>c</sup>, Jean-Baptiste Fassier<sup>c</sup>, Antonin Bal<sup>a</sup>,
Sophie Trouillet-Assant<sup>a, b</sup>, Mary-Anne Trabaud<sup>a</sup>

## ARTICLE INFO

### Keywords:

SARS-CoV-2

Neutralizing antibodies

Commercial tests

Surrogate markers of protection

Competitive anti-RBD immunoassays

## ABSTRACT

The virus neutralization test (VNT) is the reference for the assessment of the functional ability of neutralizing antibodies (NAb) to block SARS-CoV-2 entry into cells. New competitive immunoassays measuring antibodies preventing interaction between the spike protein and its cellular receptor are proposed as surrogate VNT (sVNT). We tested three commercial sVNT (a qualitative immunochromatographic test and two quantitative immunoassays named YHLO and TECO) together with a conventional anti-spike IgG assay (bioMérieux) in comparison with an in-house plaque reduction neutralization test (PRNT<sub>50</sub>) using the original 19A strain and different variants of concern (VOC), on a panel of 306 sera from naturally-infected or vaccinated patients. The qualitative test was rapidly discarded because of poor sensitivity and specificity. Areas under the curve of YHLO and TECO assays were, respectively, 85.83 and 84.07 (p-value >0.05) using a positivity threshold of 20 for PRNT<sub>50</sub>, and 95.63 and 90.35 (p-value =0.02) using a threshold of 80. However, the performances of YHLO and bioMérieux were very close for both thresholds, demonstrating the absence of added value of sVNT compared to a conventional assay for the evaluation of the presence of NAb in seropositive subjects. In addition, the PRNT<sub>50</sub> assay showed a reduction of NAb titers towards different VOC in comparison to the 19A strain that could not be appreciated by the commercial tests. Despite the good correlation between the anti-spike antibody titer and the titer of NAb by PRNT<sub>50</sub>, our results highlight the difficulty to distinguish true NAb among the anti-RBD antibodies with commercial user-friendly immunoassays.

## 1. INTRODUCTION

Coronavirus disease 2019 (COVID-19) is an emerging disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and since late 2020, vaccines against SARS-CoV-2 have been available worldwide. In recent months, a large number of commercial immunoassays have been developed for the detection of specific anti-SARS-CoV-2 antibodies (1,2). However, the presence of anti-SARS-CoV-2 antibodies does not indicate whether the antibodies are able to neutralize the virus that has been reported to have a role in the protection from COVID-19 both in animals and humans (3). The gold standard for

assessing the ability of antibodies to prevent the virus from entering into susceptible cells is the virus neutralization test (VNT) (4), but it requires a biosafety level 3 laboratory and takes approximately 10 days to complete. This has led to the development of SARS-CoV-2 surrogate virus neutralization tests (sVNT) that are more simple and rapid; these are based on the competition between patient antibodies and the angiotensin converting enzyme 2 (ACE2) receptor protein for binding to the spike receptor binding domain (RBD) that mediates the entry of the virus into susceptible cells (5).

These competitive immunoassays, which can be conducted using qualitative immunochromatographic cassettes or quantitative

<footnote><sup>a</sup> Laboratoire de Virologie, Institut des Agents Infectieux, Laboratoire associé au Centre National de Référence des virus des infections respiratoires, Hospices Civils de Lyon, IAI, Centre de Biologie Nord, Groupement Hospitalier Nord, F-69317, Lyon Cedex 04, France</footnote>

<footnote><sup>b</sup> CIRI- International Center of Research in Infectiology, INSERM U1111, CNRS UMR5308, ENS Lyon, Claude Bernard Lyon 1 University, F-69008, Lyon, France.</footnote>

<footnote><sup>c</sup> Occupational Health and Medicine Department, Hospices Civils de Lyon, Lyon, France</footnote>

<footnote><sup>d</sup> CNRS, UMR 5558, University of Lyon, Laboratoire de Biométrie et Biologie Evolutive, Equipe Biostatistique-Santé, F-69100, Villeurbanne, France</footnote>

<footnote>* Corresponding author.
E-mail address: kahina.saker@chu-lyon.fr (K. Saker).</footnote>
<figure><img src="image_3.png" /></figure>

# Evaluation of commercial Anti-SARS-CoV-2 neutralizing antibody assays in seropositive subjects

<figure><img src="image_4.png" /></figure>

Check for
updates

Kahina Saker<sup>a, *</sup>, Bruno Pozzetto<sup>b</sup>, Vanessa Escuret<sup>a, b</sup>, Virginie Pitiot<sup>c</sup>,

Amélie Massardier-Pilonchéry<sup>c</sup>, Bouchra Mokdad<sup>a</sup>, Carole Langlois-Jacques<sup>d</sup>,

Muriel Rabilloud<sup>d</sup>, Dulce Alfaiate<sup>c</sup>, Nicolas Guibert<sup>c</sup>, Jean-Baptiste Fassier<sup>c</sup>, Antonin Bal<sup>a</sup>,

Sophie Trouillet-Assant<sup>a, b</sup>, Mary-Anne Trabaud<sup>a</sup>

## ARTICLE INFO

**Keywords:**

SARS-CoV-2

Neutralizing antibodies

Commercial tests

Surrogate markers of protection

Competitive anti-RBD immunoassays

## ABSTRACT

The virus neutralization test (VNT) is the reference for the assessment of the functional ability of neutralizing antibodies (NAb) to block SARS-CoV-2 entry into cells. New competitive immunoassays measuring antibodies preventing interaction between the spike protein and its cellular receptor are proposed as surrogate VNT (sVNT). We tested three commercial sVNT (a qualitative immunochromatographic test and two quantitative immunoassays named YHLO and TECO) together with a conventional anti-spike IgG assay (bioMérieux) in comparison with an in-house plaque reduction neutralization test (PRNT<sub>50</sub>) using the original 19A strain and different variants of concern (VOC), on a panel of 306 sera from naturally-infected or vaccinated patients. The qualitative test was rapidly discarded because of poor sensitivity and specificity. Areas under the curve of YHLO and TECO assays were, respectively, 85.83 and 84.07 (p-value >0.05) using a positivity threshold of 20 for PRNT<sub>50</sub>, and 95.63 and 90.35 (p-value =0.02) using a threshold of 80. However, the performances of YHLO and bioMérieux were very close for both thresholds, demonstrating the absence of added value of sVNT compared to a conventional assay for the evaluation of the presence of NAb in seropositive subjects. In addition, the PRNT<sub>50</sub> assay showed a reduction of NAb titers towards different VOC in comparison to the 19A strain that could not be appreciated by the commercial tests. Despite the good correlation between the anti-spike antibody titer and the titer of NAb by PRNT<sub>50</sub>, our results highlight the difficulty to distinguish true NAb among the anti-RBD antibodies with commercial user-friendly immunoassays.

## 1. INTRODUCTION

Coronavirus disease 2019 (COVID-19) is an emerging disease caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), and since late 2020, vaccines against SARS-CoV-2 have been available worldwide. In recent months, a large number of commercial immunoassays have been developed for the detection of specific anti-SARS-CoV-2 antibodies (1,2). However, the presence of anti-SARS-CoV-2 antibodies does not indicate whether the antibodies are able to neutralize the virus that has been reported to have a role in the protection from COVID-19 both in animals and humans (3). The gold standard for

assessing the ability of antibodies to prevent the virus from entering into susceptible cells is the virus neutralization test (VNT) (4), but it requires a biosafety level 3 laboratory and takes approximately 10 days to complete. This has led to the development of SARS-CoV-2 surrogate virus neutralization tests (sVNT) that are more simple and rapid; these are based on the competition between patient antibodies and the angiotensin converting enzyme 2 (ACE2) receptor protein for binding to the spike receptor binding domain (RBD) that mediates the entry of the virus into susceptible cells (5).

These competitive immunoassays, which can be conducted using qualitative immunochromatographic cassettes or quantitative

<footnote><sup>a</sup> Laboratoire de Virologie, Institut des Agents Infectieux, Laboratoire associé au Centre National de Référence des virus des infections respiratoires, Hospices Civils de Lyon, IAI, Centre de Biologie Nord, Groupement Hospitalier Nord, F-69317, Lyon Cedex 04, France</footnote>

<footnote><sup>b</sup> CIRI- International Center of Research in Infectiology, INSERM U1111, CNRS UMR5308, ENS Lyon, Claude Bernard Lyon 1 University, F-69008, Lyon, France.</footnote>

<footnote><sup>c</sup> Occupational Health and Medicine Department, Hospices Civils de Lyon, Lyon, France</footnote>

<footnote><sup>d</sup> CNRS, UMR 5558, University of Lyon, Laboratoire de Biométrie et Biologie Evolutive, Equipe Biostatistique-Santé, F-69100, Villeurbanne, France</footnote>

<footnote>* Corresponding author.</footnote>

<footnote>E-mail address: kahina.saker@chu-lyon.fr (K. Saker).</footnote>
automated or manual enzyme-linked immunosorbent assay (ELISA) platforms, allow rapid and easy processing of large numbers of samples in conventional serological laboratories. However, the performance of these newly developed commercial sVNT assays by comparison to classical serological assays detecting anti-RBD IgG and/or to the reference plaque reduction neutralization test 50% (PRNT50) performed with live virus has been poorly evaluated up to now (6-9). Moreover, previous studies have evaluated the specificity using seronegative and/or prepandemic serum which do not inform if commercial sVNT can differentiate serum with or without neutralizing antibody in seropositive samples. The aim of the present study was to evaluate the performance of three commercial sVNT and of a classic anti-RBD IgG assay by comparison to NAb titers measured by a conventional PRNT50 with the original strain (clade 19A) and various clades in seropositive samples.

## 2. MATERIALS AND METHODS

### 2.1. Study

This prospective longitudinal cohort study was conducted at the laboratory associated with the national reference center for respiratory viruses (university hospital of Lyon, France). Subjects, (n=306) who were either infected with SARS-CoV-2 (n=246; 83% female; median age 41 [range: 21-66] years) or were scheduled to receive 2 doses of Pfizer BioNtech vaccine (n=31; BNT162b2/BNT162b2; 77% female; median age 41 [range: 26-69] years) or 1 dose of AstraZeneca vaccine followed by 1 dose of Pfizer BioNtech vaccine (n=29; ChAdOx1/BNT162b2; 76% female; median age 35 [range: 21-45] years) were included. For infected patients, a positive RT-PCR test was required; none of them was admitted to hospital. Blood samples were collected 6 months after infection for the convalescent cohort or 4 weeks after the two-dose vaccination for the vaccinated cohort and stored (see supplementary materials and methods).

### 2.2. Serological testing

Four assays were used according to the manufacturer's recommendations: Dynamiker Biotechnology (Tianjin, China) SARS-CoV-2 Neutralization Antibody Rapid Test, Schenzhen YHLO Biotechnologies (Schenzen, China) iFlash-2019-nCoV Nab®, TECO Medical Group (Sissach, Switzerland) SARS-CoV-2 Neutralization Antibody Assay, bioMérieux (Marcy l'Etoile, France) Vidas® SARS-CoV-2 IgG assays. The characteristics of the assays are summarized in Table 1. For the present study suppliers kindly provided all serological kits used; there were 81 Dynamiker tests available, and a sufficient number of YHLO, TECO, and bioMérieux kits for all samples tested herein (Supplementary figure S1).

YHLO and TECO are quantitative assays, bioMérieux is a semi-quantitative assay and Dynamiker is a qualitative assay. Each assay

was compared to the VNT (PRNT50); the latter was used for the detection and titration of neutralizing antibodies, as previously described (4; see supplementary materials and methods). A threshold of 20 and of 80 was used (PRNT50 ≥ 20/80); the threshold of 20 of the live virus neutralization assay was considered as the detection limit of this assay. Thus, samples with PRNT50 titers below 20 are considered as negative for the presence of neutralizing antibodies. In contrast, the PRNT50 threshold of 80 was a cutoff value assumed to discriminate high from low NAb titer.

First of all, we compared performance of sVNT assays with VNT (clade 19A) on 81 and 246 convalescent samples for the three sVNT and the two quantitative sVNT respectively. As the YHLO assay provided the best results among the investigated sVNT, the added-value of this test compared to a commercial serological assay detecting anti-RBD IgG (bioMérieux) was investigated considering both PRNT50 ≥ 20 and ≥ 80 with clade 19A, and the 246 sera from convalescent individuals and 60 sera collected 1 month post vaccination. In further experiments, we correlated the NAb titers obtained with the YHLO and bioMérieux assays to those of PRNT50 measured against various clades of SARS-CoV-2 on 60 serum specimens collected from vaccinated subjects. Each SARS-CoV-2 isolate used in this study [corresponding to 19A (B38 lineage), alpha (B.1.1.7 lineage), beta (B.1.351 lineage), gamma (P1) and delta (B.1.617.2 lineage) clades] has been sequenced to confirm the characteristic mutations of its viral clade. The sequences of the different viral strains used were deposited on Global Initiative on Sharing Avian Influenza Data (GISAID) [GISAID accession numbers: EPI_ISL_1707038 19A (B.38); EPI_ISL_1707039 Alpha (B.1.1.7); EPI_ISL_768828 Beta (B.1.351); EPI_ISL_1359892 Gamma (P.1); EPI_ISL_1904989; Delta (B.1.617.2)].

### 2.3. Statistical analyses

The correlation between Ab concentrations obtained by each assay was investigated using Pearson correlation coefficients and 95% confidence interval [95%CI]. Receiver operating characteristic curves (ROC) were built to estimate the performance of YHLO, TECO, and bioMérieux assays for the detection of the presence of neutralizing antibodies considering the VNT considered as gold standard. Area under the curves (AUC) were compared using the Delong's test. For the YHLO and TECO assays the positive threshold according to the manufacturers' instructions were found to be too low; these were recalculated using the Youden index. Results of various clades were represented with ellipses that show the 95% CIs for different clades schedules, assuming multivariate normal distributions. Statistical analyses were conducted using GraphPad Prism® software (version 8; GraphPad software, La Jolla, CA, USA) and R software, version 3.6.1 (R Foundation for Statistical Computing, Vienna, Austria). A p-value <0.05 was considered statistically significant.

<table><caption>**Table 1**

Characteristics and performance claimed by manufacturer of each assays.</caption><thead><tr><th></th><th>VNT<br>PRNT50</th><th>sVNT<br>YHLO Biotechnologies<br>iFlash</th><th>TECO Medical Group<br>TECO</th><th>Dynamiker Biotechnology<br>Dynamiker</th><th>bioMérieux<br>Vidas</th></tr></thead><tbody><tr><td>SARS-CoV-2 detected Ab</td><td>Total Ab</td><td>IgG</td><td>IgG</td><td>IgG</td><td>IgG</td></tr><tr><td>Assay type</td><td>Neutralization in cell culture</td><td>CLIA</td><td>ELISA</td><td>Immuno-chromatography</td><td>ELFA</td></tr><tr><td>Antigen</td><td>N/A</td><td>RBD</td><td>RBD</td><td>RBD</td><td>RBD</td></tr><tr><td>Positive threshold</td><td>20 or 80</td><td>AU/mL = 10</td><td>UI/mL = 20</td><td>NA</td><td>Index = 1</td></tr><tr><td>Manufacturer sensitivity, % [95%CI]</td><td>N/A</td><td>90</td><td>99.03<br>[94.07; 99.83]</td><td>95.74<br>[85.75; 98.83]</td><td>96.6<br>[82.2; 99.9]</td></tr><tr><td>Manufacturer specificity (% [95%CI])</td><td>N/A</td><td>98</td><td>100<br>[96.68; 100]</td><td>99.15<br>[95.36; 99.85]</td><td>99.9<br>[99.4; 100]</td></tr></tbody></table>

Positivity was established according to manufacturers' instructions. Sensitivity and specificity data were those described in the instruction for utilization sheet from each manufacturer. Specificity given by manufacturers was obtained from pre pandemic samples.

Abbreviations: Ab: antibodies, Ig: immunoglobulin, ELISA: enzyme-linked immunosorbent assay, CLIA: chemiluminescence immunoassay, ELFA: enzyme-linked fluorescent assay, RBD: receptor binding domain, CI: confidence interval, AU: arbitrary unit, UI: unit international.
automated or manual enzyme-linked immunosorbent assay (ELISA) platforms, allow rapid and easy processing of large numbers of samples in conventional serological laboratories. However, the performance of these newly developed commercial sVNT assays by comparison to classical serological assays detecting anti-RBD IgG and/or to the reference plaque reduction neutralization test 50% (PRNT50) performed with live virus has been poorly evaluated up to now (6-9). Moreover, previous studies have evaluated the specificity using seronegative and/or prepandemic serum which do not inform if commercial sVNT can differentiate serum with or without neutralizing antibody in seropositive samples. The aim of the present study was to evaluate the performance of three commercial sVNT and of a classic anti-RBD IgG assay by comparison to NAb titers measured by a conventional PRNT50 with the original strain (clade 19A) and various clades in seropositive samples.

## 2. MATERIALS AND METHODS

### 2.1. Study

This prospective longitudinal cohort study was conducted at the laboratory associated with the national reference center for respiratory viruses (university hospital of Lyon, France). Subjects, (n=306) who were either infected with SARS-CoV-2 (n=246; 83% female; median age 41 [range: 21-66] years) or were scheduled to receive 2 doses of Pfizer BioNtech vaccine (n=31; BNT162b2/BNT162b2; 77% female; median age 41 [range: 26-69] years) or 1 dose of AstraZeneca vaccine followed by 1 dose of Pfizer BioNtech vaccine (n=29; ChAdOx1/BNT162b2; 76% female; median age 35 [range: 21-45] years) were included. For infected patients, a positive RT-PCR test was required; none of them was admitted to hospital. Blood samples were collected 6 months after infection for the convalescent cohort or 4 weeks after the two-dose vaccination for the vaccinated cohort and stored (see supplementary materials and methods).

### 2.2. Serological testing

Four assays were used according to the manufacturer's recommendations: Dynamiker Biotechnology (Tianjin, China) SARS-CoV-2 Neutralization Antibody Rapid Test, Schenzhen YHLO Biotechnologies (Schenzen, China) iFlash-2019-nCoV Nab®, TECO Medical Group (Sissach, Switzerland) SARS-CoV-2 Neutralization Antibody Assay, bioMérieux (Marcy l'Etoile, France) Vidas® SARS-CoV-2 IgG assays. The characteristics of the assays are summarized in Table 1. For the present study suppliers kindly provided all serological kits used; there were 81 Dynamiker tests available, and a sufficient number of YHLO, TECO, and bioMérieux kits for all samples tested herein (Supplementary figure S1).

YHLO and TECO are quantitative assays, bioMérieux is a semi-quantitative assay and Dynamiker is a qualitative assay. Each assay

was compared to the VNT (PRNT50); the latter was used for the detection and titration of neutralizing antibodies, as previously described (4; see supplementary materials and methods). A threshold of 20 and of 80 was used (PRNT50 ≥ 20/80); the threshold of 20 of the live virus neutralization assay was considered as the detection limit of this assay. Thus, samples with PRNT50 titers below 20 are considered as negative for the presence of neutralizing antibodies. In contrast, the PRNT50 threshold of 80 was a cutoff value assumed to discriminate high from low NAb titer.

First of all, we compared performance of sVNT assays with VNT (clade 19A) on 81 and 246 convalescent samples for the three sVNT and the two quantitative sVNT respectively. As the YHLO assay provided the best results among the investigated sVNT, the added-value of this test compared to a commercial serological assay detecting anti-RBD IgG (bioMérieux) was investigated considering both PRNT50 ≥ 20 and ≥ 80 with clade 19A, and the 246 sera from convalescent individuals and 60 sera collected 1 month post vaccination. In further experiments, we correlated the NAb titers obtained with the YHLO and bioMérieux assays to those of PRNT50 measured against various clades of SARS-CoV-2 on 60 serum specimens collected from vaccinated subjects. Each SARS-CoV-2 isolate used in this study [corresponding to 19A (B38 lineage), alpha (B.1.1.7 lineage), beta (B.1.351 lineage), gamma (P1) and delta (B.1.617.2 lineage) clades] has been sequenced to confirm the characteristic mutations of its viral clade. The sequences of the different viral strains used were deposited on Global Initiative on Sharing Avian Influenza Data (GISAID) [GISAID accession numbers: EPI_ISL_1707038 19A (B.38); EPI_ISL_1707039 Alpha (B.1.1.7); EPI_ISL_768828 Beta (B.1.351); EPI_ISL_1359892 Gamma (P.1); EPI_ISL_1904989; Delta (B.1.617.2)].

### 2.3. Statistical analyses

The correlation between Ab concentrations obtained by each assay was investigated using Pearson correlation coefficients and 95% confidence interval [95%CI]. Receiver operating characteristic curves (ROC) were built to estimate the performance of YHLO, TECO, and bioMérieux assays for the detection of the presence of neutralizing antibodies considering the VNT considered as gold standard. Area under the curves (AUC) were compared using the Delong's test. For the YHLO and TECO assays the positive threshold according to the manufacturers' instructions were found to be too low; these were recalculated using the Youden index. Results of various clades were represented with ellipses that show the 95% CIs for different clades schedules, assuming multivariate normal distributions. Statistical analyses were conducted using GraphPad Prism® software (version 8; GraphPad software, La Jolla, CA, USA) and R software, version 3.6.1 (R Foundation for Statistical Computing, Vienna, Austria). A p-value <0.05 was considered statistically significant.

<table><caption>**Table 1**

Characteristics and performance claimed by manufacturer of each assays.</caption><thead><tr><th></th><th>VNT<br>PRNT50</th><th>sVNT<br>YHLO Biotechnologies<br>iFlash</th><th>TECO Medical Group<br>TECO</th><th>Dynamiker Biotechnology<br>Dynamiker</th><th>bioMérieux<br>Vidas</th></tr></thead><tbody><tr><td>SARS-CoV-2 detected Ab</td><td>Total Ab</td><td>IgG</td><td>IgG</td><td>IgG</td><td>IgG</td></tr><tr><td>Assay type</td><td>Neutralization in cell culture</td><td>CLIA</td><td>ELISA</td><td>Immuno-chromatography</td><td>ELFA</td></tr><tr><td>Antigen</td><td>N/A</td><td>RBD</td><td>RBD</td><td>RBD</td><td>RBD</td></tr><tr><td>Positive threshold</td><td>20 or 80</td><td>AU/mL = 10</td><td>UI/mL = 20</td><td>NA</td><td>Index = 1</td></tr><tr><td>Manufacturer sensitivity, % [95%CI]</td><td>N/A</td><td>90</td><td>99.03<br>[94.07; 99.83]</td><td>95.74<br>[85.75; 98.83]</td><td>96.6<br>[82.2; 99.9]</td></tr><tr><td>Manufacturer specificity (% [95%CI])</td><td>N/A</td><td>98</td><td>100<br>[96.68; 100]</td><td>99.15<br>[95.36; 99.85]</td><td>99.9<br>[99.4; 100]</td></tr></tbody></table>

Positivity was established according to manufacturers' instructions. Sensitivity and specificity data were those described in the instruction for utilization sheet from each manufacturer. Specificity given by manufacturers was obtained from pre pandemic samples.

Abbreviations: Ab: antibodies, Ig: immunoglobulin, ELISA: enzyme-linked immunosorbent assay, CLIA: chemiluminescence immunoassay, ELFA: enzyme-linked fluorescent assay, RBD: receptor binding domain, CI: confidence interval, AU: arbitrary unit, UI: unit international.
## 2.4. Ethics statement

Written informed consent was obtained from all participants; ethics approval was obtained from the regional review board for biomedical research in April 2020 (Comité de Protection des Personnes Sud Méditerranée I, Marseille, France; ID RCB 2020-A00932-37), and the study was registered on ClinicalTrials.gov (NCT04341142) (10).

# 3. RESULTS

## 3.1. Performance of sVNT

According to the VNT (PRNT<sub>50</sub>) with clade 19A and the positive threshold of ≥ 20, neutralizing antibodies were found in 54.5% (134/246) in the convalescent cohort and 100% (60/60) in the vaccinated cohort; using the positive threshold of ≥ 80 this was the case for 10,6% (26/246) and 95% (57/60), respectively. The performance of sVNT as- says was estimated among the 81 samples for which data was available using both VNT thresholds [PRNT<sub>50</sub> ≥ 20 (59/81) and PRNT<sub>50</sub> ≥ 80 (24/81)] and according to the positive threshold indicated by the corre- sponding manufacturer. The Dynamiker qualitative test exhibited very weak performance both in terms of sensitivity and specificity, and was discarded from the following steps of the evaluation (Table 2).

Despite the absence of NAb detected using PRNT<sub>50</sub> ≥ 20 in 112/246 samples and using PRNT<sub>50</sub> ≥ 80 in 220/246 samples, the number of

<caption>**Table 2**</caption>

Performance of the three surrogate viral neutralization test compared to PRNT<sub>50</sub> taken as gold standard.

<table><thead><tr><th></th><th>Dynamiker<br>Biotechnology</th><th>YHLO<br>Biotechnologies</th><th>TECO<br>Medical<br>Group<br>TECO</th></tr><tr><th></th><th>Dynamiker</th><th>iFlash</th><th></th></tr></thead><tbody><tr><td>PRNT<sub>50</sub> ≥20<br>(n=81)</td><td></td><td></td><td></td></tr><tr><td>Sensitivity, % [95%<br>CI]</td><td>49.2 [38.70;<br>60.10]</td><td>100 [95.60; 100]</td><td>98.30 [92.80;<br>99.60]</td></tr><tr><td>Specificity, % [95%<br>CI]</td><td>77.3 [60.10;<br>88.50]</td><td>0</td><td>18.20 [8.40;<br>34.90]</td></tr><tr><td>PRNT<sub>50</sub> ≥80<br>(n=81)</td><td></td><td></td><td></td></tr><tr><td>Sensitivity, % [95%<br>CI]</td><td>66.7 [49.90;<br>80.10]</td><td>100 [89.90; 100]</td><td>100 [89.90;<br>100]</td></tr><tr><td>Specificity, % [95%<br>CI]</td><td>68.4 [57.70;<br>77.50]</td><td>0</td><td>8.80 [4.30;<br>16.90]</td></tr><tr><td>PRNT<sub>50</sub> ≥20<br>(n=246)</td><td>N/A</td><td>85.83 [79.22-<br>88.93]</td><td>84.07 [79.22-<br>88.93]</td></tr><tr><td>AUC, % [IQR]</td><td></td><td></td><td></td></tr><tr><td>PRNT<sub>50</sub> ≥80<br>(n=246)</td><td>N/A</td><td>95.63 [93.11-<br>98.15]</td><td>90.35 [85.56-<br>95.14]</td></tr><tr><td>AUC, % [IQR]</td><td></td><td></td><td></td></tr><tr><td>Optimal threshold<br>(PRNT<sub>50</sub> ≥20;<br>n=246)</td><td>N/A</td><td>28.67 [22.58;<br>41.38] AU/mL</td><td>72.82 [53.85;<br>112] UI/mL</td></tr><tr><td>Recalculated<br>sensitivity, %<br>[95%CI]</td><td>N/A</td><td>79.1 [62.28; 91.79]</td><td>74.62 [59.7;<br>85.07]</td></tr><tr><td>Recalculated<br>specificity, %<br>[95%CI]</td><td>N/A</td><td>80.63 [65.18;<br>93.75]</td><td>83.03 [69.64;<br>94.64]</td></tr><tr><td>Optimal threshold<br>(PRNT<sub>50</sub> ≥80;<br>n=246)</td><td>N/A</td><td>70.1 [53.46; 89.47]<br>AU/mL</td><td>176.25<br>[124.96;<br>521.15] UI/<br>mL</td></tr><tr><td>Recalculated<br>sensitivity, %<br>[95%CI]</td><td>N/A</td><td>96.15 [92.31; 100]</td><td>92.31 [80.76;<br>100]</td></tr><tr><td>Recalculated<br>specificity, %<br>[95%CI]</td><td>N/A</td><td>89.09 [78.63;<br>94.09]</td><td>80 [70; 92.27]</td></tr></tbody></table>

IQR: interquartile range, CI: confidence interval, AU: arbitrary unit, UI: unit international.

samples below the manufacturer positive threshold was 0 with the YHLO assay and 21/246 (8.5%) with the TECO assay. Among the 246 samples of convalescent individuals, the median [IQR] titers obtained using the YHLO assay this was 40.69 AU/ml [21.19-156.8] and using the TECO assay was 71.4 IU/ml [36.78-238.3]. Considering the positive threshold of 20 for PRNT<sub>50</sub> as the gold standard, the AUC [95%CI] was 0.86 [0.81; 0.90] for the YHLO assay, and 0.83 [0.78; 0.88] for the TECO assay (p-value > 0.05); considering the positive threshold of 80 for PRNT<sub>50</sub> as the gold standard, the AUC [95%CI] was 0.96 [0.93; 0.98] for the YHLO assay, and 0.94 [0.90; 0.97] for the TECO assay (p-value=0.02; Fig. 1, Table 2). The combination of a good AUC but a very low specificity observed with the positive threshold indicated by the manufacturers led us to determine the best-fit cut-offs for the YHLO and TECO assays using the Youden index. Without weighting for the prevalence of NAb-positive samples, the median [IQR] positive threshold for the YHLO assay this was found to be 28.7 AU/ml [22.6-41.4] considering PRNT<sub>50</sub> ≥ 20 as the gold standard, and 70.1 AU/ml [53.5-89.5] considering PRNT<sub>50</sub> ≥ 80; for the TECO assay was found to be 72.8 IU/ml [53.9-112] considering PRNT<sub>50</sub> ≥ 20 as the gold standard, and 176.3 IU/ml [125-521.2] considering PRNT<sub>50</sub> ≥ 80 (Table 2). Using these optimal thresholds, considering the positive threshold of 20 for PRNT<sub>50</sub> as the gold standard, the sensitivity [95%CI] and the specificity [95%CI] were 79.10 [62.28; 91.79] and 80.63 [65.18; 93.75] respectively for the YHLO assay, and 74.62 [59.7; 85.07] and 83.03 [69.64; 94.64] respectively for the TECO assay; considering the positive threshold of 80 for PRNT<sub>50</sub> as the gold standard the sensitivity [95%CI] and the specificity [95%CI] were 96.15 [92.31; 100] and 89.09 [78.63; 94.09] respectively for the YHLO assay, and 92.31 [80.76; 100] and 80 [70; 92.27] respectively for the TECO assay (Table 2).

## 3.2. Comparison of YHLO assay and bioMérieux anti-RBD assay with regard to PRNT<sub>50</sub>

The correlation coefficient (ρ [IQR]) between the YHLO assay and the VNT (PRNT<sub>50</sub>) was 0.85 [0.81-0.88] (Fig. 2A), and between the VNT (PRNT<sub>50</sub>) and the bioMérieux assay it was 0.82 [0.78-0.85] (Fig. 2B). Considering PRNT<sub>50</sub> ≥20 as the gold standard, the AUC [IQR] for the YHLO was 0.90 [0.87-0.94] and for the bioMérieux assay it was 0.88 [0.85-0.92] (p-value > 0.05; Fig. 2C); considering PRNT<sub>50</sub> ≥80 as the gold standard it was 0.98 [0.96-0.99] for the YHLO assay and 0.98 [0.96-0.99] for the bioMérieux assay (p-value > 0.05; Fig. 2D).

## 3.3. Impact of viral strains on Ab neutralizing capacity

Regarding the neutralizing capacity of serum against VOCs, the median fold-reduction in Nab titers varied between 1.3 against alpha strain and 2.7 against beta strain in comparison to 19A strain (Fig. 3A and 3B).

The Pearson correlation coefficient (ρ [95%CI]) for clades 19A, Alpha, Beta, Gamma and Delta: was of 0.71 [0.26; 0.91], 0.79 [0.43; 0.93], 0.71 [0.23; 0.91], 0.76 [0.36; 0.92] and 0.72 [0.29; 0.91] respectively for the YHLO assay, and of 0.86 [0.58; 0.96], 0.96 [0.86; 0.99], 0.83 [0.49; 0.95], 0.95 [0.83; 0.98] and 0.88 [0.63; 0.96] respectively for the bioMérieux assay.

Despite good correlation between concentrations of anti RBD IgG detected with YHLO or bioMérieux assays and neutralizing antibodies titers against each variant, the same titer of binding Abs overestimates titers of variant Nabs which are lower against the variants than the wild type.

# 4. DISCUSSION

The performance of the qualitative Dynamiker assay was found to be poor, both in terms of sensitivity and in terms of specificity. The two other quantitative sVNT assays evaluated in the present study were found to be more sensitive but their specificity was extremely low since,
## 2.4. Ethics statement

Written informed consent was obtained from all participants; ethics approval was obtained from the regional review board for biomedical research in April 2020 (Comité de Protection des Personnes Sud Méditerranée I, Marseille, France; ID RCB 2020-A00932-37), and the study was registered on ClinicalTrials.gov (NCT04341142) (10).

# 3. RESULTS

## 3.1. Performance of sVNT

According to the VNT (PRNT<sub>50</sub>) with clade 19A and the positive threshold of ≥ 20, neutralizing antibodies were found in 54.5% (134/246) in the convalescent cohort and 100% (60/60) in the vaccinated cohort; using the positive threshold of ≥ 80 this was the case for 10,6% (26/246) and 95% (57/60), respectively. The performance of sVNT as- says was estimated among the 81 samples for which data was available using both VNT thresholds [PRNT<sub>50</sub> ≥ 20 (59/81) and PRNT<sub>50</sub> ≥ 80 (24/81)] and according to the positive threshold indicated by the corre- sponding manufacturer. The Dynamiker qualitative test exhibited very weak performance both in terms of sensitivity and specificity, and was discarded from the following steps of the evaluation (Table 2).

Despite the absence of NAb detected using PRNT<sub>50</sub> ≥ 20 in 112/246 samples and using PRNT<sub>50</sub> ≥ 80 in 220/246 samples, the number of

<caption>**Table 2**</caption>

Performance of the three surrogate viral neutralization test compared to PRNT<sub>50</sub> taken as gold standard.

<table><thead><tr><th></th><th>Dynamiker<br>Biotechnology</th><th>YHLO<br>Biotechnologies</th><th>TECO<br>Medical<br>Group<br>TECO</th></tr><tr><th></th><th>Dynamiker</th><th>iFlash</th><th></th></tr></thead><tbody><tr><td>PRNT<sub>50</sub> ≥20<br>(n=81)</td><td></td><td></td><td></td></tr><tr><td>Sensitivity, % [95%<br>CI]</td><td>49.2 [38.70;<br>60.10]</td><td>100 [95.60; 100]</td><td>98.30 [92.80;<br>99.60]</td></tr><tr><td>Specificity, % [95%<br>CI]</td><td>77.3 [60.10;<br>88.50]</td><td>0</td><td>18.20 [8.40;<br>34.90]</td></tr><tr><td>PRNT<sub>50</sub> ≥80<br>(n=81)</td><td></td><td></td><td></td></tr><tr><td>Sensitivity, % [95%<br>CI]</td><td>66.7 [49.90;<br>80.10]</td><td>100 [89.90; 100]</td><td>100 [89.90;<br>100]</td></tr><tr><td>Specificity, % [95%<br>CI]</td><td>68.4 [57.70;<br>77.50]</td><td>0</td><td>8.80 [4.30;<br>16.90]</td></tr><tr><td>PRNT<sub>50</sub> ≥20<br>(n=246)</td><td>N/A</td><td>85.83 [79.22-<br>88.93]</td><td>84.07 [79.22-<br>88.93]</td></tr><tr><td>AUC, % [IQR]</td><td></td><td></td><td></td></tr><tr><td>PRNT<sub>50</sub> ≥80<br>(n=246)</td><td>N/A</td><td>95.63 [93.11-<br>98.15]</td><td>90.35 [85.56-<br>95.14]</td></tr><tr><td>AUC, % [IQR]</td><td></td><td></td><td></td></tr><tr><td>Optimal threshold<br>(PRNT<sub>50</sub> ≥20;<br>n=246)</td><td>N/A</td><td>28.67 [22.58;<br>41.38] AU/mL</td><td>72.82 [53.85;<br>112] UI/mL</td></tr><tr><td>Recalculated<br>sensitivity, %<br>[95%CI]</td><td>N/A</td><td>79.1 [62.28; 91.79]</td><td>74.62 [59.7;<br>85.07]</td></tr><tr><td>Recalculated<br>specificity, %<br>[95%CI]</td><td>N/A</td><td>80.63 [65.18;<br>93.75]</td><td>83.03 [69.64;<br>94.64]</td></tr><tr><td>Optimal threshold<br>(PRNT<sub>50</sub> ≥80;<br>n=246)</td><td>N/A</td><td>70.1 [53.46; 89.47]<br>AU/mL</td><td>176.25<br>[124.96;<br>521.15] UI/<br>mL</td></tr><tr><td>Recalculated<br>sensitivity, %<br>[95%CI]</td><td>N/A</td><td>96.15 [92.31; 100]</td><td>92.31 [80.76;<br>100]</td></tr><tr><td>Recalculated<br>specificity, %<br>[95%CI]</td><td>N/A</td><td>89.09 [78.63;<br>94.09]</td><td>80 [70; 92.27]</td></tr></tbody></table>

IQR: interquartile range, CI: confidence interval, AU: arbitrary unit, UI: unit international.

samples below the manufacturer positive threshold was 0 with the YHLO assay and 21/246 (8.5%) with the TECO assay. Among the 246 samples of convalescent individuals, the median [IQR] titers obtained using the YHLO assay this was 40.69 AU/ml [21.19-156.8] and using the TECO assay was 71.4 IU/ml [36.78-238.3]. Considering the positive threshold of 20 for PRNT<sub>50</sub> as the gold standard, the AUC [95%CI] was 0.86 [0.81; 0.90] for the YHLO assay, and 0.83 [0.78; 0.88] for the TECO assay (p-value > 0.05); considering the positive threshold of 80 for PRNT<sub>50</sub> as the gold standard, the AUC [95%CI] was 0.96 [0.93; 0.98] for the YHLO assay, and 0.94 [0.90; 0.97] for the TECO assay (p-value=0.02; Fig. 1, Table 2). The combination of a good AUC but a very low specificity observed with the positive threshold indicated by the manufacturers led us to determine the best-fit cut-offs for the YHLO and TECO assays using the Youden index. Without weighting for the prevalence of NAb-positive samples, the median [IQR] positive threshold for the YHLO assay this was found to be 28.7 AU/ml [22.6-41.4] considering PRNT<sub>50</sub> ≥ 20 as the gold standard, and 70.1 AU/ml [53.5-89.5] considering PRNT<sub>50</sub> ≥ 80; for the TECO assay was found to be 72.8 IU/ml [53.9-112] considering PRNT<sub>50</sub> ≥ 20 as the gold standard, and 176.3 IU/ml [125-521.2] considering PRNT<sub>50</sub> ≥ 80 (Table 2). Using these optimal thresholds, considering the positive threshold of 20 for PRNT<sub>50</sub> as the gold standard, the sensitivity [95%CI] and the specificity [95%CI] were 79.10 [62.28; 91.79] and 80.63 [65.18; 93.75] respectively for the YHLO assay, and 74.62 [59.7; 85.07] and 83.03 [69.64; 94.64] respectively for the TECO assay; considering the positive threshold of 80 for PRNT<sub>50</sub> as the gold standard the sensitivity [95%CI] and the specificity [95%CI] were 96.15 [92.31; 100] and 89.09 [78.63; 94.09] respectively for the YHLO assay, and 92.31 [80.76; 100] and 80 [70; 92.27] respectively for the TECO assay (Table 2).

## 3.2. Comparison of YHLO assay and bioMérieux anti-RBD assay with regard to PRNT<sub>50</sub>

The correlation coefficient (ρ [IQR]) between the YHLO assay and the VNT (PRNT<sub>50</sub>) was 0.85 [0.81-0.88] (Fig. 2A), and between the VNT (PRNT<sub>50</sub>) and the bioMérieux assay it was 0.82 [0.78-0.85] (Fig. 2B). Considering PRNT<sub>50</sub> ≥20 as the gold standard, the AUC [IQR] for the YHLO was 0.90 [0.87-0.94] and for the bioMérieux assay it was 0.88 [0.85-0.92] (p-value > 0.05; Fig. 2C); considering PRNT<sub>50</sub> ≥80 as the gold standard it was 0.98 [0.96-0.99] for the YHLO assay and 0.98 [0.96-0.99] for the bioMérieux assay (p-value > 0.05; Fig. 2D).

## 3.3. Impact of viral strains on Ab neutralizing capacity

Regarding the neutralizing capacity of serum against VOCs, the median fold-reduction in Nab titers varied between 1.3 against alpha strain and 2.7 against beta strain in comparison to 19A strain (Fig. 3A and 3B).

The Pearson correlation coefficient (ρ [95%CI]) for clades 19A, Alpha, Beta, Gamma and Delta: was of 0.71 [0.26; 0.91], 0.79 [0.43; 0.93], 0.71 [0.23; 0.91], 0.76 [0.36; 0.92] and 0.72 [0.29; 0.91] respectively for the YHLO assay, and of 0.86 [0.58; 0.96], 0.96 [0.86; 0.99], 0.83 [0.49; 0.95], 0.95 [0.83; 0.98] and 0.88 [0.63; 0.96] respectively for the bioMérieux assay.

Despite good correlation between concentrations of anti RBD IgG detected with YHLO or bioMérieux assays and neutralizing antibodies titers against each variant, the same titer of binding Abs overestimates titers of variant Nabs which are lower against the variants than the wild type.

# 4. DISCUSSION

The performance of the qualitative Dynamiker assay was found to be poor, both in terms of sensitivity and in terms of specificity. The two other quantitative sVNT assays evaluated in the present study were found to be more sensitive but their specificity was extremely low since,
<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /><figcaption>**Fig. 1.** Comparison of performance of the two sVNT. ROC curves were built to estimate the performance of YHLO (in grey) and TECO (in black) assays for detecting the presence of neutralizing antibodies (PRNT<sub>50</sub> ≥20 (A)) and high neutralizing antibody titre (PRNT<sub>50</sub> ≥80 (B)) from samples of infected patients (n=246).</figcaption></figure>

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /><figcaption>**Fig. 2.** Comparison of performance of the YHLO surrogate quantitative virus neutralization test and the bioMérieux anti-RBD IgG assay with reference to the plaque reduction neutralization test 50% (PRNT<sub>50</sub>) from 246 serum specimens collected from convalescent patients. Panels A (YHLO assay) and B (bioMérieux assay) show the strong correlation between each test and PRNT<sub>50</sub> (the value of the Spearman correlation coefficient is shown on the upper right part of the panel for each test). ROC curves were built to estimate the performance of the YHLO (in grey) and bioMérieux (in black) assays. Two different positive thresholds were used for detecting neutralizing antibodies by PRNT<sub>50</sub>: ≥20 (panel C) and ≥80 (panel D). The Delong test was used to compare the areas under the curve (AUC). No statistically significant difference was observed between the two tests for both thresholds.</figcaption></figure>

at the manufacturers' cutoff, most samples (TECO assay) or all of them (YHLO assay) from convalescent individuals with no detectable NAb using the live virus neutralization assay were positive for NAb with sVNT. It can be postulated that part of the antibodies detected by these ELISA are able to interfere with the interaction between ACE receptor and the viral RBD but not to prevent cell entry of the virus; this may be related to the affinity/avidity of antibodies that is reported to be low after primary infection or first vaccine dose (11,12), and is likely to be

even more the case for the population included herein who were sampled 6 months after infection. Despite this low specificity, these assays correlated with the live virus neutralization assay as also found in other studies (5,6,13–15). The low specificity could also be attributed to a lack of sensitivity of live VNT but it is rather unlikely that decreasing the threshold below 20 would be clinically relevant, such low titer having little chance to be protective in vivo (16). In addition, the manufacturer positivity threshold was determined using pre-pandemic
<figure><img src="image_5.png" /><figcaption>**Fig. 1.** Comparison of performance of the two sVNT. ROC curves were built to estimate the performance of YHLO (in grey) and TECO (in black) assays for detecting the presence of neutralizing antibodies (PRNT<sub>50</sub> ≥20 (A)) and high neutralizing antibody titre (PRNT<sub>50</sub> ≥80 (B)) from samples of infected patients (n=246).</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Fig. 2.** Comparison of performance of the YHLO surrogate quantitative virus neutralization test and the bioMérieux anti-RBD IgG assay with reference to the plaque reduction neutralization test 50% (PRNT<sub>50</sub>) from 246 serum specimens collected from convalescent patients. Panels A (YHLO assay) and B (bioMérieux assay) show the strong correlation between each test and PRNT<sub>50</sub> (the value of the Spearman correlation coefficient is shown on the upper right part of the panel for each test). ROC curves were built to estimate the performance of the YHLO (in grey) and bioMérieux (in black) assays. Two different positive thresholds were used for detecting neutralizing antibodies by PRNT<sub>50</sub>: ≥20 (panel C) and ≥80 (panel D). The Delong test was used to compare the areas under the curve (AUC). No statistically significant difference was observed between the two tests for both thresholds.</figcaption></figure>

at the manufacturers' cutoff, most samples (TECO assay) or all of them (YHLO assay) from convalescent individuals with no detectable NAb using the live virus neutralization assay were positive for NAb with sVNT. It can be postulated that part of the antibodies detected by these ELISA are able to interfere with the interaction between ACE receptor and the viral RBD but not to prevent cell entry of the virus; this may be related to the affinity/avidity of antibodies that is reported to be low after primary infection or first vaccine dose (11,12), and is likely to be

even more the case for the population included herein who were sampled 6 months after infection. Despite this low specificity, these assays correlated with the live virus neutralization assay as also found in other studies (5,6,13–15). The low specificity could also be attributed to a lack of sensitivity of live VNT but it is rather unlikely that decreasing the threshold below 20 would be clinically relevant, such low titer having little chance to be protective in vivo (16). In addition, the manufacturer positivity threshold was determined using pre-pandemic
<figure><img src="image_9.png" /><figcaption>**Fig. 3.** Correlation between the antibody titers obtained with the YHLO test (Panel A) or the BioMérieux assay (panel B) and the plaque reduction neutralization test 50% (PRNT50). Ellipses show the 95% CIs for different clades schedules, assuming multivariate normal distributions. Tests were performed with various clades (19A, Alpha, Beta, Gamma and Delta strains) on 60 samples taken from vaccinated individuals.</figcaption></figure>

serum without any anti SARS-CoV-2 antibodies. Nevertheless, the value of sVNT use is to distinguish antibody with or without neutralizing capacity, and we designed our study to establish this performance using only seropositive samples without presence of Nab to establish specificity for sVNT assays which can explained the low specificity observed in our study.

It seems thus preferable to increase the sVNT cutoff to improve specificity, and go closer to the protective threshold. Our data from ROC curves would indicate that, for the detection of NAb, a threshold of 70 IU/ml and 30 AU/ml should be applied for the TECO an YHLO assays, respectively. However, these data have been obtained from infected subjects late after infection, at a time where antibodies are decreasing (17). This could explain the low frequency of sera with detectable NAb using the VNT, and also the discrepancy in terms of specificity between our results and previous ones using samples earlier after infection (18–21). With time, waning of antibodies could have more impact on the blocking of infection than interference with ACE binding. The study of Von Rein et al (22) suggested that the correlation between sVNT and VNT was greater at higher level of neutralization titer and they concluded that sVNT are only useful when inhibition was above 50%, which is more consistent with our data. Most of the previous studies used the cPASS assay from GeneSript, showing the correlation of competitive immunoassay to live VNT, with good sensitivity and specificity compared to VNT (5,6,9,14,15,19,20,22,23). Only a few studies reported results with the TECO (15,20) or YHLO assays (7,8,21). Of note the study of Chan et al (21) found a diagnostic cutoff, with the YHLO assay, of 27,7 AU/ml, which is closed from ours of 30 AU/ml.

Other studies have compared VNT and sVNT with assays detecting IgG binding to RBD or S proteins and observed a correlation between them (6,8,9). Fisher et al. (6) found that the correlation between sVNT and antibody binding assay is better for samples with high than low PRNT. Others studies have shown that the correlation between VNT and antibody binding assays was lower than between sVNT and antibody binding assays (9). It remains that high-throughput live virus neutralizing assays is not possible, and for this binding or competitive antibody immunoassays could be used but caution should be taken when interpreting the result, regardless of the assay used. Despite the high performance, based on the AUC of the ROC curve, of the two competitive automated immunoassays evaluated in our study, taking VNT as gold standard, we did not demonstrate any added value of sVNT compared to serological assay detecting anti-RBD IgG for evaluating the presence of Nab in seropositive subjects. These results highlight the difficulty to

distinguish the Nab among anti-RBD IgG using a standard immunoassay. This difficulty could be further extrapolated considering the antibodies able to neutralize the SARS-CoV-2 variants. Using serum collected one month post full vaccination in patients with high Nab titers, we confirmed the diminution of Nab titers against different SARS-CoV-2 variant compared to initial strain. Nevertheless, the RBD coated in these competitive sVNT is not adapted to virus evolution and are not able to detect the decrease of NAb titers. To date, VNT remains the only way to detect Nabs against VOC. Taking together, from our data and those previously published, the predictive value of surrogate neutralization assays is still not obvious in all population (infected and/or vaccinated, after priming or boost immunization, early versus late after immunization). In addition, sVNT are not able to predict neutralization of variant, and thus improvements are needed before they can be considered equivalent to VNT to detect NAbs able to protect from infection.

## 5. FUNDING

The respective suppliers kindly provided all the serological kits used in the present study.

## Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## ACKNOWLEDGMENTS

This study was supported by Hospices Civils de Lyon and Fondation des Hospices Civils de Lyon. The respective suppliers kindly provided all the serological kits used. We thank the staff of the occupational health and medicine department of the Hospices Civils de Lyon who contributed to the sample collection. We thank all Clinical Research Associates, for their excellent work. We thank Karima Brahima and all the members of the clinical research department for their reactivity (DRS, Hospices Civils de Lyon). Human biological samples and associated data were obtained from NeuroBioTec (CRB Hospices Civils de Lyon, Lyon France, Biobank BB-0033-00046). We thank all the technicians from the virology laboratory of the Hospices Civils de Lyon who performed the assays on the automated platforms. Lastly, we thank all the healthcare
<figure><img src="image_7.png" /><figcaption>**Fig. 3.** Correlation between the antibody titers obtained with the YHLO test (Panel A) or the BioMérieux assay (panel B) and the plaque reduction neutralization test 50% (PRNT50). Ellipses show the 95% CIs for different clades schedules, assuming multivariate normal distributions. Tests were performed with various clades (19A, Alpha, Beta, Gamma and Delta strains) on 60 samples taken from vaccinated individuals.</figcaption></figure>

serum without any anti SARS-CoV-2 antibodies. Nevertheless, the value of sVNT use is to distinguish antibody with or without neutralizing capacity, and we designed our study to establish this performance using only seropositive samples without presence of Nab to establish specificity for sVNT assays which can explained the low specificity observed in our study.

It seems thus preferable to increase the sVNT cutoff to improve specificity, and go closer to the protective threshold. Our data from ROC curves would indicate that, for the detection of NAb, a threshold of 70 IU/ml and 30 AU/ml should be applied for the TECO an YHLO assays, respectively. However, these data have been obtained from infected subjects late after infection, at a time where antibodies are decreasing (17). This could explain the low frequency of sera with detectable NAb using the VNT, and also the discrepancy in terms of specificity between our results and previous ones using samples earlier after infection (18–21). With time, waning of antibodies could have more impact on the blocking of infection than interference with ACE binding. The study of Von Rein et al (22) suggested that the correlation between sVNT and VNT was greater at higher level of neutralization titer and they concluded that sVNT are only useful when inhibition was above 50%, which is more consistent with our data. Most of the previous studies used the cPASS assay from GeneSript, showing the correlation of competitive immunoassay to live VNT, with good sensitivity and specificity compared to VNT (5,6,9,14,15,19,20,22,23). Only a few studies reported results with the TECO (15,20) or YHLO assays (7,8,21). Of note the study of Chan et al (21) found a diagnostic cutoff, with the YHLO assay, of 27,7 AU/ml, which is closed from ours of 30 AU/ml.

Other studies have compared VNT and sVNT with assays detecting IgG binding to RBD or S proteins and observed a correlation between them (6,8,9). Fisher et al. (6) found that the correlation between sVNT and antibody binding assay is better for samples with high than low PRNT. Others studies have shown that the correlation between VNT and antibody binding assays was lower than between sVNT and antibody binding assays (9). It remains that high-throughput live virus neutralizing assays is not possible, and for this binding or competitive antibody immunoassays could be used but caution should be taken when interpreting the result, regardless of the assay used. Despite the high performance, based on the AUC of the ROC curve, of the two competitive automated immunoassays evaluated in our study, taking VNT as gold standard, we did not demonstrate any added value of sVNT compared to serological assay detecting anti-RBD IgG for evaluating the presence of Nab in seropositive subjects. These results highlight the difficulty to

distinguish the Nab among anti-RBD IgG using a standard immunoassay. This difficulty could be further extrapolated considering the antibodies able to neutralize the SARS-CoV-2 variants. Using serum collected one month post full vaccination in patients with high Nab titers, we confirmed the diminution of Nab titers against different SARS-CoV-2 variant compared to initial strain. Nevertheless, the RBD coated in these competitive sVNT is not adapted to virus evolution and are not able to detect the decrease of NAb titers. To date, VNT remains the only way to detect Nabs against VOC. Taking together, from our data and those previously published, the predictive value of surrogate neutralization assays is still not obvious in all population (infected and/or vaccinated, after priming or boost immunization, early versus late after immunization). In addition, sVNT are not able to predict neutralization of variant, and thus improvements are needed before they can be considered equivalent to VNT to detect NAbs able to protect from infection.

## 5. FUNDING

The respective suppliers kindly provided all the serological kits used in the present study.

## Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## ACKNOWLEDGMENTS

This study was supported by Hospices Civils de Lyon and Fondation des Hospices Civils de Lyon. The respective suppliers kindly provided all the serological kits used. We thank the staff of the occupational health and medicine department of the Hospices Civils de Lyon who contributed to the sample collection. We thank all Clinical Research Associates, for their excellent work. We thank Karima Brahima and all the members of the clinical research department for their reactivity (DRS, Hospices Civils de Lyon). Human biological samples and associated data were obtained from NeuroBioTec (CRB Hospices Civils de Lyon, Lyon France, Biobank BB-0033-00046). We thank all the technicians from the virology laboratory of the Hospices Civils de Lyon who performed the assays on the automated platforms. Lastly, we thank all the healthcare
workers for their participation in this clinical study, and Philip Robinson (DRS, Hospices Civils de Lyon) for help in manuscript preparation.

## Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.jcv.2022.105169.

## References

[1] A. Bal, B. Pozzetto, M.-A. Trabaud, V. Escuret, M. Rabiloud, C. Langlois-Jacques, et al., Evaluation of high-throughput SARS-CoV-2 serological assays in a longitudinal cohort of patients with mild COVID-19: clinical sensitivity, specificity, and association with virus neutralization test, Clin. Chem. 67 (5) (2021 Apr 29) 742-752.

[2] K. Saker, V. Escuret, V. Pitiot, A. Massardier-Pilonchéry, S. Paul, B. Mokdad, et al., Evaluation of commercial Anti-SARS-CoV-2 antibody assays and comparison of standardized titers in vaccinated health care workers, J. Clin. Microbiol. 60 (1) (2022 Jan 19), e0174621.

[3] T.F. Rogers, F. Zhao, D. Huang, N. Beutler, A. Burns, W. He, et al., Isolation of potent SARS-CoV-2 neutralizing antibodies and protection from disease in a small animal model, Science 369 (6506) (2020 Aug 21) 956-963.

[4] V. Legros, S. Denolly, M. Vogrig, B. Boson, E. Siret, J. Rigaill, et al., A longitudinal study of SARS-CoV-2-infected patients reveals a high correlation between neutralizing antibodies and COVID-19 severity, Cellular & Mol. Immunol. [Internet] (2021 Jan 6) [cited 2021 Jan 11]; Available from: http://www.nature.com/articles/s41423-020-00588-2.

[5] J. Lan, J. Ge, J. Yu, S. Shan, H. Zhou, S. Fan, et al., Structure of the SARS-CoV-2 spike receptor-binding domain bound to the ACE2 receptor, Nature 581 (7807) (2020 May) 215-220.

[6] B. Fischer, C. Lichtenberg, L. Müller, J. Timm, J. Fischer, C. Knabbe, A combined strategy to detect plasma samples reliably with high anti-SARS-CoV-2 neutralizing antibody titers in routine laboratories, J. Clin. Virol. 144 (2021 Sep 16), 104984.

[7] J. Favresse, C. Gillot, L. Di Chiaro, C. Eucher, M. Elsen, S. Van Eeckhoudt, et al., Neutralizing antibodies in COVID-19 patients and vaccine recipients after two doses of BNT162b2, Viruses 13 (7) (2021 Jul 14) 1364.

[8] C. Gillot, J. Favresse, V. Maloteau, J.-M. Dogné, J. Douxfils, Dynamics of neutralizing antibody responses following natural SARS-CoV-2 infection and correlation with commercial serologic tests. A reappraisal and indirect comparison with vaccinated subjects, Viruses 13 (11) (2021 Nov 22) 2329.

[9] V. Nandakumar, T. Profaizer, B.K. Lozier, M.G. Elgort, E.T. Larragoite, E.S.C. P. Williams, et al., Evaluation of a surrogate enzyme-linked immunosorbent assay-based severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) cPass neutralization antibody detection assay and correlation with immunoglobulin G commercial serology assays, Arch. Pathol. Lab Med. 145 (10) (2021 Oct 1) 1212-1220.

[10] S. Trouillet-Assant, C. Albert Vega, A. Bal, J.A. Nazare, P. Fascia, A. Paul, et al., Assessment of serological techniques for screening patients for COVID-19 (COVID-SER): a prospective, multicentric study, BMJ Open 10 (11) (2020 Nov 24), e041268.

[11] K. Röltgen, S.D. Boyd, Antibody and B cell responses to SARS-CoV-2 infection and vaccination, Cell Host Microbe. 29 (7) (2021 Jul 14) 1063-1075.

[12] A. Sokal, G. Barba-Spaeth, I. Fernández, M. Broketa, I. Azzaoui, La de, A. Selle, et al., mRNA vaccination of naïve and COVID-19-recovered individuals elicits potent memory B cells that recognize SARS-CoV-2 variants, Immunity 54 (12) (2021 Dec 14) 2893-2907, e5.

[13] A. Cristiano, M. Nuccetelli, M. Pieri, S. Sarubbi, M. Pelagalli, G. Calugi, et al., Serological anti-SARS-CoV-2 neutralizing antibodies association to live virus neutralizing test titers in COVID-19 paucisymptomatic/symptomatic patients and vaccinated subjects, Int. Immunopharmacol. 101 (Pt B) (2021 Oct 4), 108215.

[14] C.W. Tan, W.N. Chia, X. Qin, P. Liu, M.I.-C. Chen, C. Tiu, et al., A SARS-CoV-2 surrogate virus neutralization test based on antibody-mediated blockage of ACE2-spike protein-protein interaction, Nat. Biotechnol. 38 (9) (2020 Sep) 1073-1078.

[15] A. Krüttgen, M. Lauen, H. Klingel, M. Imöhl, M. Kleines, Two novel SARS-CoV-2 surrogate virus neutralization assays are suitable for assessing successful immunization with mRNA-1273, J. Virol. Methods 299 (2021 Sep 23), 114297.

[16] S. Feng, D.J. Phillips, T. White, H. Sayal, P.K. Aley, S. Bibi, et al., Correlates of protection against symptomatic and asymptomatic SARS-CoV-2 infection, Nat. Med. 27 (11) (2021 Nov) 2032-2040.

[17] M. Betton, M. Livrozet, D. Planas, A. Fayol, B. Monel, B. Védie, et al., Sera neutralizing activities against severe acute respiratory syndrome coronavirus 2 and multiple variants 6 months after hospitalization for coronavirus disease 2019, Clin. Infect. Dis. 73 (6) (2021 Sep 15) e1337-e1344.

[18] A.K. Tiwari, G. Negi, R.M. Jaiswal, G. Aggarwal, N. Yadav, V. Kumar, et al., Correlation of sample-to-cut-off ratio of anti-SARS-CoV-2 IgG antibody chemiluminescent assay with neutralization activity: a prospective multi-centric study in India, ISBT Sci. Ser. (2021 Jun 28).

[19] R.A.P.M. Perera, R. Ko, O.T.Y. Tsang, D.S.C. Hui, M.Y.M. Kwan, C.J. Brackman, et al., Evaluation of a SARS-CoV-2 surrogate virus neutralization test for detection of antibody in human, canine, cat, and hamster sera, J. Clin. Microbiol. 59 (2) (2021 Jan 21) e02504-20.

[20] N. Kohmer, C. Rühl, S. Ciesek, H.F. Rabenau, Utility of different surrogate enzyme-linked immunosorbent assays (sELISAs) for detection of SARS-CoV-2 neutralizing antibodies, J. Clin. Med. 10 (10) (2021 May 14) 2128.

[21] K.-H. Chan, K.-Y. Leung, R.-R. Zhang, D. Liu, Y. Fan, H. Chen, et al., Performance of a surrogate SARS-CoV-2-Neutralizing antibody assay in natural infection and vaccination samples, Diagnostics (Basel) 11 (10) (2021 Sep 24) 1757.

[22] C. von Rhein, T. Scholz, L. Henss, R. Kronstein-Wiedemann, T. Schwarz, R. N. Rodionov, et al., Comparison of potency assays to assess SARS-CoV-2 neutralizing antibody capacity in COVID-19 convalescent plasma, J. Virol. Methods 288 (2021 Feb), 114031.

[23] S.C. Taylor, B. Hurst, C.L. Charlton, A. Bailey, J.N. Kanji, M.K. McCarthy, et al., A new SARS-CoV-2 dual-purpose serology test: highly accurate infection tracing and neutralizing antibody response detection, J. Clin. Microbiol. 59 (4) (2021 Mar 19) e02438-20.
workers for their participation in this clinical study, and Philip Robinson (DRS, Hospices Civils de Lyon) for help in manuscript preparation.

## Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.jcv.2022.105169.

## References

[1] A. Bal, B. Pozzetto, M.-A. Trabaud, V. Escuret, M. Rabiloud, C. Langlois-Jacques, et al., Evaluation of high-throughput SARS-CoV-2 serological assays in a longitudinal cohort of patients with mild COVID-19: clinical sensitivity, specificity, and association with virus neutralization test, Clin. Chem. 67 (5) (2021 Apr 29) 742-752.

[2] K. Saker, V. Escuret, V. Pitiot, A. Massardier-Pilonchéry, S. Paul, B. Mokdad, et al., Evaluation of commercial Anti-SARS-CoV-2 antibody assays and comparison of standardized titers in vaccinated health care workers, J. Clin. Microbiol. 60 (1) (2022 Jan 19), e0174621.

[3] T.F. Rogers, F. Zhao, D. Huang, N. Beutler, A. Burns, W. He, et al., Isolation of potent SARS-CoV-2 neutralizing antibodies and protection from disease in a small animal model, Science 369 (6506) (2020 Aug 21) 956-963.

[4] V. Legros, S. Denolly, M. Vogrig, B. Boson, E. Siret, J. Rigaill, et al., A longitudinal study of SARS-CoV-2-infected patients reveals a high correlation between neutralizing antibodies and COVID-19 severity, Cellular & Mol. Immunol. [Internet] (2021 Jan 6) [cited 2021 Jan 11]; Available from: http://www.nature.com/articles/s41423-020-00588-2.

[5] J. Lan, J. Ge, J. Yu, S. Shan, H. Zhou, S. Fan, et al., Structure of the SARS-CoV-2 spike receptor-binding domain bound to the ACE2 receptor, Nature 581 (7807) (2020 May) 215-220.

[6] B. Fischer, C. Lichtenberg, L. Müller, J. Timm, J. Fischer, C. Knabbe, A combined strategy to detect plasma samples reliably with high anti-SARS-CoV-2 neutralizing antibody titers in routine laboratories, J. Clin. Virol. 144 (2021 Sep 16), 104984.

[7] J. Favresse, C. Gillot, L. Di Chiaro, C. Eucher, M. Elsen, S. Van Eeckhoudt, et al., Neutralizing antibodies in COVID-19 patients and vaccine recipients after two doses of BNT162b2, Viruses 13 (7) (2021 Jul 14) 1364.

[8] C. Gillot, J. Favresse, V. Maloteau, J.-M. Dogné, J. Douxfils, Dynamics of neutralizing antibody responses following natural SARS-CoV-2 infection and correlation with commercial serologic tests. A reappraisal and indirect comparison with vaccinated subjects, Viruses 13 (11) (2021 Nov 22) 2329.

[9] V. Nandakumar, T. Profaizer, B.K. Lozier, M.G. Elgort, E.T. Larragoite, E.S.C. P. Williams, et al., Evaluation of a surrogate enzyme-linked immunosorbent assay-based severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) cPass neutralization antibody detection assay and correlation with immunoglobulin G commercial serology assays, Arch. Pathol. Lab Med. 145 (10) (2021 Oct 1) 1212-1220.

[10] S. Trouillet-Assant, C. Albert Vega, A. Bal, J.A. Nazare, P. Fascia, A. Paul, et al., Assessment of serological techniques for screening patients for COVID-19 (COVID-SER): a prospective, multicentric study, BMJ Open 10 (11) (2020 Nov 24), e041268.

[11] K. Röltgen, S.D. Boyd, Antibody and B cell responses to SARS-CoV-2 infection and vaccination, Cell Host Microbe. 29 (7) (2021 Jul 14) 1063-1075.

[12] A. Sokal, G. Barba-Spaeth, I. Fernández, M. Broketa, I. Azzaoui, La de, A. Selle, et al., mRNA vaccination of naïve and COVID-19-recovered individuals elicits potent memory B cells that recognize SARS-CoV-2 variants, Immunity 54 (12) (2021 Dec 14) 2893-2907, e5.

[13] A. Cristiano, M. Nuccetelli, M. Pieri, S. Sarubbi, M. Pelagalli, G. Calugi, et al., Serological anti-SARS-CoV-2 neutralizing antibodies association to live virus neutralizing test titers in COVID-19 paucisymptomatic/symptomatic patients and vaccinated subjects, Int. Immunopharmacol. 101 (Pt B) (2021 Oct 4), 108215.

[14] C.W. Tan, W.N. Chia, X. Qin, P. Liu, M.I.-C. Chen, C. Tiu, et al., A SARS-CoV-2 surrogate virus neutralization test based on antibody-mediated blockage of ACE2-spike protein-protein interaction, Nat. Biotechnol. 38 (9) (2020 Sep) 1073-1078.

[15] A. Krüttgen, M. Lauen, H. Klingel, M. Imöhl, M. Kleines, Two novel SARS-CoV-2 surrogate virus neutralization assays are suitable for assessing successful immunization with mRNA-1273, J. Virol. Methods 299 (2021 Sep 23), 114297.

[16] S. Feng, D.J. Phillips, T. White, H. Sayal, P.K. Aley, S. Bibi, et al., Correlates of protection against symptomatic and asymptomatic SARS-CoV-2 infection, Nat. Med. 27 (11) (2021 Nov) 2032-2040.

[17] M. Betton, M. Livrozet, D. Planas, A. Fayol, B. Monel, B. Védie, et al., Sera neutralizing activities against severe acute respiratory syndrome coronavirus 2 and multiple variants 6 months after hospitalization for coronavirus disease 2019, Clin. Infect. Dis. 73 (6) (2021 Sep 15) e1337-e1344.

[18] A.K. Tiwari, G. Negi, R.M. Jaiswal, G. Aggarwal, N. Yadav, V. Kumar, et al., Correlation of sample-to-cut-off ratio of anti-SARS-CoV-2 IgG antibody chemiluminescent assay with neutralization activity: a prospective multi-centric study in India, ISBT Sci. Ser. (2021 Jun 28).

[19] R.A.P.M. Perera, R. Ko, O.T.Y. Tsang, D.S.C. Hui, M.Y.M. Kwan, C.J. Brackman, et al., Evaluation of a SARS-CoV-2 surrogate virus neutralization test for detection of antibody in human, canine, cat, and hamster sera, J. Clin. Microbiol. 59 (2) (2021 Jan 21) e02504-20.

[20] N. Kohmer, C. Rühl, S. Ciesek, H.F. Rabenau, Utility of different surrogate enzyme-linked immunosorbent assays (sELISAs) for detection of SARS-CoV-2 neutralizing antibodies, J. Clin. Med. 10 (10) (2021 May 14) 2128.

[21] K.-H. Chan, K.-Y. Leung, R.-R. Zhang, D. Liu, Y. Fan, H. Chen, et al., Performance of a surrogate SARS-CoV-2-Neutralizing antibody assay in natural infection and vaccination samples, Diagnostics (Basel) 11 (10) (2021 Sep 24) 1757.

[22] C. von Rhein, T. Scholz, L. Henss, R. Kronstein-Wiedemann, T. Schwarz, R. N. Rodionov, et al., Comparison of potency assays to assess SARS-CoV-2 neutralizing antibody capacity in COVID-19 convalescent plasma, J. Virol. Methods 288 (2021 Feb), 114031.

[23] S.C. Taylor, B. Hurst, C.L. Charlton, A. Bailey, J.N. Kanji, M.K. McCarthy, et al., A new SARS-CoV-2 dual-purpose serology test: highly accurate infection tracing and neutralizing antibody response detection, J. Clin. Microbiol. 59 (4) (2021 Mar 19) e02438-20.