# Evaluation of six commercial kits for the serological diagnosis of Mediterranean visceral leishmaniasis

Maude Lévêque, Émilie Battery, Pascal Delaunay, Badre Eddine Lmimouni, Karim Aoun, Coralie L'ollivier, Patrick Bastien, Charles Mary, Christelle Pomares, Judith Fillaux, et al.

<figure><img src="image_1.png" /></figure>

## To cite this version:

Maude Lévêque, Émilie Battery, Pascal Delaunay, Badre Eddine Lmimouni, Karim Aoun, et al. Evaluation of six commercial kits for the serological diagnosis of Mediterranean visceral leishmaniasis. PLoS Neglected Tropical Diseases, 2020, 14 (3), pp.e0008139. 10.1371/journal.pntd.0008139. hal-02525150

HAL Id: hal-02525150

https://amu.hal.science/hal-02525150

Submitted on 26 May 2021

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
RESEARCH ARTICLE

# Evaluation of six commercial kits for the serological diagnosis of Mediterranean visceral leishmaniasis

Maude F. Lévêque¹, Emilie Battery², Pascal Delaunay³, Badre Eddine Lmimouni⁴,
Karim Aoun⁵, Coralie L'Ollivier⁶, Patrick Bastien¹, Charles Mary⁶, Christelle Pomares³,
Judith Fillaux⁷, Laurence Lachaud¹*

<figure><img src="image_3.png" /></figure>

1. Département de Parasitologie-Mycologie, Université de Montpellier, Centre Hospitalier Universitaire de Montpellier, Centre National de Référence des Leishmanioses, MIVEGEC, Montpellier, France,

2. Département de Parasitologie-Mycologie, Université de Montpellier, Centre Hospitalier Universitaire de Montpellier, Montpellier, France, 3. Département de Parasitologie-Mycologie Centre Hospitalier Universitaire de Nice, France, INSERM U1065 Centre Méditerranéen de Médecine Moléculaire, Nice, France,

4. Département de parasitologie, Hôpital militaire d'instruction Mohamed V, Faculté de médecine et de pharmacie, Université Mohammed V, Rabat, Maroc, 5. Département d'Épidémiologie et d'Écologie Parasitaire & LR 16-IPT-06 "Parasitologie médicale, Biotechnologie et Biomolécules », Institut Pasteur de Tunis, Université Tunis-El Manar, Tunis, Tunisie, 6. Département de Parasitologie-Mycologie, Aix Marseille Université, IRD, AP-HM, SSA, VITROME, IHU-Méditerranée Infection, Marseille, France, 7. Département de Parasitologie-Mycologie, Centre Hospitalier Universitaire de Toulouse-Purpan, Pharmaco-chimie et Biologie Pour le Développement (PHARMA-DEV), IRD UMR 152 Université Paul Sabatier, Toulouse, France

* laurence.lachaud@umontpellier.fr

<figure><img src="image_4.png" /></figure>

OPEN ACCESS

**Citation:** Lévêque MF, Battery E, Delaunay P, Lmimouni BE, Aoun K, L'Ollivier C, et al. (2020) Evaluation of six commercial kits for the serological diagnosis of Mediterranean visceral leishmaniasis. PLoS Negl Trop Dis 14(3): e0008139. https://doi.org/10.1371/journal.pntd.0008139

**Editor:** Camila I. de Oliveira, Instituto Goncalo Moniz-FIOCRUZ, BRAZIL

**Received:** October 9, 2019

**Accepted:** February 12, 2020

**Published:** March 25, 2020

**Copyright:** © 2020 Lévêque et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

**Data Availability Statement:** All relevant data are within the manuscript and its Supporting Information files.

**Funding:** Centre National de Référence des Leishmanioses (LL & PB) received financial support from Santé Publique France (ministry of Health). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

**Competing interests:** The authors have declared that no competing interests exist.

## Abstract

### Background

Zoonotic visceral leishmaniasis (VL) is endemic in the Mediterranean basin. However, large-scale comparative analyses of the commercial kits for the serological diagnosis of this neglected disease are lacking. This study compared the performances of four enzyme-linked immunosorbent assays (ELISA) and two immunochromatographic tests (ICT) as screening tests for the serodiagnosis of human VL in the Mediterranean region.

### Methodology/Principal findings

Serum samples from 319 patients living in France, Tunisia or Morocco were tested using two ICT (IT LEISH and TruQuick LEISH IgG/IgM Meridian) and four ELISA reagents (Nova-Lisa Leishmania infantum IgG, Bordier Leishmania infantum, Ridascreen Leishmania IgG, and Vircell Leishmania). The population with proven VL (n = 181) included 65 immunocompromised patients. Significantly higher percentages of false-negative results were obtained with all assays in immunocompromised patients, compared with the immunocompetent population. In the whole population, sensitivity and specificity ranged from 80.7% to 93.9% and from 95.7% to 100%, respectively. The maximum accuracy was observed with the Bordier and Vircell ELISA kits (96.2%), and the lowest accuracy with Ridascreen reagent (88.7%). New thresholds of positivity are proposed for the Bordier, Vircell and NovaLisa ELISA kits to achieve 95% sensitivity with the highest possible specificity. Western blot (WB), used as a
confirmation method, showed 100% sensitivity and identified 10.1% of asymptomatic carriers among the control population from the South of France.

## Conclusions/Significance

This is the first study that compared commercially available kits for VL serodiagnosis in the endemic region of the Mediterranean basin. It provides specific information about the tests' performance to help clinicians and biologists to select the right assay for VL screening.

## Author summary

Human visceral leishmaniasis (VL) is a neglected disease endemic in the Mediterranean region, caused by *Leishmania infantum*. Accurate VL diagnosis is critical for rapid and correct patient management. Various serological tests are used, including commercial kits that require qualifications with specific information about their performance. In this study, we compared the performance of six commercial kits, two immunochromatographic tests (ICT; IT LEISH and TruQuick LEISH IgG/IgM Meridian) and four enzyme-linked immunosorbent assays (ELISA; NovaLisa *Leishmania infantum* IgG, Bordier *Leishmania infantum*, Ridascreen *Leishmania IgG*, and Vircell *Leishmania*), available in France and used as screening tests. The Bordier and Vircell ELISA reagents displayed the highest analytic performance compared with all the other kits. However, no test displayed 95% sensitivity at the thresholds recommended by the manufacturers. To reach this sensitivity value with the highest possible specificity, new thresholds are proposed for the Bordier, Vircell and NovaLisa ELISA reagents. Positive and negative results obtained with the six screening tests were confirmed by Western blot (100% sensitivity). Overall, this study presents a comparative analysis of commercially available kits for the serodiagnosis of human VL and provides additional information on the prevalence of asymptomatic carriers in the Mediterranean basin.

## Introduction

Human visceral leishmaniasis (VL) caused by *Leishmania infantum* is a neglected, yet severe, zoonotic disease that occurs mainly in Latin America and in the Mediterranean region [1]. Dogs are the main reservoir of *L. infantum* parasites that are transmitted to humans by the bite of infected phlebotomine sandflies. In humans, the infection can be asymptomatic or lead to an acute form characterized by irregular fever, weight loss, splenomegaly, hepatomegaly, anaemia or pancytopenia [2]. If left untreated, the infection is often fatal. Accurate diagnosis, which is crucial to control the infection, relies on epidemiological data, clinical features, and laboratory tests. Parasite detection by microscopic examination of bone marrow aspirates is the gold standard method, but lacks sensitivity and requires invasive procedures [3]. The culture of parasites isolated from patients might improve the diagnostic sensitivity. The search of specific antibodies in serum or plasma, while requiring non-invasive sampling, remains the most widely used method for VL diagnosis. However, serological tests tend to be less sensitive in children younger than three years [4,5] and in immunocompromised patients [6,7]. Currently, molecular detection is considered the most reliable method [8,9], with a sensitivity higher than 95% for PCR analysis on peripheral blood [10]. Yet, such technique is restricted to well-equipped laboratories and is often combined with serological tests in the routine practices [3].
Various methods are used for VL serodiagnosis. The qualitative detection of *Leishmania* antibodies is based on Western blot (WB) method and immunochromatographic tests (ICT), whereas quantitative results are obtained by indirect immunofluorescence, direct agglutination, and enzyme-linked immunosorbent assays (ELISA). The WB technique, using *L. infantum* antigenic fractions, is a highly sensitive and specific method to confirm VL serodiagnosis [11,12]. This technique is time-consuming and quite expensive, thus complex to implement for field surveys and initial serological screenings. Conversely, ICT which are based on the recombinant *Leishmania* rk39 antigen, are well adapted for field testing. Indeed, they are fast and easy to perform, and display good performances for early VL diagnosis [13]. Among the quantitative methods, ELISA based on crude soluble antigens or the recombinant rk39 protein, are widely used for initial screening of VL [3,7], but their performances are influenced by the nature, purity and stability of the used antigens [14]. In this context, commercial kits may offer more uniform procedures for antigen preparation, compared with home-made assays, and provide standardized experimental conditions. However, large-scale studies comparing the performances of commercial assays are lacking [5,7]. A recent study in Brazil compared, for the first time, different kits tested in the same condition and in the same population [5]. Similarly, in the present study, we evaluated the performances of six commercial tests for the diagnosis of human VL in the Mediterranean region. We compared the sensitivity and specificity of two ICT and four ELISA kits, re-evaluated their positivity thresholds, and analysed their advantages and drawbacks for routine diagnosis. Finally, we use a Western blot technique to confirm the serological status of VL and healthy patients, and discuss discrepant positive results obtained in some asymptomatic patients living in the South of France, a VL endemic area in the Mediterranean basin.

## Material and methods

This study complies with the updated Standards for Reporting of Diagnostic Accuracy (STARD) statement [15]. The STARD checklist and a diagram reporting the flow of participant through the study are presented in S1 Checklist and S1 Fig, respectively.

## Study site

This study was carried out at the Parasitology and Mycology Department of Montpellier University Hospital, France, on behalf the French National Reference Centre of Leishmaniasis (Centre National de Référence des Leishmanioses, CNRL) from January to August 2019.

## Study design and human samples

Six commercial kits were evaluated using human serum samples collected between 1998 and 2018 and stored at -20°C at the University Hospitals of Montpellier (n = 171), Nice (n = 138) and Marseille (n = 5), France, at the Mohamed V Military Hospital of Rabat (n = 16), Morocco, and at the Pasteur Institute of Tunis (n = 10), Tunisia. In total, 340 samples were analysed for this comparative study. All samples were sent to the CNRL for epidemiological and analytical purposes and stored at -20°C. Among the serum samples, 202 VL-positive samples were selected to reach a sensitivity of 95% and a precision of 3%. Positive samples were from patients who lived in the Mediterranean basin and displayed at least one clinical sign of VL (i. e., splenomegaly, hepatomegaly, anaemia leukopenia, or thrombocytopenia). VL diagnosis in these patients was confirmed by microscopic detection of the parasites, culture, or PCR analysis. To reach a minimum specificity of 90% and a precision of 5%, 138 negative serum samples were included. Negative samples were from immunocompetent patients who lived in the South of France, and did not have any typical symptom or previous history of VL.
## Ethics statement

This study was approved by the Committee of Research Ethics of the Montpellier University Hospital (IRB protocol number: 2019_IRB_MTP_03-03). Human serum samples were anonymised, and diagnostic tests were performed in blinded conditions. This work was carried out in accordance with the relevant guidelines and regulations, and does not provide any information that may allow the identification of the enrolled patients.

## Serological assays

Commercial kits available in France to detect human antibodies against *Leishmania* were used in this study. Four ELISA kits were included: NovaLisa *Leishmania infantum* IgG, Bordier *Leishmania infantum*, Ridascreen *Leishmania IgG*, and Vircell *Leishmania*. Sera were incubated in microplates, containing up to 96 wells coated with kits' specific *Leishmania* antigens, for 1h using Novalisa, 45min for Vircell, 30min for Bordier and 15 min for Ridascreen. Then, wells were washed and incubated with the conjugate for 30 min using Novalisa, Bordier and Vircell, and 15 min for Ridascreen. After a second washing step, plates were incubated with the substrate for 15 min using Novalisa and Ridascreen, 20 min for Vircell and 30min for Bordier. All four assays were carried out on the Evolis (BIO-RAD) automate and the absorbance was measured at 450 nm. Results were calculated as the sample absorbance value divided by the mean of the absorbance values of the cut-off controls, multiplied by ten for Novalisa, Vircell and Ridascreen. For the screening by ICT, samples were processed manually using the IT LEISH and TruQuick LEISH IgG/IgM Meridian tests, according to the manufacturers' instructions. For western blotting, the LDBio *Leishmania* IgG kit was used with the Autoblot 3000 (MedTEC) apparatus according to the manufacturer's instructions. Specific bands at 14 kDa or 16 kDa demonstrated the presence of anti-*Leishmania* antibodies in the sample. The characteristics of each test are listed in Table 1.

## Statistical analysis

The characteristics of the studied population were described using percentages and medians along with interquartile ranges instead of means and standard deviations when distributions were found to be non-Gaussian. Screening tests were evaluated against parasitological (i.e. microscopy, culture) and molecular diagnosis. Sensitivity and specificity were calculated using all the serum samples included in the study. The sensitivity rate was calculated as the number of patients with VL who tested positive divided by the total number of patients with VL (n = 181). The specificity rate was calculated as the number of non-VL patients who tested negative divided by the total number of non-VL patients (n = 138). The accuracy rate was calculated as the number of patients with VL who tested positive plus the number of non-VL patients who tested negative divided by the total number of patients tested (n = 319) [5]. Results were analysed using Receiving Operating Characteristic (ROC) curves to determine the most suitable threshold(s) for each quantitative screening test. Sensitivity and specificity values were compared by testing the equality of proportions. The areas under the curve (AUC) were compared with the χ2 test. The significance threshold was set at 5%. All statistical tests and procedures were performed using the Intercooled Stata 9.2 statistical package (StataCorp, College Station, TX).
<table><caption>**Table 1.** Characteristics of the commercial kits for the serological diagnosis of human visceral leishmaniasis. The asterisk corresponds to the total duration for the analysis of 96 wells.</caption><thead><tr><th>Commercial kit</th><th>Manufacturer</th><th>Method</th><th>Threshold</th><th>Sample type and volume</th><th>Time</th><th>Antigen</th></tr></thead><tbody><tr><td>NOVALISA® Leishmania infantum IgG</td><td>Novatec Immundiagnostica GMBH</td><td>ELISA</td><td>Negative: &lt;9<br>Equivocal: [9-11]<br>Positive: ≥11</td><td>Serum<br>10μL</td><td>3.5h*</td><td>L. infantum antigen</td></tr><tr><td>BORDIER® Leishmania infantum</td><td>Bordier Affinity products SA</td><td>ELISA</td><td>Negative: &lt;1<br>Positive: ≥1</td><td>Serum<br>5μL</td><td>3.5h*</td><td>Soluble antigens from L. infantum promastigotes</td></tr><tr><td>RIDASCREEN® Leishmania IgG</td><td>R-Biopharm AG</td><td>ELISA</td><td>Negative: &lt;9<br>Equivocal: [9-11]<br>Positive: ≥11</td><td>Serum<br>10μL</td><td>2.5h*</td><td>Recombinant L. infantum antigen</td></tr><tr><td>VIRCELL® Leishmania IgG/IgM</td><td>Vircell S.L.</td><td>ELISA</td><td>Negative: &lt;9<br>Equivocal: [9-11]<br>Positive: ≥11</td><td>Serum<br>5μL</td><td>2.5h*</td><td>L. infantum antigen</td></tr><tr><td>IT LEISH®</td><td>BIO-RAD Laboratories, Inc.</td><td>ICT</td><td>Positive: Control + IgG bands</td><td>Serum/Blood<br>8-12μL</td><td>25min</td><td>Recombinant K39 antigen</td></tr><tr><td>MERIDIAN® TruQuick™ LEISH IgG/IgM</td><td>Meridian Bioscience</td><td>ICT</td><td>Positive: Control + IgG and/or IgM bands</td><td>Serum/Blood<br>40μL</td><td>15min</td><td>Recombinant L. donovani antigen</td></tr><tr><td>Leishmania Western blot IgG</td><td>LD BIO Diagnostics</td><td>Western Blot</td><td>Positive: 14-kD and/or 16-kD band</td><td>Serum<br>25μL</td><td>3.5h</td><td>Antigens from L. infantum promastigotes</td></tr></tbody></table>

https://doi.org/10.1371/journal.pntd.0008139.t001

# Results

## Characteristics of the population, samples, and assays

We screened serum samples from patients living in France, Tunisia and Morocco. Among the VL population, the female to male sex ratio was 2:3, and age ranged from 1 to 93 years with a median of 33 years [IQR 28–44]. Because of the low quantity of some samples, among the 340 sera included in this study, 339 sera were screened with LDBio, 338 with IT-LEISH, 335 with TruQuick, 339 with Bordier and Ridascreen, 335 with NovaLisa, and 322 with Vircell. Among these sera, a complete dataset was available for 319 samples. The VL population (n = 181) included 98 immunocompetent and 65 immunocompromised patients (HIV: n = 49, transplantation: n = 4, immunosuppressive treatments n = 12), and 18 patients with unknown immune status who were included in the immunocompetent group (n = 116). S2 Fig presents the distribution of the index values obtained using the four ELISA kits, according to patients’ immune status, and shows that kits’ specific values cannot be compared to each other.

## Comparison of the tests’ performances

To compare the performance of the different ICT and ELISA kits, we assessed their sensitivity, specificity and accuracy in the 319 samples tested with all kits (Table 2). We interpreted the results of the four ELISA kits using the cut-off values for antibody detection recommended by the manufacturers (Table 1). The results that were found in the grey zone defined by the manufacturer of Vircell (n = 16), Ridascreen (n = 10) and NovaLisa (n = 11) were considered positive. Overall, the sensitivity and specificity of the ICT and ELISA kits ranged from 80.7% to 93.9% and from 95.7% to 100%, respectively. Among all tests, the highest values were obtained with the Bordier and Vircell kits, both displaying 96.2% of accuracy. When comparing the maximum likelihood ratio of ELISA calculated by the ROC curves, we found that the Bordier kit displayed significantly higher analytical performances than the Ridascreen and NovaLisa assays, whereas it was comparable to Vircell in the whole population (Fig 1). Conversely, the
four ELISA kits performed similarly when only samples from immunocompetent patients with VL were considered (S1 Table). In this population, the sensitivity of each test was improved (S2 Table). In immunocompromised patients, the overall distribution of the index values was significantly lower than in the immunocompetent population (S2 Fig). Moreover, in the VL-positive group, the number of false-negative results was significantly higher in immunocompromised patients than in the immunocompetent population (Table 3).

## Threshold adjustment of ELISA reagents

Sensitivity and specificity of 95% and 98%, respectively, are recommended for VL diagnosis [5,16]. However, in the whole population (n = 319), none of the ELISA kits reached 95% sensitivity using the thresholds recommended by the manufacturers, although specificity ranged from 96.38% to 100%, (Table 2). Therefore, we estimated the positivity thresholds to obtain a sensitivity of 95% with the highest possible specificity (Table 4). To reach these parameters, all cut-off values had to be reduced. For instance, for the Bordier kit, a decrease of the positive threshold from 1 to 0.746 gave a specificity of 99.3%. After adjusting the positive cut-off, the specificity of the Ridascreen and NovaLisa assays, were significantly lower than that of Bordier, while there was no statistical difference in between Bordier and Vircell.

## Results confirmation by western blot

Western blotting is used in the routine practice to confirm equivocal or positive results obtained by initial screening tests. Western blot analysis of the serum samples from patients with proven VL (n = 181) confirmed the results in 100% of cases (Fig 2). On the other hand, in the control population (n = 138), 12 serum samples were identified as positive by at least one ICT or ELISA kit. Analysis of these samples by western blotting indicated that seven samples were negative (n = 4 positive with NovaLisa, n = 2 positive with TruQuick, and n = 1 positive with Ridascreen, IT LEISH and TruQuick kits). On the other hand, we confirmed positive detection in five samples (n = 1 NovaLisa, n = 1 Vircell, n = 3 TruQuick). Therefore, we analysed all samples in the control group (n = 138) to assess what was the percentage of patients who did not present any symptoms of VL but displayed positive detection by Western blot. In total, we found 14 positive samples (10.1%) in the control population.

## Discussion

During the last 70 years, various methods have been used for the serodiagnosis of VL caused by *L. infantum* in humans [17], including commercial kits that require qualifications with specific information about their performance [5]. In this study, we evaluated the analytical performances of commercially available kits based on different methods: ELISA, ICT and WB. One of the limitations of this study is that we did not include immunocompromised patients in the

<table><caption>Table 2. Sensitivity, specificity and accuracy of the tested commercial kits.</caption><thead><tr><td>Manufacturer</td><td>Sensitivity (%) [95%CI]</td><td>Specificity (%) [95%CI]</td><td>Accuracy (%) [95%CI]</td><td>p (accuracy comparison)</td></tr></thead><tbody><tr><td>NOVALISA®</td><td>89.5% [86.1–92.9]</td><td>96.4% [94.3–98.4]</td><td>92.5% [89.7–95.4]</td><td>0.043</td></tr><tr><td>BORDIER®</td><td>93.4% [90.6–96.1]</td><td>100% [100–100]</td><td>96.2% [94.1–98.3]</td><td>Reference</td></tr><tr><td>RIDASCREEN®</td><td>80.7% [76.3–85.0]</td><td>99.3% [98.3–100.2]</td><td>88.7% [85.2–92.2]</td><td>&lt;0.001</td></tr><tr><td>VIRCELL®</td><td>93.9% [91.3–96.5]</td><td>99.3% [98.3–100.2]</td><td>96.2% [94.1–98.3]</td><td>1</td></tr><tr><td>IT LEISH®</td><td>85.1% [81.2–88.9]</td><td>99.3% [98.3–100.2]</td><td>91.2% [88.1–94.3]</td><td>0.009</td></tr><tr><td>TRUQUICK®</td><td>90.1% [86.8–93.3]</td><td>95.7% [93.4–97.9]</td><td>92.5% [89.7–95.4]</td><td>0.043</td></tr></tbody></table>

https://doi.org/10.1371/journal.pntd.0008139.t002
<figure><img src="image_5.png" /></figure>

<table><caption>Fig 1. Comparison of the ROC curves for the four ELISA assays.</caption><thead><tr><th>ELISA kit</th><th>AUC [95% CI]</th><th>p (AUC comparison)</th></tr></thead><tbody><tr><td>NOVALISA</td><td>0.9773 [0.9604-0.9942]</td><td>0.0197</td></tr><tr><td>BORDIER</td><td>0.9965 [0.9931-0.9999]</td><td>Reference</td></tr><tr><td>RIDASCREEN</td><td>0.9779 [0.9657-0.9901]</td><td>0.0013</td></tr><tr><td>VIRCELL</td><td>0.9946 [0.9892-1.000]</td><td>0.3008</td></tr></tbody></table>

https://doi.org/10.1371/journal.pntd.0008139.g001

<table><caption>Table 3. Comparison of false-negative results in patients with visceral leishmaniasis according to their immune status.</caption><thead><tr><th>Manufacturer</th><th>Immunocompetent (n = 116)</th><th>Immunocompromised (n = 65)</th><th>p</th></tr></thead><tbody><tr><td>NOVALISA®</td><td>7 (6.0%)</td><td>12 (18.5%)</td><td>0.009</td></tr><tr><td>BORDIER®</td><td>2 (1.7%)</td><td>10 (15.4%)</td><td>&lt;0.001</td></tr><tr><td>RIDASCREEN®</td><td>8 (6.9%)</td><td>27 (41.5%)</td><td>&lt;0.001</td></tr><tr><td>VIRCELL®</td><td>1 (0.9%)</td><td>10 (15.4%)</td><td>&lt;0.001</td></tr><tr><td>IT LEISH®</td><td>8 (6.9%)</td><td>19 (29.2%)</td><td>&lt;0.001</td></tr><tr><td>TRUQUICK®</td><td>6 (5.2%)</td><td>12 (18.5%)</td><td>0.004</td></tr></tbody></table>

https://doi.org/10.1371/journal.pntd.0008139.t003
<table><caption>Table 4. Sensitivity and specificity of the four ELISA assays using the proposed thresholds.</caption><thead><tr><th>ELISA kits</th><th>Manufacturer threshold</th><th>Threshold</th><th>Sensitivity (%) [95%CI]</th><th>Specificity (%) [95%CI]</th><th>p (specificity)</th></tr></thead><tbody><tr><td>Vircell®</td><td>9–11</td><td>7.686</td><td rowspan="4">95.03 [92.6–97.4]</td><td>97.83 [96.2–99.4]</td><td>0.062</td></tr><tr><td>NovaLisa®</td><td>9–11</td><td>5.604</td><td>91.30 [88.2–81.88]</td><td>&lt;0.001</td></tr><tr><td>Ridascreen®</td><td>9–11</td><td>1.491</td><td>81.88 [77.6–86.1]</td><td>&lt;0.001</td></tr><tr><td>Bordier®</td><td>1</td><td>0.746</td><td>99.28 [98.3–100]</td><td>Reference</td></tr></tbody></table>

https://doi.org/10.1371/journal.pntd.0008139.t004

healthy population. Consequently, we could not compare the tests' accuracy in function of the immune status. In addition, the control group did not include patients with diseases known to cross-react with anti-Leishmania antibodies (i.e. tuberculosis, trypanosomiasis, toxoplasmosis, leprosy or auto-immune diseases [3,7]). The use of a healthy population may have influenced the high specificity observed for all tests (>95%).

Among the six assays used for antibody screening, none reached the recommended sensitivity of 95% for VL diagnosis (14). Therefore, we lowered the positivity thresholds of the ELISA reagents to obtain a sensitivity of 95% with the highest possible specificity. As no grey zone is defined by the manufacturer of Bordier, we recommend interpreting the index values between 0.7 and 1 as equivocal or positive results. By using these cut-off values, the Bordier kit displayed the highest specificity (>99%). We propose to broaden the cut-off values

<figure><img src="image_6.png" /><figcaption>**Fig 2.** Confirmation of screening tests results by Western blot. The asterisk corresponds to the same patient who was found positive with Ridascreen, IT LEISH and TruQuick kits. Positive ELISA tests' results were defined according to the cut-off values recommended by the manufacturer.</figcaption></figure>

https://doi.org/10.1371/journal.pntd.0008139.g002
recommended by the Vircell manufacturer from 9–11 to 8–11, yielding a specificity comparable to that of Bordier. We also suggest reducing the NovaLisa positivity threshold from 9 to 6, although this would entail a significant decrease of specificity and the use of additional confirmation tests, compared with the Bordier and Vircell assays. Moreover, the cut-off values (1.5–11) of the Ridascreen kit required to achieve a sensitivity of 95% significantly reduced specificity (81.9%), therefore making this test less suitable for routine screening.

Few studies evaluated ICT performance for the diagnosis of zoonotic VL [16]. However, commercial ICT kits represent the best choice for rapid, easy and efficient serological screening of VL [17]. In this study, the performances of TruQuick and IT-LEISH were lower than those of the Bordier and Vircell ELISA kits, but their estimated accuracy was comparable to that of NovaLisa and higher than that of the Ridascreen ELISA kits in the whole population. ICT present several advantages compared with ELISA kits in term of practicability. First, TruQuick and IT-LEISH can be performed also on whole blood, thus facilitating screening in field settings. It should be noted that the performances of several commercial ICT kits on whole blood were found comparable to those obtained in serum samples [18]. Second, they provide qualitative results in 15–25 minutes compared to the 2.5–3.5 hours with ELISA kits. Moreover, the sample volume required for ICT is lower than for ELISA when using automated processors.

Recently, Freire *et al.* compared one ICT (IT LEISH) and three ELISA (Vircell, Novalisa and Ridascreen) kits in Brazil [5]. The overall specificity of these commercial kits (>95% in the whole population) was comparable in our study and in the work by Freire *et al.*, except for Ridascreen (specificity <85% in Brazil). Conversely, while Ridascreen displayed the highest sensitivity value in Brazil, we found that this test had the lowest sensitivity among all kits we evaluated. On the other hand, the sensitivity and specificity of the IT LEISH ICT were similar in both studies, although our values were slightly higher in the whole population. These differences might be explained by antigen variability in parasites from different geographic regions and by cross-reactivity with other endemic pathogens, as highlighted by the World Health Organization (WHO) evaluation of ICT [19]. Finally, like Freire *et al.* [5] and other works [6,7] that reported lower performances of serodiagnostic tests in immunocompromised patients, we found a higher number of false-negative results in immunocompromised individuals with VL. In Europe, a meta-analysis concluded that serological tests should not be used to rule out a diagnosis of VL among HIV-infected patients [20]. In a recent study in Spain, the sensitivity of the rk39-ICT dropped from 78% to 67.3% in this population [21]. These results support the use of molecular assays as an alternative method for VL diagnosis in immunocompromised patients.

Regarding WB, this technique is currently used to confirm the positive results obtained by serological screening. Its specificity should be discussed by taking into account the existence of asymptomatic carriers. The Leishmanin skin test [22], blood culture [23], and WB [11,12] could indeed detect infection or parasite contact in patient without VL. Here, 10.1% of the control population was positive by WB. This value is consistent with a larger study performed in Italy showing 7.41% of asymptomatic carriers by WB [24]. Thus, a positive result by WB should be discussed and interpreted on the basis of the biological and clinical data to discriminate between disease and asymptomatic carriage.

In conclusion, the Vircell and Bordier ELISA kits displayed significantly higher accuracy rates in the whole population than the other tests. Nevertheless, we had to decrease the cut-off values to improve the performance of the four ELISA tests. For field studies, the TruQuick ICT is the most suitable with satisfactory performances. In immunocompromised patients, the index values of all ELISA reagents were significantly lower than in the immunocompetent population and false-negative results were more frequent, as shown previously [20]. This kits comparison might help to select the most suitable method, using the most appropriate thresholds, and to better understand the limitations of each test.
**Supporting information**

**S1 Checklist.** STARD checklist for reporting of studies of diagnostic accuracy. The STARD checklist describes the design of the current study in order to improve reporting accuracy and completeness.

(DOCX)

**S1 Fig.** Diagram reporting the flow of participant through the study. P: positive; N: negative; I: indeterminate; Orange case: target condition present; Green case: target condition absent.

(TIF)

**S2 Fig.** Distribution of anti-Leishmania infantum antibody titres for the four ELISA assays according to patients' immune status.

(TIF)

**S1 Table.** Comparison of the ROC curves for the four ELISA assays in the immunocompetent population.

(DOCX)

**S2 Table.** Sensitivity, specificity and accuracy of the tested commercial kits in the immuno-competent population.

(DOCX)

**S3 Table.** Spreadsheet of raw data. Negative = 0; Positive = 1; Grey cell = unknown; Pink cell = immunosuppression; Orange cell = grey zone; Yellow cell = discrepancy.

(XLSX)

**Acknowledgments**

We are grateful to Patricia Gasq, Patrick Lami, Marie-Line Loiry for technical assistance and Nicolas Molinari for help in conception of the study.

**Author Contributions**

**Conceptualization:** Maude F. Lévêque, Judith Fillaux, Laurence Lachaud.

**Data curation:** Emilie Battery.

**Formal analysis:** Emilie Battery.

**Funding acquisition:** Laurence Lachaud.

**Investigation:** Pascal Delaunay, Badre Eddine Lmimouni, Karim Aoun, Coralie L'Ollivier, Charles Mary, Christelle Pomares.

**Methodology:** Judith Fillaux, Laurence Lachaud.

**Project administration:** Patrick Bastien, Laurence Lachaud.

**Resources:** Pascal Delaunay, Badre Eddine Lmimouni, Karim Aoun, Coralie L'Ollivier, Charles Mary, Christelle Pomares.

**Software:** Judith Fillaux.

**Supervision:** Patrick Bastien, Laurence Lachaud.

**Validation:** Maude F. Lévêque.

**Writing – original draft:** Maude F. Lévêque, Emilie Battery.
**Writing – review & editing:** Maude F. Lévêque, Judith Fillaux, Laurence Lachaud.

## References

1. Ready PD. Epidemiology of visceral leishmaniasis. Clin Epidemiol. 2014; 6: 147–154. https://doi.org/10.2147/CLEP.S44267 PMID: 24833919

2. van Griensven J, Diro E. Visceral Leishmaniasis: Recent Advances in Diagnostics and Treatment Regimens. Infect Dis Clin North Am. 2019; 33: 79–99. https://doi.org/10.1016/j.idc.2018.10.005 PMID: 30712769

3. Elmahallawy EK, Sampedro Martinez A, Rodriguez-Granger J, Hoyos-Mallecot Y, Agil A, Navarro Mari JM, et al. Diagnosis of leishmaniasis. J Infect Dev Ctries. 2014; 8: 961–972. https://doi.org/10.3855/jidc.4310 PMID: 25116660

4. Cruz I, Chicharro C, Nieto J, Bailo B, Cañavate C, Figueras M-C, et al. Comparison of new diagnostic tools for management of pediatric Mediterranean visceral leishmaniasis. J Clin Microbiol. 2006; 44: 2343–2347. https://doi.org/10.1128/JCM.02297-05 PMID: 16825347

5. Freire ML, Machado de Assis T, Oliveira E, Moreira de Avelar D, Siqueira IC, Barral A, et al. Performance of serological tests available in Brazil for the diagnosis of human visceral leishmaniasis. https://doi.org/10.1371/journal.pntd.0007484 2019; 13: e0007484. PMID: 31318856

6. Alvar J, Aparicio P, Aseffa A, Den Boer M, Cañavate C, Dedet J-P, et al. The relationship between leishmaniasis and AIDS: the second 10 years. Clin Microbiol Rev. 2008; 21: 334–359, table of contents. https://doi.org/10.1128/CMR.00061-07 PMID: 18400800

7. Deniau M, Cañavate C, Faraut-Gambarelli F, Marty P. The biological diagnosis of leishmaniasis in HIV-infected patients. Ann Trop Med Parasitol. 2003; 97 Suppl 1: 115–133. https://doi.org/10.1179/000349803225002598 PMID: 14678639

8. Duthie MS, Lison A, Courtenay O. Advances toward Diagnostic Tools for Managing Zoonotic Visceral Leishmaniasis. Trends Parasitol. 2018; 34: 881–890. https://doi.org/10.1016/j.pt.2018.07.012 PMID: 30131210

9. Antinori S, Calattini S, Longhi E, Bestetti G, Piolini R, Magni C, et al. Clinical use of polymerase chain reaction performed on peripheral blood and bone marrow samples for the diagnosis and monitoring of visceral leishmaniasis in HIV-infected and HIV-uninfected patients: a single-center, 8-year experience in Italy and review of the literature. Clin Infect Dis Off Publ Infect Dis Soc Am. 2007; 44: 1602–1610. https://doi.org/10.1086/518167 PMID: 17516404

10. de Ruiter CM, van der Veer C, Leeflang MMG, Deborggraeeve S, Lucas C, Adams ER. Molecular tools for diagnosis of visceral leishmaniasis: systematic review and meta-analysis of diagnostic test accuracy. J Clin Microbiol. 2014; 52: 3147–3155. https://doi.org/10.1128/JCM.00372-14 PMID: 24829226

11. Heidari S, Gharechahi J, Mohebali M, Akhoundi B, Mirshahvaladi S, Azarian B, et al. Western Blot Analysis of Leishmania infantum Antigens in Sera of Patients with Visceral Leishmaniasis. Iran J Parasitol. 2019; 14: 10–19. PMID: 31123464

12. Mary C, Lamouroux D, Dunan S, Quilici M. Western blot analysis of antibodies to Leishmania infantum antigens: potential of the 14-kD and 16-kD antigens for diagnosis and epidemiologic purposes. Am J Trop Med Hyg. 1992; 47: 764–771. https://doi.org/10.4269/ajtmh.1992.47.764 PMID: 1281966

13. World Health Organization, Weltgesundheitsorganisation, editors. Control of the leishmaniasis: report of a meeting of the WHO Expert Committee on the Control of Leishmaniasis, Geneva, 22–26 March 2010. Geneva: World Health Organization; 2010.

14. Kar K. Serodiagnosis of leishmaniasis. Crit Rev Microbiol. 1995; 21: 123–152. https://doi.org/10.3109/10408419509113537 PMID: 7639932

15. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ. 2015; 351: h5527. https://doi.org/10.1136/bmj.h5527 PMID: 26511519

16. Boelaert M, Verdonck K, Menten J, Sunyoto T, van Griensven J, Chappuis F, et al. Rapid tests for the diagnosis of visceral leishmaniasis in patients with suspected disease. Cochrane Database Syst Rev. 2014; CD009135. https://doi.org/10.1002/14651858.CD009135.pub2 PMID: 24947503

17. Maia Z, Lírio M, Mistro S, Mendes CMC, Mehta SR, Badaro R. Comparative Study of rK39 Leishmania Antigen for Serodiagnosis of Visceral Leishmaniasis: Systematic Review with Meta-Analysis. PLoS Negl Trop Dis. 2012; 6: e1484. https://doi.org/10.1371/journal.pntd.0001484 PMID: 22303488

18. Kumar D, Khanal B, Tiwary P, Mudavath SL, Tiwary NK, Singh R, et al. Comparative evaluation of blood and serum samples in rapid immunochromatographic tests for visceral leishmaniasis. J Clin Microbiol. 2013; 51: 3955–3959. https://doi.org/10.1128/JCM.01232-13 PMID: 24048530
19. TDR | Visceral Leishmaniasis Rapid Diagnostic Test Performance. In: WHO [Internet]. [cited 16 Sep 2019]. Available: https://www.who.int/tdr/publications/tdr-research-publications/vl-rdt-evaluation/en/

20. Cota GF, de Sousa MR, Demarqui FN, Rabello A. The diagnostic accuracy of serologic and molecular methods for detecting visceral leishmaniasis in HIV infected patients: meta-analysis. PLoS Negl Trop Dis. 2012; 6: e1665. https://doi.org/10.1371/journal.pntd.0001665 PMID: 22666514

21. Bangert M, Flores-Chávez MD, Llanes-Acevedo IP, Arcones C, Chicharro C, García E, et al. Validation of rK39 immunochromatographic test and direct agglutination test for the diagnosis of Mediterranean visceral leishmaniasis in Spain. PLoS Negl Trop Dis. 2018; 12: e0006277. https://doi.org/10.1371/journal.pntd.0006277 PMID: 29494596

22. Bekele F, Belay T, Zeynudin A, Hailu A. Visceral leishmaniasis in selected communities of Hamar and Banna-Tsamai districts in Lower Omo Valley, South West Ethiopia: Sero-epidemological and Leishmanin Skin Test Surveys. PLoS One. 2018; 13: e0197430. https://doi.org/10.1371/journal.pone.0197430 PMID: 29795589

23. le Fichoux Y, Quaranta JF, Aufeuvre JP, Lelievre A, Marty P, Suffia I, et al. Occurrence of Leishmania infantum parasitemia in asymptomatic blood donors living in an area of endemicity in southern France. J Clin Microbiol. 1999; 37: 1953–1957. PMID: 10325353

24. Biglino A, Bolla C, Concialdi E, Trisciuoglio A, Romano A, Ferroglio E. Asymptomatic Leishmania infantum infection in an area of northwestern Italy (Piedmont region) where such infections are traditionally nonendemic. J Clin Microbiol. 2010; 48: 131–136. https://doi.org/10.1128/JCM.00416-09 PMID: 19923480