# A multi-country phase 2 study to evaluate the suitcase lab for rapid detection of SARS-CoV-2 in seven Sub-Saharan African countries: Lessons from the field

Arianna Ceruti, Ndongo Dia, Adeleye Solomon Bakarey, Judah Ssekitoleko, Soa Fy Andriamandimby, Padra Malwengo-Kasongo, Rasheeda H.A. Ahmed, Rea Maja Kobialka, Jean Michel Heraud, Moussa Moise Diagne, et al.

<figure><img src="image_1.png" /></figure>

## To cite this version:

Arianna Ceruti, Ndongo Dia, Adeleye Solomon Bakarey, Judah Ssekitoleko, Soa Fy Andriamandimby, et al.. A multi-country phase 2 study to evaluate the suitcase lab for rapid detection of SARS-CoV-2 in seven Sub-Saharan African countries: Lessons from the field. Journal of Clinical Virology, 2023, 162, pp.105422. 10.1016/j.jcv.2023.105422. pasteur-04096796

HAL Id: pasteur-04096796

https://riip.hal.science/pasteur-04096796

Submitted on 13 May 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>
<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

Check for
updates

A multi-country phase 2 study to evaluate the suitcase lab for rapid detection of SARS-CoV-2 in seven Sub-Saharan African countries: Lessons from the field

Arianna Ceruti<sup>a,*</sup>, Ndongo Dia<sup>b,1</sup>, Adeleye Solomon Bakarey<sup>f,1</sup>, Judah Ssekitoleko<sup>m,1</sup>, Soa Fy Andriamandimby<sup>c,1</sup>, Padra Malwengo-Kasongo<sup>i,r,1</sup>, Rasheeda H.A. Ahmed<sup>j,1</sup>, Rea Maja Kobialka<sup>a</sup>, Jean Michel Heraud<sup>b</sup>, Moussa Moise Diagne<sup>b</sup>, Marie Henriette Dior Dione<sup>b</sup>, Idrissa Dieng<sup>b</sup>, Martin Faye<sup>b</sup>, Ousmane Faye<sup>b</sup>, Jean Théophile Rafisandratantsoa<sup>c</sup>, Jean-Pierre Ravalohery<sup>c</sup>, Claudio Raharinandrasana<sup>c</sup>, Tsiry H. Randriambolamanantsoa<sup>c</sup>, Norosoa Razanajatovo<sup>c</sup>, Iony Razanatovo<sup>c</sup>, Joelinotahina H. Rabarison<sup>c</sup>, Phillipe Dussart<sup>c</sup>, Louis Kyei-Tuffuor<sup>e</sup>, Abigail Agbanyo<sup>e</sup>, Olubusuyi Moses Adewumi<sup>g</sup>, Adeola Fowotade<sup>h</sup>, Muideen Kolawole Raifu<sup>f</sup>, Patient Okitale-Talunda<sup>i,r</sup>, Gracia Kashitu-Mujinga<sup>i,r</sup>, Christelle Mbelu-Kabongo<sup>i,r</sup>, Steve Ahuka-Mundeke<sup>i,r</sup>, Anguy Makaka-Mutondo<sup>i,r</sup>, Enas M. Abdalla<sup>k</sup>, Sanaa M. Idris<sup>k</sup>, Wisal A. Elmagzoub<sup>k</sup>, Rahma H. Ali<sup>j</sup>, Eman O.M. Nour<sup>j</sup>, Rasha S.M. Ebraheem<sup>j</sup>, Huda H.H. Ahmed<sup>j</sup>, Hamadelniel E. Abdalla<sup>j</sup>, Musab Elnegoumi<sup>j</sup>, Izdihar Mukhtar<sup>j</sup>, Muatsim A.M. Adam<sup>j</sup>, Nuha Y.I. Mohamed<sup>j</sup>, Shahinaz A. Bedri<sup>j</sup>, Hamdan Mustafa Hamdan<sup>l</sup>, Magid Kisekka<sup>m</sup>, Monica Mpumwiire<sup>m</sup>, Sharley Melissa Aloyo<sup>n</sup>, Joanita Nabwire Wandera<sup>n</sup>, Andrew Agaba<sup>n</sup>, Rogers Kamulegeya<sup>n</sup>, Hosea Kiprotich<sup>n</sup>, David Patrick Kateete<sup>n</sup>, Paul Kadetz<sup>p</sup>, Uwe Truyen<sup>a</sup>, Kamal H. Eltom<sup>k,2</sup>, Anavaj Sakuntabhai<sup>q</sup>, Julius Boniface Okuni<sup>m,2</sup>, Sheila Makiala-Mandanda<sup>i,r,2</sup>, Vincent Lacoste<sup>c,2</sup>, George Olusegun Ademowo<sup>f,2</sup>, Michael Frimpong<sup>d,e,2</sup>, Amadou Alpha Sall<sup>b,2</sup>, Manfred Weidmann<sup>o</sup>, Ahmed Abd El Wahed<sup>a,2</sup>

<footnote><sup>a</sup> Institute of Animal Hygiene and Veterinary Public Health, Leipzig University, Germany</footnote>

<footnote><sup>b</sup> Virology Department, Institut Pasteur de Dakar, Dakar 12900, Senegal</footnote>

<footnote><sup>c</sup> Virology Unit, Institut Pasteur de Madagascar, Republic of Madagascar</footnote>

<footnote><sup>d</sup> Department of Molecular Medicine, School of Medicine and Dentistry, Kwame Nkrumah University of Science and Technology, Ghana</footnote>

<footnote><sup>e</sup> Kumasi Centre for Collaborative Research, Kwame Nkrumah University of Science and Technology, Ghana</footnote>

<footnote><sup>f</sup> Institute for Advanced Medical Research and Training (IAMRAT), College of Medicine, University of Ibadan, Nigeria</footnote>

<footnote><sup>g</sup> Department of Virology, College of Medicine, University of Ibadan, Nigeria</footnote>

<footnote><sup>h</sup> Department of Medical Microbiology and Parasitology, College of Medicine, University of Ibadan, Nigeria</footnote>

<footnote><sup>i</sup> Department of Virology at the Institut National de Recherche Biomédicale (INRB), Kinshasa, Democratic Republic of the Congo (DRC)</footnote>

<footnote><sup>j</sup> Tuberculosis National Reference Laboratory, National Public Health Laboratory, El Qasr Avenue, Federal Ministry of Health, Khartoum, Sudan</footnote>

<footnote><sup>k</sup> Unit of Animal Health and Safety of Animal Products, Institute for Studies and Promotion of Animal Exports, University of Khartoum, 13314 Khartoum North, Sudan</footnote>

<footnote><sup>l</sup> MDR Focal Point, El Qasr Avenue, Federal Ministry of Health, Khartoum, Sudan</footnote>

<footnote><sup>m</sup> College of Veterinary Medicine, Animal Resources and Biosecurity, Makerere University, P.o.Box 7062 Uganda</footnote>

<footnote><sup>n</sup> Department of Immunology and Molecular Biology, College of Health Sciences, Makerere University, Uganda</footnote>

<footnote><sup>o</sup> Institute of Microbiology and Virology, Medizinische Hochschule Brandenburg Theodor Fontane, Germany</footnote>

<footnote><sup>p</sup> Institute for Global Health and Development, Queen Margaret University, United Kingdom</footnote>

<footnote><sup>q</sup> Functional Genetics of Infectious Diseases Unit, Institut Pasteur, France</footnote>

<footnote><sup>r</sup> Faculty of Medicine, University of Kinshasa, Democratic Republic of the Congo (DRC)</footnote>

<footnote>* Corresponding author.
E-mail address: arianna.ceruti@uni-leipzig.de (A. Ceruti).</footnote>

<footnote n="1">¹ First joint authorship</footnote>

<footnote n="2">² Shared senior authorship</footnote>
ARTICLE INFO

Keywords:

SARS-CoV-2

Diagnostics-in-a-suitcase

Recombinase polymerase amplification assay

A B S T R A C T

**Background:** The COVID-19 pandemic led to severe health systems collapse, as well as logistics and supply de-
ivery shortages across sectors. Delivery of PCR related healthcare supplies continue to be hindered. There is the
need for a rapid and accessible SARS-CoV-2 molecular detection method in low resource settings.

**Objectives:** To validate a novel isothermal amplification method for rapid detection of SARS-CoV-2 across seven sub-Sharan African countries.

**Study design:** In this multi-country phase 2 diagnostic study, 3,231 clinical samples in seven African sites were tested with two reverse transcription Recombinase-Aided Amplification (RT-RAA) assays (based on SARS-CoV-2 Nucleocapsid (N) gene and RNA-dependent RNA polymerase (RdRP) gene). The test was performed in a mobile suitcase laboratory within 15 min. All results were compared to a real-time RT-PCR assay. Extraction kits based on silica gel or magnetic beads were applied.

**Results:** Four sites demonstrated good to excellent agreement, while three sites showed fair to moderate results. The RdRP gene assay exhibited an overall PPV of 0.92 and a NPV of 0.88. The N gene assay exhibited an overall PPV of 0.93 and a NPV 0.88. The sensitivity of both RT-RAA assays varied depending on the sample Ct values. When comparing sensitivity between sites, values differed considerably. For high viral load samples, the RT-RAA assay sensitivity ranges were between 60.5 and 100% (RdRP assay) and 25 and 98.6 (N assay).

**Conclusion:** Overall, the RdRP based RT-RAA test showed the best assay accuracy. This study highlights the challenges of implementing rapid molecular assays in field conditions. Factors that are important for successful deployment across countries include the implementation of standardized operation procedures, in-person continuous training for staff, and enhanced quality control measures.

1. Background

In March 2020, the World Health Organization (WHO) declared COVID-19 caused by severe acute respiratory syndrome coronavirus type 2 (SARS-CoV-2) as a global pandemic [1]. Early detection of infected cases is still regarded as essential to reduce the disease burden [2].

Real-time reverse transcription polymerase chain reaction (RT-PCR) is the standard approach in terms of sensitivity and accuracy. However, this technique requires well-established laboratory. Additionally, supply and delivery shortages have been reported across various sectors [3,4]. Antigen lateral flow tests were quickly developed and deployed, allowing carriers with high viral load to be diagnosed more easily. However, these tests often do not reach the WHO recommended minimum of >80% sensitivity and >97% specificity for new diagnostics tests [5]. Deployment of a simple, rapid molecular test method could offer marked advantages.

Recombinase-Polymerase/aided Amplification (RPA/RAA) assays have been described as a rapid and effective nucleic acid amplification technique, due to its simplicity and fast sample-to-result test time [6]. RPA/RAA is an isothermal probe-based nucleic acid detection method that neither requires template denaturation nor primer annealing steps [7]. The proper selection of a polymerase working at low temperature (39-42 °C), the robustness of the assay, and compactness of the fluorescence detection device make RPA/RAA an optimal technique for molecular diagnosis at the point of need. To enable the widespread use of the technology, mobile suitcase laboratories were deployed to many sub-Saharan African countries [8].

2. Objectives

To further clinically evaluate the system in real-life settings, a multi-
country single blinded phase 2 study was conducted in seven sub-
Saharan African countries. The aim was not only to determine the ac-
curacy of RAA assay for detection of SARS-CoV-2 in local African set-
tings, but also to assess performance differences between research
institutions.

3. Material and methods

3.1. Sample size calculation

The sample size was calculated using the formula for comparing two independent proportions, which is used to estimate sample size for studies comparing sensitivity and/or specificity of two tests of unpaired design [9]. A minimum of 300 samples per site were required to achieve 95% confidence.

3.2. Study site and population

A multi-country, single blinded, phase 2 diagnostic evaluation study was conducted in seven sites: Institut Pasteur de Dakar (IPD), Senegal; Institut Pasteur de Madagascar (IPM), Madagascar; Kumasi Centre for Collaborative Research (KCCR), Ghana; University of Ibadan (UI), Nigeria; Institut National de Recherche Biomédicale (INRB), Democratic Republic of the Congo; University of Khartoum (UofK), Sudan; and Makerere University (MAK), Uganda. For the purpose of this study, a total of 3231 archived samples were used to evaluate the assay. Archived samples included in this study were from patients who tested positive for SARS-CoV-2 by the reference laboratory and patients who were suspected COVID-19 and tested negative by the reference laboratory. All samples were tested irrespective of age, sex, and race. During the study, all samples were handled anonymously. Samples included nasal, mid-turbinate swab or saliva in viral transport media (VTM), PBS or stored dry and maintained in -80 °C. Additional information regarding the samples was included in supplementary file #1.

3.3. Study design

This study was conducted according to the guidelines for diagnostic kit evaluation (Fig. 1) [9].

Each country initially tested an inactivated test panel of 20 samples using RT-RAA and real-time RT-PCR reagents to assure their preparedness to perform the tests (Fig. 2). Once successful (at least 90% accurate detection rate), each country started clinical sample testing as follows: all samples were labelled with random numbers by the site principal investigator, and the laboratory personnel were divided into two teams: one team performed the real-time RT-PCR and the other performed the RT-RAA. Both teams were blinded and did not know whether the sample was from a positive or negative subject. Simultaneously, the three parts
carbonless copy paper-based laboratory report forms (LRFs) were made available in two modes, one by the unblinded staff and the other by the blinded staff (supplementary file #2). All laboratory data were reported in LRFs by both teams separately. The data was then decoded by the data management team, where it was merged and statistically analysed. Patient information was not shared between study sites, only positivity and negativity rates were recorded. To ensure the quality of the study activities, periodic monitoring was performed.

### 3.4. Mobile suitcase laboratory set up

The total set up consisted of a Glove Box (Bodo Koennecke, Berlin, Germany) and a mobile suitcase laboratory (Fig. 3). The Glove Box protects the technician while handling and inactivating the sample before nucleic acid extraction. The isothermal amplification test was performed in the Mobile Suitcase Lab (Fig. 4).

### 3.5. Laboratory analysis

#### 3.5.1. Real-time RT-PCR assay for detection of SARS-CoV-2 RNA

RNA was isolated from clinical samples of subjects, who were recently suspected of contracting COVID-19. At KCCR, UofK, UI and INRB, the QIAamp Viral RNA Mini Kit (Qiagen, Hilden, Germany) was used. At MAK, Liferiver Viral DNA/RNA isolation kit (Shanghai ZJ Biotech co. let, Shanghai, China) was used. At IPD, the Veri-Q PREP M16 Automatic Nucleic Acid Extraction System (MiCo BioMed, South Korea) was used. IPM used the NucleoSpin Dx Virus Mini kit (Macherey-Nagel GmbH, Düren, Germany).

All samples were originally tested as part of routine diagnostics and divided into positive and negative according to the locally used real-time RT-PCR results (supplementary file #2). To assure integrity of RNA on the day of testing, an additional real-time RT-PCR was performed. This result was considered as reference method (Fig. 2). The archived patient material was tested at each site with a commercially available real-time RT-PCR assay combining oligonucleotide Lightmix Modular SARS-CoV-2 RNA-dependent RNA polymerase (RdRP) and the lyophilized one-step RT-PCR Polymerase Mix kit from TIB MOLBIOL (Berlin, Germany) according to the manufacturer's instructions. The real-time RT-PCR reaction comprised 15 µL, including 0.5 µL oligonucleotide mix, 4 µL PCR-grade water, 10 µL qPCR Master mix and 5 µL

<figure><img src="image_5.png" /><figcaption>**Fig. 2.** Workflow of the performed tests at the study sites. The new real-time RT-PCR performed was used as the reference method to validate the RT-RAA assays.</figcaption></figure>

template or control. The following real-time PCR cycler was used: Bio-Rad CFX96 Touch (Bio-Rad Laboratories, Hercules, United States) at IPD, KCCR and UI, Rotorgene Q (Qiagen, Hilden, Germany) at IPM, Applied Biosystems 7500 Fast (Thermo Fisher Scientific, Waltham, United States) at INRB, qTower (Biometra, Analytik Jena, Jena, Germany) at UofK and Applied Biosystems Quantstudio 7flex (Thermo Fisher Scientific, Waltham, United States) at MAK.

#### 3.5.2. RT-RAA assay for detection of SARS-COV-2 RNA

Two RT-RAA assays were evaluated: one based on the RdRP gene and one on the Nucleocapsid (N) gene. The primers, probe and reaction conditions for SARS-CoV-2 genes N and RdRP RT-RAA assays were

<figure><img src="image_6.png" /><figcaption>**Fig. 1.** Schematic representation of the multi-country, single blinded, phase 2 study to test archived samples with the RT-RAA assay for SARS-CoV-2. Institut Pasteur de Dakar (IPD); Institut Pasteur de Madagascar (IPM); Kumasi Centre for Collaborative Research (KCCR); University of Ibadan (UI); Institut National de Recherche Biomédicale (INRB); University of Khartoum (UofK); and Makerere University (MAK).</figcaption></figure>
<figure><img src="image_7.png" /><figcaption>Fig. 3. Example of the suitcase lab, which is fully equipped to perform molecular tests in the field.</figcaption></figure>

based on a previous study [8] Primer and probe were synthesized by TIB MOLBIOL (Berlin, Germany). The RT-RAA nucleic acid amplification kit (Fluorescent RT-RAA) from Jiangsu Qitian Gene Biotechnology Co. (Wuxi, China) was used. The kit comprises lyophilized enzymes, including the reverse transcriptase necessary for RNA amplification. The RT-RAA reaction total volume was 50 µL including 21.5 µL of the oligonucleotide mix (21 pMol for forward primer, 42 pMol for reverse primer and 6 pMol for exo-probe), 25 µL rehydration buffer, 2.5 µL Magnesium Acetate and 1 µL template or control. The mix was added into the lid of the reaction tube containing the freeze-dried pellet. The tube was closed, centrifuged, mixed, centrifuged, and placed immediately into the isothermal device: UofK and IPM used the TwistDx TS1 device (Cambridge, UK). UI used the Qiagen ESEquant TS2 model (Hilden, Germany), while all other countries used the Axxin T8 Isothermal instrument (Fairfield, Australia). The reaction was incubated at 42 °C for 15 min. A mixing step was conducted after 320 s for the N gene assay and after 230 s for the RdRP gene assay. For signal interpretation, a combined threshold time (TT) and first derivative analysis was used with the corresponding software of each device.

### 3.6. Data analysis

Standard formulas were used with MedCalc [10] to determine the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) [11]. Cohen's kappa coefficient (k) and McNemar's test were performed to determine the concordance and discordance between the RT-RAA assays and real-time RT-PCR-based method. The values of Cohen's kappa coefficients were interpreted according to Landis and Koch [12] and calculated using the online Graphpad version. The McNemar test was calculated via https://epitools.ausvet.com.au/mcnemar. A P-value <0.05 was considered to be statistically

significant.

### 3.7. Mutational analysis of the RdRP amplicon

Over 10,000 SARS-CoV-2 RNA sequences from local strains in the involved countries were screened for potential mutations in the targeted RdRP region. A previously published pipeline was used [13].

## 4. Results

### 4.1. Performance of the deployed isothermal amplification assays

In total, 3231 samples were identified at study sites (Fig. 1) to validate the assays. After data curation, the N gene assay results were evaluated with a total of 1890 negative samples and 580 positive samples and the RdRP gene assay with a total of 2326 negative samples and 868 positive samples (supplementary file #3 displaying sample flowchart and raw data). The RdRP gene assay showed an overall PPV of 0.92 and a NPV of 0.88. The N gene assay showed an overall PPV of 0.93 and a NPV 0.88.

The sensitivity of both RT-RAA assays varied depending on the sample Ct values (Table 1). Real-time RT-PCR positive samples with high viral load (Ct <30) showed the best results with 90.8% and 81.8% overall sensitivities for the RdRP and N gene RT-RAA assays, respectively. When comparing sensitivity between sites, values differed considerably. For high viral load samples, the RdRP assay sensitivity ranged between 60.5 and 100%. The N gene did not perform as well as the RdRP gene, with a sensitivity between 25 and 98.6%. The specificity among all study sites ranged from 91.1 to 100% for the RdRP gene RT-RAA assay and from 94.3 to 100% for the N gene RT-RAA assay (Table 2). The overall sensitivity and specificity with both targets
<figure><img src="image_8.png" /><figcaption>Fig. 4. The suitcase lab at various study sites.</figcaption></figure>

<table><caption>**Table 1**

Overall clinical sensitivity of all samples across sites categorized according to the Ct values of real-time RT-PCR.</caption><thead><tr><th>RT-PCR Ct range</th><th>Target gene for RT-RAA</th><th>Overall Sensitivity (%)</th><th>RT-PCR positive</th><th>RT-RAA positive</th></tr></thead><tbody><tr><td rowspan="2">0–30</td><td>RdRP</td><td>90.8 (82.6 – 99.7)</td><td>493</td><td>448</td></tr><tr><td>N</td><td>81.8 (72.5 – 92)</td><td>341</td><td>279</td></tr><tr><td rowspan="2">≥30–35</td><td>RdRP</td><td>45.6 (36.3 – 56.5)</td><td>182</td><td>83</td></tr><tr><td>N</td><td>41.8 (30.98 – 55.4)</td><td>117</td><td>49</td></tr><tr><td rowspan="2">≥35–40</td><td>RdRP</td><td>13.5 (9 – 19.7)</td><td>193</td><td>26</td></tr><tr><td>N</td><td>12.3 (6.9 – 20.2)</td><td>122</td><td>15</td></tr><tr><td rowspan="2">Total</td><td>RdRP</td><td>64.2 (58.9 – 69.7)</td><td>868</td><td>557</td></tr><tr><td>N</td><td>59.1 (53 – 65.7)</td><td>580</td><td>343</td></tr></tbody></table>

combined was 58.2% and 99.2%, respectively. The agreement between test methods varied across sites (Table 3). Sites IPD, KCCR, UI and INRB showed good to excellent agreement, while sites IPM, UofK and MAK showed fair to moderate results.

## 4.2. Mutational analysis of the amplicon

No significant mutations were found in the RdRP amplicon across

countries (supplementary files #4 and #5).

## 5. Discussion

The worldwide spread of SARS-CoV-2 has taken viral diagnostics to a new level of importance and publicity. Alternative methods to real-time RT-PCR with equal sensitivity and specificity are urgently needed to overcome shortage in supply chains [14]. The WHO recommended the ASSURED criteria (affordable, sensitive, specific, user-friendly, robust, deliverable to end-users) for future diagnostics [15]. Isothermal amplification assays address most of these criteria as shown in many outbreak situations [16–18]. Although clinical research is rapidly progressing in the field of new diagnostic tests, multi-country approaches for SARS-CoV-2 are lacking, especially in sub-Saharan Africa [40]. Only one multicenter study was reported using RT-loop mediated amplification in four African countries in both east and west Africa with very promising sensitivity of 87% [19].

Using the RT-RAA technology helped to circumvent the worldwide supply shortages of real-time RT-PCR test kits [20,21]. The specificity for the two SARS-CoV-2 genome targets across sites ranged from 91.1 to 100% (RdRP gene RT-RAA) and 94.3 – 100% (N gene RT-RAA). In contrast the SARS-CoV-2 an E gene target RT-RAA, showed a high number of false positive results in a previous study and was not included in the current screening [22]. Detecting true negative samples accurately at high specificity avoids unnecessary clinical implications and social upset [23–25]. SARS-CoV-2 Rapid antigen tests have shown a higher false positive rate (96–99.7%) [26], and especially in low prevalence settings a molecular confirmatory test is needed [27].

When deploying both RT-RAA assays in this study, sensitivity
<caption>**Table 2**</caption>

Clinical sensitivity and specificity at each study site. NA is not applicable as the site did not perform the assay. The 95% confidence interval is showed in parenthesis.

<table><caption>**Table 3**</caption><thead><tr><th rowspan="2">Study site</th><th colspan="4">RdRP sensitivity (%)</th><th rowspan="2">RdRP specificity (%)</th><th colspan="4">N sensitivity (%)</th><th rowspan="2">N specificity (%)</th></tr><tr><th>&lt;30 Ct</th><th>30-35 Ct</th><th>&gt;35 Ct</th><th>Overall</th><th>&lt;30 Ct</th><th>30-35 Ct</th><th>&gt;35 Ct</th><th>Overall</th></tr></thead><tbody><tr><td>IPD</td><td>100<br>(95 - 99.9)</td><td>75<br>(47.6 - 92.7)</td><td>21.4<br>(4.7 - 50.8)</td><td>85.3<br>(76.9 - 91.5)</td><td>99<br>(97.4 - 99.7)</td><td>98.6<br>(92.5 - 99.9)</td><td>81.2<br>(54.3 - 95.9)</td><td>14.3<br>(1.8 - 42.8)</td><td>84.3<br>(75.8 - 90.8)</td><td>99<br>(97.8 - 99.8)</td></tr><tr><td>IPM</td><td>94.4<br>(89.3 - 97.5)</td><td>23.9<br>(12.6 - 38.7)</td><td>4.2<br>(0.5 - 14.2)</td><td>62.4<br>(55.9 - 68.6)</td><td>93.5<br>(89.8 - 96.2)</td><td>82.5<br>(75.3 - 88.3)</td><td>10.8<br>(3.6 - 23.6)</td><td>12.5<br>(4.7 - 25.2)</td><td>54.4<br>(47.9 - 60.9)</td><td>94.3<br>(90.8 - 96.8)</td></tr><tr><td>KCCR</td><td>95.6<br>(78 - 99.9)</td><td>84<br>(63.9 - 95.5)</td><td>47.6<br>(25.7 - 70.2)</td><td>76.8<br>(65 - 86.1)</td><td>99.7<br>(98.7 - 100)</td><td>91.3<br>(72 - 98.9)</td><td>68<br>(46.5 - 85)</td><td>19<br>(5.4 - 41.9)</td><td>60.8<br>(48.4 - 72.4)</td><td>99<br>(97.6 - 99.7)</td></tr><tr><td>UI</td><td>95.5<br>(84.8 - 99.5)</td><td>90<br>(55.5 - 99.7)</td><td>42.9<br>(9.9 - 81.6)</td><td>88.7<br>(78.1 - 95.3)</td><td>100<br>(99.1 - 100)</td><td>95.5<br>(84.8 - 99.5)</td><td>90<br>(55.5 - 99.7)</td><td>42.9<br>(9.9 - 81.6)</td><td>88.7<br>(78.1 - 95.3)</td><td>100<br>(99.1 - 100)</td></tr><tr><td>INRB</td><td>89.2<br>(79.8 - 95.2)</td><td>50<br>(28.2 - 71.8)</td><td>3.6<br>(0.1 - 18.3)</td><td>62.9<br>(53.8 - 71.4)</td><td>99.6<br>(97.9 - 100)</td><td>25<br>(8.7 - 49.1)</td><td>16.6<br>(0.4 - 64.1)</td><td>0<br>(0 - 70.7)</td><td>20.4<br>(8 - 39.7)</td><td>100<br>(96 - 100)</td></tr><tr><td>UofK</td><td>60.5<br>(40.4 - 76)</td><td>23<br>(5 - 53.8)</td><td>14.3<br>(4 - 32.7)</td><td>38<br>(27.3 - 49.6)</td><td>91.1<br>(87.2 - 94.3)</td><td>55.3<br>(38.3 - 71.4)</td><td>28.6<br>(8.4 - 58.1)</td><td>0<br>(0 - 30.9)</td><td>30.8<br>(21.1 - 42.1)</td><td>98.5<br>(96.4 - 99.6)</td></tr><tr><td>MAK</td><td>87.9<br>(79.8 - 95.6)</td><td>32<br>(19.5 - 46.7)</td><td>6.5<br>(1.4 - 18)</td><td>54.3<br>(47.09 - 61.5)</td><td>99.3<br>(97.4 - 99.9)</td><td>NA</td><td>NA</td><td>NA</td><td>NA</td><td>NA</td></tr></tbody></table>

Agreement between real-time RT-PCR and RT-RPA assays at different sites. NA is not applicable as the site did not perform the assay. Fair = 0.21 – 0.40; moderate = 0.41 – 0.60; good = 0.61 – 0.80; excellent: 0.81 – 1.

<table><thead><tr><th>Study site</th><th>Kappa value RdRp</th><th>Agreement</th><th>p-value RdRP</th><th>Kappa value N</th><th>Agreement</th><th>p-value N</th></tr></thead><tbody><tr><td>IPD</td><td>0.878</td><td>Excellent</td><td>0.022</td><td>0.854</td><td>Excellent</td><td>&lt;0.001</td></tr><tr><td>IPM</td><td>0.568</td><td>Moderate</td><td>&lt;0.001</td><td>0.497</td><td>Moderate</td><td>&lt;0.001</td></tr><tr><td>KCCR</td><td>0.843</td><td>Excellent</td><td>&lt;0.001</td><td>0.697</td><td>Good</td><td>&lt;0.001</td></tr><tr><td>UI</td><td>0.901</td><td>Excellent</td><td>0.03</td><td>0.901</td><td>Excellent</td><td>0.03</td></tr><tr><td>INRB</td><td>0.696</td><td>Good</td><td>&lt;0.001</td><td>0.283</td><td>Fair</td><td>&lt;0.001</td></tr><tr><td>UofK</td><td>0.328</td><td>Fair</td><td>0.05</td><td>0.382</td><td>Fair</td><td>&lt;0.001</td></tr><tr><td>MAK</td><td>0.570</td><td>Moderate</td><td>&lt;0.001</td><td>NA</td><td>NA</td><td>NA</td></tr></tbody></table>

showed a large range of intercountry variations. For high viral load samples, promising overall 90.8% (RdRP RT-RAA) and 81.8% (N RT-RAA) sensitivities were determined. For samples with Ct 31 – 40, values were inconsistent between sites. While higher assay accuracy was identified at three sites, two sites did not produce the expected outcomes despite the success during the preparatory phase. Compared to the RdRP gene, the N-gene RT-RAA assay demonstrated lower sensitivity. The insufficient performance of N gene RT-RAA assay led to its exclusion for further testing to maximize usage of laboratory materials and resources, further underlining the importance of adaptation to unanticipated events during a large diagnostic study. Overall, the performance of the RT-RAA assays is much better than the commercially available rapid antigen tests, whose sensitivity values differed considerably with sensitivities ranging from only 28–86% (Ct 17 – 36) [28,29]. Rapid antigen tests are suited best for detection of symptomatic carriers with high viral load [30,31]. Low viral load samples are often undetected as well as certain SARS-CoV-2 mutations [32,33].

The RdRP gene RT-RAA assay showed promising but very variable sensitivity values across sites. Some sites showed more than 90% sensitivity and others under 50% even with high viral load samples. A potential mutation in the target region of RdRP primers, was excluded by screening over 10,000 SARS-CoV-2 RNA sequences from local strains in the involved countries (supplementary file #4 and #5) using a recently published screening method [13], identifying no significant changes. Reagents deterioration during transportation was unlikely, since RAA reagents are lyophilized, cold-chain independent, robust, and

stable over long periods of time [6]. In addition, a quality control check was conducted upon delivery of the kits to exclude this possibility. Clinical sample integrity is one of the factors for decreasing assay sensitivity especially since RNA is unstable [8]. Degraded RNA or samples contaminated with RNases can lead to poor assay performance [20]. In our study, all samples were tested with real-time RT-PCR and RT-RAA in a very short time window to assure sample integrity. Thus, it can be assumed that each sample had a similar viral load when tested with both methods. A limitation represents the use of different extraction kits and amplification devices across sites. This adds on the variability of the clinical settings of the study. It is difficult to standardize protocols and equipment across healthcare laboratories. Nonetheless, all devices and equipment used were approved to be used for *in vitro* diagnostics. Nucleic acid extraction remains the bottleneck of molecular diagnostics. In this study, standardized kits were used to ensure RNA quality to validate the isothermal amplification assays. However, to implement point-of-need molecular tests, extraction protocols need to be simplified and user-friendly. Different rapid methods have been described to extract SARS-CoV-2 RNA. Combinations of detergent, heat and magnetic beads can be used for quick extraction [34,35]. RPA/RAA has been shown to be more tolerant against inhibitors in clinical samples [36–38]. Thus, rapid extraction methods are feasible with this technology, as shown in various studies [39,40]. However, further refinement is needed to enhance RNA purity and yield. In contrast, the RT-PCR is more intolerant to inhibitors from different matrices and requires highly purified RNA [41–43]. Further large clinical studies are needed to combine both rapid extraction and SARS-CoV-2 molecular assays on site.

Surprisingly, one test site discovered an unusual cluster of discordance on certain days. After retesting those samples, better results were achieved. Deviations in performance of diagnostic tests have been attributed to sample quality/quantity, settings, and operators [22]. The latter could hinder the homogeneity of sensitivity values between sites. Thus, correct sample handling is an essential factor to be considered. Although the influence of individual operators on the results of a diagnostic test cannot be fully avoided, certain actions could help to reduce such events. In this study, continuous and in-person training was not possible due to the COVID-19 pandemic and travel restrictions. Thus, the variability of the assay’s results could be partially explained due to this difference in quality of training. Furthermore, attention to details regarding workflow should not be underestimated. For example, after careful troubleshooting, inaccuracies while transcribing records and test
results, or the influence of reduced concentration while working after a
certain hour of the day were reported. As a consequence, standardized
operations and in-person training for staff are of utmost importance
before operating diagnostic samples, in addition to quality control
checks [44].

The findings of this study provide evidence for the importance of the
suitcase lab as a deployable and feasible setup for accurate, sensitive and
specific pathogen detection, particularly in low-resource settings.
Furthermore, the RT-RAA method, especially based on the RdRP gene, is
a promising on-site detection method for SARS-CoV-2 infection, overall
showing higher accuracy than commercially available rapid antigen test
and bypassing supply shortages. However, variations in assay sensitivity
between sites revealed the importance of quality control and face to face
training for the staff. The influence of global and regional disruptions
should not be underestimated in large multi-country diagnostic trials.
Additionally, sample handling by staff was regarded as a bottleneck for
test performance. Continuous in-person training is an essential tool for
successful diagnostic testing, in case of excellent quality and the quan-
tity of kits and devices. These lessons learned should be considered when
planning and performing large multi-country diagnostic clinical trials in
poor resource settings.

Funding

This project is part of the European and Developing Countries Clinical Trials Partnership (EDCTP) programme supported by the European Union (grant number: RIA2020EF-2937-Africa_Suitcaselab). The funding sources were not involved in any way regarding the views expressed in this manuscript. The authors alone are responsible for the study design, data collection, analysis and interpretation, as well as the realization and submission of this manuscript.

Ethical approval

Ethical approval was granted from each study site before performing
the testing: 00000879MSAS/DPRS/DR (National Committee for
Research and Ethics in Public Health, Senegal); CERBM: 023-MSANP/
SG/AMM/CERBM (Ethics Committee of Biomedical Research,
Madagascar); CHRPE/AP/078/21 (Committee on Human Research,
Publication and Ethics, School of Medical Sciences, Ghana); UI/EC/21/
0010 (UI/ UCH Ethics Committee, Nigeria); ESP/CE/60/2021(School of
Public Health Ethics Committee, Democratic Republic of the Congo);
2-12-20 (National Research Ethics Review Committee, Sudan); SBS-
REC-883 (School of Biomedical Sciences Research and Ethics Commit-
tee, Uganda); 2021.05.13Eb_92 (Ethics Committee of Leipzig Univer-
sity, Germany).

**Supplementary material**

Supplementary file #1: additional information of the samples collected at each study site; Supplementary file #2: laboratory report forms; Supplementary file #3: sample flowcharts, results of each study site; Supplementary file #4: raw data of the mutational analysis; Supplementary file #5: summary of the mutational analysis.

Declaration of Competing Interest

The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.

Acknowledgements

We are grateful to all involved laboratory technicians, students, and
administrative staff for their participation and support throughout this
study in Africa.

**Supplementary materials**

Supplementary material associated with this article can be found, in
the online version, at doi:10.1016/j.jcv.2023.105422.

References

[1] Organization, W.H. WHO announces COVID-19 outbreak a pandemic. 2020.

[2] P. Rai, B.K. Kumar, V.K. Deekshit, I. Karunasagar, Detection technologies and recent developments in the diagnosis of COVID-19 infection, Appl. Microbiol. Biotechnol. 105 (2021) 441–455, https://doi.org/10.1007/s00253-020-11061-5.

[3] G. Zhu, M.C. Chou, C.W. Tsai, Lessons learned from the COVID-19 pandemic exposing the shortcomings of current supply chain operations: a long-term prescriptive offering, Sustainability 12 (2020) 5858.

[4] K.P. Iyengar, R. Vaishya, S. Bahl, A. Vaish, Impact of the coronavirus pandemic on the supply chain in healthcare, British J. Healthcare Manag. 26 (2020) 1–4.

[5] W.H. Organization, Antigen-Detection in the Diagnosis of SARS-CoV-2 Infection: Interim Guidance, 6 October 2021, World Health Organization, 2021.

[6] J. Li, J. Macdonald, F. von Stetten, Review: a comprehensive summary of a decade development of the recombinase polymerase amplification, Analyst 144 (2018) 31–67, https://doi.org/10.1039/c8an01621f.

[7] J. Li, J. Macdonald, F. von Stetten, A comprehensive summary of a decade development of the recombinase polymerase amplification, Analyst 144 (2018) 31–67.

[8] A.A. El Wahed, P. Patel, M. Maier, C. Pietsch, D. Rüster, S. Böhlken-Fascher, J. Kissenkötter, O. Behrmann, M. Frimpong, M.M. Diagne, Suitcase Lab for rapid detection of SARS-CoV-2 based on recombinase polymerase amplification assay, Anal. Chem. 93 (2021) 2627–2634.

[9] C.f.P.M. Products, Guideline on clinical evaluation of diagnostic agents, EMEA (2009). CPM/P/EWP/1119/98/Rev 1: 2009.

[10] Ltd, M.S. Diagnostic test evaluation calculator, 2020. Available online: https://www.medcalc.org/calc/diagnostic_test.php (accessed on December).

[11] R. Trevethan, Sensitivity, specificity, and predictive values: foundations, pliabilities, and pitfalls in research and practice, Front. Public Health 5 (2017) 307.

[12] J.R. Landis, G.G. Koch, The measurement of observer agreement for categorical data, Biometrics (1977) 159–174.

[13] M. Weidmann, E. Graf, D. Lichterfeld, A. Abd El Wahed, M. Bekaert, Efficient screening of long oligonucleotides against hundred thousands of SARS-CoV-2 genome sequences, Front. Virol. 2 (2022), 835707.

[14] Q. Matthews, S.J.R. da Silva, M. Noruzi, L.J. Pena, K. Pardee, Adaptive, diverse and de-centralized diagnostics are key to the future of outbreak response, BMC Biol. 18 (2020) 1–5.

[15] D. Mabey, R.W. Peeling, A. Ustianowski, M.D. Perkins, Diagnostics for the developing world, Nat. Rev. Microbiol. 2 (2004) 231–240.

[16] A. Abd El Wahed, P. Patel, O. Faye, S. Thaloengsok, D. Heidenreich, P. Matangkasombut, K. Manopwisedjaroen, A. Sakuntabhai, A.A. Sall, F.T. Hufert, et al., Recombinase polymerase amplification assay for rapid diagnostics of dengue infection, PLoS ONE 10 (2015), https://doi.org/10.1371/journal.pone.0129682 e0129682.

[17] A. Abd El Wahed, S.S. Sanabani, O. Faye, R. Pessoa, J.V. Patriota, R.R. Giorgi, P. Patel, S. Böhlken-Fascher, O. Landt, M. Niedrig, Rapid molecular detection of Zika virus in acute-phase urine samples using the recombinase polymerase amplification assay, PLoS Curr. (2017) 9.

[18] O. Faye, O. Faye, B. Soropogui, P. Patel, A. Abd El Wahed, C. Loucoubar, G. Fall, D. Kiory, N.F. Magassouba, S. Keita, Development and deployment of a rapid recombinase polymerase amplification Ebola virus detection assay in Guinea in 2015, Eurosurveillance 20 (2015) 30053.

[19] M.M. Baba, M. Bitew, J. Fokam, E.A. Lelo, A. Ahidjo, K. Asmamaw, G.A. Beloumou, W.D. Bulimo, E. Buratti, C. Chenwi, Diagnostic performance of a colorimetric RT-LAMP for the identification of SARS-CoV-2: a multicenter prospective clinical evaluation in sub-Saharan Africa, EClinicalMedicine 40 (2021), 101101.

[20] Hagen, A. Laboratory supply shortages are impacting COVID-19 and non-COVID diagnostic testing. 2020.

[21] Microbiology, A.S.f. Supply shortages impacting COVID-19 and non-COVID testing. 2021.

[22] P. Ghosh, R. Chowdhury, M.E. Hossain, F. Hossain, M. Miah, M.U. Rashid, J. Baker, M.Z. Rahman, M. Rahman, X. Ma, Evaluation of recombinase-based isothermal amplification assays for point-of-need detection of SARS-CoV-2 in resource-limited settings, Int. J. Infect. Dis. 114 (2022) 105–111.

[23] A. Kretschmer, A. Kossov, B. Grüne, O. Schildgen, T. Mathes, V. Schildgen, False positive rapid antigen tests for SARS-CoV-2 in the real-world and their economic burden, J. Infect. 84 (2022) 248–288.

[24] J.P. Skittrall, M. Wilson, A.A. Smielewska, S. Parmar, M.D. Fortune, D. Sparkes, M. D. Curran, H. Zhang, H. Jalal, Specificity and positive predictive value of SARS-CoV-2 nucleic acid amplification testing in a low-prevalence setting, Clin. Microbiol. Infect. 27 (469) (2021) e469-469e415.

[25] P.S. Wikramaratna, R.S. Paton, M. Ghafari, J. Lourenço, Estimating the false-negative test probability of SARS-CoV-2 by RT-PCR, Eurosurveillance 25 (2020), 2000568.

[26] A. Krüttgen, C.G. Cornelissen, M. Dreher, M.W. Hornef, M. Imöhl, M. Kleines, Comparison of the SARS-CoV-2 Rapid antigen test to the real star Sars-CoV-2 RT PCR kit, J. Virol. Methods 288 (2021), 114024.

[27] J. Dinnes, J.J. Deeks, S. Berhane, M. Taylor, A. Adriano, C. Davenport, S. Dittrich, D. Emperador, Y. Takwoingi, J. Cunningham, Rapid, point-of-care antigen and
molecular-based tests for diagnosis of SARS-CoV-2 infection, Cochrane Database Syst. Rev. (2021).

[28] H. Scheiblauer, A. Filomena, A. Nitsche, A. Puyskens, V.M. Corman, C. Drosten, K. Zwirglmaier, C. Lange, P. Emmerich, M. Müller, Comparative sensitivity evaluation for 122 CE-marked rapid diagnostic tests for SARS-CoV-2 antigen, Germany, September 2020 to April 2021, Eurosurveillance 26 (2021), 2100441.

[29] Denzler, A., Jacobs, M.L., Witte, V., Schnitzler, P., Denkinger, C.M., Knop, M. Rapid comparative evaluation of SARS-CoV-2 rapid point-of-care antigen tests. medRxiv 2021. https://doi.org/10.1002/jmv.26830.

[30] M. Ciotti, M. Maurici, M. Pieri, M. Andreoni, S. Bernardini, Performance of a rapid antigen test in the diagnosis of SARS-CoV-2 infection, J. Med. Virol. 93 (2021) 2988-2991, https://doi.org/10.1002/jmv.26830.

[31] J.G. Routsias, M. Mavrouli, P. Tsoplou, K. Dioikitopoulou, A. Tsakris, Diagnostic performance of rapid antigen tests (RATs) for SARS-CoV-2 and their efficacy in monitoring the infectiousness of COVID-19 patients, Sci. Rep. 11 (2021) 1-9.

[32] A. Osterman, I. Badell, E. Basara, M. Stern, F. Kriesel, M. Eletreby, G.N. Öztan, M. Huber, H. Autenrieth, R. Knabe, Impaired detection of omicron by SARS-CoV-2 rapid antigen tests, Med. Microbiol. Immunol. 211 (2022) 105-117.

[33] T. Toptan, L. Eckermann, A.E. Pfeiffer, S. Hoehl, S. Ciesek, C. Drosten, V. M. Corman, Evaluation of a SARS-CoV-2 rapid antigen test: potential to help reduce community spread? J. Clin. Virol. 135 (2021), 104713.

[34] A.S. Fomsgaard, M.W. Rosenstierne, An Alternative Workflow For Molecular Detection of SARS-CoV-2 – Escape from the NA Extraction Kit-Shortage, Copenhagen, Denmark, 2020, https://doi.org/10.2807/1560-7917. ES.2020.25.14.2000398. MarchEurosurveillance 2020, 25, 2000398.

[35] Z. Zhao, H. Cui, W. Song, X. Ru, W. Zhou, X. Yu, A simple magnetic nanoparticles-based viral RNA extraction method for efficient detection of SARS-CoV-2, BioRxiv (2020).

[36] A. Ceruti, R.M. Kobialka, J. Ssekitoleko, J.B. Okuni, S. Blome, A. Abd El Wahed, U. Truyen, Rapid extraction and detection of African swine fever virus DNA based on isothermal recombinase polymerase amplification assay, Viruses 13 (2021) 1731.

[37] S. Kersting, V. Rausch, F.F. Bier, M. von Nickisch-Rosenegk, Rapid detection of Plasmodium falciparum with isothermal recombinase polymerase amplification and lateral flow analysis, Malar. J. 13 (2014) 1-9.

[38] K. Krölov, J. Frolova, O. Tudoran, J. Suhorutsenko, T. Lehto, H. Sibul, I. Mäger, M. Laanpere, I. Tulp, Ü. Langel, Sensitive and rapid detection of Chlamydia trachomatis by recombinase polymerase amplification directly from urine samples, J. Mol. Diagn. 16 (2014) 127-135.

[39] R. Chowdhury, P. Ghosh, M.A.A. Khan, F. Hossain, K. Faisal, R. Nath, J. Baker, A.A. E. Wahed, S. Maruf, P. Nath, Evaluation of rapid extraction methods coupled with a recombinase polymerase amplification assay for point-of-need diagnosis of post-kala-azar dermal leishmaniasis, Trop. Med. Infect. Dis. 5 (2020) 95.

[40] J. Yu, D. Shen, T. Dai, X. Lu, H. Xu, D. Dou, Rapid and equipment-free detection of Phytophthora capsici using lateral flow strip-based recombinase polymerase amplification assay, Lett. Appl. Microbiol. 69 (2019) 64-70.

[41] J. Bessetti, An introduction to PCR inhibitors, J. Microbiol. Methods 28 (2007) 159-167.

[42] Dalecka, B., Mezule, L. Study of potential PCR inhibitors in drinking water for *Escherichia coli* identification. 2018.

[43] M. Sidstedt, J. Hedman, E.L. Romsos, L. Waitara, L. Wadsø, C.R. Steffen, P. M. Vallone, P. Rådström, Inhibition mechanisms of hemoglobin, immunoglobulin G, and whole blood in digital and real-time PCR, Anal. Bioanal. Chem. 410 (2018) 2569-2583.

[44] H. Wang, G. Li, J. Zhao, Y. Li, Y. Ai, An overview of nucleic acid testing for the novel coronavirus SARS-CoV-2, Front. Med. (Lausanne) 7 (2021), https://doi.org/10.3389/fmed.2020.571709.