Meta-analysis of rapid direct-to-PCR assays for the qualitative detection of SARS-CoV-2
Authors: Reginald T.A. , Grippon S. , Coldwell M.J. , Chen H. , Koh L. , Jan U. , Sanchez-Bretano A. , Borley D. , Oladimeji P. , Moore N. , Kidd S.P. , Martin J.E. 
Published: May 2021

Infection with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and the ensuing COVID-19 pandemic present significant challenges to current diagnostic and therapeutic patient care pathways including whether new in vitro diagnostic tests can accurately identify and rule out current SARS-CoV-2 infection.

The gold standard diagnostic test to identify a current SARS-CoV-2 infection is a central laboratory-based molecular assay employing reverse transcription polymerase chain reaction (RT-PCR) with very high accuracy of detection; however, which typically requires 1-2 days turn-around for results. Rapid RT-PCR assays and systems have been developed which can be deployed locally (near-patient or point of care (POC)), provide faster results and not impact on already stressed central laboratory capacity. Rapid test results can be returned within the same clinical encounter, facilitating timely decisions that optimise the patient care pathway and support more rapid COVID-19 diagnosis, isolation and contract tracing activities1.

Direct-to-PCR is an evolution of RT-PCR in which the patient sample is added directly to an amplification reaction without being subjected to prior nucleic acid extraction, purification, or quantification to reduce the time and monetary resources required to process samples. Rapid, direct-to-PCR systems further increase the speed of testing by combining rapid PCR instruments with direct-to-PCR assays, to generate results in less than two hours.

This appears to be the first meta-analysis assessing the accuracy of rapid direct-to-PCR in the detection of SARS-CoV-2. In total, 1,144 unique records were identified and screened using search string evaluation, 49 full-text reports and/or supplemental materials were assessed for inclusion. This resulted in 16 studies, reporting 22 datasets with 5322 patient samples (of which 2220 were identified as positive according to centralised laboratory testing) included in the analysis.

The overall percentage agreement (OPA) between the rapid direct RT-PCR and gold standard centralised laboratory RT-PCR was 95.10% with 91.22% positive percent agreement (PPA) and 98.16% negative percent agreement (NPA). When compared to commercially available tests were considered, these were assessed to be 96.95% OPA, 94.78 % PPA and 98.34 % NPA. Furthermore, the Cohens kappa statistical coefficient k = 0.94 (0.96 for commercial only), and Youden Index = 0.893 (0.924 for commercial only) indicate an almost perfect agreement. These results therefore indicate that direct-to-PCR assays performance is equivalent to the standard centralised laboratory PCR systems for the detection of SARS-CoV-2.

Objectives

To assess the efficacy of rapid direct-to-PCR assays and systems for the detection of SARS-CoV-2 in the hospital, care home and medical research population from November 2020 to July 2021.

Search methods

Initial electronic searches of the Cochrane COVID-19 Study Register (which includes daily updates from PubMed and Embase and preprints from medRxiv and bioRxiv) were undertaken on the 30th of April 2021, with a further search undertaken on 8th July 2021 (PRISMA flow diagram, Figure 2).

Selection criteria

Studies, published in English, of subjects with either suspected SARS-CoV-2 infection, known SARS-CoV-2 infection or known absence of infection, or those who were being screened for infection were included. Commercially available and research use only rapid direct-to-PCR assays (without RNA extraction and purification reporting results within two hours) were included in the study.

Data collection, extraction and analysis

Studies were screened independently, in duplicate with any disagreements resolved by discussion with a third author. Study characteristics were extracted by one author and checked by a second; extraction of study results and assessments of risk of bias and applicability were undertaken independently in duplicate.

Where studies were not publicly available, sites that undertook in-service evaluations of rapid direct-to-PCR system were contacted and asked to supply anonymised datasets. Both reviewers independently performed data extraction and verification and calculated 2×2 contingency tables with the number of true positives, false positives, false negatives and true negatives. They resolved any disagreements by discussion and by review with the third reviewer.

Main results

In total, 22 study cohorts were included (described in 16 study reports, including 5 unpublished reports), reporting results for 5322 samples (of which 2220 were confirmed SARS-CoV-2, as determined by central laboratory testing). Studies were mainly from Europe and North America and evaluated eight commercially available direct-to-PCR assay kits/cartridges, and six developed from other reagents.

Conclusions

This appears to be the first meta-analysis assessing the accuracy of rapid direct-to-PCR in the detection of SARS-CoV-2. In total, 1,144 unique records were identified and screened using search string evaluation, 49 full-text reports and/or supplemental materials were assessed for inclusion. This resulted in 16 studies reporting 21 datasets with 5322 patient samples (2220 positive) included in the analysis.

The overall agreement between the commercially available rapid direct RT-PCR and gold standard centralised laboratory RT-PCR was 96.9% with 94.8% PPA and 98.4% NPA. Furthermore, the Cohens kappa statistical coefficient k = 0.96, indicating an almost perfect agreement and Youden Index = 0.93. These results show that direct-to-PCR assays performance is equivalent to the gold standard centralised laboratory RT-PCR systems for the detection of SARS-CoV-2.

Plain language summary

What is a rapid direct-to-PCR test for diagnosing COVID-19?

Rapid direct-to-PCR tests are rapid tests that aim to confirm or rule out the presence of SARS-CoV-2 within 2 hours without complicated processing of the sample.

How accurate is a rapid direct-to-PCR test for diagnosing COVID-19?

We compared the accuracy of rapid direct-to-PCR tests with gold standard centralised laboratory RT-PCR for the detection of SARS-CoV-2 and found that direct-to-PCR was as accurate as standard RT-PCR assays.

Why is this question important?

People with suspected COVID-19 need to know quickly whether they are infected, so that they can self-isolate, inform close contacts and possibly receive treatment. Currently, COVID-19 infection is confirmed by a laboratory test called RT-PCR, which uses specialist equipment and often takes at least 24 hours to produce a result. If they are accurate, faster diagnosis could allow people to take appropriate action more rapidly, with the potential to reduce the spread of COVID-19.1

What did we aim to find out?

Our goal was to determine if commercially available and research use rapid direct-to-PCR tests are accurate enough to detect SARS-CoV-2 in comparison to gold standard laboratory RT-PCR.

What did we do?

We looked for studies that measured the accuracy of any commercially produced and research use rapid direct-to-PCR tests, in people tested for COVID-19 using RT-PCR. People could be tested in hospital or in the community. Studies could test people with or without symptoms.

Tests had to use minimal equipment, be performed safely without risking infection from the sample, and have results available within two hours of the sample being collected.

What we found?

We include 22 studies in the review. They investigated a total of 5322 nose or throat samples; COVID-19 was confirmed in 2220 of these samples. The studies investigated 15 different direct-to-PCR tests. They took place mainly in Europe and North America.

What did we find?

Although overall results for diagnosing and ruling out COVID-19 were good (91.2% of infections correctly diagnosed and 98.3% correctly ruled out), we noted a difference in COVID-19 detection between tests, especially those available as commercial kits versus ones assembled from reagents from different sources. However, we cannot be certain about whether results will remain the same in a real-world setting. We could not investigate differences in people with or without symptoms, nor time since symptoms-onset because the studies did not consistently provide enough clinical information about their participants.

How reliable were the results of the studies?

In general, the studies included followed rigorous methods, in accordance with the tests intended use to detect COVID-19 and included at least two independent results to confirm or rule out COVID-19 infection. The results from different test brands varied and few studies compared multiple rapid-PCR tests. Most of the studies did not provide sufficient information to determine whether the detection levels would vary in people with COVID-19 symptoms versus without symptoms.

What does this mean?

On average the rapid direct-to-PCR were shown to be equivalent to gold standard laboratory-based RT-PCR tests and several direct-to-PCR tests show very high accuracy. However, for most of the tests, more evidence is needed particularly in people without symptoms, on the accuracy of repeated testing, and testing in non-healthcare settings such as schools (including self-testing).