The importance of external quality assessment data in evaluating SARS-CoV-2 virus genome detection assays – Authors‘ reply
Authors: Wing Ying Au, Peter Pak Hang Cheung
Journal: The Lancet Microbe (2022)

We thank Christoph Buchta and colleagues for their response to our systematic review and meta-analysis.1Au WY Cheung PPH Diagnostic performances of common nucleic acid tests for SARS-CoV-2 in hospitals and clinics: a systematic review and meta-analysis.Lancet Microbe. 2021; 2: e704-e714Google Scholar We agree that external quality assessment (EQA) schemes examine the diagnostic test performance of a wide range of nucleic acid assays for SARS-CoV-2 virus genome detection across different laboratories worldwide. According to WHO, EQA compares test performance within and between laboratories, and such results set a reference for the overall test accuracy and proficiency. However, most EQA done on COVID-19 diagnostics with nucleic acid assays have limited the test performance parameters to cycle threshold (Ct) value, viral concentration, and the number of correct results using synthetic samples which serve as the ground truth in diagnostic evaluation.2Matheeussen V Corman VM Mantke OD et al.International external quality assessment for SARS-CoV-2 molecular detection and survey on clinical laboratory preparedness during the COVID-19 pandemic, April/May 2020.Euro Surveill. 2020; 252001223Google Scholar, 3Wang Z Chen Y Yang J et al.External quality assessment for molecular detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in clinical laboratories.J Mol Diagn. 2021; 23: 19-28Google Scholar Unlike in the clinical evaluation of diagnostic performance, where sensitivity and specificity are computed from the number of positive and negative patient samples tested in one laboratory in each study, EQA reports the number of correct results as the total number of laboratories correctly identifying a true positive sample. This results in a small sample size per laboratory (a panel of six to seven known samples) which does not resemble real-life practice, where dozens of samples are being tested in one laboratory. Limited sample size might underestimate the false positive rate of the test.4Wikramaratna PS Paton RS Ghafari M Lourenço J Estimating the false-negative test probability of SARS-CoV-2 by RT-PCR.Euro Surveill. 2020; 252000568Google Scholar Hence, clinical evaluation remains necessary to assess practical diagnostic test performance. Furthermore, confounding factors during pre-test and post-test procedures might have an impact on the test accuracy because nucleic acid tests are vulnerable to manual errors. As nucleic acid tests target the viral genome, factors that affect the purity of viral genes—such as specimen types and RNA extraction methods—might impact the outcomes of the tests, and thus their diagnostic performance.5Tahamtan A Ardebili A Real-time RT-PCR in COVID-19 detection: issues affecting the results.Expert Rev Mol Diagn. 2020; 20: 453-454Google Scholar Also, the actual clinical samples are more complex than synthetic samples in terms of the makeup of the sample matrix and variations in the operational processes such as sample collection and preparation. Lastly, the use of synthetic samples has been shown to falsely increase the detection limit of diagnostic assays. Hence, our study compares the diagnostic performance of the three nucleic acid tests—ie, digital PCR (dPCR), quantitative PCR (qPCR), and loop-mediated isothermal amplification (LAMP), when different experimental setups are used. Our findings complement the results from EQA by looking into the experimental factors that are either not evaluated or cannot be assessed using artificial SARS-CoV-2 positive samples. Instead of testing the Ct values of qPCR and LAMP, our analysis uses a different approach in evaluating diagnostic performances as dPCR measures the absolute number of viral copies. The main analytic outcomes are sensitivity, specificity, diagnostic odd ratios, and accuracy presented in the area under the summary receiver operating characteristic curve. The false-positive rate in our findings is also shown to be very low for all three nucleic acid tests evaluated, which agrees with the results from the EQA studies.2Matheeussen V Corman VM Mantke OD et al.International external quality assessment for SARS-CoV-2 molecular detection and survey on clinical laboratory preparedness during the COVID-19 pandemic, April/May 2020.Euro Surveill. 2020; 252001223Google Scholar, 3Wang Z Chen Y Yang J et al.External quality assessment for molecular detection of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in clinical laboratories.J Mol Diagn. 2021; 23: 19-28Google Scholar Together, our study substantiates the results of EQA and confirms the test sensitivity, specificity, and accuracy of three different nucleic acid tests using clinical data. We declare no competing interests. Diagnostic performances of common nucleic acid tests for SARS-CoV-2 in hospitals and clinics: a systematic review and meta-analysisAll three nucleic acid tests consistently perform better with pharyngeal swabs using SARS-CoV-2 open reading frame 1ab primer with RNA extraction. dPCR was shown to be the most sensitive, followed by qPCR and LAMP. However, their accuracy does not differ significantly. Instead, accuracy depends on specific experimental conditions, implying that more efforts should be directed to optimising the experimental setups for the nucleic acid tests. Hence, our results could be a reference for optimising and establishing a standard nucleic acid test protocol that is applicable in laboratories worldwide. Full-Text PDF Open AccessThe importance of external quality assessment data in evaluating SARS-CoV-2 virus genome detection assaysWe read with interest the comprehensive review by Wing Ying Au and Peter Pak Hang Cheung on the diagnostic performances of common nucleic acid tests for SARS-CoV-2.1 Although there are numerous publications in this regard, no reference was made to performance data that were collected in SARS-CoV-2-related external quality assessment (EQA) schemes. Performance studies as cited in the review provide essential data on sensitivity and specificity in evaluation settings, but such studies have so far not been published for all routinely used assays. Full-Text PDF Open Access