The importance of external quality assessment data in evaluating SARS-CoV-2 virus genome detection assays
Authors: Christoph Buchta, Mathias M Müller, Andrea Griesmacher
Journal: The Lancet Microbe (2022)

We read with interest the comprehensive review by Wing Ying Au and Peter Pak Hang Cheung on the diagnostic performances of common nucleic acid tests for SARS-CoV-2.1Au WY Cheung PPH Diagnostic performances of common nucleic acid tests for SARS-CoV-2 in hospitals and clinics: a systematic review and meta-analysis.Lancet Microbe. 2021; 2: e704-e714Google Scholar Although there are numerous publications in this regard, no reference was made to performance data that were collected in SARS-CoV-2-related external quality assessment (EQA) schemes. Performance studies as cited in the review provide essential data on sensitivity and specificity in evaluation settings, but such studies have so far not been published for all routinely used assays. However, EQA schemes provide data on the diagnostic and analytic performance not only of selected individual assays, but of the entire range of assays used by laboratories in their routine work and thus real-life data. It should be noted that in the latest round of the national Austrian EQA scheme for SARS-CoV-2 virus genome detection, a total of no less than 53 different assays were registered.2Buchta C Camp J Jovanovic J et al.A look at the precision, sensitivity and specificity of SARS-CoV-2 RT-PCR assays through a dedicated external quality assessment round.Clin Chem Lab Med. 2021; 60: e34-e37Google Scholar By using the same panel of samples, these schemes show differences in the performances of individual systems, but also the intra-type variation of results—ie, the variation in results obtained from different devices of the same type. In addition, EQA is an ongoing process that monitors the performance of assays during the whole time they are approved and supplied. We therefore consider EQA data to be no less important for assessing the analytical performance of assays than performance study and evaluation data. Particularly noteworthy are the following results, which could only be found through data on SARS-CoV-2-related EQA schemes. Firstly, monitoring of dozens of different assays that are used in one or more of the hundreds of participating laboratories in one EQA round is feasible, even across national borders.2Buchta C Camp J Jovanovic J et al.A look at the precision, sensitivity and specificity of SARS-CoV-2 RT-PCR assays through a dedicated external quality assessment round.Clin Chem Lab Med. 2021; 60: e34-e37Google Scholar, 3Matheeussen V Corman VM Donoso Mantke O et al.International external quality assessment for SARS-CoV-2 molecular detection and survey on clinical laboratory preparedness during the COVID-19 pandemic, April/May 2020.Euro Surveill. 2020; 252001223Google Scholar Secondly, samples with a virus load of 100 000 copies per mL or more (ie, a cycle threshold [Ct] value of less than 30) are quite reliably detected as positive, but the detection probability decreases with decreasing virus load.2Buchta C Camp J Jovanovic J et al.A look at the precision, sensitivity and specificity of SARS-CoV-2 RT-PCR assays through a dedicated external quality assessment round.Clin Chem Lab Med. 2021; 60: e34-e37Google Scholar Thirdly, the false positive rate is very low.4Buchta C Camp JV Jovanovic J et al.The versatility of external quality assessment for the surveillance of laboratory and in vitro diagnostic performance: SARS-CoV-2 viral genome detection in Austria.Clin Chem Lab Med. 2021; 59: 1735-1744Google Scholar Fourthly, assay-specific Ct values prevent comparability of results obtained by different test systems, whereas the intra-assay and the intra-type variation of Ct values is low.4Buchta C Camp JV Jovanovic J et al.The versatility of external quality assessment for the surveillance of laboratory and in vitro diagnostic performance: SARS-CoV-2 viral genome detection in Austria.Clin Chem Lab Med. 2021; 59: 1735-1744Google Scholar, 5Buchta C Görzer I Chiba P et al.Variability of cycle threshold values in an external quality assessment scheme for detection of the SARS-CoV-2 virus genome by RT-PCR.Clin Chem Lab Med. 2020; 59: 987-994Google Scholar Lastly, the accuracy of the test systems, as challenged by samples from a dilution series and by two identical samples, is good for most assays, but not all.2Buchta C Camp J Jovanovic J et al.A look at the precision, sensitivity and specificity of SARS-CoV-2 RT-PCR assays through a dedicated external quality assessment round.Clin Chem Lab Med. 2021; 60: e34-e37Google Scholar, 4Buchta C Camp JV Jovanovic J et al.The versatility of external quality assessment for the surveillance of laboratory and in vitro diagnostic performance: SARS-CoV-2 viral genome detection in Austria.Clin Chem Lab Med. 2021; 59: 1735-1744Google Scholar In conclusion, we repeat that the performance of assays can only be fully assessed if both data from evaluation studies and EQA data are included. We declare no competing interests. AG is President of ÖQUASTA, the Austrian Association for Quality Assurance and Standardization of Medical and Diagnostic Tests. CB is Chairman of the Executive Board of EQALM, the European Organisation for External Quality Assurance Providers in Laboratory Medicine. Diagnostic performances of common nucleic acid tests for SARS-CoV-2 in hospitals and clinics: a systematic review and meta-analysisAll three nucleic acid tests consistently perform better with pharyngeal swabs using SARS-CoV-2 open reading frame 1ab primer with RNA extraction. dPCR was shown to be the most sensitive, followed by qPCR and LAMP. However, their accuracy does not differ significantly. Instead, accuracy depends on specific experimental conditions, implying that more efforts should be directed to optimising the experimental setups for the nucleic acid tests. Hence, our results could be a reference for optimising and establishing a standard nucleic acid test protocol that is applicable in laboratories worldwide. Full-Text PDF Open AccessThe importance of external quality assessment data in evaluating SARS-CoV-2 virus genome detection assays – Authors' replyWe thank Christoph Buchta and colleagues for their response to our systematic review and meta-analysis.1 We agree that external quality assessment (EQA) schemes examine the diagnostic test performance of a wide range of nucleic acid assays for SARS-CoV-2 virus genome detection across different laboratories worldwide. Full-Text PDF Open Access