Rate of no result in cell‐free DNA testing and its influence on test performance metrics
2016-10-13

The reasons for 'no result' by cell-free (cf) DNA testing are heterogeneous and vary depending on the technological platform1. Namely, they relate to: specimen handling issues (including administrative ones); insufficient (typically < 4%) or absent fetal fraction (FF)2; technical reasons (e.g. quality metrics of the extracted DNA, its amplification and raw data analysis); and biological reasons, such as cases analyzed with the wrong oocyte-donation status (heterologous donation analyzed as homologous) or cases for which the analysis was not conclusive although the pre-analytical and analytical quality controls were passed (e.g. non-informative single nucleotide polymorphisms (SNPs) within the microdeletion critical region). Background information on why a test does not provide a result is important as it may trigger a more targeted follow-up examination3. It was shown that cases with no result due to low FF carry an increased risk for trisomy 13, trisomy 18 and triploidy, but not for trisomy 21 (T21)3-5. This is in contrast with cases with no test result due to technical failure for which there is no evidence of an increased risk for aneuploidy. However, not all cfDNA tests measure FF. Tests that do not evaluate this quality metric do not fail samples with low FF. Thus, the overall rate of no test result is lower. As the screening performance in the group with low FF is lower compared to the officially claimed performance, the actual overall test metrics are worse6, 7. These elements have to be considered when evaluating the screening performance of a test. The aim of this conceptual paper is to assess how different reasons for no test result can impact the test performance. In this theoretical model, four different hypothetical cfDNA tests (A, B, C and D) claiming different reasons and different rates for no test result, but with the same test performance (99% detection rate (DR) and 0.1% false-positive rate (FPR)), are considered. The scheme of the present theoretical model is depicted in Figure S1. Tests A and B analyze all cases regardless of the FF level and, in 2% of the study population, no test result is provided due to technical reasons. In another 2% of the cases, FF is < 4% but, as these tests do not measure FF, samples with low FF are not identified and are processed with a DR and FPR of 75% and 2%, respectively. The remaining 96 000 samples (95 808 euploid and 192 T21 samples) are analyzed with the expected DR and FPR of 99% and 0.1%, respectively. As a result, the DR, FPR and positive predictive value (PPV) of samples with a result are 98.5%, 0.14% and 58.72%, respectively. However, the overall DR based on the 200 women with T21 pregnancies who requested screening is only 96.5% (Figure 1 and Table S1). Test B is the same as Test A, with the exception that the cohort with FF < 4% is analyzed at a DR of 60% and FPR of 0.1%. The DR, FPR and PPV are 98.2%, 0.10% and 66.31%, respectively. The overall DR based on the 200 cases with T21 is only 96.2% (Figure 1 and Table S1). In Test C, 96% of the population (95 808 euploid and 192 T21 samples) receive a result (2% fail for low FF and 2% for technical reasons). The DR, FPR and PPV are 99.0%, 0.10% and 66.49%, respectively, based on the population with a test result, but the DR is 95.0% if rates are based on the initial 200 samples with T21 submitted for testing (Figure 1 and Table S1). Test D is similar to Test C, but the technical failure rate is 4% instead of 2%. The test provides the same PPV and performances of the reported population as that of Test C, but the actual DR of the initial 200 samples with T21 is 93.06% (Figure 1 and Table S1). Technical reasons for no test result have a relevant effect on the submitted sample population: the higher the rate for reasons other than low FF, the lower the DR based on the overall population undergoing the test, because a larger group of non-analyzed samples will proportionally include a larger portion of non-analyzed T21 cases. This affects each test irrespective of whether it measures FF or not. Technical reasons are largely dependent on the stringency of the quality metrics that each laboratory applies for acceptance or rejection of the generated raw data; typically, the more stringent the criteria, the higher the no-result rate. It is obvious that the technical reasons for no result should be as low as possible. The aim of quality metrics is to ascertain that each sample is examined at the highest level of quality possible. In the context of clinical practice, it is challenging to apprehend the best compromise between stringency of quality metrics and technical test failure rate; more research in this field is certainly advisable. Tests with an overall low no-result rate because of the absence of FF quality metrics have a worse screening performance as the DR and FPR in the population with low FF (typically < 4%) are worse than in the population with FF ≥ 4%. This is due to the fact that there is a progressively greater overlap between the Gaussian distributions of the Z-score of the affected and unaffected populations with the decreasing level of FF percentage6, 7. The real test performance in the group with FF < 4% is yet to be determined. In our examples, we simulated the performance based on DRs and FPRs of 75% and 2%, and 60% and 0.1%, respectively. One could argue that this is just dependent on the applied cut-off and a change in the threshold would increase the DR or the FPR up to a requested and desired level. However, if the FF is not measured, the group of cases with low FF will not be identified and a post-hoc adjustment and case management will not be possible. Therefore, for those tests not measuring FF and with an overall low no-result rate, it would be most appropriate to report on slightly lower actual test performance, either based on large-scale clinical prospective studies with such a test, with follow-up at birth, or based on studies that have identified and assessed the proportion of cases with low FF and the respective test performance in this group of pregnancies. The examination of FF in each individual case and exclusion of those with a low FF results in the expected claimed test's metrics in the population that receives a test result. This is evident when the DR, FPR and PPV of Tests C and D are compared with those of the other two tests. However, the DR based on the whole tested population is different: Tests C and D (95.0% and 93.1%) have worse overall DRs rates compared to Tests A and B (96.5% and 96.2%), as cases with low FF still receive a test result with Test A and B and some of the affected cases in the population with low FF can still occasionally be identified. The effect on the FPR is expected to be less pronounced but still present. However, the overall DR in cases with no test result due to low FF could be increased again by invasive testing, with the cost of an increased invasive-procedure rate. Alternatively, cfDNA testing after a sample redraw could be proposed. In this second option, women should be counseled that, in the case of a second no test result for low FF (cfDNA test can provide a result in nearly 50% of redrawn cases8), results could potentially be provided, ignoring the FF metrics, but with the cost of worse DR and FPR. It is obvious that more research to assess the actual test performance in samples with low FF is certainly necessary and that this option will be possible only when the requested performances are available for each single cfDNA test including FF metrics. However, if the DR is demonstrated between 60–75% and the FPR between 0.1–2%, these test performances would be better than most of the available second-trimester screening tests. The present theoretical scheme could potentially be applied for modeling the effect of the no-result rate on the test metrics in more complex emerging cfDNA technologies, including those with dynamic FF thresholds or reflex cfDNA testing with increased sequencing depth on a subgroup of samples not meeting specific FF metric criteria. More research would clarify how these emerging approaches would impact test performance. In conclusion, based on the present theoretical model, we wanted to show that the proportion of cases with no test result, the reason for test failure and the management of these cases determine the overall test performance of these tests. Although tests that measure FF provide quality metrics on the DR and FPR of each individual test result, management of the cases with no test result, either for low FF and for technical reasons, remains challenging and determines the overall test performance. Our examples demonstrate the impact of the measurement of FF and the no-result rates for low FF and for technical reasons on test performances. It has been calculated that having the lowest overall test failure rate does not necessarily result in the best test metrics, above all the highest PPV. Based on the results of this study, the best advice may be to choose a cfDNA-based platform with effective FF quality metrics and with the lowest no-result rate for technical reasons. Thus, when assisting patients with the choice of a cfDNA test provider and providing pretest counseling, the presence/absence of FF quality-control metrics, their accuracy and a disclaimer regarding the percentage of no-result rate for technical or other reasons should be information as fundamental as test performance itself. In this view, every laboratory should report in published clinical studies details regarding the accuracy and the reasons of no-test result and their corresponding percentages. In addition, due to the evident relevant role of FF measurement, external quality-control assessment for validation of FF measurement technologies by comparison with a commonly assumed 'gold standard' reference method (i.e. Y-chromosome measurement) appears to be a mandatory requirement9. F.R.G. is full-time employee without ownership shares at TOMA Advanced Biomedical Assays S.p.A.. TOMA is a medical genetics laboratory performing prenatal diagnosis by cytogenetic analysis and chromosomal microarray on prenatal samples as well as cfDNA testing and serum screening for fetal aneuploidy. F. R. Grati1* and K. O. Kagan2 1TOMA, Advanced Biomedical Assays S.p.A., Via F. Ferrer 25/27, 21052 Busto Arsizio (VA), Italy; 2Department of Obstetrics and Gynaecology, University of Tuebingen, Tuebingen, Germany *Correspondence. (e-mail: [email protected]) Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.