# Results of a worldwide external quality assessment of cfDNA testing in lung Cancer

Jennifer A Fairley, Melanie H Cheetham, Simon J Patton, Etienne Rouleau, Marc Denis, Elisabeth M C Dequeker, Ed Schuuring, Kaat van Casteren, Francesca Fenizia, Nicola Normanno, et al.

► To cite this version:

Jennifer A Fairley, Melanie H Cheetham, Simon J Patton, Etienne Rouleau, Marc Denis, et al. Results of a worldwide external quality assessment of cfDNA testing in lung Cancer. BMC Cancer, 2022, 22 (1), pp.759. 10.1186/s12885-022-09849-x. doi:10.1186/s12885-022-09849-x

HAL Id: hal-04756871

https://nantes-universite.hal.science/hal-04756871v1

Submitted on 28 Oct 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
RESEARCH

Open Access

<figure><img src="image_2.png" /></figure>

Check for
updates

# Results of a worldwide external quality assessment of cfDNA testing in lung Cancer

Jennifer A. Fairley<fnref n="1" />*, Melanie H. Cheetham², Simon J. Patton², Etienne Rouleau³, Marc Denis⁴, Elisabeth M. C. Dequeker⁵, Ed Schuuring⁶, Kaat van Casteren⁵, Francesca Fenizia⁷, Nicola Normanno⁸ and Zandra C. Deans<fnref n="1" />

**Abstract**

**Background:** Circulating cell free DNA (cfDNA) testing of plasma for *EGFR* somatic variants in lung cancer patients is being widely implemented and with any new service, external quality assessment (EQA) is required to ensure patient safety. An international consortium, International Quality Network for Pathology (IQNPath), has delivered a second round of assessment to measure the accuracy of cfDNA testing for lung cancer and the interpretation of the results.

**Methods:** A collaboration of five EQA provider organisations, all members of IQNPath, have delivered the assessment during 2018–19 to a total of 264 laboratories from 45 countries. Bespoke plasma reference material containing a range of *EGFR* mutations at varying allelic frequencies were supplied to laboratories for testing and reporting according to routine procedures. The genotyping accuracy and clinical reporting was reviewed against standardised criteria and feedback was provided to participants.

**Results:** The overall genotyping error rate in the EQA was found to be 11.1%. Low allelic frequency samples were the most challenging and were not detected by some testing methods, resulting in critical genotyping errors. This was reflected in higher false negative rates for samples with variant allele frequencies (VAF) rates less than 1.5% compared to higher frequencies. A sample with two different *EGFR* mutations gave inconsistent detection of both mutations. However, for one sample, where two variants were present at a VAF of less than 1% then both mutations were correctly detected in 145/263 laboratories. Reports often did not address the risk that tumour DNA may have not been tested and limitations of the methodologies provided by participants were insufficient. This was reflected in the average interpretation score for the EQA being 1.49 out of a maximum of 2.

**Conclusions:** The variability in the standard of genotyping and reporting highlighted the need for EQA and educational guidance in this field to ensure the delivery of high-quality clinical services where testing of cfDNA is the only option for clinical management.

**Keywords:** Lung cancer, NSCLC, cfDNA, *EGFR* EQA, Mutation testing

**Introduction**

The use of molecular testing to aid determining therapy options in the treatment of non-small cell lung cancer (NSCLC) is now recommended for all patients [1, 2]. Use of formalin-fixed paraffin embedded (FFPE) material from biopsies or resected tumours is considered the gold standard for this testing, however up to 30% of patients do not have suitable material available [3]. This can be

<footnote>*Correspondence: jenni.fairley@ed.ac.uk</footnote>

<footnote n="1">¹GenQA, Nine, Edinburgh Bioquarter, 9 Little France Road, Edinburgh EH16 4SA, UK</footnote>

<footnote>Full list of author information is available at the end of the article</footnote>
due to inaccessibility of the tumour for biopsy, the insufficient amount of tissue or insufficient neoplastic cells for molecular testing. Circulating cell free DNA (cfDNA) has therefore become a recognised alternative source of testing material [1, 4].

cfDNA is comprised of short fragments of DNA (160–170 base pairs) [5, 6] found circulating within a patient's plasma and also harbours circulating tumour DNA (ctDNA). This ctDNA is shed by tumours and although the mechanisms are not fully understood it has been proposed that processes such as apoptosis and necrosis are responsible [7]. Fragmented DNA may also be released from tumour cells through vesicles [8]. Fragments of DNA from normal dividing cells are also found in the patient's plasma and, therefore, the fraction of ctDNA in the total amount of cfDNA is often less than 1%. As ctDNA is now in clinical use for the detection of somatic variants in tumours, sensitive mutant detection methods are required to detect the variants [4, 8].

External quality assessment (EQA) is part of a laboratory's quality assurance process and participation in an EQA scheme has been shown to improve the quality of testing [9–13]. In 2017, four EQA providers working under the umbrella organization, the International Quality Network for Pathology (IQN Path) delivered a pilot EQA involving 32 laboratories to determine the feasibility of offering assessment for cfDNA testing for the detection of defined pathogenic variants in cancer patients. The pilot demonstrated that the EQA could be delivered using artificial plasma with spiked-in pathogenic variants albeit using dry ice to deliver the samples to participating laboratories [14]. The pilot EQA examined the accuracy of *EGFR*, *KRAS* and *NRAS* variant testing and demonstrated a high-test error rate. In particular, it was observed that for *EGFR* testing, the majority of participants did not detect the c.2369C > T p.(Thr790Met) resistance variant in a case where both this variant and an activating deletion in exon 19 of *EGFR* were present each with a variant allelic frequency (vaf) of approximately 1%. This is a common clinical scenario and laboratories would be expected to detect all variants or at least caveat their clinical report with the limitations of the testing performed.

In addition to the high error testing rate, it was observed that there was a variability in the content of the reports submitted by the participating laboratories. Many laboratories over-interpreted wild-type results and failed to include information considered essential for the correct understanding of laboratory reports. Following release of the EQA results to participating laboratories, an international workshop was held to address some of these issues and also to further discuss

areas around testing and reporting of results [4]. Both the original pilot and the subsequent workshop provided guidance for laboratories to improve their testing and reporting in this field [4, 14].

Building on the experience of the 2017 pilot EQA, in 2018 five European EQA providers, Associazione Italiana di Oncologia Medica (AIOM), European Molecular Genetics Quality Network (EMQN), European Society of Pathology (ESP), Genomics Quality Assessment (GenQA) and Gen&Tiss, under the auspices of IQN-Path, provided an EQA for the testing of *EGFR* variants in cfDNA for NSCLC patients. Participation was open to any interested laboratory and 264 laboratories participated. In this paper we present the results of the 2018 EQA and include countries represented, methodologies employed, and the mutant detection rates determined. We demonstrate that a large-scale assessment can be delivered by multiple EQA providers, against the same standards, to promote high-quality testing and reporting in this field.

# Materials and methods

## EQA design

Five EQA provider members of IQN Path provided the EQA. It was designed to assess laboratories on the testing and reporting of plasma cfDNA for the presence of *EGFR* variants in the context of NSCLC. Participants registered through a single EQA provider and each EQA provider was responsible for the distribution of samples to their registered laboratories. Five samples were distributed for testing and all participating laboratories were supplied with identical materials. Participants were requested to test the samples according to their routine testing protocols and submit fully interpretative reports tailored to address the clinical questions detailed in the mock clinical case scenarios provided. Laboratories returned their results to the EQA provider with which they had registered. In addition, participants were requested to complete an online survey to capture relevant information, although this was not compulsory. The EQA was carried out according to the requirements of the International Standard ISO/IEC 17043:2010 [15].

Laboratories were provided with an individual score report and each EQA provider produced a general report for all their participating laboratories. In addition, an overall report of the results from all EQA providers was provided. Participating laboratories had the right to appeal if they disagreed with their scores by the assessment procedure. Appeals were considered anonymously by the expert assessment team and if upheld, scores were amended as required.
## Samples and validation

The EQA samples were manufactured to contain 250 ng of cfDNA fragmented to an average fragment size of 170 bp mimic patient cfDNA according to proprietary procedures by Horizon Dx (Cambridge, UK) Each sample was supplied as 3 mL aliquots of artificial plasma spiked with the cfDNA. The genotypes and VAF were determined by the EQA providers and incorporated common sensitising and resistance mutations at a range of frequencies. All were expected to be encountered through clinical service. Table 1 summarises the EGFR genotypes of the EQA samples. The samples were validated prior to distribution by five laboratories using a range of commonly used techniques to determine whether the expected genotyping results were obtainable and of reportable quality in different laboratories (Table 2). The techniques used were next generation sequencing (Oncomine™ Lung cfDNA Assay (ThermoFisher) and GeneRead™ QIAAct Lung UMI (Qiagen)), real time PCR (.therascreen® EGFR Plasma RGQ PCR Kit (Qiagen) and cobas® EGFR Mutation Test v2 (Roche)) and digital droplet PCR (Biorad and Stilla assays). The samples were distributed to the validating and participating laboratories at ambient temperature and storage at 4°C upon receipt was recommended.

## Assessment

Assessment was performed by two expert advisors against peer-assessed marking criteria (Supplementary

Table 1). Each EQA provider was responsible for the overall assessment of their registered laboratories. A harmonisation meeting between all the EQA providers ensured that the marking was standardised and any issues across participating laboratories were discussed, resulting in a uniform approach to assessment.

Participant reports were assessed for genotyping accuracy. In addition, four of the EQA providers (EMQN, ESP EQA, Gen&Tiss and GenQA) also reviewed the interpretation of the genotypes provided and the clerical accuracy of the reports. All categories were marked out of a maximum of 2 points; marks were deducted for genotyping errors and where aspects of a report considered to be essential were missing.

The standard of performance was based solely on the genotyping score; laboratories were considered poor performers if they scored 0 for any of the cases i.e., reported an incorrect genotyping result (false positive and/or false negative).

## Computational and statistical analysis

EQA participant and validation data were analysed using Microsoft Excel 2013 (Microsoft, Redmond, WA, United States of America). The overall error rate was calculated by dividing the total number of false positive, false negative and 'incorrect variant reported' results over the total number of genotypes reported by the participants. Technical failures and unmarked reports were not included in the total numbers.

<table><caption>**Table 1** Summary of clinical cases and expected results</caption><thead><tr><td>Sample</td><td>Case</td><td>Reason for referral</td><td>Genotype</td></tr></thead><tbody><tr><td>IQN Path Sample 2018 – A</td><td>1</td><td>Never smoker patient, diagnosed with metastatic lung adenocarcinoma at age 62. EGFR testing performed on the patient's tumour biopsy specimen failed. Testing for EGFR gene mutations on the patient's plasma sample has been requested.</td><td>c.2236_2250del p.(Glu746_Ala750del) (1.3% VAF)</td></tr><tr><td>IQN Path Sample 2018 – B</td><td>2</td><td>Patient with metastatic lung adenocarcinoma diagnosed at age 80. After resection, tumour tissue was analysed and no EGFR variant was detected. EGFR gene testing has been requested on the patient's plasma sample.</td><td>No mutations detected within regions tested</td></tr><tr><td>IQN Path Sample 2018 – C</td><td>3</td><td>Patient with metastatic lung adenocarcinoma, diagnosed at age 68. Patient received first line EGFR-TKI treatment and is now in clear clinical progression. No tissue sample or cytology specimen of progressing disease is available due to their poor clinical condition. Testing of the patient's plasma sample for EGFR gene variants has been requested.</td><td>c.2369C>T p.(Thr790Met) (5.1% VAF) and c.2573T>G p.(Leu858Arg) (4.7% VAF)</td></tr><tr><td>IQN Path Sample 2018 – D</td><td>4</td><td>Patient diagnosed with metastatic lung adenocarcinoma at age 55. The patient was found to have an EGFR mutation and received first line treatment with an EGFR-TKI. At progression of the disease on TKI, the patient had a tissue biopsy but no tumour cells were present. EGFR gene testing has been requested on the patient's plasma sample</td><td>c.2236_2250del p.(Glu746_Ala75del) (6.2% VAF)</td></tr><tr><td>IQN Path Sample 2018 – E</td><td>5</td><td>Patient diagnosed with EGFR-mutant metastatic lung adenocarcinoma at age 65. The patient has a radiological progression of their primary tumour wheras the metastatic lesions are stable. Testing for EGFR gene variants on patient's plasma sample has been requested.</td><td>c.2369C>T p.(Thr790Met) (0.81% VAF) and c.2573T>G p.(Leu858Arg) (0.49% VAF)</td></tr></tbody></table>
<table><caption>**Table 2** Validated results for EQA samples</caption><thead><tr><th rowspan="2">Expected results (manufacturer's data using ddPCR)</th><th colspan="4">Sample / Case</th><th rowspan="2">Testing method</th></tr><tr><th>A / 1</th><th>B / 2</th><th>C / 3</th><th>D / 4</th><th>E / 5</th></tr></thead><tbody><tr><td>Laboratory 1</td><td>Deletion in exon 19 (uncharacterised)</td><td>WT</td><td>c.2236_2250del p.(Glu746_Ala750del) (1.3% VAF)</td><td>c.2369C>T p.(Thr790Met) (5.1% VAF) and c.2573T>G p.(Leu858Arg) (4.7% va)</td><td>c.2369C>T p.(Thr790Met) (0.81% VAF) and c.2573T>G p.(Leu858Arg) (0.49% VAF)</td><td>therascreen® EGFR Plasma RGQ PCR Kit (Qiagen)</td></tr><tr><td>Laboratory 1</td><td>c.2236_2250del p.(Glu746_Ala750del) (1.11% VAF)</td><td>WT</td><td>c.2369C>T p.(Thr790Met) (4.46% VAF)</td><td>c.2236_2250del p.(Glu746_Ala750del) (6.26% VAF)</td><td>c.2369C>T p.(Thr790Met) (0.66% VAF)</td><td>Oncomine™ Lung cfDNA Assay (Life Technologies) on S5XL System (Life Technologies)</td></tr><tr><td>Laboratory 1</td><td>c.2236_2250del p.(Glu746_Ala750del) (1.17% VAF)</td><td>WT</td><td>c.2369C>T p.(Thr790Met) (4.56% VAF)</td><td>c.2236_2250del p.(Glu746_Ala750del) (2.62% VAF)</td><td>c.2573T>G p.(Leu858Arg) (0.33% VAF)</td><td>GeneRead™ QIAact Lung UMI Panel (Qiagen) on GeneReader NGS System (Qiagen)</td></tr><tr><td>Laboratory 2</td><td>Deletion in exon 19 (uncharacterised)</td><td>WT</td><td>c.2369C>T p.(Thr790Met) c.2573T>G p.(Leu858Arg)</td><td>Deletion in exon 19 (uncharacterised)</td><td>c.2573T>G p.(Leu858Arg)</td><td>Stilla ddPCR Technologies</td></tr><tr><td>Laboratory 3</td><td>Deletion in exon 19 (uncharacterised)</td><td>WT</td><td>c.2369C>T p.(Thr790Met) c.2573T>G p.(Leu858Arg)</td><td>Deletion in exon 19 (uncharacterised)</td><td>c.2369C>T p.(Thr790Met)</td><td>cobas® EGFR Mutation Test v2 (Roche)</td></tr><tr><td>Laboratory 4</td><td>N/A</td><td>N/A</td><td>c.2369C>T p.(Thr790Met) c.2573T>G p.(Leu858Arg)</td><td>N/A</td><td>c.2369C>T p.(Thr790Met)</td><td>cobas® EGFR Mutation Test v2 (Roche)</td></tr><tr><td>Laboratory 5</td><td>Deletion in exon 19 (uncharacterised) (2.7% VAF)</td><td>WT</td><td>c.2369C>T p.(Thr790Met) (4.5% VAF)</td><td>Deletion in exon 19 (uncharacterised) (5.6%)</td><td>c.2369C>T p.(Thr790Met) (0.65% VAF)</td><td>ddPCR (BioRad-assays)</td></tr><tr><td>Laboratory 5</td><td>N/A</td><td>N/A</td><td>c.2369C>T p.(Thr790Met) c.2573T>G p.(Leu858Arg) (5.2% VAF)</td><td>N/A</td><td>c.2573T>G p.(Leu858Arg) (0.59% VAF)</td><td>cobas® EGFR Mutation Test v2 (Roche)</td></tr></tbody></table>

N/A Not analysed; WT wild type (no mutation detected); ddPCR digital droplet PCR; Ex19del deletion in exon 19 of EGFR; VAF variant allele frequency

Nomenclature according to EGFR gene reference sequence LRG_304t1

# Results

## Participation

A total of 304 laboratories from 45 different countries registered across the five EQA providers (Fig. 1) and 264 (87%) submitted results. Two of the EQA providers are National Schemes (AIOM – Italy and Gen&Tiss - France) therefore the laboratories participating with these providers were from single countries.

Five laboratories were unable to extract cfDNA from the artificial plasma samples provided. All five of these laboratories used the same DNA extraction method which suggested an issue of incompatibility of the artificial plasma with the particular DNA extraction technique (Helix Circulating Nucleic Acid (Diatech Pharmacogenetics, Italy); these laboratories were
<figure><img src="image_3.png" /></figure>

excluded from the results of the EQA. No other laboratories using this method participated in the EQA.

A total of 1316 reports were assessed. One laboratory only submitted results for case 1, without explanation and was excluded from the results for the other cases. The technical failure rate of the EQA was just under 2% (26/1316 samples reported). A small number of laboratories reported leaking of the plasma from the tubes during transportation; the majority were supplied with replacement samples to ensure they were not disadvantaged by sub-optimal EQA material. An audit was performed of the EQA results from laboratories that did test the leaking samples and all reported the correct genotyping results, confirming that the leakage did not cause any subsequent testing issues.

Three laboratories had at least one of their reports not marked; one laboratory submitted screenshots of

the online survey and the data was not assessed for all the cases, one laboratory did not perform testing of cases 1 and 2 as their testing strategy only included testing of recurrence cases for the EGFR c.2369C > T p.(Thr790Met) variant; the scenario provided for these cases was for a primary tumour so these cases were not marked for this laboratory and the third laboratory did not submit a report for case 5.

### Genotyping accuracy

The mean scores obtained by participating laboratories are shown in Table 3. The genotyping results were further sub-divided as to the accuracy of the result or type of error made (Table 4).

Overall, the results for cases 1, 2, 3 and 4 were of a very high standard with most laboratories reporting the correct genotyping results (92% for case 1, 94% for case

<table><caption>**Table 3** Mean scores of all participating laboratories (maximum score = 2)</caption><thead><tr><th>Category</th><th>Case 1</th><th>Case 2</th><th>Case 3</th><th>Case 4</th><th>Case 5</th><th>All cases</th></tr></thead><tbody><tr><td>Mean Genotyping Score</td><td>1.83</td><td>1.93</td><td>1.82</td><td>1.86</td><td>1.83</td><td>1.85</td></tr><tr><td>Mean Interpretation Score<sup>a</sup></td><td>1.58</td><td>1.45</td><td>1.54</td><td>1.40</td><td>1.49</td><td>1.49</td></tr><tr><td>Mean Clerical Accuracy Score<sup>a</sup></td><td>1.89</td><td>1.9</td><td>1.9</td><td>1.89</td><td>1.89</td><td>1.89</td></tr></tbody></table>

<footnote>a Not assessed for AIOM laboratories</footnote>
<table><caption>**Table 4** Breakdown of results submitted by participating laboratories</caption><thead><tr><th></th><th>Case 1</th><th>Case 2</th><th>Case 3</th><th>Case 4</th><th>Case 5</th></tr></thead><tbody><tr><td>Correct result and correct nomenclature used</td><td>106</td><td>248</td><td>193</td><td>122</td><td>115</td></tr><tr><td>Correct result (variant uncharacterised)</td><td>125</td><td>N/A</td><td>10</td><td>88</td><td>4</td></tr><tr><td>Correct result (incorrect nomenclature)</td><td>12</td><td>N/A</td><td>43</td><td>39</td><td>26</td></tr><tr><td>One of the two variants present not reported</td><td>N/A</td><td>N/A</td><td>7</td><td>N/A</td><td>79</td></tr><tr><td>False negative result</td><td>13</td><td>N/A</td><td>3</td><td>3</td><td>24</td></tr><tr><td>False positive result</td><td>2</td><td>5</td><td>1</td><td>5</td><td>0</td></tr><tr><td>Incorrect EGFR variant reported</td><td>1</td><td>N/A</td><td>1</td><td>1</td><td>6</td></tr><tr><td>Technical failure</td><td>3</td><td>8</td><td>4</td><td>4</td><td>7</td></tr><tr><td>Not marked</td><td>2</td><td>2</td><td>1</td><td>1</td><td>2</td></tr></tbody></table>

N/A Not applicable

2, 94% for case 3 and 95% for case 4) not taking into account the correct use of mutation nomenclature. There were 19 (1.8%) false negative results, 13 (1.2%) false positive results and 3 (0.3%) reports where an incorrect *EGFR* variant was reported. The sample provided for case 3 had two *EGFR* variants present: seven (2.7%) laboratories only reported the presence of one variant.

Cases 1 and 4 both contained the same deletion in exon 19 of the *EGFR* gene. Laboratories used incorrect nomenclature to describe the variants as follows: 12 (5%) of laboratories in case 1 and 39 (15%) of laboratories in case 4, resulting in genotyping score deductions. For case 1, 125 (46%) laboratories and 8 (33%) for case 4 were unable to characterise the mutations due to the technology used for testing (mainly PCR based technologies) which did not enable the mutation present to be fully characterised. These laboratories were not penalised because the exact genotyping is not required for treatment-decision-making. The case 3 variants were reported using incorrect nomenclature by 43 (16%) laboratories resulting genotyping score deductions.

Case 5 was a more challenging sample with two variants present in *EGFR*: c.2369C > T p.(Thr790Met) at 0.81% allelic frequency and c.2573T > G p.(Leu858Arg) at 0.49% allelic frequency. The variant allele frequencies (vafs) for both had previously been measured using digital droplet PCR by the commercial manufacturer of the materials. Validation results for these samples were inconsistent with regards to the detection of the presence of the *EGFR* c.2573T > G p.(Leu858Arg) variant and the c.2369C > T variants in the sample (see Table 2). Two of the validation techniques (Oncomine™ Lung cfDNA assay and Bio-Rad ddPCR assay) were able to consistently

detect both mutations but the cobas® EGFR Mutation Test v2 (Roche) was not in 3 independent laboratories, along with Silla ddPCR technology, the therascreen EGFR Plasma RGQ PCR kit (Qiagen) and GeneRead™ QIAact Lung UMI Panel (Qiagen) on GeneReader NGS System (Qiagen). In addition, inconsistency in the reporting of the results was observed between laboratories using the same methodology during the EQA. For example of the 72 laboratories using cobas® EGFR Mutation Test v2 (Roche), only 18 (25%) reported the presence of both variants, a further 42 (58%) only reported the presence of the c.2369C > T p.(Thr790Met) variant, one laboratory (1%) reported no variants and one laboratory (1%) reported a false positive. Due to this inconsistency in the validation and during the EQA and the fact that only 55% of laboratories reported the correct result, a problem with the sample could not be ruled out and therefore the marking criteria was adjusted for this case as detailed in Supplementary Table 2. This ensured that the sample was included in the EQA assessment, and that laboratories were only deducted a full 2 marks if they reported an incorrect *EGFR* variant. The marking was also dependent on the sensitivity of the assay used, as reported by laboratories. If a laboratory reported a limit of detection (LOD) of their assay higher than that of the validated variant allele frequency of the specific variant, no marks were deducted.

The case 3 sample had two *EGFR* variants, c.2573T > G p.(Leu858Arg) and c.2369C > T p.(Thr790Met) with VAF 4.7 and 5.1% and 246 of 263 participants submitting reports for this case (94%) correctly detected both variants. Interestingly, in case 5 with the same variants but now with VAF 0.49 and 0.81%, both variants were reported by only 145 of 263 participants (55%). Twenty-four (9%) laboratories did not report the presence of both *EGFR* variants and 79 (33%) laboratories stated the presence of only one variant: 62 (23%) reported the presence of the c.2369C > T p.(Thr790Met) variant only. In addition, there were six (2%) laboratories that reported an incorrect *EGFR* mutation in this case. Taking into account the LOD of the assays used and described clearly in the report, along with the revised marking criteria, the mean genotyping score for this case (see Supplemental Table 2), however, was concordant to that of the other four cases (Table 3).

## Testing methodologies

The methods used by laboratories to extract DNA and test the samples were collected using the online survey and the submitted EQA reports. The five most frequently used commercial kits for DNA extraction were used by more than 80% of participating laboratories (see
<figure><img src="image_4.png" /></figure>

Fig. 2). All were designed specifically for cfDNA extrac-
tion. Extraction kits not specifically designed for use
for cfDNA were also applied by some laboratories e.g.
QIAamp DNA Blood Mini Kit and the QIAamp DNA
FFPE Tissue Kit. No specific data regarding the perfor-
mance of these kits in this EQA is available. As men-
tioned above, an issue was observed with laboratories
using the Helix Circulating Nucleic Acid (Diatech Phar-
macogenetics) and it is likely that the artificial plasma
used was incompatible with the commercial DNA extrac-
tion method.

The most frequently used testing strategy was Real-
Time PCR (51% of laboratories) (Fig. 3) of which the
cobas® EGFR Mutation Test v2 (Roche) assay was the
most commonly used test (27% of total laboratories).
Next Generation Sequencing (NGS) represented the
next most common testing strategy used by participat-
ing laboratories (27% of laboratories). A breakdown of
the NGS panels used is shown in Fig. 4. The most fre-
quently used panel was the Oncomine™ Lung cfDNA
Assay (8% of all laboratories). No other NGS panel was

used by more than eight laboratories with the major-
ity of panels being used by only one or two partici-
pants. As was the case for DNA extraction methods,
some laboratories used tests that were not specifically
designed for cfDNA, including NGS panels and other
technologies.

There was no correlation between any testing method
and results obtained for the EQA. As the EQA examined
the end-to-end testing strategy then both DNA extrac-
tion and testing methods were combined and the perfor-
mance of individual test methods was not possible.

**Reporting of results**

Laboratories registered for this EQA with EMQN, Gen&Tiss, ESP-EQA and GenQA were awarded scores for the interpretation of their reports (Table 3) using the expert opinion on the contents of cfDNA reports from an international workshop [4, 14] as a basis for the marking criteria (Supplementary Table 1). The mean scores for interpretation of the cases were lower than those for genotyping.
<figure><img src="image_5.png" /></figure>

The majority of reports (54%) for cases 1, 3 and 4 correctly described the mutations using Human Genome Variation Society (HGVS) nomenclature [16]. A further 28% of reports stated that techniques were used where it was not possible to characterize the variants detected and therefore use of HGVS nomenclature was not required. Some reports (12%) used either incorrect nomenclature or only reported the results at the amino acid level. General omissions from the reports included failure to provide correct gene reference sequences, reporting the sample type incorrectly (e.g. FFPE instead of plasma) or use of the terms ‘positive’ or ‘negative’ to describe the mutation status.

## Discussion

cfDNA testing is now a recognised method for testing for *EGFR* variants in NSCLC to influence targeted therapy treatment decisions [1], with its use complementing that of testing of tumour tissue. There is therefore a requirement to ensure that such testing is accurate and reported in a clear and precise manner. EQA can play an important role in demonstrating this by providing an external

measure of the standard of a clinical service and providing educational support when appropriate [17–19].

EQA enables the benchmarking of laboratory services through the provision of the same material to be tested and reported. The use of real patient material for large scale cfDNA EQAs is not feasible due to the amount of patient plasma that would be required. Previous EQAs delivered for testing of cfDNA *EGFR* variants have used alternative materials, for example, [20] provided DNA for testing but this approach had a limitation in that the complete testing process, including DNA extraction, was not able to be assessed. An EQA using plasma or blood samples spiked with DNA [14, 21] were delivered to a relatively small number of participants with sample transportation either temperature controlled by use of dry ice, or by next day delivery, if participants were localised to the same country. Such approaches are feasible when the volume of packages are small and the geographical distance travelled is minimised. In order to deliver EQA to multiple participants worldwide, this EQA utilised artificial plasma which enabled distribution at ambient temperature as it remained stable for a few days, long enough
<figure><img src="image_6.png" /></figure>

to reach participants in multiple continents. The procurement of the artificial plasma was harmonised between the five European EQA providers which enabled a large-scale comparison of laboratories globally and allowed increased efficiency and reduced cost of the material used. However, the artificial plasma appeared incompatible with one particular commercial DNA extraction method, but the technical failure rates reported were less than 2%, suggesting that the material is suitable for most DNA extraction methods. Today only a few inter-laboratory comparisons studies for somatic testing were performed to validate various cfDNA extraction methods with multiple testing methods, mostly involving a small numbers of participants using spiked-in DNA in pooled plasma [22], commercial reference-material [23] and patients-derived plasma [24]. Future EQA runs involving >250 participants would benefit from more extensive validation by incorporating multiple DNA extraction methods with multiple testing methods.

The overall genotyping error rate was 11.7% which was lower than that reported for previous EQAs where blood or plasma was the initial starting material [14, 21] but slightly higher than that where extracted cfDNA only was distributed [20]. In this EQA the error rate for samples harbouring no pathogenic variants or *EGFR* variants present at an allele frequency of above 1% (cases 1–4)

was 4.1% (42/1028). However for the sample which had two variants in *EGFR* present both below 1% (case 5), the error rate was 43% (109/254). It is recognised that the genotype of this sample was challenging and participating laboratories may not have used testing strategies with sufficient sensitivity to detect alleles at such low frequencies. Limited detection of these variants may also be assigned to the use of methods that were not specifically designed for cfDNA, both for extraction and testing. It is expected that this approach will be impacted upon when the IVD R is implemented and laboratories and the diagnostic industry will be required to complete robust validtaions for defined clinical utility [25]. An issue with the EQA sample cannot be excluded as testing gave inconsistent results during validation with only two of the techniques able to detect both the *EGFR* c.2573T > G p.(Leu858Arg) and the *EGFR* c.2369C > T p.(Thr790Met) present. Furthermore, in addition to having low variant allele frequencies for both variants, the *EGFR* c.2369C > T p.(Thr790Met) is associated with acquired resistance to EGFR tyrosine kinase inhibitors (EGFR-TKIs) [26] and would be expected to be present at a lower level than the original sensitising mutations in a tumour. However, in case 5 of this EQA the c.2369C > T variant was present at a higher level that the sensitising c.2583T > G sensitising variant The sample therefore may not be truly
representative of a clinical sample and assays may not be optimised for this scenario which may account for some of the variability in genotyping seen. Analysis of the same *EGFR* p.(Leu858Arg) and p.(Thr790Met) mutations with higher VAF (4.7 and 5.1%) in case 3 revealed correct identification of both variants by 246 of 263 participants (94%). This illustrates that the assays used overall have high specificity for these mutations. Although the genotyping case 5 was comparable to the analysis of the other cases taking into account the LOD of the assays used, the results from case 5 demonstrated that the detection of mutations at VAF < 1% requires more sensitive assays than those used by > 40% of the participating laboratories.

The scope of *EGFR* testing was variable. A small number of laboratories performed testing only for the c.2369C > T p.(Thr790Met) resistance variant. In the case of first line treatment with EGFR tyrosine kinase inhibitors (as in cases 1 and 2) it is inappropriate to only test for this resistance variant as this strategy will not provide the correct information for a clinician to determine the possibility of response to treatment. In the case of progressive disease, it is good practice to test for the original *EGFR* variant identified as this can provide useful data that tumour DNA has been tested and also on the sensitivity of the assay performed i.e. if a resistance mutation is not detected and the original variant is not detected, consideration must be given to the likelihood of insufficient test sensitivity [27]. However, it must be acknowledged that this EQA was run in 2018, when osimertinib was available only for patients progressing on EGFR TKI and with a p.(Thr790Met) variant [28]. The increasing use of osimertinib in the first line therapy of *EGFR* mutant lung cancer is leading a change to testing strategies. Although the same actionable *EGFR* mutations (e.g. deletions in exon 19 and c.2573 T > G p.(Leu858Arg) are also targets for osimertinib at first line, the resistance mechanisms on osimertinib are very different and do only show very low occurrence of T790M [28].

Including the scope of testing on the report is also of importance as assays may not be able to detect all clinically relevant variants e.g. insertions of exon 20 in *EGFR* or in the case of ddPCR less common hotspot variants such as those at position 719 or 861 of *EGFR*.

Many laboratories over-interpreted a “no mutations detected” result, advising that the absence of a mutation indicated that the patient would be unlikely to respond to EGFR TKIs. This interpretation is inappropriate, and more caution is advised; cfDNA analyses are known to have reduced sensitivity for detecting variants [29–31] and there is a chance that either no tumour DNA was tested or the variant level was below the limits of detection of the assay performed. As

observed in the IQNPath pilot EQA delivered in 2017 [14], some reports provided insufficient information regarding the limitations of the test performed; these are required to determine how reliable a negative result is [4]. This will impact on the clinical interpretation of a test result with a negative result in a less sensitive assay being more likely to be a false negative. Circulating DNA has been shown to represent a small fraction (< 1%) of total cell free DNA [32] therefore detection of variants with allelic frequencies below 1% in cfDNA is thought to be important. As not all techniques have the same limits of detection, including inter-laboratory, it is important for this information to be included in clinical reports so the reader can determine whether there is a likelihood of a false negative result.

To ensure standardization in the reporting of genomic changes, the use of HGVS nomenclature and inclusion of an appropriate reference sequence in reports is important [33]. Laboratories followed HGVS guidance variably. The most frequent error was in the reporting of variants using only the predicted amino acid change(s) and not reporting the nucleotide variant(s). As the assays performed are DNA based tests, the variants detected should be described in terms of the nucleotide change(s) and ideally with the predicted amino acid changes also provided.

## Conclusions

This EQA demonstrated the feasibility of delivering a large worldwide EQA assessing accuracy and reporting of cfDNA testing for *EGFR* variants in lung cancer. However, the EQA has highlighted issues across the process in both testing of the samples and reporting of the results. Laboratories should follow up any errors highlighted in this and endeavour to implement process to improve the clinical pathway. Subsequent runs of EQA should aid in the quality assessment process.

## Abbreviations

AIOM: Associazione Italiana di Oncologia Medica; cfDNA: Circulating cell-free DNA; ddPCR: Droplet digital polymerase chain reaction; EGFR: Epidermal growth factor receptor; EMQN: European Molecular Genetics Quality Network; EQA: External quality assessment; ESP: European Society of Pathology; GenQA: Genomics Quality Assessment; HGVS: Human Genome Variation Society; IQNPath: International Quality Network for Pathology; NGS: Next Generation Sequencing; LOD: Limit of detection; NSCLC: Non-small-cell lung cancer; QC: Quality control; VAF: Variant allele frequency.

## Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1186/s12885-022-09849-x.
**Acknowledgments**

N/A

**Authors' contributions**

JAF, MHC, SJP, ER, MD, EMCD, KVC, FF, NN and ZCD and EMCD were responsible for the setup and delivery of the EQA, data collection and statistical analysis. JAF interpreted the data and wrote the manuscript. ES provided technical assistance for the EQA and took part in assessing the submissions. All authors critically revised the manuscript for important intellectual content. The author(s) read and approved the final manuscript.

**Funding**

The EQA was supported by an unrestricted educational grant from AstraZeneca and further financial support was provided by Novartis. Both sources of funding were awarded irrespective of the presented research; grant numbers are not applicable.

**Availability of data and materials**

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

**Declarations**

**Ethics approval and consent to participate**

Not applicable as the study did not involve human participants or human material.

**Consent for publication**

Not applicable.

**Competing interests**

ZCD: Advisory Boards for AstraZeneca, lectures and webinars for AstraZeneca and Merck MSD.

JAF Advisory Board for Eli Lilly, lectures for AstraZeneca.

ES reports providing lectures for Bio-Rad, Novartis, Roche, Biocartis, Illumina, Pfizer, AstraZeneca, and Agena Bioscience, advisory board presence for AstraZeneca, Roche, Pfizer, Novartis, Bayer, Lilly, BMS, Amgen, Biocartis, Illumina, Agena Bioscience and MSD/Merck, and research grants from Pfizer, Biocartis, Invitae-ArcherDX, AstraZeneca, Agena Bioscience, BMS, Bio-Rad, Roche, Boehringer Ingelheim, all outside the submitted work and all financial supports transferred to Institute.

MD Personal fees (advisory board and lectures) from AstraZeneca, Takeda, BMS, Boehringer Ingelheim, Pfizer, Roche Diagnostics, and Amgen. Grants from BluePrint Medicines and Takeda.

NN Personal and/or Institutional financial interests: Thermofisher, QIAGEN, Roche, AstraZeneca, Biocartis, Illumina.

ER Personal fees (consulting and travel) - AstraZeneca, BMS, Roche.

SJP, MHC, KVC, FF and EMCD have no interests to declare.

**Author details**

¹GenQA, Nine, Edinburgh Bioquarter, 9 Little France Road, Edinburgh EH16 4SA, UK. ²EMQN CIC, Unit 4, Enterprise House, Pencroft Way, Manchester Science Park, Manchester M15 6SE, UK. ³Medical Biology and Pathology Department, Gustave Roussy, Villejuif, France. ⁴Department of Biochemistry and INSERM U1232, Centre Hospitalier Universitaire de Nantes, 9 quai Moncousu, F-44093 Nantes Cedex, France. ⁵Department of Public Health and Primary Care, Biomedical Quality Assurance Research Unit, KU Leuven, Kapucijnenvoer 35d, 3000 Leuven, Belgium. ⁶Department of Pathology and Medical Biology, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands. ⁷IQN Path ASBL, Luxembourg, Luxembourg. ⁸Istituto Nazionale Tumori-IRCCS-Fondazione G. Pascale, 80131 Napoli, Italy.

Received: 16 March 2022 Accepted: 1 July 2022

Published online: 12 July 2022

**References**

1. Kalemkerian GP, Narula N, Kennedy EB, Biermann WA, Donington J, Leighl NB, et al. Molecular testing guideline for the selection of

patients with lung cancer for treatment with targeted tyrosine kinase inhibitors: American society of clinical oncology endorsement of the college of American pathologists/ international association for the. J Clin Oncol. 2018;36(9):911-9.

2. Rolfo C, Mack P, Scagliotti GV, Aggarwal C, Arcila ME, Barlesi F, et al. Liquid biopsy for advanced NSCLC: a consensus statement from the International Association for the Study of Lung Cancer. J Thorac Oncol. 2021;16(10):1647-62.

3. Goldman JW, Noor ZS, Remon J, Besse B, Rosenfeld N. Are liquid biopsies a surrogate for tissue EGFR testing? Ann Oncol. 2018;29:i38-46.

4. Deans ZC, Butler R, Cheetham M, Dequeker EMC, Fairley JA, Fenizia F, et al. IQN path ASBL report from the first European cfDNA consensus meeting: expert opinion on the minimal requirements for clinical ctDNA testing. Virchows Arch. 2019;474(6):681-9.

5. Snyder MW, Kircher M, Hill AJ, Daza RM, Shendure J. Cell-free DNA comprises an in vivo nucleosome footprint that informs its tissues-of-origin. Cell. 2016;164(1-2):57-68.

6. Heitzer E, Haque IS, Roberts CES, Speicher MR. Current and future perspectives of liquid biopsies in genomics-driven oncology. Nat Rev Genet. 2019;20(2):71-88.

7. Mouliere F, Rosenfeld N. Circulating tumor-derived DNA is shorter than somatic DNA in plasma. Proc Natl Acad Sci U S A. 2015;112(11):3178-9.

8. Normanno N, Denis MG, Thress KS, Ratcliffe M, Reck M. Guide to detecting epidermal growth factor receptor (EGFR) mutations in ctDNA of patients with advanced non-small-cell lung cancer. Oncotarget. 2017;8(7):12501-16.

9. Deans ZC, Wallace A, O'Sullivan B, Purvis A, Camus S, Fairley JA, et al. External quality assessment of BRAF molecular analysis in melanoma. J Clin Pathol. 2014;67(2):120-4.

10. Deans ZC, Bilbe N, O'Sullivan B, Lazarou LP, de Castro DG, Parry S, et al. Improvement in the quality of molecular analysis of EGFR in non-small-cell lung cancer detected by three rounds of external quality assessment. J Clin Pathol. 2013;66(4):319-25.

11. Deans ZC, Tull J, Beighton G, Abbs S, Robinson DO, Butler R. Molecular genetics external quality assessment pilot scheme for kras analysis in metastatic colorectal cancer. Genet Testing Mol Biomarkers. 2011;15(11):777-83.

12. Tack V, Lichtenberg MJL, Tembuyser L, Normanno N, vander Borgh S, Han van Krieken J, et al. External quality assessment unravels Interlaboratory differences in quality of RAS testing for anti-EGFR therapy in colorectal Cancer. Oncologist. 2015;20(3):257-62.

13. Dequeker EMC, Keppens C, Egele C, Delen S, Lamy A, Lemoine A, et al. Three rounds of external quality assessment in France to evaluate the performance of 28 platforms for multiparametric molecular testing in metastatic colorectal and non-small cell lung Cancer. J Mol Diagnostics. 2016;18(2):205-14.

14. Keppens C, Dequeker EMC, Patton SJ, Normanno N, Fenizia F, Butler R, et al. International pilot external quality assessment scheme for analysis and reporting of circulating tumour DNA. BMC Cancer. 2018;18(1).

15. Conformity assessment - general requirements for proficiency testing. ISO/IEC 17043:2010 ISO; 2010.

16. Sequence Variant Nomenclature [Internet]. [cited 2022 Mar 7]. Available from: https://varnomen.hgvs.org/

17. Keppens C, Schuuring E, Dequeker EM. Managing deviating EQA results: a survey to assess the corrective and preventive actions of medical laboratories testing for oncological biomarkers. Diagnostics. 2020;10(10):837.

18. Keppens C, Schuuring E, Dequeker EMC. Causes behind error rates for predictive biomarker testing: the utility of sending post-EQA surveys. Virchows Arch. 2021;478(5):995-1006.

19. Tack V, Schuuring E, Keppens C, Hart N 't, Pauwels P, van Krieken H, et al. Accreditation, setting and experience as indicators to assure quality in oncology biomarker testing laboratories. Br J Cancer. 2018;119(5):605-14.

20. Chai SY, Peng R, Zhang R, Zhou L, Pillay N, Tay KH, et al. External quality Assurance of Current Technology for the testing of Cancer-associated circulating free DNA variants. Pathol Oncol Res. 2020;26(3):1595-603.

21. Fassunke J, Ihle MA, Lenze D, Lehmann A, Hummel M, Vollbrecht C, et al. EGFR T790M mutation testing of non-small cell lung cancer tissue and blood samples artificially spiked with circulating cell-free tumor DNA: results of a round robin trial. Virchows Arch. 2017;471(4):509-20.

22. Lampignano R, Neumann MHD, Weber S, Kloten V, Herdean A, Voss T, et al. Multicenter evaluation of circulating cell-free DNA extraction and
downstream analyses for the development of standardized (pre) analyti-
cal work flows. Clin Chem. 2020;66(1):149-60.

23. Weber S, Spiegl B, Perakis SO, Ulz CM, Abuja PM, Kashofer K, et al. Techni-
cal evaluation of commercial mutation analysis platforms and reference
materials for liquid biopsy profiling. Cancers. 2020;12(6):1588.

24. van der Leest P, Boonstra PA, ter Elst A, van Kempen LC, Tibbesma M, Koo-
pmans J, et al. Comparison of circulating cell-free DNA extraction meth-
ods for downstream analysis in Cancer patients. Cancers. 2020;12(5):1222.

25. Lubbers BR, Schilhabel A, Cobbaert CM, Gonzalez D, Dombrink I, Brügge-
mann M, et al. The new EU regulation on in vitro diagnostic medical
devices: implications and preparatory actions for diagnostic laboratories.
HemaSphere. 2021;5(5):e568.

26. Kobayashi S, Boggon TJ, Dayaram T, Jänne PA, Kocher O, Meyerson M,
et al. EGFR mutation and resistance of non-small-cell lung Cancer to
Gefitinib. New Engl J Med. 2005;352(8):786-92 Feb 24 [cited 2021 May 10]
Available from: http://www.nejm.org/doi/abs/10.1056/NEJMoa044238.

27. Normanno N, Maiello MR, Chicchinelli N, Iannaccone A, Esposito C, de
Cecio R, et al. Targeting the EGFR T790M mutation in non-small-cell lung
cancer. Expert Opin Ther Targets. 2017;21(2):159-65.

28. Schoenfeld AJ, Yu HA. The evolving landscape of resistance to Osimertinib. J Thorac Oncol. 2020;15(1):18-21.

29. Thress KS, Brant R, Carr TH, Dearden S, Jenkins S, Brown H, et al. EGFR mutation detection in ctDNA from NSCLC patient plasma: a cross-platform comparison of leading technologies to support the clinical development of AZD9291. Lung Cancer. 2015;90(3):509-15.

30. Douillard J-Y, Ostoros G, Cobo M, Ciuleanu T, Cole R, McWalter G, et al. Gefitinib treatment in EGFR mutated Caucasian NSCLC: circulating-free tumor DNA as a surrogate for determination of EGFR status. J Thorac Oncol. 2014;9(9):1345-53.

31. Denis MG, Lafourcade M-P, le Garff G, Dayen C, Falchero L, Thomas P, et al. Circulating free tumor-derived DNA to detect EGFR mutations in patients with advanced NSCLC: French subset analysis of the ASSESS study. J Thor Dis. 2019;11(4):1370-8.

32. Diaz LA, Bardelli A. Liquid biopsies: genotyping circulating tumor DNA. J Clin Oncol. 2014;32(6):579-86.

33. Tack V, Deans ZC, Wolstenholme N, Patton S, Dequeker EMC. What's in a name? A coordinated approach toward the correct use of a uniform nomenclature to improve patient reports and databases. Hum Mutat. 2016;37(6):570-5.

**Publisher’s Note**

Springer Nature remains neutral with regard to jurisdictional claims in pub-
lished maps and institutional affiliations.

Ready to submit your research? Choose BMC and benefit from:

- fast, convenient online submission

* thorough peer review by experienced researchers in your field

- rapid publication on acceptance

- support for research data, including large and complex data types

* gold Open Access which fosters wider collaboration and increased citations

* maximum visibility for your research: over 100M website views per year

At BMC, research is always in progress.

Learn more biomedcentral.com/submissions

<figure><img src="image_7.png" /></figure>