# Multicenter Analytical Validation of Aβ40 Immunoassays

Linda van Waalwijk van Doorn, Luka Kulic, Marleen Koel-Simmelink, H. Bea Kuiperij, Alexandra Versleijen, Hanne Struyfs, Harry Twaalfhoven, Anthony Fourier, Sebastiaan Engelborghs, Armand Perret-Liaudet, et al.

<figure><img src="image_1.png" /></figure>

► To cite this version:

Linda van Waalwijk van Doorn, Luka Kulic, Marleen Koel-Simmelink, H. Bea Kuiperij, Alexandra Versleijen, et al. Multicenter Analytical Validation of Aβ40 Immunoassays. Frontiers in Neurology, 2017, 8, pp.310. 10.3389/fneur.2017.00310. hal-01758275

HAL Id: hal-01758275

https://hal.science/hal-01758275

Submitted on 7 Jun 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

Check for
updates

# Multicenter Analytical Validation of Aβ40 Immunoassays

Linda J. C. van Waalwijk van Doorn¹,², Luka Kulic³, Marleen J. A. Koel-Simmelink⁴, H. Bea Kuiperij¹,², Alexandra A. M. Versleijen², Hanne Struyfs⁵, Harry A. M. Twaalfhoven⁴, Anthony Fourier⁶, Sebastiaan Engelborghs⁵,⁷, Armand Perret-Liaudet⁶, Sylvain Lehmann⁸, Marcel M. Verbeek¹,², Eugeen J. M. Vanmechelen⁹ and Charlotte E. Teunissen⁴*

¹Department of Neurology, Radboud University Medical Center, Radboud Alzheimer Centre, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, Netherlands, ²Department of Laboratory Medicine, Radboud University Medical Center, Radboud Alzheimer Centre, Donders Institute for Brain, Cognition and Behaviour, Nijmegen, Netherlands, ³Institute for Regenerative Medicine (IREM), University of Zurich, Schlieren, Switzerland, ⁴Neurochemistry Laboratory and Biobank, Department of Clinical Chemistry, VU University Medical Center, Neurocampus, Amsterdam, Netherlands, ⁵Reference Center for Biological Markers of Dementia (BIODEM), Institute Born-Bunge, University of Antwerp, Antwerp, Belgium, ⁶Neurobiology Laboratory, Centre for Memory Resources and Research (CMRR), Groupement Hospitalier Est (GHE), Hôpitaux de Lyon, Université Lyon 1, CNRS UMR5292, INSERM U1028, Lyon, France, ⁷Memory Clinic and Department of Neurology, Hospital Network Antwerp (ZNA) Middelheim and Hoge Beuken, Antwerp, Belgium, ⁸CHU de Montpellier and Université de Montpellier, IRMB, Laboratoire de Biochimie Protéomique Clinique, Montpellier, France, ⁹R&D, ADx NeuroSciences, Ghent, Belgium

OPEN ACCESS

**Edited by:**
Raymond Scott Turner,
Georgetown University, United States

**Reviewed by:**
Luke Esposito,
Allen Institute for Brain Science,
United States
Alberto Lleo,
Hospital Sant Pau, Spain

*Correspondence:
Charlotte E. Teunissen
c.teunissen@vumc.nl

**Specialty section:**
This article was submitted
to Neurodegeneration,
a section of the journal
Frontiers in Neurology

**Received:** 05 May 2017
**Accepted:** 14 June 2017
**Published:** 03 July 2017

**Citation:**
van Waalwijk van Doorn LJC, Kulic L, Koel-Simmelink MJA, Kuiperij HB, Versleijen AAM, Struyfs H, Twaalfhoven HAM, Fourier A, Engelborghs S, Perret-Liaudet A, Lehmann S, Verbeek MM, Vanmechelen EJM and Teunissen CE (2017) Multicenter Analytical Validation of Aβ40 Immunoassays. *Front. Neurol.* 8:310. doi: 10.3389/fneur.2017.00310

**Background:** Before implementation in clinical practice, biomarker assays need to be thoroughly analytically validated. There is currently a strong interest in implementation of the ratio of amyloid-β peptide 1-42 and 1-40 (Aβ42/Aβ40) in clinical routine. Therefore, in this study, we compared the analytical performance of six assays detecting Aβ40 in cerebrospinal fluid (CSF) in six laboratories according to a recently standard operating procedure (SOP) developed for implementation of ELISA assays for clinical routine.

**Methods:** Aβ40 assays of six vendors were validated in up to three centers per assay according to recently proposed international consensus validation protocols. The performance parameters included sensitivity, precision, dilutional linearity, recovery, and parallelism. Inter-laboratory variation was determined using a set of 20 CSF samples. In addition, test results were used to critically evaluate the SOPs that were used to validate the assays.

**Results:** Most performance parameters of the different Aβ40 assays were similar between labs and within the predefined acceptance criteria. The only exceptions were the out-of-range results of recovery for the majority of experiments and of parallelism by three laboratories. Additionally, experiments to define the dilutional linearity and hook-effect were not executed correctly in part of the centers. The inter-laboratory variation showed acceptable low levels for all assays. Absolute concentrations measured by the assays varied by a factor up to 4.7 for the extremes.

**Conclusion:** All validated Aβ40 assays appeared to be of good technical quality and performed generally well according to predefined criteria. A novel version of the validation SOP is developed based on these findings, to further facilitate implementation of novel immunoassays in clinical practice.

**Keywords:** method validation, Alzheimer's disease, cerebrospinal fluid, amyloid, Immunoassays

**Abbreviations:** Aβ, amyloid-β peptide; CSF, cerebrospinal fluid; SOP, standard operating procedure; AD, Alzheimer's disease; t-tau, total tau protein; p-tau, tau phosphorylated at threonine 181; MCI, mild cognitive impairment; PET, positron emission tomography; LLOQs, lower limits of quantification; 4PL, four parameter logistic; %CV, coefficient of variation; QC, quality control; ELISA, enzyme-linked immunosorbent assay; CE, European Conformity.
# INTRODUCTION

Cerebrospinal fluid (CSF) biomarkers have proven to be helpful in early diagnosis of Alzheimer's disease (AD). They reflect preclinical early events in AD by as many as 10-15 years before clinical symptoms occur (1). Therefore, CSF biomarkers have been incorporated in the diagnostic criteria of AD (2-4).

The CSF biomarkers most prominently used in Alzheimer's diagnostics are amyloid-β peptide 1-42 (Aβ42), total tau protein (t-tau), and tau phosphorylated at threonine 181 (p-tau), because they reflect the pathological hallmarks of AD (5, 6). A decrease in CSF Aβ42 levels probably reflects the extent of amyloid-β accumulation in the formation of plaques in the brain, while an increase in CSF t-tau and p-tau levels likely reflects neuronal degeneration and intracellular tangle formation. The ratio of Aβ42/Aβ40 is helpful to correct levels of Aβ42 for the total amyloid production (7) and provide better assessment of the presence of amyloid pathology in case of for instance cerebral amyloid angiopathy (8). For example, recent studies demonstrated that the CSF Aβ42/Aβ40 ratio significantly improved the diagnostic performance compared to CSF Aβ42 alone in distinguishing controls or non-AD patients from mild cognitive impairment or AD patients (9-13). The concordance with amyloid positron emission tomography increased when the CSF Aβ42/Aβ40 ratio was used as compared to CSF Aβ42 alone (14). Therefore, there is currently a strong interest to implement the Aβ42/Aβ40 ratio into clinical practice. However, before implementation, it is important to assess the quality of the Aβ40 assays and validation is an essential step in implementation of novel assays in clinical routine. In this study, we evaluated the performance parameters of six Aβ40 assays currently commercially available and one in-house assay according to an international consensus protocol following the ISO 15189 guidelines (15, 16). The results will provide insight into the real-life use of this standard operating procedure (SOP) for implementation of novel immunoassays in clinical routine. Moreover, we determined the inter-laboratory variation and compared the quality and outcomes of the ELISA assays.

# MATERIALS AND METHODS

## CSF Samples

The inter-laboratory variation was tested by using 20 CSF samples centrally distributed in aliquots by one center. The samples were shipped on dry ice and upon receipt stored at -80°C in all laboratories. These samples were patient samples from an outpatient clinic, mostly patients who came for dementia diagnostic screening. Participants or their legal representatives gave informed consent. The study conforms with The Code of Ethics of the World Medical Association (Declaration of Helsinki) (17). Furthermore, for all the other parameters, samples were used that were available in each of the laboratories participating in this study. The samples were diluted according to protocols provided by the manufacturers (Table 1).

## Participants and Assay Kits

Six laboratories participated in this study and validated seven Aβ40 assays (Table 1). Every commercial assay was validated

by two or three experienced laboratories, which collaborated within the EU Joint Programme Neurodegenerative disease (JPND) BIOMARKAPD consortium. Additionally, the in-house Aβ40 assay was only tested by the developers. The following assays for quantification of Aβ40 concentrations were included: Aβ Peptide Panel I (4G8) kit [Cat#: K15199E, Meso Scale Discovery (MSD), Rockville, MD, USA<fnref n="1" />], amyloid-β (1-40) CSF ELISA [Cat#: RE59651, IBL-international (IBL), Hamburg, Germany<fnref n="2" />], ELISA Kit Human Aβ40 (Cat#: LNB0001, Invitrogen, Carlsbad, CA, USA<fnref n="3" />), Human Aβ40 Singleplex Bead Kit (Cat#: LHB3481, Novex, Invitrogen, Carlsbad, CA, USA, see text footnote 3), INNOTEST β-Amyloid (1-40) (Cat#: 81585, Fujirebio, Gent, Belgium<fnref n="4" />), β-amyloid (1-40) ELISA (Cat#: EQ 6511-9601-L, Euroimmun, Luebeck, Germany<fnref n="5" />), and an in-house Aβ40 ELISA [Ref. (18), VUmc, Amsterdam, The Netherlands]. Aβ40 assays of the same production batches from each vendor were directly distributed to the participating laboratories. The assays were performed manually according to the manufacturers' protocols, some laboratories used plate washers for the washing steps. The laboratories tested all performance parameters according to the BIOMARKAPD SOPs (15, 16). Calculations of the analytical performance parameters were done using the corresponding Data Sheets S2 and S3 in Supplementary Material (15).

## Sensitivity

For the determination of the LLOQs, 16 blank samples were measured in one plate for each assay. The calibration curves were calculated using a four parameter logistic curve fit for all assays, which gave the optimal fit.

## Precision

Intra-assay variation (repeatability) was determined by analysis of samples (n = 15) in four replicates within one plate. Some deviations were made from the original protocol: in Lab #6, n = 3 samples instead of n = 15 samples were analyzed due to technical reasons. In Lab #2, n = 14 samples for the MSD assay and, in Lab #5, n = 16 samples for the Invitrogen assay were tested. The mean coefficient of variation (%CV) was calculated by averaging the CVs of all tested samples. A %CV <20% was defined as acceptable.

Inter-assay variation (intermediate precision) was measured to determine the variation of analyses between different days. To quantify inter-assay variation, samples with low, medium, and high concentrations were selected from the samples used for the intra-assay variation [quality control (QC) low, QC medium and QC high]. These samples were measured in duplicate in four different plates at identical positions in the assay plates on four different days. In Lab #2, three different plates were tested on three different days for the Invitrogen enzyme-linked immunosorbent

<footnote n="1">¹www.mesoscale.com.</footnote>

<footnote n="2">²www.ibl-international.com.</footnote>

<footnote n="3">³www.thermofisher.com.</footnote>

<footnote n="4">⁴www.fujirebio-europe.com.</footnote>

<footnote n="5">⁵www.euroimmun.com.</footnote>
<table><caption>**TABLE 1** | Overview of Aβ40 assays tested in different laboratories.</caption><thead><tr><th>Vendor</th><th>Kit</th><th>Cat (#)</th><th>Read-out</th><th>Standards (#)</th><th>Controls (#)</th><th>Sample dilution factor</th><th>Standard range (pg/mL)</th><th>Lab #1</th><th>Lab #2</th><th>Lab #3</th><th>Lab #4</th><th>Lab #5</th><th>Lab #6</th></tr></thead><tbody><tr><td>MSD</td><td>Aβ peptide panel I (4GB)</td><td>K15199E</td><td>Electrochemiluminescence</td><td>8</td><td></td><td>2x</td><td>4–15,000</td><td>x</td><td>x</td><td></td><td>x</td><td></td><td></td></tr><tr><td>IBL</td><td>Amyloid-β (1-40)</td><td>RE59651</td><td>Optical density</td><td>6</td><td>2</td><td>20x</td><td>0–1,900</td><td>x</td><td>x</td><td>x</td><td></td><td></td><td></td></tr><tr><td>Invitrogen</td><td>CSF ELISA</td><td>LNB0001</td><td>Optical density</td><td>8</td><td></td><td>4x</td><td>0–500</td><td></td><td>x</td><td></td><td>x</td><td>x</td><td>x</td></tr><tr><td>Novex</td><td>Human Aβ40</td><td>LHB3481</td><td>Luminescence</td><td>8</td><td></td><td>2x</td><td>0–5,000</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td></tr><tr><td>Fujirebio</td><td>Singleplex bead kit</td><td>81585</td><td>Optical density</td><td>6</td><td>2</td><td>100x</td><td>5–1,000</td><td>x</td><td>x</td><td></td><td></td><td>x</td><td></td></tr><tr><td>Euroimmun</td><td>INNOTEST β-amyloid (1-40)</td><td>EQ 6511-9601-L</td><td>Optical density</td><td>6</td><td>2</td><td>21x</td><td>0–800</td><td></td><td></td><td>x</td><td>x</td><td></td><td>x</td></tr><tr><td>VUmc</td><td>β-amyloid (1-40) ELISA In-house Aβ40</td><td>(18)</td><td>Optical density</td><td>8</td><td>2</td><td>25x</td><td>0–25,000</td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

Aβ, amyloid-β; Cat (#), catalog number; CSF, cerebrospinal fluid; ELISA, enzyme-linked immunosorbent assay; IBL, IBL-international; MSD, Meso Scale Discovery; VUmc, VU university medical center.

assay (ELISA). The mean %CV was calculated for all samples. A %CV <20% was defined as acceptable.

Intra-plate variation was determined to explore the influence of different positions within a plate on the measured concentrations (stability of the plate). QC low, QC medium, and QC high samples were measured in four replicates at different positions of the plate (columns 3/4 vs. columns 11/12). The mean %CV was calculated for all samples. A %CV <20% was defined as acceptable.

## Dilutional Linearity

Three different CSF samples were used to perform the dilution linearity experiments. In Table S1 in Supplementary Material, the spiked Aβ40 calibrator concentration and the dilution factors used to serially dilute the samples per laboratory and assay are summarized. These samples were analyzed in duplicate. The dilutional linearity was calculated and expressed as follows:

$$ \% \text{Linearity} = \frac{(\text{observed C} * \text{dilution factor})}{(\text{previous observed C} * \text{previous dilution factor})} * 100 $$

$$ C = \text{concentration}(\text{pg/mL}) $$

A linearity between 80 and 120% was defined as acceptable.

To compare dilutional linearity for the kits from the various vendors and analyses by the different labs, we determined the length of dilution range in which dilutional linearity was within the acceptable range of 80–120%. This dilution range length was calculated using the following formula:

### Dilution range length

$$ = \frac{\text{highest dilution factor in which the curve is linear}}{\text{lowest dilution factor in which the curve is still linear}} $$

Despite that protocols were distributed among the participating laboratories, some of them deviated from this protocol in the execution of the experiments to assess the dilutional linearity, since these laboratories did not spike the calibrator into the CSF samples. In addition, different interpretations of the distributed protocols may have resulted in the large differences in spiked Aβ40 calibrator concentrations by the various laboratories to assess dilutional linearity (Table S1 in Supplementary Material).

## Recovery

Five different CSF samples, measured in duplicate, were diluted according to manufacturers' protocols and spiked with recombinant Aβ40 calibrator at three different levels. An overview of the spike low, medium, and high concentrations for each laboratory in every assay can be found in Table S2 in Supplementary Material. For neat samples, buffer without calibrator was added
to the diluted CSF. Spike recoveries were calculated according to the formula:

$$ \% Recovery = \frac{(C spike sample-C neat sample)}{theoretical C spike} * 100 $$

$$ C = concentration (pg/mL) $$

A recovery between 80 and 120% was defined as acceptable.

## Parallelism

Five different CSF samples, with high endogenous protein concentrations, were serially diluted. Both reciprocal relative dilution factor and OD450 absorbance signals of the samples and calibrator were log-transformed to be able to use linear regression to calculate the slopes of the sample and calibrator curves. The slope of the linear parts of the log-log transformed calibrator and sample dilution series were compared to determine the degree of parallelism by calculating the “in range%” using the following formula:

$$ in range\% = \frac{slope of sample dilution series}{slope of calibration curve} * 100 $$

A calculated in range% between 80 and 120% was defined as acceptable.

## Statistical Analysis

Bland-Altman plots were drawn to define if differences in results between assays were dependent on the concentration and to define the % deviation of each assay from the overall mean results for all clinical CSF samples. Correlation coefficients for comparison of results between vendors and between labs were performed by Spearman's p. A p-value of 0.05 was considered significant.

# RESULTS

## Sensitivity

An overview of the mean LLOQs per assay kit and for each laboratory per assay are presented in Table 2 and Table S3 in Supplementary Material, respectively. The mean LLOQs of the MSD and Fujirebio assays showed the largest variation between laboratories (CV = 99%), while the Euroimmun assay showed the smallest variation between laboratories (CV = 8%).

## Precision

The mean intra-assay CVs in all laboratories were below the predefined value of 20% for all assays (Table 2). However, it should be noted that the CVs of some individual samples were above this threshold for specific tests in single laboratories (Figure S1A in Supplementary Material).

The mean inter-assay CVs for all assays, but not for all individual laboratories, were below the predefined value of 20% (Table 2; Figure S1B in Supplementary Material).

The mean intra-plate CVs for all assays in all laboratories were below the predefined value of 20% (Table 2; Figure S1C in Supplementary Material). Only one individual sample in one laboratory showed an intra-plate CV above 20% (Figure S1C in Supplementary Material).

## Dilutional Linearity and Hook-Effect

No correct dilutional linearity data were obtained for the Fujirebio assay, Lab #2 and Lab #5. The remaining results are shown in Table 2 (overview) and Table S4 in Supplementary Material. The MSD assay showed an acceptable dilutional linearity between the predefined ranges of 80–120% over the longest dilution range, whereas this dilution range with acceptable linearity was the shortest for the Euroimmun and VUmc assays. Of note, the dilution range lengths with acceptable dilutional linearity may, in some cases, be longer than reported in Table S4 in Supplementary Material (as indicated by footnotes), since in these cases the highest or lowest dilution factor in which the curve was linear, corresponded to the highest or lowest dilution factor tested by the laboratory.

A hook effect, i.e., suppression of signal at concentrations above the upper limit of quantification, was observed for one sample tested in the Invitrogen ELISA by Lab #2 (between dilution factor 50 and 250; data not shown). Other samples tested by this and other laboratories neither showed a hook effect for the Invitrogen ELISA nor for any other assay (data not shown).

## Recovery

The results of the recovery (%R) experiments for all assays of different vendors are detailed in Table S5 in Supplementary Material. A large variation in recovery results between laboratories was observed for several assays. As a result, acceptable recovery within the predefined range of 80–120% for all three laboratories was only obtained for the IBL assay. In addition, acceptable recovery was obtained by individual laboratories for the MSD (1/3 laboratories), Invitrogen (1/3 laboratories), Novex (1/2 laboratories), Euroimmun (1/3 laboratories), and Fujirebio (1/3 laboratories) assays. Due to this large inter-laboratory variation, an overview of recovery results per vendor is not included in Table 2.

## Parallelism

Parallelism results are displayed in Table 2 (overview) and Table S6 in Supplementary Material. The mean percentage parallelism (%P) results for the assays of MSD, IBL, Novex, Euroimmun, and VUmc were within the predefined ranges of 80–120% (Table S6 in Supplementary Material), respectively: MSD (92%), IBL (100%), Novex (95%), Euroimmun (98%), and VUmc (93%). A large inter-laboratory variation for %P was observed using the Invitrogen and Fujirebio assays. For both these assays, the %P was within the predefined range for two out of three laboratories, while one laboratory per assay showed a strongly deviating result.

## Variation in Clinical Sample Concentrations between Laboratories

The intra-assay variation in Aβ40 concentrations was below 10% for all CSF samples (n = 20) in all laboratories for all different
<table><caption>**TABLE 2** | Overview of analytical performance parameters.</caption><thead><tr><th rowspan="3">Vendor</th><th rowspan="3">Kit</th><th colspan="2">Sensitivity</th><th colspan="2">Precision</th><th rowspan="2">Dilutional linearity range (pg/ml)</th><th rowspan="2">Parallelism (%P)</th><th rowspan="2">Intra-assay variation in clinical samples n = 20/ laboratory (%CV)</th><th rowspan="2">Correlation coefficient (ρ) between laboratories</th></tr><tr><th colspan="2">LLOQ (pg/mL)</th><th colspan="2">Intra-assay (%CV)</th></tr><tr><th>Mean (SD)</th><th>%CV</th><th>Mean (SD)</th><th>Mean (SD)</th><th>Range (length)</th><th>Mean (SD)</th><th>Mean (SD)</th><th>Mean (SD)</th></tr></thead><tbody><tr><td>MSD</td><td>Aβ peptide panel I (4G8)</td><td>65 (64)</td><td>99</td><td>7.1 (5.1)</td><td>13 (2.7)</td><td>6.1 (0.8)</td><td>7–2,825 (348)</td><td>92 (18)</td><td>5.8 (3.6)</td><td>0.86 (0.09)</td></tr><tr><td>IBL</td><td>Amyloid-β (1-40) CSF ELISA</td><td>41 (32)</td><td>79</td><td>3.0 (0.5)</td><td>10 (3.8)</td><td>3.2 (1.9)</td><td>15–480 (32)</td><td>100 (5.5)</td><td>2.4 (1.3)</td><td>0.93 (0.03)</td></tr><tr><td>Invitrogen</td><td>ELISA kit human Aβ40</td><td>28 (12)</td><td>40</td><td>7.3 (5.3)</td><td>18 (14)</td><td>7.0 (1.7)</td><td>2,500–97,656 (39)</td><td>189 (134)</td><td>3.7 (1.1)</td><td>0.91 (0.04)</td></tr><tr><td>Novex</td><td>Human Aβ40 singleplex bead kit</td><td>64 (36)</td><td>56</td><td>14 (1.9)</td><td>17 (8.0)</td><td>12 (5.4)</td><td>2,500–97,656 (39)</td><td>95 (12)</td><td>8.4 (1.1)</td><td>0.82 (–)</td></tr><tr><td>Fujirebio</td><td>INNOTEST β-amyloid (1-40)</td><td>1.8 (1.7)</td><td>99</td><td>5.1 (3.5)</td><td>13 (2.8)</td><td>4.0 (1.9)</td><td></td><td>73 (42)</td><td>3.6 (1.3)</td><td>0.80 (0.11)</td></tr><tr><td>Euroimmun</td><td>β-amyloid (1-40) ELISA</td><td>23 (1.9)</td><td>8</td><td>4.1 (2.5)</td><td>14 (6.0)</td><td>5.2 (3.6)</td><td>2,500–15,625 (6)</td><td>98 (6.7)</td><td>2.9 (1.8)</td><td>0.89 (0.03)</td></tr><tr><td>VUMc</td><td>In-house Aβ40</td><td>1,089 (–)</td><td>–</td><td>1.9 (–)</td><td>12 (–)</td><td>1.7 (–)</td><td>25–125 (5)</td><td>93 (–)</td><td>1.1 (–)</td><td>–</td></tr></tbody></table>

Aβ, amyloid-β; CSF, cerebrospinal fluid; ELISA, enzyme-linked immunosorbent assay; IBL, IBL-international; LLOQ, lower limit of quantification; MSD, Meso Scale Discovery; VUMc, VU university medical center; %CV, percentage coefficient of variation; %P, percentage parallelism.

vendors, well below the predefined CV of 20% (Table 2). Per vendor, a strong correlation (ρ > 0.8) between sample Aβ40 concentrations assayed by the different laboratories was observed, except for the results by Lab #1 and Lab #2 that showed a lower correlation coefficient for the MSD (ρ = 0.77) and Fujirebio (ρ = 0.67) assays (Figure 1). The best mean correlation coefficient between laboratories was measured for Aβ40 concentrations using the IBL kit (ρ = 0.93) (Table 2). In terms of absolute concentrations, results for the Novex assay had the lowest similarity in mean values of the clinical samples between laboratories (Figure 1).

The mean Aβ40 concentrations of all CSF samples varied up from ~2,000 pg/mL for the Novex assay at the one extreme to ~9,500 pg/mL for the IBL assay at the other end. The %difference in Aβ40 concentrations per assay compared to the mean concentration of all assays per sample showed that the highest deviation was obtained by the IBL and Novex assays (mean deviation of 65 and -64%, respectively) (Bland-Altman plots in Figure 2). The lowest variation (within 20% deviation from the mean Aβ40 concentration of all assays per sample) was observed for the MSD assay.

## DISCUSSION

The CSF Aβ42 levels reflect the extent of amyloid-β accumulation in the form of plaques in the brain. However, it has been shown in previous studies that the concentration of Aβ42 not only depends on the presence or absence of plaques but also may on the total turnover of Aβ peptides in the brain, resulting in different total Aβ concentrations in the CSF (11, 19–21). Besides, recent studies indicated that absorption of Aβ42 to collection and storage tubes

can be overcome by calculation of the ratio of Aβ42/Aβ40, since both peptides will be absorbed to a similar degree (22, 23). The ratio of Aβ42/Aβ40 is helpful to correct levels of Aβ42 for the total amyloid production. In this study, we validated Aβ40 assays from six different vendors and one in-house developed assay for future use in clinical practice.

The assignment of the different assays to the laboratories was performed first based on the availability of the MSD and Luminex systems in the laboratories and second as random as possible, to overcome that two laboratories measured the same set of assays as a source of center bias. We have chosen to have the validation performed by up to three centers per assays, which we expect to provide sufficient insight into real-life performance of the assays and deemed to be cost-effective. We reasoned that testing all assays in a larger number of laboratories would not lead to dramatically different results for the assays than presented in this study.

The performance parameters of all assays were within the predefined ranges, except for the recovery results, for which no definite conclusions could be drawn due to large inter-laboratory variation, and the parallelism results of Invitrogen and Fujirebio assays, for each of which one out of three laboratories obtained results out of the predefined range. The variation between laboratories in the Aβ40 concentrations in the clinical samples was small in the duplicate measurements for each assay in every laboratory (%CV <10%). Furthermore, high mean correlation coefficients (ρ > 0.8) per vendor between sample concentrations were found for all assays. In terms of variation in absolute values between vendors, the least deviation was found for the MSD assay compared to the mean Aβ40 concentration of all assays per sample.
<figure><img src="image_4.png" /><figcaption>B IBL Amyloid-beta (1-40) CSF ELISA</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>C
Invitrogen ELISA Kit Human A β</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>D
Novex Human Aβ40 Singleplex Bead Kit</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>E
Fujirebio INNOTEST β-Amyloid (1-40)</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>F
Euroimmun Beta-Amyloid (1-40) ELISA</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>G
Vumc in-house Aβ40</figcaption></figure>

<figure><img src="image_10.png" /></figure>

**FIGURE 1** | Correlation of concentrations in clinical cerebrospinal fluid (CSF) samples between laboratories. The mean concentrations of *n* = 20 samples tested in each laboratory for Meso Scale Discovery (MSD) Aβ peptide panel I (4G8) (A), IBL amyloid-β (1-40) CSF ELISA (B), Invitrogen ELISA kit human Aβ (C), Novex Human Aβ40 Singleplex Bead Kit (D), Fujirebio INNOTEST β-Amyloid (1-40) (E), Euroimmun β-Amyloid (1-40) ELISA (F), Vumc in-house Aβ40 (G) was plotted. The best correlation between laboratories was measured for concentrations using the IBL kit (mean ρ = 0.93). On the other hand, the lowest correlation was found in concentrations obtained by the Fujirebio assay (mean ρ = 0.80). ρ = Pearson's correlation coefficient.

The LLOQs were slightly higher compared to the kit specifications provided by most vendors, but well below the ranges found in clinical samples. The IBL and Euroimmun assays are the

exceptions, because the LLOQ was lower in our results compared to the kit specifications (data not shown). It should be noted, however, that the LLOQs are calculated differently by the vendors
<figure><img src="image_11.png" /><figcaption>**FIGURE 2** | Variation in Aβ40 concentrations in clinical cerebrospinal fluid (CSF) samples between assays. Bland–Altman plot: the %difference (y-axis) in mean Aβ40 concentrations in each clinical CSF sample per assay (n = 20 samples, 7 assays) is compared to the mean Aβ40 concentration of all assays per sample. Each sample per assay is plotted on the x-axis at its mean Aβ40 concentration of all assays.</figcaption></figure>

(mean + 2 to 3\*SD), while we did use a more strict method of calculation (mean + 10\*SD).

Although the mean precision data per assay were well below the predefined CV of 20% for all assays, if we would consider a more strict maximum of a mean CV of all laboratories below 15%, the results of Invitrogen and Novex assays are out of range.

The dilutional linearity description in the SOP gave room for many different interpretation options and should be revised. Some laboratories pre-diluted the CSF samples and, therefore, did not test the lower dilutions in which a hook effect still could be present. In the Invitrogen assay, a high variation was found, probably because Lab #6 tested lower steps of dilution (a factor 2.5) than the other two labs did (factor 4 and 5). Also, the amount of spiked antigen differed per laboratory; some laboratories did not spike antigen and, therefore, actually performed a parallelism experiment instead of the dilutional linearity experiment and had to be excluded from analysis for this parameter.

The recovery was out-of-range for the majority of the experiments. This could be due to the large variability between the laboratories in terms of preparation of the samples for the recovery experiments, e.g., the variation in concentrations chosen for spiking. In some laboratories, too high concentrations of Aβ40 were spiked, which resulted in an overflow or too low concentrations were spiked, resulting in very low precision of the result and thus low recovery. We have adapted the wording in the SOP to increase the compliance in execution (Supplemental Data for the improved version), e.g., by expanding on the choice of

spikes and adding notes to standardize the spike concentrations in multicenter studies. Moreover, we added a proposal to include analysis of the actual spiked concentrations in reagent diluent for calculation instead of using a theoretical spiked value, in order to track possible dilution errors.

The parallelism results for the majority of the assays were within the predefined ranges, except the results of the Invitrogen and Fujirebio assays, because one laboratory was out of range. No explanation could be given for these two deviating results. However, with regard to this occurrence, to get more insight, it may be helpful to include more laboratories to test each validation parameter in future validation studies.

The IBL, Invitrogen, and Euroimmun Aβ40 assays seem to perform slightly better in terms of inter-laboratory comparisons, as shown by consistent high correlation coefficients for the clinical samples between laboratories. The mean Aβ40 concentrations of all CSF samples varied between the vendors. This is a hurdle in implementation of the ratio of Aβ42/Aβ40 in clinical practice to improve diagnostic accuracy (9–13). As long as this variation between assays is not overcome by the use of certified reference methods and materials, which are being developed (24), cut-offs will depend on the combination of assays used at every site. Usage of automated platforms instead of manual performance of biomarker measurements could mitigate inter-laboratory variations.

In conclusion, all Aβ40 assays perform generally well and are promising for implementing in clinical practice. We showed some deviating results in specific parameters, but these are more
likely the result of inter-laboratory variation and the lack of refer-
ence materials. Furthermore, of note is that the absolute Aβ40
concentrations in clinical CSF varied between assays. Therefore,
the assays could be improved by the availability of certified refer-
ence materials to calibrate kits as well as usage of automated plat-
forms. The upcoming availability of Aβ40 assays in a European
Conformity (CE)-certified format is an important first step for
their routine implementation in Europe.

## ETHICS STATEMENT

Participants or their legal representatives gave informed con-
sent. The study conforms with The Code of Ethics of the World
Medical Association (Declaration of Helsinki) (World_Medical_
Association 1964). The biobank containing the coded patient
samples was approved by the Local Ethical committee of the VU
University Medical Center Amsterdam.

## AUTHOR CONTRIBUTIONS

LWD: performed data analysis and drafted the manuscript. LK: supervision of data collection and study design, critical review of the manuscript. MK-S: study design and coordination, data collection, and analysis. HK, SE, AP-L, SL, MV, and EV: supervision of data collection and study design, critical review of the manuscript. AV: data collection, critical review of the manuscript. HS, HT, and AF: data collection and analysis, critical review of the manuscript. EV: supervision of data collection and study design, critical review of the manuscript. CT: study design and coordination, data analysis, critical review of the manuscript.

## ACKNOWLEDGMENTS

We would like to thank Kees van Uffelen from VUmc for his
technical assistance. Additionally, all used kits were provided
for free by the vendors involved in this study. This work was
supported by the BIOMARKAPD research project within the
EU JPND. The project is supported through national funding
organizations under the aegis of JPND (http://www.jpnd.eu).
For the Netherlands, the Netherlands Organisation for Health

Research and Development (ZonMw); for France, the French
National Research Agency (ANR); for Switzerland, the Swiss
National Science Foundation (SNSF). BIODEM UAntwerp
was in part supported by the University of Antwerp Research
Fund and the Alzheimer Research Foundation (SAOFRA); the
Agency for Innovation by Science and Technology (IWT) (www.
iwt.be); the Research Foundation Flanders (FWO) (www.fwo.
be); the Belgian Science Policy Office Interuniversity Attraction
Poles (IAP) program (BELSPO) (www.belspo.be); the Flemish
Government initiated Methusalem excellence grant (EWI)
(www.ewi-vlaanderen.be); the Flanders Impulse Program on
Networks for Dementia Research (VIND). Part of this project
is covered by a bilateral research agreement between University
of Antwerp and ADx NeuroSciences. The Radboudumc was
supported by the CAVIA project (ZonMw-nr. 733050202),
part of "Memorabel", the research and innovation program
for dementia, as part of the Dutch national "Deltaplan for
Dementia" (zonmwl/dementiaresearch). The CAVIA project
(www.caviaproject.nl) is a consortium of Radboudumc, LUMC,
Erasmus MC, VUmc, ADX NeuroSciences, Philips Healthcare,
Stony Brook University and Massachusetts General Hospital. EV
is co-founder of ADx NeuroSciences. SE was/is consultant for
and/or received research funding from Janssen Pharmaceutica,
ADx NeuroSciences, Innogenetics/Fujirebio Europe, Lundbeck,
Pfizer, Novartis, UCB, Roche diagnostics, Nutricia/Danone, and
Eli Lilly. MV serves on an advisory board for Roche Diagnostics.
CT served on the advisory board of Fujirebio and Roche,
received research consumables from Euroimmun, IBL, Fujirebio,
Invitrogen, and Meso Scale Discovery, and performed contract
research for IBL, Shire, Boehringer, Roche, and Probiodrug;
and received grants from the European Commission, the Dutch
Research Council (ZonMw), Association of Frontotemporal
Dementia, ISAO, and the Alzheimer's Drug Discovery
Foundation. The other co-authors do not have any conflicts of
interest to report.

## SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at
http://journal.frontiersin.org/article/10.3389/fneur.2017.00310/
full#supplementary-material.

## REFERENCES

1. Jack CR Jr, Knopman DS, Jagust WJ, Shaw LM, Aisen PS, Weiner MW, et al. Hypothetical model of dynamic biomarkers of the Alzheimer's pathological cascade. *Lancet Neurol* (2010) 9:119-28. doi:10.1016/
S1474-4422(09)70299-6

2. Albert MS, DeKosky ST, Dickson D, Dubois B, Feldman HH, Fox NC, et al. The diagnosis of mild cognitive impairment due to Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease. *Alzheimer Dement* (2011) 7:270-9. doi:10.1016/j.jalz.2011.03.008

3. McKhann GM, Knopman DS, Chertkow H, Hyman BT, Jack CR Jr, Kawas CH, et al. The diagnosis of dementia due to Alzheimer's disease: recommendations from the National Institute on Aging-Alzheimer's Association workgroups on diagnostic guidelines for Alzheimer's disease. *Alzheimer Dement* (2011) 7:263-9. doi:10.1016/j.jalz.2011.03.005

4. Dubois D, Feldman HH, Jacova C, Hampel H, Molinuevo JL, Blennow K, et al. Advancing research diagnostic criteria for Alzheimer's disease: the IWG-2 criteria. *Lancet Neurol* (2014) 13:614-29. doi:10.1016/S1474-4422(14)70090-0

5. Strozyk D, Blennow K, White LR, Launer LJ. CSF Abeta 42 levels correlate with amyloid-neuropathology in a population-based autopsy study. *Neurology* (2003) 60:652-6. doi:10.1212/01.WNL.0000046581.81650.D0

6. Tapiola T, Overmyer M, Lehtovirta M, Helisalmi S, Ramberg J, Alafuzoff I, et al. The level of cerebrospinal fluid tau correlates with neurofibrillary tangles in Alzheimer's disease. *Neuroreport* (1997) 8:3961-3. doi:10.1097/00001756-199712220-00022

7. Janelidze S, Zetterberg H, Mattsson N, Palmqvist S, Vanderstichele H, Lindberg O, et al. CSF Abeta42/Abeta40 and Abeta42/Abeta38 ratios: better diagnostic markers of Alzheimer disease. *Ann Clin Transl Neurol* (2016) 3:154-65. doi:10.1002/acn3.274

8. Renard D, Wacongne A, Ayrignac X, Charif M, Fourcade G, Azakri S, et al. Cerebrospinal fluid Alzheimer's disease biomarkers in cerebral
amyloid angiopathy-related inflammation. *J Alzheimers Dis* (2015) 50:759–64.
doi:10.3233/JAD-150621

9. Slaets S, Le Bastard N, Martin JJ, Sleegers K, Van Broeckhoven C, De Deyn PP, et al. Cerebrospinal fluid Abeta1-40 improves differential dementia diagnosis in patients with intermediate P-tau181P levels. *J Alzheimers Dis* (2013) 36:759–67. doi:10.3233/JAD-130107

10. Sauvee M, DidierLaurent G, Latarche C, Escanye MC, Olivier JL, Malaplate-Armand C. Additional use of Abeta(4)(2)/Abeta(4)(0) ratio with cerebrospinal fluid biomarkers P-tau and Abeta(4)(2) increases the level of evidence of Alzheimer's disease pathophysiological process in routine practice. *J Alzheimers Dis* (2014) 41:377–86. doi:10.3233/JAD-131838

11. Lewczuk P, Lelental N, Spitzer P, Maler JM, Kornhuber J. Amyloid-β 42/40 cerebrospinal fluid concentration ratio in the diagnostics of Alzheimer's disease: validation of two novel assays. *J Alzheimer Dis* (2015) 43:183–91. doi:10.3233/JAD-140771

12. Slemmon JR, Shapiro A, Mercken M, Streffer J, Romano G, Andreasen N, et al. Impact of cerebrospinal fluid matrix on the detection of Alzheimer's disease with Abeta42 and influence of disease on the total-Abeta42/Abeta40 ratio. *J Neurochem* (2015) 135:1049–58. doi:10.1111/jnc.13297

13. Dorey A, Perret-Liaudet A, Tholance Y, Fourier A, Quadrio I. Cerebrospinal fluid Abeta40 improves the interpretation of Abeta42 concentration for diagnosing Alzheimer's disease. *Front Neurol* (2015) 6:247. doi:10.3389/fneur.2015.00247

14. Leuzy A, Chiotis K, Hasselbalch SG, Rinne JO, de Mendonça A, Otto M, et al. Pittsburgh compound B imaging and cerebrospinal fluid amyloid-beta in a multicentre European memory clinic study. *Brain* (2016) 139:2540–53. doi:10.1093/brain/aww160

15. Andreasson U, Perret-Liaudet A, van Waalwijk van Doorn LJ, Blennow K, Chiasserini D, Engelborghs S, et al. A practical guide to immunoassay method validation. *Front Neurol* (2015) 6:179. doi:10.3389/fneur.2015.00179

16. van Waalwijk van Doorn LJ, Koel-Simmelink MJ, Haußmann U, Klafki H, Struyfs H, Linning P, et al. Validation of soluble APP assays as diagnostic CSF biomarkers for neurodegenerative diseases. *J Neurochem* (2016) 137:112–21. doi:10.1111/jnc.13527

17. World_Medical_Association. Human experimentation: code of ethics of W.M.A. *Br Med J* (1964) 2:177–177. doi:10.1136/bmj.2.5402.177

18. Verwey NA, Veerhuis R, Twaalfhoven HA, Wouters D, Hoozemans JJ, Bollen YJ, et al. Quantification of amyloid-beta 40 in cerebrospinal fluid. *J Immunol Methods* (2009) 348:57–66. doi:10.1016/j.jim.2009.06.011

19. Wiltfang J, Esselmann H, Bibl M, Hüll M, Hampel H, Kessler H, et al. Amyloid β peptide ratio 42/40 but not Aβ42 correlates with phospho-Tau in

patients with low- and high-CSF Aβ40 load. *J Neurochem* (2007) 101:1053–9.
doi:10.1111/j.1471-4159.2006.04404.x

20. Hansson O, Zetterberg H, Buchhave P, Andreasson U, Londos E, Minthon L, et al. Prediction of Alzheimer's disease using the CSF Aβ42/Aβ40 ratio in patients with mild cognitive impairment. *Dement Geriatr Cogn Disord* (2007) 23:316–20. doi:10.1159/000100926

21. Spies PE, Verbeek MM, van Groen T, Claassen JA. Reviewing reasons for the decreased CSF Abeta42 concentration in Alzheimer disease. *Front Biosci (Landmark Ed)* (2012) 17:2024–34. doi:10.2741/4035

22. Vanderstichele HM, Janelidze S, Demeyer L, Coart E, Stoops E, Herbst V, et al. Optimized standard operating procedures for the analysis of cerebrospinal fluid Abeta42 and the ratios of abeta isoforms using low protein binding tubes. *J Alzheimers Dis* (2016) 53:1121–32. doi:10.3233/JAD-160286

23. Willemse E, van Uffelen K, Brix B, Engelborghs S, Vanderstichele H, Teunissen CE. How to handle adsorption of cerebrospinal fluid amyloid-β (1-42) in laboratory practice? Identifying problematic handlings and resolving the issue by use of the Aβ42/Aβ40 ratio. *Alzheimers Dement* (2017). doi:10.1016/j.jalz.2017.01.010

24. Pannee J, Portelius E, Minthon L, Gobom J, Andreasson U, Zetterberg H, et al. Reference measurement procedure for CSF amyloid beta (Aβ)1-42 and the CSF Aβ1-42/Aβ1-40 ratio – a cross-validation study against amyloid PET. *J Neurochem* (2016) 139:651–8. doi:10.1111/jnc.13838

**Conflict of Interest Statement:** SE was/is consultant for and/or received research funding from Janssen Pharmaceutica, ADx NeuroSciences, Innogenetics/Fujirebio Europe, Lundbeck, Pfizer, Novartis, UCB, Roche diagnostics, Nutricia/Danone, and Eli Lilly. MV serves on an advisory board for Roche Diagnostics. CT served on the advisory board of Fujirebio and Roche, received research consumables from Euroimmun, IBL, Fujirebio, Invitrogen, and Meso Scale Discovery, and performed contract research for IBL, Shire, Boehringer, Roche, and Probiodrug. All other authors declare no conflict of interest.

Copyright © 2017 van Waalwijk van Doorn, Kulic, Koel-Simmelink, Kuiperij, Versleijen, Struyfs, Twaalfhoven, Fourier, Engelborghs, Perret-Liaudet, Lehmann, Verbeek, Vanmechelen and Teunissen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.