# Classification Performance of the Ages and Stages Questionnaire: Influence of Maternal Education Level

Marguerite Lockhart, Robin Chaux, Mathilde Chevin, Magali Celle, Tiphaine Raia-Barjat, Hugues Patural, Stéphane Chabrier, Antoine Giraud

► To cite this version:

Marguerite Lockhart, Robin Chaux, Mathilde Chevin, Magali Celle, Tiphaine Raia-Barjat, et al. Classification Performance of the Ages and Stages Questionnaire: Influence of Maternal Education Level. Children, 2023, 10 (3), pp.449. 10.3390/children10030449. hal-04858471

HAL Id: hal-04858471

https://hal.science/hal-04858471v1

Submitted on 29 Dec 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Article

# Classification Performance of the Ages and Stages Questionnaire: Influence of Maternal Education Level

Marguerite Lockhart<sup>1,2</sup>, Robin Chaux<sup>3</sup>, Mathilde Chevin<sup>4 ID</sup>, Magali Celle<sup>5</sup>, Tiphaine Raia-Barjat<sup>2 ID</sup>, Hugues Patural<sup>1,2 ID</sup>, Stéphane Chabrier<sup>2</sup> and Antoine Giraud<sup>1,2,* ID</sup>

<figure><img src="image_2.png" /></figure>

check for
updates

**Citation:** Lockhart, M.; Chaux, R.; Chevin, M.; Celle, M.; Raia-Barjat, T.; Patural, H.; Chabrier, S.; Giraud, A. Classification Performance of the Ages and Stages Questionnaire: Influence of Maternal Education Level. *Children* **2023**, *10*, 449. https://doi.org/10.3390/children10030449

Academic Editor: Mark Dzietko

Received: 5 January 2023

Revised: 18 February 2023

Accepted: 22 February 2023

Published: 25 February 2023

<figure><img src="image_3.png" /></figure>

**Copyright:** © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

**Abstract:** (1) Background: The Ages and Stages Questionnaire—Third Edition (ASQ-3) is a parental screening questionnaire increasingly being used to evaluate the development of preterm children. We aimed to assess the classification performance of the ASQ-3 in preterm infant follow-up. (2) Methods: In this cross-sectional study, we included 185 children from the SEVE longitudinal cohort born <33 weeks of gestational age between November 2011 and January 2018, who had both an ASQ-3 score at 24 months of corrected age (CA) and a revised Brunet-Lézine (RBL) scale score at 30 months of CA. The ASQ-3 overall score and sub-scores were compared to the RBL developmental quotient (DQ) scores domain by domain. The diagnostic performance of the ASQ-3 was evaluated with the RBL as the reference method by calculating sensitivity, specificity, and positive and negative likelihood ratios. A multivariate analysis assessed the association between low maternal education level and incorrect evaluation with the ASQ-3. (3) Results: The ASQ-3 overall score had a specificity of 91%, a sensitivity of 34%, a positive likelihood ratio of 3.82, and a negative likelihood ratio of 0.72. Low maternal education level was a major risk factor for incorrectly evaluating children with the ASQ-3 (odds ratio 4.16, 95% confidence interval 1.47–12.03; *p* < 0.01). (4) Conclusions: Regarding the low sensitivity and the impact of a low maternal education level on the classification performance of the ASQ-3, this parental questionnaire should not be used alone to follow the development of preterm children

**Keywords:** prematurity; neurodevelopment; assessment; maternal instruction

## 1. Introduction

Preterm children are at high risk of neurodevelopmental impairment [1,2]. According to the EPIPAGE-2 cohort study, the rates of severe and moderate neurodevelopmental impairments at five years of age are estimated from 28% to 12%, for children born between 24 and 26 weeks of gestational age and between 32 and 34 weeks of gestational age, respectively [1]. The early evaluation of the developmental trajectory of each preterm child is essential to identify the children to whom to propose an early developmental intervention, to promote their future activity and participation [3,4]. A meta-analysis of eight studies, including 1436 children born before 37 weeks of gestational age evaluated between three and five years of age, had shown that such early developmental intervention improved intelligence quotient at preschool age (standardised mean difference 0.43 standard deviations, 95% confidence interval (CI) from 0.32 to 0.54) [4].

The two hetero-assessment scales used by professionals for assessing neurodevelopment during the first years of life and validated in France are the revised BrunetLézine (RBL)

<footnote n="1">¹ Neonatal Intensive Care Unit, Department of Paediatrics, Centre Hospitalier Universitaire de Saint-Etienne, 42270 Saint-Etienne, France</footnote>

<footnote n="2">² INSERM, U1059 Sainbiose, Université Jean Monnet, 42100 Saint-Etienne, France</footnote>

<footnote n="3">³ Department of Clinical Research and Pharmacology, Centre Hospitalier Universitaire de Saint-Etienne, 42270 Saint-Etienne, France</footnote>

<footnote n="4">⁴ Department of Paediatrics, McGill University, Montreal, QC H3A 0G4, Canada</footnote>

<footnote n="5">⁵ Coordination du Réseau SEVE, Centre Hospitalier Universitaire de Saint-Etienne, 42270 Saint-Etienne, France</footnote>

<footnote>* Correspondence: antoine.giraud@univ-st-etienne.fr</footnote>
scale and the Bayley Scales of Infant and Toddler Development—Third Edition (Bayley-III) [5,6]. The RBL is the predominant validated hetero-assessment scale used to evaluate the neurodevelopment of children at preschool age in France. It calculates a global developmental quotient (DQ) by computing four DQs for sociability, gross motor function, visuospatial coordination, and language [5]. The Bayley-III evaluates the same neurodevelopmental domains, has the same age groups, and uses a similar scoring technique as the RBL [6].

The Ages and Stages Questionnaire—Third Edition (ASQ-3) is a validated parental screening questionnaire increasingly being used to evaluate the development of preterm children [7,8]. It evaluates the same neurodevelopmental domains as RBL, in addition to the problem-solving domain, and consists of six questions for each domain of child development [8]. A single study compared the ASQ-3 to the RBL scale in this population and found a good correlation between the overall ASQ-3 score and the global DQ of the RBL in children with neurodevelopmental impairment at 24 months of corrected age (CA) [9].

The aim of our study was to assess the classification performance of the ASQ-3 in preterm children and the impact of maternal education level on the classification performances of the ASQ-3, using the RBL scale as a reference. As the ASQ-3 is a parental questionnaire, we hypothesised that the ASQ-3 has low sensitivity and that low maternal education level was associated with incorrect classification with the ASQ-3.

**2. Materials and Methods**

This cross-sectional study was performed and reported according to the STROBE guidelines for observational studies [10].

2.1. *Population Sample*

The *Suivi des Enfants Vulnérables sur le Réseau Elena* (SEVE) Network is a regional cohort ensuring a free-of-charge and standardised follow-up of vulnerable children, as described [11]. As part of the follow-up, parents were encouraged to complete the ASQ-3 at 24 months of corrected age (CA) and a systematic encounter with an experienced neuropsychologist was offered at 30 months of CA to perform an RBL assessment. All children born before 33 weeks of gestational age (GA) between November 2011 and January 2018 with an ASQ-3 evaluation and an RBL evaluation were included in our study. There were no exclusion criteria.

Data were longitudinally collected by using standardised questionnaires throughout the predefined follow-up protocol and included maternal age, maternal education level, antenatal steroids administration, multiple pregnancy, vaginal delivery versus caesarean-section, GA at birth, birth measurements, Apgar score at 1 min and 5 min, the use of surfactant, cerebral haemorrhage, periventricular leukomalacia, necrotising enterocolitis, ventilatory support duration, hospitalisation duration, and feeding at discharge. Information on the maternal education level was recorded during the neuropsychologist consultation. A low maternal education level was defined as the highest attained education below secondary education (i.e., baccalaureate degree or less in the French school system). A high education level was defined by any high school degree. Birth measurement z-scores were calculated by GA and sex using the INTERGROWTH-21st International Newborn Size at Birth Standards application v1.3.5 (University of Oxford, UK).

2.2. *Neurodevelopmental Assessment*

Neurodevelopmental evaluation at 30 months of CA was performed by an experienced neuropsychologist blinded to the ASQ-3 results, using the RBL scale. The RBL is the predominant scale utilised to evaluate the neurodevelopment of children from 2 to 30 months of CA in France [5,12]. It was validated on a sample of 1032 children born at term [5]. A global DQ was established by computing four DQs for sociability, gross motor function, visuospatial coordination, and language. Each DQ is normalised to the CA and has a mean of 100 and a standard deviation (SD) of 15 [5]. A DQ < 85 is defined as abnormal [5].
Parents completed the French ASQ-3 adapted for children at 24 months of CA. The ASQ-3 consists of six questions for five domains of child development: communication, gross motor, fine motor, problem-solving, and personal-social [8]. A score <−2 SDs is defined as abnormal [8]. The overall ASQ-3 score is considered abnormal if at least one domain is abnormal [8].

## 2.3. Ethics

Written informed parental consent was mandatory to be included in the SEVE Network. The French National Technologies and Civil Liberties Commission (CNIL) approved data collection (authorisation no. 1530737). This study was authorised by the Ethical Committee of the Saint-Étienne University Hospital (April 2020, IRBN392020/CHUSTE).

## 2.4. Statistical Analysis

Categorical variables are expressed as absolute numbers (percentage). Continuous variables are expressed as mean (SD) or median (interquartile range) according to their distribution.

The ASQ-3 was compared to the RBL domain by domain: communication, gross motor, fine motor, personal-social domains, and overall ASQ scores were compared to language, gross motor function, visuospatial coordination, sociability, and global DQs, respectively. The classification performance of the ASQ-3 was evaluated by generating contingency tables for each score. The RBL was defined as the reference method. DQs < 85 were considered as positive, i.e., abnormal development [5]. ASQ-3 domain scores and overall ASQ score with one or more domains <−2 SDs were considered as positive, i.e., abnormal development [8]. The children considered correctly classified with the ASQ-3 were true positives and true negatives. The children considered incorrectly classified were false positives and false negatives.

Using that classification contingency table, sensitivity (true positive rate) and specificity (true negative rate) with their 95% CIs were computed. To summarize the information contained in both sensitivity and specificity, positive and negative likelihood ratios were also computed for each score using Formulas (1) and (2) [13].

$$ \text{Positive likelihood ratio} = \frac{\text{sensitivity}}{1 - \text{specificity}} = \frac{\text{Pr}(T + |D+)}{\text{Pr}(T + |D-)} \quad (1) $$

$$ \text{Negative likelihood ratio} = \frac{1 - \text{sensitivity}}{\text{specificity}} = \frac{\text{Pr}(T - |D+)}{\text{Pr}(T - |D-)} \quad (2) $$

"T+" and "T−" represent the positivity or negativity of the test, respectively. "D+" and "D−" represent the presence or absence of abnormal development, respectively. A positive likelihood ratio > 10 would be considered satisfying, as well as a negative likelihood ratio < 0.1, whereas positive or negative likelihood ratios = 1 would mean that the test does not discriminate against individuals with or without abnormal development.

Diagnostic odds ratios with their 95% CI were also calculated to describe the odds of having a positive ASQ-3 test in children with abnormal development relative to the odds of a positive ASQ-3 test in children with normal development. The calculation of the diagnostic odds ratio and its relations to positive and negative likelihood ratios is described in Formula (3) [13].

$$ \text{Diagnostic odds ratio} = \frac{\text{positive likelihood ratio}}{\text{negative likelihood ratio}} \quad (3) $$

Then, the performance of the ASQ-3 diagnostic was assessed using the area under curve (AUC) of receiver operating characteristic (ROC) curves and their 95% CIs, by means of 10,000 iterations of bootstrap. A secondary analysis stratified on maternal education level was also performed using the same methodology.

Univariate and multivariate logistic regression models were used to estimate the independent association between the classification performance of ASQ-3 and low maternal
education level. The dependent variable ASQ-3 was dichotomized as previously and treated as a binary variable, with an overall ASQ-3 score < -2 SDs considered positive and an overall ASQ-3 score ≥ -2 SDs considered negative. The multivariate model was built a priori, using known potential confounders according to literature data and experts' opinions. Sex, GA, hospitalization duration, and ventilatory support duration were included in the multivariate model as adjustment covariates as potential confounders. Quantitative covariates were coded as continuous covariates.

The results are reported as odds ratios (ORs) and their 95% CIs. All tests were two-sided, with a *p*-value <0.05 considered as statistically significant. Missing data represented 2.5% of the data and were not considered in the analysis. Analyses were performed using R software version 4.0.3 (R Foundation for Statistical Computing, Vienna, Austria).

### 3. Results

One hundred and eighty-five children had an RBL and ASQ-3 evaluation and were included in the current study. The features of the population sample are described in Table 1. The mean CA at RBL evaluation was 29.0 months with an SD of 2.7.

<table><caption>**Table 1.** Features of included children.</caption><thead><tr><th>Features</th><th>Population (n = 185)</th></tr></thead><tbody><tr><td>Maternal age, years, mean (SD)</td><td>30.5 (5.1)</td></tr><tr><td>Multiple pregnancy, n (%)<sup>a</sup></td><td>43 (23.4)</td></tr><tr><td>Antenatal steroids administration</td><td></td></tr><tr><td>Incomplete, n (%)</td><td>27 (15.5)</td></tr><tr><td>Complete, n (%)</td><td>125 (71.8)</td></tr><tr><td>Vaginal delivery, n (%)</td><td>168 (70.6)</td></tr><tr><td>GA, weeks, mean (SD)</td><td>30.6 (2.9)</td></tr><tr><td>Birth weight, g, mean (SD)</td><td>1440 (550)</td></tr><tr><td>Birth weight z score, mean (SD)</td><td>-0.24 (0.99)</td></tr><tr><td>Birth height, cm, mean (SD)</td><td>39.0 (4.7)</td></tr><tr><td>Birth height z score, mean (SD)</td><td>-0.49 (1.25)</td></tr><tr><td>Birth HC, cm, mean (SD)</td><td>27.7 (3.1)</td></tr><tr><td>Birth HC z score, mean (SD)</td><td>-0.12 (1.21)</td></tr><tr><td>Male sex, n (%)</td><td>104 (56.2)</td></tr><tr><td>Apgar score at 1 min &lt; 7, n (%)</td><td>82 (44.8)</td></tr><tr><td>Apgar score at 5 min &lt; 7, n (%)</td><td>21 (11.7)</td></tr><tr><td>Surfactant administration</td><td></td></tr><tr><td>1 dose, n (%)</td><td>69 (43.9)</td></tr><tr><td>2 doses, n (%)</td><td>13 (8.3)</td></tr><tr><td>Hospitalisation duration, days, mean (SD)</td><td>57.9 (27.9)</td></tr><tr><td>VS duration, weeks, median (IQR)</td><td>1 (0–10)</td></tr><tr><td>Cerebral haemorrhage, n (%)</td><td></td></tr><tr><td>Grade 1, n (%)</td><td>6 (3.2)</td></tr><tr><td>Grade 2, n (%)</td><td>3 (1.6)</td></tr><tr><td>Grade 3, n (%)</td><td>4 (2.2)</td></tr><tr><td>Grade 4, n (%)</td><td>1 (0.5)</td></tr><tr><td>Necrotizing enterocolitis, n (%)</td><td>5 (2.7)</td></tr><tr><td>Feeding at discharge</td><td></td></tr><tr><td>Breastfeeding, n (%)</td><td>49 (27.1)</td></tr><tr><td>Artificial, n (%)</td><td>100 (55.2)</td></tr><tr><td>Mixed, n (%)</td><td>32 (17.1)</td></tr><tr><td>Low maternal educational level, n (%)</td><td>45 (26)</td></tr><tr><td>High maternal educational level, n (%)</td><td>128 (74)</td></tr></tbody></table>

The results of the neurodevelopmental assessments by ASQ and RBL are summarised in Table 2. The included children had a median overall ASQ-3 score of 265, with an IQR

<footnote><sup>a</sup> Indicates more than one foetus (i.e., twins, triplets). Mean, SD, median, IQR, and percentages were calculated after the exclusion of missing data. Abbreviations: GA, gestational age; HC, head circumference; IQR, interquartile range; n (%), number of patients (percentage); SD, standard deviation; VS, ventilatory support.</footnote>
from 235 to 285, and a mean global DQ score of 91.5, with an SD of 10.2. Twenty-six (14.3 %) children had an abnormal developmental evaluation with the ASQ-3, whereas 36 (19.7%) children had an abnormal developmental evaluation with the RBL scale. The ASQ-3 evaluation identified 8 (4.3%) children with abnormal gross motor abilities, 16 (8.6%) children with abnormal personal-social abilities, 10 (5.4%) children with abnormal communication abilities, 6 (3.2%) children with abnormal fine motor abilities, and 15 (8.2%) children with abnormal problem-solving abilities. According to the RBL evaluation, 35 (18.9%) children had an abnormal gross motor development, 38 (20.5%) children had an abnormal personal-social development, 68 (37.2%) children had an abnormal communication development, and 35 (18.9%) children had an abnormal fine motor development (Table 2).

<table><caption>**Table 2.** Neurodevelopmental assessments.</caption><thead><tr><th>Domains</th><th>Ages and Stages Questionnaire (ASQ)—Third Edition *</th><th>Revised Brunet-Lézine (RBL) Scale *</th></tr></thead><tbody><tr><td>Overall score</td><td>265 (235–285)</td><td>91.5 (10.2)</td></tr><tr><td>Abnormal overall test, n (%)</td><td>26 (14.3)</td><td>36 (19.7)</td></tr><tr><td>Gross motor score</td><td>60 (50–60)</td><td>92.9 (11.7)</td></tr><tr><td>Abnormal gross motor, n (%)</td><td>8 (4.3)</td><td>35 (18.9)</td></tr><tr><td>Personal-social score</td><td>50 (45–60)</td><td>93.5 (12.2)</td></tr><tr><td>Abnormal personal-social, n (%)</td><td>16 (8.6)</td><td>38 (20.5)</td></tr><tr><td>Communication score</td><td>55 (50–60)</td><td>87.1 (13.1)</td></tr><tr><td>Abnormal communication, n (%)</td><td>10 (5.4)</td><td>68 (37.2)</td></tr><tr><td>Fine motor score</td><td>50 (50–60)</td><td>92.8 (11.4)</td></tr><tr><td>Abnormal fine motor, n (%)</td><td>6 (3.2)</td><td>35 (18.9)</td></tr><tr><td>Problem-solving score</td><td>50 (45–60)</td><td>NA</td></tr><tr><td>Abnormal problem-solving, n (%)</td><td>15 (8.2)</td><td>NA</td></tr></tbody></table>

The classification performance of the ASQ-3 is detailed in Table 3. The overall ASQ-3 score had a sensitivity of 0.34 (95% CI from 0.27 to 0.41) and a specificity of 0.91 (95% CI from 0.87 to 0.95). The cross-tabulation of the overall ASQ score and the global DQ score found a positive likelihood ratio of 3.82, a negative likelihood ratio of 0.72, a diagnostic odds ratio of 5.29 (95% CI from 4.43 to 6.17), and an area under curve of 0.67 (95% CI from 0.56 to 0.77). Regarding domain-by-domain evaluation, ASQ-3 had a sensitivity of 0.11 (95% CI from 0.07 to 0.16) for the gross motor domain, 0.08 (95% CI from 0.04 to 0.12) for the personal-social domain, 0.13 (95% CI from 0.08 to 0.18) for the communication domain, and 0.09 (95% CI from 0.05 to 0.13) for the fine motor domain. The ASQ-3 specificity was 0.97 (95% CI from 0.95 to 0.99) for the gross motor domain, 0.98 (95% CI from 0.96 to 0.99) for the personal-social domain, 0.99 (95% CI from 0.98 to 1.00) for the communication domain, and 0.98 (95% CI from 0.96 to 1.00) for the fine motor domain. The ASQ-3 positive likelihood ratios were 4.29, 3.87, 15.22, and 4.29 for the gross motor domain, the personal-social domain, the communication domain, and the fine motor domain, respectively. The ASQ-3 negative likelihood ratios were 0.72, 0.91, 0.94, 0.88, and 0.93 for the gross motor domain, the personal-social domain, the communication domain, and the fine motor domain, respectively. ASQ-3 had a diagnostic odds ratio of 4.71 (95% CI from 3.29 to 6.13) for the gross motor domain, 4.11 (95% CI from 2.49 to 5.74) for the personal-social domain, 17.39 (95% CI from 15.32 to 19.46) for the communication domain, and 4.59 (95% CI from 2.96 to 6.22) for the fine motor domain. The ASQ-3 area under curve was 0.66 (95% CI from 0.48 to 0.85) for the gross motor domain, 0.65 (95% CI from 0.43 to 0.87) for the personal-social domain, 0.78 (95% CI from 0.68 to 0.88) for the communication domain, and 0.66 (95% CI from 0.44 to 0.88) for the fine motor domain (Table 3).

<footnote>* ASQ scores are expressed as median (IQR); RBL DQs are expressed as mean (SD). Abbreviations: DQ, developmental quotient; IQR, interquartile range; n (%), number of patients (percentage); SD, standard deviation.</footnote>
<table><caption>**Table 3.** Classification performance of the Ages and Stages Questionnaire—Third Edition.</caption><thead><tr><th>Domains</th><th>Overall</th><th>Gross Motor</th><th>Personal-Social</th><th>Communication</th><th>Fine Motor</th></tr></thead><tbody><tr><td>Sensitivity, % (95% CI)</td><td>0.34 (0.27-0.41)</td><td>0.11 (0.07-0.16)</td><td>0.08 (0.04-0.12)</td><td>0.13 (0.08-0.18)</td><td>0.09 (0.05-0.13)</td></tr><tr><td>Specificity, % (95% CI)</td><td>0.91 (0.87-0.95)</td><td>0.97 (0.95-0.99)</td><td>0.98 (0.96-0.99)</td><td>0.99 (0.98-1.00)</td><td>0.98 (0.96-1.00)</td></tr><tr><td>Positive likelihood ratio</td><td>3.82</td><td>4.29</td><td>3.87</td><td>15.22</td><td>4.29</td></tr><tr><td>Negative likelihood ratio</td><td>0.72</td><td>0.91</td><td>0.94</td><td>0.88</td><td>0.93</td></tr><tr><td>DOR (95% CI)</td><td>5.29 (4.43-6.17)</td><td>4.71 (3.29-6.13)</td><td>4.11 (2.49-5.74)</td><td>17.39<br>(15.32-19.46)</td><td>4.59 (2.96-6.22)</td></tr><tr><td>AUC (95% CI)</td><td>0.67 (0.56-0.77)</td><td>0.66 (0.48-0.85)</td><td>0.65 (0.43-0.87)</td><td>0.78 (0.68-0.88)</td><td>0.66 (0.44-0.88)</td></tr></tbody></table>

Abbreviations: 95% CI, 95% confidence interval; AUC, area under curve; DOR, diagnostic odds ratio.

The subgroup analysis of the classification performance of the ASQ-3 in children with a low maternal education level is detailed in Table 4. The overall ASQ-3 score had a sensitivity of 0.35 (95% CI from 0.21 to 0.49) and a specificity of 0.88 (95% CI from 0.78 to 0.97) in children with a low maternal education level. In these children, the cross-tabulation of the overall ASQ score and the global DQ score found a positive likelihood ratio of 2.80, a negative likelihood ratio of 0.74, a diagnostic odds ratio of 3.77 (95% CI from 2.38 to 5.16), and an area under curve of 0.66 (95% CI from 0.49 to 0.83) (Table 4).

<table><caption>**Table 4.** Subgroup analysis of the classification performance of the Ages and Stages Questionnaire—Third Edition in children with low maternal education level.</caption><thead><tr><th>Domains</th><th>Overall</th><th>Gross Motor</th><th>Personal-Social</th><th>Communication</th><th>Fine Motor</th></tr></thead><tbody><tr><td>Sensitivity, % (95% CI)</td><td>0.35 (0.21-0.49)</td><td>0.14 (0.04-0.25)</td><td>0.20 (0.08-0.32)</td><td>0.17 (0.06-0.28)</td><td>0.14 (0.04-0.25)</td></tr><tr><td>Specificity, % (95% CI)</td><td>0.88 (0.78-0.97)</td><td>0.92 (0.84-0.99)</td><td>0.97 (0.91-1)</td><td>NA</td><td>0.90 (0.82-0.99)</td></tr><tr><td>Positive likelihood ratio</td><td>2.80</td><td>1.81</td><td>6</td><td>NA</td><td>1.48</td></tr><tr><td>Negative likelihood ratio</td><td>0.74</td><td>0.93</td><td>0.83</td><td>0.83</td><td>0.95</td></tr><tr><td>DOR (95% CI)</td><td>3.77 (2.38-5.16)</td><td>1.94 (NA-4.32)</td><td>7.25 (4.95-9.55)</td><td>NA</td><td>1.56 (NA-3.40)</td></tr><tr><td>AUC (95% CI)</td><td>0.66 (0.49-0.83)</td><td>0.55 (0.30-0.80)</td><td>0.73 (0.47-0.98)</td><td>0.69 (0.62-0.77)</td><td>0.55 (0.29-0.80)</td></tr></tbody></table>

Abbreviations: 95% CI, 95% confidence interval; AUC, area under curve; DOR, diagnostic odds ratio; NA, non-available.

The subgroup analysis of the classification performance of the ASQ-3 in children with a high maternal education level is detailed in Table 5. In these children, the overall ASQ-3 score had a sensitivity of 0.36 (95% CI from 0.27 to 0.44) and a specificity of 0.92 (95% CI from 0.87 to 0.97). In children with a high maternal education level, the ASQ-3 also had a positive likelihood ratio of 4.37, a negative likelihood ratio of 0.70, a diagnostic odds ratio of 6.23 (95% CI from 4.98 to 7.49), and an area under curve of 0.64 (95% CI from 0.50 to 0.77) (Table 5).

<table><caption>**Table 5.** Subgroup analysis of the classification performance of the Ages and Stages Questionnaire—Third Edition in children with a high maternal education level.</caption><thead><tr><th>Domains</th><th>Overall</th><th>Gross Motor</th><th>Personal-Social</th><th>Communication</th><th>Fine Motor</th></tr></thead><tbody><tr><td>Sensitivity, % (95% CI)</td><td>0.36 (0.27-0.44)</td><td>0.08 (0.03-0.12)</td><td>NA</td><td>0.08 (0.04-0.13)</td><td>0.06 (0.01-0.09)</td></tr><tr><td>Specificity, % (95% CI)</td><td>0.92 (0.87-0.97)</td><td>0.99 (0.97-1)</td><td>0.99 (0.97-1)</td><td>0.99 (0.97-1)</td><td>NA</td></tr><tr><td>Positive likelihood ratio</td><td>4.37</td><td>7.85</td><td>NA</td><td>7.58</td><td>NA</td></tr><tr><td>Negative likelihood ratio</td><td>0.70</td><td>0.93</td><td>1.01</td><td>0.93</td><td>0.94</td></tr><tr><td>DOR (95% CI)</td><td>6.23 (4.98-7.49)</td><td>8.42 (5.99-10.84)</td><td>NA</td><td>8.18 (5.90-10.46)</td><td>NA</td></tr><tr><td>AUC (95% CI)</td><td>0.64 (0.50-0.77)</td><td>0.74 (0.41-0.99)</td><td>0.41 (NA-NA)</td><td>0.97 (0.49-0.99)</td><td>0.93 (NA-NA)</td></tr></tbody></table>

Abbreviations: 95% CI, 95% confidence interval; AUC, area under curve; DOR, diagnostic odds ratio; NA, non-available.

The results of the analyses calculating the association between the classification performance of ASQ-3 and a low maternal education level are displayed in Table 6. A Low maternal education level was a major risk factor for incorrectly evaluating children with the ASQ-3 on multivariate analysis (OR = 4.16, 95% CI from 1.47 to 12.03; *p* < 0.01) (Table 6).
<table><caption>**Table 6.** Factors associated with incorrect classification with the Ages and Stages Questionnaire—Third Edition.</caption><thead><tr><th rowspan="2">Variables</th><th colspan="2">Univariate Analysis</th><th colspan="2">Multivariate Analysis *</th></tr><tr><th>OR (95% CI)</th><th>p-Values</th><th>OR (95% CI)</th><th>p-Values</th></tr></thead><tbody><tr><td>Low maternal education level</td><td>3.37 (1.52–7.47)</td><td>&lt; 0.01</td><td>4.16 (1.47–12.03)</td><td>&lt; 0.01</td></tr><tr><td>Gestational age</td><td>0.98 (0.96–1.00)</td><td>0.13</td><td>1.01 (0.97–1.05)</td><td>0.59</td></tr><tr><td>Male sex</td><td>0.85 (0.41–1.77)</td><td>0.65</td><td>1.06 (0.39–2.99)</td><td>0.92</td></tr><tr><td>Hospitalisation duration</td><td>1.01 (0.99–1.02)</td><td>0.11</td><td>1.02 (0.99–1.06)</td><td>0.15</td></tr><tr><td>VS duration</td><td>1.01 (0.97–1.04)</td><td>0.59</td><td>0.98 (0.93–1.02)</td><td>0.42</td></tr></tbody></table>

Abbreviations: 95% CI, 95% confidence interval; OR, odds ratio; VS, ventilatory support. * Adjusted for low maternal education level, gestational age, male sex, hospitalisation duration, and VS duration.

## 4. Discussion

Compared to the RBL, the ASQ-3 had excellent specificity and low sensitivity to assess the neurodevelopment of preterm children. A low maternal education level was a major risk factor for incorrectly evaluating children with the ASQ-3.

This association between the maternal education level and the classification performance of the ASQ-3 is a significant result to consider because a low maternal education level is also a risk factor for impaired neurodevelopmental outcomes in preterm children [14,15]. Altogether, a low maternal education level can be considered as a double-risk factor in preterm children: a risk factor for an impaired neurodevelopmental outcome, and a risk factor for having such neurodevelopmental outcome incorrectly evaluated if the ASQ-3 is used alone to assess the neurodevelopment [14,15].

The Bayley-III and RBL scales evaluate the same neurodevelopmental domains, have the same age groups, and use a similar scoring technique. The accuracy of the RBL scale compared to the Bayley-III scale is 86.4% for motor skills, 90.9% for cognitive skills, 94.3% for communication skills, and 88.6% for sociability skills [16]. Our findings for the specificity and sensitivity of the ASQ-3 agree with most studies comparing the ASQ-3 to the Bayley-III at this age [17–21]. The ASQ-3 is a screening tool rather than a diagnostic assessment [8]. Some studies considered that the ASQ-3 is not reliable for screening neurodevelopment in children [22,23]. One prospective study considered the ASQ-3 equivalent to the Bayley-III at 2 years of CA to detect moderate to severe neurodevelopmental impairment [24]. A recent meta-analysis including 36 studies concluded that the ASQ-3 performance to identify developmental delay in children aged between 12 and 60 months was moderate, with a pooled sensitivity of 0.77 (95% CI, 0.64–0.86), a pooled specificity of 0.81 (95% CI, 0.75–0.86), a pooled positive likelihood ratio of 4.10 (95% CI, 3.17–5.30), and a pooled negative likelihood ratio of 0.28 (95% CI, 0.18–0.44) [25].

In France, the ASQ-3 has been used as an evaluating method based on a study that compared the ASQ-3 scores and RBL DQ scores at 24 months of CA [9]. Compared to our study, the previous study included more patients and found good sensitivity (88%) and specificity (57%) for the ASQ-3 [9]. The high sensitivity might be explained by the RBL being administered earlier, at 24 months of CA, because the neurodevelopmental trajectory becomes complex with age [26]. Additionally, the authors compared the overall ASQ-3 score to the global RBL DQ and did not focus on each domain of the ASQ-3 [9]. Domain-by-domain evaluation allows for more accurate identification of impairment. Besides language skills, which are reported to more strongly predict impairment than other cognitive domains, early developmental milestones before age 24 months poorly predict neurodevelopmental impairment at school age [27]. This is particularly important for fine motor skills, which involve complex neurodevelopmental functions [28].

Although the parental point of view is essential, a professional assessment of the neurodevelopmental trajectory of preterm children is critical. Parent and professional views are different; the threshold level of impairment can differ: professionals tend to prioritize functional goals, whereas parents tend to prioritize activity and participation [29]. Additionally, a professional assessment is limited to a short time window of evaluation,
whereas parental evaluation reflects on an average function over time. Alongside the neurodevelopmental trajectory evaluation, a follow-up of the quality of life is also essential in this population [30,31].

Our study is the first to compare the ASQ-3 and RBL scales domain by domain and the second to compare those two neurodevelopmental assessment scales. A limitation is the population sample size of 185, but the features of this sample are comparable to those of the whole population included in the SEVE Network [11].

**5. Conclusions**

The ASQ-3 has a low sensitivity to assess the neurodevelopment of preterm children. A low maternal education level is a major risk factor for incorrectly evaluating children with the ASQ-3 at 24 months of CA. The ASQ-3 should not be used alone to follow the development of preterm children. Although the parental point of view is essential, a hetero-assessment by a professional using a validated scale is critical to evaluate the neurodevelopment of preterm children.

**Author Contributions:** A.G. conceptualised and designed the study, reviewed and revised the manuscript. M.C. (Magali Celle) performed neuropsychological assessments. M.L. collected data, drafted the initial manuscript, reviewed and revised the manuscript. R.C. performed the statistical analysis, reviewed and revised the manuscript. M.C. (Mathilde Chevin) reviewed and revised the manuscript. T.R.-B. reviewed and revised the manuscript. H.P. reviewed and revised the manuscript. S.C. reviewed and revised the manuscript. All authors have read and agreed to the published version of the manuscript.

**Funding:** This work was funded by the *Agence Régionale de Santé Auvergne-Rhône-Alpes*. The funding source was not involved in the study design; in the collection, analysis, and interpretation of the data; in the writing of the report; and in the decision to submit the paper for publication.

**Institutional Review Board Statement:** The study was conducted in accordance with the Declaration of Helsinki and approved by the Ethics Committee of the Saint-Étienne University Hospital (April 2020, IRBN392020/CHUSTE).

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** Anonymised data are available from the corresponding authors upon reasonable request.

**Acknowledgments:** We are grateful to all health professionals working within the SEVE Network.

**Conflicts of Interest:** The authors declare no conflict of interest.

**References**

1. Pierrat, V.; Marchand-Martin, L.; Marret, S.; Arnaud, C.; Benhammou, V.; Cambonie, G.; Debillon, T.; Dufourg, M.-N.; Gire, C.; Goffinet, F.; et al. Neurodevelopmental outcomes at age 5 among children born preterm: EPIPAGE-2 cohort study. *BMJ* **2021**, *373*, n741. [CrossRef]

2. Mercier, C.; Deforge, H.; Hascoët, J.-M. Neurodevelopment at seven years and parents' feelings of prematurely born children. *Front. Pediatr.* **2022**, *10*, 1004785. [CrossRef] [PubMed]

3. Msall, M.E. Neurodevelopmental surveillance in the first 2 years after extremely preterm birth: Evidence, challenges, and guidelines. *Early Hum. Dev.* **2006**, *82*, 157–166. [CrossRef] [PubMed]

4. Spittle, A.; Orton, J.; Anderson, P.J.; Boyd, R.; Doyle, L.W. Early developmental intervention programmes provided post hospital discharge to prevent motor and cognitive impairment in preterm infants. *Cochrane Database Syst. Rev.* **2015**, *11*, CD005495. [CrossRef]

5. Brunet, O.; Lézine, I. Brunet-Lézine Révisé: Échelle de Développement Psychomoteur de la Première Enfance; Pearson: London, UK, 2001.

6. Armstrong, K.H.; Agazzi, H.C. The Bayley-III Cognitive Scale. In *Bayley-III Clinical Use and Interpretation*; Academic Press: Cambridge, MA, USA, 2010; pp. 29–45.

7. Pierrat, V.; Marchand-Martin, L.; Arnaud, C.; Kaminski, M.; Resche-Rigon, M.; Lebeaux, C.; Bodeau-Livinec, F.; Morgan, A.S.; Goffinet, F.; Marret, S.; et al. Neurodevelopmental outcome at 2 years for preterm children born at 22 to 34 weeks' gestation in France in 2011: EPIPAGE-2 cohort study. *BMJ* **2017**, *358*, j3448. [CrossRef] [PubMed]

8. Squires, J.; Bricker, D.; Twombly, E.; Yockelson, S.; Davis, M.S.; Kim, Y. *Ages & Stages Questionnaires: A Parent-Completed, Child-Monitoring System*, 3rd ed.; Paul H. Brookes Publishing: Baltimore, MD, USA, 2009.
9. Flamant, C.; Branger, B.; Tich, S.N.T.; de La Rochebrochard, E.; Savagner, C.; Berlie, I.; Rozé, J.-C. Parent-Completed Developmental Screening in Premature Children: A Valid Tool for Follow-Up Programs. *PLoS ONE* **2011**, *6*, e20004. [CrossRef]

10. Von Elm, E.; Altman, D.G.; Egger, M.; Pocock, S.J.; Gøtzsche, P.C.; Vandenbroucke, J.P.; Strobe Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: Guidelines for reporting observational studies. *J. Clin. Epidemiol.* **2008**, *61*, 344–349. [CrossRef]

11. Giraud, A.; Chaux, R.; Allard, M.-J.; Celle, M.; Teyssier, G.; Roche, F.; Chapelle, C.; Chabrier, S.; Sébire, G.; Patural, H. Perinatal inflammation is associated with social and motor impairments in preterm children without severe neonatal brain injury. *Eur. J. Paediatr. Neurol.* **2020**, *28*, 126–132. [CrossRef]

12. Baud, O.; Trousson, C.; Biran, V.; Leroy, E.; Mohamed, D.; Alberti, C.; PREMILOC Trial Group. Association Between Early Low-Dose Hydrocortisone Therapy in Extremely Preterm Neonates and Neurodevelopmental Outcomes at 2 Years of Age. *JAMA* **2017**, *317*, 1329–1337. [CrossRef]

13. Eusebi, P. Diagnostic accuracy measures. *Cerebrovasc. Dis.* **2013**, *36*, 267–272. [CrossRef]

14. Benavente-Fernandez, I.; Siddiqi, A.; Miller, S.P. Socioeconomic status and brain injury in children born preterm: Modifying neurodevelopmental outcome. *Pediatr. Res.* **2020**, *87*, 391–398. [CrossRef]

15. Mangin, K.S.; Horwood, L.J.; Woodward, L.J. Cognitive Development Trajectories of Very Preterm and Typically Developing Children. *Child Dev.* **2017**, *88*, 282–298. [CrossRef] [PubMed]

16. Cardoso, F.G.C.; Formiga, C.K.M.R.; Bizinotto, T.; Tessler, R.B.; Rosa, F. Concurrent validity of the brunet-lézine scale with the bayley scale for assessment of the development of preterm infants up to two years. *Rev. Paul. Pediatr.* **2017**, *35*, 144–150. [CrossRef] [PubMed]

17. Williams, J.; Lee, K.J.; Anderson, P.J. Prevalence of motor-skill impairment in preterm children who do not develop cerebral palsy: A systematic review. *Dev. Med. Child Neurol.* **2010**, *52*, 232–237. [CrossRef] [PubMed]

18. Gollenberg, A.L.; Lynch, C.D.; Jackson, L.W.; McGuinness, B.M.; Msall, M.E. Concurrent validity of the parent-completed Ages and Stages Questionnaires, with the Bayley Scales of Infant Development II in a low-risk sample: Validity of Ages and Stages Questionnaires vs. BSID II. *Child Care Health Dev.* **2010**, *36*, 485–490. [CrossRef]

19. Schonhaut, L.; Pérez, M.; Armijo, I.; Maturana, A. Comparison between Ages & Stages Questionnaire and Bayley Scales, to predict cognitive delay in school age. *Early Hum. Dev.* **2020**, *141*, 104933.

20. Smith, C.; Wallen, M.; Walker, K.; Bundy, A.; Rolinson, R.; Badawi, N. Validity of the Fine Motor Area of the 12-Month Ages and Stages Questionnaire in Infants Following Major Surgery. *Phys. Occup. Ther. Pediatr.* **2011**, *32*, 260–271. [CrossRef]

21. Steenis, L.J.; Verhoeven, M.; Hessen, D.J.; van Baar, A.L. Parental and professional assessment of early child development: The ASQ-3 and the Bayley-III-NL. *Early Hum. Dev.* **2015**, *91*, 217–225. [CrossRef]

22. Simard, M.-N.; Luu, T.M.; Gosselin, J. Concurrent Validity of Ages and Stages Questionnaires in Preterm Infants. *Pediatrics* **2012**, *130*, e108–e114. [CrossRef]

23. Woodward, B.J.; Papile, L.-A.; Lowe, J.R.; Laadt, V.L.; Shaffer, M.L.; Montman, R.; Watterberg, K.L. Use of the Ages and Stages Questionnaire and Bayley Scales of Infant Development-II in neurodevelopmental follow-up of extremely low birth weight infants. *J. Perinatol.* **2011**, *31*, 641–646. [CrossRef]

24. Agarwal, P.K.; Shi, L.; Daniel, L.M.; Yang, P.H.; Khoo, P.C.; Quek, B.H.; Zheng, Q.; Rajadurai, V.S. Prospective evaluation of the Ages and Stages Questionnaire 3rd Edition in very-low-birthweight infants. *Dev. Med. Child Neurol.* **2017**, *59*, 484–489. [CrossRef] [PubMed]

25. Muthusamy, S.; Wagh, D.; Tan, J.; Bulsara, M.; Rao, S. Utility of the Ages and Stages Questionnaire to Identify Developmental Delay in Children Aged 12 to 60 Months: A Systematic Review and Meta-analysis. *JAMA Pediatr.* **2022**, *176*, 980–989. [CrossRef] [PubMed]

26. Berardi, N.; Sale, A.; Maffei, L. Brain structural and functional development: Genetics and experience. *Dev. Med. Child Neurol.* **2015**, *57*, 4–9. [CrossRef]

27. Peyre, H.; Charkaluk, M.L.; Forhan, A.; Heude, B.; Ramus, F.; EDEN Mother-Child Cohort Study Group. Do developmental milestones at 4, 8, 12 and 24 months predict IQ at 5–6 years old? Results of the EDEN mother–child cohort. *Eur. J. Paediatr. Neurol.* **2017**, *21*, 272–279. [CrossRef] [PubMed]

28. Bos, A.F.; Van Braeckel, K.N.J.A.; Hitzert, M.M.; Tanis, J.C.; Roze, E. Development of fine motor skills in preterm infants. *Dev. Med. Child Neurol.* **2013**, *55*, 1–4. [CrossRef]

29. Salah, L.; Chabrier, S.; Thébault, G.; Kossorotoff, M. Parental and professional opinion regarding outcome after neonatal stroke. *Dev. Med. Child Neurol.* **2020**, *62*, 1450–1451. [CrossRef]

30. Gire, C.; Resseguier, N.; Brévaut-Malaty, V.; Marret, S.; Cambonie, G.; Souksi-Medioni, I.; Müller, J.-B.; Garcia, P.; Berbis, J.; Tosello, B.; et al. Quality of life of extremely preterm school-age children without major handicap: A cross-sectional observational study. *Arch. Dis. Child.* **2019**, *104*, 333–339. [CrossRef]

31. Gire, C.; The GPQoL Study Group; Tosello, B.; Marret, S.; Cambonie, G.; Souksi-Medioni, I.; Müller, J.-B.; Garcia, P.; Berbis, J.; Auquier, P.; et al. Specific cognitive correlates of the quality of life of extremely preterm school-aged children without major neurodevelopmental disability. *Pediatr. Res.* **2020**, *88*, 642–652. [CrossRef]

**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.