Testing two screening instruments for autism spectrum disorder

SIR–The article by Charman et al. has many strengths and presents interesting results, but we disagree with one major conclusion. One of the strengths of the study is that it follows reallife clinical decision-making, rather than being done in a research-supported setting. Another is that the population consisted largely of minority families, many with low educational levels and additional languages. It is unfortunate that screening instruments do not generally work as well in such populations. Our own data suggest that specificity rather than sensitivity is more affected, with higher rates of false positives in families with lower education, and this is consistent with the Charman et al. findings. As a population screener, the Modified Checklist for Autism in Toddlers (M-CHAT) is designed to identify children who may need further services as early as possible. Therefore, we prioritize sensitivity over specificity. The sensitivity of the M-CHAT for autism reported in this study (82% for the total sample, 93% for the younger half of the sample, for which the M-CHAT is intended) is satisfactory, and is quite consistent with other M-CHAT studies by us and other international research groups (mchatscreen.com). In the Charman et al. study, the screener was used for a somewhat different purpose, that is, as a level 2 screener for children already identified with developmental concerns. Specificity is expected to be lower in such a sample, and was probably also lowered by the lack of follow-up interview, which eliminates many false-positive cases. We are currently testing an online version in which the followup questions are administered seamlessly. This will eliminate the physician burden of the follow-up and hopefully also decrease false positives. The clinician concerns to which the screeners were compared were presumably from specialists and not primary care pediatricians. Physician concern in general pediatrics, by itself, is inadequate in detecting autism, hence the American Academy of Pediatrics (AAP) recommendation for developmental surveillance combined with standardized screening. It should also be noted that the older children in the sample were out of the proper age range for the MCHAT, and the younger ones appeared too young for the usual age range for the Social Communication Questionnaire (SCQ). Mean age at screening was 35 months, above the recommended M-CHAT age range of 16 to 30 months and older than the AAP recommendations for universal screening (18mo and 24mo). Together with the screening of already identified children, as well as the good sensitivity reported, using the findings from this study to argue against population-based screening seems unwarranted. Regarding psychometrics, the authors state ‘. the specificity of both the M-CHAT (used in a one-stage manner) and the SCQ has been low (50% or lower), meaning that reliance on screening . could result in many screen “false negatives”. ’ The authors presumably meant ‘false positives’, which the follow-up interview was designed to reduce. They then state ‘. if the sensitivity of screening instruments is low, there is a cost to children and families in potential delayed assessment and diagnosis, as well as the distress of unnecessary referrals. ’ Unnecessary referral is generally an issue when specificity (rather than sensitivity) is low. Such false positives can indeed raise unwarranted parent concern and strain resources, but most children without autism who screen positive on the M-CHAT have other developmental conditions, mitigating this concern. The impact of false negatives (missed cases) seems to us worse than potential false positives, especially in light of recent studies suggesting that very early intervention (before age 2 or 3) can lead to a better outcome. The authors state that ‘Based on their performance in this study, reliance on either of the screening instruments in isolation would lead to considerable under identification of children likely to go on to receive an ASD diagnosis . ’ While M-CHAT false positives might be high without the follow-up and in clinically referred samples, sensitivity appears high enough not to lead to considerable underidentification. In fact, we completely agree with the authors’ conclusion: ‘. clinical judgement in combination with information gained from administering specific tests and assessments (including information from ASD screening instruments), as well as the presence of parental concerns . should inform the decision to refer for more specialized assessment rather than reliance on the result of a screening instrument in isolation. ’ This conclusion seems quite consistent with the recommendations of the AAP and with our support of screening in general pediatrics in the context of ongoing developmental surveillance.