Pros and Cons of Using Intelligence Batteries for the Study of Clinical Populations: A Response to Beaujean (2017)

In his commentary to our study (Toffalini, Giofrè, & Cornoldi, 2017), Beaujean (2017) raised some important issues. In particular, he suggested that the reliability of the WISC-IV indexes may be inadequate for clinical interpretation, that abnormal differences should be discussed rather than statistical significance, and that evidence of the treatment utility of WISC-IV indexes is limited. We agree with Beaujean that using cognitive strengths and weaknesses in isolation for individual clinical decisions is dangerous. However, investigation of groups is somewhat different; comparing group performances under different subtests or conditions, without definitive conclusions, can enhance clinical psychological research. Concerning technical aspects of Beaujean’s commentary, it must be noted that he calculated the “unique” reliability with the omega-s (ωs), considering a bifactor model of intelligence. Not all researchers agree with this model, suggesting that the superiority of the bifactor over the higher order model could be biased (Murray & Johnson, 2013; but see Gignac, 2016, and Molenaar, 2016, for a different argument). We agree with Beaujean, however, that the bifactor model may provide relevant information. For this reason, we applied the bifactor model to our data on 1,383 children with Specific Learning Disability (SLD; Giofrè, Toffalini, Altoè, & Cornoldi, 2017). The ωs was . 46 for Verbal Comprehension Index (VCI), . 20 for Perceptual Reasoning Index (PRI), . 37 for Working Memory Index (WMI), and . 57 for Processing Speed Index (PSI), which is in line with recent studies of children with SLD in the United States (Styck & Watkins, 2016). Notably, ωs measures the effect of reliability of each index after controlling for the g-factor, and criteria (e.g., cutoffs) for evaluating ωs are not clear. However, some criteria have been proposed based on a simulation study (Gignac & Kretzschmar, 2017). In fact, following these criteria, the unique reliability of indexes in general is relatively large (ωs > .30), with the exception of PRI. Demonstrably, the indexes can provide some useful information. Concerning abnormal differences versus clinical significance, we agree that statistical significance is often arbitrary. However, abnormal differences can also be arbitrary. For example, abnormal differences are defined by fixed criteria (e.g., 5% or 1%) in the standardization sample (Silverstein, 1989). Both statistical significance and abnormal differences raise problems, and the “clinical significance” should be considered instead (Kazdin, 1999). We recently showed that differences within the intellectual profile can be clinically relevant in discriminating between typically developing children vs. children with SLD (with adequate sensitivity and specificity; Giofrè et al., 2017). We agree, however, that distinguishing between different subtypes of SLD on the sole basis of the intellectual profile is problematic because similarities tend to prevail over differences. In our study (Toffalini et al., 2017), we mentioned this fact, but we noted that even small differences can be relevant from a theoretical point of view and should not be neglected. Regarding the treatment utility, we agree that evidence is insufficient to support the exclusive use of cognitive ability tests, particularly WISC-IV indexes, for designing intervention. Intervention techniques are beyond the goals of our study and have not been 720716 CPXXXX10.1177/2167702617720716Toffalini et al.Response to Beaujean research-article2017