Detection of Response Bias in Applied Assessment: Comment on McGrath et al. (2010)

In their 2010 article, McGrath, Mitchell, Kim and Hough (2010) issued a provocative challenge to researchers and clinicians who utilize indicators of various response styles in interpreting results from psychological measures. In their view, the utility of response bias indicators rests upon two assumptions: (1) that response bias suppresses or moderates the criterion-related validity of substantive psychological indicators and (2) that bias indicators are capable of detecting the presence of response bias. In reviewing the psychological assessment literature on response bias, their conclusion was that there was “failure to support the overall hypothesis” (McGrath et al., 2010, p. 450) with respect to these assumptions, and thus, they concluded that “it is difficult to justify the use of response bias indicators” (p. 465). This conclusion was met with some controversy in the assessment field. For example, a response from Rohling, Larrabee, Greiffenstein, Ben-Porath, Lees-Haley, Green and Greve (2011) noted a number of issues with the conclusions offered by McGrath et al. (2010), based both upon the data provided in McGrath et al.’s review as well as upon data not covered by that review. Rohling et al. offered three broad critiques of the review. First, the inclusion criteria for McGrath et al.’s review were thought to be too narrow; second, the presence of errors in interpreting the research that they did include; and finally, a tendency to overlook evidence confirming moderation effects. The purpose of this commentary is to extend some of the discussion offered by Rohling et al., offering additional interpretations and caveats that might further “moderate” the conclusions drawn by McGrath et al. In contrast to the primarily neuropsychological focus of Rohling et al. (2011), my examples will be drawn from the assessment of personality and psychopathology, although the issues raised are typically equally applicable to other assessment domains. Furthermore, I will draw examples from the literature on my instrument, the Personality Assessment Inventory (PAI: Morey, 1991, 2007), although there are numerous examples from other measures as well. Nonetheless, because the PAI includes a variety of these response bias indicators, and because I continue to recommend that clinicians make use of these indicators, I feel that it is important to provide my rationale in support of such a recommendation in response to the issues highlighted by McGrath et al. (2010). Thus, it is worth discussing, as suggested by McGrath et al. (2010), whether (1) there is little or no evidence that bias indicators can detect the presence of response bias and (2) that there is little or no evidence of validity moderation from bias indicators—and if so, why this might be. In doing so, it is important to underscore that many of the issues covered were mentioned (sometimes briefly) by McGrath et al. (2010), so my discussion is meant to elaborate further on the implications of these issues, particularly for their conclusion that “it is difficult to justify the use of response bias indicators” (p. 465).