Structural equation models of faking ability in repeated measures designs

Models were compared on data in which a situational judgment test and measures of the Big Five were administered under honest and fake good instructions. A model with latent variables representing the six measures and a latent variable representing faking ability proved to be a useful representation of the data. PRESS PARAGRAPH Many experimental studies of applicant faking have employed the difference between test performance when instructed to fake with performance when instructed to respond honestly as a measure of faking ability. There have been many arguments against using such difference scores as measures of change in the psychological literature. This study investigated an alternative approach to the conceptualization of behavior in such studies using structural equation modeling techniques. The model developed provided a reasonable fit to the data and allowed an examination of applicant faking that could not have been carried out as efficiently using the difference score approach. Structural equation models of faking ability 2 Structural equation models of faking ability in repeated measures designs One result of the increase in importance of noncognitive measures in selection processes has been a renewed focus on applicant faking and its implications for selection processes. The issue of applicant faking is of little importance when cognitive tests are used for selection. However, when noncognitive tests, such as personality tests, are used, the possibility of choosing employees because of their ability to fake appropriate scores on such tests rather than their actual personality characteristics presents a dilemma for selection specialists. Of course, if the ability to fake is a predictor of performance or other attributes important for the job, the issue of faking is less problematic. And if the amount of faking is constant among the applicant population, then, again, the existence of such faking will not present a problem in personnel selection. But if there are individual differences in faking across the applicant population and if that faking is not related to important job attributes, then the potential impact of such faking on validity of selection instruments is of concern to the selection specialist. The measurement of faking behavior depends on the experimental design in which it is studied. In studies in which no manipulation of faking instruction is employed, investigators have typically used separate measures of faking, such as social desirability scales (e.g., Hough, Eaton, Dunnette, Kamp, & McCloy, 1990; Rosse, Stecher, Miller, & Levine, 1998). In other studies, participants have been instructed to respond honestly at one point in the research and to respond in a faking good fashion in another part. In these repeated measures designs, difference scores have been used to measure the extent of faking (e.g., McFarland & Ryan, 2000, 2003). In addition to differences in the operationalization of faking behavior, some studies have focused on the tendency of respondents to fake in various contexts (e.g., applicant vs. job incumbent; Hough et al., 1990), while other, recent studies have examined the ability to fake Structural equation models of faking ability 3 regardless of context (e.g., McFarland & Ryan, 2000; 2003). The focus of the present research is the ability of applicants to fake. Following McFarland and Ryan (2000, 2003) we employed a repeated measures design. Specifically, we investigated structural equation models as alternatives to the use of difference scores as measures of faking in such designs. The difference between test performance obtained under conditions to fake good and the performance under instructions to respond honestly, referred to here as F-H, is an example of a difference score encountered in a variety of research contexts within psychology. Difference scores have been employed as independent variables in studies to represent person-environment fit, value fulfillment, met expectations, and self-other agreement to name just a few (Edwards, 2002). They also have been considered as dependent variables in studies investigating fakingpersonality relationships (e.g., McFarland & Ryan, 2000). As dependent variables, difference score measures of faking are in good company with measures of pre-post change in the two-level repeated measures designs (e.g., Cribbie, & Jamieson, 2000). The models investigated in this study are not restrictive concerning the independent/dependent variable role of applicant faking. Many arguments have been presented against the use of difference scores, i.e., F-H, as both independent (Edwards, 2002; Cribbie & Jamieson, 2000) and dependent variables (Edwards, 1995) in psychological research. First of all, the predominant argument is the fact that in some circumstances they may have very low reliability (e.g., Edwards, 2002). Secondly, when treated as dependent variables (i.e., change scores) in pretest-posttest designs, it has been pointed out that such use is a special case of analysis of covariance in which within-groups slopes relating posttest to pretest are assumed to equal one (Pedhazur & Schmelkin, 1991). Thirdly, difference or change scores suffer from the fact that they are inherently inversely related Structural equation models of faking ability 4 to pretest values preventing examination of the precursors of change in some instances and sometimes leading to incorrect conclusions concerning such precursors (Cohen & Cohen, 1983). In spite of the arguments against the use of difference scores, they have been employed in recent studies of applicant faking of noncognitive measures. For example, McFarland and Ryan (2000) found generally positive correlations among difference scores from NEO-PI measures of the Big Five personality dimensions (Costa and McCrea, 1989), a bio-data instrument, and an integrity test, leading to the suggestion that there are consistent individual differences in the extent of faking that are not test specific. Their study provided “initial evidence that even when people fake responses, there are individual differences in the extent to which people fake. ” (McFarlend & Ryan, 2000, p. 817-818). In spite of these positive results, there were certain tests that could not be conducted as efficiently as might have been desired using difference scores. These were tests of the relationship of personality traits to faking. Since faking was defined as the F-H test performance difference faking scores for each personality trait were negatively correlated with the corresponding trait scores preventing a clear cut examination of the relationship of faking ability to respondent attributes as represented by the Big Five personality traits. Moreover, even though McFarland and Ryan (2000) found evidence consistent with the existence of a single individual difference variable representing faking ability, their analyses were conducted separately using faking scores for each personality dimension. An approach, which allows examination of a single faking ability variable, would have been desirable. The present study was designed to address the issue of difference scores as a measure of applicant faking and whether there existed a single faking ability trait through the use of structural equation modeling techniques. The data to which the model was applied (Nguyen, 2002) consisted of noncognitive measures administered twice, once under instructions to respond Structural equation models of faking ability 5 honestly and again under instructions to fake good. A series of structural equation models was applied to the data. Models incorporating the relationships of a faking latent variable to latent variables representing personality dimensions and to cognitive ability were assessed. Method Two hundred three undergraduate and graduate students from two southeastern public universities participated in the study in exchange for partial course credit. All participants were given the Wonderlic Personnel test (Form A). After that all participants completed a 31 item situational judgment test (SJT) and a 50-item instrument measuring the Big Five personality dimensions twice, once under instructions to respond honestly and also under instructions to respond faking good. The order of instructions was counterbalanced with half of the participants completing the measures under the “honest” instructions first and the other half under the “faking good” instructions first. The SJT was the Work Judgment Survey described by Smith and McDaniel (1998). The test consists of 31 problem scenarios or situations with five possible responses. Empirical keying had identified one response as the “Most likely” response to the situation and another response as the “Least likely” response. Respondents were asked to indicate which response they would most likely make and which response they would least likely make to the situation. Estimates of internal consistency reliability were . 74 and . 78 for the Honest and Faking instructional conditions respectively. We used the Big 5 personality inventory developed by Goldberg (http://ipip.ori.org/ipip). Each dimension was indicated by 10 Likert-type items. Respondents were asked to indicate the accuracy of each item as a descriptor of themselves with a number ranging from 1 (very inaccurate) to 5 (very accurate). The scales have been validated against other established scales (e.g., NEO-PI) and shown to have good reliabilities (Goldberg, 1999; in press). Estimates of reliability ranged from . 74 to . 90. Structural equation models of faking ability 6 In order to provide adequate indicators of each of the latent variables in the model, three “testlets” for the SJT and for each of the personality dimensions were formed. Bandalos (2002) found that the use of item testlets or parcels resulted in better fitting solutions when items had a unidimensional structure. Previous work with the scales used here suggested that each was essentially