Decomposed and Holistic Job Analysis Judgments: The Moderating Role of Experience

We investigated whether holistic judgments in job analysis are more susceptible to inflation than decomposed judgments. Moreover, we examined whether experience moderates the relationship between decomposed judgments and holistic judgments. We find that more experienced incumbents will display less convergence in their decomposed and holistic judgments than more experienced incumbents. PRESS PARAGRAPH It is a widespread belief that more experienced employees should be used in job analysis because of their superior knowledge of the job. As such, they are expected to provide more accurate judgments in job analysis. Building on decision making and memory literature, however, we suggest that the opposite might in fact be true: More experienced employees tend to use heuristics in their decision making process which can be detrimental to the overall quality of their judgment. We suggest that there is an optimal level of work experience at which job incumbents can rate their jobs with the highest accuracy. Decomposed and Holistic 3 Job analysis is a fund amental research area in the field of industrial and organizational psychology. Also, its practical relevance for HR managers and practitioners is widely recognized (Shippmann, Ash, Battista, Carr, Eyde, Hesketh, et al., 2000). In general, job analysis describes “a wide variety of systematic procedures for examining, documenting, and drawing inferences about work activities, worker attributes, and work context” (Sackett & Laczo, 2003). In particular, job analysis forms the basis for such key HR processes and activities as job grading/job evaluation, hiring, performance management and compensation, the design of career succession models, or the design of individual and organizational development programs. Because of this, it is essential that job analysis information be accurate and free from bias. Accordingly, scholars have begun to investigate the range of factors that may systematically influence job analysis judgments (Morgeson & Campion, 1997). At least two issues arise when investigating job analysis judgments. The first concerns what kinds of judgments are made during the data collection process. Traditionally, job analysis has focused on highly specific, “decomposed” judgments in which numerous specific job tasks are rated. Recently, however, there has been a trend toward more “holistic” judgments in which more general judgments are made. Such a trend toward holistic judgments is most evident in competency modeling efforts where very general job aspects are rated (Shippmann et al., 2000). Although research has been conducted comparing decomposed and holistic judgments, evidence in the job analysis domain has been equivocal. The second issue that arises concerns the role that job incumbent experience plays in job analysis judgments. Some have suggested that more experienced incumbents provide different information (Borman, Dorsey, & Ackerman, 1992; Landy & Vasey, 1991). There are two conceptional reasons for this finding: First, more experienced employees have greater knowledge Decomposed and Holistic 4 of the job. Second, with increasing experience, job incumbents perform the job differently. Others have suggested that less experienced individuals provide more accurate information because they are more likely to systematically process information about the job (Richman & Quinones, 1996). Still others have suggested that experience has no relationship with job analysis judgments (Mullins & Kimbrough, 1988; Schmitt & Cohen, 1989; Silverman, Wexley, & Johnson, 1984). To address these gaps in the job analysis literature, we directly compare decomposed and holistic judgments using a within-subjects research design. This will add needed insight into effects of using more holistic judgments in job analysis. We then seek to understand the role of work experience in job analysis judgments by investigating how a multidimensional conceptualization and operationalization of work experience (following the work of Tesluk and Jacobs, 1998) influences convergence in decomposed and holistic job analysis judgments. This contributes to the job analysis literature by enhancing our understanding of which incumbents (in terms of work experience) are best able to make holistic job analysis judgments. Human Judgment in Job Analysis Rating job content and job requirements is a complex cognitive process involving considerable rater judgment (Cornelius & Lyness, 1980; Morgeson & Campion, 1997). Raters must take into account numerous factors before judging the importance of various attributes of a specific job. Although it is commonly assumed that job analysis methods produce reliable and valid information, research in social and cognitive psychology has demonstrated that human judgment is limited in numerous ways. Morgeson and Campion (1997) have highlighted a number of potential social and cognitive sources of inaccuracy in job analysis, and recent research suggests that some of these processes operate in job analysis contexts (Morgeson, Decomposed and Holistic 5 Delaney-Klinger, Mayfield, Ferrara, & Campion, 2004). Consequently, connecting basic research on human judgment and decision making with job analysis research can improve our understanding of the potential sources of inaccuracy in job analysis (Cornelius & Lyness, 1980). Miller (1956) is among the first to acknowledge the limitations of human judgment. He suggests that humans can only handle a limited number of conceptual units when making judgments and decisions. In a similar fashion, Driver and Streufert (1969) present a theoretical model according to which decision makers are no longer able to incorporate the number of external stimuli available to them when making judgments once a certain level of complexity has been reached. For the job analysis domain, this would imply that human judgment becomes increasingly unreliable as the complexity of the rating stimuli increases. Interestingly, a major recent trend in job analysis has been a movement toward competency modeling approaches (Shippmann et al., 2000). One of the key ways in which competency modeling differs from traditional methods of job analysis is in a stronger emphasis on holistic judgments. In traditional job analysis measurement, job incumbents are often asked to rate a set of very specific tasks, which are then combined into a more general behavioral category (or competency). This can be thought of as a decomposed rating strategy. In competency modeling approaches, however, job incumbents are often asked to make judgments of very general categories of behavior. This can be thought of as a holistic rating strategy. Although each approach has its strengths and weaknesses, it is clear that a holistic rating strategy is more complex and places higher cognitive demands on the rater. A long tradition of research in the decision making literature has supported the view that decomposed judgments are superior to holistic judgments (Miller, 1956; Driver & Streufert, 1969). Yet the superiority of decomposed judgment strategies in the domain of job analysis has Decomposed and Holistic 6 not been fully supported. For example, some studies have found only small or non-significant differences when comparing outcomes in job analyses for decomposed or holistic judgments (Cornelius & Lyness, 1980; Sackett, Cornelius, & Carron, 1981). Yet other studies report significant differences between the two judgment strategies. For example, Butler and Harvey (1988) found “near-zero convergence between the holistic and decomposed dimension ratings” (p.761). More recently, Morgeson et al. (2004) were able to show that decomposed task ratings in job analysis are less susceptible to inflation than holistic competency ratings. Given these divergent findings, additional research is needed comparing decomposed and holistic job analysis judgments for different jobs and in different organizational contexts. In addition, to investigate this issue it is important to use a within-subjects research design in which the two different types of judgments can be directly compared. We expect that decomposed job analysis judgments will be less inflated (lower mean ratings) than holistic judgments due to the reduced complexity of the judgment task. Hypothesis 1: Decomposed job analysis ratings will be less inflated than holistic job analysis ratings. The Role of Job Experience in Job Analysis Judgments Although we expect decomposed job analysis ratings will be less subject to inflation than holistic ratings, no previous research has investigated whether some job incumbents are better able to make holistic judgments. There is reason to believe that some incumbents are more capable of making holistic judgments than others. We propose that job experience influences the degree to which raters can make accurate holistic competency judgments. Two different arguments have been offered to explain why job experience predicts job analysis outcomes. The first, rather intuitive explanation highlights the fact that more experienced employees will know their jobs in greater detail than less experienced employees. Decomposed and Holistic 7 Consequently, they would be in a better position to rate the content of jobs in job analysis. Also, Landy and Vasey (1991), Borman, Dorsey, and Ackerman (1992) note that with increasing experience, incumbents tend to emphasize different aspects of their jobs. An alternative explanation has been offered more recently by Richman and Quinones (1996). Drawing on research into human memory, they suggest that increased experience leads to routines that make it very difficult for job incumbents to remember the frequency with which certain tasks have been performed. In a lab experiment, Richman and Quinones found support for their hypothesis that low experience participants would evidence greater accuracy than participants in the high experience condition. These findings are consistent with resea