EXPLAINING INTERRATER RELIABILITY (PERFORMANCE APPRAISAL, HOSPITAL NURSES).
Contributors: Michigan State University.
Published: 1984
Number of pages: 394
Subjects: Psychology, Industrial.
Description: The purpose of this study was to empirically evaluate three hypotheses about the causes for low interrater reliability in performance ratings from supervisors, peers, and incumbents. Low reliability has been attributed to different observations by supervisors and peers, different behaviors exhibited by incumbents in the presence of supervisors and peers, and different perspectives of the incumbent's job. Theories and research findings from cognitive-social psychology and role theory were reviewed to explain the rationale for the hypotheses. Ratings of observational frequency of incumbents' job behaviors were obtained from supervisors and peers to assess the first explanation. The frequency with which rates enacted behaviors in the presence of supervisors and peers were obtained from incumbents to measure the second proposition. Supervisors', peers', and incumbents' job performance standards and judgments about the importance of job dimensions were obtained to examine the third assertion. Ratings of incumbents' job performance were obtained from supervisors, peers, and incumbents in order to relate each of the hypothesized explanations to performance rating interrater reliability and performance rating variance.

Head nurses (supervisors) and staff nurses (peers and incumbents) employed by 16 hospitals throughout Michigan participated in the research. Results indicated differences in behavioral frequency of incumbents when in the presence of their supervisors and peers. There were no differences between supervisors' and peers' observational frequency or among the three rater groups' performance standards or importance ratings. Further, there was no relationship between any of the explanations and performance rating interrater reliability or performance rating variance. Results are discussed in terms of distortions attributed to cognitive schemas and the limitations of the sample. Possible future research utilizing videotapes to establish greater control over the situation is described.
Notes: Source: Dissertation Abstracts International, Volume: 45-08, Section: B, page: 2723.

Thesis (PH.D.)--MICHIGAN STATE UNIVERSITY, 1984.

School code: 0128.