Contrast Effects in Evaluating Essays.

In recent years a number of investigations have identified characteristics of essays that affect their evaluation. Much of this research is focused on the development and utilization of rating procedures that yield moderate to high agreement among judges (Diederich, 1974) and cues people use in forming impressions of written samples (Freedman, 1979; Harris, 1977; Malgady & Barcher, 1979; Nold & Freedman, 1977; Remondino, 1959; Scannel & Marshall, 1966). Little attention has been paid to biases that affect raters in evaluating essays. One potential bias, that introduced by the quality of preceding essays on the evaluation of subsequent essays, is the focus of this investigation. When evaluating writing, readers often are asked to judge a series of essays. This process of reviewing and evaluating a number of writing samples, one after another, may produce a significant bias in the form of a contrast effect between preceding essays and the essay currently being read. Contrast effects refer to the influences of previous stimuli on the evaluation or judgment of a new stimulus. Studies of visual perception (Helson, 1947, 1964), social judgments (DiVesta, 1961; Holmes & Berkowitz, 1961; Hovland, Harvey & Sherif, 1957), weight estimates (Sherif, Taub, & Hovland, 1958), attraction ratings of females (Kenrick & Gutierres, 1980), and interview judgments (Carlson, 1970; Hakel, Ohnesorge & Dunnette, 1970; Kopelman, 1975; Landy & Bates, 1973; Rowe, 1967; Wexley, Yukl, Kovacs & Sanders, 1972) have obtained contrast effects by examining judgments of a stimulus that follows, and is different from, earlier stimuli on an important dimension. In each case, characteristics of previous stimuli affect judgments of subsequently presented stimuli, often leading to less than optimal judgments. Although in a single session most evaluators of writing examine a number of essays which vary in quality, only limited research considers whether the order in which written samples are read affects their evaluation. Hales and Tokar (1975) asked student-teachers to read a group of essays where the criterion average essay followed a block of either poor or excellent papers. They found a significant effect attributable to the quality of preceding .essays. The criterion paper was rated more positively when it came after the group of poor essays than when it followed the good papers. In a follow-up study, Hughes, Keeling, and Tuch (1980) used a similar design but added conditions that varied according to whether the block of excellent or poor essays were read by the subject before or after a group of variable quality essays had been read. In every condition, the criterion average paper followed the block of good or poor papers. They found that the quality of preceding essays affected judgments of the criterion paper regardless of whether the experimental block came first or after a group of randomly selected papers. Both the Hales and Tokar (1975) and Hughes et al. (1980) investigations are limited by the absence of adequate comparison control groups. In both studies, the only rating of the criterion essay was obtained after subjects had read an experimental block of either good or bad essays. Finding differences between these two experimental conditions does not demonstrate a meaningful effect in the absence of two critical control conditions. The