EVALUATION INFORMATION' AND DECISION MAKING IN ORGANIZATIONS: SOME CONSTRAINTS ON THE UTILIZATION OF EVALUATION RESEARCH

Versus t'ororete information. Typically evaluations are conducted to assess whether a program meets its goals, is successful compared to other projects, or in order to provide for the feedback of information to improve performance in succeeding time periods. Underlying these reasons is the idea that evaluation information will allow decision makers to derive inferences about causal relationships (for example, between educational 41, interventions and student achievement). An important postulate of this process holds that causal explanations will be influenced by consensus information, i.e., information concerning base rates and how a given project fares compared to the base rate. Attribution theory researchers (e.g., Nisbett, Borgida, Crandall & Reed, 1976) have drawn attention to the sub. stantiayMounI of evidence which has failed tasupport,the postulated effect of consensus information. For instance, Tvers,XY and Kahneman (1973) have demonstrated this point by asking subjects to judge the probability that a target individual, described in a brief personality sketch, was an engineer, given: (a) that he was drawn from a population of 70 engineers and 30 lawyers, or (b) that he was drawn from a population of 70 lawyers and 30 engineers. , 4 Knowledge of the population base rate for occupational categories had no effect' whatever on judgments of the probability that the target individual was an engineer. Instead, subjed,ts relied exclusively on the personality sketch and based the decision on the degree to which the description fitted the stereotypic engineer or lawyer. Numerous other examples are available which demonstrate that decision.makers, and even scientists familiar with statistics, habitually ignore information about the population and draw recklessly strong inferences about the underlying population from knowledge of a very small sample. Kahneman and Tversky (1973) refer to this as the "law of small numbers. " O Observers offer some reasons why even expert decision makers ignore base rate or consensus information: Kahnem;n and Tversky, for example, speculate that people may not know how to combine base rate information and, therefore, ignore It. Nisbett, et al, (1976) propose,that base rate information by its very nature, is abstract and pallid, and may simply lack the force to persuade subjects'to attend to tt and to use it. Outsiders, it seems, are unmoved by dry, statistical data dear to the hearts of scientists and evaluators. As Bertrand Russell observed, "Popular induction depends on the emotional interest of the instances, not upon their number'\ (1927, p. 269). Individuals respond to vivid, concrete information and ignOre abstract data. Nisbett, et al. (1976). offer several examples of this tendency. Consumers have long ignored medical advice to quit smoking and safety advice to fasten seat belts. Such appeals typically report numbers such as the probability of being a victim and are largely ignored. Yet when a highly vis ible and concrete incident occurs which people can focus on, results are often dramatic. For example, medical exhortation on the value of early