Process Versus Content in the Study of Judgmental Accuracy

Arie Kruglanski has offered a breathtaking integration of the varying rubrics under which psychologists have studied human judgment. In a single, simple, 2 x 2 model, he has integrated such traditionally diverse areas as attribution theory, attitude change, cognitive therapy, and even philosophy of science. The benefits of his integrative model are many. For instance, by pointing out how consistency, consensus, and distinctiveness are all special cases of the covariation principle, Kruglanski's discussion of attribution theory brings that body of research much closer to being a conmprehensive and plausible account of interpersonal judgment. And, Kruglanski provides a valuable insight when he discusses how shifts in scientific viewpoints and even paradigms can be accounted for through generic processes of attitude formation and change. In short, I believe that Kruglanski's model of knowledge acquisition (or "epistemics," to use his preferred term) is itself a significant contribution to knowledge. However, the reach of Kruglanski's model is not unlimited, as I suspect its author would wish to be the first to acknowledge. I do not know what all its limits might be, but one seems rather obvious to me, perhaps because of my own epistemic motivations. Lay epistemic theory is an integrative model-an exceptionally wide-ranging model-of the process of judgment. As such, and despite the author's valiant attempts, it cannot address judgmental accuracy. As Kruglanski himself notes more than once, accuracy is a matter of content. The distinction between process and content is both fundamental and subtle. Consider an example: Somebody once pointed out that if you want to understand completely how a chess player thinks, and in particular if you want to be able to predict his or her next move, it will not suffice to gamer knowledge, no matter how extensive, about the chess player's cognitive processes (his or her "software") or brain functioning (his or her "hardware"). You will also have to acquire an understanding of chess. I hope the point is clear. Cognitive mechanisms serve to process information, but they are not the information itself. Chess is a domain of content in its own right, and you will never be able to predict the player's future behavior, or even be able to determine whether somebody is a good player, by studying only his or her processes of judgment. You will also need to understand the game itself, what the player thinks about the game and how often, against what players, and how, he or she tends to win. This analogy comes close to explaining why studies of judgmental process, even when they are viewed through such an admirably integrative model as Kruglanski's, will never get us far toward understanding accuracy. Kruglanski allows as much when he states that "it is doubtful whether a general process may be uncovered whereby accurate judgments are reached" (emphasis in the original). Exactly -ight, except that it's not just "doubtful"-it is impossible. This is why I argued a couple of years ago (Funder, 1987) that research on judgmental error is utterly uninformative about accuracy: Studies of error tell us about the process of judgment rather than about its content. In particular, I argued, the existence of errors, demonstrated in laboratory settings, cannot be taken as evidence that judgment in real life is generally inaccurate. I also argued, in a seldomnoticed passage (1987, p. 87), that when subjects make judgments correctly in the lab, this cannot constitute evidence that they are generally accurate in real life. Thus, I think Kruglanski misrepresents my point of view at least slightly when he cites my 1987 article as having claimed that "persons' judgments are more likely to be accurate in natural versus artificial settings. " That was not the point I was trying to make. I Rather, my point was that the accuracy of judgments that subjects render in artificial settings is simply not informative about the accuracy of their judgments in real life. For all I know, in real life, accuracy is even worse. But evidence is not being gathered one way or the other as long as research remains inside the laboratory, where it can address only process and not content. That said, and at the risk of reinstating the misunderstanding I have just tried to eliminate, let me add that the laboratory situations in which judgmental errors are demonstrated are not merely artificial in some neutral way. Rather, they are specifically designed to fool subjects. (Procedures that don't succeed at fooling subjects are eliminated in the pretesting phase.) Subjects are shown impossible combinations of stimuli, or are presented irrelevant information under circumstances designed to imply that the information is relevant, or even are told outright lies (e.g., "This sociable-unfriendly person is real, and was evaluated by trained psychologists") th t cannot help but lead subjects down the primrose path toward what the experimenter will triumphantly label an erroneous inference (see Funder, 1987, for several specific examples). 2 I own up to a belief that subjects' judgments in real-life settings, which often are confusing but are seldom so perverse, will be more accurate than they are in artificial settings like these. But, let me reiterate, it would be trivially easy to design a study that would lead subjects to make correct judgments of experimental stimuli. Unfortunately, such a study would not prove that people reason will in real life, for exactly the same reason that studies of error do not show that people reason poorly. So, how does one study judgmental accuracy? If we want to understand the judgments of a chess player, we must understand chess. It would follow that to understand the judgments of a weather forecaster, we should learn something about meteorology (e.g., Lusk, Hammond, & Steward,