Enhancing Calibration of Comprehension

SUMMARY Calibration of comprehension is the correlation between subjective assessments of knowledge gained from reading and performance on an objective test. Contrary to intuition, this correlation is typically close to zero. This article is structured around four points concerning calibration of comprehension. First, poor calibration of comprehension is the rule, rather than the exception, a fact that has been repeatedly demonstrated in our laboratory and in others. Poor calibration is also typical in at least one other domain: solving insight problems. The high levels of calibration reported in studies on the calibration of probabilities and feeling-of-knowing research may be dependent on using feedback from taking the test to assess the probability of correct performance on the test. Second, we present two experiments that demonstrate that poor calibration of comprehension is not associated with a particular type of performance test but is found with inference tests, verbatim recognition tests, and idea recognition tests. For the most part, poor calibration is found when the test is given immediately after reading as well as when the test is given after a delay. Also, we demonstrate that poor calibration cannot be attributed to unreliable testing procedures. Third, the evidence from three experiments indicates that a likely reason for poor calibration is that subjects assess familiarity with the general domain of a text instead of assessing knowledge gained from a particular text. Assessing domain familiarity is probably easier than assessing knowledge gained from a particular text. Also, under some conditions, applying a domain familiarity strategy does result in spurious calibration, which thereby reinforces application of the strategy. Fourth, we demonstrate that calibration of comprehension can be enhanced if subjects are given a pretest that provides (self-generated) feedback. Even this ability is limited, however. Calibration is enhanced only when the processes and knowledge tapped by the pretest are closely related to the processes and knowledge required on the criterion test. Under these conditions, subjects apparently use feedback from the pretest to predict criterion test performance with a modest degree of accuracy. We briefly discuss the implications of these results for theories of representation of knowledge gained from reading. In preparing for a test of learning, a rational strategy is to our claim that beliefs about how much has been learned are study until one believes that the material is learned. Studying often uncorretated with performance on a test of comprehenfor less time is risky; studying for more time may be wasteful. sion. Second, we will demonstrate that the lack of correlation is For this strategy to be effective, however, beliefs and judgments not due to some methodological artifact, but that it is represenabout how much has been learned must be calibrated. That is, tative of a wide range of situations. Third, we present data supthese beliefs must be correlated with performance. Unfortu- porting one general account for the lack of correlation. Finally, nately for learners, calibration of comprehension often is close we will demonstrate a method for enhancing calibration, to zero. We believe that the results of these studies have implications