A STUDY OF RELIABILITY AND VALIDITY EFFECTS OF TOTAL AND PARTIAL IMMEDIATE FEEDBACK IN MULTIPLE‐CHOICE TESTING

For some years it has been optimistically thought that measurement of partial knowledge in multiple-choice testing would enhance reliability and validity. Confidence weighting of all options, and elimination of variable numbers of (presumed incorrect) options have been among the most frequently attempted means of tapping partial knowledge (Hakstian & Kansup, 1975). These techniques have generally failed to produce reliability and validity coefficients that are consistently greater than those obtained with conventional procedures; the methods have been especially disappointing when the relative amount of testing time they demand has been considered (Hakstian & Kansup, 1975). A different approach to measuring partial knowledge involves securing an examinee's second, third, etc. choices of options only when previous choices are wrong. This method has several advantages over confidence weighting. The first four of these advantages also apply when this approach is compared to a method described by Diamond (1975). First, the method may save testing time; for those items answered correctly on an examinee's first attempt, no consideration of other options is necessary. Second, the immediate feedback inherent in this approach may enhance learning. Next, students can score their own classroom achievement tests (for their own information if not for "the record") before turning them in; the need to return papers or post scores may be obviated. Fourth, scoring may be simpler. Finally, the directions given examinees will, if followed, maximize their expected scores; this ethical consideration is unsatisfied by several schemes for confidence weighting (Kansup & Hakstian, 1975). The information supplied the examinee may either be total feedback, wherein (s)he is informed as to whether each answer is correct, and if incorrect (s)he is to continue selecting answers until feedback on a subsequent trial indicates that (s)he has selected the keyed option, or it may be partial feedback, wherein (s)he is merely informed whether hir answer to each question is right or wrong. The major lure of testing with immediate feedback during the last half century has been the learning thought to accompany it (e.g., Angell, 1949; Pressey, 1950). While studies of the learning effects of immediate feedback testing may use both total feedback, i.e., answer-until-correct (AUC), and partial feedback experimental conditions (e.g., Wentling, 1973; Hanna, 1976), previous investigations of the measurement concomitants of immediate feedback testing have been limited to total feedback conditions This is appropriate, since partial feedback testing seems to offer no better provision for measurement of partial knowledge than does no feedback, i.e., conventional, testing. The major attractions of total feedback testing over conventional testing from a measurement standpoint center on (1) the realness-to-life and/or content validity of continuing to respond in some situations until feedback signifies success and (2) the possibly enhanced reliability accruing from the measurement of partial knowledge. Gilman and Ferry (1972) used an AUC carbon erasure procedure with graduate students of educational measurement. For each examinee, an AUC score was de-