The Confidence-Accuracy Relationship in Diagnostic Assessment: The Case of the Potential Difference in Parallel Electric Circuits.

(ProQuest: . denotes formulae omitted)Research in science education has revealed that students have conceptions that differ from the accepted scientific ones in many science subjects (Driver, Guesne, & Tiberghien, 1985; Driver, Squires, Rushworth, & Wood-Robinson, 1994; Duit, 2009). Although several research methods have been employed to elicit these alternative ways of reasoning (White & Gunstone, 1992), written diagnostic questions have been the primary method for data collection from a large sample in a relatively short time. A diagnostic question analyzes an individuals performance to uncover his or her strengths or weaknesses in the subject tested. In addition to the information provided by students' answers to such questions, the knowledge of their confidence in their answers may prove to be useful. Confidence judgements provide insights into "how well a person evaluates and monitors his or her performance'' (Stankov & Crawford, 1996, p. 971). Students with unwarranted high confidence in their knowledge in a given domain are considered "overconfident,'' whereas those with unwarranted low confidence are "under-confident. '' Overconfidence in a given area of knowledge indicates that (a) "such knowledge is frequently invoked," (b) "such knowledge is rarely questioned or checked against other criteria, so that wrong conclusions are not detected," and (c) "such knowledge may be maintained, even in the face of explicit counterevidence, so that wrong conclusions persist" (Reif & Allen, 1992, p. 28). Conversely, under-confidence in a given area indicates that "such knowledge may not be believed, even if correct, and may be rejected in favor of incorrect conclusions" (Reif & Allen, 1992). Research on over/under-confidence has revealed that many people are overconfident in their knowledge (Lichtenstein, Fischhoff, & Phillips, 1982). In the research literature, overconfidence and under-confidence are related to the concept of calibration investigating the relationship between confidence in and accuracy of performance. Alexander (2013) defined calibration as "the distance between perceived and demonstrated levels of understanding, capability, competence, or preparedness" (p. 1). She indicated that there is an increasing interest in the topic of calibration among educational researchers and argued that one reason for this is that constructs and processes, such as conceptual change, metacognition, and self-regulation, are closely linked to learners' ability to accurately assess their own capacities. Furthermore, Alexander argued that learners' subsequent efforts and strategic behaviors will be affected by their calibration.Another important concept in the research literature on the confidence-accuracy relationship is resolution (Mengelkamp & Bannert, 2012). Sharp, Cutler, and Penrod (1988) defined resolution as students' ability "to discriminate correct from incorrect judgments by differentially assigning confidence judgments to accurate and inaccurate judgments" (p. 272). Students who can appropriately discriminate between what they know and what they do not (i.e., the students with good resolution) may use their study time more effectively than the students with poor resolution of confidence.Data on calibration and resolution are considered valuable because the findings from the research on the confidence-accuracy relationship imply that students' confidence in their science knowledge is a part of their understanding of natural phenomena.Confidence Judgements in Science Education ResearchSeveral studies in science education research have employed the Certainty of Responses Index (CRI) proposed by Hasan, Bagayoko, and Kelley (1999) to distinguish a lack of knowledge from alternative conceptions. In their study, the confidence levels of the CRI were "totally guessed answer," "almost a guess," "not sure," "sure," "almost certain," and "certain. " According to Hasan et al. …