EVALUATING SCIENTIFIC REASONING ABILITY: THE DESIGN AND VALIDATION OF AN ASSESSMENT WITH A FOCUS ON REASONING AND THE USE OF EVIDENCE

The study of scientific reasoning ability (SRA) is one of the frequently discussed topics in science education. In decades, researchers and educators have devoted great efforts to exploring how students develop their scientific reasoning ability and how their scientific reasoning ability affects their learning achievements (Coletta, Phillips & Steinert, 2007; Ding, 2018; Johnson & Lawson, 1998). In science education, the development of scientific reasoning ability is an essential goal that has long been pursued and prioritized (Bao et al., 2009; Engelmann et al., 2016). Thus, one of the primary tasks of science education is to cultivate students into good reasoners and become scientific literate (Lawson, 2004). K-12 science educators have also made great efforts to foster students’ scientific thinking and reasoning through their engagement in familiar phenomena in daily life contexts (Kind & Osborne, 2017; van der Graaf et al., 2019). To cultivate student reasoning ability in and for science learning, many studies have investigated the nature of reasoning (Driver et al., 1994) and its development via teaching practices (Lawson, 2004; Zimmerman, 2000; 2007). Leveraging these research efforts, a batch of assessments of scientific reasoning using standardized tests have been constructed and implemented. These assessments, which mainly focused on the evaluation of the level and complexity of reasoning involved in the processes of solving science problems, can inform and improve educational practices of science (Kind, 2013; Kalinowski & Willoughby, 2019; Lee & She, 2010). However, the valid and reliable assessments of scientific reasoning ability are still needed to diagnose research gaps and identify areas of improvement in science learning and teaching. The recent discussion on scientific reasoning in science education uncovers and underscores the significance of evidence use. Evidence is viewed as the premise and basis of valid reasoning in the fields of logic and cognitive psychology (Toulmin, 2003), and the collection and analysis of evidence are necessary and vital for the formation of scientific reasoning and thinking (Kanari & Millar, 2004). Scientific evidence and evidence-based reasoning should be the core of students’ science learning experiences (Duschl, 2003). Abstract. Scientific reasoning ability (SRA) is widely recognized as an essential goal for science education. There is much discussion on the design and development of assessment frameworks as viable tools to foster SRA. However, established assessments mostly focus on the level of students reasoning attainment. Student ability to use evidence to support reasoning is not adequately addressed and evaluated. In this study, the 6-level SRA assessment framework was conceptualized and validated iteratively via synthesizing literature and a Delphi study. Guided by the framework, an SRA assessment tool adopting and adapting PISA test items and self-created items was developed and administered to 593 secondary students (including 318 8th Graders and 275 9th Graders) in mainland China. Pearson correlation analysis of SRA assessment score and their scores in scientific reasoning provided criterion-related validation for the former (Pearson correlation = .527). Rasch analysis conducted further confirmed the validity and reliability of the SRA test and the assessment framework. Combing quantitative and qualitative methods, the study provides a valid and reliable analytical framework of SRA. It can inform the design of SRA assessments in various science education contexts for diversified audiences.