Student Understanding of Science and Scientific Inquiry (SUSSI): Revision and Further Validation of an Assessment Instrument

This paper presents the revision and validation of the Student Understanding of Science and Scientific Inquiry (SUSSI) instrument based on the data collected from pre-service teachers in the USA, China, and Turkey. Built on the current national and international science education standards documents and existing literature in science education, SUSSI blends Likert-type items and related open-ended questions to assess students’ understanding about how scientific knowledge develops. It is suggested that SUSSI can be used as either a summative or a formative assessment tool in small or large-scale studies. SUSSI will also be most suitable for conducting cross-cultural comparison studies. The combined quantitative and qualitative methods enhance the sensitivity of the instrument for detecting cultural influences. Student Understanding of Science and Scientific Inquiry 3 Introduction The purpose of this paper is to report the development, revision, and validation of an instrument entitled Student Understanding of Science and Scientific Inquiry (SUSSI). In this project, scientific inquiry is used as a term related to the process by which scientific knowledge is developed, whereas the nature of science (NOS) and scientific inquiry refers to the epistemology of science, the values and beliefs inherent to scientific knowledge and its development (Lederman, 1992, 2004). Understanding of NOS as one of the goals of science instruction in the USA can at least be traced to the beginning of the 20 century (Central Association of Science and Mathematics Teachers, 1907). In the most recent science education reform movements, scientific inquiry and NOS have been identified as critical elements for developing scientific literacy of all learners at both national and international levels (American Association for the Advancement of Science, 1993; National Research Council, 1996; McComas & Olson, 1998; Ministry of Education of the People’s Republic of China, 2001; Ministry of Education, 2003; Turkish Ministry of National Education-Turkey’s National Board of Education, 2005). However, NOS studies consistently show that neither students nor schoolteachers have clear ideas about how science operates or how scientific knowledge develops (e.g., Aikenhead 1987; Cooley & Klopfer, 1963; Lederman, 1992; Rubba & Anderson, 1978; Abd-El-Khalick & Lederman, 2000a, 2000b). This has become a serious concern for many science educators, curriculum developers, and science education researchers at both national and international levels. Furthermore, the assessment of learners’ views of nature of science and scientific inquiry remains an issue in research. A valid and meaningful instrument, which can be used as either a summative or formative assessment tool in small and/or large scale studies, is much needed to track learners’ growth and promote evidence-based practice in the learning and teaching of Student Understanding of Science and Scientific Inquiry 4 science. This has led to the development of the SUSSI instrument. It is envisioned that SUSSI can create a shared context to discuss issues related to learning and teaching the nature of science and scientific inquiry, both locally and globally. Relevant Research on the Nature of Science and Scientific Inquiry Instruments In the last decades, both quantitative and qualitative questionnaires have been developed and used in conducting NOS related research. Examples of traditional quantitative instruments include the Test on Understanding Science (Cooley & Klopfer, 1961), Science Process Inventory (Welch, 1966), Nature of Science Scale (Kimball, 1967), Nature of Scientific Knowledge Scale (Rubba, 1977), and Modified Nature of Scientific Knowledge Scale (Meichtry, 1992). These instruments contain multiple-choice or Likert-type questionnaires and were usually written from perspectives of experts. Jungwirth (1974) and Alters (1997) criticized that those experts did not adequately represent perspectives of scientists, philosophers, and science educators. Moreover, items on these instruments often assumed that all scientists had the same view and behaved in the same way. Views of NOS in these instruments were oversimplified and over generalized. Furthermore, traditional instruments were developed based on an assumption that students perceive and interpret the statements in the same way as researchers do. The instruments failed to detect the respondents’ perceptions and interpretations of the test items. However, research has indicated that students and researchers used language differently and this mismatch has almost certainly led to misinterpretation of students’ views of NOS in the past (Lederman & O’Malley, 1990). Aikenhead, Fleming, and Ryan (1987) also found that students may agree upon a statement because of extremely different reasons. It was suggested that empirically derived, multiple-choice responses could reduce the ambiguity to a level between 15% and 20% (Aikenhead, 1988). Accordingly, Aikenhead and Ryan (1992) developed an Student Understanding of Science and Scientific Inquiry 5 instrument entitled the Views on Science-Technology-Society (VOSTS) over a six-year period. They analyzed 50 to 70 paragraphs written by Canadian students (grades 11-12) in response to two statements representing both sides of an NOS issue, to ensure that all VOSTS items represent common viewpoints possessed by students. Furthermore, “VOSTS items focus on the reasons that students give to justify an opinion” (p.480). The reasons underlying the students’ choices of items are particularly meaningful for teachers to make informed decisions in teaching and for researchers to interpret students’ beliefs appropriately. Nevertheless, several problems were found with the use of VOSTS. For instance, some VOSTS items appeared redundant, and/or had ambiguous positions and overlapping meanings (Chen, in press). Researchers also pointed out that respondents might have combinations of views that would not be reflected in the multiple-choice format (Lederman, Abd-El-Khalick, Bell, & Schwartz, 2002; Abd-El-Khalick & BouJaoude, 1997; Chen, in press). But this particular problem may be resolved by using the Likert scale and scoring model proposed for the use of VOSTS by Vazquez-Alonso and Manassero-Mas (1999). Their proposed scale and scoring scheme allow researchers to make maximum use of the VOSTS items because respondents circle their views on all items, and create data that can be applied to inferential statistics. Currently, the most influential NOS assessment tools on views of the nature of science perhaps are the Views of Nature of Science questionnaires (VNOS), developed by Lederman, Abd-El-Khalick, Bell, and Schwartz (2002). There are several forms of VNOS (e.g., Form A, B, C, D). With certain variations in length and complexity of language used in the questionnaires, all VNOS instruments consist of openended questions accompanied by follow-up interviews. For instance, the VNOS C is composed of 10 free-response questions and takes 45-60 minutes for undergraduate and graduate college students to complete the survey. This presents a Student Understanding of Science and Scientific Inquiry 6 challenging task to respondents with limited knowledge of NOS and writing skills. Most often, students who are not equipped to fully express their own ideas in an open-ended format tend to respond in a few words or simply leave several items blank. This limits the potential of the VNOS instruments as formative classroom assessment forms and/or accurate research tools as anticipated. Other research methods such as follow-up interviews become necessary to clarify the participants’ beliefs. In summary, significant efforts have been made to modify and/or develop instruments aimed at increasing validity and minimizing the chance of mis-interpretation of respondents’ perceptions over the last decades. It appeared that the open-ended questionnaires accompanied interviews would yield valid and meaningful assessment outcomes. However, it may not appropriate as a standardized tool in large-scale assessments. On the other hand, previous research suggested that empirically derived assessment tools would significantly reduce the ambiguity caused by the problem of language. We therefore have developed the SUSSI instrument, by combining both quantitative and qualitative approaches to assess students’ views about how scientific knowledge develops. Methodology SUSSI was developed through a four-phase process. During phase one, the National and International Science Education Standards documents and related literature were examined to select target ideas about the nature of science and scientific inquiry to be included in the instrument. A draft form of SUSSI was developed built on existing instruments and literature and first piloted in the USA and China. In the second phase, a modified version of SUSSI (SUSSI –1) was produced based on the pilot study and expert reviews. Students were also interviewed for content clarification. During phase Student Understanding of Science and Scientific Inquiry 7 three, SUSSI –1 was administered in the USA (in English), China (Chinese translation), and Turkey (Turkish translation). The results were presented at the Eighth International History, Philosophy, Sociology and Science Teaching Conference (Author, 2005). During the last phase, SUSSI was further revised and the current version (SUSSI-2) was administered in the aforementioned three countries again to further examine the validity and reliability of the instrument. Target Ideas about the Nature of Science and Scientific Inquiry Whereas the nature of science and scientific inquiry involves a wide variety of topics in history, philosophy, and sociology of science, SUSSI focuses on the following seven essential elements that are emphasized in national and international K-12 science education standards documents and have been widely discussed in literature (e.g., AAAS, 1