Improving the Feedback Cycle to Improve Learning in Introductory Biology Using the Digital Dashboard

At Carnegie Mellon we are integrating timely and targeted feedback for the students and real-time student progress reports for the instructor to create an online learning environment that engages the student, improves learning and allows immediate adaptation of instruction. The Digital Dashboard is a dynamic portal into the continuous data provided by student use of online instructional tools and assessments within the Open Learning Initiative environment. The compiled data presentation from each lesson is a guide to changing and targeting instruction in the classroom. The Digital Dashboard organizes information and is being developed to link concepts, categorize answers to open-ended questions and provide a use-index to better inform the instructor of student learning. The information provides tools for making the classroom more dynamic and data for research on student learning. There are many challenges involved in teaching introductory Biology courses effectively. Some of these stem from the fact that introductory courses generally have high enrollments, and others relate more specifically to the structure of knowledge in Biology. First, with high course enrollments, there is great variability in students’ background knowledge, relevant skills, and future goals, which makes it difficult for the instructor to address students’ diverse needs (Felder & Brent, 1996; Fink, 2003). Furthermore students arrive with preconceived views of the topics and a false sense of security in having heard many of the topics in previous courses. Second, the larger one’s class, the harder (and more costly) it is to employ the best teaching practices that foster deep learning, e.g., personalized instruction, rich and timely feedback, and interactive learning environments (CFE, 2003; NRC, 2000). Third, even though the conceptual structure of knowledge in Biology is clear to experts, the array of new ideas and unfamiliar terminology in introductory courses tends to overwhelm students into memorizing a bunch of isolated facts (Chi, 2005; diSessa, 1993). Fourth, within Biology curricula, concepts are introduced initially in basic form and then applied and relied upon in multiple contexts, and yet students often compartmentalize what they learn, missing opportunities to connect their knowledge and generalize their understanding. From research on how students learn, two well-supported principles have emerged that are particularly relevant to addressing these challenges. First, students’ learning improves and their understanding deepens when they are given timely and targeted feedback on their work (Butler & Winne, 1995; Corbett & Anderson, 2001; NRC, 2000, 2001). By “feedback” we refer to corrections, suggestions, cues, and explanations that are tailored to the individual’s current performance and that encourage revision and refinement. Second, for students to benefit from a conceptual framework that organizes the material they are learning, instruction needs to make that framework salient, and students need to practice making connections between related ideas (Eylon & Reif, 1984). A less studied area of research involves how instructors can best use information on their students’ progress to effectively adapt their teaching. And yet, adapting instruction to students’ needs is also consistent with the principle of student-centered teaching and hence should derive similar benefits (McKeachie, 2001; Slunt & Giancarlo, 2004). This presentation will describe a set of tools designed to take advantage of these learning principles by improving the feedback cycle both for students and for instructors, in introductory Biology courses in particular and in other courses in general. The presentation will demonstrate innovative feedback techniques based on existing technologies that connect diverse data sources on students’ work via an instructor’s digital dashboard that facilitates meaningful display and interpretation of student progress in real time. The Benefits of Feedback The benefits of feedback have long been known (e.g., Thorndike, 1913), and yet recent research has greatly refined our understanding of when and how feedback should be given to best suit particular educational goals. Regarding timing and frequency of feedback, the best learning outcomes occur when feedback comes immediately after the students’ response but not before the student is ready to make adjustments in his or her performance or understanding (Corbett & Anderson, 2001). Regarding the nature of feedback – as with instruction in general – it is more effective when presented in a way that relates to students’ prior knowledge (NRC, 2000). Perhaps the most important finding regarding the effectiveness of feedback is that the feedback must lead students to revisit the activity that led to the feedback in the first place (Butler & Winne, 1995). Only then will the students get practice that exercises and extends their improved understanding. Together, these results lead to a model that explains why timely and targeted feedback is so helpful for learning. In essence, effective feedback (1) provides information targeted to the individual’s recent responses and prior knowledge and (2) guides the student to refine his or her understanding through continued practice. A unique feature of our approach is that we also emphasize the role of feedback to the instructor. In this feedback cycle, the instructor gets information about students’ current state of knowledge (e.g., common areas of strength or difficulty) and can refine his or her teaching to better address students’ needs. In both cycles, feedback offers information – based on data gathered from students’ learning experiences – that students and instructors can use to adjust their behavior. Examples of feedback to students include explanation of a misconception evidenced by the student’s performance, cues to important conceptual relationships that the student missed, and suggestions to try a new exercise or a different strategy based on the prediction that it will aid the student’s future learning. Examples of feedback to the instructor include summaries of class performance on particular activities or topics, catalog of identified areas of student difficulty, and reports on how far students have progressed through the instructional material. A guiding philosophy applies throughout the development of these tools and includes the following tenets: • Develop flexible, modular, and modifiable tools and materials that instructors can and will use. • Use technology judiciously but effectively. • Bring together a multi-disciplinary team to address a common set of challenges and build a culture of educational research and development. • Incorporate research from multiple literatures, including cognitive psychology, education, educational technology, and science education. • Progressively refine our work based on formative assessment data. The feedback tools discussed in this presentation form an important part of the course and its design. In particular, the design and implementation of the feedback tools is always conducted with a keen eye toward keeping students actively working with the material and offering frequent opportunities for self-assessment of their own learning progress. The instructional tools, some developed specifically for this context and others adapted to it, are all designed in light of best practices for teaching and learning. Finally, although the online course integrates these components into a coherent whole, we know different instructors will have differing needs. So, we make the components modular, adaptable, and usable in stand-alone form. Putting it into Practice The framework for this presentation is based on the collaborative development of an online Modern Biology course (www.cmu.edu/oli) with a team of domain experts, educational technologists and cognitive scientists. This work is funded by a grant from The William and Flora Hewlett Foundation. The theme of course development involves use of the best available research from the cognitive and learning sciences and continual evaluation and revision based on actual student-learning data. It is also important to note that, even though this is an online course for Modern Biology, its approach and many of the underlying concepts relate directly to Biochemistry and Chemistry courses as well. Hence, faculty from all three courses have been involved in the development of the online course, contributing their expertise both to help the online course and to inform the teaching and use of resources in their own courses. The online Biology course also represents a model of educational research and development – that of a multidisciplinary team all focused on improving instruction. It has been through this project that we have come to appreciate even more the complementary areas of expertise that each member of the team brings to the table and to learn a great deal by working with each other. Development of the online Biology course involves creating instruction that goes far beyond a computerized textbook. For each module in the course, we create course content that interleaves text, graphics, animations, voice-over videos, simulators, interactive mini-tutors, comprehension and reflection questions, and more. From students’ point of view, the pattern for each module is always one of reading the learning objectives for the given topic, gaining some initial exposure, and then continually learning and checking their understanding as they move through the online material (Brown, Bajzek, & Burnette, 2006). The Tools Within the OLI Environment Within the OLI environment there are many opportunities available for use by the instructor to assess student learning. In the online Biology course some of the instructional tools that provide feedback are:, • Interactive animations: animations that allow students to visualize complicated biological processes and interact with th