THE EFFECTS OF TECHNOLOGY-INFUSED INSTRUCTIONAL STRATEGIES ON STUDENT LEARNING OUTCOMES IN ECONOMICS: A COMPARATIVE STUDY

This study investigates technology-infused teaching strategies with a view to identifying and selecting a teaching method and practice that will improve student learning in principles of macro and microeconomics. Two teaching strategies, clickers or personal response system and “just-in-time teaching”, were used and compared. Each method was applied in different classes for principles of macro and microeconomics courses taught by two instructors over two semesters. This classroom research found that clickers contributed more to student learning outcome than “just –in-time teaching” even after controlling for selected observable determinants of academic performance. Bezuneh & Yiheyis Electronic Teach SBAJ: Vol 12 (1&2):48-70 49 INTRODUCTION Economics is perceived by many undergraduate students as one of the most challenging disciplines in business and social sciences. An increasingly fewer number of students choose economics as their major or minor in undergraduate studies compared to related disciplines. While this phenomenon could be attributed to differences in job opportunities and other factors, at the University where the sample of this study is drawn, the perception of relative difficulty is anecdotally cited as one explanation (Knoedler and Underwood, 2003). This perception is a problem not only in recruiting students who would potentially major or minor in economics but also for those who take economics courses as part of their requirements for their chosen fields of study. At this University, principles of micro and macroeconomics courses are taken each semester by non-economics majors most of whom are students in majoring in accounting and business administration. The average level of performance of students in these courses has been worsening over the years, with a failure/withdrawal rate surpassing that in comparable courses. To be sure, observed students’ relative performance in these courses cannot be entirely attributed to perception/misperception and apprehension, although these are expected to play a part in isolation or in interaction with other factors, which may include inadequate relevant academic background, lack of motivation and effective Bezuneh & Yiheyis Electronic Teach SBAJ: Vol 12 (1&2):48-70 50 engagement on the students’ part, and perhaps inappropriate or nonadaptive teaching methods on the instructors’ part. Economics at the undergraduate level has been commonly taught using the traditional “talk and chalk” or direct instruction method (Becker & Watts, 1996, 2001). Although the use of other types of teaching methods is slowly on the rise, standard lectures and chalkboard presentations still dominated the instruction of economics by 2005 (Becker & Watts, 2007). And the way economics has been taught compared to other disciplines has been asserted as one of the factors for the relative decline in economics enrollment in the 1990s (Becker, 1997; Knoedler & Underwood, 2003). Evidently, the traditional method is becoming less and less effective for a student body with the aforesaid attributes. Given the appeal of technology to this generation of students and the technological savvy they readily display, and given the lack of active engagement asserted to be associated with the traditional method (Cooper, 1995; Schank, Berman, & Macpherson, 1999), it would be reasonable to inquire and explore what technology-infused course delivery may be better suited to the way students learn and, as a consequence, would exert a greater impact on teaching and learning effectiveness. Put differently, what instructional technology would be more effective in dealing with the problems of motivation, study habit, class attendance and engagement and thereby enhance student learning and, in the process, help change students’ perception about the discipline of economics? Bezuneh & Yiheyis Electronic Teach SBAJ: Vol 12 (1&2):48-70 51 Needless to say, there is no single “all-cure” method that addresses observed student learning problems. Of the instructional strategies suggested in the relevant literature, we considered personal response system (PRS) and Just-in-Time Teaching (JiTT) and compared their effectiveness in teaching principles of economics. These methods have not been extensively used in teaching economics, and the empirical evidence on their effectiveness in the discipline is thus rather thin. The purpose of this study is, therefore, to compare the two technology-infused teaching strategies with a view to identifying and selecting teaching methods and practices that will improve student learning. The remainder of the paper is organized as follows. The next section describes the instructional methods under study. The third section describes the research method and the profile of the study sample, followed by a presentation and discussion of results. The final section summarizes and concludes. AN OVERVIEW OF THE PEDAGOGICAL TOOLS USED This section briefly reviews the two technology-infused and potentially active learning-based instructional strategies under study, PRS and JiTT. PRS or Clickers: PRS is a form of classroom technology that allows an instructor to ask multiple-choice or true/false questions and Bezuneh & Yiheyis Electronic Teach SBAJ: Vol 12 (1&2):48-70 52 instantaneously receive responses individually from students on a hand-held wireless transmitter. The responses from students are aggregated and the results are anonymously displayed on a screen for them to see. This technology is asserted to have multiple benefits including the following. First, PRS helps gauge student comprehension of the course material covered, allowing the instructor to identify misconceptions or lack of understanding and address them in the subsequent lecture segment during the same class session. This is expected to have a direct favorable effect on student learning outcome. Second, the system encourages active student learning and engagement as students know that they will be quizzed on the material covered right after the lecture. Third, given that some students struggle to maintain concentration levels for the entire class period of 50-75 minutes, administering quizzes in the middle (or at some appropriately designated intervals) of the lecture provides the necessary break which will help minimize the incidence of boredom and passivity. Lastly, the system can be used to take attendance, and this may decrease unexcused absences from class. The existing empirical evidence on wireless classroom communications systems overwhelmingly points to its effectiveness in one or other dimension of the learning process. Some results based on classroom research conducted in a number of disciplines suggest that the use of clickers positively influences class attendance and engagement (e.g. Chemistry: Ward, Reeves, & Heath, 2003; Economics: Elliott, 2003; Biology: Ribbens, 2007; Psychology: Stowell Bezuneh & Yiheyis Electronic Teach SBAJ: Vol 12 (1&2):48-70 53 & Nelson, 2007; Shapiro, 2009), although there are reports who found no clear evidence for this to be the case (Morling, et al., 2008, Judson and Sawada, 2002). As well, a favorable effect on learning outcomes , as measured by exam scores and other assessment instruments, is reported by a number of researchers (e.g. Physics: Poulis, Massen, Robens, & Gilbert, 1998; Biology: Hatach, Jensen, & Moore, 2005; Economics: Ball, Eckel , & Rojas, 2006; Biology: Ribbens, 2007; Statistics: Lass, Morzuch, & Rogers, 2007; Physiology: Gauci, Dantas, Williams, & Kemm, 2009 ; Psychology: Morling et al.), while mixed or conditional results are documented by other studies [e.g. Computing Science : Kennedy & Cutts, 2007; Psychology: Stowell & Nelson, 2007). Just in-Time Teaching Method: As described by Novak, Patterson, Gavrin,& Wolfgang (1999), the original developers of this teaching technique, Just-in-Time Teaching is a strategy that comprises of classroom activities that encourage active learning and web-based activities designed to enhance the former (see also Simkins & Maier, 2010). The JiTT strategy involves completing and electronically submitting appropriately designed exercises on material that will be covered in the next class, requiring students to read ahead on their own prior to class coverage. The instructor reviews students’ responses prior to class and uses them to modify the next lecture and classroom activities that will overcome students’ misunderstanding, or lack of understanding, of concepts. This strategy’s in-class component complements the out-of-class pre-lecture activity component, Bezuneh & Yiheyis Electronic Teach SBAJ: Vol 12 (1&2):48-70 54 creating a positive “feedback loop” that enhances student learning by (1) encouraging students to come to class better prepared, (2) allowing immediate feedback on students’ comprehension of course material, and (3) enabling “just-in-time” modifications of class activities and discussion on the basis of their performance on the outside-of-class activities. The modification of lectures informed by students’ work heightens their sense of ownership of their learning, motivating them to complete assignments and to engage actively in classroom activities and discussions. There are classroom experiment reports suggestive of the effectiveness of this strategy. Simkins and Maier (2004) find that it exerted a statistically significant effect on student learning in introductory economics. As well, research findings reported by authors in various disciplines (biology, geosciences, physical sciences, history, and in a course on critical thinking in humanities) show that this pedagogical tool contributed to student learning outcomes (Simkins & Maier, 2010). Likewise, completing online homework assignments, even without the other components of JiTT, is reported to have improved exam scores in statistics classes (Lass et al., 2007). RESEARCH METHOD AND STUDY SAMPLE The classroom research for this study was carried out for two economics