Student Outcomes in Economics Principles: Online vs. Face-to-Face Delivery.

This study looks at the performance of students in an online and face-to-face section of economic principles with the same instructor. After controlling for the bias of students selecting the online section and observable characteristics, we did not find any statistical difference in the exam performance of students across delivery modes of the course. This work contributes to the literature on student outcomes across delivery modes with the implementation of an experimental design in that the instructor purposefully made the sections as similar as possible including the same homework and exams. 1 Author Note: Thank you to Mike Allgrunn and James Murray for econometric consulting. Kathryn Birkeland, Mandie Weinandt & David L. Carr Student Outcomes in Economics Principles: Online vs. Face-to-face Delivery 42 Journal of Learning in Higher Education 43 Fall 2015 (Volume 11 Issue 2) BACKGROUND LITERATURE Studies comparing course delivery modes have looked at differences in student satisfaction, student perceptions of quality, and student performance. Other studies have sought to ascertain whether the learning style of the student leads to different outcomes when students choose online or traditional courses. Our study focuses on student performance, differentiated by course delivery mode. Within this specific literature, there is not a consensus as to the impact of course delivery mode and student performance. We begin with papers finding generally inferior outcomes for online courses. Brown and Liedholm (2002) find that online principles of microeconomics courses are inferior to traditional courses, and find that students who chose an online course would have scored significantly higher had they chosen the traditional mode instead. Crouse (2002) found that students in principles of agricultural economics scored generally lower on exams in the online section. Coates, Humphreys, Kane, and Vachris (2004) found, using two-stage least squares, that students who took principles of economics online scored lower on the Test of Understanding College Economics (TUCE) than students who took the same course face-to-face. They also noted that failing to account for sample selection bias in online and face-to-face sections biases this difference towards zero giving the appearance of no significant difference. Anstine and Skidmore (2005) find the online learning environment for M.B.A. courses to be less effective than the traditional classroom. They use a two-stage least squares analysis and a switching model to account for the fact that a student’s choice of course delivery mode may be related to learning. The difference in student performance shows up only after accounting for the selection bias. Stephenson, McGuirk, Zeh, and Watts Reaves (2005) examined students in an introductory agricultural microeconomics course and found that students with average or below average aptitude test scores had worse outcomes in online sections. Ury, McDonald, McDonald, and Dorn (2006) looked at outcomes for students in computer science and information systems courses. After accounting for student characteristics, students in large courses serving several majors fared worse in online sections than those in traditional sections. Students in upper-level courses focused on a particular major show no difference. Farinella (2007) found that students in introductory finance online courses under-performed their counterparts in the traditional sections. More recently, Gratton-Lavoie and Stanley (2009) found, after correcting for differences in student demographics, the online teaching mode had narrowly insignificant or negative effect. Howsen and Lile (2008) used a twostage least squares model and found students in online sections of principles of macroeconomics scored almost two letter grades lower than those in traditional sections. Trawick, Lile, and Howsen (2010) studied macroeconomics students and found that not only did online students performed worse overall, but also found no selection bias for students choosing the online sections. Scherrer (2011) examines undergraduate statistics courses in traditional, hybrid and online delivery modes, and finds better performance from the traditional sections than in the others. Verhoeven and Wakeling (2011) look at students in an upper-level, quantitative methods, business core course and find students performed significantly lower in the online sections, regardless of their course grade in a lower-level statistics course. While numerous studies have concluded online courses produce inferior outcomes, many others have found no difference, or even slightly superior outcomes for online courses. Navarro and Shoemaker (2000) looked at introductory economics courses and found cyberlearners performed as well or better than traditional students, regardless of student characteristics. Neuhauser (2002) found students in a principles of management course had no significant difference in test scores, assignments, participation and course grades. McLaren (2004) found, for business statistics students who persist, there is no significant difference in final course grade for online and traditional learners. McFarland and Hamilton (2005) found that students in a senior-level undergraduate MIS course had no performance difference in online and traditional sections. Unal (2005) found no statistically significant difference in student performance in an educational technology course. Bennett, Padgham, McCarty, and Carter (2007) examined online and traditional students in micro and macroeconomics courses. They found no difference in performance overall, but found student in microeconomics did better in the traditional setting, while students in macroeconomics had higher scores in online sections. Daymont and Blau (2008) looked at outcomes for students in undergraduate management courses, and found that online students performed as well as traditional students, after controlling for student characteristics. Harmon and Lambrinos (2008) used exam questions rather than students as the unit of observation. In MBA introductory economics courses delivered in a hybrid format, they found students had a significantly greater chance of answering a question correctly if the chapter was taught online. For principles of microeconomics, they also found positive and significant results for the online sections. Bennett, McCarty, and Carter (2011) studied MBA managerial economics courses and did not find overall differences in final grades for online and traditional sections, but they did find that the difference between stronger and weaker students was larger in online sections. Student strength was measured by aptitude and effort. Ary and Brune (2011) found students in personal finance courses performed the same regardless of delivery mode, but that incoming GPA and ACT scores better predicted student success. Euzent, Martin, Moskal, and Moskal (2011) compared introductory economics courses with traditional and lecture capture delivery. Results demonstrated no significant differences in student performance across the delivery modes. Driscoll, Jicha, Hunt, Tichavsky, and Thompson (2012) studied students in online and traditional introductory sociology courses. They found the difference in student performance between delivery modes is accounted for by the presence of a selection effect, and the delivery mode does not have an impact on student performance.