Effectiveness of Online Algebra Learning: Implications for Teacher Preparation

This study of an online algebra course looked at the overall effectiveness of the course and at the development, implementation, and evaluation of interactive tools for graphing linear equations. The first part of the study compared algebra achievement of virtual school students to that of public classroom-based students in the same state. The second part of the study focused on an interactive tool that was evaluated with virtual school algebra students. The performance of these students on the component was compared to the performance of students who did not use the intervention. The achievement of online algebra students was equivalent to that of students in faceto-face algebra courses, as was the performance of students learning in the online course with the interactive tools as compared to students not using the tools. The implications of the unique nature of the online algebra course for teacher preparation are discussed. The quality of algebra curricula and instruction has received special attention in recent years when the mathematics test scores of students in the U.S. have been compared to scores of students in other countries, scores on state mathematics tests have been used to evaluate schools, and the need for qualified scientists and engineers has grown. As a gateway course to further education, algebra is a critical milestone in secondary education. Algebra is increasingly integrated into elementary school mathematics, and the majority of secondary level students take a stand-alone algebra course. States, districts, schools, and instructors need information about effective approaches to teaching algebra. Hundreds of online algebra courses are currently offered in North America annually, and their enrollment growth shows no signs of slowing. This paper describes the results of a study of the effectiveness of one online algebra course, and explores the preparation of teachers for online algebra courses. Research has provided knowledge of effective algebra instruction in face-to-face classrooms (Beatty, 2005; McCoy, 2005; Gamoran & Hannigan, 2000; Carpenter, Franke & Levi, 2003). Over the last decade several studies have compared the mathematics achievement of secondary students in distance education programs to the achievement of their counterparts in traditional classrooms (Calderoni, 1998; Ryan, 1996; Yasin & Luberisse, 1998), and several states and provinces have released reports comparing mathematics performance of virtual charter school students with classroom-based students (Alaska, 2003; Colorado, 2003; Indiana, 2004; Minnesota, 2003; Schollie, 2001; Texas, 2003; Washington, 2003). While individual results have been mixed, taken together these studies and reports indicate that students in virtual school mathematics courses can be expected to perform at levels at least as high as students in traditional mathematics courses (Cavanaugh et al., 2004). However, each study and report has used a different measure of mathematics achievement, and none have looked specifically at algebra. In addition, the number of students in the past studies was as low as 7. This study focused on the quality of an online algebra course, looking at the overall effectiveness of the course, and at the development, implementation, and evaluation of interactive tools for a particularly complex and abstract component of the course. The first part of the study was designed to learn how algebra achievement of virtual school students compares to that of public classroom-based students in the same state. The second part of the study focused on a specific component of the online algebra course that has been the most difficult for students, according to data collected on over 1300 students over the last four years at the virtual school where the study took place. An interactive tool was designed and developed, and it was implemented with a sample of virtual school algebra students to address the problematic component of the course, and the performance of these students on the component were compared to students who did not use the intervention. The Setting for the Study A state virtual school was selected as the setting for this study because of its long and successful history of offering high school courses to students online. The school was established by the state legislature in 1996 to offer education in cooperation with public schools, and now it operates as an independent nonprofit school serving students across the U.S. and internationally. The school educates thousands of students each year from a wide diversity of geographic locations, socioeconomic groups, and school backgrounds. Thus, the sample of students participating in the study was seen as representative of virtual school students nationwide. Because each course offered by the school is developed by certified and qualified school staff, there was flexibility to adapt and evaluate course components. Courses are aligned with state and national standards, a factor that makes them similar to algebra courses offered in other settings. The algebra course is among the courses with the highest enrollment at the school, and the instructors of the course regularly collect data on students’ performance in each module of the course as they work on continuous improvement of the course. Quality is a prime goal of the school, and it is enhanced through the use of frequent course reviews and outside evaluations. Therefore the school administrators and staff had a strong interest in acquiring knowledge via this study that would have the potential to improve the quality of the algebra course. The Effectiveness of the Online Algebra Course The algebra achievement of two groups of algebra students was measured using an established, reliable algebra exam. The groups were: 1. Students enrolled in online algebra courses developed by the Virtual School 2. Students enrolled in classroom-based algebra courses taught in state public schools All students were taught by state certified mathematics teachers following the state standards, and using state adopted materials. Algebra achievement was measured when students had reached 70% completion of the course using the Assessment of Algebraic Understanding (AAU) published by Educational Testing Service (ETS). The AAU is a 50-item, multiple-choice test that was designed “for determining the extent to which students have mastered algebraic concepts. The items are aligned with the Algebra Standards as described by the National Council of Teachers of Mathematics (NCTM)” (ETS, 2004). The test can be administered according to any schedule. The reliability coefficient (KR-20) for the 2003 form of the Assessment of Algebraic Understanding is 0.8655. The virtual school students who had reached the same point in the algebra course as face-to-face classroom students (70% completion) by the target date received a copy of the AAU exam, answer form, and consent form by mail. Students enrolled in classroom-based algebra courses taught in a public school in a district served by the virtual school also received the tests and consent forms during the same time period. The tests at all sites were administered according to the guidelines provided by Educational Testing Service. Data Analysis and Results The public classroom-based school had 97 out of the 98 students participate and complete the ETS AAU. Twelve out of 139 virtual school students completed the exam. A higher number of returned tests would be expected if the testing had been done later in the year when more students had reached the 70% completion point in the course, if the test could have been administered online, and if the course could have been designed from the beginning of the year to include the test as a requirement rather than as a voluntary activity. The ETS AAU full scores for the virtual group ranged from 19 to 32 while the full scores for the classroom-based group ranged from 10 to 33. Means for the ETS AAU full score were 24.08 for the virtual group and 19.43 for the classroom-based group. The differences in the variability of the scores may be due in part to the larger, more variable sample of the classroom-based group and to the assessment being administered in a group setting within the classroom. Contrarily, the virtual school students completed the assessment individually, primarily at home, and at their own leisure. The voluntary participation of the assessment, the fact that the assessment did not count toward their school grades, and the test completion setting may be factors in the low completion rate of the virtual students. Further, the virtual students who did complete the assessment may be somehow different from the virtual students who did not complete the assessment. For instance, these students may be more academically motivated as well as naturally higher achieving than those who did not complete the assessment. Further, the virtual group may have had access to other resources at their disposal whereas the classroom-based group did not. Due to these assorted reasons, one should take care in drawing conclusions from the overall higher scores in the virtual group compared to the classroombased group. Interactive Tools for Graphing Linear Equations Over time, mathematics educators at the Virtual School have identified five components of the algebra course that tend to be most problematic for students. One of the components, graphing linear equations, is the focus of this part of the study. Graphing equations is a representational activity, involving conceptual understanding of mathematical concepts, operations, and relations, along with strategic competence to represent the information (NRC, 2001, p. 257). To become proficient in any representational activity, students need to learn and use anticipatory thinking that comes from making meaning for the rules that will be applied. Students need experience generalizing, and the use of tools such as spreadsheets, graphing c