Mixed Methods Evaluation of Statewide Implementation of Mathematics Education Technology for K-12 Students
Author: Brasiel, Sarah
Date: 2016-00-00
Type: Reports - Research
Subjects: Mixed Methods Research,Program Evaluation,Program Effectiveness,Mathematics Education,Educational Technology,Technology Integration,Elementary Secondary Education,Teacher Surveys,Technological Literacy,Pedagogical Content Knowledge,State Programs,Fidelity,Benchmarking,Technology Uses in Education,Barriers,Teacher Attitudes,Feedback (Response),User Satisfaction (Information)

An extensive body of research has demonstrated that the use in a K-12 classroom of technology, such as the Internet, computers, and software programs, enhances the learning of mathematics (Cheung &amp; Slavin, 2013; Cohen &amp; Hollebrands, 2011). In particular, growing empirical evidence supports that certain types of technology, such as intelligent tutoring systems and adaptive learning systems, have a positive impact on students' academic achievement in math and their attitudes toward math (Arroyo, Burleson, Tai, Muldner, &amp; Woolf, 2013; Ma, Adesope, Nesbit, &amp; Liu, 2014; Pane, Griffin, McCaffrey, &amp; Karam, 2013; Steenbergen-Hu &amp; Cooper, 2013). These kinds of learning systems yield positive effects by providing students with personalized instruction tailored to &quot;the pace, order, location, and content of a lesson uniquely for each student&quot; (Enyedy, 2014, p. 3). However, despite the recognized benefits, many teachers still struggle with successfully integrating technology into their instruction. Through funding from the state legislature, over 200,000 K-12 students were given access to 11 mathematics education technology products. The authors surveyed teachers to understand the implementation successes and challenges. The authors' review of prior research and the TPACK framework informed the research as the authors analyzed open-ended survey data on teachers' perceptions of the education technology implementation over one school year. At the end of the year, researchers collected data from the state office of education on student assessment and demographic characteristics for use in a quasi-experiment to understand the impact of the mathematics technology products. The following research questions were addressed: (1) Is there any significant effect of using mathematics education technology through the statewide grant program on student state achievement; (2) Is there any significant effect of using the technology for students who met the fidelity of implementation benchmark; (3) How were the education technology products being used; (4) With what features of the products or experiences are teachers most satisfied; (5) What concerns or challenges have teachers experienced with use of the products; (6) What barriers limit teachers from using the products to their desired level; and (7) How have teachers used the performance management features of the products? While the state assessment data is very important, the authors provide a detailed overview of teacher feedback because it sheds light on their experiences implementing the products/programs and opportunities to learn lessons from implementation that can inform future years of implementation. Tables and figures are appended. [SREE documents are structured abstracts of SREE conference symposium, panel, and paper or poster submissions.]