The Effect of Ongoing Professional Development on Interactive Whiteboard Use: A Report of a Randomized Experiment in Forsyth County Schools
Author: Cabalo, Jessica V.
Date: 2007-03-01
Type: Reports - Research
Subjects: Faculty Development,Interaction,Visual Aids,Program Effectiveness,Mathematics Achievement,Elementary School Teachers,Teacher Attitudes,Teaching Methods

Introduction: The Forsyth County School District sought scientifically based evidence for the effect of ongoing, district-developed professional development (PD) on the usage of Promethean Interactive Whiteboards. A randomly selected subset of teachers received PD in addition to basic training sessions offered in Fall 2005 to all district teachers for using interactive whiteboards (IWBs). District decision-makers were interested in learning about the impact of the additional PD on student achievement outcomes in core subjects as well as on teacher classroom practices. The district, considered part of metro Atlanta, GA has an annual growth rate of 8-10% and serves mainly White students. The PD is an ongoing, systematic training structured to include 1- to 2-hour workshops and supplemental resources from the school IT specialist. The objective was to train teachers to perform advanced IWB operations to enhance their instruction and engage their students in learning. We conducted an experiment comparing outcomes for classes taught by teachers randomly assigned to receive the PD and classes taught by teachers assigned to only the basic training on IWBs that all teachers received. Interviews, surveys, and observations allowed us to characterize the use of IWBs both qualitatively and quantitatively. Findings: We found that students of the teachers who received the PD had lower scores on Georgia's state assessment (CRCT) in Math than students of the other teachers. We also found a negative impact on the amount of IWB usage. These unexpected results led us to additional exploratory analyses and considerations. The figure below shows the analysis approach. To understand the mechanism through which the PD resulted in both lower IWB usage and lower Math scores, we investigated whether the difference in IWB usage was associated with the Math outcomes. Here we found some indication of a positive relationship. It therefore became very important to understand how it was possible for additional PD to have a negative effect on the two outcomes. We considered two different phenomena. First, the control students' better performance could perhaps be attributed to non-PD teachers' eagerness to compensate for not having the additional PD, making a greater effort to obtain information from other sources in the school. The second is what researchers call &quot;contamination&quot; of the control group that resulted from PD teachers making all the materials to support whiteboard use available on the school's computer network (the &quot;shared drive&quot;). Contamination is unfortunate from a research standpoint because it eliminates the differences between treatment and control groups. On the other hand, the sharing of resources is consistent with the culture of cooperation within schools, which appears to be an effective tool for multiplying the impact of the PD and spreading information and expertise, allowing teachers to maximally utilize their resources within their schools. Although our goal in this research was to provide the district with evidence that would be useful in determining the impact of their PD program, we found that the value of the PD appeared to be eclipsed by the culture of sharing in the schools. Insofar as: (1) the PD assisted the schools in developing IWB resources for shared use; and (2) the resources helped to increase overall usage for all teachers, the PD can be considered a success. The suggestive finding from the exploratory analysis that whiteboard use may have an impact on achievement remains to be confirmed with properly controlled comparisons. Our sample of teachers was small (15) and for the analysis involving test scores, the study could not use seven of the teachers in grades where no state test results were available. Because few teachers participated, there is a danger of bias being introduced by, for example, more enthusiastic teachers falling by chance into the PD or control group. With a larger pool of teachers, the likelihood of this bias occurring would have been lower. We were able to eliminate technology expertise and interest as potential sources of bias, but others may have been present. Analysis: Statistical analyses involving CRCT scores were based on eight teachers/classes (four PD and four control) and 125 students in grades 3 to 5. We investigated three separate outcomes: (1) the primary topic of our experiment was the impact of additional professional development. We first looked at the impact of PD on the CRCT. We found negative effect sizes in English Language Arts, Reading, and Math, indicating that the students in control classes performed better than their counterparts in PD classes. (Note that nearly all students, regardless of condition, improved on the CRCT in Reading.); (2) second, we examined whether teachers given the additional PD used IWBs more or less than control teachers. We found a statistical difference between PD and control teachers in IWB usage, control teachers using them considerably more than PD teachers. As noted, we attribute this to the culture of sharing often found in successful schools. Surveys revealed that PD teachers made their materials available to control teachers through their schools' shared drive. The table shows that, while PD teachers shared more of their materials, control teachers accessed these materials more than PD teachers; and (3) third, we asked whether the amount of IWB usage was related to the student outcomes, in particular in Math. If there were an impact of PD on student outcomes, we wanted to understand the extent to which that impact was mediated by IWB usage. The significant differences we found in student performance appear to be related to the degree of IWB usage. A test of the correlation reveals a positive relationship between IWB usage and student outcome. This is evident in the graph, which shows a positive slope in the predicted relationship between the z-transformed scores and IWB usage. The p value for this effect is 0.14, which gives us some confidence that the true slope is in fact different from zero. Overall Teacher Impressions: Surveys and interviews revealed overall satisfaction with IWBs and increased student interest and engagement compared to the previous year before their introduction. One teacher referred to the IWB as a &quot;magnet&quot; that &quot;gets the focus of the whole class. All eyes are on the whiteboard.&quot; When asked how they had their students use the IWBs, PD group teachers said their students used them more for basic presentation than for critical thinking activities, whereas the opposite was true for control teachers. Teachers in both groups, however, expressed frustration over the lack of time for planning and practicing with their IWBs and indicated that integrating such technology into their classroom instruction would be a lengthy process.