Training teachers to use new technologies impacts multiple ecologies: Evidence from a national initiative

A pair of papers re-examined the evidence from a national initiative to train all teachers in England to bring them up to the level of newly qualified teachers, who are required to know when to use and when not to use information and communication technologies (ICT) in their professional practice. The first paper confirmed that multilevel evaluation of professional development was robust for ICT teacher training. This second paper contrasts the highest and lowest rated designs for ICT teacher training: an ‘organic’ approach that provided training in schools was highly rated, whereas a centralised computerassisted learning approach with online access to trainers was the lowest rated design. The study supports an ecological view of the diffusion of ICT innovations in education and recommends that ICT teacher training be designed to support evolution of each teacher’s classroom, school and region, as well as the training of the ICT teacher trainers. Introduction Teachers’ use of information and communication technologies (ICT) has an important role in education in the 21st century (Hinostroza, 2008; UNESCO, 2008). Realising this, many countries and regions are designing ICT-related teacher training like Africa (Evoh, 2007). UNESCO (2008) has recently published guidance for this. Research has established that effective ICT teacher training is an important pillar for successful integration and sustainability of ICT in education (Culp, Honey & Mandinach, 2003; Haydon & Barton, 2007; Somekh, 2008). Therefore, research to inform the design of effective ICT teacher training is urgent, and given the demand, it is important to know how provision may be scaled up, possibly mobilizing computer-based training (CBT) and distance learning that has proved successful in some industries. Lawless and British Journal of Educational Technology Vol 40 No 5 2009 861–878 doi:10.1111/j.1467-8535.2008.00875.x © 2008 The Authors. Journal compilation © 2008 British Educational Communications and Technology Agency. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA. Pellegrino’s (2007) recent literature review of technology-related professional development also calls for a study that compares models of technology-related professional development and the study of scaling such interventions from a local to a national level. A national initiative in England from 1999 to 2002 that stimulated a range of approaches to ICT teacher training within a common evaluation framework provided an opportunity to study different approaches to ICT teacher training within the same population of teachers. This initiative was known as New Opportunities Funding (NOF) after its source of charitable funding. As described by Preston (2004), 47 approaches to training provision were authorised by the Teacher Training Agency (TTA) and all providers improved their approaches following several rounds of evaluation. Overall, that evaluation had many negative points to make: the financial and promotional incentives for training were inadequate: the estimated costs were based on extensive online delivery, but teachers lacked the skills and broadband provision was unstable; and the evaluators found that it was difficult to estimate what the teachers had achieved because there was no directive for national accreditation standards and only two models required the development of portfolios. Another key problem was the lack of time release in the school day and the after-school training was found to be tiring. The reputation of NOF in the first 2 years was also damaged by two reports by national inspectors that suggested that there had been no impact on the learning of the students (Ofsted 2001, 2002). Evaluation of the impact of training on the students’ learning very early in the teachers’ learning cycle indicated how little was understood nationally about the challenges of integrating ICT into teaching and learning. Despite nearly 30 years of government initiatives, many teachers in England did not have the computer skills at the start of this project to explore the potential for teaching and learning. However, at the end of the 4-year period, some of the smaller training models were beginning to have an effect on their regions. In-depth feedback both from the teachers and ICT experts was used to improve all the professional development models so they improved significantly by the end of the period, except for the few training programmes that closed following the loss of accreditation. Preston’s (2004) evaluation provided a broad national picture for the funding agency and participants. This national programme has international significance so the evidence was revisited to provide a more in-depth analysis. The main objective of this paper is to inform the design of ICT-related teacher training through the contrast of two teacher-training designs in order to examine their characteristics and the application of ecological theory to ICT teacher training, including the scaling up of provision to train large numbers of teachers. Methodology As noted in the abstract, Davis, Preston and Sahin’s (in press) was the first of this pair of papers and it described the MirandaNet NOF data set for this opportunity sample covering 11 of the 47 different providers of ICT teacher training in England for whom quality assurance reports by ICT experts were accessible and for whom there were at least 15 responses from the national survey of teachers. A well-respected multilevel 862 British Journal of Educational Technology Vol 40 No 5 2009 © 2008 The Authors. Journal compilation © 2008 British Educational Communications and Technology Agency. framework to evaluate teacher professional development in general provided by Guskey (2002) laid out five critical levels for the evaluation of professional development in general: (1) participants’ reactions, (2) participants’ learning, (3) organisational support and change, (4) participants’ use of new knowledge and skills, and (5) students’ learning outcomes. Davis et al (in press) confirm that this multilevel evaluation framework also applied to ICT teacher training. This paper uses part of the same data set and digs deeper into the data for the two approaches that were evaluated as the most and least effective. The most effective approach to ICT teacher training may be characterised by its organic approach, so it will be called ‘organic’. The least effective approach may be characterised by its CBT approach, so it will be called CBT. The further analysis of this data followed the methodology outlined by Stake (1995) wherein all available data for the two approaches were used and comparisons were made with the entire data set of the contrasted approach. Both approaches had 15 surveys each and the reports provided to the TTA by ICT experts. The organic approach had two reports, with the second confirming that the design had remained effective with increased number of teachers in training. In contrast, the CBT approach had five reports that signalled improvement of this training design over time. The apparent elements of an ethnographical approach are derived from the detailed observations in the reports by the ICT experts. This further analysis naturally fell into stages that were performed on each approach separately before comparisons were made: (1) extract and confirm a description of the approach and its evaluation across Guskey’s (2002) five levels; (2) identify processes for training the ICT trainers; and (3) analyse and contrast the engagement of the two approaches within three interrelated ecosystems: the teacher’s classroom, the school and the ecosystem involved in the training of ICT trainers. The context First, it is important to set the scene for these two case studies of ICT-related teacher training in England in 2000–03. Following evaluation in 2003, the MirandaNet Fellowship provided a description of the large scale of this national initiative to train around 395 000 teachers in England when schools had ICT and Internet in every school but not in every classroom. There was a related investment of £1.9 billion in the National Grid for Learning, but every teacher did not have access to ICT or broadband Internet connections. Considerable scaling up was required in order for all teachers and librarians in England to receive training as mandated by the government. The focus was planned to be the pedagogy of ICT in the classroom and much of the training was expected to be online. However, the programme had a difficult start. The intention that schools would update their ICT skills before the start of the programme was not realised. The idea that schools would receive their government-funded ICT equipment and broadband connections before they started their training were overambitious (Preston, 2004). Within this national context, we contrast the most and least effective approaches to ICT teacher training, both of which were designed for primary and secondary teachers and librarians. The two designs were significantly different from each other. A nonICT teacher training impacts multiple ecologies 863 © 2008 The Authors. Journal compilation © 2008 British Educational Communications and Technology Agency. parametric test on the teachers’ survey data showed a significant difference (both = 15, Mann–Whitney U test z = -3.713, p < 0.001). The teachers in the organic approach (overall training M = 21.47, or an average strongly agree) reported higher levels of benefits in training than the teachers in CBT approach (overall training M = 9.53, or an average disagree). A summary of the two approaches is presented in Tables 1 and 2 organised across Guskey’s (2002) five levels plus a foundational level of opinion about the approach of ICT teacher training including illustrative evidence from each data source, where found. Overall, the teachers strongly agreed with the organic approach, eg, te