Evaluation of Quality Teaching for English Learners (QTEL) Professional Development. Final Report. NCEE 2012-4005
Author: Bos, Johannes M.
Date: 2012-03-00
Type: Numerical/Quantitative Data
Subjects: Middle School Teachers,English Teachers,Teacher Effectiveness,English Language Learners,Faculty Development,Intervention,Academic Achievement,Standardized Tests,Scores,Middle School Students,Program Effectiveness,Teacher Attitudes,Knowledge Base for Teaching,Teacher Characteristics,Statistical Analysis,Teacher Surveys

To add to the evidence base on effective strategies for teaching English language learner students, the 2006-2011 REL West at WestEd conducted a rigorous study, using an intent-to-treat model, of the impact on middle grades student achievement of offering teachers the opportunity to participate in a teacher professional development program. The program, developed by WestEd, is Quality Teaching for English Learners (QTEL), an approach to improving the teaching of English language learner students at the secondary level. It aligns with the broader democratic goals of equal access and equal opportunity for all students (Walqui and van Lier 2010). QTEL targets the teachers of English language learner students classified as limited English proficient and those reclassified as fluent English proficient and placed in mainstream classrooms. By enhancing the ability of teachers to work with English language learner students, the intervention also seeks to increase the quality of instruction for all other students in the mainstream classroom. This study is a school-level randomized controlled trial to test the effectiveness of offering this intervention, rather than that of participating in it. Teachers eligible for the intervention included those of English language arts and English language development in the schools assigned to receive an invitation to participate in treatment. Teachers in the control group followed the &quot;business as usual&quot; non-QTEL professional development of their schools. These professional development activities, and other implementation contexts, are discussed in chapter 3. From a sample of 52 middle schools, Berkeley Policy Associates randomly selected 26 for the intervention group. Teachers in intervention schools were offered QTEL; teachers in control schools were not. Most teachers who received an invitation to participate in QTEL did not receive the services offered, for a variety of reasons. Still, because this was an intent-to-treat study, outcomes were measured for intervention teachers and their students whether or not the teachers received the services offered. No evidence was found that offering QTEL resulted in statistically significant impacts on the English language arts achievement of any of the four English language learner status subgroups in either grade 7 or grade 8. For teacher characteristics, offering QTEL was estimated to have increased the test scores of grade 8 students who were in schools where more than 43.5 percent of the teachers had an advanced degree. The difference of 10.40 points translated to an effect size of 0.17 standard deviation in these scores. This impact estimate was statistically significant (p = 0.027) and statistically significantly different from the estimated impact on the test scores of grade 8 students in schools with less highly educated teachers (effect size = 0.01, p = 0.167). No other impact estimates related to teacher characteristics were statistically significant or differed significantly across the subgroups. The exploratory analysis of the four subscales of the Program Aligned Classroom Observation (PACO) instrument resulted in one statistically significant positive impact estimate: the intervention-control difference on the &quot;student interaction&quot; subscale translated to an effect size of 0.445 (p = 0.005). This subscale measures the degree of academically oriented student-student interaction within the classroom. The study found no evidence that offering QTEL resulted in statistically significant impacts on any Sheltered Instruction Observation Protocol subscales. The exploratory analysis suggests that offering QTEL might have differential effects on the students of teachers with different levels of education. It also suggests that there might be a positive relationship between offering QTEL and the amount of student-student interaction within classrooms, as measured by an observation instrument aligned with the intervention. These findings point to the need for more research on how offering QTEL interacts with teachers' prior learning and how it affects teacher practice. Appended are: (1) Statistical Power Analysis; (2) Random Assignment; (3) Data Collection Instruments; (4) Unadjusted Means for Primary Student-Level Outcomes and Secondary Teacher-Level Outcomes; (5) Sensitivity Analyses; and (6) Detailed Tables and Discussion of Student Sample. (Contains 10 figures, 61 tables and 53 footnotes.)