and less than 2 percent received a score of less than 5. Similarly, with Student Engagement, 72 percent received a 6 or higher and less than 10 percent received a score of less than 5. With such little variation, there would be very little possibility of capturing measurable changes in these domains.

More variation appears when examining by specific dimensions. While a few showed little variation (e.g. Negative Climate, Productivity, and Behavior Management), others had a good deal of variation with even distributions (e.g. Quality of Feedback, Emotional Support, and Teacher Sensitivity). There were also scores that were particularly low and where measurable improvements could emerge through training activities (e.g. Instructional Dialogue, Analysis and Inquiry, and Regard for Student Perspectives).

The most interesting comparisons are weaker domains and dimensions, and the international comparisons. Observed teachers received higher scores in Classroom Organization, and lower scores in Emotional Support and Instructional Support. This is comparable with other CLASS assessments in the United States. Elementary-school teachers scored marginally higher than middle-school teachers in areas of Emotional Support, while teachers with more experience tended to have higher scores in Instructional Support.

On interregional comparisons, the three pilot counties did not exhibit significant differences in scores. This may be because the differences between the counties were not large enough—a comparison with teachers in the best schools in Guangzhou or Shanghai might reveal larger differences. A similar result occurred on the urban/rural distinction: given the overall level of development of the pilot counties, the distinctions between the two types of schools were not significant in affecting scores.

There were minor differences based on teacher type, although it was not as clear cut as saying that backbone teachers always outperformed new teachers on all dimensions.

The pilot study showed that the CLASS tool could be used in the context of the three pilot counties. The exploration of differences between teacher types, as well as by county and training, did not reveal as much difference as originally posited. To further explore these differences, a larger sample size would be necessary.

# **4.2 Contextualizing results through participant feedback**

In June 2017, a workshop was held to discuss these findings and gather input from participants on their experiences of using CLASS and their perception of the tool’s applicability. The participants included coders, teachers who were observed, Department of Education officials, and university personnel. There was strong agreement on the usefulness and general applicability of CLASS, but issues were raised on the challenge of coding specific dimensions. Results of the coding indicated that there were greater differences in scoring among pairs of coders for Teacher Sensitivity, Regard for Student Perspectives, Instructional Dialogue, and Analysis and Inquiry. Participants indicated that such discrepancies could in part be reduced through more intensive training specifically on these dimensions to develop a better understanding of the indicators. But some participants indicated that cultural aspects were an important factor in coding discrepancies for Teacher Sensitivity and Regard for Student Perspectives. They suggested that these dimensions should be revisited to consider how the instructions for coding might be modified to be more applicable to the Guangdong context. Overall, the participants in the workshop felt that the pilot study provided useful information on teacher strengths and areas for improvement.

They also felt that the lessons learned from the pilot study could also be used to improve the existing system for delivering teacher training. While continuing to develop and deliver teacher training as it has always done, the government could use the results of classroom observation as a results measure under a resultsbased financing mechanism. Doing so would give the government a consistent yardstick with which to provide content developers and service providers (often the same university) an added incentive of receiving a bonus based on the effectiveness of their training programs. Effectiveness could be gauged by improvements in teacher knowledge (as measured by pre- and post-training assessments) and by changes 20