low range) describes few instances of the behavioral markers; a score of 3, 4 or 5 (the mid-range) describes intermittent or mixed instances of the behavioral markers; and a score of 6 or 7 (the high range) describes frequent and consistent instances of the behavioral markers (table 4). Quality and depth of indicators is also taken into account when judging score.

|**Low**|**Range**|**Table**|**4: CLASS Lik**<br>**Mid Range**|**ert Scale 1–7**<br>|**High**|**Range**|
|---|---|---|---|---|---|---|
|1|2|3|4|5|6|7|
|The low-range<br>description fits<br>the classroom/<br>teacher very<br>well. All, or<br>almost all,<br>relevant<br>indicators in<br>the low range<br>are present.|The low-<br>range<br>description<br>mostly fits the<br>classroom/<br>teacher but<br>there are one<br>or two<br>indicators that<br>are in the mid<br>range.|The mid-<br>range<br>description<br>mostly fits the<br>classroom/<br>teacher but<br>there are one<br>or two<br>indicators in<br>the low range.|The mid-range<br>description fits<br>the classroom/<br>teacher very<br>well. All, or<br>almost all,<br>relevant<br>indicators in<br>the mid-range<br>are present.|The mid-range<br>description<br>mostly fits the<br>classroom/<br>teacher but there<br>are one or two<br>indicators in the<br>high range.|The high-range<br>description<br>mostly fits the<br>classroom/<br>teacher but<br>there are one or<br>two indicators<br>in the mid<br>range.|The high-range<br>description fits the<br>classroom/teacher<br>very well. All, or<br>almost all, relevant<br>indicators in the<br>high range are<br>present.|

Source: Upper Elementary CLASS Manual, page 12. Used with permission of Teachstone.

Raters documented evidence of each indicator as they occurred within a 15–20-minute period, and then rated the frequency of the observed behaviors as in the low, mid, or high range. The behavioral markers are intended to capture overall quality of the interaction and are not intended to serve as a checklist. The notes corresponding to each score in table 4 are not meant to be comprehensive but rather abbreviated guides around which a rater was allowed to recall the frequency of such instances. Finally, dimension scores within specific domains were averaged to come to an average score for Emotional Support, Classroom Organization, and Instructional Support, respectively. The only dimension for which this is not the case is Negative Climate, where lower scores are better, as they indicate an absence of negativity.<sup>5</sup> For the purposes of calculating the Classroom Organization domain score, the Negative Climate score is reversed by subtracting it from 8, allowing the dimension to be averaged with others in the domain.

# **2.4 Inter-rater reliability**

This section presents the results of a series of analyses to gauge whether the raters’ results were reliable. These results are then compared against other CLASS studies, identifying strengths and weaknesses in the results from the pilot counties. The pilot study had 12 raters, forming 13 different pairs, making a total of 302 individual observations over the course of the study. Each rater had received and passed CLASS training 18 weeks before rating the observed teachers.

Three comparator studies, which took place in a U.S. context, are presented in table 5 to provide a reference for other implementations of the CLASS tool in upper elementary and junior secondary.

> 5 For more guidance on computing CLASS scores, see pages 17-19, K-3 CLASS Manual.

8