The NCTE Transcripts: A Dataset of Elementary Math Classroom Transcripts (2023)

Abstract: Classroom discourse is a core medium of instruction analyzing it can provide a window into teaching and learning as well as driving the development of new tools for improving instruction. We introduce the largest dataset of mathematics classroom transcripts available to researchers, and demonstrate how this data can help improve instruction. The dataset consists of 1,660 45-60 minute long 4th and 5th grade elementary mathematics observations collected by the National Center for Teacher Effectiveness (NCTE) between 2010-2013. The anonymized transcripts represent data from 317 teachers across 4 school districts that serve largely historically marginalized students. The transcripts come with rich metadata, including turn-level annotations for dialogic discourse moves, classroom observation scores, demographic information, survey responses and student test scores. We demonstrate that our natural language processing model, trained on our turn-level annotations, can learn to identify dialogic discourse moves and these moves are correlated with better classroom observation scores and learning outcomes. This dataset opens up several possibilities for researchers, educators and policymakers to learn about and improve K-12 instruction. The dataset can be found at https://github.com/ddemszky/classroom-transcript-analysis .

## introduction improving k-12 mathematics instruction in the aftermath of the covid-12 pandemic is a major national priority, drawing support from both the u.s. government (u.s. department of education) foot_0 ) and major foundations (e.g., gates, spencer). 2 a key step in this direction is to measure and facilitate the use of effective mathematics teaching practices, an effort that draws on a long history of research (e.g., @xcite . instructional measurement has traditionally relied on resourceintensive classroom observation. recent natural language processing models, trained on manually scored classroom transcripts, enable measuring effective instructional practices in scalable and adaptable ways @xcite @xcite @xcite . however, a common barrier to evaluating such measures is the lack of comprehensive data sources that link classroom transcripts to external variables, such as student and teacher demographics and learning outcomes. to address this, we introduce a dataset of 1,660 u.s. 4th and 5th grade elementary math classroom transcripts collected by the national center for teacher effectiveness (ncte) between 2010-2013 @xcite . the anonymized transcripts represent data from 317 teachers across 4 school districts serving largely historically marginalized student populations. the transcripts are associated with a wide range of metadata: (i) turn-level annotations for discourse moves, (ii) classroom observation scores, (iii) questionnaires that capture teacher background, beliefs and classroom practices, (iv) student administrative data, (v) questionnaires describing student background and classroom experiences, (vi) value added scores, which estimate teachers' contribution to students' academic performance. to our knowledge, this is the largest dataset of math classroom transcripts with linked outcomes available to researchers. to illustrate how this data can be used to identify effective instructional practices, we build classifiers for discourse moves and validate these measures by correlating them with instructional outcomes. these discourse moves include on vs off task instruction, teachers' uptake of student ideas @xcite , teachers' focusing questions @xcite and student reasoning -the latter three of which are indicators of dialogic instruction, where students are active participants of the learning process @xcite @xcite . we show that a roberta classifier can learn to predict these discourse moves with moderate to high accuracy, leaving some room for improvement for future work. importantly, we find that predictions for all of these discourse moves correlate significantly with classroom observation scores that measure instructional quality, teacher sensitivity, and classroom climate, among other items, while controlling for teacher and classroom covariates. predictions for dialogic moves (teacher uptake, focusing questions and student reasoning) also show a significant positive correlation with teachers' value added scores. taken together, these results demonstrate the value of this dataset for developing measures that can help us understand and facilitate effective instruction, for example, by powering automated feedback tools for teachers @xcite . ## related work we provide an overview of related corpora and methods pertaining to the computational analysis of classroom discourse. ## dataset description the dataset consists of 1,660 anonymized transcripts of whole lessons, collected as part of the national center for teacher effectiveness (ncte) main study @xcite . 3 the observations took place between 2010-2013 in 4th and 5th grade elementary math classrooms across four districts serving largely historically marginalized students. in the first two years, teachers and students were assigned to each classroom according to their school's usual procedure for forming classes. in the third year, the ncte project team randomly assigned teachers to rosters of students provided by the school. ## computational analysis we illustrate how the ncte transcript dataset can serve as a valuable resource for developing computational measures of classroom discourse. first, we train models that automatically identify dialogic discourse moves by leveraging the turn-level annotations. we then study how these discourse moves correlate with observation scores and teachers' value added scores. ## discussion we find that our measures of discourse moves, which we identified by consulting research on mathematics instruction, correlate with human raters' perceptions of lesson quality, and with students' learning outcomes. these results are significant in multiple ways. ## conclusion we introduce a dataset of elementary math classroom transcripts, associated with a rich source of linked variables. we train classifiers on turn-level annotations to predict discourse moves and show that predictions for these moves correlate significantly with observation scores and value added scores. these results demonstrate how the ncte dataset can serve as a valuable resource for understanding classroom interactions and for powering tools that seek to facilitate instruction.