Abstract: Classroom discussions in English Language Arts have a positive effect on students' reading, writing and reasoning skills. Although prior work has largely focused on teacher talk and student-teacher interactions, we focus on three theoretically-motivated aspects of high-quality student talk: argumentation, specificity, and knowledge domain. We introduce an annotation scheme, then show that the scheme can be used to produce reliable annotations and that the annotations are predictive of discussion quality. We also highlight opportunities provided by our scheme for education and natural language processing research.

Introduction: Current research, theory, and policy surrounding K-12 instruction in the United States highlight the role of student-centered disciplinary discussions (i.e. discussions related to a specific academic discipline or school subject such as physics or English Language Arts) in instructional quality and student learning opportunities BIBREF0 , BIBREF1 . Such student-centered discussions – often called “dialogic" or “inquiry-based” – are widely viewed as the most effective instructional approach for disciplinary understanding, problem-solving, and literacy BIBREF2 , BIBREF3 , BIBREF4 . In English Language Arts (ELA) classrooms, student-centered discussions about literature have a positive impact on the development of students' reasoning, writing, and reading skills BIBREF5 , BIBREF6 . However, most studies have focused on the role of teachers and their talk BIBREF7 , BIBREF2 , BIBREF8 rather than on the aspects of student talk that contribute to discussion quality. Additionally, studies of student-centered discussions rarely use the same coding schemes, making it difficult to generalize across studies BIBREF2 , BIBREF9 . This limitation is partly due to the time-intensive work required to analyze discourse data through qualitative methods such as ethnography and discourse analysis. Thus, qualitative case studies have generated compelling theories about the specific features of student talk that lead to high-quality discussions, but few findings can be generalized and leveraged to influence instructional improvements across ELA classrooms. As a first step towards developing an automated system for detecting the features of student talk that lead to high quality discussions, we propose a new annotation scheme for student talk during ELA “text-based" discussions - that is, discussions that center on a text or piece of literature (e.g., book, play, or speech). The annotation scheme was developed to capture three aspects of classroom talk that are theorized in the literature as important to discussion quality and learning opportunities: argumentation (the process of systematically reasoning in support of an idea), specificity (the quality of belonging or relating uniquely to a particular subject), and knowledge domain (area of expertise represented in the content of the talk). We demonstrate the reliability and validity of our scheme via an annotation study of five transcripts of classroom discussion.

Question: what were the baselines?
Answer: 2008 Punyakanok et al. 
2009 Zhao et al. + ME 
2008 Toutanova et al. 
2010 Bjorkelund et al.  
2015 FitzGerald et al. 
2015 Zhou and Xu 
2016 Roth and Lapata 
2017 He et al. 
2017 Marcheggiani et al.
2017 Marcheggiani and Titov 
2018 Tan et al. 
2018 He et al. 
2018 Strubell et al. 
2018 Cai et al. 
2018 He et al. 
2018 Li et al.