Improving Classroom Dialogue Act Recognition from Limited Labeled Data with Self-Supervised Contrastive Learning Classifiers (2023)

Abstract: Recognizing classroom dialogue acts has significant promise for yielding insight into teaching, student learning, and classroom dynamics. However, obtaining K-12 classroom dialogue data with labels is a significant challenge, and therefore, developing data-efficient methods for classroom dialogue act recognition is essential. This work addresses the challenge of classroom dialogue act recognition from limited labeled data using a contrastive learning-based self-supervised approach (SSCon). SSCon uses two independent models that iteratively improve each other’s performance by increasing the accuracy of dialogue act recognition and minimizing the embedding distance between the same dialogue acts. We evaluate the approach on three complementary dialogue act recognition datasets: the TalkMoves dataset (annotated K-12 mathematics lesson transcripts), the DailyDialog dataset (multi-turn daily conversation dialogues), and the Dialogue State Tracking Challenge 2 (DSTC2) dataset (restaurant reservation dialogues). Results indicate that our self-supervised contrastive learning-based model outperforms competitive baseline models when trained with limited examples per dialogue act. Furthermore, SSCon outperforms other few-shot models that require considerably more labeled data.

## introduction dialogue analysis offers significant potential for improving our understanding of classroom learning and teaching by modeling dialogue between students and teachers. studies of classroom dialogue can provide deep insight into how students learn most effectively and engage with each other and with teachers @xcite @xcite . a long-standing goal in analyzing classroom dialogue is to understand how student-student and studentteacher dialogues lead to better student learning outcomes @xcite . this work addresses the problem of dialogue act recognition in k-12 classroom dialogues. dialogue act recognition has garnered considerable attention and is useful for many tasks such as dialogue generation and understanding @xcite @xcite . recent efforts in dialogue act recognition are built on large-scale pre-trained language models @xcite @xcite @xcite . these models demonstrate high performance on standard datasets but require substantial labeled training data and, in some cases, combine other corroborative labels such as sentiment. finding labeled public datasets of k-12 classroom dialogues is challenging for several reasons. first, there are concerns about participants' privacy and security. second, researchers often develop individualized coding schemes specific to their design framework, and research context @xcite @xcite . therefore, even when labeled datasets are available, the assortment of coding schemes makes it challenging to cross-train across datasets. third, classroom dialogue utterances are usually specific to a given subject matter, making generic labeled dialogue datasets less useful as auxiliary data. it is, therefore, essential to develop the capability to build dialogue act recognition models from limited labeled training data. our research addresses the lack of large-scale labeled classroom dialogue datasets by using a self-supervised contrastive learning-based model (sscon) trained using limited labeled data. ss-con uses contrastive learning to transform the dialogue utterance representation into a new embedding space where identical dialogue acts cluster together, and distinct dialogue acts are pushed further apart. the system iteratively improves per-formance, as the contrastive learning step benefits self-supervision, even when presented with limited labeled data. experiments show that sscon outperforms competitive baselines with just tens of labeled examples per dialog act in both a k-12 mathematics classroom dataset and an everyday conversation dataset, dailydialog. our key contributions are the following: ## results the first dataset we consider is the talkmoves dataset. to evaluate the models in a few-shot learning scenario, we experiment using 10-70 labeled instances per dialogue act type (less than 1% of the overall data). the second dataset we consider is the dailydialog dataset with four possible dialogue acts. like the talkmoves dataset, we consider 10 to 70 labeled examples per class. we also compare our results on the dstc2 dataset on restaurant inquiries. this dataset differs from the other two as each utterance can have multiple dialogue act labels. sscon performs better on the dstc2 dataset than the baseline self-supervising student-teacher model tod-bert-st @xcite ) by 3 to 10%, depending on the size of the labeled training dataset. the tod-bert-st model uses a data augmentation approach, while we use a contrastive learning approach. both these approaches are complementary, and future work should explore using them together. 5 sscon component analysis ## conclusion classroom dialogue analysis can yield significant insight into student learning. however, collecting and coding classroom dialogue datasets is very labor-intensive. to address this problem, we introduce a novel self-supervised contrastive learning approach that can automate a portion of this process and make it more efficient even when limited labeled is available. we show that our approach improves on other methods that work on limited labeled datasets. the results also show that our approach can match and exceed the performance of some models trained on the fully labeled dataset.