Edu- C onvo K it: An Open-Source Library for Education Conversation Data (2024)

Abstract: We introduce Edu-ConvoKit, an open-source library designed to handle pre-processing, annotation and analysis of conversation data in education. Resources for analyzing education conversation data are scarce, making the research challenging to perform and therefore hard to access. We address these challenges with Edu-ConvoKit. Edu-ConvoKit is open-source [1], pip-installable [2], with comprehensive documentation [3]. Our demo video is available at: https://youtu.be/zdcI839vAko?si=h9qlnl76ucSuXb8-. We include additional resources, such as Colab applications of Edu-ConvoKit to three diverse education datasets [4] and a repository of Edu-ConvoKit-related papers [5].[1] https://github.com/stanfordnlp/edu-convokit[2] https://pypi.org/project/edu-convokit/[3] https://edu-convokit.readthedocs.io/en/latest/[4] https://github.com/stanfordnlp/edu-convokit?tab=readme-ov-file#datasets-with-edu-convokit[5] https://github.com/stanfordnlp/edu-convokit/blob/main/papers.md

## introduction language is central to educational interactions, ranging from classroom instruction to tutoring sessions to peer discussions. it offers rich insights into the teaching and learning process that go beyond the current, oversimplified view of relying on standardized test outcomes @xcite @xcite . the landscape of natural language processing (nlp) and education is rapidly evolving, with an increase of open-sourced education conversation datasets (e.g., from @xcite al. (2020) challenges and consequences. while the interest in this interdisciplinary field is growing, our conversations with education data science and nlp researchers both in academia and industry have surfaced several challenges that hinder research progress. first, there is no centralized tool or resource that assists in analyzing education data, or helps researchers understand different tradeoffs in methods. for example, researchers expressed uncertainty about pre-processing the data, such as "the best way to anonymize the data to protect the privacy of students and teachers". they also wanted an "easily accessible collection of language tools and models that can detect insightful things." the lack of these tools and resources makes the research harder to conduct. second, there is a high learning curve for performing computational analyses. for example, many education researchers are trained in qualitative research; even though they want to use computational tools for quantitative analyses at scale, they often do not know how to start or have the readily available compute to try out the tools. ## design principles edu-convokit follows these principles: i. minimalistic data structure. the system transforms all data inputs (e.g., csv and json files) into a dataframe. edu-convokit only needs the speaker and text columns to be uniquely identifiable, which is the case in the datasets we surveyed and applied edu-convokit to. ii. efficient execution. the system should be able to run on a cpu and support large-scale pre-processing, annotation and analysis. ## additional resources: basic tutorials, case studies, and paper repository we create a suite of introductory tutorials and case studies of edu-convokit as colab notebooks (link). to demonstrate its wide applicability and generalizable design structure, we apply edu-convokit to three different education transcript datasets developed by different authors: ncte, an elementary school classroom dataset @xcite ; talkmoves, a k-12 classroom dataset @xcite ; and amber, a one-on-one 8th-9th grade tutoring dataset @xcite . for space reasons, we omit the findings of the case studies in this paper, but they can be found in our github repository. to centralize research efforts, we additionally contribute a paper repository that include papers that have used edu-convokit or have features incorporated into edu-convokit (link). ## conclusion we introduce edu-convokit, an open-source library designed to democratize and enhance the study of education conversation data. implemented in python and easily accessible via github and pip installation, it offers a user-friendly interface complete with extensive documentation, tutorials, applications to three diverse education datasets, and paper repository resource. based on extensive research experience, it incorporates best practices for pre-processing data and a series of different annotation measures grounded in prior literature, such as measuring student reasoning and talk time. it additionally supports several analysis modules, such as temporal analyses (e.g., talk time ratios), lexical analyses (e.g., word usage) and gpt-powered analyses (e.g., summarization). fostering a collaborative environment through its open-source nature, edu-convokit and its resources unify research efforts in this exciting interdisciplinary field to improve teaching and learning.