“Mistakes Help Us Grow”: Facilitating and Evaluating Growth Mindset Supportive Language in Classrooms (2023)

Abstract: Teachers’ growth mindset supportive language (GMSL)—rhetoric emphasizing that one’s skills can be improved over time—has been shown to significantly reduce disparities in academic achievement and enhance students’ learning outcomes. Although teachers espouse growth mindset principles, most find it difficult to adopt GMSL in their practice due the lack of effective coaching in this area. We explore whether large language models (LLMs) can provide automated, personalized coaching to support teachers’ use of GMSL. We establish an effective coaching tool to reframe unsupportive utterances to GMSL by developing (i) a parallel dataset containing GMSL-trained teacher reframings of unsupportive statements with an accompanying annotation guide, (ii) a GMSL prompt framework to revise teachers’ unsupportive language, and (iii) an evaluation framework grounded in psychological theory for evaluating GMSL with the help of students and teachers. We conduct a large-scale evaluation involving 174 teachers and 1,006 students, finding that both teachers and students perceive GMSL-trained teacher and model reframings as more effective in fostering a growth mindset and promoting challenge-seeking behavior, among other benefits. We also find that model-generated reframings outperform those from the GMSL-trained teachers. These results show promise for harnessing LLMs to provide automated GMSL feedback for teachers and, more broadly, LLMs’ potentiality for supporting students’ learning in the classroom. Our findings also demonstrate the benefit of large-scale human evaluations when applying LLMs in educational domains.

## introduction growth mindset supportive language (gmsl) refers to rhetoric highlighting the potential for individuals to develop their skills and abilities over time. research on gmsl has shown positive effects on crucial aspects of student learning, such as resilience, motivation, and performance @xcite . this language also helps reduce educational gaps linked to race, ethnicity, and social class @xcite . despite many teachers embracing the importance of gmsl, most find it difficult to implement @xcite due to misconceptions as to what constitutes a growth mindset and the lack of effective coaching in this domain. prior work has shown that many well-intentioned teachers often believe they are using gsml (for example, through simply adding more positive expressions-"i believe in you"), without comprehensively understanding the tenets of gmsl needed to apply it in practice @xcite . helping teachers adopt growth mindset principles requires coaching and individualized feedback that is currently not available at scale @xcite . recent advancements in large language models (llms) have led to unparalleled abilities for models to follow instructions, which suggests potential for prompting them with growth mindset principles to generate suggestions for teachers. although unexplored with gmsl, llms have shown success as automated educational tools in other settings. for example, khanmigo (khan @xcite , an llm-assisted tutoring bot was successfully prompt-engineered to follow principles of socratic questioning and not provide the answer to students directly. at the same time, recent work has also shown that chatgpt -a model from the same family as khanmigo -was not successful at providing teachers with high-level, insightful pedagogical feedback @xcite . thus, the question whether llms are capable of generating gmsl for teachers remains an open question. in this paper, we seek to assess the ability of gpt-4, a state-of-the-art llm, to generate gmsl. in doing so, we make several contributions. first, working with teachers trained in growth mindset principles (henceforth referred to as "expert teachers"), we develop an annotation framework to guide teachers' understanding of gmsl. leveraging a dataset of elementary math classroom transcripts @xcite , we further collaborate with these teachers to create an open-source parallel dataset that pairs 100 unsupportive utterances from the transcripts with their expert-revised gmsl reframings. having outlined the tenets of gmsl, we prompt engineer @xcite to generate gmsl reframings of the unsupportive utterances. we utilize recent advancements in prompting and self-critique @xcite techniques to design a detailed, gmsl-specific prompt. via our prompt design, we ensure that the reframings are personalized for each specific classroom instance and adhere to our gmsl tenets. critically, we also propose a novel framework for evaluating gmsl, assessing teacher and model responses by recruiting 174 teachers and 1,006 students (13-15 years old) to evaluate the perceived impact that these responses would have on students. our participants are given excerpts from a conversation between a teacher and student and asked to rate the teachers on their (1) perceived growth mindset, (2) promotion of challengetaking behavior, (3) alleviation of students' feelings of shame, and (4) increase of respect felt by students. some excerpts contain the original teacher responses whereas others contain reframings (either by the expert teachers or model). we adapt these questions from instruments developed and tested by growth mindset experts @xcite . in our evaluation, we find that both expert teachers' and model reframings better support growth mindset, outperforming the original utterances across all metrics. we discover that model reframings slightly surpass even the expert teachers' reframings and that the model with self-critique performs best. we also conduct a lexical analysis, identifying that all reframings include more explicit growth mindset supportive language than the original (e.g. empathetic validation, positioning oneself as a collaborative resource, autonomy supportive language), with models including even more specific diction compared to teacher utterances. our findings not only highlight the need for encouraging gmsl in classrooms but also offer valuable insight into the role llms can provide in this endeavor. furthermore, our results highlight the importance of extensive student-centered evaluations when examining the usefulness of llms in educational contexts, offering a framework for future studies in this area. we demonstrate that through closely working with students and teachers when developing and evaluating llm-based technologies, we can help ensure their positive influence within the classroom. ## modeling gmsl we utilize a dataset of elementary math classroom transcripts @xcite and gpt-4 prompting techniques to create four different types of responses: the subsequent sections detail the development of these four types of responses. ## evaluation we develop a framework, grounded in psychological theory, for evaluating gmsl in generated responses with the help of teachers and students (section 4.1). in addition, we also conduct a lexical analysis to understand linguistic differences among the different types of responses (section 4.2). ## discussion & conclusion in this study, we take steps towards helping teachers adopt growth mindset supportive language (gmsl) in their classroom. we establish an annotation guide, create a parallel dataset, develop an automated coaching tool utilizing gpt-4, and propose a framework grounded in psychological theory for evaluating gmsl's effectiveness for both teachers and students. our findings demonstrate that expert-teacher and model-generated reframings significantly improved upon the original unsupportive utterances in promoting a growth mindset, encouraging challenge-seeking behavior, alleviating feelings of shame, and enhancing the perceived respect by students. interestingly, model reframings surpassed expert-teacher reframings, highlighting the potential of llms in fostering gmsl adoption within educational contexts. our work serves as a foundation for eliciting and evaluating the use of gmsl by teachers and llms through collaboration with students and teachers. the insights derived from our analysis open avenues for further inquiry into the role llms can play in the future to support teachers' communication strategies and facilitate better student-teacher interactions in the classroom. future research could build upon this foundation to explore new dimensions of gmsl, examine the long-term impact of gmsl coaching, and extend the application of llms alongside our psychologically-grounded evaluation framework to address other outstanding challenges across various educational domains.