Can you Summarize my learnings? Towards Perspective-based Educational Dialogue Summarization (2023)

Abstract: The steady increase in the utilization of Virtual Tutors (VT) over recent years has allowed for a more efficient, personalized, and interactive AI-based learning experiences. A vital aspect in these educational chatbots is summarizing the conversations between the VT and the students, as it is critical in consolidating learning points and monitoring progress. However, the approach to summarization should be tailored according to the perspective. Summarization from the VTs perspective should emphasize on its teaching efficiency and potential improvements. Conversely, student-oriented summaries should distill learning points, track progress, and suggest scope for improvements. Based on this hypothesis, in this work, we propose a new task of Multi-modal Perspective based Dialogue Summarization (MM-PerSumm), demonstrated in an educational setting. Towards this aim, we introduce a novel dataset, CIMA-Summ that summarizes educational dialogues from three unique perspectives: the Student, the Tutor, and a Generic viewpoint. In addition, we propose an Image and Perspective-guided Dialogue Summarization (IP-Summ) model which is a Seq2Seq language model incorporating (i) multi-modal learning from images and (ii) a perspective-based encoder that constructs a dialogue graph capturing the intentions and actions of both the VT and the student, enabling the summarization of a dialogue from diverse perspectives. Lastly, we conduct detailed analyses of our model’s performance, highlighting the aspects that could lead to optimal modeling of IP-Summ.

## introduction artificial intelligence (ai) is making significant strides in the field of education, paving the way for numerous innovative applications. the power of natural language processing (nlp) and advanced machine learning techniques have given rise to virtual tutors (vts) @xcite figure 1 : sample instance of mm-persumm task 2022), which, unlike traditional methods, are accessible to anyone, anywhere, anytime foot_0 . vts are revolutionizing the education sector, breaching the barriers of geographical boundaries, and democratizing learning on a global scale. they offer a tailored learning approach, adapting to each learner's pace, aptitude, and preferences, thereby catering to a diverse array of learning styles. while the implementation of ai in education are evident, extracting valuable insights from these virtual educational interactions remains a challenging task. summarizing insights from vt carries significant implications for both the improvement of aibased educational systems and the learning outcome of students. with a growing reliance on these ai-powered tools for teaching and learning, it becomes crucial to gauge their teaching efficiency, highlight potential areas of improvement, and assess their overall impact on learners' progress. for learners, summaries provide a distillation of key learning points, tracking their learning journey and suggesting areas for future study, which could be pivotal in informing their learning path. while dialogue summarization techniques @xcite have seen extensive development in several domains, their application to educational interactions, particularly with vts, has not been thor-oughly explored. furthermore, the importance of summarizing these dialogues from multiple perspectives -those of the tutor, the student, and a neutral observer -is a distinct requirement in this domain @xcite . these perspectives can reveal diverse, valuable insights that can contribute to the evolution of virtual tutoring systems and the individual learning journey of students. thus, this area holds vast potential for exploration and advancement in the field of ai-driven education. to address this significant gap in the field, the contributions of this paper are : (i) propose a new task of multi-modal perspective based dialogue summarization (mm-persumm) which is demonstrated in an educational setting; (ii) as a step towards this goal, we extend an existing education based conversational dataset, cima @xcite by summarizing the dialogues from three different perspectives: that of the student, the tutor, and a generic viewpoint to create cima-summ which serves as the foundation for our research; ## related works in this section, we detail some relevant works in the context of educational-nlp and dialogue summarization in general. education and ai. with the advent of ai, tailored educational materials can be generated based on individual learning styles and abilities @xcite . automated feedback systems offer timely evaluations of assignments and homework @xcite . educational question generation is another area that has witnessed significant advancements @xcite . in the context of virtual tutoring, advancements in nlg have paved the way for interactive and immersive learning experiences. this led to the creation of ai-related datasets, such as cima @xcite and the teacher-student cha-troom corpus @xcite . recently, a lot of attempts have been made to create a vt system utilizing these datasets @xcite . dialogue understanding. in the context of the social good theme, there have been an upsurge of research in the latest time focused on understanding dialogues. for example works in mental health conversations @xcite , disease diagnosis assistant @xcite , education @xcite etc. a broad spectrum of research has been done on dialogue summarization spanning different domains. one of the notable sectors includes meeting summarization, where advancements have been observed in both extractive and abstractive techniques @xcite . similarly, chat summarization has been a focus due to the explosion of messaging apps and business communication platforms @xcite . email thread summarization @xcite is another domain that has seen significant research interest. the complexity of the task lies in identifying the salient information buried in long and often nested conversations @xcite . the domain of customer service summarization @xcite has a more specific goal, which is to extract customer issues. finally, medical dialogue summarization @xcite @xcite has seen substantial development, driven by the need for concise patient-doctor conversation records. ## dataset to facilitate research in educational dialogue summarization, we create the cima-summ dataset, which is based on an existing conversational dataset for educational tutoring. ## proposed methodology in this section, we discuss the problem statement and proposed methodology in detail. problem description. our proposed task multimodal perspective based dialogue summarization (mm-persumm) is defined as follows : input : the input comprises of three entries: (1) the source dialogue output : the output is a natural language sequence, y , representing the summarized dialogue from the perspective, p . given an instance of d, i, and p , y can be manifested as a diverse summary conditioned on the perspective and image. ## experiments and results this section details the experimental setup followed by results and analysis of ip-summ. ## conclusion the rapid adoption of vts has brought forth novel challenges and opportunities in enhancing the learning experience amongst which dialogue summarization stands out as a pivotal area warranting dedicated attention. our work addresses this crucial aspect by proposing a new task of multi-modal perspective-based dialogue summarization (mm-persumm) in an educational setting, which paves the way for a holistic understanding of the vtstudent exchanges. we introduce a novel dataset, cima-summ, that features dialogue summaries from diverse perspectives, offering a multi-faceted view of the learning interaction. we further propose a model, image and perspective-guided dialogue summarization (ip-summ), which effectively incorporates both image context and dialogue perspectives into the summarization process.