C o M u MDR : Code-mixed Multi-modal Multi-domain corpus for Discourse pa R sing in conversations (2025)

Abstract: Discourse parsing is an important task useful for NLU applications such as summarization, machine comprehension, and emotion recognition. The current discourse parsing datasets based on conversations consists of written English dialogues restricted to a single domain. In this resource paper, we introduce CoMuMDR: Code-mixed Multi-modal Multi-domain corpus for Discourse paRsing in conversations. The corpus (code-mixed in Hindi and English) has both audio and transcribed text and is annotated with nine discourse relations. We experiment with various SoTA baseline models; the poor performance of SoTA models highlights the challenges of multi-domain code-mixed corpus, pointing towards the need for developing better models for such realistic settings.

## introduction discourse structures @xcite capture relationships between clauses (e.g., causality, contrast, elaboration, and temporal sequencing) and are crucial to understanding the logical flow of information. these have been utilized in various tasks such as text summarization @xcite , language understanding, machine reading comprehension @xcite , dialog generation @xcite @xcite and emotion recognition @xcite . researchers have created annotated discourse corpora from humanto-human dialogues for a single language such as english (e.g., stac @xcite and molweni @xcite ). however, many modern-day conversations are audio-based and often involve code-mixing of multiple languages, such as hindi and english (hinglish). understanding the discourse structure in such code-mixed audio-based conversations would be interesting. in this paper, qacp ack continuation dc dc figure 1 : an example of a two-party call-center conversation (in code-mixed language) regarding a complaint on product return. utterances corresponding to the customer center representative are shown in green boxes, and those of the customer are in purple boxes. here, the relation types of question answer complaint pair and acknowledgment are shortened to qacp and ack, respectively. here, dc is the correction by the annotator on incorrect diarization. ## related work discourse parsing has been an active research area in the nlp community @xcite . discourse parsing consists of three main components: discourse segmentation @xcite @xcite , discourse link prediction and discourse relation classification. discourse segmentation divides a text corpus into elementary discourse units (edus) for further processing. discourse link prediction predicts a directed link between two edus, and relation classification assigns a relation type to the link (also check discourse theories in app. a.1). datasets: in the context of english, two main text-0 5 10 15 20 25 % of relations question-answer complaint pair correction conditional contrast background continuation acknowledgment question_answer_pair question_extension figure 2: distribution of discourse labels in comumdr. based corpora have been proposed for discourse parsing: stac (asher et al., 2016) and molweni (li et al., 2020) (check details in app. a.2). ## comumdr creation comumdr consists of two-party customer call center interactions. we obtained the data via a joint research collaboration with a call center company (they own the data) and want to automate customer call understanding. the calls mainly cater to indian customers and companies. we ensure that the privacy of customers and companies mentioned in a call is maintained during annotation. the audio data is transcribed using the existing asr (automatic speech recognition) system @xcite (details in app. b.1) and diarized into utterances (koluguri et al., 2021) (details in app. b.2). subsequently, the data is anonymized for customer names and other private information. a team of three professional annotators further annotated the transcribed and diarized data. discourse labels: we used nine discourse labels: acknowledgment, question-answer pair, question-answer complaint pair, background, contrast, correction, question extension, conditional, and continuation. app. annotation details: annotators were tasked to identify the elementary discourse units (edus), predict links between them to form a dag, and assign a label to the relation between edus. a team of two annotators independently annotated the data, and another annotator verified the annotations and marked batches for re-annotation if deemed necessary. previous studies have shown that identifying complex discourse units (cdus) foot_0 is a challenging task and have relied on combining edus using discourse relations to get a cdu @xcite . prior discourse parsing models have used various strategies to convert cdus to edus for efficient parsing @xcite . we identified similar challenges in annotation and hence, instructed the annotators to connect an edu with only the head (i.e., the first edu) of a cdu @xcite . appendix c presents more details regarding the annotation. we used the inception software @xcite for annotation ( §c.1). we provide details of annotators, instructions, and processes in the app. c.2, app. c.3, and app. c.4, respectively. inter-annotator agreement: ## experiments, results and analysis discourse modeling: a dialogue consists of a list of utterances between two speakers. the utterances are further divided into elementary discourse units (i.e., clauses @xcite ) {u 0 , u 1 , ..., u n }, where u 0 is a dummy root edu. discourse parsing involves predicting a directed link between two edus u j and u i and assigning a relation label r ji between edus u j and u i . ## future directions and conclusion this paper presents comumdr, a new discourse corpus for multi-modal, multi-domain, and codemixed conversations from various customer call centers. we transcribed the audio and diarized the text into utterances. we annotated the edus using nine discourse labels by combining a few closely related labels from the sdrt format as they formed a more appropriate flow of discourse in a two-party conversation on customer support calls. in this work, we experimented with sota models; however, these do not perform well on comumdr. in the future, we plan to develop more advanced models incorporating audio modality information.