Multi-level Contrastive Learning for Script-based Character Understanding (2023)

Abstract: In this work, we tackle the scenario of understanding characters in scripts, which aims to learn the characters’ personalities and identities from their utterances. We begin by analyzing several challenges in this scenario, and then propose a multi-level contrastive learning framework to capture characters’ global information in a fine-grained manner. To validate the proposed framework, we conduct extensive experiments on three character understanding sub-tasks by comparing with strong pre-trained language models, including SpanBERT, Longformer, BigBird and ChatGPT-3.5. Experimental results demonstrate that our method improves the performances by a considerable margin. Through further in-depth analysis, we show the effectiveness of our method in addressing the challenges and provide more hints on the scenario of character understanding. We will open-source our work in this URL.

## introduction as one essential element in stories, character comprehension is a popular research topic in literary, psychological and educational research @xcite @xcite . to fully understand characters, individuals must empathize with characters based on personal experiences @xcite , construct profiles according to characters' identities, and inference about characters' future actions @xcite . according to the data modality and format, character comprehension can be categorized into several classes @xcite . in this work, we focus on character understanding in scripts @xcite . scripts are written text for plays, movies, or broadcasts @xcite . typically, scripts are often structured with several text fields, including scene description, conversation, transition and summary @xcite . character sheldon jennifer story title tbbt the test dataset tvshowguess rocstories text length 528832 41 character's related text sheldon: " ... we take on koothrappaliand his dog. really give ourselves a challenge." jennifer has a big exam tomorrow. ... jennifer felt bittersweet about it. comparison between a script from tvshowguess @xcite and a narrative from rocstories @xcite . although pre-trained language models (plms) have demonstrated their effectiveness in language and vision research fields @xcite , script-based character understanding is yet a hard task, as shown in our experiments. here we highlight two challenges. the first one is text type. as scripts mainly consist of conversations between different characters, at the core of script-based character understanding is conversation understanding. especially, scripts often involve multi-party conversations where multiple characters talk and interact with each other in a single scene. considering other common issues in conversation understanding, it is non-trivial for plms to comprehend characters based on finegrained conversation information @xcite @xcite . the other challenge of applying plms to script-based character understanding is text length. to address the aforementioned challenges, we propose a multi-level contrastive learning framework and capture both fine-grained and global information using two devised contrastive losses. for fine-grained character information, we build a summary-conversation contrastive loss by comparing character representations from different sources. specifically, we leverage two text fields in scripts, i.e., summary and conversation, and then extract character representations from the corresponding field. the representations of the same character are then treated as the postive pairs, while those of different characters are negative pairs. to model the global information, we also propose a novel cross-sample contrastive loss as inspired by @xcite . by aligning the same character's representation in different samples, the model overcomes the input length limitation and learns the global information of each character. to validate the effectiveness of our framework, we benchmark the performances of several plms, including spanbert, longformer, bigbird, and chatgpt-3.5, on three widely-adopted character understanding tasks. in general, our contributions are as follows: • we identify two critical challenges for character understanding in scripts and propose a multilevel contrastive learning framework to address them. ## preliminaries generally, character understanding tasks require the model to predict character's information given a segment of text. for script-based character understanding, the provided texts often consist of conversations within scripts. in this work, we also leverage script summaries as an additional source. we provide detailed examples in appendix a. in practice, the model first generates character's embeddings e in the representation learning step. subsequently, a feed-forward network ffn is often adopted as the classifier with the cross-entropy loss: our work presents a multi-level contrastive learning framework for character representation learning. firstly, we follow a general encoding process to obtain character representations from conversations and summaries. then, we describe two novel contrastive losses to capture fine-grained and global information at both in-sample and cross-sample levels. finally, we propose a two-stage training paradigm that applies different losses in different learning stages. figure 1 illustrates an overview pipeline of our method. ## discussion about llms on character understanding in this section, we go deeper to discuss the unsatisfied performance when adopting the icl of llms to perform character understanding tasks. one possible reason for this is the script-based character understanding we focus on requires the model to learn the character information globally. for example, in character guessing, anonymous speakers sometimes need to be identified with some global evidence, like linguistic style and the character's relationship with others. these subtle cues are usually not included in the current sample and thus require the model to learn them globally from other samples @xcite . however, due to the finetuned unavailability of icl, llms can only utilize local information from the current sample and limited demonstrations to make inferences. we believe this is the reason that llms don't perform well in our script-based character understanding scenario. additionally, we notice icl also falls short in some other tasks that involve learning a domain-specific entity or individual across multiple samples, like knowledge graph completion @xcite . this shortcoming in the global learning scenario, which is similar to hallucination @xcite and the reverse problem @xcite , can limit llms' application in many downstream tasks. it appears that augmenting the number of demonstrations in the prompt could be a potential strategy for enhancing the capabilities of llms in these global learning tasks. nonetheless, it's essential to note that incorporating an excessive number of relevant samples as demonstrations faces practical challenges, primarily due to constraints related to input length and efficiency considerations. in the future, more efforts are needed to explore optimal ways of harnessing the icl method of llms in such global learning scenarios. ## conclusions in this work, we focus on addressing two key challenges, text length and text type in scriptbased character understanding. to overcome these challenges, we propose a novel multi-level contrastive framework that exploits in-sample and cross-sample features. the experimental results on three tasks show that our method is effective and compatible with several sota models. we also conduct in-depth analysis to examine our method detailedly and provide several hints in the character understanding tasks. in the future, we plan to apply contrastive learning to other long-form document understanding tasks, such as long document matching @xcite and fiction understanding @xcite .