Symbolization, Prompt, and Classification: A Framework for Implicit Speaker Identification in Novels (2023)

Abstract: Speaker identification in novel dialogues can be widely applied to various downstream tasks, such as producing multi-speaker audiobooks and converting novels into scripts. However, existing state-of-the-art methods are limited to handling explicit narrative patterns like “Tom said, '...'", unable to thoroughly understand long-range contexts and to deal with complex cases. To this end, we propose a framework named SPC, which identifies implicit speakers in novels via symbolization, prompt, and classification. First, SPC symbolizes the mentions of candidate speakers to construct a unified label set. Then, by inserting a prompt we re-formulate speaker identification as a classification task to minimize the gap between the training objectives of speaker identification and the pre-training task. Two auxiliary tasks are also introduced in SPC to enhance long-range context understanding. Experimental results show that SPC outperforms previous methods by a large margin of 4.8% accuracy on the web novel collection, which reduces 47% of speaker identification errors, and also outperforms the emerging ChatGPT. In addition, SPC is more accurate in implicit speaker identification cases that require long-range context semantic understanding.

## introduction speaker identification in novel dialogues, also known as dialogue attribution @xcite , aims at identifying the speaking characters of utterances in fiction texts @xcite . it is an important task for various downstream applications like automatically assigning appropriate voices to utterances in audiobook production @xcite and novel-to-script conversion @xcite . as dialogues serve as type speaker sentence n1 -jin xiu shouted with enthusiasm, u1 jin xiu "let's sing a song together!" u2 wu zhongping "old songs or new songs?" n2 -said wu zhongping. u3 lanxiang "you should say the songs of the present or the songs of the past." n3 -lanxiang corrected her friend with a laugh. u4 wu zhongping "well, you're right. for the songs of the past i only know let's sway twin oars." u5 gu yangmin "it's just the time for it." n4 -said gu yangmin. the major interaction between characters in novels, automatic identification of speakers can also be useful for novel-based knowledge mining tasks such as social network extraction @xcite and personality profiling of the characters @xcite . most recent approaches for speaker identification @xcite @xcite are based on pre-trained language models (plms) like bert @xcite . plm-based methods enjoy the advantage of the plm's internal linguistic and commonsense knowledge obtained from pre-training. however, two main downsides should be highlighted for these methods. on the one hand, some methods truncate a context into shorter textual segments before feeding them to plms @xcite . intuitively, this approach inevitably introduces a bias to focus on short and local texts, and to identify speakers by recognizing explicit narrative patterns usually in the local context of the utterance. it may fail in implicit speaker identification cases when such patterns are not available and long-range semantic understanding is indispensable. on the other hand, some methods adopt an end-to-end setting @xcite that sometimes extracts uninformative speakers like personal pronouns. besides, they only perform mention-level speaker identification in which two extracted aliases of the same character won't be taken as the same speaker. in recent months, large language models (llms) have become the most exciting progress in the nlp community. although llms show impressive zero-shot/few-shot capabilities in many benchmarks @xcite @xcite , how well they work for speaker identification remains unknown. drawing inspiration from the successful application of prompt learning and pattern-exploiting training in various tasks like sentiment analysis @xcite , we propose a framework to identify speakers in novels via symbolization, prompt, and classification (spc). spc first symbolizes the mentions of candidate speakers to construct a unified label set for speaker classification. then it inserts a prompt to introduce a placeholder for generating a feature for the speaker classifier. this approach minimizes the gap between the training objectives of speaker identification and the pre-training task of masked language modeling, and helps leverage the internal knowledge of plms. in previous studies, the interutterance correlation in conversations was shown to be useful for speaker identification in sequential utterances @xcite @xcite . spc also introduces auxil-iary character classification tasks to incorporate supervision signals from speaker identification of neighborhood utterances. in this way, spc can learn to capture the implicit speaker indication in a long-range context. to measure the effectiveness of the proposed method and to test its speaker identification ability, spc is evaluated on four benchmarks for speaker identification in novels, specifically, the web novel collection, world of plainness, jin-yong novels, and csi dataset. compared to the previous studies that conduct experiments on merely 1-18 labeled books @xcite , the web novel collection dataset contains 230 labeled web novels. this dataset enables us to fully supervise the neural models and to analyze their performance at different training data scales. experimental results show that the proposed method outperforms the best-performing baseline model by a large margin of 4.8% accuracy on the web novel collection. besides, this paper presents a comparison with the most popular chatgpt foot_0 , and results indicate that spc outperforms it in two benchmarks. to facilitate others to reproduce our results, we have released our source code foot_1 . to sum up, our contributions in this paper are three-fold: @xcite we propose spc, a novel framework for implicit speaker identification in novels via symbolization, prompt, and classification. (2) the proposed method outperforms existing methods on four benchmarks for speaker identification in novels, and shows superior cross-domain performance after being trained on sufficient labeled data. ## related work in recent years, deep neural networks have shown great superiority in all kinds of nlp tasks @xcite , including textbased speaker identification. candidate scoring network (csn) @xcite is the first deep learning approach developed for speaker identification and outperforms the manual featurebased methods by a significant margin. for each candidate speaker of an utterance, csn first encodes the shortest text fragment which covers both the utterance and a mention of the candidate speaker with a bert. then the features for the speaker classifier are extracted from the output of bert. in this way, the model learns to identify speakers by recognizing superficial narrative patterns instead of understanding the context. cuesta-lázaro et al. (2022) migrates a dialogue state tracking style architecture to speaker identification. it encodes each sentence separately with a distill-bert @xcite before modeling cross-sentence interaction with a gated recurrent unit @xcite . then a matching score is calculated between each character and each utterance. however, this method still just utilizes the plm to model local texts and results in poor performance. @xcite adopts an end-to-end setting that directly locates the span of the speaker in the context. it feeds the concatenation of the utterance and its context to a roberta @xcite , after which the start and end tokens are predicted on the output hidden states. yet under end-to-end setting, only mention-level speaker identification is performed, in which two extracted aliases of the same character are taken as two different speakers. previous studies have shown the great advantage of applying deep-learning methods to speaker identification in novel dialogues, but these methods either are limited to recognizing superficial patterns or only identify speakers at mention level. our study proposes a method to identify speakers based on context understanding and improves the performance on implicit speaker identification problems when long-range semantic information is needed. the proposed method outperforms other competitors given sufficient labeled data, and is more efficient in data utilization. ## experiments 4.1 datasets we conducted experiments on four chinese speaker identification datasets which are the web novel collection (wn), world of plainness (wp) @xcite 4 , jin-yong novels (jy) (jia et al., 2020) 5 , and end-to-end chinese speaker identification dataset (csi) @xcite 6 respectively. wn is a large internal speaker identification dataset. its annotation details can be referred to in appendix d. wn and csi both consist of web novels of various genres and writing styles, while wp and jy are printed literature with more normative writings. wp and jy can serve as cross-domain evaluation datasets for wn. as no test data was provided in csi, we use the development set of csi as the test data and randomly sampled 10% of instances from the training set for validation. the number of novels and utterances in each subset of each dataset is shown in for clarity, each subset of a dataset is named as "dataset-subset", e.g., "wn-train" for the training set of wn. to evaluate the impact of a smaller training set, we also sampled 5 novels from wntrain to make a smaller training set with about 31k utterances. to distinguish between the whole training set and the sampled one, we referred to them as wn-large and wn-small respectively. ## conclusions in this paper, we propose spc, an effective framework for implicit speaker identification in novels via symbolization, prompt, and classification. experimental results show spc's superiority on four speaker identification benchmarks and its remarkable cross-domain capability. furthermore, spc significantly outperforms the strongest competitor csn in implicit speaker identification cases that require deeper context understanding. we also evaluate chatgpt on two speaker identification benchmarks and present a comparison with spc. in the future, we hope to harness llms with longer input length limits to further improve speaker identification performance.