Recent Trends in Linear Text Segmentation: A Survey (2024)

Abstract: Linear Text Segmentation is the task of automatically tagging text documents with topic shifts, i.e. the places in the text where the topics change. A well-established area of research in Natural Language Processing, drawing from well-understood concepts in linguistic and computational linguistic research, the field has recently seen a lot of interest as a result of the surge of text, video, and audio available on the web, which in turn require ways of summarising and categorizing the mole of content for which linear text segmentation is a fundamental step. In this survey, we provide an extensive overview of current advances in linear text segmentation, describing the state of the art in terms of resources and approaches for the task. Finally, we highlight the limitations of available resources and of the task itself, while indicating ways forward based on the most recent literature and under-explored research directions.

## introduction linear text segmentation, also known as topic segmentation, is the task of identifying topic boundaries in a text using coherence modeling and/or local cues @xcite . the attribute 'linear' derives from the fact that in this setting, which is the most popular but not the only one, topics are considered "linearly" as following one another in documents and, as such, linear text segmentation ignores any sub-topic or hierarchic structure and focus on finding the boundaries between the topics thus linearly defined. this is also distinguished from topic classification, which relates to classifying text with the correct topic class; while linear text segmentation is strictly tasked with identifying the part of a text in which a topic boundary occurs. such boundaries then have a relevant role in a variety of contexts, such as finding individual 1 chris newell is no longer part of bbc r&d, but the paper was authored when still working at the organisation. this survey aims to give a comprehensive, yet brief overview of the field, highlighting the evolution of the approaches used to tackle the task as well as the available metric and resources and what remains to be done. such a survey is much needed as previous surveys on the topic are mostly outdated at this point (see, e.g., @xcite . crucially, previous surveys lack an in-depth exploration of the use of language models for the task, where transformer-based language models and large language models (llms) have now become, as in other areas of nlp, central for the task. in this survey, then, we aim to fill this gap by showing how the field has slowly shifted to use features from transformer-based language models and supervised learning as the framework of choice and how llms are just starting to get traction. in doing so, we will also highlight the various problems of resources and evaluation which, we argue, are central for further developments in the field. finally, we discuss future directions. this work is a necessary step for summarising and grounding recent research in the field, while pointing towards future developments which are worth the focus of future research. note that this survey does not touch upon sub-areas like multi-modality and more niche domains like videolecture segmentation: we focus on nlp and on the domains in which topic segmentation has traditionally been seen as a central task. future research might integrate the current work with these aspects. ## datasets many datasets for topic segmentation have been released, but very few have been widely adopted. in this paragraph, we focus on domains that are arguably the most represented in the literature and we divide them in two distinct macro-domains: namely, written text and dialogue. we mostly discuss english datasets, but we will mention in the open challenges the lack of multilingual resources. ## metrics even though traditional classification metrics like f1 and accuracy have been used and continue to be used in the field, specific evaluation metrics for topic segmentations have been variously suggested during the years as traditional classification metrics over-penalize near misses (i.e. a topic boundary placed close to a real one), while evidence suggests human annotators tend to disagree where exactly to place topic boundaries @xcite . evaluation in topic segmentation, however, is not a solved problem and specific metrics proposed for evaluating segmentation systems face a number of problems, mostly related to different types of errors (not including a topic boundary, including additional topic boundaries, or placing an existing topic boundary in an incorrect place) and how to quantify and to balance them. broadly speaking, segmentation metrics can be categorised into three groups: window-based, boundary similarity-based and embedding-based metrics. window-based metrics, exemplified by p k @xcite and windowdiff @xcite , employ a sliding window approach, comparing reference and hypothesis boundaries in the window. to overcome certain limitations of early window-based approaches winpr (scaiano and inkpen, 2012) was proposed to extend the common f1, precision, and recall measures to include a tolerance window. drawing on this, boundary similarity @xcite , proposed more recently to overcome some of the problems with window-based metrics, works by representing the input sequence using the identity of the topic segment each element in the sequence belongs to. given such a representation for both the hypothesized and reference segmentation, edit distance is used to quantify the error. finally, reference-free embeddings use notions of embedding similarities to measure similarity within (and/or difference between) hypothesized topic segments, but they still lag behind referencebased metrics in assigning best scores to systems that more closely reflect human-annotated topic boundaries, while also being highly dependent on the quality of the embeddings @xcite . figure 1 summarises the three different methods just described. p k , windowdiff, and f1 are the most used metrics in the field. p k and windowdiff, however, have been shown to have specific flaws related to penalizing certain types of errors more than others and to behave inconsistently in certain edge cases @xcite . alternatives like boundary similarity, which was proposed to overcome some of the limitations, are not as popular with few works using it and most literature preferring p k , notwithstanding its limitations @xcite . this is evident in figure 2 showing how popular different metrics are in the literature by the occurrences of different metrics as used in a sample of recent works (i.e. published after 2020) we cited. furthermore, figure 3 shows that p k tends to co-occur with windowdiff and with f1 in recent literature, while it never appeared together with boundary similarity. in our systems comparison, we also used p k , but we suggest that future research look into complementing or substituting this metric with more modern ones like boundary similarity to overcome well-known evaluation problems with p k @xcite . ## systems comparison having described unsupervised and supervised approaches for linear text segmentation proposed during the years, in choosing the systems to be compared we have prioritised the inclusion of at least one example for the different kinds of systems described in the previous section (the kind column in the table) and, where possible, for each system kind we have included at least one example using sentences as basic units and one using words. a final consideration was that of reporting what in our knowledge were the current state-of-the-art for the given benchmarks. the choice of systems for which to report results is also limited by the fact that metrics and datasets used vary greatly in existing literature, making the comparison harder. at first glance, in fact, it can be observed how sparse the tables are: this is due to the long period considered which implies several changes of popular benchmarks over the years, but it also reflects a wider problem in the field for which benchmarks are not consistently used, especially when dealing with domains such as meetings and multimedia content. on another side, it can be seen how supervised models in given the highlighted problem of generalizability, unsupervised systems are still relevant, as the comparatively good performance of bayesseg on the small icsi dataset in ## conclusions: open challenges and future opportunities the above discussion has shown how one of the major challenge in the field is the availability and the adoption of datasets (especially related to dts). when enough data are available supervised systems can be trained for both written text topic segmentation and dts generally showing improvements over unsupervised methods. at the same time, the large number of empty spots in our system comparison tables shows that no single dataset has ever been established as a widely recognized benchmark in the field. such empty spots are also partly explained by the variety of different metrics for segmentation evaluation, as the lack of a single, widely recognised standard metric means that different works often use different metrics. moreover, reported performance often does not reflect performance in real-world use cases, because of flaws of existing metrics like p k @xcite . future research should, in certain cases like podcast shows segmentation, propose new resources, but mostly it should establish which existing datasets and metrics are best suited to be used as benchmarks and evaluation metrics so that the numerous and rapid advances in this fast-evolving field can be compared in a fair and widely accepted setting. apart from resource limitations, methods for topic segmentation often assume a high level of agreement among human annotators, which isn't always the case @xcite . identifying topics can be straightforward in domains like news shows but more challenging in contexts such as multi-party dialogue. even when segmenting articles from wikipedia, decisions must be made about what constitutes a significant enough topic shift @xcite . previous research has explored hierarchical segmentation approaches, moving away from linear text segmentation @xcite . recent end-to-end systems have lagged in this aspect, but the cited work combining topic segmentation and topic modelling @xcite is a promising step forward to exploit knowledge about the topic structure rather than just local cues and coherence. modern llms might be particularly suited to combine different tasks in a multi-task and/or zero-shot framework, as initially explored by @xcite . our discussion primarily focused on english resources. recently, more diverse linguistic resources have been suggested, especially for mandarin @xcite , with two german datasets also noted @xcite . few examples of datasets for other languages exist, except for the multilanguage dataset proposed by @xcite , which remains underutilized. multilinguality is crucial to democratize and broaden the scope of nlp research. to summarise, in this work we have traced the various existing trends in literature for linear text segmentation within nlp and we have identified the following main challenges: lack of publicly available datasets: this problem affects mostly dts (specifically the media domain) and it is crucial as recent supervised systems greatly outperform unsupervised ones. as a subset of this problem, we have also mentioned the need for standard benchmarks for the task to better track the advances in the field. pitfalls in existing metrics: the most popular metric, p k has a number of well-documented shortcomings. even though newer metrics like boundary similarity have been proposed, p k is the most used even in recent works. low generalizability we have also discussed how the field has individuated generalizability as a key problem for the task, as many well-performing supervised systems might just be overfitting on specific cue phrases. we suggest the following future directions as open opportunities for researchers in the field: use of llms: the rise of llms has already reshaped many areas in nlp, and there is similar scope in this context, especially given the problems of generalizability and the lack of resources which affect the field. advances in multi-task learning: we highlight the combination of modern segmentation systems with topic modelling ones as a research direction worth developing, having deep roots in the field and narrowing the gap with hierachical segmentation, which is useful for overcoming the problem of arbitrary definition of topic granularity. advances in evaluation resources and metrics: we stress the importance of having a stable evaluation framework for the task. advances in metrics are useful to deepen our understanding of a task having low human annotators agreement. multi-lingual datasets, instead, can widen the reach of the available technology to less-resourced languages.