A Span-based Dynamic Local Attention Model for Sequential Sentence Classification (2021)

Abstract: Sequential sentence classification aims to classify each sentence in the document based on the context in which sentences appear. Most existing work addresses this problem using a hierarchical sequence labeling network. However, they ignore considering the latent segment structure of the document, in which contiguous sentences often have coherent semantics. In this paper, we proposed a span-based dynamic local attention model that could explicitly capture the structural information by the proposed supervised dynamic local attention. We further introduce an auxiliary task called span-based classification to explore the span-level representations. Extensive experiments show that our model achieves better or competitive performance against state-of-the-art baselines on two benchmark datasets.

## introduction the goal of sequential sentence classification (ssc) is to classify each sentence in a document based on rhetorical structure profiling process @xcite , and the rhetorical label of each sentence is related to the surrounding sentences, which is different from the general sentence classification that does not involve context. an example is shown in figure 1 , the document is divided into rhetorical labels such as "background" and "outcome" for five sentences in nicta dataset. the ssc task is crucial for downstream domains such as information retrieval @xcite , question answering @xcite and so on. traditional statistical methods, such as hmm @xcite , crf @xcite , etc., heavily rely on numerous carefully hand-designed features. in contrast, recent methods based on end-to-end neural networks utilize hierarchical sequence * *corresponding author figure 1 : an example of nicta dataset for ssc task. the text has five sentences and is divided into two segments {(s 1 , s 2 ), (s 3 , s 4 , s 5 )} by labels. nevertheless, the above-mentioned methods ignore the latent structural information (e.g. segmentation) in the document, which is the grouping of content into topically coherent segments. intuitively, a segment with several continuous sentences is expected to be more coherent semantics than the text spanning different segments, e.g., the text with two segments in figure 1 . in this paper, we propose a novel span-based dynamic local attention model to explore the latent segment structure in a document for ssc task. first, we introduce dynamic local attention guided by segmentation supervision signal to focus on the surrounding sentences with coherent semantics, called supervised dynamic local attention (sdla). furthermore, we introduce an auxiliary task called span-based classification, which calculates semantic representations of spans and performs span classification on them to obtain predicted rhetorical labels. the dynamic local attention mechanism and the auxiliary task complement each other to enhance the model capacity to perceive segment structure and improve the performance of ssc task. the results on two benchmark datasets show that our method achieves better or competitive performance than state-of-the-art baselines. ## proposed method in this paper, we propose a span-based dynamic local attention model for sequential sentence classification with two novel components: supervised dynamic local attention and auxiliary span-based classification task, respectively. the architecture of our model is shown in figure 2 . ## conclusion in this paper, we propose a novel model for ssc task, which includes a supervised dynamic local attention to explore the latent segment structure of the document, and an auxiliary task to improve the performance at span-level representations. we demonstrate the effectiveness of our model on two datasets and find that our model also performs well in the text segmentation scenario. in future work, we will consider joint learning sequential sentence classification and text segmentation.