Joint Learning-based Heterogeneous Graph Attention Network for Timeline Summarization (2022)

Abstract: Previous studies on the timeline summarization (TLS) task ignored the information interaction between sentences and dates, and adopted pre-defined unlearnable representations for them. They also considered date selection and event detection as two independent tasks, which makes it impossible to integrate their advantages and obtain a globally optimal summary. In this paper, we present a joint learning-based heterogeneous graph attention network for TLS (HeterTls), in which date selection and event detection are combined into a unified framework to improve the extraction accuracy and remove redundant sentences simultaneously. Our heterogeneous graph involves multiple types of nodes, the representations of which are iteratively learned across the heterogeneous graph attention layer. We evaluated our model on four datasets, and found that it significantly outperformed the current state-of-the-art baselines with regard to ROUGE scores and date selection metrics.

## introduction timeline summarization (tls) is designed to extract sentences that describe an evolutionary story from a massive amount of web articles with respect to a specific topic in chronological order. tls has drawn much attention in recent years @xcite @xcite @xcite since it releases people from burdensome manual creation of summaries and gives readers a faster but comprehensive access to track events from many aspects, such as start and end, causality, and the main protagonists involved. most studies on tls seek ways to combine two individual subtasks: date selection and event detection. depending on different strategies for them, current methods are generally divided into three categories (ghalandari and ifrim, 2020): 1) direct summarization approaches @xcite @xcite directly identify topic-related sentences from a collection of news articles to form a timeline; 2) date-wise summarization methods @xcite @xcite first select salient dates then construct a timeline for each date individually with sentences of the highest score; and 3) event detection algorithms @xcite @xcite detect events by clustering sentences from multi-timeline news articles then identify several of the most important events and summarize them separately. although great successes have been achieved in conducting tls, several issues remain unsolved. first, current tls methods mainly adopt statistical hand-designed features to represent dates, e.g., the number of published articles and topic-related sentences in a specific time duration @xcite , and employ sentence-bert (reimers and gurevych, 2019) and other predefined unchangeable representations for sentences. the low-level or unlearnable representations tend to ignore the semantic and temporal information interaction between sentences and dates, which significantly degrades the performance of downstream tasks. secondly, traditional approaches focus on either date selection or event detection. although excellent date selection algorithms can pinpoint accurate timeline dates, they usually extract topic-irrelevant sentences. while event detection algorithms are capable of avoiding redundant summaries by various clustering strategies, they sometimes capture wrong timeline dates. to the best of our knowledge, there is no framework that jointly learns the advantages of the above two subtasks to accurately capture salient dates and eliminate topic-irrelevant sentences in a timeline. to circumvent the above dilemma, we propose to jointly learn date/sentence representations and event detection-based sentence clustering in a het-erogeneous graph attention network (han) for tls. specifically, we construct a heterogeneous graph with dates, words, and sentences as semantic units to solve the first problem. in this graph, words act as a bridge between dates and sentences, enabling date nodes to learn different granularities (word-and sentence-level) of semantic information and sentence representations to be complemented with a date-related intra-and cross-sentence message. as for the second issue, semi-supervised date prediction and event detection-based clustering are integrated into an overall objective, where labeled dates guide and facilitate sentence clustering, and sentence-level clustering information indicating main events improves the accuracy of unlabeled date prediction. note that we create a new way beyond the above-mentioned three categories as a joint end-to-end approach since we no longer have to handle each subtask step by step. we highlight our contributions as follows: