A Proposal: Interactively Learning to Summarise Timelines by Reinforcement Learning (2021)

Abstract: Timeline Summarisation (TLS) aims to generate a concise, time-ordered list of events described in sources such as news articles. However, current systems do not provide an adequate way to adapt to new domains nor to focus on the aspects of interest to a particular user. Therefore, we propose a method for interactively learning abstractive TLS using Reinforcement Learning (RL). We define a compound reward function and use RL to fine-tune an abstractive Multi-document Summarisation (MDS) model, which avoids the need to train using reference summaries. One of the sub-reward functions will be learned interactively from user feedback to ensure the consistency between users’ demands and the generated timeline. The other sub-reward functions contribute to topical coherence and linguistic fluency. We plan experiments to evaluate whether our approach could generate accurate and precise timelines tailored for each user.

## introduction notable events often happen over a long period. for example, covid-19 caused immeasurable damage around the world, lasting for more than a year. when reviewing different aspects of the disaster, the huge number of reports and news articles makes it difficult to trace the development of events such as outbreaks, policy interventions and vaccination efforts. tls can solve this problem by identifying significant dates and summarising events of sub-topics. most prior tls works focused on producing extractive timelines, which copies the original sentences from input documents @xcite @xcite . irrelevant and repeated information may be extracted in this process, decreasing the quality of the generated timelines. abstractive timeline summari-sation methods can address this problem @xcite but few neural network models have been proposed due to the lack of reference timelines for supervised learning. producing reference timelines by human requires expertise to capture important temporal information and sub-events from the source documents, thus it is extremely expensive. in mds tasks, researchers have tried heuristics-based and unsupervised methods to address the reference data shortage problem @xcite . however, their results on some evaluation metrics, like rouge-2, only reached half of the upper bound. @xcite showed that interactive learning could improve the performance of an mds system via leveraging users' preference, which is relatively easy to obtain, and does not require reference summaries. therefore, we take inspiration from their work to propose an interaction-based abstractive tls framework. @xcite treated the tls task as an mds task and proposed a modular summarisation method, which achieved the state of the art and is adaptable. however, its adaptation requires abstracting mathematical constraints from concrete requirements. this contrasts with interactive learning (il), which greatly decreases the cognitive burden for humans by receiving user feedback to refine summaries @xcite . comparing to traditional approaches, interaction enables the model to learn from the users, thus it is possible to accurately tailor and refine timeline summaries according to users' demands. in this paper, we propose an interaction-based abstractive timeline summarisation framework using deep rl. by learning a reward signal from user feedback, we can fine-tune a pretrained mds model for the tls task via a small number of interactive learning rounds. therefore, our framework should be capable of generating timeline summaries with high text quality after enough episodes of training. and we plan both simulation and realuser experiments to evaluate the framework on two benchmark tls datasets, timeline17 @xcite and crisis @xcite . the workflow of our model (figure 1 ) mainly follows the event detection method, clust (ghalandari and ifrim, 2020), which identifies subevents first and then generates summaries for them. due to the rl-based interactive learning process in the framework, our model can be automatically adapted to new topics and adjusted by users' interests. our main contribution is a proposed interactive method for generating timelines for news, which adapts to user feedback through rl fine-tuning. interactive summarisation instead of producing reference texts by crowdsourcing, obtaining information (e.g., keywords) via user interaction can be more practical to obtain training data. @xcite outperformed previous extractive mds approaches on rouge-based metrics by querying topic words from users. @xcite collected pairwise comparisons between summaries from simulated users, which are then used to train a ranker without any reference data, and fixed the efficiency issue of il. due to the similarity between the mds and tls task, il is expected to solve the reference timelines shortage problem as well, without increasing many computation expenses. so we introduce interaction into an rl-based tls model for the first time. reinforcement learning in natural language generation (nlg) recent research on applying rl on nlg tasks has received some success. some prior works on dialogue systems @xcite utilized rl-based fine-tuning method to ensure the factual consistency of the response. in automatic summarisation @xcite @xcite , il is applied to learn a reward function from users, so that rl agents could learn a policy to summarise text indirectly under users' guidance. however, for the tls task, we are the first to use rl to generate summaries for key dates. ## method all components of our method shown in figure 1 will be introduced below. ## summary we propose an interactive method to summarise timelines without reference data. in each interactive learning round, we first update the reward function, and then use rl to fine-tune a huge neural network model. then the model will generate summaries for each of the important sub-events, which are identified by textual similarity to the articles in the corpus. all the summaries will be ordered by their assigned dates to form a timeline. the user can preview the timeline and give feedback to start another round of interactive learning. part of our method has been implemented, including pega-sus to summarise event clusters but without rl or user feedback. given the current experiment results, we can expect better performance after the interaction part implemented. the challenge remains in rl and designing suitable modes of interaction. we will move forward to our planned experiments and report our results in future work.