Interactive Query-Assisted Summarization via Deep Reinforcement Learning (2022)

Abstract: Interactive summarization is a task that facilitates user-guided exploration of information within a document set. While one would like to employ state of the art neural models to improve the quality of interactive summarization, many such technologies cannot ingest the full document set or cannot operate at sufficient speed for interactivity. To that end, we propose two novel deep reinforcement learning models for the task that address, respectively, the subtask of summarizing salient information that adheres to user queries, and the subtask of listing suggested queries to assist users throughout their exploration. In particular, our models allow encoding the interactive session state and history to refrain from redundancy. Together, these models compose a state of the art solution that addresses all of the task requirements. We compare our solution to a recent interactive summarization system, and show through an experimental study involving real users that our models are able to improve informativeness while preserving positive user experience.

## introduction integrating human interaction into nlp tasks has been gaining the interest of the nlp community. human-machine cooperation can improve the general quality of results, as well as provide a higher sense of control for the targeted consumer. we focus on the task of interactive summarization (intsumm: @xcite which enables information exploration within a document set on a topic, by means of user-guided summarization. as illustrated in figure 1 , a user can incrementally expand on a summary by submitting requests to the system, in order to expose the information of interest within the topic. a proper exploration session demands access to all information within the document set, and fast reaction time for smooth human a user interactively submits queries in order to expand on the information. the system is required to process the full document set for comprehensive exploration, respond quickly, and expose nonredundant salient information that also complies to the input queries. see real example in figure 5 . engagement @xcite . in addition, presented information must consider the session history to refrain from repetitiveness. while it is worthwhile to apply recent nlp advances that excel at extracting salient and querybiased information, those advances usually come at a cost of rather small input size limits or heavy computation time. indeed, all previous interactive summarization systems we know of either apply traditional methods or are inadequate for real-time processing due to high latency ( §2). our goal is to overcome these obstacles, and leverage advanced methods to improve information exposure while keeping latency acceptable for interaction. as depicted in figure 1 , an intsumm system provides an initial generic summary as an overview of the topic, after which a user can iteratively issue queries to the system for summary expansions on subtopics of interest. to support querying, the system offers a list of suggested queries, hinting at information concealed within the document set. we address the intsumm task components through two subtasks: (1) generating the initial summary and query responses, and (2) generating lists of suggested queries. for each of the subtasks we propose a deep reinforcement learning (rl) algorithm that addresses the respective sub-task requirements. to enable comprehensive topic exploration, our models speedily process the full document set, as inspired by @xcite . additionally, they are able to peek at session history to comply to the current state of the interaction. the model for the query-assisted summarization subtask, m summ , incorporates the query sequence by (1) encoding a query into the contextual sentence representations, (2) attending the representations using a new query-biased variant of the maximal marginal relevance (mmr: @xcite ) function, and (3) a dual reward mechanism for policy optimization @xcite which we adapt to consider both reference summaries and the query ( §3). the model for the suggested queries list generation subtask, m sugg , works at the phrase level, as opposed to the sentence level, to enable extraction of important phrases that serve as suggested queries. similarly to m summ , the model learns importance with consideration to session history, but without an input query -as its role is to suggest such a query ( §4). the models are trained on the duc 2 2007 multidocument summarization (mds) news-domain dataset, with adaptions for our task setting. for testing, we follow the intsumm evaluation framework of @xcite to run simulations, collect real user sessions, and assess the results, using duc 2006. in principle, summary informativeness, i.e. general salience, could potentially come at the expense of query responsiveness, but importantly, our results show that our rl-based solution is able to significantly improve information exposure over the baseline of @xcite , without compromising user experience ( §5). ## background and related work interactive summarization facilitates user-guided information navigation within document sets. the task suffered from a lack of a methodological evaluation, until @xcite formalized the intsumm task with a framework consisting of a benchmark, evaluation metrics, a session collection process and baseline systems. this framework, that we leverage, enables comparison and analysis of systems, allowing principled research on the task and accelerated development of algorithms. to the best of our knowledge, all previous works on intsumm have either applied more traditional text-processing methods or require costly prepro-2 https://duc.nist.gov/ cessing of inputs to facilitate seamless interaction. @xcite used surface-form features for processing content, and @xcite adapted classic mds algorithms like lexrank @xcite and klsum @xcite . @xcite optimized discourse graphs and @xcite relied on a knowledge representation, both expensively pre-generating hierarchical summaries that limit expansions to pre-prepared information selections. @xcite applied advanced coreference resolution algorithms that take several hours for preprocessing a document set. the two intsumm baseline systems of @xcite use sentence clustering or textrank @xcite for summarization, sentence similarity heuristics for query-responses, and n-gram frequency or textrank for suggested query extraction. moreover, their query-response generators strictly consider a given query, ignoring history or global informativeness. our proposed algorithms significantly improve information exposure over the latter baselines, using advanced deep rl methods, working in real time. we next review some recent techniques in mds, query-focused summarization and multi-document keyphrase extraction, all of which relate to the intsumm task and our choice of algorithms. the subtask of query-assisted summarization. non-interactive mds has been researched extensively, with few recent neural-based methods that can handle relatively large inputs. for example, @xcite use graph neural networks to globally score sentence salience, @xcite summarize using longformers @xcite , and @xcite combine a longformer with bart @xcite and incorporate graphical representation of information. @xcite apply deep rl for autoregressive sentence selection, and, in contrast to most other neural methods, can ingest the full document set. in the query-focused summarization (qfs) task summaries are biased on a query. to accommodate a query, @xcite use conditional selfattention to enforce dependency of the query on source words. @xcite and @xcite hierarchically encode a query with the documents. these and other qfs methods require large training sets, and limit the allowed input size @xcite . relatedly, incremental update summarization @xcite ) marks queryrelevant information as reported texts stream in, avoiding repeating information marked earlier. interactivity is not a constraining factor here, yielding solutions with relatively high computation time. with respect to the above related work, we develop a model inspired by @xcite , which is closest to our requirements. to facilitate an interactive setting, our model (1) enables query+history injection, (2) supports full input processing, necessary for complete information availability during exploration, (3) has low latency at inference time, and (4) requires a relatively small training set. the subtask of suggested-queries list generation. extracting suggested queries on a document set most resembles the multi-document keyphrase extraction (mdke) task since it aims to identify salient keyphrases @xcite . mdke was mostly addressed using traditional heuristics or graph-centrality algorithms applied over the documents (e.g. @xcite . in contrast to mdke, the suggested queries extraction subtask is a new paradigm that updates "keyphrases" with respect to session history. while previous methods for keyphrase extraction could potentially be adapted for our dynamic setting, we choose to focus in this work on a deep rl architecture for suggested queries that resonates our model for query-assisted summarization and allows sharing insights between the models. ## query-assisted summarization model the subtask of query-assisted summarization covers two main components of the intsumm task: the generators of an initial summary and of queryresponses. the initial summary concisely specifies some central issues from the input topic (not biased on a query) to initiate the user's understanding of the topic and to motivate further exploration. then, for each user submitted query, the query-response generator non-redundantly expands on the previously presented information with topically salient responses that are also biased around the query. we next formally define the subtask and then describe our rl model for it. ## experiments we ran several experiments for the assessment of our m summ and m sugg models, applying the intsumm evaluation framework of @xcite . the goals of the experiments are to compare varying configurations of our models and to evaluate against an intsumm baseline system. the experiments include both simulations and interactive sessions with human users. ## conclusion interactive summarization for information exploration is a task that requires compliance to user requests and session history, while comprehensively handling a large input document set. these requirements pose a challenge for advanced text processing methods due to the need for fast reaction time. we present novel deep reinforcement learning based algorithms that answer to the task requirements, improving salient information exposure while satisfying user queries and keeping user experience positive. we note that while m summ is designed for the intsumm task, it may potentially be serviceable for standard mds, qfs, update summarization and combinations thereof. this can be accommodated by a proper choice of input, e.g., qfs can be addressed by giving m summ as input a query, an empty history and target summary length. in future work, we may study the performance of our solutions for such tasks, as well as strive to further improve their performance on both ends of the intsumm task -selecting topically salient information and responding to user queries.