Towards Human-Centered Summarization: A Case Study on Financial News (2021)

Abstract: Recent Deep Learning (DL) summarization models greatly outperform traditional summarization methodologies, generating high-quality summaries. Despite their success, there are still important open issues, such as the limited engagement and trust of users in the whole process. In order to overcome these issues, we reconsider the task of summarization from a human-centered perspective. We propose to integrate a user interface with an underlying DL model, instead of tackling summarization as an isolated task from the end user. We present a novel system, where the user can actively participate in the whole summarization process. We also enable the user to gather insights into the causative factors that drive the model’s behavior, exploiting the self-attention mechanism. We focus on the financial domain, in order to demonstrate the efficiency of generic DL models for domain-specific applications. Our work takes a first step towards a model-interface co-design approach, where DL models evolve along user needs, paving the way towards human-computer text summarization interfaces.

## introduction the ever increasing amount of online text documents, such as blog posts, newswire articles and academic publications, during the last decades, has created the urgent need for appropriate natural language understanding tools. summarization, i.e., shortening an initial text document by keeping only the most important information, plays a key role in addressing this information overload. a lot of sophisticated summarization models have been proposed in the past, with a recent focus on deep learning (dl) architectures. dl models @xcite @xcite @xcite @xcite achieve great results in the task of summarization, outperforming most of the previously used methods. typical dl models involve sequence to sequence architectures with rnns @xcite often combined with attention mechanisms @xcite , as well as transformers @xcite @xcite . despite the success of dl models, some significant challenges remain. the low interpretability of these models @xcite @xcite is a major drawback that limits significantly the trust of users in the whole process. in addition, existing pipelines do not adequately engage the human in the summarization process @xcite , providing isolated and static predictions. the engagement of users and their feedback in the whole process can be a key factor in creating high-quality models and improving the quality of existing models @xcite . to overcome the above limitations, we revisit the task of neural summarization from a humancentered perspective and with a unifying view of user interfaces and underlying summarization models. more specifically, we present a system that allows the active involvement of the user, setting the basis for human-computer text summarization interfaces. our system allows users to choose over different decoding strategies and control the number of alternative summaries that are generated. users can give their feedback by combining parts of the different generated summaries as a target summary for the corresponding input. these summaries are recorded, and can then be used as additional training examples, which in turn will improve the performance of the model and customize it to the preferences of the users. in addition, our system provides useful insights about the inner workings of the model, based on the self-attention mechanism of transformers. knowing which parts of the source document are most important for the generation of the final summary, can build up the trust between users and the machine. we present a case study of the proposed system on the challenging, domain-specific task of financial articles summarization, to demonstrate the ability of the suggested approach to successfully employ generic dl models for domain-specific applications that often have different requirements. indeed, domain-focused summarization models @xcite are generally more challenging, as they require deeper knowledge of the specific domain intricacies in order to generate salient summaries with logical entailment. to this end, we compiled a novel financial-focused dataset, which consists exclusively of financial articles from bloomberg foot_0 . the rest of this paper is structured as follows. the main features of the proposed human-centered system are detailed in section 2. the case study on financial news summarization is presented in section 3. finally, conclusions and interesting future research directions are discussed in section 4. ## hci meets summarization in this section we will introduce the main features of our human-centered summarization system. we first present the approach used for interpreting the summaries generated by the model. then we present the different decoding strategies we employ during inference. finally, we explain how users can interact with our system. ## case study: financial summarization in this section, we detail our experiments with the case study of financial summarization. we first describe the data collection process and the preprocessing steps we followed. then we discuss the models that we constructed and their evaluation. finally we discuss concrete examples of the user experience. the code and instructions for this case study of our system is publicly available foot_1 . ## conclusions and future work we presented a novel system for human-centered summarization that actively engages the user into the whole process of summarization, enabling personalized summaries and models. the users interact with the model, by entering a source text, selecting different decoding strategies, viewing a visualization of the model's attention and synthesizing a final summary from parts of multiple summaries, which can be used for further fine-tuning. we also presented a case study of our work, along with a novel dataset, on summarizing financial news. we observed that pre-trained pegasus models adapt quickly to our dataset, generating salient financially-focused summaries. our work aims to inspire future research in human-centered techniques for neural summarization systems. in future work, human involvement in the summarization process could be enhanced by using approaches that allow users to control different aspects of the generated summaries, such as length @xcite @xcite , style @xcite or generation based on a specific entity of the text @xcite . the interface we designed can be also further extended, allowing the user to evaluate the generated summaries, assessing different aspects of the text, such as salience, readability and coherence. finally, more advanced approaches can be explored for leveraging the user submitted feedback in order to further improve the underlying model @xcite .