S umm H elper: Collaborative Human-Computer Summarization (2023)

Abstract: Current approaches for text summarization are predominantly automatic, with rather limited space for human intervention and control over the process. In this paper, we introduce SummHelper, and screencast demo at https://www.youtube.com/watch?v=nGcknJwGhxk a 2-phase summarization assistant designed to foster human-machine collaboration. The initial phase involves content selection, where the system recommends potential content, allowing users to accept, modify, or introduce additional selections. The subsequent phase, content consolidation, involves SummHelper generating a coherent summary from these selections, which users can then refine using visual mappings between the summary and the source text. Small-scale user studies reveal the effectiveness of our application, with participants being especially appreciative of the balance between automated guidance and opportunities for personal input.

## introduction text summarization is the task of generating a condensed version of a given text. most summarization approaches operate in a fully automated pipeline. while efficient, fully automatic summarization does not flexibly enable human intervention and control during the summarization process, which could potentially tune the process to better accommodate user preferences, as well as rectify inevitable mistakes made by models. our objective in this paper is to promote such a human-involved approach to summarization, allowing to better tailor the eventual output to real-world user needs, and to synergize the efficiency of the computer with the quality of the human @xcite @xcite . the process can conveniently support a range of practical scenarios that require individual preferences, such as editors preparing summaries of articles, students condensing notes, or legal practitioners abridging contracts. to advance such direction, we present summhelper, a 2-stage summarization assistant, which decomposes the summarization pipeline into two natural subtasks-content selection followed by summary generation-and facilitates human-machine cooperation in each subtask. on an input document, the process starts with the selection of content for the summary ( §3.1). summhelper suggests possible salient content, efficiently pointing users to central information within the text (see @xcite in figure 1a ). users may accept or reject suggested spans, or highlight any other content to include in the summary @xcite in figure 1a ). upon receiving highlighted content within the text, summhelper subsequently consolidates it and generates a coherent summary ( §3.2). this step coincides with the recently introduced controlled text reduction task (ctr; @xcite , which produces a coherent fused version of the content of marked spans ("highlights") in a source document, as interpreted within the context of the full text. once ready, users can review the generated summary and edit any unsatisfactory content. to facilitate inspection, users are presented with a side-by-side display of the summary and the highlighted input (see figure 1b ), with clearly marked alignments between summary spans and corresponding source text spans @xcite in figure 1b ). the automatic alignments aid users in navigating through the input text and identifying summary content that may need editing. to assess summhelper's usefulness for generating customized summaries, we conduct two user studies ( §4), following common human-computer interaction (hci) methodologies and applying prominent usability questionnaires. these studies indicate the system's utility and user-friendly design for a thorough collaborative summarization process. notably, users valued the tool's guidance throughout the process, while also appreciating their continuous involvement in refining automatic decisions. ## background and related work this section provides a brief overview of related lines of work in summarization. these include strategies offering some level of user control ( §2.1), and modular summarization pipelines that separate the task into distinct subtasks ( §2.2). ## the summhelper application summhelper is a web application designed for human-computer cooperation in generating humancontrolled summaries, shown in figure 1 . it consists of two stages: (i) computer-assisted content selection via highlighting ( §3.1), and (ii) automated summary generation according to the selected content followed by machine-assisted reviewing and editing of the generated summary ( §3.2). ## experiments and evaluation we assess summhelper via two user studies with human subjects, using standard human-computer interaction (hci) questionnaires. in the first study, we examine the usability of summhelper for carrying out its purpose, i.e., summarizing an article in a collaborative manner, granting control to the user throughout the process. the second study compares summhelper to a conventional summarization setup, where a standard auto-generated summary can simply be post-edited without any specialized automated assistance, aiming to assess summhelper's comparative utility. ## conclusion in this paper, we presented summhelper, a novel summarization assistant, which collaborates with users across two steps: content selection and content consolidation. the system facilitates user intervention and supervision along the summarization process, in order to achieve the most suitable output tailored to specific needs. preliminary user studies illustrate summhelper's potential for a thorough and collaborative summarization process, with users expressing satisfaction with the process, as well as the final output. future work may include investigating more effective semantic strategies to locate summarysource alignments with acceptable latency. additionally, in light of some user feedback, another interesting extension includes developing more abstractive consolidation and fusion models, which would offer control over the level of abstractness in the outputs. lastly, exploring strategies to scale summhelper to a multi-document setting presents another promising avenue for future investigation.