D ynamic TOC : Persona-based Table of Contents for Consumption of Long Documents (2022)

Abstract: Long documents like contracts, financial documents, etc., are often tedious to read through. Linearly consuming (via scrolling or navigation through default table of content) these documents is time-consuming and challenging. These documents are also authored to be consumed by varied entities (referred to as persona in the paper) interested in only certain parts of the document. In this work, we describe DynamicToC, a dynamic table of content-based navigator, to aid in the task of non-linear, persona-based document consumption. DynamicToC highlights sections of interest in the document as per the aspects relevant to different personas. DynamicToC is augmented with short questions to assist the users in understanding underlying content. This uses a novel deep-reinforcement learning technique to generate questions on these persona-clustered paragraphs. Human and automatic evaluations suggest the efficacy of both end-to-end pipeline and different components of DynamicToC.

## introduction documents such as financial statements, reports and contracts are often long and comprehensive, replete with domain-specific description and information. they are meant to be consumed by several entities or personas, e.g. legal department of companies, customers or financial organizations such as banks. as these documents contain vital information about the business, the business personas are often required to read through and analyze the documents in details. these personas are often interested in different sections of the document, based on the business requirements. for example, employees might be interested in the stock programs of the company, whereas the lenders and investors would like to read through profit statements. the traditional technology to navigate long documents is through a table of contents (toc) populated with the heading of each section and chapters. however, the table of contents does not show the information present in the underlying paragraphs of a section, and there is no way to highlight information relevant to different personas. to this effect, we propose dynamictoc, an intelligent table of contents-based navigator. dy-namictoc provides user the flexibility to choose the persona and read the document from its lens. for the current work, we focus on the finance and legal domain, and hence, personas are taken as commonplace entities like investors, lenders, financial bodies, etc. dynamictoc highlights the relevant sections of the document as per the persona. for this, the input finance or contract document is segmented at the paragraph level and a cluster of "aspects or topics" is inferred for each para. these are then mapped to the interest topics of the personas. further, dynamictoc has a novel question-based guided experience, to enhance the visibility of underlying information. studies have shown that questions are more intuitive and informative than headings and hence can provide a better understanding of what the paragraph talks about. the overall interface is shown in figure 1 . first step is to acquire a list of keyword candidates (e.g., n-grams or chunks) with heuristic methods @xcite , then rank them in accordance with their importance to the document @xcite @xcite . another task is compact and informative headline generation from a document @xcite . text summarization is the process of generating natural language summaries from an input document retaining the most important information @xcite . recently, an outline generation task was introduced by @xcite as a hierarchical structured prediction problem. given a document, their aim is to first predict a sequence of section boundaries and then a sequence of section headings accordingly to come up with a table of contents for the same. a related direction of work to ours is of aspect detection, which has been explored in the literature largely using user reviews for products. early works focused on rule-based approaches using lexicons and dependency relations, and utilize manually defined rules to identify patterns and extract aspects @xcite , which require domain-specific knowledge and human expertise. supervised approaches formulate aspect extraction as a sequence labelling problem that can be solved by hidden markov models (hmm) @xcite , conditional random fields (crf) @xcite @xcite , and recurrent neural networks (rnn) @xcite . these approaches have shown better performance compared to the rule-based ones, but require large amounts of la-belled data for training. early unsupervised systems are dominated by latent dirichlet allocation (lda)-based topic models @xcite @xcite . recently, deep learning based topic models @xcite @xcite have shown strong performance in extracting coherent aspects in an unsupervised manner. none of the prior works on aspect detection have worked with contracts or financial documents that are quite long (50-100 pages) in comparison to user reviews. even if we break the document at paragraph level, it can still go over tens of lines. hence, the importance of word frequency is much more in our case. we bridge the gap between directly using the unsupervised aspect detection frameworks for the financial documents by adding a tf-idf based weighing parameter while training. moreover, there are no gold standards for aspect detection for contract or finance domain, hence, we use unsupervised clustering based metrics for validating the output detection. further, it has been shown that question-answers play a critical role in scientific inquiry, informationseeking dialogue, and knowledge acquisition @xcite . in a dialogue system, question generation is used to obtain specific information from the user or make the conversation more pleasant @xcite . hence, we hypothesize that augmenting the default toc with questions that give a high level overview of the paragraphs can enhance the reading experience of the users. question generation can also be seen as a summarization or seq2seq task. various pre-trained language models like bart @xcite , pegasus @xcite , etc., have shown excellent results for these tasks. researchers have worked on top of these language models & proposed various rewards for qgen to optimize these models. @xcite has employed bleubased rewards, @xcite have used answerability rewards, whereas @xcite has used a combination of fluency, relevance, and answerability rewards. previous question generation literature has focused on generating questions based on an entity, phrase, or sentence. in this work, we explore longform question generation, i.e., question-based long text. we explore deep reinforcement learning techniques for the same. as they have shown competitive results in various natural language generation tasks such as summarization @xcite , style transfer @xcite , question generation @xcite etc. motivated by this we use bart as our base model. as there is a lack of labeled question datasets in financial domain, to overcome the domain shift problem, we train the model with additional rewards in a reinforcement learning setup to make more suitable for a general domain. there are several commercial products for reading documents across devices, but all of them have a fixed document navigation, based on chapters and headings. to the best of our knowledge, there is no prior art looking into providing an end-to-end persona-based navigation. the mentioned technologies address only part of the required solutions. following are the key contributions of our work: ## methodology in this section, different components of the dy-namictoc are described in details. ## results & discussion in order to evaluate the different parts of the pipeline, we employ metric based evaluation schemes. since there is no off-the-shelf criterion to evaluate all the stages of the pipeline together, we have provided independent evaluations for each of the sub-modules. however, the intended goal of our pipeline is to facilitate the readers in consuming long documents through the lens they deem most suitable for them. to facilitate an end-to-end evaluation and to understand whether the persona-based document segmentation with enhanced table of content is informative or not, we have conducted a small scale human evaluation. both metric-based and human-based evaluations are discussed in the following subsections. ## conclusion in this work, we have proposed a novel dynam-ictoc framework for consumption of long documents. financial documents are high value documents for businesses, and are often long and complex. the default toc-based reading experience is quite limited and document consumption can be enhanced using intelligent technologies. dy-namictoc is one of the first works to pursue this exciting research direction. dynamictoc would benefit from in-domain learning of aspect keywords and questions. evaluation of paragraph segmentation and mapping of personas to the aspects are future directions. a better understanding of personas would generalize the work to different domains.