SCE - SUMMARY at the FNS 2020 shared task (2020)

Abstract: With the constantly growing amount of information, the need arises to automatically summarize this written information. One of the challenges in the summary is that it’s difficult to generalize. For example, summarizing a news article is very different from summarizing a financial earnings report. This paper reports an approach for summarizing financial texts, which are different from the documents from other domains at least in three parameters: length, structure, and format. Our approach considers these parameters, it is adapted to hierarchical structure of sections, document length, and special “language”. The approach builds an hierarchical summary, visualized as a tree with summaries under different discourse topics. The approach was evaluated using extrinsic and intrinsic automated evaluations, which are reported in this paper. As all participants of the Financial Narrative Summarisation (FNS 2020) shared task, we used FNS2020 dataset for evaluations.

## introduction the area of text summarization exists for several decades, since the first work of luhn @xcite . the summarization approaches developed from extractive unsupervised statistical approaches to abstractive supervised methods, using deep learning models @xcite . however, the most advanced seq2seq models (transformers) are very limited in input size and, therefore, are inapplicable to long texts. also, only few of state-of-the-art summarizers consider hierarchical structure of the input documents @xcite , their key concepts @xcite or topics @xcite and build a hierarchical summary @xcite . usually, hierarchical summary is built per document collection. the top level of hierarchy provides a general overview and users can navigate the hierarchy to drill down for more details on topics of interest. there is a growing interest in the application of automatic and computer-aided approaches for extracting, summarising, and analysing both qualitative and quantitative financial data, as a series of fnp and related workshops @xcite recently demonstrates. however, summarization of documents in financial domain is usually limited to summarization of financial news @xcite @xcite @xcite which are not very different from the general news in length and format. only few attempts were made to summarize financial reports @xcite , which are different from the news articles in at least four parameters: length, structure, format, and lexicon. this paper reports an approach for hierarchical summarization of financial reports. financial annual reports in the data of financial narrative summarisation (fns 2020) shared task 1 (el-haj et al., 2020) are long, have many sections, and are written in "financial" language using many special terms, numerical data, and tables. our system for summarization and hierarchical visualization financial reports (named by sce-summ) considers discourse and topic hierarchical structure and builds an hierarchical view of the summarized report with interactive user interface. text preprocessing, that includes text cleaning, sentence splitting and tokenization. we developed our own tool that cleaned text before segmenting it to sentences and tokens. financial reports usually contain a lot of sections, figures, and tables. because the text files in the fns-2020 dataset were obtained by converting pdf files to plain texts, these texts contain a lot of "noise" left from broken tables and meta-data such as section and page numbers. we cleaned the noise by measuring the ratio between text and numbers and ratio between number of words and whitespaces. lines with ratio less than 0.4 were removed. then, regular expressions were applied to find and mark such entities as url, phone number, date, time, email. finally, non-unicode characters were filtered out. section segmentation, where section headers are identified and a document is segmented into sections. the section titles were extracted following the heuristic rules saying that (1) each title appears in a separate line, (2) does not end with period mark, and (3) contains only few (up to 5) words with (4) each word either starting with capital case letter or containing only upper case letters. the extracted candidates were then compared against the list of 13 manually edited titles foot_1 . the candidate that obtained jaccard similarity above 0.4 to one of the titles from the list was extracted as a title. the text body between two consequent titles was marked as a section. discourse parsing of each section. for discourse parsing we used the codra parser @xcite . codra parser performs two-part process: (1) a discourse segmenter creates a segmentation analysis on the sentence level and edu's for the discourse parsing process and (2) a discourse parser parses the text on sentence level and document level to identify relations between parts of sentences and sentences in the document. the rhetorical analysis of the parser starts from a breaking a text into elementary discourse units (edus). because edus do not span across multiple sentences, this segmentation task finds edus inside the sentence boundaries. as a result, some sentences (actually, most, according to our observations) are split into edus. every edu is marked as a nucleus (an essence part) or a satellite (a complementary part of the related nucleus), based on the relation that they are connected to. internal (relation) nodes represent different inter-sentence relations: elaboration, same-unit, etc. topic modeling. for topic modeling we applied latent dirichlet allocation (lda) model @xcite . it was applied on all files in the fns-2020 dataset with predefined number of topics 3 . topic-to-text assignment, where each sentence (or sentence part) represented by a leaf node of the discourse tree, is assigned to one of the topics obtained by lda. we refer topic probabilities p(t|w) for all sentence s words @xmath0. therefore, we extract a dominant topic (@xmath1, as a topic with the maximal normalized sum of topic probabilities for all sentence words @xmath2: max @xmath3. topic distribution smoothing. we noticed that after single text nodes (that stand for sentences or sentence parts) are assigned to topics, we can get unexpected topic distribution where two parts of the same sentence or two adjacent sentences inside the same paragraph and/or belonging to the same discourse relation are assigned to different topics, and transition from one topic to another is not coherent. 4 we decided to smooth topic distribution by extrapolating one dominant topic on entire block of adjacent sentences and sentence parts, connected by a direct discourse relation. we denote nodes with at least one leaf node as "simple" and all leaves in its sub-tree are finally assigned to one dominant topic, so that a "random" noise is left out. the implement this approach as follows. we know that all leaf nodes are arranged in the natural sequential order of their texts from right-to-left (top-down) in a discourse tree. we assume that the important information usually comes first (important part of a sentence usually precedes its complementary part, and a sentence stating some fact usually precedes a sentence that elaborates more about this fact) and, therefore, upper right nodes and nucleuses should propagate their topics on their siblings. according to this assumption and our empirical observations on each parameter's influence, the final impact factor n i of node n is calculated as follows. if n is nucleus 0, else w 1 = 0.5, w 2 = 0.3, and w 3 = 0.2. then, the final dominant topic for a "simple" sub-tree is calculated as follows: max @xmath4,n }. after topic-to-sentence assignment (at previous stage), every leaf node has non-zero value for only one dominant topic, other topics have score t,@xmath5. summarization of entire report (regardless visualization) and of each section (for visualization needs) was performed as follows. all topics t are ranked by their importance t i(t) (normalized sum of their probabilities for all document/section words). then, summaries are created by extraction of nucleuses from each topic, in the topics' importance order, until the maximum length limit is reached. as for entire report a summary should not exceed 1000 words according to the shared task instructions, we limit a section summary to 100 words. hierarchical visualization. at this stage sce-summ creates an interactive html file with the data from all the stages for a user to browse. the file contains the following sections: (1) original text; (2) processed xml text after cleaning and section segmentation; (3) discourse trees for all the sections; (4) sentences (nodes) with assigned topics after smoothing; (5) the final hierarchical tree with the section summaries, and (6) a general report summary. for visualization and interactive user's navigation, the following tree structure of a document is built and present to a user: root represents an entire document and points to its sections, each section is split to major topics inside this section after smoothing, and each topic points to a summary of this particular section focused on the chosen topic. visualization is performed in interactive manner, upon a user's request. demo video foot_5 demonstrates all interactive options provided by the system. ## conclusions and future work this paper describes a new method for hierarchical summarization of financial reports, based on integrating the discourse structure and topic modeling. our approach is mainly based on heuristic assumptions. in future, we intend to develop a new supervised method utilizing the discourse and topics knowledge. also, we would like to apply this method and its extension to educational materials, which also have highly hierarchical structure and an evolving flow of topics in a discourse. hierarchical summarization can help to organize those materials in a hierarchical structure and provide users with interactive navigation to the topics of interest. sce-summ's source code is available foot_8 and can be run using the provided instructions foot_9 .