A Compare-and-contrast Multistage Pipeline for Uncovering Financial Signals in Financial Reports (2023)

Abstract: In this paper, we address the challenge of discovering financial signals in narrative financial reports. As these documents are often lengthy and tend to blend routine information with new information, it is challenging for professionals to discern critical financial signals. To this end, we leverage the inherent nature of the year-to-year structure of reports to define a novel signal-highlighting task; more importantly, we propose a compare-and-contrast multistage pipeline that recognizes different relationships between the reports and locates relevant rationales for these relationships. We also create and publicly release a human-annotated dataset for our task. Our experiments on the dataset validate the effectiveness of our pipeline, and we provide detailed analyses and ablation studies to support our findings.

## introduction with the rapid growth of information, many tasks in the field of natural language processing (nlp) involve streamlining information comprehension. one such task is summarization, which selects a subset of sentences or generates new content that best represents the given document @xcite @xcite . this task helps humans save time and effort by identifying important information in a text. in the finance context, comprehending regulatory narrative reports is a classic example of efficiently mining signals from a large amount of text. as these reports often contain rich information concerning specific financial entities, discovering valuable insights is crucial for academia and the finance industry. much research has shown that textual features from financial reports contain valuable financial signals about future firm performance and market reactions (e.g., @xcite @xcite . however, authorities such as the securities and exchange commission (sec) require that companies provide comprehensive and detailed information about their current status in these reports, which often contain much unimportant and already-known information. for example, the token overlap ratio between annual 10-k reports of the same company between adjacent years is often high, foot_0 making it a challenging and tedious task to acquire important signals in new reports (termed as the overlapping characteristic hereafter). recent advances in nlp technology have included attempts to efficiently and effectively comprehend lengthy financial documents. one approach to address this problem is through summarization (e.g., @xcite @xcite . other approaches additionally leverage numerical metrics, such as stock return volatility and abnormal trading volumes, to locate essential financial signals in reports (e.g., @xcite @xcite @xcite . however, these approaches often require high-quality human annotation or suitable financial measures, which poses significant limitations in practical scenarios. in this study, we approach financial report comprehension from a novel perspective by leveraging the intrinsic year-to-year characteristic of reports (i.e., the overlapping characteristics). specifically, for a particular company, we use the document published in the previous year as an information anchor (i.e., the reference) to construct a year-to-year structure and locate important financial signals in the report of the subsequent year (i.e., the target). this inherent structure enables us to mine financial signals in a compare-and-contrast self-supervised manner, compared to existing supervised approaches. based on the year-to-year structure, we propose a compare-and-contrast multistage pipeline to effectively locate financial signals in reports. we first identify a few types of relationships between reference and target financial reports at the segment level. then, using these recognized relationships, we present a novel financial signal-highlighting task together with a domain-adaptive highlighting model. the goal of this task is to identify the rationales, represented by the importance of certain words, for a specific pair of year-to-year segments. therefore, the words with high importance are deemed to be crucial financial signals in these reports. for experiments, we present a synthetic dataset consisting of 30,400 reference-to-target segment pairs for financial signal highlighting. 2 experimental results validate the effectiveness of the proposed pipeline; detailed analyses and ablation studies are also provided. ## problem definition the year-to-year nature of financial reports allows us to take advantage of the differences between a company's documents in consecutive years. these differences may reveal complex but insightful relationships within a pair of documents. to better understand these relationships, we investigate them through rationales (represented by the word importance), which are considered essential signals in financial reports. ## (ref.) our most critical accounting policies relate to revenue recognition, inventory, pension and other postretirement benefit costs, goodwill, ... ## (target) our most critical accounting policies relate to revenue recognition, inventory, pension and other postretirement benefit costs, goodwill, ... net sales in the americas increased 5%, or @xmath04,302.9 million. ## (target) net sales in the americas decreased 1%, or @xmath14,513.8 million. to locate meaningful financial signals revealed by segment pair differences, we further classify each pair (r, t) ∈ t into the following two sets: 1. t β contains reference-to-target segment pairs with largely similar meanings (see note that all the aforementioned terminologies and notations can also be found in figure 1 ; they will be used throughout the following sections of this paper. ## proposed pipeline here we describe the proposed multistage pipeline for discovering the rationale behind the referenceto-target structure in financial reports, as illustrated in figure 1 . ## the final dataset we constructed final (financial-alpha), a financial signal highlighting dataset, consisting of 30,400 reference-to-target segment pairs @xmath6. ## related work research on financial report analysis has been ongoing for many years, with various studies utilizing both textual and numerical features to identify signals in reports. for instance, some researchers have used the relationship between tokens and quantitative indicators from the financial market to identify financial risks (e.g., @xcite @xcite . others have adapted unsupervised methods to recognize information and classify risk factors in financial reports (e.g., @xcite . however, previous research has mostly focused on risk factors in a global context rather than companyspecific signals, which is the focus of this study. recently, transformer-based language models such as bert, gpt-3, and t5 @xcite @xcite have made significant strides in the summarization task. in 2020, @xcite proposed the financial narrative summarization shared task (fns 2020), which aims to summarize annual uk reports. while some methods for this task have achieved satisfactory performance using rouge as a metric (e.g., @xcite @xcite , they have been criticized for sometimes omitting essential signals under a rouge-guided policy. additionally, the signals discovered through these approaches are heavily dependent on high-quality human-annotated summaries, making it challenging to apply them in real-world scenarios. in the field of nlp, some research has focused on developing rationalizing models related to the concept of our highlighting model. for example, @xcite proposed a method for learning the rationale (words) to justify a model's prediction by selecting a subset of text inputs. more recently, some studies have proposed methods that can rationalize the relationship of sentence pairs, such as natural language inference @xcite and query-document relevance @xcite . additionally, deyoung et al. (2020) released a benchmark to facilitate the development of interpretable nlp models with faithfulness. ## conclusion this paper addresses the task of identifying rationales as insightful financial signals between two narrative financial reports in consecutive years. we use the reference-to-target structure of financial reports to develop a compare-and-contrast multistage pipeline, comprising mainly of relation recognition and signal highlighting stages. in particular, we propose domain-adaptive learning strategies for the financial signal highlighting task, including out-ofdomain warm-up and in-domain fine-tuning. our empirical results confirm the effectiveness of the proposed approaches. we also present the newly constructed final dataset. some future works include (a) improving highlighting effectiveness by developing multitask learning on large financial corpora as financial pretrained representations; (b) increasing efficiency by integrating dense retrieval methods in the relation recognition stage; (c) analyzing broader relationships beyond the year-to-year ones (e.g., crosscompany); (d) identifying important words in both reference segments and target segments (i.e., twoway rationalization), which may provide a more in-depth financial analysis. and we believe this research can facilitate nlp techniques applied in finance domain.