A Computational Analysis of Financial and Environmental Narratives within Financial Reports and its Value for Investors (2020)

Abstract: Public companies are obliged to include financial and non-financial information within their cor- porate filings under Regulation S-K, in the United States (SEC, 2010). However, the requirements still allow for manager’s discretion. This raises the question to which extent the information is actually included and if this information is at all relevant for investors. We answer this question by training and evaluating an end-to-end deep learning approach (based on BERT and GloVe embeddings) to predict the financial and environmental performance of the company from the “Management’s Discussion and Analysis of Financial Conditions and Results of Operations” (MD&A) section of 10-K (yearly) and 10-Q (quarterly) filings. We further analyse the mediating effect of the environmental performance on the relationship between the company’s disclosures and financial performance. Hereby, we address the results of previous studies regarding environ- mental performance. We find that the textual information contained within the MD&A section does not allow for conclusions about the future (corporate) financial performance. However, there is evidence that the environmental performance can be extracted by natural language processing methods.

## introduction understanding the textual information in corporate disclosures is an important part of financial accounting research. it provides additional information about numerical financial data and aids the reader to understand the managerial information sets. in addition, the manager's communication patterns can reveal certain behavioral traits -e.g., in case the company provides truthful and meaningful information or if the company tries to obfuscate or withhold information. the textual information can, hence, guide users of corporate disclosures to better understand the firm's behavior and decisions @xcite -in particular, when information asymmetries exist @xcite . however, corporate disclosures often tend to be boilerplate @xcite @xcite . as a consequence, the securities and exchange commission @xcite , concerned that a substantial part of the corporate disclosures includes generic language and immaterial detail, issued a guideline to elicit more meaningful disclosure. in 2010, the sec further published an interpretive release on climate change related disclosures "to remind companies of their obligations under existing federal securities laws and regulations to consider climate change and its consequences as they prepare disclosure documents", @xcite . therefore, the company's disclosures should not only consist of financial narratives but also contain information about the environmental aspects concerning the firm. whether these corporate disclosures are truly informative @xcite or are just boilerplate generic disclosures @xcite @xcite remains an empirical question, which we address in this paper. while in the financial domain, the financial narratives of corporate reports have been analysed with respect to dictionary-based approaches or standard classification methods like naïve bayes @xcite , little research has focused on more sophisticated methods @xcite . this equally applies for studies analysing the non-financial information foot_0 content @xcite . the most common methods for analysing the non-financial, narrative information content still remain a manual-or dictionary-based approach @xcite @xcite @xcite . also, only few studies concerning non-financial information focus on the actual narrative information content, and rather address the quantity of non-financial information published @xcite . in this paper, we analyse the information content of the management's discussion and analysis of financial conditions and results of operations (md&a) section of 10-k and 10-q filings with respect to the financial performance and environmental performance using a variety of natural language processing (nlp) methods. our contributions are that we, (1), evaluate if the md&a section contains information that can be used to predict financial and environmental performance in a machine learning setting, (2), analyse if the environmental information complements the financial information in explaining the future financial performance. 2 as more than 2000 studies have linked non-financial performance to financial performance @xcite , we assume that the narrative per se also contributes to the financial performance. hence, we directly train the classifier on the financial performance and the environmental performance using a sentence-wise bert-and word-wise glove-embedding model, instead of regressing a certain measure on a disclosure-score previously derived from either a manually constructed word-list or a tfidf-weighting scheme, like most other studies. the hypothesese we discuss in this paper are: (h1) corporate disclosures, particularly the md&a section, contain information regarding the company's financial performance. (h2) corporate disclosures, particularly the md&a section, contain information regarding the company's environmental performance. (h3) narrative, non-financial information within the md&a section, complements narrative financial information in explaining the financial performance, i.e., environmental performance serves as a mediator for financial performance. ## study design and research methodology our aim is to examine whether nlp methods can capture the meaning of the md&a section of corporate disclosures with respect to financial and non-financial narratives. if we find evidence, this is an indicator that such information is contained, if not, this is an indicator that this information might not be available, which, however, would motivate future work to investigate this further. therefore, we directly train classifiers on the text in the md&a section to predict financial and environmental performance. environmental performance text indep. variable figure 1 elicits our approach: first, we predict the financial performance from the md&a section, i.e., direct path. second, we predict the environmental performance from the text, represented by path env. third, we build an extended model, which predicts the financial performance from the text while also considering the (predicted) environmental performance, i.e., combined path. in case all paths show a meaningful relation to their labels and the combined path shows an improved f 1 -score compared to the direct path, we conclude that the environmental narrative complements the financial narrative in explaining the financial performance. ## empirical results ## analysis and discussion analysing the reports revealed that the md&a section varies largely in content and size. while 10-q reports are rather short and often refer to the 10-k report for more detail, the 10-k reports tend to be rather extensive, with some reports ranging up to more than 100 pages. however, the heterogeneity of 10-k reports make inter-comparisons quite challenging. for our sample, we could not show that the md&a sections have information content with respect to financial or environmental performance. given our sample, we cannot reject the null hypothesis for h1, h2, and h3 and, hence, conclude that the md&a section is not informative with respect to financial and environmental information. however, this can be due to the low performance of the classifiers. as performance of trained classifiers could not be generalised, it is likely that there is not sufficient, material environmental and financial information within the md&a section, i.e., reports tend to be boilerplate. analysing the top 200 weights of the tfidf-classifier, we observe that particularly for the environmental labels the majority of words can be related to the pharmaceutical or chemical industry. the top 200 weights show that for the short-term environmental performance 25 words can be related to the pharmaceutical or chemical industry (12.5%) and 41 words for the long-term environmental performance (20.5%). filtering for reports which include at least 5 of these words reveals that these reports are related to the manufacturing division (particularly sic showing pharmaceutical preparations; surgical&medical instruments; biological products industry etc.). this indicates that the classifier rather learns industries instead of environmental performance, even as we are controlling for industry and market capitalization. however, it is important to note that including at least five of these words in a report does not necessarily result in a performance improvement. while for ∆eps i,q 21 words can be categorized as financial words, the top 200 words appear to be quite random for bhar i,t . ## conclusion in this paper, we analysed the information content of the md&a section with respect to financial and environmental narratives employing various nlp methods. we proxy environmental performance with the change in esg rating percentile scores and the financial performance with earnings and buy-and-holdreturns. although, we sought to implement a model that uses the predicted environmental performance and text to predict financial performance, results proved to be not as promising. therefore, we implemented the model with the original environmental performance data instead of the predicted data. we find that our classifiers are not able to surpass a baseline model of always guessing an improvement in financial and environmental performance. as a consequence, we were not able provide evidence that the md&a section has information content with respect to financial and environmental narratives, and the section appears to be rather boilerplate. this asks for more structured reports with respect to volume and content as well as the inclusion of more material information. however, it is clearly a limitation of this study to focus solely on one esg rating provider. as rating construction among providers varies, this raises general data quality concerns @xcite and limits the generalisability of our study. although other information sources were neglected, corporate disclosures should be by itself useful, informative and reliable. while one could argue that the information is already priced into stock prices shortly after dissemination, the general information content with respect to earnings remains remarkably low. surprisingly, the usefulness of the md&a section appears to be quite limited with respect to future corporate performance. an explanation are the reports ambiguity and low polarization. as other researchers have shown, specific word lists are more likely to capture the specific context in this setting. alternatively, a larger sample could overcome the distinctions of the reports for classification purpose. future research might use different rating providers, support the classifiers by humanly annotation, or apply bert models specific for the financial domain, like the recently published finbert @xcite .