Climate-related financial disclosures aim at increasing the transparency to guide investment and lending decisions in the financial sector. They reflect not only the physical effects of a changing climate, also the transition processes to decarbonize the economy that bear financial risks and opportunities. Such transition risks and opportunities include policy and legal change, technology and market shifts, and reputation-related risks (ClimateWise, 2019;Semieniuk et al., 2020). Transition risks are found to be more imminent than physical risks from climate change, and there are indications that some of those risks are already financially priced (Kölbel et al., 2020). In fact, the G20 Financial Stability Board has established a Task Force on Climate-related Financial Disclosures (TCFD) to issue recommendations for disclosure of climate-related risks and opportunities (CRO)

Tackling Climate Change with Machine Learning Workshop at ICML 2021. in corporate reporting (TCFD, 2019). However, recent analyses of the current standard of reporting have found insufficient clarity of CRO reporting (meaning that disclosures are not specific enough to allow a judgement whether CRO are material, i.e. significant, for future financial performance), with some recent improvements (TCFD, 2019; Climate Disclosure Standards Board (CDSB), 2020; Demaria & Rigot, 2019). To establish more stringent reporting standards on CRO, policy makers and financial regulators have begun establishing regulations (Steffen, 2021)  (French Republic, 2015), and a recent Executive Order on Climate-Related Financial Risk in the US (The White House, 2021). To monitor the state of climate-related disclosure and establish the effectiveness of related regulation, financial reports need to be analyzed to assess the extent and quality of such disclosures by companies and organizations. Typically, these analyses are conducted manually, which is timeintensive: recent examples covered only the 40 (Demaria & Rigot, 2019) and 50 largest companies (Climate Disclosure Standards Board (CDSB), 2020) of different listings. Constraining the analysis to only a small set of the largest companies risks introducing bias or preventing important insights: first, the financially strongest companies might not sufficiently represent sectors that are most relevant for lowcarbon transitions (e.g., most freight shipping companies are typically very small, while having a significant carbon footprint (Teter et al., 2017)). Second, there are equity concerns if CROs of smaller companies are not monitored (e.g., they could experience significant transition-related opportunities but investments are not directed to them). This bias might cause the financial impacts of climate change and transitions to be incorrectly priced, and result in an inefficient allocation of capital during the low-carbon transition (Semieniuk et al., 2020). the TCFD has conducted an "AI review," using a supervised learning approach that is not further detailed. They identified compliance with the TCFD Recommended Disclosures, but did not assess the quality of the disclosed information nor the type of risk (TCFD, 2019). This approach was refined by Luccioni et al. (2020), who developed a question answering approach to identify passages in climate disclosures that answer the 14 TCFD recommendations, and make their trained model accessible as a tool for sustainability analysts. Bingler et al. (2021) developed "ClimateBERT" to analyze compliance with TCFD recommendations in a variety of corporate reporting globally, and find mostly disclosure of non-material TCFD categories. Kölbel et al. (2020) were able to identify an increase in disclosure of transition risks in 10-K reports that outpaced those of physical risks, based on their measure of climate disclosure using a fine-tuned BERT model. Finally, Sautner et al. (2020) use a rule-based approach for identifying CRO-related language in corporate conference calls. They use machine learning (ML) for expanding their set of keywords, which was also proposed by Luccioni & Palacios (2019) for analyzing climate-related disclosure. All of these studies have analyzed the number of mentions of climate-related disclosures, however, the quality and materiality of the disclosures remains largely unclear. Analyzing the types of reported risks is a step into this direction, allowing potential investors to better judge the materiality of reported risks. We expand on Kölbel et al. (2020) by introducing more fine-grained risk categories and detect them in free text such as European annual reports (instead of 10-Ks). While most previous work has taken the approach to classify at the sentence level, we observe that more context is needed for disclosing risks, and we classify at the paragraph-level. To carry out our project, we create a novel dataset based on a refined labeling scheme to distinguish different types of climate-related risks (Section 3). We then train different classification algorithms to identify and categorize paragraphs in free-text annual reports that disclose such climate-related risks (Section 4). Finally, we apply the model to analyze climate-related disclosure in annual reports of 337 European firms over the past decade (Section 5). We created our own labeled dataset for the task of classifying paragraphs according to disclosure of climate risks. 1 We built a corpus of annual reports from the 50 largest publicly traded companies (STOXX Europe 50) and more than half of the European firms in the STOXX Europe 600 index for the last 20 years (where available), which we obtained from the companies' investors relation websites and Refinitiv 1 Available upon request. Eikon (Refinitiv Eikon, 2021).We then parsed the PDF files using the Apache Tika package2 and split the documents on each page into paragraphs using a rule-based approach (regex). The paragraphs were annotated by student assistants familiar with climate policy, who were trained by the authors and followed our code book (see Appendix). The five risk categories include two types of physical risks and three types of transitions risks. "Acute" and "chronic" physical risks denote those from increases in extreme weather events and those that develop slowly like changes in precipitation patterns, respectively. Transition risks include those related to the potential introduction or strengthening of climate policies ("policy & legal"), to changing market and technological environments ("tech. & market"), and to the reputation of corporations or products ("reputation"). For the test and validation datasets, we labeled 120 STOXX Europe 50 reports in their entirety.
We sampled stratified by years and industries in order to avoid bias in the dataset and later evaluate the model performance across industries and time. To reduce the number of pages to screen, we pre-selected those pages that included at least one match with an extensive list of relevant keywords (see Appendix), and their neighboring pages. On average, a report consisted of 34 relevant pages with 16 paragraphs each. All paragraphs on those selected pages were then annotated with the five categories allowing multiple labels per paragraph. Paragraphs without risk disclosure on relevant pages were considered "negative examples", and perceived edge cases were labeled as "hard negatives." We randomly split the dataset in test and validation data, and ensured that each contains a separate set of companies to avoid spill-overs. The classes are highly imbalanced, which is why we employed a greedier approach for the training dataset focused on covering the variance among positive examples and including only hard negative examples. We extracted relevant pages from annual reports of STOXX Europe 600 companies using a more tailored keyword list than for the test dataset, and then selectively labeled relevant paragraphs. The resulting datasets are summarized in Table 1. To assess the inter-coder reliability, two coders independently labeled 20 reports resulting in a Krippendorff's alpha of α = 0.20 (union) and α = 0.69 (intersection) for 5 classes. We divide the task of classifying climate risks in three tasks of increasing difficulty: Binary (classification in "risk" or "no risk"), and multi-label with two (physical and transi- tion risks) and five classes (all risk categories). On all of these tasks we evaluate a baseline model, pretrained Distil-BERT (Sanh et al., 2019), and RoBERTa (Liu et al., 2019). As a baseline model, we selected a support vector machine (SVM) (Cortes & Vapnik, 1995) as a one-versus-rest classifier and applied standard preprocessing to the input such as stop-word removal, lemmatization and TF-IDF weighting. We addressed class imbalances between negatives and positives with class weights and used Precision-Recall AuC on the validation set for scoring. To leverage context-specific word embeddings, we finetuned different variants of pretrained BERT-related models (Devlin et al., 2019) such as DistilBERT (Sanh et al., 2019) and RoBERTa Large (Liu et al., 2019) on our training dataset using negative log-likelihood loss with a softmax activation function for binary classification and a binary cross-entropy loss for the multi-label classification tasks. Again, we calculated class weights to address the class-imbalance. We trained the models for 4 epochs, using early stopping and limited hyperparameter search on the validation set. We also determined the optimal class probability thresholds by maximizing the F1-score on the validation set. Training is estimated to have emitted less than 7.12 kg CO 2 e in total. We evaluate the models on the three tasks defined above as well as in the following settings: (1) discriminatory where no negative paragraphs are present, (2) hard negatives setting with paragraphs that are edge cases, and (3) realistic setting with all negatives from pre-selected pages. We choose the best model for inference based on the F1-score on the validation set for the realistic setting and five risk categories. RoBERTa achieves the best performance in the 5class/realistic setting (Table 2). As RoBERTa is the largest among the models compared, this confirms expectations. We evaluate the model performance on held-out test data in the realistic setting (Table 3). In general, the model suffers from a relatively low recall of 0.360, and its precision does not exceed 0.5. This is explained by the fact that the task of identifying disclosures requires domain expertise and is also rather difficult for humans. For comparison, after refining the coding scheme, we conducted a review of test and validation data, resulting in a precision of 0.59 for the preliminary coding of binary, which is lower than what the model achieves on the same task. We find a large variance in performance across classes, with reputational transition risks being the hardest to identify (F1-score of 0.140) and acute physical risks the easiest (F1-score of 0.537). We also observe that physical risk classes exhibit a considerably higher precision and lower recall than transition risk classes, which are more balanced. We determined the number of risk mentions for 4,498 annual reports by all 337 companies in the dataset by performing inference on pages in proximity of a keyword match. Out of 2.7m analyzed paragraphs, 3892 paragraphs were predicted to contain at least one risk (total of 5501 risk mentions). Figure 1(a) shows the average number of mentions per report of physical and transition risks over time, which grew slowly until 2015, after which it increased rapidly. This growth is particularly high for transition risks, resulting in about

In the present article, we developed an approach to automatically identify climate risk disclosures in corporate annual reports, and used it to analyze disclosure of 337 European companies over 20 years. We find that the number of risk mentions (especially of transition risks) started to rise sharply around 2015. It appears likely that public policies played a role for this development, as numerous policies to encourage or mandate climate risk reporting have been enacted in Europe since 2015 (Steffen, 2021). To assess whether specific policies indeed caused the development, however, requires further research. Potential empirical designs to that end include difference-in-differences approaches, or models with country-and industry-fixed effects. Our approach is well suited to the deliver the dependent variable for such analysis. Next steps for refining our analysis will focus on appropriately quantifying the uncertainty of model predictions, and working to reduce it further by exploring a hierarchical classification approach, and adding more training data. The analysis can also be expanded to a broader set of company types and communication channels beyond annual reports. Finally, it should be kept in mind that improved transparency on climate risks should not automatically be expected to change investor behavior in a meaningful way (Ameli et al., 2020).
More research is needed to understand how capital is (re)allocated based on better climate risk disclosures; in this context our approach can be useful to deliver the explana-tory variable for such future analyses. Ultimately, both the effectiveness of policies to trigger climate risk disclosures, and the effectiveness of such disclosures to change investment behavior, are required for financial investments to help achieve the targets of the Paris Agreement. Recent work has shown first successful computerized analysis of climate-related financial disclosures. For example,

https://tika.apache.org/