F in S im4- ESG Shared Task: Learning Semantic Similarities for the Financial Domain. Extended edition to ESG insights (2022)

Abstract: This paper describes FinSim4-ESG 1 shared task organized in the 4th FinNLP workshopwhich is held in conjunction with the IJCAI-ECAI-2022 confer- enceThis year, the FinSim4 is extended to the Environment, Social and Government (ESG) insights and proposes two subtasks, one for ESG Taxonomy Enrichment and the other for Sustainable Sentence Prediction. Among the 28 teams registered to the shared task, a total of 8 teams submitted their systems results and 6 teams also submitted a paper to describe their method. The winner of each subtask shows good performance results of 0.85% and 0.95% in terms of accuracy, respectively.

## introduction the finsim shared task aims to spark interest from communities in nlp, ml/ai, knowledge engineering and financial document processing. going beyond the mere representation of words is a key step to industrial applications that make use of natural language processing (nlp). this is typically addressed using either 1) unsupervised corpus-derived representations like word embeddings, which are typically opaque to human understanding but very useful in nlp applications or 2) supervised approach to semantic representations learning, which typically requires an important volume of labeled data, but has high coverage for the target domain or 3) manually labeled resources such as corpora, lexica, taxonomies and ontologies, which typically have low coverage and contain inconsistencies, but provide a deeper understanding of the target domain. these approaches form a different spectrum which a number of them have attempted to combine, particularly in tasks aiming at expanding the coverage of manual resources using automatic methods. this new edition of finsim4-esg is extended to the "environment, social and governance (esg)" related issues based on the sustainability reports, esg reports, environment report and annual reports periodically published by financial companies. the esg criteria is a set of standards for a company's behavior used by socially conscious investors to screen potential investments. environmental criteria consider how a company safeguards the environment, including corporate policies addressing climate change, for example. social criteria examine how it manages relationships with employees, suppliers, customers, and the communities where it operates. governance deals with a company's leadership, executive pay, audits, internal controls, and shareholder rights. for example, in financial domain, the esg criteria are applied to assess the companies risk on these esg aspects, so that help investors supporting business aligned with green initiatives. according to the european commission, from the end of 2022, companies providing investment products that make sustainability or environmental claims will be required to disclose how their portfolios align with the eu taxonomy foot_2 and esg regulations for sustainable activities. the objective of this shared task is to elaborate an esg taxonomy, esg concepts representations, based on the data like companies' sustainability reports, annual reports, environment reports, etc. and make use of them to analyze how an economic activity complies with the taxonomy. consequently, it allows us to know how an investment product aligns with esg regulations. ## related works the finsim4-esg proposes two subtasks: esg taxonomy enrichment and sustainability prediction. the subtask1 is similar to the previous tasks of finsim shared tasks, given a training set of terms and a fixed set of concepts, participants are asked to propose systems allowing to categorize new terms to their most likely concepts. a term-concept pair has a hierarchical and semantic relation if one of them can be conceived as a more generic term (e.g. emissions -greenhouse gas emissions). and the subtask2 proposes to solve a sentence classification problem in order to classify each sentence extracted from the esg related reports into sustainable or unsustainable. a taxonomy represents a semantic relation of term pairs, which is "isa" pairs, largely used in nlp and ie tasks. the taxonomy extraction task from a domain specific corpora as a competition is first proposed by the shared task texeval @xcite and texeval-2 @xcite as part of semeval-2015 and 2016. several works introduced methods to learn hypernymy from the corpora and showed how to induce taxonomies from "isa" pairs. while those systems largely exploit semantic lexical resources like wordnet, babelnet, yago, wiki, dbpedia, distributional approach was not meaningfully adopted. more recently, as part of semeval 2020, the shared task on predicting multilingual and cross-lingual (graded) lexical entailment @xcite proposed a challenge for detecting semantic hierarchical relation, hypernym-hyponym, from multilingual and cross-lingual datasets. the distributional track was newly added in order to evaluate distributional systems. the participating systems make use of rulebased approach by exploiting wiktionary definitions of concepts @xcite or distributional approach combining distributional word vectors, multilingual lexical resources and translated parallel corpora to obtain cross lingual synonyms, then to extract a set of terms which are semantically most similar to a seed term @xcite . also, in the previous finsim shared tasks @xcite , the authors proposed various approaches for the hypernyms and synonyms ranking of financial terms by using the state-of-art models like bert and its variants (e.g. finbert) with good performance results. the distributional semantic models are widely explored in different nlp based financial data analysis. a lot of studies make use of the fine-tuned models from various extensions of bert model like sentence-bert [reimers and gurevych, 2019], roberta @xcite , distilbert @xcite . for example, the fine-tuned model on the financial data, @xcite , is largely used and esg-bert @xcite is also trained on sustainability corpus with the growing interest in the esg data analysis. recently, we observe that the esg related data like 10k report, 10q filling or annual reports has been studied for automating the esg ratings of the companies. @xcite proposes a linear svm classifier to predict market performance based on narrative disclosures of 10k reports while @xcite and @xcite investigate the effect of the environmental performance of a company on the relationship between the company's disclosures and financial performance. @xcite also proposes an approach to automatically convert unstructured data into esg scores using a pre-trained bert model. @xcite proposes a diachronic analysis of esg terms in uk annual reports at the first computing social responsibility workshop-nlp approaches to corporate social responsibilities (csr-nlp foot_3 ). @xcite proposes nlp based methods for the analysis of financial reports in order to identify climate-relevant sections based on a question answering approach and @xcite develops a pipeline of esg news extraction, news representations, and bayesian inference of deep learning models. ## task description the new edition proposes two subtasks: esg taxonomy enrichment and sustainability prediction. ## results and analysis in all the participating teams commonly explored bert models along with its variants to measure the semantic relatedness between terms and concepts. the fine-tuned models on the esg corpus on a basis of bert @xcite , sentence-bert [reimers and gurevych, 2019], distilbert @xcite , roberta @xcite , linkbert @xcite , @xcite , al-bert @xcite are proposed by most of the participating systems either for the word representations in vector space or for the term/sentence classification task and the classical logistic regression model is trained for the classification task giving the most performing results. ## conclusions and perspectives the finsim4-esg proposed two subtasks: esg taxonomy enrichment and sustainability prediction. among the 8 participating teams, 6 teams submitted the systems runs to the subtask1 and all submitted to the subtask2. all of the system runs showed very promising results using state-of-art nlp and ml techniques and features. as the first edition about esg taxonomy and sustainability prediction, the systems with the best performance achieved a good accuracy of 83%∼85% for the subtask1 and a high accuracy of 94%∼95% for the subtask2. all the participating systems largely exploited distributional methods for the similarity measures between terms and sentences, and for the classification task, and the results showed that using distributed and contextual features improve the performance of their systems. especially, several experiments from the participating systems show that the fine-tuned roberta on the esg data outperforms other models like bert, sentence-bert, fin-bert, linkbert and the linear classifier performs better than non linear classifiers for both subtasks. and the results confirm that the data augmentation helps improve the overall results as also shown in the results of the previous editions of the finsim shared task. the impact of ai technologies grows more and more in esg related domains like esg ratings for analyzing the sustainable activities of the companies, green investments supporting activities aligned with environmentally friendly business and helping investors to hold green bonds, green etfs, green funds or stock of the companies supporting green initiatives, esg risk assessment, esg databases, etc. and this requires a study on how to exploit a large scale of esg related concepts and build a knowledge representation of those concepts. the eu taxonomy was already released in the objectives of european green deal 6 but they needs to extend the scope of the concepts toward social and government topics. in this shared task, we elaborated and provided a first version of esg taxonomy taking into account the eu taxonomy along with the esg criteria proposed by the well known esg data providers like refinitiv and moody's. it will be possible to improve finsim-esg task by proposing to increase the coverage of esg concepts and its terms as the proposed concepts are still limited to those observed in the corpus composed of a limited number of reports. also, the current task is focused on a monolingual data processing. knowing that esg data analysis is gaining increasing global attention and has become an increasingly important part of the investment, every year, more companies publish their activities related to esg topics in non financial reports in different languages from different countries. the majority of the esg concepts are language-independent, so it will be interesting to extend the task to a multilingual data processing.