Ranking Environment, Social And Governance Related Concepts And Assessing Sustainability Aspect of Financial Texts (2022)

Abstract: Understanding Environmental, Social, and Governance (ESG) factors related to financial products has become extremely important for investors. However, manually screening through the corporate policies and reports to understand their sustainability aspect is extremely tedious. In this paper, we propose solutions to two such problems which were released as shared tasks of the FinNLP workshop of the IJCAI-2022 conference. Firstly, we train a Sentence Transformers based model which automatically ranks ESG related concepts for a given unknown term. Secondly, we fine-tune a RoBERTa model to classify financial texts as sustainable or not. Out of 26 registered teams, our team ranked 4th in sub-task 1 and 3rd in sub-task 2. The source code can be accessed from https://github.com/sohomghosh/Finsim4_ESG

## introduction these days a lot of investors have become socially responsible and environmentally conscious 1 . they tend to choose stocks and funds which do not harm the environment 2 . keeping this in mind, organizations are also putting in efforts to increase their environmental, social, and governance (esg) ratings. they tend to publish reports mentioning the esg aspect of their policies. however, reading through all such reports is time-consuming and inefficient. this brings in the need for an automated system for mapping terms to esg concepts and classifying financial texts as sustainable or not. finnlp workshop of ijcai-2022 conference hosted a shared task with these problems. we present an example of this in figure 1 . our team lipi participated in 1 https://news.gallup.com/poll/389780/ investors-stand-esg-investing.aspx (accessed on 10 june 2022) 2 https://bwdisrupt.businessworld.in/article/sustainableinvesting-to-surge-to-125-b-in-india-by-2026-report/0906-2022-432078/ (accessed on 10 june 2022) the shared task and ranked 4 th and 3 rd in sub-tasks 1 and 2 respectively. in this paper, we describe our solutions. ## related works the sub-task of mapping terms with high level concepts is similar to hypernym detection. for the natural language processing (nlp) community, hypernym detection has been an active area of research. several semeval tasks ( @xcite , @xcite , @xcite , (camacho-collados et al., 2018)) were organized on this topic. subsequently, three editions of finsim @xcite , @xcite , @xcite ) shared task were held which adapted the task of hypernym detection for the financial domain. this year while organizing finsim-4, this was extended to esg insights. with the rising popularity of green investing, understanding the sustainability aspect of financial texts has become extremely important. smeuninx et al. @xcite studied the readability of annual reports of several organizations. they highlighted how formula-based readability scores classified these texts as complex documents. they also mentioned the need for nlp based techniques to comprehend the readability of such documents. luccioni et al. @xcite fine-tuned roberta-base @xcite model to develop a question-answering based tool, climateqa for extracting sections related to climate from financial reports. guo et al. @xcite proposed a framework esg2risk for predicting stock prices by analyzing esg related events from financial news. they specifically used sentiments from these events. nugent et al. @xcite pre-trained a bert @xcite model with financial news articles from reuters news archive for predicting esg related controversies. furthermore, they used it for mapping financial news into one of the united nations sustainable development goals. ## problem statements the fourth edition of finsim presented two subtasks. they are as follows: given a set j consisting of n tuples of terms and their high level concepts i.e. @xmath0, c 1 ), (t 2 , c 2 ), ...(t n , c n )} where c i represents the high level concept corresponding to the i th term t i and c i ϵ set of concepts mentioned in the evaluation metrics for this sub-task were accuracy and mean rank. as per the evaluation script shared by the organizers, the rank of an instance was calculated by checking the presence of the true value in the first three elements of the predicted ranked list. ## data the data sets provided by the organizers consist of a set of 190 documents in pdf format, 651 terms mapped to 24 concepts and 2265 financial texts labelled as sustainable or unsustainable. we provide more details about the data set in the following sections. ## system description as per the rules, for every team, the number of submissions for each sub-task was restricted to two. we describe each of our submissions here. we pictorially depict our methodology in figure 2 . ## experiments and results we initiated by fine-tuning the all-mpnet-base-v2 model ( song et al., 2020) using sentence transformer architecture. our objective was to reduce the multiple negatives ranking loss as well as the online contrastive loss for the task of information retrieval 4 . we also studied the effect of changing this model with the sbert-un model, adding negative samples and concepts as it is. we further experimented with different sampling methods as mentioned in section 4.2. furthermore, we finetuned a roberta-base (liu et al., 2019) based model to classify terms into 24 pre-defined concepts or classes. group-1 group-2 group-3 group-4 carbon factor employee development injury frequency rate audit oversight emissions recruiting and retaining employees injury frequency rate for subcontracted labour shareholder rights energy efficiency and renewable energy future of work human rights executive compensation employee engagement share capital group-5 group-6 group-7 waste management sustainable transport board independence water waste-water management sustainable food agriculture board make-up for sub-task-2, we fine-tuned four models for classifying financial texts into two classes sustainable and unsustainable. these models are: roberta-base @xcite , finbert @xcite , sbert-un and sbert-un fine-tuned for sub-task 1. we present the results in we present the test set results in ## conclusion and future work in this paper, we elaborate on our team lipi's approach toward solving the finsim-4-esg sub-tasks. as per the official report, out of 28 registered teams, 6 and 8 teams participated in sub-task 1 and 2 respectively. for sub-task 1, our team ranked 4 th whereas for sub-task 2, our team ranked 3 rd . in future, we would like to collect more data and work towards improving the model performance. developing a user-friendly tool for assigning terms to concepts and automatically evaluating the sustainable aspect of financial texts are other directions of future work. tim nugent, nicole stelea, and jochen l. leidner. 2020. detecting esg topics using domain-specific language models and data augmentation approaches. nils reimers and iryna gurevych. 2019. sentence-