GPT -based Solution for ESG Impact Type Identification (2023)

Abstract: In this paper, we present our solutions to the ML-ESG-2 shared task which is co-located with the FinNLP workshop at IJCNLP-AACL-2023. The task proposes an objective of binary classification of ESG-related news based on what type of impact they can have on a company - Risk or Opportunity. We report the results of three systems, which ranked 2nd, 9th, and 10th in the final leaderboard for the English language, with the best solution achieving over 0.97 in F1 score.

## introduction in an era characterized by increasing environmental, social, and governance (esg) awareness, investors are becoming increasingly conscious of such issues, and companies' esg performance affects their financial performance @xcite . it has been shown that esg-related news have become a significant driver of market volatility, as both good and bad news can have a considerable impact @xcite . ## related work following the trend for esg awareness, natural language processing (nlp) is commonly used to analyze texts, usually reports and news, related to these types of issues. new tools and data sets are being developed, going further than just being specific to the financial domain, e.g. esg-bert @xcite and dynamicesg @xcite . last year's shared task focused on esg taxonomy enrichment and sustainable sentence prediction @xcite . @xcite shows that gpt-based models, while producing impressive results, still fall behind domain-specific large language models (llms), such as finbert @xcite . in this work, we wanted to test this hypothesis, as the proposed task differs from sentiment analysis in the complexity of the relation between a given text and its label. we believe that gpt models can reason better in this task as they seem to be able to utilize the context (or factual knowledge) they already have about the world, thus being able to grasp complicated causation even in zero-and few-shot settings @xcite @xcite . ## data the data was provided by the organizers, see @xcite . the english training data set contained 808 entries, each entry consisting of url, news title, news content, and impact type (class). the classes were quite imbalanced, so for our first system's training we also used an additional 360 entries of the risk class from the french data set, automatically translated to english using deepl python library foot_0 . the dataset statistics are presented in ## results the organizers provided the micro-, macro-, and weighted averaged f1 scores (see as we can see, the few-shot approach outperforms the other two and reaches over 0.97 f1 score, ranking second among 21 total submissions for the english language. ## conclusions and further work we conducted several experiments, showing that even with limited data pre-trained llms are capable of achieving high scores (> 0.94 weighted f1) in risk vs. opportunity classification. we show that gpt outperforms finbert in both zero-in few-shot settings. for further work, we consider fine-tuning gpt and esg-bert models, while also exploring gpt's capabilities to reasonably explain its classification decisions in such a task, especially gpt-4. we also consider applying and evaluating the same approaches with the data in other languages, namely french, as even the top scores for it are at least 0.1 lower than for english. another exciting direction would be exploring alternative translation models such as gpt-3.5, gpt-4, and flan-t5 @xcite .