Multi-Lingual ESG Impact Type Identification (2023)

Abstract: Assessing a company’s sustainable development goes beyond just financial metrics; the inclusion of environmental, social, and governance (ESG) factors is becoming increasingly vital. The ML-ESG shared task series seeks to pioneer discussions on news-driven ESG ratings, drawing inspiration from the MSCI ESG rating guidelines. In its second edition, ML-ESG-2 emphasizes impact type identification, offering datasets in four languages: Chinese, English, French, and Japanese. Of the 28 teams registered, 8 participated in the official evaluation. This paper presents a comprehensive overview of ML-ESG-2, detailing the dataset specifics and summarizing the performance outcomes of the participating teams.

## introduction in the rapidly shifting global business milieu, relying solely on traditional financial metrics to gauge companies is no longer adequate. the imperative for companies to make meaningful contributions to society and the environment, underpinned by robust governance, has intensified. these principles, often denoted as environmental, social, and governance (esg) factors, have surged in significance for stakeholders from investors to consumers. evaluating a company's adherence to these principles is crucial not only for its sustainable growth but also for setting the benchmark for corporate responsibility in the modern era. the increasing need to precisely appraise and contrast esg ratings across corporations has ignited extensive research and discourse. the influence of news and contemporaneous events on these ratings offers a captivating subject for study. this dynamic environment necessitates a resilient and malleable approach. while numerous rating systems exist, our focus leans towards the msci esg rating guidelines as the foundation for the multi-lingual esg (ml-esg) shared task series. in this backdrop, the ml-esg shared task series was initiated to cultivate a deeper understanding of news-driven esg ratings. in a world growing more intertwined, the imperative of incorporating multiple languages in these assessments is paramount. in the inaugural ml-esg, our focus was on pinpointing esg issues @xcite . progressing to the second iteration, ml-esg-2 hones in on the classification of impact types, featuring datasets in chinese, english, french, and japanese. 1 the objective of ml-esg-2 is discerning whether news pieces indicate risks or opportunities for company operations through an esg lens. this paper aims to shed light on the methodologies, datasets, and discoveries of ml-esg-2. through the concerted effort of diverse teams, we aspire to offer a comprehensive perspective on the latest advancements in multi-lingual esg impact type identification. our endeavor is to equip both scholars and industry professionals with insights into this emerging domain. ## dataset and task setting ## experimental results tables 3, 4, 5, and 6 report the performances of ml-esg-2 participants on english, french, chinese, and japanese datasets, respectively. drawing from the outcomes of both english and french datasets, it becomes evident that finetuning roberta using these datasets produces the most optimal results, as documented in @xcite . when juxtaposed with the outcomes of fine-tuning llama-2 (spevft) (winatmoko and septiandri, 2023), there is a pronounced disparity in performance for english, though this difference is less marked for 7 https://huggingface.co/humarin/chatgpt_ paraphraser_on_t5_base in the evaluation of the chinese dataset, team lipi @xcite achieved the highest performance during the formal evaluation. their strategy employed all datasets available in ml-esg-2 to facilitate data augmentation. the predictions from their model are generated using fin-bert with english inputs. a noteworthy observation from their study is the superior performance achieved using translated inputs compared to the original data. subsequent to the formal evaluation, team 231 @xcite embarked on further exploration. their findings indicate that a simple conversion from traditional chinese to simplified chinese can enhance performance. furthermore, they observed that utilizing a summary from chatgpt yields better results than employing the complete news content. however, there were contrasting findings among the participants: while team 231's experiments suggest that tf-idf fea in the evaluation of the japanese dataset, team lipi's performance emerged superior compared to other participating teams. this notable advantage is attributed to their t5-based paraphrasing module, which demonstrably bolstered the system's efficacy. notably, despite the evident disparity in document genres -transitioning from news to annual securities reports -their methodology retained its effectiveness. this suggests that certain topics, especially those pertinent to "opportunity" (positive) and "risk" (negative) annotations, exhibit shared characteristics across different document genres and across languages to some degree. team finnlu adopted an early ensemble fusion approach, which proved to be equally effective for the japanese context. the inclusion of lsa features also enhanced their performance metrics, mirroring the positive outcomes observed in other languages. on the other hand, team spevft explored a prompt engineering approach tailored to the japanese language. they employed a sentence similarity method with training examples, revealing its efficacy, albeit to a limited extent. ## conclusion in this study, we provided an overview of the ml-esg-2 shared task featured at finnlp@ijcai-2023. our findings indicate that for the english and french datasets with paragraph-based annotations, the top-performing models were fine-tuned roberta or llama-2. conversely, for the chinese and japanese datasets that used article-based and sentence-based structures, translating the content into english yielded commendable outcomes. such findings bring forth pivotal questions. our investigation into baseline models suggests that their adeptness, even in their simplicity, might surpass certain specialized methods. additionally, it hints at the possibility that massive datasets might not be essential for fine-tuning (large) language models in esg impact type identification, particularly for languages like english and french. having delved into esg issue identification and esg impact type identification tasks, our next venture will be into esg impact duration inference under ml-esg-3. through the series of ml-esg shared tasks, we pave the way for a holistic approach to dynamic esg ratings based on news articles. anna polyanskaya and lucas fernández brillet. 2023. gpt-based solution for esg impact type identification. in proceedings of the sixth workshop on financial technology and natural language processing. le qiu, bo peng, jinghang gu, yu-yin hsu, and emmanuele chersoni. 2023. identifying esg impact with key information. in proceedings of the sixth workshop on financial technology and natural language processing. yu-min tseng, chung-chi chen, hen-hsen huang, and hsin-hsi chen. 2023. dynamicesg: a dataset for dynamically unearthing esg ratings from news articles.