ESG Impact Type Classification: Leveraging Strategic Prompt Engineering and LLM Fine-Tuning (2023)

Abstract: In this paper, we describe our approach to the ML-ESG-2 shared task, co-located with the FinNLP workshop at IJCNLP-AACL-2023. The task aims at classifying news articles into categories reflecting either “Opportunity” or “Risk” from an ESG standpoint for companies. Our innovative methodology leverages two distinct systems for optimal text classification. In the initial phase, we engage in prompt engineering, working in conjunction with semantic similarity and using the Claude 2 LLM. Subsequently, we apply fine-tuning techniques to the Llama 2 and Dolly LLMs to enhance their performance. We report the results of five different approaches in this paper, with our top models ranking first in the French category and sixth in the English category.

## introduction natural language processing (nlp) is pivotal in the finance sector, extracting valuable semantic information from vast unstructured data sources like reports, news, and social media. this extraction is crucial for identifying scenarios and analyzing risks, especially in the rising field of environmental, social, and governance (esg) considerations, as the global economy shifts towards sustainability. financial markets and investors play a central role in supporting companies with strong esg principles amidst growing interest in corporate sustainability performance. using nlp, investors can swiftly and efficiently analyze sustainability reports and news, simplifying the traditionally complex manual process. with automatic text classification and sentiment analysis, nlp models identify crucial esg topics and sentiments, saving time and resources while enabling informed and timely sustainable investment decisions. ## multi-lingual esg impact type identification shared task the "multi-lingual esg impact type identification (ml-esg-2)" shared task, introduced by @xcite , aims to discern the type of esg impact news articles exert on companies, focusing on articles written in chinese @xcite ), english, french, and japanese @xcite ). each article is systematically categorized as either "opportunity", "risk", "cannot distinguish", or in the case of japanese texts, "positive", "negative", or "n/a", serving as indicative labels of potential impacts. this task is a sequel to the initial ml-esg shared task @xcite and is meticulously designed under the rigorous esg rating guidelines provided by msci, seeking the development and evaluation of systems adept at accurately classifying articles into specific impact types. within resource and time constraints, our team focused on classifying esg impacts in english and french articles with comprehensive strategies, while solely applying prompt engineering to japanese articles, demonstrating a tailored approach to multilingual classification. ## techniques explored this section elucidates the various techniques employed during the competition. our discussion will not only delve into the specifics of each method utilized but also provide a comprehensive understanding of the applied techniques. ## lessons learned several key lessons emerged from our work. we omitted translating the french dataset, and training on non-translated data didn't enhance classification scores for french samples. this raises questions about the potential impact of initially translating news articles and integrating them into templates, as well as the appropriateness of the french and japanese translations in the prompt templates. further experimentation and investigation are warranted to clarify these issues. second, while our approach was potent, it was also resource-intensive. the demanding nature of llms prompts consideration of using more cost-effective alternatives. additionally, considering the superior performance often exhibited by fine-tuned pre-trained models for discriminative tasks, we should explorre the use of models like roberta. furthermore, future efforts should also involve experimenting with diverse and concise prompt templates. ## conclusion this paper presents a nuanced approach developed for the finnlp 2023 ml-esg-2 shared task, with a focus on classifying news articles into "opportunity" or "risk" categories for companies from an esg perspective. our dual-strategy method seamlessly integrated prompt engineering with semantic similarity and subsequent fine-tuning of llms, notably enhancing their performance. with our models, we achieved a commendable 6th position in the english category and 1st in the french category. particularly noteworthy was the superior performance demonstrated by the fine-tuned llama 2 model, highlighting its promising application potential in esg texts. our findings offer valuable insights, contributing significantly to the field of esg impact classification and suggesting promising directions for future research in this domain. in proceedings of the 32nd acm international conference on information and knowledge management (cikm'23). thomas wolf, lysandre debut, victor sanh, julien chaumond, clement delangue, anthony moi, pierric cistac, tim rault, remi louf, morgan funtowicz, joe davison, sam shleifer, patrick von platen, clara ma, yacine jernite, julien plu, canwen xu, teven le scao, sylvain gugger, mariama drame, quentin lhoest, and alexander rush. 2020. transformers: state-of-the-art natural language processing. in proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pages 38-45, online. association for computational linguistics.