Enhancing ESG Impact Type Identification through Early Fusion and Multilingual Models (2023)

Abstract: In the evolving landscape of Environmental, Social, and Corporate Governance (ESG) impact assessment, the ML-ESG-2 shared task proposes identifying ESG impact types. To address this challenge, we present a comprehensive system leveraging ensemble learning techniques, capitalizing on early and late fusion approaches. Our approach employs four distinct models: mBERT, FlauBERT-base, ALBERT-base-v2, and a Multi-Layer Perceptron (MLP) incorporating Latent Semantic Analysis (LSA) and Term Frequency-Inverse Document Frequency (TF-IDF) features. Through extensive experimentation, we find that our early fusion ensemble approach, featuring the integration of LSA, TF-IDF, mBERT, FlauBERT-base, and ALBERT-base-v2, delivers the best performance. Our system offers a comprehensive ESG impact type identification solution, contributing to the responsible and sustainable decision-making processes vital in today’s financial and corporate governance landscape.

## introduction in the rapidly evolving landscape of finance and corporate governance, environmental, social, and corporate governance (esg) considerations have gained unprecedented prominence @xcite . investors, stakeholders, and businesses increasingly recognize the importance of esg factors in decision-making processes, portfolio management, and corporate strategy @xcite . the "e" in esg shines a spotlight on environmental sustainability, compelling companies to address pressing issues such as climate change and resource conservation. recognizing that eco-friendly practices are ethical and financially savvy, businesses are embracing environmental responsibility as a key driver of longterm success @xcite . simultaneously, the "s" in esg emphasizes social responsibility, pushing corpora-tions to foster diversity, inclusivity, and community engagement @xcite . companies that prioritize these social factors not only enhance their reputation but also attract top talent and build resilience in a socially conscious world. finally, the "g" underscores the importance of sound corporate governance, providing the foundation for trust and stability in financial markets. investors now realize that transparent decision-making, ethical conduct, and effective risk management are essential elements for ensuring a company's long-term viability @xcite . in this context, the ml-esg-2 shared task is pivotal in facilitating a deeper understanding of how esg information can be extracted and utilized from the vast sea of textual data, contributing to the broader endeavor of responsible and sustainable finance. this specific task of esg impact type identification, where machine learning models classify news articles or sentences as "opportunity", "risk", or "cannot distinguish", carries profound significance. for investors, this knowledge translates into well-informed decisions with the potential to impact portfolios significantly @xcite . positive esg signals can guide investments toward companies poised for sustainable growth while identifying esg risks enables proactive risk mitigation. in the corporate sphere, grasping the esg implications of textual data like news articles and annual reports fosters strategic decision-making and risk management. companies can leverage positive esg news to enhance their sustainability efforts and attract responsible investors while swiftly addressing negative news to minimize reputational damage and regulatory challenges @xcite . furthermore, automated text analysis tools expedite these processes, enabling scalable and efficient esg information extraction. as global issues like climate change, social inequality, and ethical governance become increas-ingly prominent, harnessing esg information from textual sources is a means to hold corporations accountable. by scrutinizing relevant text like news articles or annual securities reports for their esg impact, stakeholders can encourage responsible behavior, promote sustainable practices, and ensure that businesses prioritize societal and environmental concerns @xcite . this task not only empowers decision-makers with the ability to navigate the complexities of esg information but also contributes to a broader cultural shift towards responsible investment and corporate citizenship. in essence, esg information extraction from text represents a vital step towards a more sustainable and equitable future, where financial decisions align with environmental and social responsibility goals. in natural language processing (nlp), text classification is a critical enabler for tasks like esg impact type identification, bridging the gap between the vast expanse of textual data and actionable insights. nlp techniques have been widely used in the financial domain @xcite . leveraging nlp techniques, machine learning models can autonomously categorize news articles or sentences into predefined esg impact types, such as "opportunity", "risk", or "cannot distinguish". these models harness the power of feature extraction, supervised learning, feature engineering, and even advanced transfer learning from pre-trained language models to accurately assess the implications of textual content on a company's esg performance. the synergy between nlp and esg, impact type identification, is instrumental in empowering stakeholders, including investors and corporations, to efficiently navigate the complexities of esg information, make informed decisions, and contribute to a more sustainable and responsible financial and corporate governance landscape @xcite . this paper presents the system description of our submission to ml-esg-2, a shared task hosted by the finnlp workshop in conjunction with ijcnlp-aacl 2023. in this task, we delve into the critical intersection of natural language processing (nlp) and esg impact type identification. ## system description in this section, we provide a detailed overview of our system architecture and methodology for the ml-esg-2 shared task on esg impact type identification. our approach is founded on the principles of ensemble learning, utilizing both early fusion and late fusion techniques to harness the strengths of multiple language models and feature representations. in order to approach this task, we use four different models. ## results in the ml-esg-2 shared task for esg impact type identification, we evaluated the performance of our system across multiple languages using different ensemble configurations and fusion techniques. the micro f1, macro f1, and weighted f1 scores offer a comprehensive assessment of our system's effectiveness in capturing nuances within different linguistic contexts. our system secured notable rankings in the shared task, claiming the fourth position for english, the third position for both japanese and chinese, and the fifth position for the french language. we employed various ensemble configurations, including late fusion models combining flaubert, mbert, and albert, incorporating tf-idf and latent semantic analysis (lsa) features. the results demonstrate significant improvements when transitioning to early fusion techniques, particularly when combining tf-idf, lsa, and language model representations, as shown in in the english language, our best early fusion ensemble achieved a micro f1 score of 0.9633, a while our ensemble system demonstrates commendable performance in english (as shown in ## conclusion in the rapidly evolving finance and corporate governance landscape, the ml-esg-2 shared task has presented a unique opportunity to explore esg impact type identification. our comprehensive system leverages the collective intelligence of multiple language models and feature representations through advanced ensemble learning techniques, resulting in high performance. while our system excels in english, it faces hurdles in accurately identifying esg impact types in japanese, chinese, and french languages. these challenges underscore the importance of further research and fine-tuning to adapt our approach to diverse linguistic contexts.