ESG -Kor: A K orean Dataset for ESG -related Information Extraction and Practical Use Cases (2024)

Abstract: With the expansion of pre-trained language model usage in recent years, the importance of datasets for performing tasks in specialized domains has significantly increased. Therefore, we have built a Korean dataset called ESG-Kor to automatically extract Environmental, Social, and Governance (ESG) information, which has recently gained importance. ESG-Kor is a dataset consisting of a total of 118,946 sentences that extracted information on each ESG component from Korean companies’ sustainability reports and manually labeled it according to objective rules provided by ESG evaluation agencies. To verify the effectiveness and applicability of the ESG-Kor dataset, classification performance was confirmed using several Korean pre-trained language models, and significant performance was obtained. Additionally, by extending the ESG classification model to documents of small and medium enterprises and extracting information based on ESG key issues and in-depth analysis, we demonstrated potential and practical use cases in the ESG field.

## introduction pre-trained language models are exhibiting astonishing performances in various natural language processing (nlp) tasks, including classification, question answering (qa), machine translation, summarization, and conversation generation, and are currently being utilized in numerous fields. moreover, pre-trained language models specialized domains, such as finbert @xcite , med-bert @xcite , and legal-bert @xcite , which train the text of the respective domains, have emerged and shown superior performance in solving problems of each domain @xcite . however, these pre-trained language models show their limits when * corresponding author applied to more detailed subfields, leading to an increased significance of datasets from such detailed subfields @xcite . despite this, the construction of datasets from these detailed subfields necessitates the involvement of relevant experts, thereby requiring more time and costs and resulting in limited construction of related datasets. moreover, most publicly available datasets and pretrained language models are built based on english, limiting their applicability for performing specific tasks in non-english languages. while there have been pre-trained language models released for supporting multiple languages @xcite , they do not perform as well as the models based on a single language. consequently, studies are being conducted to build datasets for specific languagebased tasks by translating pre-constructed english data into an other language @xcite . nevertheless, these exhibit limited performance compared to english-based pre-trained language models and are especially difficult to use in highly specialized domains, prompting ongoing research into constructing datasets for specialized domains in diverse languages @xcite . in this paper, we introduce esg-kor, a largescale korean dataset that encompasses environmental, social, and governance (esg) information. considering the importance of ethical management, social issues such as racial and gender discrimination, and environmental problems like climate change, esg is essential for a sustainable society. the importance of esg is increasing globally as a measure of non-financial corporate activities. esg information is mainly provided as text without a standardized format, making it very challenging to obtain only the necessary esg information from a vast amount of data @xcite , and small-scale companies do not provide reports for the disclosure of esg information. therefore, the potential benefits are significant if esg information can be extracted from various text data through an esg-based language model. for this, we used sustainability reports related to esg in this study to build a korean esg dataset. we constructed a dataset of 118,946 sentences, manually labeled by three annotators possessing relevant knowledge. the annotators were conducted based on objective rules provided by esg rating providers. we verified the performance by fine-tuning the existing pre-trained language models with the constructed esg-kor dataset and validated the effectiveness and applicability of esg-kor. additionally, by extending our methodology and utilizing additional techniques, we demonstrate potential and practical use cases, such as the expansion of esg into the small and medium enterprises (smes) environment and in-depth analysis of corporate esg based on esg key issues. our contributions are as follows: esg-kor is the only korean-based professional nlp benchmark dataset manually labeled in the field of esg. the entire construction process of esg-kor is illustrated in figure 1 . this dataset can be leveraged by diverse stakeholders, and the data construction process can be used as a benchmark for constructing esg datasets in various languages. we release the esg-kor dataset using a data repository to advance esg research. we have created temporary accounts with guaranteed anonymity to provide some dataset samples. the dataset can be accessed at https://github.com/nowzer0/ esg-kor . ## related work for pre-trained language models to be effectively utilized in diverse nations and highly specialized domains, datasets and related research tailored to different languages and specialized domains are essential. moreover, despite the increasing importance of esg, research leverages machine learning techniques in the field of esg is limited @xcite . we present research focused on specialized domains and minority languages, as well as research related to esg. ## experiments 4.1 experimental settings to validate the effectiveness and applicability of esg-kor, we used models based on bert @xcite . since esg-kor is a korean dataset, we utilized popular korean pre-trained language models such as kobert (sktbrain, 2019), dis-tilkobert @xcite , kb-albert (kb-ai-research, 2021), and klue-roberta @xcite . we fine-tuned these pre-trained models using the esg-kor dataset. the sentence x of the esg-kor dataset is represented by the [cls] token through the layers of the model, and the model calculates the probability of belonging to classes c using [cls] token c and classification layer w , as shown in equation 1. the model is trained to minimize the cross-entropy loss, as shown in equation 2 . we set the batch size to 64, maximum length to 256, optimizer to adamw, epsilon to 1e-08, and the learning rate of kobert, distilkobert, kb-albert, and klue-roberta to 1e-04, 5e-04, 5e-05, and 5e-05, respectively. the dataset was split into 70% training, 15% validation, and 15% test data. all experiments were conducted identically on nvidia geforce rtx 3090 4ea. the model parameters and the time taken for each experiment are provided in the appendix a.1. ## use cases we propose two potential and practical cases to demonstrate additional applications of the esg-kor dataset and model. the first shows the scalability of esg in the sme environment using a model trained on the esg-kor dataset. the second case shows a method that utilizes the trained model and additional methodologies for in-depth analysis of corporate esg based on esg key issues. ## conclusions we manually constructed the high-quality, largescale esg-related esg-kor dataset to effectively utilize pre-trained models in the esg field to extract and classify esg information. we collected and preprocessed sustainability reports containing esg information to construct the dataset and annotated them manually according to various esg criteria. the esg-kor dataset consists of 118,946 sentences, focusing on the automobile, aviation, electronics, finance, heavy industry, steel industry, and it sectors. we conducted experiments using korean pre-trained language models to verify the effectiveness and applicability of the esg-kor dataset. as a result of the experiments, the klue-roberta-large model recorded accuracies of 85.87% in multiclass classification and 86.67% in binary classification, confirming that the model appropriately learned and performed the task. this suggests that applying the esg-kor dataset to existing pre-trained language models allows for proper extraction and classification of relevant information in the field of esg of korea, a specialized domain in a particular country. we also show use cases of how our methodology can be extended to the sme environment and the extraction of esg key issues. by collecting sme esg reports and labeling them, we evaluated the performance through a model trained with the esg-kor dataset, achieving an accuracy of 86.72%. furthermore, by combining our model with nlp techniques to extract esg information based on esg key issues, we validated its usefulness and potential in the esg field through human evaluation. in countries that use minority languages, research in specialized domains like esg is limited due to the lack of datasets. many datasets for minority languages are constructed by translating major languages, but translating specialized domains is challenging, and its performance is limited. various countries are currently publishing documents such as sustainability reports that contain esg information. therefore, the entire construction process of the esg-kor dataset can serve as a benchmark for building esg datasets in various countries. furthermore, our approach can contribute to various esg stakeholders, including researchers and practitioners, and can be utilized in multiple countries.