Constructing a K orean Named Entity Recognition Dataset for the Financial Domain using Active Learning (2020)

Abstract: The performance of deep learning models depends on the quality and quantity of data. Data construction, however, is time- consuming and costly. In addition, when expert domain data are constructed, the availability of experts is limited. In such cases, active learning can efficiently increase the performance of the learning models with minimal data construction. Although various datasets have been constructed using active learning techniques, vigorous studies on the construction of Korean data on expert domains are yet to be conducted. In this study, a corpus for named entity recognition was constructed for the financial domain using the active learning technique. The contributions of the study are as follows. (1) It was verified that the active learning technique could effectively construct the named entity recognition corpus for the financial domain, and (2) a named entity recognizer for the financial domain was developed. Data of 8,043 sentences were constructed using the proposed method, and the performance of the named entity recognizer reached 80.84%. Moreover, the proposed method reduced data construction costs by 12–25%

## introduction rapid advancements in the field of artificial intelligence have contributed to their increased use in various fields. in the field of natural language processing, deep learning models have demonstrated a higher performance than humans in benchmark tests such as glue @xcite and squad @xcite . such deep learning models require expert knowledge and time, in addition to the cost involved in computing power for model training and data construction. therefore, it is difficult to apply deep learning models in industries readily. in particular, it is not easy to reduce the costs of learning data construction when compared to that of hardware because it requires human effort. in addition, specific fields, such as the financial domain, requires workers with expert domain knowledge and, therefore, requires more time and is more expensive. various studies have been conducted in areas such as unsupervised learning @xcite and crowdsourcing @xcite to reduce the data construction costs of expert domains. the number of data constructed through such methods is larger than the number of annotations created by experts, but the quality of the constructed data is relatively low. active learning is a technique used by experts to construct data, with the goal of achieving optimum performance with fewer data. the active learning technique determines which data are to be learned first by interacting with users when learning data are limited. the goal of the interaction is to identify data that can be used to improve the performance of the learning model efficiently. during the early learning stages, the model is trained with a small number of learning data, and it submits queries requesting additional data that can efficiently improve its performance, thereby reducing development costs. the active learning technique is known to be effective in ## construction of a named entity corpus for the financial domain using active learning in this study, the ner corpus was constructed by applying active learning to the financial domain. ner identifies and classifies plo entities in a given sentence. in the financial domain, this is an annotation task that distinguishes financial and general institutions and requires expert knowledge for tagging named entities of financial products (e.g., funds) and financial theories and phenomena. however, there are practical limitations such as the requirement for experts who perform the annotation tasks and the costs of data construction. therefore, in this study, we investigate whether the performance of a model can be improved by minimizing the annotation tasks using active learning to handle such problems. figure 1 presents the overall diagram of the named entity corpus construction task for the financial domain using active learning. the task sequence is as follows. (1) first, 500 arbitrary data, which are to be annotated, are selected from the financial domain raw corpus. (2) a named entity annotation task is performed on the corresponding data by an annotation platform. (3) the created data are then used to train the model. ## conclusions in this study, a ner corpus for the financial domain was constructed through active learning. the active learning technique could rapidly and efficiently improve the performance of the model. tag sets that could not be easily analyzed by the model were constructed first, and it was found that more meaningful quality datasets were constructed through machine learning. from the 8,043 sentences constructed, a named entity recognizer with a f1 performance of 80.84% was developed for the financial domain. further, it was experimentally verified that the active learning technique could provide a cost reduction of 12.5-25%.