Break it Down into BTS : Basic, Tiniest Subword Units for K orean (2022)

Abstract: We introduce Basic, Tiniest Subword (BTS) units for the Korean language, which are inspired by the invention principle of Hangeul, the Korean writing system. Instead of relying on 51 Korean consonant and vowel letters, we form the letters from BTS units by adding strokes or combining them. To examine the impact of BTS units on Korean language processing, we develop a novel BTS-based word embedding framework that is readily applicable to various models. Our experiments reveal that BTS units significantly improve the performance of Korean word embedding on all intrinsic and extrinsic tasks in our evaluation. In particular, BTS-based word embedding outperforms the state-of-theart Korean word embedding by 11.8% in word analogy. We further investigate the unique advantages provided by BTS units through indepth analysis.

## introduction distributed word representations, known as word embeddings, play a fundamental role in natural language processing (nlp). with their capability to formulate semantic and syntactic relationships between words, word embeddings serve as a useful feature for various nlp applications, such as machine translation @xcite , sentence classification @xcite , and named entity recognition @xcite . initial approaches to word embeddings, such as word2vec @xcite and glove @xcite , represent each word in a vocabulary using distinct vectors. to improve the quality of word representations, many approaches have considered the internal structure of words with subword information. @xcite @xcite @xcite . for non-english languages, prior studies @xcite have demonstrated that word embeddings should be designed to consider their linguistic structures. in particular, korean has a unique and complex linguistic structure as a morphologically rich and agglutinative language @xcite . in the realm of korean word embedding, decomposing a word into jamo letters foot_0 is believed to be the best feasible option for capturing syntactic and semantic information of korean words, which has been empirically supported by @xcite . however, previous approaches have overlooked the academic background of the korean alphabet, hangeul. according to hunminjeongeum foot_1 , hangeul involves five basic consonants (ㄱ, ㄴ, ㅁ, ㅅ, ㅇ) and three basic vowels ( • , ㅡ, ㅣ). other letters are made from the basic letters by adding strokes or combining together @xcite . this unique invention principle of hangeul indeed helps to understand the korean linguistic structure. although this principle is wellrecognized by the korean linguistic community, its application to korean nlp is yet to be explored. in this paper, we firstly bring attention to the overlooked background of hangeul for korean nlp. we propose novel decomposition units to analyze the korean linguistic structure called basic, tiniest subword (bts) units, based on the invention principle of hangeul. bts units involve the basic consonants and vowels of hangeul and represent all jamo letters by themselves or through expansion. we implement this concept by developing a simple-yet-effective bts-based word embedding framework. to verify the effectiveness of bts units, we evaluate our bts-based embeddings on the intrinsic tasks of word analogy and similarity, and the extrinsic task of sentiment analysis. the results demonstrate that our embeddings consistently outperform the state-of-the-art korean word embeddings on all evaluation tasks. through our in-depth analysis, we verify that bts units improve both syntactic and semantic information in korean words, leading to consistent performance improvements. we summarize our contributions as follows: ## method -bts in this section, we introduce bts units for korean and present the implementation details. ## experiments in this section, we demonstrate the effectiveness of bts units through the evaluation of intrinsic and extrinsic tasks. see appendix c for details on datasets in these experiments. ## conclusion until now, among the korean nlp community, various efforts have been made to reflect the unique writing system of korean to nlp systems, but they have overlooked how hangeul was created. in this paper, we have developed an unprecedented approach to korean nlp inspired by the invention principle of hangeul, leading to novel bts units for analyzing the korean linguistic structure. instead of relying on the conventional approach of using korean consonant and vowel letters, we have formed letters from bts units by adding strokes or combining them together. we have shown that our bts units effectively improve the quality of korean word embeddings, which is a fundamental and versatile component of nlp models. we have demonstrated the efficacy of bts units for both intrinsic and extrinsic tasks by outperforming existing korean word embeddings methods on whole tasks in our evaluation. we hope that our bts will light the way up like dynamite foot_6 for future research on korean nlp.