Text Classification through Glyph-aware Disentangled Character Embedding and Semantic Sub-character Augmentation (2020)

Abstract: We propose a new character-based text classification framework for non-alphabetic languages, such as Chinese and Japanese. Our framework consists of a variational character encoder (VCE) and character-level text classifier. The VCE is composed of a β-variational auto-encoder (β -VAE) that learns the proposed glyph-aware disentangled character embedding (GDCE). Since our GDCE provides zero-mean unit-variance character embeddings that are dimensionally independent, it is applicable for our interpretable data augmentation, namely, semantic sub-character augmentation (SSA). In this paper, we evaluated our framework using Japanese text classification tasks at the document- and sentence-level. We confirmed that our GDCE and SSA not only provided embedding interpretability but also improved the classification performance. Our proposal achieved a competitive result to the state-of-the-art model while also providing model interpretability.

## introduction some asian languages (e.g., chinese and japanese) use glyphs to give visual meaning to characters. for example, the following japanese characters have a common form of "辶," which is a sub-character meaning of the related word road: "迫" (approach: come near the destination by road) and "追" (follow: track the road). in consideration of these characteristics of the language, several glyph-aware natural language processing (nlp) models have been proposed @xcite @xcite . these deep-learning-based models train input text as a sequence of character images and learn character embeddings from the images. in general, the interpretability of the nlp model is important in terms of its reliability, as well as providing the required performance for the task. if imaged-based models can learn these subcharacters in a way that is interpretable, it helps greatly in improving the overall interpretability of the models. in terms of improving the interpretability of models, disentangled representation learning method has received a great deal of attention in recent years, such as infogan @xcite and β-variational auto-encoder (β-vae) @xcite . this learning method transforms the input data into low-dimensional representations that are independent of each other while still retaining the important content. although it has been actively discussed in the field of computer vision, there are few applications in the field of nlp. in terms of ensuring model robustness, data augmentation is necessary and essential in machine learning today. with regard to this desirable feature, glyph-aware embedding (i.e., image-based character embedding) allows data augmentation without contextual consideration, such as word dropout @xcite and wildcard training @xcite . simple data augmentation based on dropout does not consider the features of the input space. if the nlp method based on glyph-aware embedding is highly interpretive, such as a disentangled representation, an effective data augmentation method can be achieved. this improves not only the robustness of the model but also its interpretability. in this paper, we propose a general-purpose text classification framework that gives interpretability to data augmentation for image-based glyph-aware character embedding, which has the various advantages mentioned above. the framework consists of two novel methods: (1) glyph-aware disentangled character embedding (gdce) and (2) semantic subcharacter augmentation (ssa). each method has the following simple but effective features: our framework improves the interpretability of character embedding by the gdce, and the ssa provides interpretable data augmentation suitable for the gdce. we verified the text classification ability of our proposed framework using japanese text classification tasks. 1 2 related work ## methodology in this paper, we propose a new character-based text classification framework that includes a new character embedding method, consisting of glyphaware disentangled character embedding (gdce) and semantic sub-character augmentation (ssa). figure 1 shows an overview of the proposed text classification framework. ## results and discussion first, as a comparison of embedding methods, we compared the gdce with the conventional caebased embedding @xcite . second, as a comparison of data augmentation methods for image-based character embedding, we also compared the proposed ssa with the conventional wt, the latter of which has reported excellent results but offers no way of interpreting the change on the embedding space. ## conclusion we propose a new character-based text classification framework for non-alphabetic languages. as the name implies, the combination of our gdce and ssa not only provided embedding interpretability but also improved the text classification performance. our gdce provided better text classification performance than conventional cae-based character embedding without data augmentation. finally, our framework achieved a competitive result to the conventional state-of-the-art cae-based embedding with wt while also providing model interpretability.