Complex Word Identification in V ietnamese: Towards V ietnamese Text Simplification (2022)

Abstract: Text Simplification has been an extensively researched problem in English, but has not been investigated in Vietnamese. We focus on the Vietnamese-specific Complex Word Identification task, often the first step in Lexical Simplification (Shardlow, 2013). We examine three different Vietnamese datasets constructed for other Natural Language Processing tasks and show that, like in other languages, frequency is a strong signal in determining whether a word is complex, with a mean accuracy of 86.87%. Across the datasets, we find that the 10% most frequent words in many corpus can be labelled as simple, and the rest as complex, though this is more variable for smaller corpora. We also examine how human annotators perform at this task. Given the subjective nature, there is a fair amount of variability in which words are seen as difficult, though majority results are more consistent.

## introduction text simplification is a task that focuses on improving the readability and understandability of text while preserving the original content and meaning. text simplification applications have been shown to benefit a variety of target audiences, including readers with low-literacy levels @xcite , non-native speakers @xcite , language learners @xcite , deaf people @xcite , people with reading comprehension problems such as aphasia @xcite and dyslexia @xcite , and people with autistic spectrum disorder @xcite . it is also a useful preprocessing step for other nlp tasks, including parsing @xcite , information extraction @xcite , and question generation @xcite . although significant progress has been made in text simplification in multiple languages, including english @xcite @xcite , spanish @xcite , portuguese @xcite , japanese @xcite , korean @xcite , and italian @xcite , the problem remains a relatively new area of research in vietnamese, a language spoken by over 70 million people @xcite in vietnam, the south east asia region, france, australia, and the united states. sentence splitting has been conducted for the vietnamese -english machine translation task @xcite , which can be helpful as an initial step for text simplification, but no further work has been recorded. other tasks in vietnamese have been explored, from core problems such as dependency parsing, word segmentation, and part-of-speech parsing to more recent ones such as sentiment analysis, automatic speech recognition, and question answering. 1 text summarization is the most closely related task to text simplification that has been attempted in vietnamese. progress on the specific task of complex word identification in vietnamese has not been reported so far. although the terms complex words and simple words have appeared in literature on the word segmentation task, such as in @xcite , @xcite , they refer to the length of each word (whether they are monosyllabic or polysyllabic words such as compound and reduplicative words) rather than the understandability and readability of each word in the context of text simplification. we implement two approaches to solve the complex word identification task in vietnamese: frequency-based and classification-based with support vector machines. we conclude with an experiment involving human annotators to predict the suitability of our datasets for this task. ## characteristics of vietnamese the characteristics presented in this section are extracted from @xcite and @xcite . ## data we conduct two experiments across three vietnamese corpora of various sizes extracted from different domains. we obtain a simple word list, a stopword list, and use the two lists to extract three complex word lists from the three corpora for evaluation purposes. the simple and complex wordlists for the three corpora are available online. foot_1 ## methods for each dataset, we have the simple word list and the list of unique complex words. this creates three complex word identification tasks to identify whether a word is simple or complex. we examine two approaches for the this task: frequency threshold and feature-based using support vector machines. ## experiments we evaluate the performance of the two approaches on the three corpora based on overall accuracy and precision, recall, and f1 (for identifying simple words). ## human annotation to quantify the quality of the datasets for the automated cwi task in vietnamese, three participants were asked to manually classify 199 words as simple or complex, with 100 words randomly picked from the simple words list and 99 words from the readability complex word list. the words were presented by themselves without any additional context. all participants were native vietnamese speakers pursuing a college degree in the united states. the instructions were provided in vietnamese, in which an example of one simple word and one complex word is demonstrated. the participants were reassured that there are no right or wrong answers, encouraged to use their intuition when making the decision, and to label a word as complex when in doubt. results are reported under two circumstances: a word gets assigned a label during this collective classification process if (a) the label is chosen by all 3 of the participants and (b) the label is chosen by a majority (i.e., 2 out of 3) participants. ## discussion frequency is an overpowering signal in determining whether a word is complex or simple as shown by the accuracy, precision, recall and f1 scores of the frequency threshold experiment, which are all are greater than 0.8 (see we analyze three corpora to try understand how consistent frequency is. for the larger corpora, it is surprisingly consistent with words in the top 10% most frequent words as simple. for smaller corpora this is more varied. there are some shortcomings in the datasets that may affect the performance. there exist words in the simple word list that are acronyms that may be obvious to a certain target audience but not for the majority of vietnamese readers (such as "ubnd", which stands for "uỷ ban nhân dân" (people's committee)), and can mean different things in different contexts (such as tp, which can mean "thành phố" (city) or "thành phần" (ingredient)). the cluster and classification datasets also involve foreign words, especially english words, that can add noise to the data. support vector machines are also explored to incorporate additional information into the prediction task. three more features are added in addition to frequency for the svm model: number of syllables, number of characters, and number of characters and diacritics. we hypothesize that longer words and words with more diacritics will be harder to recognize and understand. for example, "cỏ cây" (trees and plants) can be perceived as a simpler word to understand than "đường sá" (streets). however, results show that using svm with more features do not improve the performance of the classification task compared to using a frequency threshold. in fact, we observe a decline in precision (from 92.40% to 81.95%) and f1 score (from 94.73% to 89.39%) on the readability dataset. this can be explained by the fact that surface-level word features do not necessarily make the word more complex in terms of readability and understandability. coming back to our example, although the former word "cỏ cây" is shorter and has fewer diacritics, it can also be simpler because both words have clear meanings ("cỏ" -grass and "cây" -plant), while the second syllable of the latter word "đường sá" is a sino-vietnamese word that may not be clearly decipherable. because of this reason, "trung kiên" (loyal), which is a sino-vietnamese word, can be viewed as more complex than "phương hướng" (direction), which is a more common word. again, this particular example shows that frequency gives a very strong signal. the human annotation experiment shows a great difference between labeling based on the agreement between all three annotators or between the majority of annotators (2 out of 3 annotators). the accuracy and recall scores nearly double, and the precision score is 1.0 for the majority vote. this means that the majority of annotators' labeling of complex words is consistent with the data we obtain, which can indicate the suitability of the readability dataset for the cwi training purposes. ## conclusions and future work several next steps can be taken beyond this project: more salient features: features that describe a word's characteristics beyond its pronunciation can be helpful to obtain a better classification performance. some examples include sense count (number of entries in a dictionary for example), synonym count, and word type (whether the word is loan word). context: the approach we explore predicts words as simple/complex regardless of their context. in some cases, the context information can help provide additional information and additional features to help the identification @xcite .