C hinese Long and Short Form Choice Exploiting Neural Network Language Modeling Approaches (2020)

Abstract: This paper presents our work in long and short form choice, a significant question of lexical choice, which plays an important role in many Natural Language Understanding tasks. Long and short form sharing at least one identical word meaning but with different number of syllables is a highly frequent linguistic phenomenon in Chinese like 老虎-虎(laohu-hu, tiger)

## introduction the long and short form of an elastic word refers to words have different word length (i.e. number of syllables) but share at least one identical word meaning such as 丢失-丢 (diushi-diu, lose). @xcite points out that as high as 80% percent of chinese words has both long and short forms, therefore chinese speakers need to make the choice between long and short forms during daily communication. like human speakers and writers, the long and short form choice task also needs to be carefully resolved for various domain including natural language generation @xcite @xcite , and style transfer @xcite . in this work, we focus on long and short forms that share at least one same word meaning and one same morpheme, but compose of different number of syllables. the long and short form choice task is formulated as fill-in-the-blank (fitb) task @xcite , whose goal is to select a missing word for a sentence from a set of candidates. a fitb example used in this work is shown in ## related work a lot of words can be expressed by either a long form or a short form @xcite , for instance, elastic word, abbreviation, reduplication. in this work, we focus on the choice of long and short form of elastic words, that is, to choose between the long form (disyllabic) and short form (monosyllabic) of an elastic word that shares one morpheme and at least one same word meaning, and are interchangeable in some contexts @xcite . previous work @xcite @xcite show that as high as 90% chinese word has long and short forms, which is a key issue in chinese lexical choice. @xcite investigated the problem of long and short form choice through human and corpus-based approaches, whose results support the statistical significant correlation between word length and the predictability of its context. most previous work investigate the distribution and preference of long and short form based on corpus. it is still an open question to automated choose between long and short forms for a given context. we framed the choosing between long and short forms as a fitb task proposed by @xcite in english near-synonyms choice. unsupervised statistical approaches were applied to accomplish fitb task in near-synonym choice, for instance, co-occurrence networks @xcite and pointwise mutual information (pmi) @xcite were used to build up nearsynonym choice model separately. @xcite explore lexical choice problem by capturing high dimensional information of target words ant their contexts thorough latent semantic space. language models have obtained excellent performance in many language processing tasks, thus they have been also used to tackle the lexical choice task. a 5-gram language model(islam and inkpen, 2010) was trained from a large-scale web corpus to choosing among english nearsynonyms, following which @xcite implemented n-gram language model to chinese near-synonym choice. n-gram model shows a better accuracy than pmi in near-synonym choice which is similar to our task. neural language models overcome the limitation of n-gram language model by its powerful capability of long-range dependency. recurrent neural networks (rnn) @xcite and it variation long-short term memory (lstm) @xcite . zweig et al. @xcite tackled the sentence completion problem with various approaches like language models. nnlms achieved a better performance in these work, whose improvement can be attributed to its capability of capturing global information. ## long and short form choice via language models language modeling is an effective approach to solve the task by computing occurrence probability of each candidate words. given a context, the best long and short form can be chosen according to the probability acquired from language models. the state-of-the-art language modeling techniques and apply them to our task is described in this section. ## experiments and results our baseline is a smoothed 4-gram language model, described in section 3.1. in our training data set, we keep the words occurring at least 50 times, and filter out 2-gram, 3-gram, and 4-gram that occur less than three times. for the model based on lstm-rnn lm, we set the word embeddings as 300, the lstm hidden states as 128, sentence max length as 50, and learning rate as 0.1. ## conclusion in this paper, we have investigated methods for answering long short form choice question. this question is significant because it is a key aspect of lexical choice which is still not well solved by many language processing systems. through this work, we find that both all nnlm-based models do obviously outperform than ngram lm. and our results show that all models perform very well in x-xx category but not very well in x-0x category. our future work will be in the direction of eliminating the bias from nnlms. human evaluation for long and short form choice models also will be our further research content.