Chinese spell check is an important component for many NLP applications, including word processors, search engines, and automatic essay rating. However, compared to spell checkers for alphabetical languages (e.g., English or French), Chinese spell checkers are more difficult to develop, because there are no word boundaries in Chinese writing system, and errors may be caused by various Chinese input methods. Chinese spell check involves automatically detecting and correcting typos, roughly corresponding to misspelled words in English. Liu et al. (2011) show that people tend to unintentionally generate typos that sound similar (e.g., * 措折 [cuo zhe] and 挫折 [cuo zhe]), or look alike (e.g., *固難 [gu nan] and 困難 [kun nan]). The methods for spell check can be broadly classified into two types: rule-based methods (Ren et al., 2001; Jiang et al., 2012) and statistical methods (Hung & Wu, 2009; Chen, 2010) . Rule-based methods use knowledge resources such as a dictionary to identify a word as a typo. Statistical methods tend to use a large monolingual corpus to create a language model to validate the correction hypotheses. Consider the sentence "心是很重要的。" [xin shi hen zhong yao de] which is correct. However, "心" and "是" are likely to be regarded as an error by a rule-based model for the word "心事" with identical pronunciation. In statistical methods, "心" and "是" are a bigram which has high frequency in a monolingual corpus, so we may determine that "心是" is not a typo after all. In this paper, we propose a model that combines rule-based and statistical approaches. Probable errors, proposed by the rule-based detection module, are verified using statistical machine translation (SMT) model. Our model treats spell check and correction as a kind of translation, where typos are translated into correctly spelled words according to the translation probability and the language model probability. We describe three modules for solving the problem of Chinese spell check: word segmentation, error detection, and error correction. The first module segments the input sentence into word tokens in an attempt to reduce the search space and the probability of false alarm. The second module detects probable errors in the segmented tokens. Any sequences of two or more singleton words are considered likely to contain an error. However, over-segmentation might lead to falsely identified errors. For example, phrases like "超世之 才" [chao shi zhi cai] tend to be over-segmented to "超世/之/才" which might lead to false alarms. So we use additional lexicon items and reduce the chance of generating false alarms.
REF explored statistical machine translation model to translate the sentences containing typos into correct ones.
機器翻譯為本的中文拼字改錯系統 (Chinese Spelling Checker Based on Statistical Machine Translation)
ROCLING
Computer Science