Linguist Code Switching (LCS) is a situation where two or more languages show up in the context of a single conversation. For example, in EnglishChinese code switching, there might be a sentence like "· ‚15©¨ k ‡meeting (We will have a meeting in 15 minutes)". Traditional machine translation (MT) systems treat LCS data as noise, or just as regular sentences. However, if LCS data is processed intelligently, it can provide a useful signal for training word alignment and MT models. Moreover, LCS data is from non-news sources which can enhance the diversity of training data for MT. In this paper, we first extract constraints from this code switching data and then incorporate them into a word alignment model training procedure. We also show that by using the code switching data, we can jointly train a word alignment model and a language model using cotraining. Our techniques for incorporating LCS data improve by 2.64 in BLEU score over a baseline MT system trained using only standard sentence-aligned corpora.
While code-switched data is traditionally viewed as noise when training machine translation systems, REF showed that appropriately detecting codeswitching can help inform word alignment and improve machine translation quality.
Improving Word Alignment Using Linguistic Code Switching Data
EACL
Computer Science