Character-based neural machine translation (NMT) models alleviate out-ofvocabulary issues, learn morphology, and move us closer to completely end-toend translation systems. Unfortunately, they are also very brittle and easily falter when presented with noisy data. In this paper, we confront NMT models with synthetic and natural sources of noise. We find that state-of-the-art models fail to translate even moderately noisy texts that humans have no trouble comprehending. We explore two approaches to increase model robustness: structure-invariant word representations and robust training on noisy texts. We find that a model based on a character convolutional neural network is able to simultaneously learn representations robust to multiple kinds of noise. Character-based neural machine translation (NMT) models alleviate out-ofvocabulary issues, learn morphology, and move us closer to completely end-toend translation systems. Unfortunately, they are also very brittle and easily falter when presented with noisy data. In this paper, we confront NMT models with synthetic and natural sources of noise. We find that state-of-the-art models fail to translate even moderately noisy texts that humans have no trouble comprehending. We explore two approaches to increase model robustness: structure-invariant word representations and robust training on noisy texts. We find that a model based on a character convolutional neural network is able to simultaneously learn representations robust to multiple kinds of noise. 1 (ySbHhm), roughly translating to a blessing for Good Morning, was confused for ABSTRACT Character-based neural machine translation (NMT) models alleviate out-ofvocabulary issues, learn morphology, and move us closer to completely end-toend translation systems. Unfortunately, they are also very brittle and easily falter when presented with noisy data. In this paper, we confront NMT models with synthetic and natural sources of noise. We find that state-of-the-art models fail to translate even moderately noisy texts that humans have no trouble comprehending. We explore two approaches to increase model robustness: structure-invariant word representations and robust training on noisy texts. We find that a model based on a character convolutional neural network is able to simultaneously learn representations robust to multiple kinds of noise.
REF show that machine translation models trained on noisy source text are more robust to the corresponding type of noise.
Synthetic and Natural Noise Both Break Neural Machine Translation
ArXiv
ArXiv
Computer Science