Adaptation of neural machine translation systems to domain-specific vocabulary

State-of-the-art neural machine translation (NMT) systems are sequence-to-sequence neural networks which take the input sentence in the source language, process it and output a translation in target language. These neural networks can be divided into different components: encoder, decoder and attention mechanism. The goal of this work is to solve the vocabulary size limitation that current NMT systems have and also automate the translation enforcement for some domain-specific words, which means that some words will always need to be translated to other specific words as synonyms do not have the same exact meaning. The first problem was addressed by adding an extra layer to the neural network, which extracts the embeddings (numeric representations of each word that include information such as meaning, context, etc.) generated by the decoder and the attention matrix for every sentence; this way it was possible to associate each produced unknown word to its embedding and to its source word. The actual translation for that source word is provided using an external method and afterwards it is stored along the embedding in an external dictionary. This way, if anytime later this word appears again, the system would be able to find its embedding close enough to one in the dictionary and will automatically replace the unknown token in the target sentence by the corresponding word in the dictionary. On the other hand, the second problem also uses a dictionary. However, as it is intended to directly replace a given source word by a specific target word, this dictionary only contains those two words. A modification to the neural network is applied in order to be able to retrieve the attention matrix to align the source and target words so that the system knows which word the NMT system produced for each source word and can therefore easily replace the words contained in the dictionary. For the experiments, the starting point was a NMT system trained to translate from Catalan into Spanish. For training this neural system, a mix of the Diari Oficial de la Generalitat de Catalunya (DOGC) 1 and El Periodico 2 corpora was used, and for testing the corpus from Consumer Eroski [Alcázar, 2005] was used. The results obtained during the evaluation of the first problem (the unknown words problem) http://dogc.gencat. cat/ https://www.elperiodico. com/ i show that after translating 10,038 test sentences (with a total of 243,221 words), a total amount of 10,219 unknown words were generated, of which the system proposed was able to replace 58.68%. Moreover, 68% of those replaced words (39.94% of the total unknown words generated) were corrected by the same word that the reference translation uses. As for the BLEU score, the translation provided by the original NMT system scores 61.0 BLEU points while the proposed system gets a score of 64.6. Therefore, from the results of the experiments we can conclude that the addition of this system increased 3.6 BLEU points and helped correcting at least a 39.94% of the unknown words. The results of the second problem show that out of 255 words, 72% were correctly replaced and 28% were not replaced as expected. This means that using this system around 72% of the domain-specific words are correctly translated, which is a great improvement.