Question: Can the approach be generalized to other technical domains as well? 
Evidence: FLOAT SELECTED: Figure 2: NMT training after replacing technical term pairs with technical term tokens “TTi” (i = 1, 2, . . .)
FLOAT SELECTED: Figure 3: NMT decoding with technical term tokens “TTi” (i = 1, 2, . . .) and SMT technical term translation
FLOAT SELECTED: Figure 4: NMT rescoring of 1,000-best SMT translations with technical term tokens “TTi” (i = 1, 2, . . .)
In this paper, we propose a method that enables NMT to translate patent sentences with a large vocabulary of technical terms. We use an NMT model similar to that used by Sutskever et al. Sutskever14, which uses a deep long short-term memories (LSTM) BIBREF7 to encode the input sentence and a separate deep LSTM to output the translation. We train the NMT model on a bilingual corpus in which the technical terms are replaced with technical term tokens; this allows it to translate most of the source sentences except technical terms. Similar to Sutskever et al. Sutskever14, we use it as a decoder to translate source sentences with technical term tokens and replace the tokens with technical term translations using statistical machine translation (SMT). We also use it to rerank the 1,000-best SMT translations on the basis of the average of the SMT and NMT scores of the translated sentences that have been rescored with the technical term tokens. Our experiments on Japanese-Chinese patent sentences show that our proposed NMT system achieves a substantial improvement of up to 3.1 BLEU points and 2.3 RIBES points over a traditional SMT system and an improvement of approximately 0.6 BLEU points and 0.8 RIBES points over an equivalent NMT system without our proposed technique.
One important difference between our NMT model and the one used by Sutskever et al. Sutskever14 is that we added an attention mechanism. Recently, Bahdanau et al. Bahdanau15 proposed an attention mechanism, a form of random access memory, to help NMT cope with long input sequences. Luong et al. Luong15b proposed an attention mechanism for different scoring functions in order to compare the source and target hidden states as well as different strategies for placing the attention. In this paper, we utilize the attention mechanism proposed by Bahdanau et al. Bahdanau15, wherein each output target word is predicted on the basis of not only a recurrent hidden state and the previously predicted word but also a context vector computed as the weighted sum of the hidden states.
According to the approach proposed by Dong et al. Dong15b, we identify Japanese-Chinese technical term pairs using an SMT phrase translation table. Given a parallel sentence pair $\langle S_J, S_C\rangle $ containing a Japanese technical term $t_J$ , the Chinese translation candidates collected from the phrase translation table are matched against the Chinese sentence $S_C$ of the parallel sentence pair. Of those found in $S_C$ , $t_C$ with the largest translation probability $P(t_C\mid t_J)$ is selected, and the bilingual technical term pair $\langle t_J,t_C\rangle $ is identified.
For the Japanese technical terms whose Chinese translations are not included in the results of Step UID11 , we then use an approach based on SMT word alignment. Given a parallel sentence pair $\langle S_J, S_C\rangle $ containing a Japanese technical term $t_J$ , a sequence of Chinese words is selected using SMT word alignment, and we use the Chinese translation $t_C$ for the Japanese technical term $t_J$ .
Figure 3 illustrates the procedure for producing Chinese translations via decoding the Japanese sentence using the method proposed in this paper. In the step 1 of Figure 3 , when given an input Japanese sentence, we first automatically extract the technical terms and replace them with the technical term tokens “ $TT_{i}$ ” ( $i=1,2,\ldots $ ). Consequently, we have an input sentence in which the technical term tokens “ $TT_{i}$ ” ( $i=1,2,\ldots $ ) represent the positions of the technical terms and a list of extracted Japanese technical terms. Next, as shown in the step 2-N of Figure 3 , the source Japanese sentence with technical term tokens is translated using the NMT model trained according to the procedure described in Section "NMT Training after Replacing Technical Term Pairs with Tokens" , whereas the extracted Japanese technical terms are translated using an SMT phrase translation table in the step 2-S of Figure 3 . Finally, in the step 3, we replace the technical term tokens “ $TT_{i}$ ” ( $i=1,2,\ldots $ ) of the sentence translation with SMT the technical term translations.
FLOAT SELECTED: Figure 2: NMT training after replacing technical term pairs with technical term tokens “TTi” (i = 1, 2, . . .)
FLOAT SELECTED: Figure 3: NMT decoding with technical term tokens “TTi” (i = 1, 2, . . .) and SMT technical term translation
FLOAT SELECTED: Figure 4: NMT rescoring of 1,000-best SMT translations with technical term tokens “TTi” (i = 1, 2, . . .)
We use an NMT model similar to that used by Sutskever et al. Sutskever14, which uses a deep long short-term memories (LSTM) BIBREF7 to encode the input sentence and a separate deep LSTM to output the translation. We train the NMT model on a bilingual corpus in which the technical terms are replaced with technical term tokens; this allows it to translate most of the source sentences except technical terms. Similar to Sutskever et al. Sutskever14, we use it as a decoder to translate source sentences with technical term tokens and replace the tokens with technical term translations using statistical machine translation (SMT). We also use it to rerank the 1,000-best SMT translations on the basis of the average of the SMT and NMT scores of the translated sentences that have been rescored with the technical term tokens.
In this paper, we utilize the attention mechanism proposed by Bahdanau et al. Bahdanau15, wherein each output target word is predicted on the basis of not only a recurrent hidden state and the previously predicted word but also a context vector computed as the weighted sum of the hidden states.
According to the approach proposed by Dong et al. Dong15b, we identify Japanese-Chinese technical term pairs using an SMT phrase translation table.
For the Japanese technical terms whose Chinese translations are not included in the results of Step UID11 , we then use an approach based on SMT word alignment
Finally, in the step 3, we replace the technical term tokens “ $TT_{i}$ ” ( $i=1,2,\ldots $ ) of the sentence translation with SMT the technical term translations.
Answer: There is no reason to think that this approach wouldn't also be successful for other technical domains. Technical terms are replaced with tokens, therefore so as long as there is a corresponding process for identifying and replacing technical terms in the new domain this approach could be viable.