Improving Neural Machine Translation by Multi-Knowledge Integration with Prompting
Authors: Ke Wang
Jun Xie
Yuqi Zhang
Yu Zhao
Venue: EMNLP 2023 Findings (2023)
Keywords: Machine Translation
Multi-Knowledge Integration
Prompting

Abstract: Improving neural machine translation (NMT) systems with prompting has achieved significant progress in recent years. In this work, we focus on how to integrate multi-knowledge, multiple types of knowledge, into NMT models to enhance the performance with prompting. We propose a unified framework, which can integrate effectively multiple types of knowledge including sentences, terminologies/phrases and translation templates into NMT models. We utilize multiple types of knowledge as prefix-prompts of input for the encoder and decoder of NMT models to guide the translation process. The approach requires no changes to the model architecture and effectively adapts to domain-specific translation without retraining. The experiments on English-Chinese and English-German translation demonstrate that our approach significantly outperform strong baselines, achieving high translation quality and terminology match accuracy.

Improving Neural Machine Translation by Multi-Knowledge Integration with Prompting
Ke Wang, Jun Xie", Yuqi Zhang, Yu Zhao
Alibaba Group
(wk258830, qingjing. xj, chenwei www.:eeiizzyy@llbbbbbba... inc. com kongyu@taobao... com
Xiong, 2018: Liu al., 2019a) or translation tem- plates (Yang et al., 2020).
Abstract
As primary technique to utilize terminology dictionary, lexically constrained translation allows for explicit phrasc-based constraints be placed on target output strings (Hu et al., 2019). Several research works (Hokamp and Liu, 2017; Post and Vilar, 2018) impose lexical constraints by modify- ing the beam search decoding algorithm. Another line of approach trains the model to copy the tar- get constraints by data augmentation (Song et al., 2019; Dinu et al., 2019; Chen et al., 2020). Some searchers (Li al., 2019; Wang al., 2022b) intro- duce attention modules in the architecture of NMT models to integrate constraints. These methods using terminologies or phrases as the knowledge suffer from either high computational overheads or low terminology translation success rates.
Improving neural machine translation (NMT) systems with prompting has achieved signif- icant progress in recent years. In this work, we focus on how to integrate muuli-kknolledge multiple types of knowledge, into NMT mod- els to enhance the performance with prompting. We propose unified framework, which can in- tegrate effectively multiple types of knowledge including sentences, terminologies/prrasee and translation templates into NMT models. We utilize multiple types of knowledge prefix- prompts of input for the encoder and decoder of NMT models to guide the translation pro- cess. The approach requires no changes to the model architecture and effectively adapts domain-specific translation without retrain- ing. The experiments on English-Chinese Chinese and English-Germi translation demonstrate that approach significantly outperform strong baselines, achieving high translation quality and terminology match accuracy.
In the majority of methods that utilize sentence pairs, the most similar nauree-target sentence pairs are retrieved from a translation memory (TM) for the input source sentence (Liu et al., 2019a; Huang et al., 2021; He et al., 2021). Several approaches focus integrating TM into statistical machine translation (SMT) (Ma al., 2011; Wang et al., 2013; Liu et al., 2019b). Some researchers use a TM augment an NMT model, including using n- grams from a TM to reward translation (Zhang et al., 2018b), employing an auxiliary network to integrate similar sentences into the NMT (Gu et al., 2018; Xia et al., 2019) and data augmenta- tion based on TM (Bulte and Tezcan, 2019a; Xu et al., 2020). These methods consume considerable computational overheads in training testing.
1 Introduction
In the workflow of translation, human translators generally utilize different types of external knowl- edge to simplify the process and improve transla- tion quality and speed, such as matching terminolo- gies and similar example sentences. The knowl- edge used machine translation mainly includes high-quality bilingual sentences, a bilingual termi- nology dictionary and translation templates. Intu- itively, it is reasonable to believe that is beneficial for improving translation quality to integrate mul- tiple types of knowledge into NMT models in flexible and efficient way. However, most existing methods focus on only how integrate single type of knowledge into NMT models, either a ter- minology dictionary (Dinu et al., 2019; Dougal and Lonsdale, 2020), bilingual sentences (Cao and
Although these approaches have demonstrated the benefits of combining an NMT model with single type of knowledge, how integrate multi- ple types of knowledge into NMT models remains challenge. In this work, we propose a prompt- based neural machine translation that can integrate multiple types of knowledge including both sen-
"Corresponding author.

---

tences, terminologiees/phaass and translation tem- plates into NMT models in unified framework. Inspired by (Brown et al., 2020), which has re- defined different NLP tasks as fill in the blanks problems by different prompts, we concatenate the source and target side of the knowledge as prefix- prompts of input for the encoder and decoder of NMT models, respectively. During training, this model learns dynamically to incorporate helpful information from the prefixes into generating trans- lations. At inference time, new knowledge from multiple sources can be applied in real time. The model has automatic domain adaption capability and can be extended to new domains without up- dating parameters. We evaluate the approach in two tasks domain adaptation and soft lexical (ter- minology) constraint. The metric of 'exact match' for terminology match accuracy has significantly improved compared strong baselines both in En- glish to German and English to Chinese transla- tion. This approach has shown its robustness in domain adaptation and performs better than fine- tuming when there are domain mismatch or noise data.
et al., 2019) proposes vectorized dynamic beam allocation (VDBA) improve the efficiency of the decoding algorithm further. The beam search de- coding algorithm by adding lexical constraints is still significantly slower than the beam search algo- rithm. Some data augmentation works propose to replace the corresponding source phrases with the target constraints (Song et al., 2019), to integrate constraints inline annotations in the source sen- tence (Dinu et al., 2019), to insert target constraints using an alignment model (Chen al., 2021) and to append constraints after the source sentence with a separation symbol (Chen et al., 2020; Jon et al., 2021). These data augmentation methods can not guarantee the presence of the target constraints in the output.
Some works concentrate on adapting the ar- chitecture of NMT models to add lexical con- straints. (Susanto al., 2020) invokes lexical con- straints using a non-autoregressive decoding ap- proach. (Zhang et al., 2021) introduces explicit phrase alignment into the translation process of NMT models by building a search space similar to phrase-based SMT. (Li et al., 2019) proposes to use external continuous memory to store constraints and integrate the constraint memories into NMT models through the decoder network. (Wang al., 2022a) proposes template- based method for con- strained translation while maintaining the inference speed. (Wang et al., 2022b) proposes to integrate vectorized lexical source and target constraints into attention modules of the NMT model to model con- straint pairs. These methods may still suffer from low match accuracy of terminology when decoding without the VDBA algorithm.
The contributions of this work include:
We propose simple and effective approach to integrate multi- kowlieknowwedge into NMT models with prompting.
We demonstrate that an NMT model can benefit from multiple types of knowledge si- multaneously, including sentence, terminol- ogy/phrases and translation template knowl- edge.
2 Related Work
The use of TM is very necessary for computer- aided translation (Yamada, 2011) and computa- tional approaches for machine translation (Koehn and Senellart, 2010). Similar sentence pairs re- trieved from TM are also utilized as a type of knowledge to enhance the translation (Liu et al.. 2019a; He et al., 2021; Khandelwal et al., 2021). (Farajian et al., 2017) exploits the retrieved sen- tence pairs from a TM to update the generic NMT models on-the-fly. (Zhang et al., 201 8b) utilizes translation pieces based on n-grams extracted from TM during beam search by adding rewards for matched translation pieces into the NMT model output layer. (He et al., 2019) proposes to add the word position information from a TM as additional rewards to guide the decoding of NMT models.
NMT is increasingly improving translation quality, However, the interpolation of the reasoning process has been less clear due the deep neural architec- tures with hundreds of millions of parameters. How to guide an NMT system with user- user-specified dif- ferent types of knowledge an important issue of NMT applications in real world.
The first and most studied knowledge is simple constraints such as lexical constraints or in-domain dictionaries. (Hokamp and Liu, 2017) proposes grid beam search (GBS) by modifying the decod- ing algorithm add lexical constraints. (Post and Vilar, 2018) introduces dynamic beam allocation to reduce the runtime complexity of GBS by di- viding a fixed size of beam for candidates. (Hu

---

Tixt
Knowledge
Source/laget
X
Food Forward
ENCODER
XM
Feod Forward
Eacoder-Decoder Attention
Self-Allention
Self-Atiention
Bentenoc)
he energy motion is called kinetic
[Template]
DU NN VII PP VIIN
[Temn]
kinctic enogy
[ngut
Inputs
The enerpy possessed by an object by vittue of its motica called kinetie eneggy, or enargy metioon.
[entence]
9633 slit
Pamplate]]
DEC N NN.
Datgut
HORE
Figure I: Knowledge integration framework with prompting and an example representation.
(Gu et al., 2018) uses an auxiliary network to fuse information from the source sentence and the re- trieved value from a TM and then integrate into the NMT model architecture. (Xia et al., 2019) proposes to pack TM into compact graph corre- sponding to multiple words for different sentences in TM, and then encode the packed graph into deep representation during the decoding phase. (Xu al., 2020) utilizes data augmentation to train NMT model whose training instances are bilingual sentences augmented with the translation retrieved from the TM. For input sentences that are not very similar to their TMs, the translation performance of these methods suffers significantly. (He et al., 2021) introduces Example Layer consisting of multi-head attention and cross attention to translate any in- put sentences whether they similar their TM no. (Cai et al., 2021) extends the TM from the bilingual setting to the monolingual setting through learnable memory retrieval in a cross-lingual man- ner. The key idea TM is to integrate the retrieved sentence pairs from a TM into the NMT architec- ture for accurate translations. Most of the works in- tegrate the TM knowledge via model modification and the models need be retrained when loading another TM in new domains.
tures as soft target templates incorporate the tem- plate information into the encoder-decoder frame- work. (Shang et al., 2021) introduce to a template- based machine translation (TBMT) model to inte- grate the syntactic knowledge of the retrieved target template in the NMT decoder. (Zhang et al., 201 8a) represents prior knowledge sources features in log-lineat model to guide the learning process of NMT models. These approaches have demon- strated the clear benefits by incorporating different single types of knowledge into NMT models. Our approach designed to integrate multiple types knowledge into NMT models through an unified framework.
3 Approach
As shown in Figure 1, we use source-side knowl- edge sequence and target-side knowledge sequence to prepend a source sentence and a target sentence as a prefix separately. The model uses multiple types of knowledge as prefix-promple of the input to guide the process of translating target sentence. The form is intended lead the NMT model how to utilize relevant information from the redundant prefixes to guide the translation process for improv- ing translation quality. We use three types of spe- cial tokens to separate different types of knowledge sequence, source sentence and target sentence.
The general knowledge integration NMT is
an ongoing work (Tang et al., 2016; Liu et al., 2016; Zhang et al., 2018a). (Yang et al., 2020) pro- poses to use extracted templates from tree struc-
[Sentence]] / [Term] / [Template]] It indicates similar sentences, matching termi-

---

nologies and translation templates respec- tively. The first token of each knowledge se- quence is always the special token.
1 D
C
Σ (x.))ED
log Ply X Xk:
(2)
[Input]: It is used to separate the knowl- edge sequence and the source sentence.
Note that the proposed method is different from the priming method (Bulte and Tezcan, 2019b; Pham et al., 2020). The priming techniques retrieve sim- ilar source sentences and corresponding transla- tions, and then the similar sentence pairs can be used as an input prompt for NMT models. First, we train the model based on our proposed loss function Equation 2 and the NMT model is trained with stan- dard loss function in their works. Also, our method can be applied to multi-knowleegee and their work can only be is limited sentences.
[Output]] J: It is used to separate the knowl- edge sequence and the target sentence.
For each sentence pair y). corresponding similar sentence pairs, matching terminologies and translation templates
concatenated into source knowledge sequence Xk and target knowledge se- quence Yk with the corresponding special token [Sentence], [Term] and [Template] on the source and target side, respectively. Then the sentence pair yx) are preprocessed as follows: the source sentence x and target sentence y are concatenated with source knowledge sequence Xk and target knowledge sequence Yk. respectively. An Example of the format of the input and output sequences is given in Figure 1. When similar sen- tences and matching terminologies retrieved are empty, the input sequence and output sequence contain only translation template knowledge se- quence. Although in this paper we integrate similar sentences, terminologies and translation templates into the NMT models, our approach can utilize more types of knowledge improve translation performance by using this labeling strategy.
For model optimization, we adopt a two-stage training strategy. In the first stage, we train the standard NMT model based on the standard train- ing objective using the original training data set. Then, in the second stage, we training data constructed from multiple types of knowledge to learn the model parameters based on Equation 2. The proposed model can also be initialized with pre-trained NMT models and then trained on the training data set that contains multi-knowledge.
3.2 Inference
The model receives the whole input sequence and the target prefix composed of target knowledge sequence during decoding. Before beginning trans- lation sequence generation, the encoder encodes the input sequence, and the decoder encodes the target prefix. The initial steps of the beam search use the given prefix Yk to decode the tokens after the special separator token [Output] in forced decoding mode. The decoder can gain indirect ac- cess to whole input sequence tokens while also gaining direct access to target prefix tokens by self- attention and cross-attention mechanisms. It en- ables the NMT model learn to how to extract and make use of valuable information from the redun- dant prefixes during training, and use the prefixes to guide the translation process during inference.
3.1 Training
Given a source sentence x, the conditional proba- bility of the corresponding target sentence y by in- corporating source and target knowledge sequence (Kk.yk) defined as follows:
12
P(y|x, Xxy/xxxk.1 Xk. Fki
II Plyj,,y Xk. Nk:
(1)
where is set of model parameters, y<j ...... Bj denotes a sequence of translation prefix
tokens at time step j and n is length of the target sentence y.
Similar to the vanilla NMT, we use the maximum likelihood estimation (MLE) loss function to find a set of model parameters training set D. In order focus the model on learning the target sentence, we utilize only tokens from the target sentence to calculate the loss function instead of the whole output sentence that contains knowledge sequence. Formally, we minimize the following loss function:
3.3 Knowledge Acquisition
We describe the methods employed in this work to how to obtain knowledge from bilingual sen- tences (sentence knowledge), terminology dietio- naries (terminology knowledge) and translation templates (template knowledge).

---

Retrieving Similar Sentence For each source sentence x, we retrieve the most similar bilin- gual sentences (xs. Ys/ from sentence knowledge. We use token-based edit distance (Levenshtein, 1965) to calculate the similarity score. Formally, for a given souree sentence x, similarity score sim(x, x Xx) between two source sentences and Xg:
source sentence and corresponding source template sequence as inputs and generate template sequence as outputs.
4 Experiments
In this section, we validate the effectiveness of the proposed approach on translation quality and ter- minology match accuracy by comparing with the previous methods used only single type of knowl- edge. We evaluate translation quality with the case- insensitive detokenized SacreBLEU score (Post, 2018) and terminology match accuracy with ex- act match accuracy (Anastasopoulos et al., 2021) which is defined as the ratio between the number of matched source term translations in the output and the total number of source terms.
Ex X3) sim(x, Xx) max(xx] Xxs)
where (x( denotes the Edit Distance be- tween x and Xg, and |X] is the length of x. Xş is a source sentence from the sentence knowledge. Each source sentence x is compared to all the sources from the sentence knowledge using the similarity score. We ignore perfect matches and keep the single best match sentence pair if its simi- larity score is higher than specified threshold X.
4.1 Setup
Corpus We evaluate our approach on English- Chinese (En-Zh) and English-German (En-De) translation tasks. For English-German, German, we use the WMT16 dataset as the training corpus of our model, consisting of 4.5M sentence pairs. We randomly divided the corpus into 4,000 sentences for the validation set and the rest for training. For English- Chinese, we train model CCMT2022 Cor- pus, containing 8.2M sentence pairs. The WMT newsdev2017 used as the validation set.
Matching Terminology Terminology dictionar- ies specify phrase-level corresponding relation- ships of sentence pair. When two matching ter- minologies from a sentence pair have overlapping ranges, a 'hard' match selects only one of them as matching results, such as maximum matching us- ing the longest matching terminologies. The match strategy causes boundary errors, which could nega- tively impact the quality of the translation. There- fore, we adopt 'soft' match strategy to utilize all matching terminologies from sentence pairs. For each source x and the corresponding target y, we record any matching bilingual terminologies that are fully contained in both the source x and target y. For each sentence pair, source tokens of the matching terminologies are concatenated into Xtm with the special token [Term], and corresponding target tokens are similarly concatenated into Ytm. The NMT model learns automatically choose the proper terminologies based on the terminological context by redundant prefixes that contain the all matching bilingual terminologies.
We measure the effectiveness of our model on multi-domain test sets. For English-German, we use the multi-domain English-Germat parallel data (Aharoni and Goldberg, 2020) as in-domain test sets, which include IT. Medical, Koran, and Law. For English-Chinese, We use the multi- domain Inglish-Chiness parallel dataset (Tian et al., 2014) as in-domain test sets, including Sub- titles, News and Education. To distinguish the multi-domain sets for testing and the WMT16 or CCMT2022 sets for training, we call the multi- domain datasets as the in-domain training set or in-domain test set. We use only the training sets to train the models and evaluate results in-domain test sets our experiments. We retrieve similar sen- tence pairs from corresponding in-domain training sets for each in-domain test set. We used randomly selected 2,000 sentences from UM-Corpus as the validation sets of Fine-Tuning models. The sen- tence statistics of datasets are illustrated in Table 1. For all datasets, we tokenize English and Ger- man text with Moses and the Chinese text with
Translation Template Prediction To construct translation template sequence Xtp- we fol- low (Yang et al., 2020) to extract templates from a sub-tree by pruning the nodes deeper than a specific depth on the sentence corresponding constituency- based parser tree. We gain a parallel training data using the source sentences, extracted source and target templates. The constructed data is employed to train a sequence generation model to predict tar- get template sequence. The model is to take the

---

Task
Domain
Train
Vaild
Test
En-De
En-Zh
Suhtitles
298K
2,000
579
Sentence
33.57%
39.28%
News
448K
2,000
1,500
Terminology
42.66%
18.23%
Education
448K
2,000
790
En-Zh
223K
2,000
2,000
Table Percentage of source sentences with similar
En-De
Medical
248K
2,000
2,000
sentences and with matching terminologies on the train-
Law
467K
2,000
2,000
ing sets.
Koran
18K
2,000
2,000
Table I: The number of training, validation, and test data sets of English-German and English-Ching Chinese multi- domains.
model to force the model to generate trans- lation.
VecConstNMT (Wang et al., 2022b): It vectorizes and integrates lexical constraints (matching terminologies) into NMT models by attention modules. The method outper- forms several strong baselines, including the works (Song et al., 2019; Chen et al., 2021).
Jieba tokenizer. We train a joint Byte Pair En- coding (BPE) (Sennrich et al., 2016) with 32k merge operations and use joint vocabulary for both source and target text. The models in all exper- iments follow the statc-of-the-
Transformer base architecture (Vaswani al., 2017) implemented in the Fairseq toolkit (Ott et al., 2019). The models are trained on NVIDIA V100 GPUs and optimized with Adam algorithm (Kingma and Ba, 2015) with 31 = 0.9 and B2 = 0.98. We set the learning rate to 0.0007. In all experiments, the dropout rate is set to 0.3 for English-Genne German and 0.1 for English- Chinese. We use early stopping with patience of 30 for all experiments. We averaged the last checkpoints in all testing.
Training Data The similar sentence pairs are extracted from an training data set using speci- fied similarity threshold of 0.4 in our experiments. For terminology knowledge, we extract a bilingual terminology dictionary from the training data set using term extraction tool TM2TB with default parameters and use the dictionary to match each source and target sentences in the training data set by the 'soft' match strategy. We use Stanford parser (Manning al., 2014) to generate source and target templates based on specific depth 4 from the training data. We build our method's s train- ing data by combining corresponding the similar sentence pair, matching terminologies and trans- lation templates for each sentence pair from the training data set. Table 2 provides the percentage of source sentences with similar sentence pair where the score is higher than the similarity thresh- old of 0.4 and the percentage of sentences with matching terminologies in the training data set. We use the vanilla NMT to train our proposed models based on the training data.
Baseline We compare our approach with the fol- lowing representative baselines:
Vanilla NMT (Vaswani et al., 2017): We di- rectly train a standard Transformer base model using the training set.
Fine-Tuning: The model is fine-tuned us- ing each in-domain training data set based on vanilla NMT. As a single-domain model with an upper bound on the performance, loses the ability of multi-domain adaption.
ANN-MT (Khandelwal et al., 2021): The non- parametric method combines a NMT model with token-level i-rearest-neighbor((NNN by retrieving relevant token examples. It uses an in-domain training data set for domain adap- tation tasks without additional training. The datastore is generated by in-domain train- ing set.
Test Data For each test set of multi-domain sets, we retrieve similar sentence pairs from the training data set and corresponding in-domain training data set. The number of terminologies extracted by TM2TB with the default threshold of 0.9 is not sufficient validate terminology accuracy in the in- domain test sets. Therefore, similarity threshold of 0.7 to extract the bilingual terminologies from the in-domain test set, and then
TM2TB with
Priming- NMT (Pham et al., 2020): It only uses similar sentences as prefixes of NMT
hhaps://iitubb.oom//sssii/iih
Phtps://iiinih.co///ssmmmm/////

---

English-German
English-Chinese
Metric
Method (Knowledge)
IT
Medical
Law
Koran
Avg.
Subtitles
News
Education
Avg.
Fine-Turee Tuning
40.79
53.14
44.67
27.53
33.91
47.90
36,47
Vanilla NMT
23.07
30.72
35.57
10.16
24.88
24.34
31.43
38.54
31.44
NcT (Term.)
23.74
30.41
35.19
10.21
24.89
24.70
31.61
38.78
31.70
BLEU
Priming- NMT (Sent.)
25.93
37.94
41.28
11.41
29.14
38.79
34.83
52.83
42.15
ANN-MT (Sent)
31.00
45.59
50,70
17.66
36.23
41.31
35.07
51.32
42.57
Ours (Term Temp.)
32.43
45.14
36.62 62
37.87
55.25
44.42
Fine-Tuning
59.41
69.70
66.29
35.45
57.71
53.25
58.29
65.89
59.14
Vanilla NMT
34.09
41.92
43.29
19.09
34.60
59.17
55.50
60.75
58.47
VcT (Term.)
80.91
83.99
83.87
77.27
81.51
92.31
88.01
94.39
91.57
Match
Priming-Nie Mominng-NMT (Senz.)
37.48
50.00
49.38
15.45
38.
66.86
56.32
66.82
63.33
ANN-MT (Sent.)
43.56
61.15
61.16
24.55
47.61
50.30
47.87
63.55
53.91
Ours (Term, +Snnt, + Temp.)
80.20
86.58
89.52
81.82
84.53
92.90
94.66
90.65
92.74
Table 3: Evaluation results the English-German and English-Chinese multi-domain test sets, reported on BLEU and exact match accuracy of terminology. Term., Sent. and Temp. indicate terminology, sentence and template knowledge, respectively.
use the lerminologies to match each sentence pair. We train a model based on the pre-trained model mBART (Liu et al., 2020) using source sentences, source and target templates extracted from parse tree in-domain training data. Then we use the model predict target templates in-domain test data.
ter BLEU in News and Education domains com- pared to kNN-MT. Therefore, our approach has stronger generalization ability and significant per- formance improvements in these domains com- pared to the baselines.
4.3 Ablation Studies
4.2 Main Results
In this subsection, we perform ablation experiments on proposed models in order to better understand their relative importance. Table 4 shows evaluation results of the proposed model using only one two type of knowledge on in-domain test sets. Our proposed method (Sent.) using sentence knowledge outperforms the strong baseline Priming-NMB by 5.11 BLEU on English-German on average, which indicates that the loss function Equation 2 could sig- nificantly enhance the performance during training. Our method (Term.) using terminology knowledge outperforms the strong baseline VecConstNMT in terms of exact match accuracy on average.
Table shows the BLEU and exact match accu- racy on Fine-Tuning, NMT, VecConsINMT, kNN- MT, Prim and our proposed method on the Anglish-German and English-Chiness multi- domain test data sets. Our method outperforms all baselines in terms of exact match accuracy on average, demonstrating the benefits of integrating sentence, terminology and template knowledge into NMT models. On Boglish-Germinn multi-domain test sets, our method improves an average of 10.74 BLEU and 49.93% exact match accuracy over vanilla NNT. Compared with Priming-NMTT us- ing sentence knowledge, formance by up to 7.48 BLEU average. Our method outperforms than kNN-MT in the IT and Koran domains.
method enhances per-
Our method (Term.+Sent) using sentence and terminology knowledge achieves both BLEU and exact match accuracy improvements compared with our methods used only sentence or termi- nology knowledge. When sentence, terminology and template knowledge are used simultaneously. our method (Term.+ Temp.) outperforms the method
For English-Chinese multi-domain test sets, our method performs better than Fine-Tuning. The Subtitles, News and Education three domains train- ing data contains noises, such sentence pairs domain mismatches. The performance improve- ment of Fine-Tuning relied on the quality of the in-domain the training data is not significant. Sim- ilarly, the performance of ANN-MT depends on training data quality, and our method achieves bet-
using sentence and termi- nology knowledge by BLEU on English-German and by 0.68 BLEU on average on English-Chinee on average respectively, which shows that the trans- lation templates could effectively improve the trans- lation performance.

---

English-German
English-Chinese
Metric
Method
Knowledge
Medical
Law
Koran
Avg.
Subtitles
News
Education
Avg.
Prining-NMT
Sent.
25.93
37.94
41.28
11.41
29.14
38.79
34,83
52.83
42.15
Term.
25.50
31.88
35.24
10.33
25.74
25.32
33.67
41.47
33.49
BLBU
Sent.
29.49
44.11
49.52
13.86
34.25
39.03
35.03
52.39
42.15
Ours
Term. +Sent.
31.34
43.74
49.77
13.61
34.62
39.95
37.53
53.73
43.74
Term +lemp.
32.43
45.14
50.00
18.89
36.62
40.13
37.87
55.25
44.42
VoeConstNMT
Terma
80.91
83,99
83.87
77.27
81.51
92.31
88.01
94.39
91.57
Term.
88.26
92.85
91.09
87.27
89.87
92,90
91.54
97.20
93.88
Exact Match
Sent.
39.75
56.83
57.72
23.64
44.49
52.94
57,20
61.11
57.08
Term SSet.
80.05
86.89
89.97
78.18
83,77
92.31
94,91
92.52
93.25
Term. + +Sett. Temp.
80.20
86.58
89.52
81.82
84.53
92.90
94.66
90.65
92.74
Table 4: Ablation result BLEU and exact match accuracy using only partial type of knowledge the English- German and English-Chiness
A 2 0.4
2 0.5
2
4.5 Inference Speed
32.43
31.61
30.91
We report the inference speed of our approach rel- ative to the vanilla NMT in Table 6. We use the multiple types of knowledge as prefixes of the en- coder and decoder of the NMT model, increasing the extra calculating time during decoding. The speed of the proposed method using beam search is 1.6 1.9 times slower than the vanilla NMT and mainly depends on the number of tokens of the prefixes on the target during decoding. Compared to ANN-MT with a generation speed that two orders of magnitude slower than the vanilla NMT, our method is more easily acceptable in terms of inference speed.
Medical
45.14
44.41
42.85
Law
50.00
49.10
47.76
Koran
18.89
17.56
16.38
Avg 36.62
35.67
34.46
Table 5: Effect of the threshold for similar sentence retrieval BL. the Finlishh German.
IT
Medical
Law
Koran
Avg.
#Term.
1.6
4.3
5.4
0.2
2.9
#Sent.
6.5
12.8
17.8
11.2
12.1
#Temp.
5.6
4.5
2.7
8.1
5.2
#Know.
13.7
21.6
25.9
19.5
20.2
5
Conclusions
Speed
1.6
1.9
1.9
1.7
1.8
In this paper, we propose unified framework integrate multi-knowledge into NMT models. We utilize multiple types of knowledge as prefixes the encoder and decoder of NMT models, which guides the NMT model's translation process. Espe- cially, our approaches do not actually require the model to see the domain-specific data in training.
Table 6: Relative inference speed for our method com- pared to the vanilla NMT in English to German multi- domain test sets. The batch size is 32. #Term., #Sent., #Temp., and #Know. indicate the average number of tokens of matching terminologies, similar sentences, predicted templates and whole knowledge sequences on the target, respectively.
The model has automatic domain adaption capa- bility and can be extended to new domains with- out updating parameters. The experimental results on multi-domain translation tasks demonstrated that incorporating multiple types of knowledge into NMT models leads to significant improvements in both translation quality and exact match accuracy. Monolingual data is valuable to improve the translation quality of NMT models. In the future, would like to integrate monolingual knowledge into the NMT model. Furthermore, our approach can be applied for tasks where there are multiple types of knowledge, such as Question Answering and Image to Text.
4.4 Effect of Similarity Threshold
Most works (Bulte and Tezcan, 2019a; Xu et al., 2020; Pham et al., 2020) use 0.5 or 0.6 as a sim- ilarity threshold. Table shows the effect of the threshold for similar sentence retrieval transla- tion quality. We find that retrieving similar sen- tences using lower threshold leads improve- ments. The average best performance on in-domain test sets can be achieved by our method based on the 0.4 threshold.

---

Limitations
Guanhua Chen, Yun Chen, and Victor O.K. Li. 2021. Lexically constrained neural machine translation with explicit alignment guidance. Proceedings of the AAAI Conference on Artificial Intelligence, 35(14):12630-11 16630-12688.
As with the majority of studies, the design of the current approach is subject to limitations. We in- tegrate multiple types of knowledge additional prefixes of NMT models and add time consumption in the training and inference stages. The experi- mental results show the added time cost of the pro- posed method is acceptable. Our approach depends on multiple types of knowledge and obtaining the knowledge may be difficult in some practical appli- cations.
Guanhua Chen, Yun Chen, Yong Wang, and Victor O.K. Li. 2020. rexical-constraint-awaaa neural machine translation via data augmentation. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pages 3587-3593. International Joint Conferences on Artificial Intelli- gence Organization. Main track.
Georgiana Dinu, Prashant Mathur, Marcello Federico, and Yaser Al-Onaizan, 2019. Training neural ma- chine translation to apply terminology constraints. In Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 3063- 3068, Florence, Italy. Association for Computational Linguistics.
References
Roce Aharoni and Yoav Goldberg, 2020. Unsupervised domain clusters in pretrained language models. In Proceedings of the 58th Annual Meeting of the Associ- ation for Computational Linguistics ( (Volume I: Long Papers). Association for Computational Linguistics.
Duane K. Dougal and Deryle Lonsdale. 2020. Improv- ing NMT quality using terminology injection. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4820-4827, Marseille, France. European Language Resources Association.
Antonios Anastasopoulos, Laurent Besacier, James Cross, Matthias Gallé, Philipp Koehn, Vassilina Nikoulina, et al. 2021. On the evaluation of ma- chine translation for terminology consistency, arXiv preprint arx:v:2 206. 18911.
M. Amin Farajian, Marco Turchi, Matteo Negri, and Marcello Federico. 2017. Multi-domain neural ma- chine translation through unsupervised adaptation. In Conference on Machine Translation.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah. Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 13:1877-1901.
Jiatao Gu, Yong Wang, Kyunghyun Cho, and Victor OK 2018. Search engine guided neural machine trans- lation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.
Bram Bulte and Arda Tezcan. 2019a. Neural fuzzy re- pair: Integrating fuzzy matches into neural machine translation. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 1800-1809, Florence, Italy, Association for Computational Linguistics.
Qiuxiang He, Guoping Huang. Qu Cui, Li Li, and Lemao Liu. 2021. Fast and accurate neural machine translation with translation memory. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3170-314 3180.
Bram Bulte and Arda Tezcan. 2019b. Neural fuzzy repair: Integrating fuzzy matches into neural chine translation. In 57th Annual Meeting of the (ACL).
Qiuxiang He, Guoping Huang, Lemao Liu, and Li Li. 2019. Word position aware translation memory for neural machine translation. In Natural Language Processing and Chinese Computing: 8th CCF In- ternational Conference, NLPCC 2019, Dunhuang, China, October 9-14, 2019, Proceedings, Part I, page 367-379, Berlin, Heidelberg. Springer- Verlag.
Association-for Computational-Lnnaiiie
pages 1800-1809,
Deng Cai, Yan Wang. Huayang Li. Wai Lam, and Lemao Liu. 2021. Neural machine translation with monolingual translation memory. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1ihh International Joint Conference on Natural Language Processing ( (Volume 1: Long Papers), pages 7307-7318.
Chris Hokamp and Qun Liu. 2017. Lexically con- strained decoding for sequence generation using grid beam search. In Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guisties (Volume 1: Long Papers). pages 1546. Vancouver, Canada. Association for Computational Linguistics.
Qian Cao and Deyi Xiong. 2018. Encoding gated translation memory into neural machine translation. In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 3042-3080 3047, Brussels, Belgium. Association for Com- putational Linguistics.
Edward Hu, Huda Khayrallah, Ryan Culkin, Patrick Xia, Tongfei Chen, Matt Post, and Benjamin Van Durme. 2019. Improved lexically constrained

---

decoding for translation and monolingual rewriting. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, Volume 1 (Long and Short Pupers), pages 839-850, Minneapolis, Minnesota. Association for Computa- tional Linguistics.
Yang Liu, Kun Wang, Chengqing Zong, and Keh-Yit Su. 2019b. A unified framework and models for integrating translation memory into phrase-based sta- tistical machine translation. Comput. Speech Lang.. 14:176-206.
Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Chazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual denoising pre- training for neural machine translation. Transac- rions of the Association for Computational Linguis- tics, 8:726-742.
Guoping Huang, Lemao Liu, Xing Wang. Longyue Wang, Huayang Li. Zhaopeng Tu. Chengyan Huang. and Shuming Shi. 2021. TranSmart: A Practical In- teractive Machine Translation System, arXiv e-prints, page arXiv:2105 2001.1307.222
Yanjun Ma, Yifan He, Andy Way, and Josef van Gen- abith. 2011. Consistent translation using discrimi- native learning- a translation memory-inspired ap- proach. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Hu- man Language Technologies, pages 1239-1248, Port- land, Oregon, USA. Association for Computational Linguisties.
Josef Jon, João Paulo Aires, Dusan Varis, and Ondrej Bojar. 2021. End-to-end lexically constrained ma- chine translation for morphologically rich languages. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the I 11th International Joint Conference on Natural Lan- guage Processing ( (Volume 1: Long Papers), pages 4019-4033, Online. Association for Computational Linguistics.
Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language pro- cessing toolkit. In Proceedings of 52nd Annual Meet- ing of the Association for Computational Linguis- rics: System Demonstrations, pages 55-60, Balti- more, Maryland. Association for Computational Lin- guistics.
Urvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2021. Nearest neigh- bor machine translation. In International Conference on Learning Representations.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings.
Myle Out, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Con- ference of the North American Chapter of the Associa- tion for Computational Linguistics (Demonstrations). pages 48-53, Minneapolis, Minnesota. Association for Computational Linguistics.
Philipp Koehn and Jean Senellart. 2010. Convergence of translation memory and statistical machine transla- tion. In Proceedings of the Second Joint EM E/NGL Workshop: Bringing MT to the User: Research on Integrating MT in the Translation Industry, pages 21- 32, Denver, Colorado, USA. Association for Machine Translation in the Americas.
Minh Quang Pham, Jitao Xu, Josep-Maria Crego, Jean Senellart, and François Yvon. 2020. Priming neu- ral machine translation. In Conference on Machine Translation.
Vladimir I. Levenshtein. 1965. Binary codes capable of correcting deletions, insertions, and reversals. Sovier physics. Doklady, 10:707-710.
Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186- 191, Brussels, Belgium. Association for Computa- tional Linguistics.
Huayang Li. Guoping Huang, Deng Cai, and Lemao Liu. 2019. Neural machine translation with noisy lex- ical constraints. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 28: 28:18644-188 864-1874.
Matt Post and David Vilar. 2018. Fast lexically con- strained decoding with dynamic beam allocation for neural machine translation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, Volume I (Long Pa- pers), pages 1314-1324, New Orleans. Louisiana. Association for Computational Linguisties.
Chunyang Liu, Yang Liu, Maosong Sun, Huanbo Luan, learning of
and Heng Yu. 2016. Agreement-basee parallel lexicons and phrases from non-pan pa-aallllll cor- pora. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Vol- ume 1: Long Papers). pages 1024-1033, Berlin, Ger- many. Association for Computational Linguistics.
Yang Liu, Kun Wang, Chengqing Zong, and Keh- Yih Su. 2019 9a. A unified framework and models for integrat- ing translation memory into phrase-based statistical machine translation. Computer Speech & Language, 54: 17-2206
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Papers), pages 1715-1725,

---

Berlin, Germany, Association for Computational Lin- guistics.
Mengzhou Xia, Guoping Huang. Lemao Liu, and Shum- ing Shi. 2019. Graph based translation memory for neural machine translation. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelli- gence and Thirty-Firsl Innovative Applications of Ar- tificial Intelligence Conference and Ninth AAAI Sym- posium on Educational Advances in Artificial Intelli- gence, AAАI11//А 19/ AAAI Press.
Wei Shang, Chong Feng, Tianfu Zhang, and Da Xu. 2021. Guiding neural machine translation with re- trieved translation template. In 2021 International Joint Conference on Neural Networks (JCCNN) pages 1-7.
Kai Song, Yue Zhang, Heng Yu, Weihua Luo, Kun Wang, and Min Zhang. 2019. Code-switching for enhancing NMT with pre-specified translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 449-459, Minneapolis, Minnesota. Association for Computa- tional Linguistics.
Jitao Xu, Josep Crego, and Jean Senellart. 2020. Boost- ing neural machine translation with similar transla- tions. In Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, pages 1580-1590, Online. Association for Computational Linguistics.
Masaru Yamada. 2011. The effect of translation mem- ory databases on productivity.
Jian Yang, Shuming Ma, Dongdong Zhang, Zhoujun Li, and Ming Zhou. 2020. Improving neural machine translation with soft template prediction. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics. pages 5979-5989, On- line. Association for Computational Linguisties.
Raymond Hendy Susanto, Shamil Chollampatt, and Lil- ing Tan. 2020. Lexically constrained neural machine translation with Levenshtein transformer. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3536-3543, On- line. Association for Computational Linguistics.
Jiacheng Zhang, Yang Liu, Huanbo Luan, Jingfang Xu, and Maosong Sun. 2018a. Prior knowledge integra- tion for neural machine translation using posterior regularization. arXiv preprint arXiv: 111.01111 1.000
Yaohua Tang, Fandong Meng, Zhengdong Lu, Hang Li, and P. Yu. 2016. Neural machine translation with external phrase memory. ArXiv, abs/1606.0179
Liang Tian, Derek F. Wong, Lidia S. Chao, Paulo Quaresma, Francisco Oliveira, Lu, Shuo Li, Yim- ing Wang. and Longyue Wang. 2014. UM-corpus: A large Singissh-Chnnese parallel corpus for statistical machine translation. In Proceedings of the Ninth In- ternational Conference on Language Resources and Evaluation (LREE'144, pages 1837-1842, Reykjavik, Iceland. European Language Resources Association (ELRA).
Jiacheng Zhang, Huanbo Luan, Maosong Sun, Feifci Zhai, Jingfang Xu, and Yang Liu. 2021. Neural machine translation with explicit phrase alignment. IEFF/ACM Transactions on Audio, Speech, and Lan- guage Processing. 29:1001-108 1010.
Jingyi Zhang, Masao Utiyama, Eicchro Sumita, Graham Neubig, and Satoshi Nakamura. 2018b. Guiding neural machine translation with retrieved translation pieces. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume I (Long Pupers), pages 325-1335, New Orleans, Louisiana. Association for Computa- tional Linguisties.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Sys- tems, NIPS' 17, page USA. Curran Associates Inc.
Red Hook, NY,
Kun Wang, Chengqing Zong, and Keh- Yih Su. 2013. Integrating translation memory into phrase- phrase-based based ma- chine translation during decoding. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11-21, Sofia, Bulgaria. Association for Com- putational Linguistics.
Shuo Wang, Peng Li, Zhixing Tan, Zhaopeng Tu, Maosong Sun, and Yang Liu. 2022a. A template- based method for constrained neural machine transla- tion. ArXiv, abs/2205.1 1255.
Shuo Wang, Zhixing Tan, and Yang Liu. 2022b. Inte- grating vectorized lexical constraints for neural ma- chine translation. arXiv preprint arXiv:2203. 12210.


Reviews:
[{"review_id": "bQFTXykf50", "reviewer": "Reviewer_m2EQ", "summary": "", "questions": "- Do you have results on En-Zh Spoken and Science domains? If not, consider dropping them from Table 1 for consistency.\n- If I understand correctly, terminology match accuracy is based solely on matching terminology on the source and checking for its translation in the output, completely ignoring the reference translation. Are there cases where a terminology translation is missing in a reference translation?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback on our manuscript. We apologize for the oversight regarding the typo you pointed out and will ensure its correction in the revised manuscript.\nFirst, The evaluation method of terminology is fair to the method VecConstNMT that include terminology knowledge. Our method and VecConstNMT use same terminology constraints in training and testing. We add Inference time of The mBART model in revised manuscript.   We remove  En-Zh Spoken and Science domains from Table 1 and thank you for reminding. The matching bilingual terminologies are from source sentences and its translations and the terminologies matched only to the source sentences  are ignored in our paper.\nWe hope the above explanations could help to clarify part of your concerns. Thank you once again for your comments and suggestions to improve this paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The method is straightforward and robust\n- Moderately promising results", "reasons_to_reject": "Although there is no major show stopper, there are a couple questionable points, that may be clarified based on authors response:\n\n- Terminology match accuracy score seems to unfairly favour methods that include terminology knowledge, because the same terminology extraction method is both used during evaluation and to extract knowledge during model inference. Results in Table 3 show a huge difference between methods that use terminology constraints and those that don't. While I believe this still shows an actual improvement of terminology translation, ideally the structure of the evaluation method should not overlap with model architecture improvements to avoid biases to a particular terminology extraction method.\n- Including translation template knowledge requires running an mBART model before the translation, which seems to not be clearly mentioned in the inference speed section. To avoid misleading conclusions, this point should be elaborated in the paper.", "questions_to_authors": "- Do you have results on En-Zh Spoken and Science domains? If not, consider dropping them from Table 1 for consistency.\n- If I understand correctly, terminology match accuracy is based solely on matching terminology on the source and checking for its translation in the output, completely ignoring the reference translation. Are there cases where a terminology translation is missing in a reference translation?", "paper_topic_and_main_contributions": "This paper explores augmenting Neural Machine Translation by integrating multiple types of additional knowledge as prefixes in the model encoder and decoder, that the model can refer to to improve translation. The following types of knowledge are considered:\n\n(1) similar source-target translation pair, if it exists, extracted from an in-domain parallel corpus, based on Edit Distance.\n\n(2) translations of any specific terminology found in the source sentence, extracted from a bilingual dictionary, which itself is derived also from an in-domain parallel corpus.\n\n(3) predicted translation template, which is generated by a model, trained to predict a pruned constituency tree in the target language, corresponding to the source sentence and source tree.\n\nA baseline NMT model is trained on examples augmented with knowledge to learn to leverage it and then applied for translation inference. Although the use of each of those types of knowledge has been explored in previous papers, to the best of my knowledge, this is the first work that puts those methods in a common framework and reports results on combining all knowledge types in one model. Experiments are performed on En-De and En-Zh tasks and compared to relevant methods from the literature. BLEU scores and terminology accuracies are reported, showing improvement of the introduced method.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- The point you are making in line 525 that this method may work better than fine-tuning when the data is noisy or there is a domain mismatch seems pretty important, it could be pointed our more explicitly.\n- line 354: cloud -> could"}}, {"review_id": "5nNo4tGERw", "reviewer": "Reviewer_GTCU", "summary": "", "questions": "n/a", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback on our manuscript. Please let us address your concerns.\nWe proposed method is to integate external knowledge into NMT models.  Compared with kNN-MT, our method achieving high translation quality in some domains. Our method get better performance than fine-tuning when the domain data is noisy. Our approach has stronger generalization ability and significant performance improvements. And we prove the importance of external knowledge to NMT models. We hope the above explanations could help to clarify part of your concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Experimental results show significant improvements compared to baselines.\nThis paper is well written and the proposed method is easy to reproduced.", "reasons_to_reject": "1. From the ablation experiments, the improvements brought by the proposed method mainly come from utilizing different types of external knowledge. However, other methods, such as kNN-MT, can also leverage this knowledge with simple modifications, making the comparison of experimental results insufficient.\n2. The author needs to consider the performance of the proposed method when external knowledge contains noise.\n3. As a training approach, the innovation of the proposed method is limited.\n4. Large language models can easily incorporate different types of external knowledge into translation results. The author needs to prove the necessity of the method proposed in this paper.", "questions_to_authors": "n/a", "paper_topic_and_main_contributions": "In this paper, the authors propose a unified framework for integrating multiple types of knowledge into Neural Machine Translation (NMT) models. By using the knowledge as prefixes for the encoder and decoder, the model is guided in learning how to use the knowledge effectively. This method is not domain-specific and does not require extra training for specific domains. Experiments on multi-domain translation tasks show significant improvements in translation quality and exact match accuracy.", "excitement": 2, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "n/a", "Typos_Grammar_Style_and_Presentation_Improvements": "n/a", "Justification_for_Ethical_Concerns": "n/a"}, "extra_text": {}}, {"review_id": "uYBxMBXCEy", "reviewer": "Reviewer_wucw", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your feedback on our manuscript. Please let us address your concerns and provide answers to your questions.\nOur method is a supplement of previous work Priming-NMT.  The Priming-NMT retrieve similar source sentences and corresponding translations, and then the similar sentence pairs can be used as an input prompt for NMT models. First, we train the model based on our proposed loss function Equation2 and the NMT model is trained with standard loss function in their works. Also, our method can be applied to multi-knowledge and their work can only be is limited to sentences.  The performance of kNN-MT depends on training data quality, and Prompt-NMT achieves better BLEU on News and Education domains in table 3. Our method get better performance than fine-tuning when the domain data is noisy. Our approach has stronger generalization ability and significant performance improvements. We hope the above explanations could help to clarify part of your concerns.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper is relatively easy to understand.\n2. The effect of this paper on English-German multi-domain translation is okay.\n3. The inference speed of this method is acceptable.\n", "reasons_to_reject": "1. The novelty of this paper is not enough and seems to be a supplement to some previous works (like Priming-NMT).\n2. This paper does not show significant improvement over other methods in terms of the BLUE metric, and even has a noticeable gap compared to kNN-MT on the English-Chinese dataset.\n3. Comparison to the latest methods in this paper is lacking. Additionally, the authors should demonstrate the performance of Language Model-based approaches such as LLaMA.\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper is about improving neural machine translation (NMT) performance through multi-knowledge integration and prompting. The authors propose a unified framework that can effectively integrate multiple types of knowledge, including sentences, terminologies/phrases, and translation templates, into NMT models to guide the translation process. The approach requires no changes to the model architecture and can adapt to domain-specific translation without retraining. The experiments on English-Chinese and English-German translation demonstrate that the proposed approach outperforms baselines. The contributions of this work include proposing a simple and effective approach to integrate multi-knowledge into NMT models with prompting and demonstrating that an NMT model can benefit from multiple types of knowledge simultaneously.", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a method that mixes LLM-style prompting and seq2seq encoder-decoder methods in NMT. The result allows for combining terminology integration, few-shot translation and domain adaptation.

Pros: reviewers have relatively high opinions and raise valid but non-critical concerns.

Cons: the authors fail to address the raised concerns.

As a conclusion, the contribution is exciting, but there are major unaddressed questions with the paper: namely, that the terminology metric relies on the same terminology as the method (and thus additionally penalizes methods that do not use this terminology), no comparison to existing LLMs is given.
Decision: Accept-Findings