Efficient Cluster-Based k -Nearest-Neighbor Machine Translation (2022)

Abstract: k -Nearest-Neighbor Machine Translation ( k NN-MT) has been recently proposed as a non-parametric solution for domain adaptation in neural machine translation (NMT). It aims to alleviate the performance degradation of advanced MT systems in translating out-of-domain sentences by coordinating with an additional token-level feature-based retrieval module constructed from in-domain data. Previous studies (Khandelwal et al., 2021; Zheng et al., 2021) have already demonstrated that non-parametric NMT is even superior to models fine-tuned on out-of-domain data. In spite of this success, k NN retrieval is at the expense of high latency, in particular for large datastores. To make it practical, in this paper, we explore a more efficient k NN-MT and propose to use clustering to improve the retrieval efficiency. Concretely, we first propose a cluster-based Compact Network for feature reduction in a contrastive learning manner to compress context features into 90+% lower dimensional vectors. We then suggest a cluster-based pruning solution to filter out 10% 40% redundant nodes in large datastores while retaining translation quality. Our proposed methods achieve better or comparable performance while reducing up to 57% inference latency against the advanced non-parametric MT model on several machine translation benchmarks. Experimental results indicate that the proposed methods maintain the most useful information of the original datastore and the Compact Network shows good generalization on unseen domains. Codes are available at https://github.com/tjunlp-lab/PCKMT .

## introduction recently, non-parametric approaches @xcite @xcite machine translation (nmt) for domain adaptation with retrieval pipelines. given an advanced mt model, they generally involve two steps: the accessibility of any provided datastore during translation makes them interpretable. meanwhile, the reliability of these approaches gives the credit to the datastore quality. in spite of significant translation improvements, analyses on the datastore behavior have not been fully explored yet. we empirically observe that the construction of datastore is not optimal for retrieval from two aspects: retrieval latency and semantic distribution. retrieval latency. as shown in ## related work and background in this section, we will briefly introduce the background of the adaptive knn-mt @xcite . adaptive knn-mt is derived from knn-mt @xcite by inserting a lightweight meta-k network that fuses knn retrievals with various k to alleviate the possible noise induced by a single k. formally, it is formulated as two steps: target-side datastore creation and metak network predictions. target-side datastore creation. the datastore constists of a set of key-value pairs. given a bilingual sentence pair (s, t) in a corpus (s, t ), a pretrained general domain nmt model autoregressively extracts the context representation h i of the i-th target token conditioned on both source and target context (s, @xmath0, denoted as h @xmath1, @xmath2. the datastore is finally constructed by taking h i as keys and t i as values: meta-k network prediction. meta-k network (f β ) is a two-layer feed-forward network followed by a non-linear activation function. based on the constructed datastore, it considers a set of different ks that are smaller than an upper bound k. the standard setting for k is @xmath3, k @xmath4. k nearest neighbors of the current context query ĥi from the datastore are first retrieved at the i-th decoding step. then the square of l 2 distance from ĥi to each neighbor (h j , v j ) is denoted as d @xmath5, ĥi 2 . and the number of distinct values in top j neighbors are denoted as c j . the normalized weights of each available k are computed as: note that a validation set is usually required to study the meta-k network before predicting on test sets. during training, only the parameters of the meta-k network need to update. ## our approach as shown in figure 2 , our proposed approach focuses on datastore reconstruction from the perspectives of feature compression and size pruning by utilizing cluster-based signals. ## experiments we carried out a series of experiments to evaluate the proposed non-parametric nmt against the previous advanced counterpart on several translation benchmarks.