user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: A cluster-to-cluster framework for neural machine translation

Paper content:
The quality of a machine translation system depends largely on the availability of sizable parallel corpora.
For the recently popular Neural Machine Translation (NMT) framework, data sparsity problem can become even more severe.
With large amount of tunable parameters, the NMT model may overfit to the existing language pairs while failing to understand the general diversity in language.
In this paper, we advocate to broadcast every sentence pair as two groups of similar sentences to incorporate more diversity in language expressions, which we name as parallel cluster.
Then we define a more general cluster-to-cluster correspondence score and train our model to maximize this score.
Since direct maximization is difficult, we derive its lower-bound as our surrogate objective, which is found to generalize point-point Maximum Likelihood Estimation (MLE) and point-to-cluster Reward Augmented Maximum Likelihood (RAML) algorithms as special cases.
Based on this novel objective function, we delineate four potential systems to realize our cluster-to-cluster framework and test their performances in three recognized translation tasks, each task with forward and reverse translation directions.
In each of the six experiments, our proposed four parallel systems have consistently proved to outperform the MLE baseline, RL (Reinforcement Learning) and RAML systems significantly.
Finally, we have performed case study to empirically analyze the strength of the cluster-to-cluster NMT framework.
Recently, an encode-decoder neural architecture has surged and gained its popularity in machine translation.
In this framework, the encoder builds up a representation of the source sentence and the decoder uses its previous RNN hidden state and attention mechanism to generate target translation.
In order to better memorize the input information, an attention mechanism has been exploited to further boost its performance.
In order to train the attentive encoder-decoder architecture, Maximum Likelihood Estimation (MLE) algorithm has been widely used, which aims at maximizing the point-to-point (one sentence to one sentence) log-likelihood of data pairs in a given dataset.
However, this algorithm has severely suffered from data sparsity problem, or in other word, maximizing only likelihood the existing language pairs might make the model blind to all the non-existing similar sentence pairs.
Thus, the large neural model might overfit to certain prototypes existing in the training set while failing to generalize more unseen but similar scenarios in test time.hurting its semantic meaning.
2) Model-Centroid Augmentation (RL), and BID13 leverage model-generated candidates as pseudo training samples, which are weighted with rewards to enhance the model learning.
By exploring self-generated candidates, the model is able to understand the diversity in the output space.
In pseudo-learning algorithms, both RAML and RL can be interpreted as broadcasting a target ground truth as a cluster of analogues while leaving the source input untouched, which though helps the model understand target diversity, fails to capture the input diversity.
In order to explore both sides' diversity, we advocate a novel and general cluster-to-cluster framework of pseudo learning, which first broadcasts both source and target sentence as clusters and then train the model to comprehend their correspondence, as described in FIG0 .In
this paper, we first introduce the concept of parallel cluster, then design the cluster-to-cluster correspondence score as our optimization objective, based on which, we derive its lower bound KL-divergence as our surrogate objective for model training. In
order to realize our proposed framework, we design four parallel systems and apply them to three recognized machine translation tasks with both forward and reverse translation directions, these four systems have all demonstrated their advantages over the existing competing algorithms in six translation tasks. In
the appendices, we draw samples from the parallel clusters and further analyze their properties to verify our motivation.The contributions of our paper can be summarized as follows: 1)
We are the first to propose the concept of cluster-to-cluster framework, which provides a novel perspective to current sequence-tosequence learning problems. 2)
We delineate the framework and arrive in a novel KL-divergence loss function and generalizes several existing algorithms as special cases, which provides a highlevel understanding about the previous algorithms.2 RELATED
LITERATURE
In this paper, we propose a cluster-to-cluster learning framework and incorporate this concept into neural machine translation.
Our designed systems have proved to be efficient in helping current NMT model to generalize in both source and target sides.
In the cluster-to-cluster framework, the cooperation of four agents can augment valuable samples and alleviate data sparsity, and achieve significant improvement compared with strong baseline systems.
We believe the concept of clusterto-cluster learning can be applicable to a wide range of natural language or computer vision tasks, which will be explored in the future.
Appendices A SYSTEM-DESIGN Sequence to sequence problem (machine translation) can be considered to produce an output sequence Y = (y 1 , y 2 , . . . , y T ), y t ∈ A given an input X. Given input-target pairs (X, Y * ), the generated sequence Y on test is evaluated with task-specific score R(Y, Y * ).
Recurrent neural networks have been widely used in sequence to sequence prediction tasks.
As proposed in and , the basic idea is to first encode the input sequence as a variablelength feature vectors, then apply attention mechanism to compute weighted average over the input vectors and summarize a context vector, with which, previous hidden states and previous label are fed into the decoder RNN to predict the next state and its label.
In our approach, attention-based encoder-decoder is leveraged for both the translation and cluster models, shown as: DISPLAYFORM0 A.1
RL NMT In order to train our RL system as well as adaptive cluster, we need to define a task-level reward as driving signal.
Instead of directly applying BLEU or other evaluation metric, we advocate to use a surrogate n-gram match interpolation, as shown as: DISPLAYFORM1 where N n denotes the number of n-gram match between Y and Y * .
In order to alleviate sequencereward sparseness, we further split it as a series of local reward to drive model's policy search at every time step.
Formally, we write the step-wise reward r(y t |y 1:t−1 , Y * ) as following.
(22) where N (Y,Ỹ ) represents the occurrence of n-gramỸ in sequence Y , specifically, if a certain nsequence y t−n+1:t appears in reference and it's not repeating more than needed, then we assign a corresponding matching score to y t , the policy gradient is described as: DISPLAYFORM2 DISPLAYFORM3 A.2
RAML NMT In order to sample from the intractable payoff distribution for system-A/B as well as our implemented RAML system, we adopt stratified sampling technique described in .
Given a sentence Y * , we first sample an edit distance m, and then randomly select m positions to replace the original labels.
For each sentence, we randomly sample four candidates to perform RAML training.
DISPLAYFORM4 B MATHEMATICAL ANALYSIS We optimize the model parameters of our cluster-to-cluster models by minimizing the lower-bound KL-divergence instead of maximizing the original correspondence score, to characterize the difference between the two objective function, we analyze the relationships between these two functions below: DISPLAYFORM5 which can be further written as: DISPLAYFORM6 therefore, we can derive: DISPLAYFORM7 Since both cluster and translation confidence score c(Y |Y * , X * ) and w(Y |X, X * ) require computing the marginalized probability p(Y |X * ) known to be intractable for variable-length sequences, here we adopt different mechanisms to approximate them.
In system-A and C, we simplify DISPLAYFORM8 pη(Y |X * ) .
In system-B and D, since Y is broadcast through the translation system, the marginalized probabilityp(Y |X * ) is close to one, we discard this factor and approximate c(Y |Y DISPLAYFORM9

assistant: text: We invent a novel cluster-to-cluster framework for NMT training, which can better understand the both source and target language diversity.