Neural-Driven Multi-criteria Tree Search for Paraphrase Generation
Abstract
A good paraphrase is semantically similar to the original sentence but it must be also well formed, and syntactically different to ensure diversity.
To deal with this tradeoff, we propose to cast the paraphrase generation task as a multi-objectives search problem on the lattice of text transformations.
We use BERT and GPT2 to measure respectively the semantic distance and the correctness of the candidates.
We study two search algorithms: Monte-Carlo Tree Search For Paraphrase Generation (MCPG) and Pareto Tree Search (PTS) that we use to explore the huge sets of candidates generated by applying the PPDB-2.0 edition rules.
We evaluate this approach on 5 datasets and show that it performs reasonably well and that it outperforms a state-of-the-art edition-based text generation method.
Introduction
The generation of paraphrase, i.e. the transformation of a sentence into a well-formed but lexically different one while preserving its original meaning, is a fundamental task of NLP .
Its ability to provide diversity and coverage ﬁnds applications in several domains like question answering [5], machine-translation [4], dialog systems [15], privacy [22] or more recently adversarial learning [19].
The formal deﬁnition of paraphrase may vary according to the targeted application and the tolerance we set along several axes including:
the semantic distance from the source that we want to minimize; 2.
the quality of the syntax that we want to keep high; 3.
the lexical distance from the source that we want to maximize to ensure diversity.
Automatically generating paraphrases that are semantically accurate, diverse and well-formed remains a challenging problem and with the lack of large and high-quality aligned paraphrase corpora, it is difﬁcult to train generic supervised models on this task.
On the other hand, assessing the quality of a given sentence as a paraphrase has become a much more tractable task.
Indeed, the availability of contextual embeddings networks trained on huge corpora have led to the development of high-quality semantic distances [18, 17, 25, 26].
Regarding the quality of the syntax , the perplexity of a large-scale language model like GPT2 [24] is an excellent ranking criterion as well.
For the lexical distance we use a simple Levenshtein edition distance.
Leveraging on these three metrics, we cast the paraphrase generation task as a multi-criteria search problem.
In this experiment we use PPDB 2.0 [13], a large-scale database of rewriting rules derived from bilingual corpora, to potentially generate billions of ’naive’ candidate paraphrases by edition from the source sentence.
To sort efﬁciently the good candidates from the others, we experiment two algorithms.
The ﬁrst one, called MCPG for Monte-Carlo Paraphrase Generation , is a variant of the Monte-Carlo Tree Search algorithm ( MCTS ) [6, 7, 9].
This algorithm is famous for its successes on mastering the – highly combinatorial – game of Go [7, 14].
The second one is a novel search algorithm that we call Pareto Tree Search ( PTS ).
In contrast to MCTS which is a single-criterion search algorithm, we designed PTS to retrieve an approximation of the whole Pareto optimal set.
This allows for more ﬂexibility on paraphrase generation where the balance between semantic distance , lexical distance , and syntax quality is hard to tune a priori .
Paraphrase Generation Lattice
We model the paraphrase generation as a sequence of editions and transformations from a source sentence into its paraphrase.
In this work, we only consider local transformations, i.e. replacement of certain words or phrases by others, but the method we propose should work with more sophisticated transformations schemes as well.
[Table]
The Paraphrase Database ( PPDB ) [12, 13] is a large collection of paraphrase edition rules that was automatically constructed from various bilingual corpora using a pivoting alignment method [8].
The PPDB database is divided into increasingly large and decreasingly accurate subsets 1 We used the XL subset, and we removed the rules labeled as ” Independent ”.
This left us with a set of 5 .million rewriting rules.
We give some examples of these rules on Table 1.
By iteratively applying the rules from a source sentence like the one on Table 1, we obtain a vast lattice of candidate paraphrases.
Some of these candidates like ” he’s talking on june 14 ” are well formed, but many like ” he is speak now on june 14 ” are syntactically broken.
The number of rules that apply, depends on the size of the source sentence and the words it contains.
For instance on the MSRPARAPHRASE dataset (see section 4), the sentences are quite long and the median number of PPDB - XL rules that apply is around 450 .
After two steps of rewriting, the median number of candidates is around 10 5 and by iterative rewriting, we quickly reach a number of paraphrase candidates that is greater than 10 8 .
Searching Algorithms
[Table]
Searching for good paraphrases in the large lattice of candidates generated by PPDB is a costly task.
The two algorithms that we propose are described as pseudocode on Tables 2 and 3.
They share a similar structure: the outer loop explores the lattice at different depths, while the inner loop explores the candidates at each depth.
Both algorithms are anytime: they return the best solutions found so far when the time or space budget is depleted.
MCPG: Monte-Carlo Tree Search for Paraphrase
Generation Following the idea of [9], we used
Monte-Carlo Tree Search ( MCTS ) to explore the PPDB lattice.
The three key ingredients of MCTS are: the use of a bandit policy at each node of a search-tree to select the most promising paths, the use of randomized roll-outs to estimate the quality of these paths, and the back-propagation of rewards along the paths to update the bandit.
We opted here for the EXP 3 [2] randomized bandit policy which proved to be robuster
[Figure]
The MCTS algorithm being not designed for multi-objective problems, we had to combine semantic similarity BERT S , syntax correctness GPT2 and surface diversity Lev S into a single criterion.
The simplest option was a linear scalarization but after some quantitative analysis of the scores distributions like in 1, we realized that it was easy for the search algorithm to maximize the score by just applying a lot of editions to the source sentence.
To overcome this, we replaced the Lev S term by
Lev S · BERT S which led us to the polynomial form α · BERT S + β · Lev S · BERT S − γ · GPT2 .
After a few experiments on train sets, we tuned empirically the weights to α = 3 , β = 0 .
5 and γ = 0 .
025 in order to obtain a balanced policy.
[Table]
PTS: Pareto Tree Search in the paraphrase lattice The
MCTS algorithm is powerful but we observed two drawbacks for paraphrase generation.
First, it was designed for combinatorial problems like Go where the evaluation is only possible on the leaves of the search tree.
This is not the case for paraphrase generation where our neural models can cheaply evaluate any rewriting step and where rewriting from good candidates is more likely to provide good paraphrases than rewriting from bad ones.
Secondly, because it was designed for single criterion search it requires to set the balance between semantic similarity, lexical distance and syntax quality deﬁnitively before any paraphrase search begins.
This is not very ﬂexible and it becomes painful when we want to generate sets of candidates.
By plotting the distributions of the scores as on Figure 1, we noticed that most of the candidates were dominated in the Pareto sense: it was possible to eliminate most of the bad candidates without any hyper-parameter tuning.
We hence adapted MCPG in order to explore the paraphrase lattice and recover an approximation of the Pareto front, postponing the balance between the criteria as a quick post-optimization stage.
This ﬂexibility is also useful for speciﬁc tasks like adversarial learning [19] where a valid paraphrase is selected in post-optimization according to its ability to deceive a given text classiﬁcation model.
Experiments
Datasets and metrics To evaluate the paraphrase generation models, we rely on machine translation metrics that compare the generated paraphrase to one or several references [11].
We report the surface metric BLEU [10] and the semantic metric BERT [26] : the average BERT score of the generated sentence with respect to the ground truth sentences 2 .
[Table]
On Table 4 we give a summary of the datasets we used.
On one hand we have large datasets like MSCOCO or OPUSPARCUS ( OPUSPAR .)
that are noisy and very speciﬁc.
On the other hand we have MSRPARAPHRASE ( MSRPARA .), a high-quality but small human-labeled dataset.
Baseline Constrained Sentence Generation by Metropolis-Hastings Sampling ( CGMH ) [20] is an approach that uses
Metropolis-Hastings sampling [3] for constrained sentence generation
[Table]
The PTS random policy picks a random candidate in the Pareto set. The PTS upper-bound ( upper-b. ) policy is ”cheating” by taking a candidate that maximizes individual BLEU against the target. These two former policies are given to assess the ”quality” of the generated set of candidates.
Starting from the source sentence, the CGMH model samples a sequence of sentences by using local editions: word replacement, deletion, and insertion.
For paraphrase generation, CGMH constraints the sentence generation using a matching function that combines a measure of semantic similarity and a measure of English ﬂuency.
This model is therefore directly comparable with the MCPG and PTS approaches.
Results On Table 5 we report the evaluation of the CGMH , MCPG and PTS models on the ﬁve datasets of aligned paraphrases pairs.
MCPG and PTS models outperform the CGMH baseline model on all corpora except on the MSCOCO dataset where the results are mixed.
The low
PTS upper-bound on MSCOCO and OPUSPARCUS indicates that PPDB - XL vocabulary does not match well with these datasets.
Discussion
We experimented two search-based approaches for paraphrase generation.
These approaches are pragmatic and ﬂexible.
When compared against CGMH , another search-based and weakly-supervised method, our algorithms proved to be faster and more efﬁcient.
The use of local transformations however lead our models to produce paraphrases that are a bit too conservative, especially when starting from short sentences.
For future work, we plan to reﬁne the scoring with deep reinforcement learning techniques and enrich the rules with more sophisticated patterns like phrases permutations.
We also plan to hybridize the search method with supervised seq2seq models [16] by data-augmentation, or through a planning-then-realization scheme like in [23] and [21].