Venue: NAACL 2018

# Adversarial Example Generation

with Syntactically Controlled Paraphrase Networks

Mohit Iyyer\({}^{\clubsuit\)\(\clubsuit\)

so, we face two new challenges: (1) obtaining a large amount of paraphrase pairs for training, and (2) defining syntactic transformations with which to label these pairs.

Since no large-scale dataset of sentential paraphrases exists publicly, we follow Wieting et al. (2017) and automatically generate millions of paraphrase pairs using neural _backtranslation_. Backtranslation naturally injects linguistic variation between the original sentence and its backtranslated counterpart. By running the process at a very large scale and testing for the specific variations we want to produce, we can gather ample input-output pairs for a wide range of phenomena. Our focus is on syntactic transformations, which we define using templates derived from linearized constituency parses (SS2). Given such parallel data, we can easily train an encoder-decoder model that takes a sentence and target syntactic template as input, and produces the desired paraphrase.1

Footnote 1: Code, labeled data, and pretrained models available at [https://github.com/miyyer/scpn](https://github.com/miyyer/scpn).

A combination of automated and human evaluations show that the generated paraphrases almost always follow their target specifications, while paraphrase quality does not significantly deteriorate compared to vanilla neural backtranslation (SS4). Our model, the syntactically controlled paraphrase network (scpn), is capable of generating adversarial examples for sentiment analysis and textual entailment datasets that significantly impact the performance of pretrained models (Figure 1). We also show that augmenting training sets with such examples improves robustness without harming accuracy on the original test sets (SS5). Together these results not only establish the first general purpose syntactically controlled paraphrase approach, but also suggest that this general paradigm could be used for controlling many other aspects of the target text.

## 2 Collecting labeled paraphrase pairs

In this section, we describe a general purpose process for gathering and labeling training data for controlled paraphrase generation.

### Paraphrase data via backtranslation

Inducing paraphrases from bilingual data has long been an effective method to overcome data limitations. In particular, bilingual pivoting Banard and Callison-Burch (2005) finds quality paraphrases by pivoting through a different language. Mallinson et al. (2017) show that neural machine translation (NMT) systems outperform phrase-based MT on several paraphrase evaluation metrics.

In this paper, we use the ParaNMT-50M corpus from Wieting and Gimpel (2017). This corpus consists of over 50 million paraphrases obtained by backtranslating the Czech side of the CzEng Bojar et al. (2016) parallel corpus. The pretrained Czech-English model used for translation came from the Nematus NMT system Sennrich et al. (2017). The training data of this system includes four sources: Common Crawl, CzEng 1.6, Europarl, and News Commentary. The CzEng corpus is the largest of these four and was found to have significantly more syntactic diversity than the other data sources Wieting and Gimpel (2017).2

Footnote 3: Similar automated filtering could be used to produce data for many other transformations, such as tense changes, point-of-view shifts, and even stylometric pattern differences Feng et al. (2012). This is an interesting area for future work.

### Automatically labeling paraphrases with syntactic transformations

We need labeled transformations in addition to paraphrase pairs to train a controlled paraphrase model. Manually annotating each of the millions of paraphrase pairs is clearly infeasible. Our key insight is that target transformations can be detected (with some noise) simply by parsing these pairs.4

Footnote 4: Because of the large dataset size, we use the faster but less accurate shift-reduce parser written by John Bauer.

Specifically, we parse the backtranslated paraphrases using the Stanford parser Manning et al. (2014),5 which yields a pair of constituency parses \(\langle p_{1},p_{2}\rangle\) for each sentence pair \(\langle s_{1},s_{2}\rangle\), where \(s_{1}\) is the reference English sentence in the CzEng corpus and \(s_{2}\) is its backtranslated counterpart. For syntactically controlled paraphrasing, we assume \(s_{1}\) and \(p_{2}\) are inputs, and the model is trained to produce \(s_{2}\). To overcome learned biases of the NMT system, we also include reversed pairs \(\langle s_{2},s_{1}\rangle\) during training.

Footnote 5: Some similar automated filtering could be used to produce data for many other transformations, such as tense changes, point-of-view shifts, and even stylometric pattern differences Feng et al. (2012). This is an interesting area for future work.

#### 2.2.1 Syntactic templates

To provide syntactic control, we linearize the bracketed parse structure without leaf nodes (i.e., tokens). For example, the corresponding linearized parse 

tree for the sentence "_She drove home._" is (S(NP(PRP)) (VP(VPD) (NP(NN))) (.*)). A system that requires a complete linearized target parse at test-time is unwieldy; how do we go about choosing the target parse? To simplify test-time usage, we relax the target syntactic form to a parse _template_, which we define as the top two levels of the linearized parse tree (the level immediately below the root along with the root); the prior example's template is S\(\rightarrow\)NP VP. In the next section, we design models such that users can feed in either parse templates or full parses depending on their desired level of control.

## 3 Syntactically Controlled Paraphrase Networks

The scpn encoder-decoder architecture is built from standard neural modules, as we describe in this section.

### Neural controlled paraphrase generation

Given a sentential paraphrase pair \(\langle s_{1},s_{2}\rangle\) and a corresponding target syntax tree \(p_{2}\) for \(s_{2}\), we encode \(s_{1}\) using a bidirectional LSTM Hochreiter and Schmidhuber (1997), and our decoder is a two-layer LSTM augmented with soft attention over the encoded states Bahdanau et al. (2014) as well as a copy mechanism See et al. (2017). Following existing work in NMT Sennrich et al. (2015), we preprocess \(s_{1}\) and \(s_{2}\) into subword units using byte pair encoding, and we perform decoding using beam search. For all attention computations, we use a bilinear product with a learned parameter matrix **W**: given vectors \(\mathbf{u}\) and \(\mathbf{v}\), we score them by \(\mathbf{u}^{T}\textbf{W}\mathbf{v}\).

We incorporate the target syntax \(p_{2}\) into the generation process by modifying the inputs to the decoder. In particular, a standard decoder LSTM receives two inputs at every time step: (1) the embedding \(\mathbf{w}_{t-1}\) of the ground-truth previous word in \(s_{2}\), and (2) an attention-weighted average \(\mathbf{a}_{t}\) of the encoder's hidden states. We additionally provide a representation \(\mathbf{z}_{t}\) of the target \(p_{2}\), so at every time step the decoder computes

\[\mathbf{h}_{t}=\text{LSTM}([\mathbf{w}_{t-1};\mathbf{a}_{t};\mathbf{z}_{t}]). \tag{1}\]

Since we preserve bracketed parse structure, our linearized parses can have hundreds of tokens. Forcing all of the relevant information contained by the parse tree into a single fixed representation (i.e., the last hidden state of an LSTM) is difficult with such large sequences. Intuitively, we want the decoder to focus on portions of the target parse tree that correspond with the current time step. As such, we encode \(p_{2}\) using a (unidirectional) LSTM and compute \(\mathbf{z}_{t}\) with an attention-weighted average of the LSTM's encoded states at every time step. This attention mechanism is conditioned on the decoder's previous hidden state \(\mathbf{h}_{t-1}\).

### From parse templates to full parses

As mentioned in Section 2.2.1, user-friendly systems should be able to accept high-level parse templates as input rather than full parses. Preliminary experiments show that scpn struggles to maintain the semantics of the input sentence when we replace the full target parse with templates, and frequently generates short, formulaic sentences. The paraphrase generation model seems to rely heavily on the full syntactic parse to determine output length and clausal ordering, making it difficult to see how to modify the scpn architecture for template-only target specification.

Instead, we train another model with exactly the same architecture as scpn to generate complete parses from parse templates. This allows us to do the prediction in two steps: first predict the full syntactic tree and then use that tree to produce the paraphrase. Concretely, for the first step, assume \(t_{2}\) is the parse template formed from the top two levels of the target parse \(p_{2}\). The input to this _parse generator_ is the input parse \(p_{1}\) and \(t_{2}\), and it is trained to produce \(p_{2}\). We train the parse generator separately from scpn (i.e., no joint optimization) for efficiency purposes. At test time, a user only has to specify an input sentence and target template; the template is fed through the parse generator, and its predicted target parse is in turn sent to scpn for paraphrase generation (see Figure 2).

### Template selection and post-processing

By switching from full parses to templates, we have reduced but not completely removed the burden of coming up with a target syntactic form. Certain templates may be not be appropriate for particular input sentences (e.g., turning a long sentence with multiple clauses into a noun phrase). However, others may be too similar to the input syntax, resulting in very little change. Since template selection is not a major focus of this paper, we use a relatively simple procedure, selecting the twenty most frequent templates in ParaNMT 

50M.5

Footnote 5: However, we do provide some qualitative examples of rare and medium-frequency templates in Table 3.

Since we cannot generate a valid paraphrase for every template, we postprocess to remove non-sensical outputs. In particular, we filter generated paraphrases using n-gram overlap and paraphrastic similarity, the latter of which is computed using the pretrained word,triavg sentence embedding model from Wieting and Gimpel (2017).6 These paraphrastic sentence embeddings significantly outperform prior work due to the ParaNMT-50M data.

Footnote 6: After qualitatively analyzing the impact of different filtering choices, we set minimum n-gram overlap to 0.5 and train our experiments.

## 4 Intrinsic Experiments

Before using scpn to generate adversarial examples on downstream datasets, we need to make sure that its output paraphrases are valid and grammatical and that its outputs follow the specified target syntax. In this section, we compare scpn to a neural backtranslation baseline (nmt-bt) on the development set of our ParaNMT-50M split using both human and automated experiments. nmt-bt is the same pretrained Czech-English model used to create ParaNMT-50M; however, here we use it to generate in both directions (i.e., English-Czech and Czech-English).

### Paraphrase quality & grammaticality

To measure paraphrase quality and grammaticality, we perform a crowdsourced experiment in which workers are asked to rate a paraphrase pair \(\langle s,g\rangle\) on the three-point scale of Kok and Brockett (2010), where \(s\) is the source sentence and \(g\) is the generated sentence. A **0** on this scale indicates no paraphrase relationship, while **1** means that \(g\) is an ungrammatical paraphrase of \(s\) and **2** means that \(g\) is a grammatical paraphrase of \(s\). We select 100 paraphrase pairs from the development set of our ParaNMT-50M split (after the postprocessing steps detailed in Section 3.3) and have three workers rate each pair.7 To focus the evaluation on the effect of syntactic manipulation on quality, we

Figure 2: scpn implements parse generation from templates as well as paraphrase generation from full parses as encoder-decoder architectures (attention depicted with dotted lines, copy mechanism with double stroked lines). While both components are trained separately, at test-time they form a pipelined approach to produce a controlled paraphrase from an input sentence \(s_{1}\), its corresponding parse \(p_{1}\), and a target template \(t_{2}\).

 

only select sentences whose top-level parse templates differ (i.e., \(t_{s}\neq t_{g}\)), ensuring that the output of both systems varies syntactically from the source sentences.

The results (Table 1) show that the uncontrolled nmt-bt model's outputs are comparable in quality and grammaticality to those of scpn; neither system has a significant edge. More interestingly, we observe no quality drop when feeding templates to scpn (via the parse generator as described in Section 3.2) instead of complete parse trees, which suggests that the parse generator is doing a good job of generating plausible parse trees; thus, for all of the adversarial evaluations that follow, we only use the templated variant of scpn.

### Do the paraphrases follow the target specification?

We next determine how often scpn's generated paraphrases conform to the target syntax: if \(g\) is a generated paraphrase and \(p_{g}\) is its parse, how often does \(p_{g}\) match the ground-truth target parse \(p_{2}\)? We evaluate on our development set using _exact template match_: \(g\) is deemed a syntactic match to \(s_{2}\) only if the top two levels of its parse \(p_{g}\) matches those of \(p_{2}\). We evaluate two scpn configurations, where one is given the full target parse \(p_{2}\) and the other is given the result of running our parse generator on the target template \(t_{2}\). As a sanity check, we also evaluate our parse generator using the same metric.

The results (Table 2) show that scpn does indeed achieve syntactic control over the majority of its inputs. Our parse generator produces full parses that almost always match the target template; however, paraphrases generated using these parses are less syntactically accurate.8 A qualitative inspection of the generated parses reveals that they can differ from the ground-truth target parse in terms of ordering or existence of lower-level constituents (Table 6); we theorize that these differences may throw off scpn's decoder.

Footnote 8: With that said, exact match is a harsh metric; these paraphrases are more accurate than the table suggests, as often they differ by only a single constituent.

The nmt-bt system produces paraphrases that tend to be syntactically very similar to the input sentences: 28.7% of these paraphrases have the same template as that of the input sentence \(s_{1}\), while only 11.1% have the same template as the ground-truth target \(s_{2}\). Even though we train scpn on data generated by NMT backtranslation, we avoid this issue by incorporating syntax into our learning process.

## 5 Adversarial example generation

The intrinsic evaluations show that scpn produces paraphrases of comparable quality to the uncontrolled nmt-bt system while also adhering to the specified target specifications. Next, we examine the utility of controlled paraphrases for adversarial example generation. To formalize the problem, assume a pretrained model for some downstream task produces prediction \(y_{x}\) given test-time instance \(x\). An _adversarial_ example \(x^{\prime}\) can be formed by making label-preserving modifications to \(x\) such that \(y_{x}\neq y_{x^{\prime}}\). Our results demonstrate that controlled paraphrase generation with appropriate template selection produces far more valid adversarial examples than backtranslation on sentiment analysis and entailment tasks.

### Experimental setup

We evaluate our syntactically adversarial paraphrases on the Stanford Sentiment Treebank Socher et al. (2013, SST) and SICK entailment detection Marelli et al. (2014). While both are relatively small datasets, we select them because they offer different experimental conditions: SST contains complicated sentences with high syntactic variance, while SICK almost exclusively consists of short, simple sentences. As a baseline, we compare the ten most probable beams from nmt-bt to controlled paraphrases generated by scpn using ten templates randomly sampled from the template set described in Section 3.3.9 We also need pretrained models

\begin{table}
\begin{tabular}{l l} Model & Parse Acc. \\ \hline scpn w/ gold parse & 64.5 \\ scpn w/ generated parse & 51.6 \\ Parse generator & 99.9 \\ \hline \end{tabular}
\end{table}
Table 2: The majority of paraphrases generated by scpn conform to the target syntax, but the level of syntactic control decreases when using generated target parses instead of gold parses. Accuracy is measured by exact template match (i.e., how often do the top two levels of the parses match).

 

for which to generate adversarial examples; we use the bidirectional LSTM baseline for both SST and SICK outlined in Tai et al. (2015) since it is a relatively simple architecture that has proven to work well for a variety of problems.10 Since the SICK task involves characterizing the relationship between two sentences, for simplicity we only generate adversarial examples for the first sentence and keep the second sentence fixed to the ground truth.

Footnote 10: We initialize both models using pretrained GloVe embeddings Pennington et al. (2014) and set the LSTM hidden dimensionality to 300.

### Breaking pretrained models

For each dataset, we generate paraphrases for held-out examples and then run a pretrained model over them.11 We consider a development example _x broken_ if the original prediction \(y_{x}\) is correct, but the prediction \(y_{x^{\prime}}\) for at least one paraphrase \(x^{\prime}\) is incorrect. For SST, we evaluate on the binary sentiment classification task and ignore all phrase-level labels (because our paraphrase models are trained on only sentences). Table 4 shows that for both datasets, scpn breaks many more examples than nmt-bt. Moreover, as shown in Table 5, nmt-bt's paraphrases differ from the original example mainly by lexical substitutions, while scpn often produces dramatically different syntactic structures.

Footnote 11: Since the SICK development dataset is tiny, we additionally generate adversarial examples on its test set.

### Are the adversarial examples valid?

We have shown that we can break pretrained models with controlled paraphrases, but are these paraphrases actually valid adversarial examples? After all, it is possible that the syntactic modifications cause informative clauses or words (e.g., negations) to go missing. To measure the validity of our adversarial examples, we turn again to crowdsourced experiments. We ask workers to choose the appropriate label for a given sentence or sentence pair (e.g., positive or negative for SST), and then we compare the worker's judgment to the original development example's label. For both models, we randomly select 100 adversarial examples and have three workers annotate each one. The results (Table 4) show that on the more complex SST data, a higher percentage of scpn's paraphrases are valid adversarial examples than those of nmt-bt, which is especially encouraging given our model also generates significantly _more_ adversarial examples.

### Increasing robustness to adversarial examples

If we additionally augment the _training_ data of both tasks with controlled paraphrases, we can increase a downstream model's robustness to adversarial examples in the development set. To quantify this effect, we generate controlled paraphrases for the training sets of SST and SICK using the same templates as in the previous experiments. Then, we include these paraphrases as additional training examples and retrain our biLSTM task models.12 As shown by Table 4, training on scpn's paraphrases significantly improves robustness to syntactic adversaries without affecting accuracy on the original test sets. One im

\begin{table}
\begin{tabular}{l l} \hline \hline template & paraphrase \\ \hline original & with the help of captain picard , the borg will be prepared for everything . \\  (SBARQ (ADVP) (,) (S) (,) (SQ)) & now, the bog will be prepared by picard , will it ’? \\  (S (NP) (ACVP) (VP)) & the bore here will be prepared for everything . \\  (S(S) (,) (CC) (S) (:) (FRAG)) & with the help of captain picard , the borg will be prepared , and the borg will be prepared for everything ... for everything . \\ \hline   (FRAG(INT) (,) (S) (,) (NP)) & oh , come on captain picard , the borg line for everything . \\ \hline   original & you seem to be an excellent burglar when the time comes . \\  (S(BAR) (,) (NP) (VP)) & when the time comes , you ‘ll be a great thief . \\  (S(’!’) (UQP) (’’) (VP)) & ”you seem to be a great burgler , when the time comes .” you said . \\  (SO(ND) (SDARQ)) & can i get a good burglar when the time comes ? \\  (S(NP) (IN) (NP) (NP) (UP)) & look at the time the their comes . \\ \hline \end{tabular}
\end{table}
Table 3: Syntactically controlled paraphrases generated by scpn for two examples from the ParaNMT-50M development set. For each input sentence, we show the outputs of four different templates; the fourth template is a failure case (highlighted in green) exhibiting semantic divergence and/or ungrammaticality, which occurs when the target template is unsuited for the input.

 

portant caveat is that this experiment only shows robustness to the set of templates used by scpn; in real-world applications, careful template selection based on the downstream task, along with using a larger set of templates, is likely to increase robustness to less constrained syntactic adversaries. Augmentation with nmt-bt's paraphrases increases robustness on SICK, but on SST, it degrades test accuracy without any significant gain in robustness; this is likely due to its lack of syntactic variation compared to scpn.

## 6 Qualitative Analysis

In the previous section, we quantitatively evaluated the scpn's ability to produce valid paraphrases and adversarial examples. Here, we take a look at actual sentences generated by the model. In addition to analyzing scpn's strengths and weaknesses compared to nmt-bt, we examine the differences between paraphrases generated by various configurations of the model to determine the impact of each major design decision (e.g., templates instead of full parses).

Syntactic manipulation:Table 3 demonstrates scpn's ability to perform syntactic manipulation, showing paraphrases for two sentences generated using different templates. Many of the examples exhibit complex transformations while preserving both the input semantics and grammaticality, even when the target syntax is very different from that of the source (e.g., when converting a declarative to question). However, the failure cases demonstrate that not every template results in a valid paraphrase, as nonsensical outputs are sometimes generated when trying to squeeze the input semantics into an unsuitable target form.

Adversarial examples:Table 5 shows that scpn and nmt-bt differ fundamentally in the type of adversaries they generate. While scpn mostly avoids lexical substitution in favor of making syntactic changes, nmt-bt does the opposite. These examples reinforce the results of the experiment in Section 4.2, which demonstrates nmt-bt's tendency to stick to the input syntax. While scpn is able to break more validation examples than nmt-bt, it is alarming that even simple lexical substitution can break such a high percentage of both datasets we tested.

Ebrahimi et al. (2017) observe a similar phenomenon with HotFlip, their gradient-based substitution method for generating adversarial examples. While nmt-bt does not receive signal from the downstream task like HotFlip, it also does not require external constraints to maintain grammaticality and limit semantic divergence. As future work, it would be interesting to provide this downstream signal to both nmt-bt and scpn; for the latter, perhaps this signal could guide the template selection process, which is currently fixed to a small, finite set.

Templates vs. gold parses:Why does the level of syntactic control decrease when we feed scpn parses generated from templates instead of gold parses (Table 2)? The first two examples in Table 6 demonstrate issues with the templated approach. In the first example, the template is not expressive enough for the parse generator to produce slots for the highlighted clause. A potential way to combat this type of issue is to dynamically define templates based on factors such as the length of the input sentence. In the second example, a parsing error results in an inaccurate template which in turn causes scpn to generate a semantically-divergent paraphrase. The final two examples

\begin{table}
\begin{tabular}{c c c c c c c}  & & & **No augmentation** & \multicolumn{2}{c}{**With augmentation**} \\ \cline{4-8} Model & Task & Validity & Test Acc & Dev Broken & Test Acc & Dev Broken \\ \hline scpn & SST & 77.1 & 83.1 & 41.8 & 83.0 & 31.4 \\ nmt-bt & SST & 68.1 & 83.1 & 20.2 & 82.3 & 20.0 \\ scpn & SICK & 77.7 & 82.1 & 33.8 & 82.7 & 19.8 \\ nmt-bt & SICK & 81.0 & 82.1 & 20.4 & 82.0 & 11.2 \\ \end{tabular}
\end{table}
Table 4: scpn generates more legitimate adversarial examples than nmt-bt, shown by the results of a crowdsourced validity experiment and the percentage of held-out examples that are broken through paraphrasing. Furthermore, we show that by augmenting the training dataset with syntactically-diverse paraphrases, we can improve the robustness of downstream models to syntactic adversaries (see “Dev Broken” before and after augmentation) without harming accuracy on the original test set.

 

show instances where the templated model performs equally as well as the model with gold parses, displaying the capabilities of our parse generator.

Removing syntactic control:To examine the differences between syntactically controlled and uncontrolled paraphrase generation systems, we train an scpn without including \({\mathbf{z}_{t}}\), the attention-weighted average of the encoded parse, in the decoder input. This uncontrolled configuration produces outputs that are very similar to its inputs, often identical syntactically with minor lexical substitution. Concretely, the uncontrolled scpn produces a paraphrase with the same template as its input 38.6% of the time, compared to nmt-bt's 28.7% (Section 4.2).13

Footnote 13: A configuration without the copy mechanism copies input syntax even more, with a 47.7% exact template match.

## 7 Related Work

Paraphrase generation Androutsopoulos et al. (2010); Madnani and Dorr (2010) has been tackled using many different methods, including those based on hand-crafted rules McKeown (1983), synonym substitution Bolshakov and Gelbukh (2004), machine translation Quirk et al. (2004), and, most recently, deep learning Prakash et al. (2016); Mallinson et al. (2017); Dong et al. (2017). Our syntactically controlled setting also relates to controlled language generation tasks in which one desires to generate or rewrite a sentence with particular characteristics. We review related work in both paraphrase generation and controlled language generation below.

### Data-driven paraphrase generation

Madnani and Dorr (2010) review data-driven methods for paraphrase generation, noting two primary families: template-based and translation-based. The first family includes approaches that use hand-crafted rules McKeown (1983), thesaurus-based substitution Bolshakov and Gelbukh (2004); Zhang and LeCun (2015), lattice matching Barzilay and Lee (2003), and template-based "shake & bake" paraphrasing Carl et al. (2005). These methods often yield grammatical outputs but they can be limited in diversity.

The second family includes methods that rewrite the input using methods based on parallel text Bannard and Callison-Burch (2005), machine translation Quirk et al. (2004); Napoles et al. (2016); Suzuki et al. (2017), or related statistical techniques Zhao et al. (2009). Of particular relevance to our work are methods that incorporate syntax to improve fluency of paraphrase output. Callison-Burch (2008) constrains paraphrases to be the same syntactic type as the input, though he was focused on phrase-level, not sentential, paraphrasing. Pang et al. (2003) learn finite-state automata from translation pairs that generate syntactic paraphrases, though this requires multiple translations into the same language and cannot be used to generate paraphrases outside this dataset. Shen et al. (2006) extend this to deeper syntactic analysis. All of these approaches use syntax to

\begin{table}
\begin{tabular}{p{113.8pt} p{113.8pt}|p{113.8pt}} \hline
**template** & **original** & **paraphrase** \\ \hline (S (ADVP) (NP) (VP)) & moody , heartbreaking , and filmed in a natural , unforced style that makes its characters seem entirely convincing when his script is not . \\ (S (PP) (,) (NP) (VP)) & there is no pleasure in watching a child suffer . \\ (S (S) (,) (CC) (S)) & the characters are interesting and often very creatively constructed from figure to backstory . \\ \hline  & every nanosecond of the the new guy reminds you that you could be doing something else far more pleasurable. harris commands the screen , using his wavleases to suggest the raveases of life corruption and ruthlessness. \\ \end{tabular}
\end{table}
Table 5: Adversarial sentiment examples generated by scpn (top) and nmt-bt (bottom). The predictions of a pretrained model on the original sentences are correct (red is negative, blue is positive), while the predictions on else much more enjoyable . harris commands the screen , using his weakness to suggest the raveases of life of corruption and recklessness .

 

improve grammaticality, which is handled by our decoder language model.

Recent efforts involve neural methods. Iyyer et al. (2014) generate paraphrases with dependency tree recursive autoencoders by randomly selecting parse trees at test time. Li et al. (2017) generate paraphrases using deep reinforcement learning. Gupta et al. (2017) use variational autoencoders to generate multiple paraphrases. These methods differ from our approach in that none offer fine-grained control over the syntactic form of the paraphrase.

### Controlled language generation

There is growing interest in generating language with the ability to influence the topic, style, or other properties of the output.

Most related to our methods are those based on syntactic transformations, like the tree-to-tree sentence simplification method of Woodsend and Lapata (2011) based on quasi-synchronous grammar Smith and Eisner (2006). Our method is more general since we do not require a grammar and there are only soft constraints. Perhaps the closest to the proposed method is the conditioned recurrent language model of Ficler and Goldberg (2017), which produces language with user-selected properties such as sentence length and formality but is incapable of generating paraphrases.

For machine translation output, Niu et al. (2017) control the level of formality while Sennrich et al. (2016) control the level of politeness. For dialogue, Li et al. (2016a) affect the output using speaker identity, while Wang et al. (2017) develop models to influence topic and style of the output. Shen et al. (2017) perform style transfer on non-parallel texts, while Guu et al. (2017) generate novel sentences from prototypes; again, these methods are not necessarily seeking to generate meaning-preserving paraphrases, merely transformed sentences that have an altered style.

## 8 Conclusion

We propose scpn, an encoder-decoder model for syntactically controlled paraphrase generation, and show that it is an effective way of generating adversarial examples. Using a parser, we label syntactic variation in large backtranslated data, which provides training data for scpn. The model exhibits far less lexical variation than existing uncontrolled paraphrase generation systems, instead preferring purely syntactic modifications. It is capable of generating adversarial examples that fool pretrained NLP models. Furthermore, by training on such examples, we increase the robustness of these models to syntactic variation.

#### Acknowledgments

We thank the reviewers for their insightful comments. We would also like to thank Mark Yatskar for many useful suggestions on our experiments.

\begin{table}
\begin{tabular}{l l} template & (S(CC) (S) (, (NP) (ADVP) (VP)) \\ original & damian encouraged me , criticized , he ... he always made me go a little deeper . \\ scpn parse & but damian, he supported me , he told me , he always made me go a little deeper . \\ scpn template & but damian supported me , he always made me go a little deeper . \\ \hline template & (S(S) (, (NP) (VP)) \\ original & zacharias did n’t deserve to die , grishanov thought , and he was aware of the huge irony of his situation \\ scpn parse & zacharias did not deserve to die , grishanov told himself , realizing the greatest irony of all . \\ scpn template & Zacharias did not deserve to die , he was aware of the great irony of his situation . \\ \hline template & S(S) (,) (S) \\ original & give me some water , my lips are dry , and i shall try to tell you . \\ scpn parse & give me some water , i have just a dry mouth . \\ scpn template & give me some water , my lips are dry . \\ \hline template & (S(NP) (,) (ADVP) (,) (VP)) \\ original & in the meantime , the house is weakened , and all its old alliances and deals are thrown into doubt . \\ scnp parse & the house , meanwhile , is weakening , which will be all of its old alliances and business . \\ scpn template & the house , meanwhile , is weakened , and its old alliances and deals are thrown into doubt . \\ \end{tabular}
\end{table}
Table 6: Examples from ParaNMT-50M comparing the output of two scpn configurations, one with gold target parses (scpn parse) and one with parses generated from templates (scpn template), where templates are the top two levels of the gold parses. The first two examples demonstrate issues with missing information caused by inexpensive templates and parsing errors, respectively. The remaining examples, in which both configurations produce syntactically similar paraphrases, showcase the ability of the parse generator to produce viable full parses.

 



## References

* Androutsopoulos and Malakasiotis (2010) Ion Androutsopoulos and Prodromos Malakasiotis. 2010. A survey of paraphrasing and textual entailment methods. _Journal of Artificial Intelligence Research_ 38.
* Bahdanau et al. (2014) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. In _International Conference on Learning Representations_.
* Bannard and Callison-Burch (2005) Colin Bannard and Chris Callison-Burch. 2005. Paraphrasing with bilingual parallel corpora. In _Proceedings of the Association for Computational Linguistics_.
* Barzilay and Lee (2003) Regina Barzilay and Lillian Lee. 2003. Learning to paraphrase: an unsupervised approach using multiple-sequence alignment. In _Conference of the North American Chapter of the Association for Computational Linguistics_.
* Bojar et al. (2016) Ondrej Bojar, Ondrej Dusek, Tom Kocmi, Jindrich Libovicky, Michal Novak, Martin Popel, Roman Sudarikov, and Dusan Varis. 2016. CzEng 1.6: Enlarged Czech-English Parallel Corpus with Processing Tools Decdered. In _Text, Speech, and Dialogue: 19th International Conference, TSD 2016_.
* Bolshakov and Gelbukh (2004) Igor Bolshakov and Alexander Gelbukh. 2004. Synonymous paraphrasing using WordNet and Internet. _Natural Language Processing and Information Systems_ .
* Callison-Burch (2008) Chris Callison-Burch. 2008. Syntactic constraints on paraphrases extracted from parallel corpora. In _Proceedings of Empirical Methods in Natural Language Processing_.
* Carl et al. (2005) Michel Carl, Paul Schmidt, and Jorg Schutz. 2005. Reversible template-based shake & bake generation. In _MT Summit X_.
* Dong et al. (2017) Li Dong, Jonathan Mallinson, Siva Reddy, and Mirella Lapata. 2017. Learning to paraphrase for question answering. In _Proceedings of Empirical Methods in Natural Language Processing_.
* Ebrahimi et al. (2017) Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2017. Hotflip: White-box adversarial examples for nlp. _arXiv preprint arXiv:1712.06751_ .
* Ettinger et al. (2017) Allyson Ettinger, Sudha Rao, Hal Daume III, and Emily M Bender. 2017. Towards linguistically generalizable nlp systems: A workshop and shared task. In _Proceedings of the First Workshop on Building Linguistically Generalizable NLP Systems_.
* Feng et al. (2012) Song Feng, Ritwik Banerjee, and Yejin Choi. 2012. Characterizing stylistic elements in syntactic structure. In _Proceedings of Empirical Methods in Natural Language Processing_.
* Ficler and Goldberg (2017) Jessica Ficler and Yoav Goldberg. 2017. Controlling linguistic style aspects in neural language generation. _arXiv preprint arXiv:1707.02633_ .
* Goodfellow et al. (2015) Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In _Proceedings of the International Conference on Learning Representations_.
* Gupta et al. (2017) Ankush Gupta, Arvind Agarwal, Prawaan Singh, and Piyush Rai. 2017. A deep generative framework for paraphrase generation. _arXiv preprint arXiv:1709.05074_ .
* Guu et al. (2017) Kelvin Guu, Tatsunori B Hashimoto, Yonatan Oren, and Percy Liang. 2017. Generating sentences by editing prototypes. _arXiv preprint arXiv:1709.08878_ .
* Hochreiter and Schmidhuber (1997) Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long short-term memory. _Neural computation_ .
* Iyyer et al. (2014) Mohit Iyyer, Jordan Boyd-Graber, and Hal Daume III. 2014. Generating sentences from semantic vector space representations. In _NIPS Workshop on Learning Semantics_.
* Jia and Liang (2017) Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In _Proceedings of Empirical Methods in Natural Language Processing_.
* Kok and Brockett (2010) Stanley Kok and Chris Brockett. 2010. Hitting the right paraphrases in good time. In _Conference of the North American Chapter of the Association for Computational Linguistics_.
* Li et al. (2016a) Jiwei Li, Michel Galley, Chris Brockett, Georgios Spithourakis, Jianfeng Gao, and Bill Dolan. 2016a. A persona-based neural conversation model. In _Proceedings of the Association for Computational Linguistics_.
* Li et al. (2016b) Jiwei Li, Will Monroe, and Dan Jurafsky. 2016b. A simple, fast diverse decoding algorithm for neural generation. _arXiv preprint arXiv:1611.08562_ .
* Li et al. (2017) Zichao Li, Xin Jiang, Lifeng Shang, and Hang Li. 2017. Paraphrase generation with deep reinforcement learning. _arXiv preprint arXiv:1711.00279_ .
* Liang et al. (2017) Bin Liang, Hongcheng Li, Miaoqing Su, Pan Bian, Xirong Li, and Wenchang Shi. 2017. Deep text classification can be fooled. _arXiv preprint arXiv:1704.08006_ .
* Madnani and Dorr (2010) Nitin Madnani and Bonnie J Dorr. 2010. Generating phrascal and sentential paraphrases: A survey of data-driven methods. _Computational Linguistics_ 36(3).
* Mallinson et al. (2017) Jonathan Mallinson, Rico Sennrich, and Mirella Lapata. 2017. Paraphrasing revisited with neural machine translation. In _Proceedings of the European Chapter of the Association for Computational Linguistics_.
* Manning et al. (2014) Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven J. Bethard, and David McClosky. 2014. The Stanford CoreNLP natural language processing toolkit. In _Association for Computational Linguistics System Demonstrations_

Marco Marelli, Luisa Bentivogli, Marco Baroni, Raffaella Bernardi, Stefano Menini, and Roberto Zamparelli. 2014. Semeval-2014 task 1: Evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. _SemEval 2014_ .
* McKeown (1983) Kathleen R McKeown. 1983. Paraphrasing questions using given and new information. _Computational Linguistics_ 9(1).
* Napoles et al. (2016) Courtney Napoles, Chris Callison-Burch, and Matt Post. 2016. Sentential paraphrasing as black-box machine translation. In _Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations_.
* Niu et al. (2017) Xing Niu, Marianna Martindale, and Marine Carpuat. 2017. A study of style in machine translation: Controlling the formality of machine translation output. In _Proceedings of Empirical Methods in Natural Language Processing_.
* Pang et al. (2003) Bo Pang, Kevin Knight, and Daniel Marcu. 2003. Syntax-based alignment of multiple translations: Extracting paraphrases and generating new sentences. In _Conference of the North American Chapter of the Association for Computational Linguistics_.
* Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In _Proceedings of Empirical Methods in Natural Language Processing_.
* Prakash et al. (2016) Aaditya Prakash, Sadid A Hasan, Kathy Lee, Vivek Datla, Ashequl Qadir, Joey Liu, and Oladimeji Farri. 2016. Neural paraphrase generation with stacked residual LSTM networks. In _Proceedings of International Conference on Computational Linguistics_.
* Quirk et al. (2004) Chris Quirk, Chris Brockett, and William Dolan. 2004. Monolingual machine translation for paraphrase generation. In _Proceedings of Empirical Methods in Natural Language Processing_.
* See et al. (2017)bigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. In _Proceedings of the Association for Computational Linguistics_.
* Sennrich et al. (2017) Rico Sennrich, Orhan Firat, Kyunghyun Cho, Alexandra Birch, Barry Haddow, Julian Hitschler, Marcin Junczys-Dowmunt, Samuel Laubli, Antonio Valerio Miceli Barone, Jozef Mokry, et al. 2017. Nematus: a toolkit for neural machine translation. _arXiv preprint arXiv:1703.04357_ .
* Sennrich et al. (2015) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. Neural machine translation of rare words with subword units. In _Proceedings of the Association for Computational Linguistics_.
* Sennrich et al. (2016) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Controlling politeness in neural machine translation via side constraints. In _Conference of the North American Chapter of the Association for Computational Linguistics_.
* Shen et al. (2006)Siwei Shen, Dragomir R Radev, Agam Patel, and Gunes Erkan. 2006. Adding syntax to dynamic programming for aligning comparable texts for the generation of paraphrases. In _Proceedings of International Conference on Computational Linguistics_.
* Shen et al. (2017)Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. 2017. Style transfer from non-parallel text by cross-alignment. In _Proceedings of Advances in Neural Information Processing Systems_.
* Smith and Eisner (2006) David A Smith and Jason Eisner. 2006. Quasi-synchronous grammars: Alignment by soft projection of syntactic dependencies. In _Proceedings of the Workshop on Statistical Machine Translation_.
* Socher et al. (2013)Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In _Proceedings of Empirical Methods in Natural Language Processing_.
* Suzuki et al. (2017)Yui Suzuki, Tomoyuki Kajiwara, and Mamoru Komachi. 2017. Building a non-trivial paraphrase corpus using multiple machine translation systems. In _Proceedings of ACL Student Research Workshop_.
* Tai et al. (2015) Kai Sheng Tai, Richard Socher, and Christopher D Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. In _Proceedings of the Association for Computational Linguistics_.
* Wang et al. (2017)Di Wang, Nebojsa Jojic, Chris Brockett, and Eric Nyberg. 2017. Steering output style and topic in neural response generation. In _Proceedings of Empirical Methods in Natural Language Processing_.
* Wieting and Gimpel (2017)John Wieting and Kevin Gimpel. 2017. Pushing the limits of paraphrastic sentence embeddings with millions of machine translations. _arXiv preprint arXiv:1711.05732_ .
* Wieting et al. (2017) John Wieting, Jonathan Mallinson, and Kevin Gimpel. 2017. Learning paraphrastic sentence embeddings from back-translated bitext. In _Proceedings of Empirical Methods in Natural Language Processing_.
* Woodsend and Lapata (2011)Kristian Woodsd and Mirella Lapata. 2011. Learning to simplify sentences with quasi-synchronous grammar and integer programming. In _Proceedings of Empirical Methods in Natural Language Processing_.
* Zhang and LeCun (2015)Xiang Zhang and Yann LeCun. 2015. Text understanding from scratch. _arXiv preprint arXiv:1502.01710_ .
* Zhao et al. (2009)Shiqi Zhao, Xiang Lan, Ting Liu, and Sheng Li. 2009. Application-driven statistical paraphrase generation. In _Proceedings of the Association for Computational Linguistics_