A Probabilistic Formulation of Unsupervised Text Style Transfer
Authors: Junxian He
Xinyi Wang
Graham Neubig
Taylor Berg-Kirkpatrick
Venue:  (2020)
Keywords: unsupervised text style transfer
deep latent sequence model

Abstract: We present a deep generative model for unsupervised text style transfer that unifies previously proposed non-generative techniques. Our probabilistic approach models non-parallel data from two domains as a partially observed parallel corpus. By hypothesizing a parallel latent sequence that generates each observed sequence, our model learns to transform sequences from one domain to another in a completely unsupervised fashion. In contrast with traditional generative sequence models (e.g. the HMM), our model makes few assumptions about the data it generates: it uses a recurrent language model as a prior and an encoder-decoder as a transduction distribution. While computation of marginal data likelihood is intractable in this model class, we show that amortized variational inference admits a practical surrogate. Further, by drawing connections between our variational objective and other recent unsupervised style transfer and machine translation techniques, we show how our probabilistic view can unify some known non-generative objectives such as backtranslation and adversarial loss. Finally, we demonstrate the effectiveness of our method on a wide range of unsupervised style transfer tasks, including sentiment transfer, formality transfer, word decipherment, author imitation, and related language translation. Across all style transfer tasks, our approach yields substantial gains over state-of-the-art non-generative baselines, including the state-of-the-art unsupervised machine translation techniques that our approach generalizes. Further, we conduct experiments on a standard unsupervised machine translation task and find that our unified approach matches the current state-of-the-art.

Published as a conference paper at ICLR 2020
A PROBABILISTIC FORMULATION OF UNSUPERVISED TEXT STYLE TRANSFER
Junxian He Xinyi Wang", Graham Neubig Cameeiie Mellon University (junxi anh, xi xinyiwl, gneubi g écs. cmu edu
Taylor Berg-Kirkpatrick
University of California San Diego tbergeeng ucsdd edu
ABSTRACT
MXXvv22002.339121 [cs.CL] 10 Feb 2020
We present deep generative model for unsupervised text style transfer that unifies previously proposed non-generative techniques. Our probabilistic approach models non-parallel data from two domains as a partially observed parallel corpus. By hypothesizing a parallel latent sequence that generates each observed sequence, our model learns to transform sequences from one domain another in a completely unsupervised fashion. In contrast with traditional generative sequence models (e.g. the HMM), our model makes few assumptions about the data it generates: uses a recurrent language model as prior and an encoder-decoder as a transduction distribution. While computation of marginal data likelihood is intractable in this model class, we show that amortized variational inference admits practical surro- gate. Further, by drawing connections between our variational objective and other recent unsupervised style transfer and machine translation techniques, we show how our probabilistic view can unify some known non generettetive objectives such backtranslation and adversarial loss. Finally, we demonstrate the effectiveness of our method on a wide range of unsupervised style transfer tasks, including senti- ment transfer, formality transfer, word decipherment, author imitation, and related language translation. Across all style transfer tasks, our approach yields substantial gains over state-of state-of-the-ar the--rr non-generative baselines, including the state-of-the-an unsupervised machine translation techniques that our approach generalizes. Further, we conduct experiments on a standard unsupervised machine translation task and find that our unified approach matches the current state-of-tt of-the-art]
INTRODUCTION
Text sequence transduction systems convert a given text sequence from one domain to another. These techniques can be applied to a wide range of natural language processing applications such as machine translation (Bahdanau et al. 2015), summarization (Rush et al., 2015), and dialogue response generation (Zhao et al. 2017). In many cases, however, parallel corpora for the task at hand are scarce. Therefore, unsupervised sequence transduction methods that require only non- non--arallll
data are appealing and have been receiving growing attention (Bannard & Callison-Bure
2005
Ravi & Knight 2011: Mizukami et al. 2015 Shen et al., 2017 Lample et al. 2018 2019)) This trend is most pronounced in the space of text style transfer tasks where parallel data particularly challenging to obtain (Hu et al. 2017 Shen et al.] [017]] Yang et al. 20188. Style transfer has historically referred sequence transduction problems that modify superficial properties of text i.e. style rather than content
2 We focus on a standard suite of style transfer tasks, including formality transfer (Rao Tetreault 2018). author imitation (Xu et al. (2122). word decipherment (Shen et al.) 2017), sentiment transfer (Shen et al.) (017)), and related language translation (Pourdamghami & Knight 2017). General unsupervised translation has not typically been considered style transfer, but for the purpose of comparison we also conduct evaluation on this task (Lample et al.) 2017).
Equual Contribution.
Code and data are available at http::/hhh...ooooooiiii//iiaaaap
NNotably, some tasks evaluate on do change content to some degree, such as sentiment transfer, but for conciseness use the term "style transfer" nonetheless.

---

Published as a conference paper at ICLR 2020
Recent work on unsupervised text style transfer mostly employs non-generative non crobbbbbaiilistic modeling approaches. For example, [hhen et al.] (20(7) and Yang et al. (2018) design adversarial discriminators shape their unsupervised objective introduces training instability. Other work focuses on directly designing unsupervised training objectives by incorporating intuitive loss terms (e.g. backtranslation loss), and demonstrates state-of- the- performance on unsupervised machine translation (Lample et al. | Artetxe et al. 2019) and style transfer (Lample et al.) 2019). However, the space of possible unsupervised objectives is extremely large and the underlying modeling assumptions defined by each objective can only be reasoned about indirectly. As a result, the process of designing such systems is often heuristic.
approach that can be effective, but often
In contrast, probabilistic models (e.g. the noisy channel model (Shannon 1948)) define assumptions about data more explicitly and allow us to reason about these assumptions during system design. Further, the corresponding objectives are determined naturally by principles of probabilistic inference, reducing the need for empirieal search directly in the space of possible objectives. That said, classical probabilistic models for unsupervised sequence transduction (e.g. the HMM or semi-HMM) typically enforce overly strong independence assumptions about data to make exact inference tractable (Knight et al., 2006 Ravi & Knight 2011 Pourdamghani & Knight 2017). This has restricted their develop- ment and caused their performance to lag behind unsupervised neural objectives on complex tasks. Luckily, in recent years, powerful variational approximation techniques have made it more practical to train probabilistic models without strong independence assumptions (Miao & Blunsom 2016 Yin al. 2018). Inspired by this, we take a new approach to unsupervised style transfer.
We directly define generative probabilistic model that treats a non-parallel corpus in two domains as partially observed parallel corpus. Our model makes few independence assumptions and its true posterior is intractable. However, we show that by using amortized variational inference (Kingma & Welling, 2013), principled probabilistic technique, natural unsupervised objective falls out of our modeling approach that has many connections with past work, yet is different from all past work in specific ways. experiments across suite of unsupervised text style transfer tasks, we find that the natural objective of our model actually outperforms all manually defined unsupervised objectives from past work, supporting the notion that probabilistic principles can be a useful guide even in deep neural systems. Further, in the case of unsupervised machine translation, our model matches the current state-of-the-ar non-generative approach.
2 UNSUPERVISED TEXT STYLE TRANSFER
We first overview text style transfer, which aims to transfer a text (typically single sentence or a short paragraph- - for simplicity we refer to simply "sentences" below) from one domain to another while preserving underlying content. For example, formality transfer (Rao & Tetreauli (018) is the task of transforming the tone of text from informal to formal without changing its content. Other examples include sentiment transfer (Shen al. 2017). word decipherment (Knight et al. 2006). and author imitation (Xu et al. 2012). If parallel examples were available from each domain (i.c. the training data is bitext consisting of pairs of sentences from each domain), supervised techniques could be used to perform style transfer (e.g. attentional Seq2Seq (Bahdanau et al. (015) and Transformer ( (Vaswani al., 2017)). However, for most style transfer problems, only non-parallel corpora (one corpus from each domain) can be easily collected. Thus, work on style transfer typically focuses on the more difficult unsupervised setting where systems must learn from non-parallel data alone.
The model we propose treats an observed non non-parallel text corpus as partially observed parallel corpus. Thus, we introduce notation for both observed text inputs and those that we will treat as latent variables. Specifically, we let X (1)1), (2())
(m)) represent observed data from domain DI, while we let Y (m++)) y (1++2) ..... (")) represent observed data from domain D2. Corresponding indices represent parallel sentences. Thus, none of the observed sentences share indices. In our model, we introduce latent sentences to complete the parallel corpus. Specifi- cally, X (z(m+1) (,,+++),))2)))) (mm)))m.......... _.(n)} represents the set of latent parallel sentences in D1, while Y= ( ( ... glm)) represents the set of latent parallel sentences in D2. Then the goal of unsupervised text transduction is to infer these latent variables conditioned the observed non non-parallel corpora; that is, to learn p(u]ar) and p(Ely).
2

---

Published as a conference paper at ICLR 2020
Domain D2
j
gitm
Transcuerion ply(S
PD, (
Transduction Model p(x (0) ( (sto)
LM Prior Pn, (262)
pirne
p
Domain D1
Figure 1: Proposed graphical model for style transfer via bitext completion. Shaded circles denote the observed variables and unshaded circles denote the latents. The generator is parameterized encoder-decoder architecture and the prior on the latent variable is pretrained language model.
3 THE DEEP LATENT SEQUENCE MODEL
First we present our generative model of bitext, which we refer to as a deep latent sequence model. We then describe unsupervised learning and inference techniques for this model class.
3.1 MODEL STRUCTURE
Directly modeling p( æ)) and p( in the unsupervised setting difficult because we never directly observe parallel data. Instead, we propose generative model of the complete data that defines joint likelihood,
Y)). order to perform text transduction, the unobserved halves can be treated latent variables: they will be marginalized out during learning and inferred via posterior inference at test time.
Our model assumes that each observed sentence is generated from an unobserved parallel sentence in the opposite domain, as depicted Figure Specifically, each sentence 2(()) in domain D2 is generated as follows: First, latent sentence ( in domain D2 is sampled from a prior, PD, (1) Then, (8) sampled conditioned on g( from transduction model, (1) (g()). Similarly, each observed sentence y( in domain D2 is generated conditioned on latent sentence, æ6), domain Dj via the opposite transduction model, (0 (z()) and prior, PD, (z()). We let 8z19 and Byl= represent the parameters of the two transduction distributions respectively. We assume the prior distributions are pretrained the observed data in their respective domains and therefore omit their parameters for simplicity of notation. Together, this gives the following joint likelihood:
p(X, X, Y, 0y1z)
[
( (0ww)) (07))
(
II j=m+1
1( (yiz)op (I
The log marginal likelihood of the data, which we will approximate during training, is:
log p(x. 0 0-1=) log E E, P(X.XX 0yja)
Note that if the two transduction models share no parameters, the training problems for each observed domain are independent. Critically, we introduce parameter sharing through our variational inference procedure, which describe in more detail in Section 3.2
Architecture: Since we would like be able to model a variety of transfer tasks, we choose parameterization for our transduction distributions that makes no independence assumptions. Specifi- cally, we employ an encoder-decoder architecture based on the standard attentional Seq2Seq model which has been shown to be successful across various tasks (Bahdanau et al.) 2015] Rush et al. (015). Similarly, our prior distributions for each domain are parameterized as recurrent language models which, again, make no independence assumptions. In contrast, traditional unsupervised generative sequence models typically make strong independence assumptions to enable exact inference (e.g. the HMM makes Markov assumption the latent sequence and emissions one-to-one). Our model more flexible, but exact inference via dynamic programming will be intractable. We address this problem in the next section.

---

Published as a conference paper at ICLR 2020
Shared transduction purameters
Domain D2
gl
Ogly
yb
(
p
CD2
CD2
Dec
CD₁
Enc
Dec
CD
ERc
p
2.(+)
q i yy)
y
Domain D1
Shared transduction perameters
Figure Depiction of amortized variational approximation. Distributions (()(x) and ((Ely) represent inference networks that approximate the model's true posterior. Critically, parameters are shared between the generative model and inference networks to tie the learning problems for both domains.
3.2 LEARNING
Ideally, learning should directly optimize the log data likelihood, which is the marginal of our model shown in Eq. [ However, due to our model's neural parameterization which does not factorize, computing the data likelihood cannot be accomplished using dynamic programming
be done with simpler models like the HMM. To overcome the intractability of computing the true data likelihood, we adopt amortized variational inference (Kingma & Welling 2013) in order to derive a surrogate objective for learning, the evidence lower bound (ELBO) on log marginal likelihood
p(X, Y; 0=15: 0y1=) LLLOO (X, y=
Oz 5. 0 @alv: 0pt=)
Σ,
F (utgi [oggpp Oniu]] - DKL (s(gl= d Ilpo, (lp (i))
Σ, [B [Buceuu flo ply ( aa; 0-1=)] 1 DEL (a(*ly 1) mno) (z))
Reconstraction likelihood
KL regulazizrr
The surrogate objective introduces (f) dglz) and (Z) @2ly), which represent two separate inference network distributions that approximate the model's true posteriors, (()]x((0; 021p) and (()) (y)= respectively. Learning operates by jointly optimizing the lower bound over both variational and model parameters. Once trained, the variational posterior distributions can be used directly for style transfer. The KL. terms in Eq. [3] that appear naturally the ELBO objective, can be intuitively viewed as regularizers that use the language model priors to bias the induced sentences towards the desired domains. Amortized variational techniques have been most commonly applied to continuous latent variables, in the of the variational autoencoder (VAE) (Kingma & Welling 2013). Here, we use this approach for inference over discrete sequences, which has been shown to be effective in related work on supervipe task (Miao & Blunsom] 2016).
Inference Network and Parameter Sharing: Note that the approximate posterior on one domain aims to learn the reverse style transfer distribution, which is exactly the goal of the generative distribution in the opposite domain. For example, the inference network (f) dzj=)) and the generative distribution ply (());; Oylz) both aim transform D, to D2. Therefore, we use the same architecture for each inference network as used in the transduction models, and tie their parameters:
Φzy dzig: dyjl= 8y|z. This means we learn only two encoder-decoders overall which are parameterized by 6ztø and fylz respectively to represent two directions of transfer. In addition to reducing the number of learnable parameters, this parameter tying couples the learning problems for both domains and allows us to jointly learn from the full data. Moreover, inspired by recent work that
Note that in practice, we add weight À (the same to both domains) to the KL term in ELBO since the regularization strength from the pretrained language model varies depending on the datasets, training data size, or language model structures. Such reweighting has proven necessary in previous work that is trained with
ELBO (Bowman et al. 2016 Miao & Blunsom 2016 Yin et al. 2018).
4

---

Published as a conference paper at ICLR 2020
builds universal Seq2Seq model to translate between different language pairs (Johnson et al. 2017) we introduce further parameter tying between the two directions of transduction: the same encoder is employed for both and y. and domain embedding is provided to the same decoder to specify the transfer direction, as shown in Figure 2 Ablation analysis in Section 5.3 suggests that parameter sharing is important to achieve good performance.
Approximating Gradients of ELBO: The reconstruction and KL terms in Eq. 3 still involve intractable expectations due to the marginalization over the latent sequence. thus we need to approxi- mate their gradients. Gumbel-softmax (ang et al. 2017) and REINFORCE (Sutton al. 2000) are often used as stochastic gradient estimators in the discrete case. Since the latent text variables have an extremely large domain, we find that REINFORCE-basse gradient estimates result in high variance. Thus, we use the Gumbel-softmax straight-throur estimator to backpropagate gradients from the KL terms However, However, we find that approximating gradients of the reconstruction loss is much more challenging both the Gumbel-softmax estimator and REINFORCE are unable to outperform simple stop-grudient method that does not back-propagate the gradient of the latent sequence to the inference network. This confirms similar observation in previous work on unsupervised machine translation (Lample et al.] 2018). Therefore, we use greedy decoding without recording gradients to approximate the reconstruction term.] | Note that the inference networks still receive gradients from the prior through the KL, term, and their parameters are shared with the decoders which do receive gradients from reconstruction. We consider this to be the best empirical compromise the moment.
Initialization. Good initialization is often necessary for successful optimization of unsupervised learning objectives. In preliminary experiments, we find that the encoder-decoder structure has difficulty generating realistic sentences during the initial stages of training. which usually results in a disastrous local optimum. This is mainly because the encoder-decode is initialized randomly and there is no direct training signal specify the desired latent sequence in the unsupervised setting. Therefore, we apply self-reconstruction loss free at the initial epochs of training. We denote the output the encoder as e(-) and the decoder distribution as Pdecs then
Lre = =aa E,Iree(e(xr" (,)) - a E, E,laaceelv))) cg)l,
cx)
(4)
a decays from 1.0 to 0.0 linearly in the first & epochs. Is is a tunable parameter and usually less than 3 in all our experiments.
4 CONNECTION TO RELATED WORK
Our probabilistic formulation can be connected with recent advances in unsupervised text transduction methods. For example, back translation loss (Sennrich et al. (016)) plays an important role in recent unsupervised machine translation (Artetxe al. [0188] Lample et al. 2018 Artetxe et al. 2019) and unsupervised style transfer systems (Lample et al. 2019). order to incorporate back translation lation loss the source language is translated to the target language y form pseudo-paralles corpus, then a translation model from to x can be learned on this pseudo bitext just as in supervised setting. While back translation was often explained as data augmentation technique, in our probabilistic formulation it appears naturally with the ELBO objective as the reconstruction loss term.
Some previous work has incorporated a pretrained language models into neural semi-supervised or unsupervised objectives. He et al.) (2016) uses the log likelihood of pretrained language model as the reward to update supervised machine translation system with policy gradient. Artetxe al.] (2019) utilize a similar idea for unsupervised machine translation. Yang et al.) (201 8) employed similar approach, but interpret the LM as an adversary, training the generator to fool the LM. We show how our ELBO objective is connected with these more heuristic LM regularizers by expanding the KL loss term (assume x is observed):
DKL.
- H, - E,goog (T)).
Note that the loss used in previous work does not include the negative entropy term, -Hy. Our objective results in this additional "regularizer", the negative entropy of the transduction distribution, - H, Intuitively, - H, helps avoid peaked transduction distribution, preventing the transduction
"We use one sample to approximate the expectations.
WWe compare greedy and sampling decoding in Section
5

---

Published as a conference paper at ICLR 2020
from constantly generating similar sentences satisfy the language model. In experiments will show that this additional regularization is important and helps bypass bad local optima and improve performance. These important differences with past work suggest that probabilistic view of the unsupervised sequence transduction may provide helpful guidance in determining effective training objectives.
5 EXPERIMENTS
We test our model on five style transfer tasks: sentiment transfer, word substitution decipherment, formality transfer, author imitation, and related language translation. For completeness, we also
evaluate the task of general unsupervised machine translation using standard benchmarks.
We compare with the unsupervised machine translation model (UNMT) which recently demonstrated ofte-of-thhe-ar performance transfer tasks such as sentiment and gender transfer (Lample et al.. 2019) To validate the effect of the negative entropy term in the KL loss term Eq. 5 we remove and train the model with back-translation loss plus language model negative log likelihood loss (which we denote as BT+NLL) as an ablation baseline. For each task, we also include strong baseline numbers from related work if available. For our method we select the model with the best validation ELBO, and for UNMT or BT+NLL we select the model with the best back-translation loss. Complete model configurations and hyperparameters can be found in Appendix A.1
5.1 DATASETS AND EXPERIMENT SETUP
Word Substitution Decipherment. Word decipherment aims to uncover the plain text behind corpus that was enciphered via word substitution where word in the vocabulary mapped a unique type in a cipher dictionary (Dou & Knight 2012 Shen et al. 2017 Yang et al. 2018). In our formulation, the model is presented with a non-parallel corpus of English plaintext and the ciphertext. We use the data in (Yang et al. 2018) which provides 200K sentences from each domain. While previous work (Shen et al. 2017 Yang et al. 2018) controls the difficulty of this task by varying the percentage of words that are ciphered, we directly evaluate on the most difficult version of this task 100% of the words are enciphered (i.e. no vocabulary sharing in the two domains). We select the model with the best unsupervised reconstruction loss, and evaluate with BLEU score the test set which contains 100K parallel sentences. Results are shown in Table 2
Sentiment Transfer. Sentiment transfer is task of paraphrasing sentence with different sentiment while preserving the original content. Evaluation of sentiment transfer is difficult and is still an open research problem (Mir et al. 2019). Evaluation focuses on three aspects: attribute control, content preservation, and fluency. A successful system needs to perform well with respect to all three aspects. We follow prior work by using three automatic metrics (Yang et al. 2018: Lample et al. 2019): classification accuracy, self-BLEU (BLEU of the output with the original sentence as the reference), and the perplexity (PPL) of each system's 's output under an external language model. We pretrain convolutional classifier (Kim) (014) to assess classification accuracy, and use LSTM language model pretrained on each domain to compute the PPL of system outputs.
We use the Yelp reviews dataset collected by Shen et al. (2017) which contains 250K negative sentences and 380K positive sentences. We also use a small test set that has 1000 numan-annotaan parallel sentences introduced in Li et al. (2018). We denote the positive sentiment domain D1 and the negative sentiment as domain D2. We use Self-BLEU and BLEU to represent the BLEU score of the output against the original sentence and the reference respectively. Results shown in Table []
Formality Transfer. Next, we consider a harder task of modifying the formality of a sequence. We the GYAFC dataset (Rao & Tetreault] 2018), which contains formal and informal sentences from two different domains. In this paper, we use the Entertainment and Music domain, which has about 52K training sentences, 5K development sentences, and 2.5K test sentences. This dataset actually contains parallel data between formal and informal sentences, which we use only for evaluation. We follow the evaluation of sentiment transfer task and test models on three axes. Since the test set is
'The model they used is slightly different from the original model of Lample et al. (018)) in certain details e.g. the addition of pooling layer after attention. We re-implement their model in our codebase for fair comparison and verify that our re-implemennation achieves performance competitive with the original paper.

---

Published as a conference paper at ICLR 2020
Table 1: Results on the sentiment transfer, author imitation, and formality transfer. We list the PPL of pretrained LMs the test sets of both domains. We only report Self- ELff on the sentiment task to compare with existing work.
Task
Model
Acc.
BLEU
Self-BLEU
PPL.,,
PPL.,
Test Set
31.97
21.87
Shen et al
2017
79.50
6.80
12.40
50,40
52.70
Hu ct al.
201
87.70
65.60
115.60
239.80
Sentiment
Yang
2018
83.30
13.40
38.60
30.30
42.10
UNM1
87.17
16.99
44.88
26.53
35.72
BT+NLL
88.36
12.36
31.48
8.75
12.82
Ours
87.90
18.67
48.38
27.75
35.61
Test Set
132.95
85.25
UNMT
80.23
7.13
40.11
39.38
Author Imitation
BT4NLL
76.98
10.80
61.70
65.51
Ours
81.43
10.81
49.62
44.86
Test Set
71.30
135.50
UNMT
78.06
16.11
26.70
10.38
Formality
BT+NLL
82.43
8.57
6.57
8.21
80.46
18.54
22.65
17.23
a parallel corpus, we only compute reference BLEU and ignore self-BLEU. We use Dj to denote formal text, and D2 to denote informal text. Results are shown in Table
Author Imitation. Author imitation is the task of paraphrasing a sentence to match another author's style. The dataset use is collection of Shakespeare's 's plays translated line by line into modern English. It was collected by Xu et al. (2012 and used in prior work on supervised style trans- fer (Jhamtani et al. 2017). This is parallel corpus and thus follow the setting in the formality transfer task. We use 2, to denote modern English, and D2 to denote Shakespeare-style English. Results shown in Table
Related Language Translation. Next, we test our method on a challenging related language translation task (Pourdamghani Knight 2017 Yang al. 2018). This task is natural test bed for unsupervised sequence transduction since the goal is to preserve the meaning of the source sentence while rewriting it into the target language. For our experiments. we choose Bosnian (bs) and Serbian (sr) as the related language pairs. We follow Yang et al. (2018) to report BLEU-1 score on this task since BLEU-4 score is close zero. Results are shown Table
Unsupervised MT. In order to draw connections with related work on general unsupervised machine translation, we also evaluate on the WMT' 16 German English translation task. This task is substantially more difficult than the style transfer tasks considered so far. We compare with the state-of-the-art UNMT system using the existing implementation from the XLM codebase and implement approach in the same framework with XLM initialization for fair comparison. We train both systems on 5M non non-parlllel sentences from each language. Results are shown in Table [
In Tables D we also list the PPL of the test set under the external LM for both the source and target domain. PPL of system outputs should be compared to PPL of the test set itself because extremely low PPL often indicates that the generated sentences are short or trivial.
Table 2: BLEU for decipherment, related language translation (Sr-Bs), and general unsupervised transla- tion (En-De).
Model
Decipher
Sr-Bs
Bs-Sr
En-De
De-En
Shen al.
2017
50.8
Yang al
2018
49.3
31.0
33,4
UNMT BT+NLL Ours
76.4
31.4
33.4
26.5
32.2
78.0
29.6
31.4
5.2 RESULTS
78.4
36.2
38.3
26.9
32.0
Tables and demonstrate some general
trends. First, UNMT is able to outperform
other prior methods in unsupervised text style transfer, such as (Yang et al.) 2018 Hu et al. 2017
Shen et al. 2017). The performance improvements UNMT indicate that flexible and powerful

---

Published as a conference paper at ICLR 2020
architectures crucial (prior methods generally do not have attention mechanism). Second, our model achieves comparable classification accuracy to UNMT but outperforms it in all style transfer tasks in terms of the reference- BLEU,, which is the most important metric since it directly measures the quality of the final generations against gold parallel data. This indicates that our method is both effective and consistent across many different tasks. Finally, the BT+NLLL baseline is sometimes quite competitive, which indicates that the addition of a language model alone can be beneficial. However, our method consistently outperforms the simple BT+NLL method, which indicates the effectiveness of the additional entropy regularizer in Eq. 5 that is the byproduct of our probabilistic formulation.
Next, we examine the PPL of the system outputs under pretrained domain LMs, which should be evaluated in comparison with the PPL of the test set itself. For both the sentiment transfer and the formality transfer tasks in T BT+NIL achieves extremely low PPI., lower than the PPI. of the test corpus in the target domain. After a close examination of the output, we find that it contains many repeated and overly simple outputs. For example, the system generates many examples of "I love this place" when transferring negative to positive sentiment (see Appendix [A.3 for examples). It is not surprising that such a trivial output has low perplexity, high accuracy, and low BLEU score. On the other hand, our system obtains reasonably competitive PPL., and our approach achieves the highest accuracy and higher BLEU score than the UNMT baseline.
5.3 FURTHER ABLATIONS AND ANALYSIS
60%
Improverent Percentage
Parameter Sharing. We also conducted an experiment on the word substitution decipherment task, where we remove parame- ter sharing (as explained in Section(.22) between two directions of transduction distributions, and optimize two encoder-decoder instead. We found that the model only obtained an extremely low BLEU score and failed to generate any meaningful outputs.
Äuthor
43.70%
27.5%
Sr.Bs
Formality
11.25%
Sentiment
Decipher En-De
Performance vs. Domain Divergence. Figure plots the rel- ative improvement of our method over UNMT with respect to accuracy of a naive Bayes' classifier trained to predict the domain of test sentences. Tasks with high classification accu- racy likely have more divergent domains. We can see that for decipherment and en-de translation, where the domains have different vocabularies and thus are easily distinguished, our method yields a smaller gain over UNMT. This likely indicates that the (discrimination) regularization effect of the LM priors is less important or necessary when the two domains are very different.
-5%
Naive Bayes Classiffication Accuracy
Improvement over UNMT vs. classification accuracy.
Figure 3:
Why does the proposed model outperform UNMT? Finally, we examine in detail the output of our model and UNMT for the author imitation task. We pick this task because the reference outputs for the test set are provided, aiding analysis. Examples shown in Table demonstratee that UNMT tends to make overly large changes to the source so that the original meaning is lost, while our method is better at preserving the content of the source sentence. Next, we quantitatively examine the outputs from UNMT and our method by comparing the Fl measure of words bucketed by their syntactic tags. We use the open- soureedd compare-m tool (Neubig et al.) 2019), and the results are shown in Figure
Our system has outperforms UNMT in all word categories. In particular, our system is much better at generating nouns, which likely leads to better content preservation.
Table 3: Examples for author imitation task
Mithods
Stocksspcnn to Medlen
Souren
to father's.
Refenmar
Nut father's hemse
UNMT
Nut beother-
Ouzs
fulher's louse
Senil
away.
Refonenne
Seni your away.
0.2
UNMT
Send encelllentt
Ouzs
Send your
Source
Way should full into dayy O''
Reference
Way should you fill into deep monn
UNMT
Way should you carry nicely. but have your legs
Figure 4: Word Fl score by POS tag.
Outs
Way should you full inso deep a sin
8

---

Published as a conference paper at ICLR 2020
Table 4: Comparison of gradient approximation on the sentiment transfer task.
Method
train F.L.O
test FIIBO
Acc.
BLEU,,
BLEU.
PPL.,
PPL..,
Sample-based Greedy
-3.51
-3.79
87.90
13.34
33.19
24.55
25.67
-2.05
-2.07
87.90
18.67
48.38
27.75
35.61
Table 5: Comparison of gradient propagation method on the sentiment transfer task.
Method
train ELBO
test ELBO
Acc.
BLEU,
BLEU.
PPLe,
PPLo,
Gumbel Softmax
-2.96
-2.98
81.30
16.17
40.47
22.70
23.88
REINFORCE
-6.07
-6.48
95.10
4.08
9.74
6.31
4.08
Stop Gradient
-2.05
-2.07
87.90
18.67
48.38
27.75
35.61
Greedy vs. Sample-based Gradient Approximation. In our experiments, we use greedy decoding from the inference network to approximate the expectation required by ELBO, which is biased estimator. The main purpose of this approach is to reduce the variance of the gradient estimator during training, especially in the early stages when the variance of sample-hased approaches is quite high. As an ablation experiment on the sentiment transfer task we compare greedy and sample-based gradient approximations in terms of both train and test ELBO, well as task performance corresponding to best test ELBO. After the model is fully trained, we find that the cumple-based approximation has low variance. With a single sample, the standard deviation of the EBLO is less than 0.3 across 10 different test repetitions. All final reported ELBO values are all computed with this approach, regardless of whether the greedy approximation was used during training. The reported ELBO values are the evidence lower bound per word. Results are shown in Table where the sampling-based training underperforms on both ELBO and task evaluations.
5.4 COMPARISON OF GRADIENT PROPAGATION METHODS
As noted above, to stabilize the training process, stop gradients from propagating to the inference network from the reconstruction loss. Does this approach indeed better optimize the actual proba- bilistic objective (i.c. ELBO) only indirectly lead to improved task evaluations? In this section use sentiment transfer as an example task to compare different methods for propagating gradients and evaluate both ELBO and task evaluations.
Specifically, we compare three different methods:
Stop Gradient: The gradients from reconstruction loss are not propagated to the inference network. This is the method we use in all previous experiments.
Gumbel Softmax (Jang et al., 2017): Gradients from the reconstruction loss are propagated to the inference network with the straight-througe Gumbel estimator.
REINFORCE (Sutton et al., 2000): Gradients from reconstruction loss are propagated to the inference network with ELBO as a reward function. This method has been used in
previous work for semi-supervised sequence generation (Miao & Blunsom 2016 Yin et al. 2018), but often suffers from instability issues.
We report the train and test ELBO along with task evaluations in Table 5 and plot the learning curves on validation set in Figure 5 While being much simpler, we show that the stop-gradient trick produces superior ELBO over Gumbel Softmax and REINFORCE. This result suggests that stopping gradient helps better optimize the likelihood objective under our probabilistic formulation in comparison with other optimization techniques that propagate gradients, which is counter innuttrrintiiivee likely explanation is that as gradient estimator, while clearly biased, stop-gradd stop-gradient has substantially reduced variance. In comparison with other techniques that offer reduced bias but extremely high variance when applied to our model class (which involves discrete sequences as latent variables). stop-grade nop-gradient actually leads to better optimization of our objective because it achieves better balance of bias and variance overall.
We remove REINFORCE from this figure since it is very difficutt to stabilize training and obtain reasonable
results (c.g. the BLBO value is much worse than others in Table

---

Published as a conference paper at ICLR 2020
-2.5
EL8O
-3.5
-4.0
gumbel snflmma
gradient
-5.0
10000
20000
30000
40000
50000
step
Figure 5: ELBO the validation set the number training steps.
CONCLUSION
We propose probabilistic generative forumalation that unites past work unsupervised text style transfer. We show that this probabilistic formulation provides different way to reason about unsupervised objectives in this domain. Our model leads to substantial improvements five text style transfer tasks, yielding bigger gains when the styles considered are more difficult to distinguish.
ACKNOWLEDGEMENT
The work of Junxian He and Xinyi Wang supported by the DARPA GAILA project (award (IR0011190006) and the Tang Family Foundation respectively. The authors would like to thank Zichao Yang for helpful feedback about the project.
REFERENCES
Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. Unsupervised neural machine translation. In Proceedings of ICLR, 2018.
Mikel Artetxe, Gorka Labaka, and Eneko Agirre. An effective approach to unsupervised machine translation. arXiv preprint 190.00190..01313 2019.
Dzmitry Bahdanau. Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In Proceedings of ICLR, 2015.
Colin Bannard and Chris Callison-Buul Burchh Paraphrasing with bilingual parallel corpora. In Proceedings ACL, 2005.
Samuel R Bowman, Luke Vilnis, Oriol Vinyals, Andrew Dai, Rafal Jozefowicz, and Samy Bengio. Generating sentences from continuous space. In Proceedings of ConNLL, 2016.
Qing Dou and Kevin Knight. Dependency-base based decipherment for resource-limited machine transla- tion. Proceedings of EMNLP, 2012.
Di He, Yingce Xia, Tao Qin, Liwei Wang, Nenghai Yu, Tie-Yan Liu, and Wei-Ying Ma. Dual learning for machine translation. In Proceedings of NeurIPS, 2016.
Zhiting Hu, Zichao Yang. Xiaodan Liang, Ruslan Salakhutdinov, and Eric P Xing. Toward controlled generation of text. In Proceedings of ICML, 2017.
Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax. In Proceedings of ICLR, 2017.
Harsh Jhamtani, Varun Gangal, Edward Hovy, and Eric Nyberg. Shakespearizing modern language using copy-enriched sequence-to-sequence models. Proceedings of EMNLP, 2017.
10

---

Published as a conference paper at ICLR 2020
Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg. Greg Corrado, et al. Google's 's multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational Linguistics, 2017.
Yoon Kim. Convolutional neural networks for sentence classification. In Proceedings of EMNLP, 2014.
Diederik P Kingma and Max Welling. arXiv:1312.61 14, 2013.
Auto-encoding variational bayes. arXiv preprint
Kevin Knight, Anish Nair, Nishit Rathod, and Kenji Yamada. Unsupervised analysis for decipherment problems. In Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions, pp. 499- 506, 2006.
Guillaume Lample. Alexis Conneau, Ludovic Denoyer, and Mare' Aurelio Ranzato. Unsupervised machine translation using monolingual corpora only. arXiv preprint arXiv: 1.00043, 2017.
Guillaume Lample, Myle Ott, Alexis Conneau, Ludovic Denoyer, and Marc' Aurelio Ranzato. Phrase- based & neural unsupervised machine translation. arXiv preprint arXiv:180.007755, 2018.
Guillaume Lample, Sandeep Subramanian, Eric Smith, Ludovic Denoyer, Marc' Aurelio Ranzato, and Y-Lan Boureau. Multiple-atrribut text rewriting. In Proceedings of ICLR, 2019.
Juncen Li, Robin Jia, He He, and Percy Liang. Delete, retrieve, generate: A simple approach to sentiment and style transfer. arXiv preprint arXiv:1804.06433 2018.
Yishu Miao and Phil Blunsom. Language latent variable: Discrete generative models for sentence compression. In Proceedings of EMNLP, 2016.
Ronen Mir, Bjarke Felbo, Nick Obradovich, and Iyad Rahwan. Evaluating style transfer for text. In Proceedings of NAACL, 2019.
Masahiro Mizukami, Graham Neubig, Sakriani Sakti, Tomoki Toda, and Satoshi Nakamura. Linguis- tic individuality transformation for spoken language. In Natural Language Dialog Systems and Intelligent Assistants. 2015.
Graham Neubig, Zi-Yi Dou, Junjie Hu, Paul Michel, Danish Pruthi, and Xinyi Wang. compare-mt: A tool for holistic comparison of language generation systems. In Meeting of the North American Chapter of the Association for Computational Linguistics (NAACL) Demo Track, Minneapolis, USA, June 2019. URL
Nima Pourdamghani and Kevin Knight. Deciphering related languages. Proceedings of EMNLP. 2017.
Sudha Rao and Joel Tetreault. Dear sir or madam, may i introduce the gyafc dataset: Corpus, benchmarks and metrics for formality style transfer. arXiv preprint wrXiv:/803.06533 2018.
Sujith Ravi and Kevin Knight. Deciphering foreign language. In Proceedings of ACL, 2011.
Alexander M Rush, Sumit Chopra, and Jason Weston. A neural attention model for abstractive sentence summarization. In Proceedings of EMNLP, 2015.
Rico Sennrich, Barry Haddow, and Alexandra Birch. Improving neural machine translation models with monolingual data. In Proceedings of ACL, 2016.
Claude Elwood Shannon. A mathematical theory of communication. Bell system technical journal, 27(3):79-423, 1948.
Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. Style transfer from non-parallel text by cross-alignment. In Proceeings of NIPS, 2017.
Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. Proceedings of NeurIPS, 2000.
11

---

Published as a conference paper at ICLR 2020
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. In Proceedings of NeurlPS, 2017.
Wei Xu, Alan Ritter, William B. Dolan, Ralph Grishman, and Cherry Colin. Paraphrasing for style. COLING, 2012.
Zichao Yang, Zhiting Hu, Chris Dyer, Eric P Xing, and Taylor Berg- Kerg-Kirkpatrick. Unsupervised text stylee transfer using language models as discriminators. In Proceedings of NeurIPS, 2018.
Pengcheng Yin. Chunting Zhou, Junxian He, and Graham Neubig. Structvae: Tree-structured latent variable models for semi-supervised semantic parsing. In Proceedings of ACL, 2018.
Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In Proceedings of ACL,, 2017.
12

---

Published as a conference paper at ICLR 2020
A APPENDIX
A.1 MODEL CONFIGURATIONS.
We adopt the following attentional encoder-decoder architecture for UNMT, BT+NLL, and our method across all the experiments:
We use word embeddings of size 128.
We use layer LSTM with hidden size 512 both the encoder and decoder.
We apply dropout to the readout states before softmax with rate of 0.3.
pooling operation over the encoder hidden states before feeding it to the decoder. Intuitively the pooling window size would control how much information is preserved during transduction. A window size of is equivalent to standard attention mechanism, and large window size corresponds to no attention. See Appendix A.2 for how to select the window size.
Following Lample et al. (2019), add
There is noise function for UNMT baseline in its denoising autoencoder loss (Lample et al.] 2017 2019), which is critical for its success. We use the default noise function and noise hyperparameters in Lample al. (2017) when running the UNMT model. For BT+NLL and our method we found that adding the extra noise into the self-reconstruction loss (Eq. is only helpful when the two domains
relatively divergent (decipherment
and related language translation tasks) where the language models play a less important role. Therefore, we add the default noise from UNMT Eq. for decipherment and related language translation tasks only, and do not use any noise for sentiment, author imitation, and formality tasks.
A.2 HYPERPARAMETER TUNING.
We vary pooling windows size {1,5), the decaying patience hyperparameter k for self- reconstruction loss (Eq. 4) as (1,2,3}. For the baseliens UNMT and BT+NLL, we also try the option of not annealing the self-reconstrrutionn loss all as in the unsupervised machine translation task (Lample et al. 2018). We vary the weight for the NLL term (BT+NLL) or the KL term (our method) {0.001, 0.01, 0.03, 0.05, 0.1).
A.3 SENTIMENT TRANSFER EXAMPLE OUTPUTS
We list some examples of the sentiment transfer task in Table Notably, the BT+NLL method tends to produce extremely short and simple sentences.
A.4 REPETITIVE EXAMPLES OF BT+NLL
In Section 5 we mentioned that the baseline BT+NLL has low perplexity for some tasks because tends to generate overly simple and repetitive sentences. From Table I we see that two representative tasks are sentiment transfer and formatliy transfer. In Appendix A.3 we have demonstrated some examples for sentiment transfer, next we show some repetitive samples of BT+NLL in Table 7
13

---

Published as a conference paper at ICLR 2020
Table 6: Random Sentiment Transfer Examples
Methods
negative to positive
Original UNMT
the cake portion was extremely light and a bit dry- the cake portion was extremely light and a bit spicy the cake portion was extremely light and bit dry the cake portion was extremely light and a bit fresh
BT+NLL
Ours
Original UNMT BT+NLL Ours
the chicken strip were paper thin oddly flavored strips
the chicken were extra crispy noodles were fresh and incredible.
the service was great.
the chicken strip were paper sweet & juicy flavored.
Original
i could give them zero star review would if could give them zero star review would love this place. love the restaurant and give great review would
UNMT
BT+NLL
Ours
positive to negative
Original UNMT
great food, staff is unbelievably nice
no food is n't particularly friendly.
BT+NLL
will be back
Ours
no apologies, staff is unbelievably poor.
Original UNMT
my wife and love coming here my wife and do n't come here will not be back
BT+NLL
Ours
my wife and walked out the last time.
Original
my wife and love coming here my wife and do n't come here will not be back my wife and walked out the last time.
UNMT
BT+NLL
Ours
Original UNMT
the premier hookah lounge of las vegas! !
the worst museum of las vegas ! the worst frame shop of las vegas the hallways scam lounge of las vegas
BT+NLL
Ours
Table 7: Repetitive examples of BT+NLL baseline on Formality transfer.
Original
Transferred
formal informal
I like Rhythm and Blue music
like her and don't know
There's nothing he needs to change
don't know. but I don't know
enjoy watching my companion attempt to role @- play with them watching right now
don't know. hut don't know know,
don't don't 't know. but don't know
That the key point, . that you fell asteep.
informal formal
its great source just download it. Happy Days. it was the coolest
do not know but I do not know I do not know but I do not know
I used to play flute but once started sax, got hooked.
not know but do not know
The word you looking for ........ strengths Plus you can tell she really cared about her crew.
The word you looking for
Plus you can tell she really cared about her crew
14


Reviews:
[{"review_id": "SkxpKw0nYB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors propose a probabilistic framework for unsupervised text style transfer. Given two non-parallel corpora X,Y in different domains, the authors introduce unobserved corpora \\bar{X}, \\bar{Y}. These are used as latent variables that control the generation of the observed data. To train models, the paper proposes to optimize the evidence lower bound of the log marginal likelihood. To facilitate training, multiple techniques are suggested, such as parameter sharing, some gradient approximations and initialization with a reconstruction objective. The approach is evaluated on five style transfer tasks, as well as unsupervised machine translation. Models are evaluated with multiple metrics, and generally obtain reasonably strong performance.\n\nI lean towards the acceptance of the paper because the approach is fairly simple and elegant, while obtaining promising results. The connections to back-translation and language models are also potentially interesting. However, while the paper aims to suggest a principled approach to style transfer, using greedy samples biases the reconstruction objective, and as such the method does not really optimize the ELBO.\n\nCasting style transfer as data completion is a straight-forward idea that doesn't introduce unnecessary or too simplistic assumptions. Optimizing the ELBO follows naturally, and can lead to more diverse outputs than the BT+NLL approach, which misses the negative entropy term. Reference BLEU scores on all tasks are competitive, and sometimes clearly better, with strong baselines.\n\nGreedily sampling latent sequences during training should ideally be justified more carefully as it biases the objective function. In particular, an experimental comparison to stochastic sampling, which should more closely approximate the expectation, would be appreciated. Additionally, detailing the similarities and differences between the proposed approach and current UNMT techniques could be helpful to some readers.\n\nQuestions:\n\nCould you present the validation and test evidence lower bounds? If so, how is sampling performed?\n\nIn footnote 2, you mention tuning the strength of the KL regularizer. As the KL can be decomposed into 2 terms (Eq. 5), would it be beneficial to control each term separately?", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "SJl7ALxqYH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The main contribution of this paper is a principled probabilistic framework of unsupervised sequence to sequence transfer (text to text in particular). \n\nHowever, I believe there is a large disconnect between the probabilistic formulation written it section 3 and whats actually happening experimentally in section 5. It is not clear whether the model is *actually* optimizing an ELBO because the gradients from sequence reconstruction loss are not backpropogated to the inference network as explained in paragraph on Approximating Gradients of ELBO. Moreover this restriction makes the authors method almost the same as the one used for unsupervised neural machine translation by Lample et al 2017 and Artetxe et al 2017. I would like to see a more detailed analysis from authors on how far the performance of Gumbel-softmax estimator and REINFORCE estimator is from simple stop-gradient estimator used in experiments.\n\nIn terms of experimental setup I like that the authors considered a large suite of experiments across various tasks. Although the evaluation metrics on text style transfer tasks like sentiment transfer, formality transfer, author imitation are in line with previous work ideally the human evaluation needs to be done to truly see how well each method performs. On unsupervised machine translation, authors show a large improvement on Serbian-Bostian translation. I am a bit skeptical since as I wrote above the proposed method is very similar to previously proposed unsupervised neural machine translation approaches and it is not clear why we are seeking such a large gain of 5 BLEU points.\n\nOverall I think it is a well written paper with a large experimental suite, although I am skeptical of actual connection between probabilistic formulation and whats actually happening in practice.\n\n================================================\nUpdate: I have raised the score from 3 to 6.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}, {"review_id": "B1gcj6Fe5r", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThis paper introduces a probabilistic generative model for\nunsupervised style transfer of text. The approach introduced in\nthe paper does not require paired training data. An\nencoder-decoder model is trained to transfer text from one style\nto another and back.\n\nReview:\n\nThis work is very well-written and easy to follow. The\ncontribution is clearly articulated as while there are\nprobabilistic generative models for transfer in the\nliterature (Shen et al does include one) they don't perform as\nwell. Ablation studies further confirm the need for the\nparticular kind of parameter sharing used in the model in the\npaper. Great results are shown on 5 text transfer problems.\n\nClarifications and improvements:\n\nJust for clarity, in the last paragraph on page 4. It says two encoder-decoder\nmodels are learnt, but isn't the idea that there is effectively only one\nencoder and one decoder learned that just put together in different ways\nduring training? I'm also curious why the baseline of BT+NLL was so strong? Is\nhaving the loss of a language model work that much better than the regular\nentropy term?\n\nI would also like if possible if you could share some of the repetitive examples\ncreated by BT-NLL which explain its low PPL.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes an unsupervised text style transfer model which combines a language model prior with an encoder-decoder transducer. They use a deep generative model which hypothesises a latent sequence which generates the observed sequences. It is trained on non-parallel data and they report good results on unsupervised sentiment transfer, formality transfer, word decipherment, author imitation, and machine translation. The authors responded in depth to reviewer comments, and the reviewers took this into consideration. This is a well written paper, with an elegant model and I would like to see it accepted at ICLR. 
Decision: Accept (Spotlight)