Contextual Text Style Transfer
Authors: Yu Cheng
Zhe Gan
Yizhe Zhang
Oussama Elachqar
Dianqi Li
Jingjing Liu
Venue:  (2020)

Abstract: In this paper, we introduce a new task, Contextual Text Style Transfer, to translate a sentence within a paragraph context into the desired style (e.g., informal to formal, offensive to non-offensive). Two new datasets, Enron-Context and Reddit-Context, are introduced for this new task, focusing on formality and offensiveness, respectively. Two key challenges exist in contextual text style transfer: 1) how to preserve the semantic meaning of the target sentence and its consistency with the surrounding context when generating an alternative sentence with a specific style; 2) how to deal with the lack of labeled parallel data. To address these challenges, we propose a Context-Aware Style Transfer (CAST) model, which leverages both parallel and non-parallel data for joint model training. For parallel training data, CAST uses two separate encoders to encode each input sentence and its surrounding context, respectively. The encoded feature vector, together with the target style information, are then used to generate the target sentence. A classifier is further used to ensure contextual consistency of the generated sentence. In order to lever-age massive non-parallel corpus and to enhance sentence encoder and decoder training, additional self-reconstruction and back-translation losses are introduced. Experimental results on Enron-Context and Reddit-Context demonstrate the effectiveness of the proposed model over state-of-the-art style transfer methods, across style accuracy, content preservation, and contextual consistency metrics.

Under review as a conference paper at ICLR 2020
CONTEXTUAL TEXT STYLE TRANSFER
Anonymous authors
Paper under double-blind review
ABSTRACT
In this paper, we introduce a new task, Contextual Text Style Transfer, to trans- late sentence within paragraph context into the desired style (e.g., informal to formal, offensive to non-offensive). Two new datasets, Enron-Context and Reddit- Context, are introduced for this new task, focusing formality and offensiveness, respectively. Two key challenges exist in contextual text style transfer: 1) how to preserve the semantic meaning of the target sentence and its consistency with the surrounding context when generating an alternative sentence with a specific style; 2) how to deal with the lack of labeled parallel data. To address these challenges, we propose a Context- Aware Style Transfer (CAST) model, which leverages both parallel and non non-paralle non-parallel data for joint model training. For parallel training data, CAST uses two separate encoders to encode each input sentence and its surround- ing context, respectively. The encoded feature vector, together with the target style information, are then used to generate the target sentence. A classifier is further used to ensure contextual consistency of the generated sentence. order lever- age massive non-parallel corpus and to enhance sentence encoder and decoder training, additional relostecosttuttionn and back-translation losses are introduced. Experimental results on Caron-Context and Conditt-Cotextttt demonstrate the effec- tiveness of the proposed model over state-of ofte--ff-hhhrrrrrr style transfer methods, across style accuracy, content preservation, and contextual consistency metrics!!
INTRODUCTION
Text style transfer has recently been applied to many applications with remarkable success (c.g., sentiment manipulation, formalized writing). Early work relied on parallel corpora with a sequence- to-sequence learning framework (Bahdanau al. 2015 Jhamtani et al. 2017). However, collecting annotations for parallel data is highly time-consuming. There has been a recent surge of interest in developing text style transfer models using non-parallel data (Hu al. 2017 Li al., 2018 Prab- humoye et al. 2018)) Subramanian et al. 2018), assuming that tisentangling style information from semantic content can achieved in an auto-encoding fashion with the introduction of additional regularizers (e.g., adversarial discriminators (Shen et al. 2017) or language models (Yang et al., 2018).
Despite promising results, these techniques still have a long way towards practical use. Specifically, existing models mostly focus on sentence-level rewriting. However, real-world applications, sen- tences typically reside in proper context such as paragraph. For example, in the formalized writing task, the rewritten span should align well with the surrounding context (e.g., personal email, scientific content) to keep coherent text flow. Taking single sentence as the sole input of style transfer model may fail to preserve topical coherency of the generated sentence with its surrounding context, resulting in poor semantic and logical consistency the paragraph level (see Example C in 4
Motivated by this, we propose and investigate paragraph, the system aims to automatically edit sentences into a desired style, while keeping the edited section topically coherent with its surrounding context. To achieve this goal, we propose novel Context-Aware Style Transfer (CAST) model, by jointly considering style transfer and con- text alignment. For parallel training data. CAST uses two separate encoders encode the source sentence and its surrounding context, respectively, and decoder to translate the encoded features
task Contextual Text Style Transfer. Given
Source code, and collected new datasets will he released upon acceptance.

---

Under review as a conference paper at ICLR 2020
into the target sentence. A pre-trained coherence classifier is further applied to regularize the gen- erated target sentence to be consistent with the context. To overcome the data sparsity issue, we further leverage non- non-parallel data by using hybrid approach. With large-scale non-paraia non-parallel corpus, the training of the sentence encoder and decoder are enhanced via additional seff-reccostrructiin and fack-translation objectives. A pre-trained style classifier is also used for style regularization. The final CAST model is jointly trained with both parallel and non non-parallel data.
As this is newly proposed task, we also introduce two new datasets, Enron-Context and Reddit- Context, collected via crowdsourcing. The former contains 14,734 formal vs. informal paired sam- ples from Enron (Klimt& Yang 2004) (an email dataset), and the latter contains 23,158 offensive vs. non-offensive paired samples from Reddit (Serban et al.) (017). Each paired sample contains an original sentence and a nuwrttrewritten sentence with the desired style, accompanied by its para- graph context. Besides this, in order to enhance model training, we exploit additional 28.375/29,777 7774 formal/informal non non-parallel sentences from GYAFC (Rao & leetreault 2018), and 53,028/53,714 offeessieemnon-offenss
non-parallel sentences from Reddit (dos Santos al. 2018).
The main contributions of this work are summarized as follows: (i) We propose a new task Con- textual Text Style Transfer, which aims to translate an input sentence into a desired style, while preserving its style-irrelevant semantics and topical consistency with the surrounding context. (it) We introduce two new datasets for this task, Enron-Context and Reddit-Context which provide liable benchmarks for measuring contextual style transfer models. (fii) We present a new model Context-Aware Style Transfer (CAST), which jointly optimizes the generation quality of the target sentence and its topical coherency with adjacent sentences. Extensive experiments on these two new datasets demonstrate that the proposed CAST model outperforms state-of-the-an baselines.
RELATED WORK
Text Style Transfer Text style transfer aims to modify an input sentence into a desired style while preserving its style-independent semantics. Previous work has explored this as sequence-to- sequence learning task using parallel corpora with paired source/target sentences different styles. For example, Jhamtani et al. (2017) pre-trained word embeddings by leveraging external dictionar- ies mapping Shakespearean words to modern English words and additional text. However, available parallel data in different styles very limited. Therefore, there is recent surge of interest in con- sidering a more realistic setting, where only non-parallel stylized corpora are available. A typical approach is: (i) disentangling latent space as content and style features; then (ii) generating stylistic sentences by tweaking style-relevant features and passing them through a decoder, together with the original content-relevan features (Xu et al. 2018
Many of these approaches borrowed the idea of adversarial discriminnator/lassifee from the Gener- ative Adversarial Network (GAN) framework (Goodfellow al., 2014). For example, [henn et al.
(017); Fu et al. 2018);; Lample et al. (2088) used adversarial classifiers to force the decoder to transfer the encoded source sentence into a different style/language. Alternatively, [i al. (2018) achieved disentanglement by filtering stylistic words of input sentences. Another direction for text style transfer without parallel data is using back-translation (Prabbumoye et al. (2018) with a de-
noising auto-encoding objective (Logeswaran et al. 2018: Subramanian et al. 2018))
Regarding the tasks, sentiment transfer one of the most widely studied problems. From informality to formality (Rao & Tetreault, 2018) is another direction of text style transfer, aiming change the style of a given sentence to more formal text. dos Santos et al. (2018) presented an approach to transferring offensive text to non-offensive based social network data. In Prabhumoye et al. 2018), the authors proposed the political slant transfer task. However, all these previous studies did not directly consider context-aware text style transfer, which is the main focus of this work.
Context-aware Text Generation Our work is related to context- aware text generation (Mikolov & Zweig 2012 Tang et al., 2016), which can be applied to many NLP tasks (Mangrulkar et al. 2018). For example, previous work has investigated language modeling with context information (Wang &
Cho 2015; Wang et al. 2017). treating the preceding sentences as context. There are also studies on response generation for conversational systems (Sordoni et al. 2015b Wen et al. 2015). where dialogue history is treated as a context. Zang & Wan (2017) introduced a neural model to generate long reviews from sentieetentiment scores given the topics. Vinyals & Le (2015) proposed model
2

---

Under review as a conference paper at ICLR 2020
Context
Encoder
(c(c,)
Context C;
Classifier
cohere
(Cantextual Coherence Loss)
Target style 1;
Sentence
Target
L
(Cantextual Loss)
Input
Sentence Encoder
E, (x.)
CAST with paraile!
Original style L;
Sentence Decoder
Recon. X;
1 resuor
(Reconstruction Loss)
Input
Sentence Encoder
{x,
Style Classifier
I Classfication
Target style I;
Sentence Decoder
Trans. x,
Sentence
(8.)
Sentence
Decoder
Recon. X
Bback-trans
(Bark. Transkation loss)
Original
CAST non pon--aralllll
Figure I: Model architecture of the proposed CAST model for contextual text style transfer. Both the training paths share the same sentence encnder and decoder. See Sec. for details.
to predict the next sentence given the previous sentences in dialogue session. (Sordoni et al. (2015a) presented hierarchical recurrent encoder-decoder model to encode dialogue context. Our work the first to explore context information in the text style transfer task.
3 CONTEXTUAL TEXT STYLE TRANSFER
In this section, we first describe the problem definition and provide an overview of the model ar- chitecture in Section 3.1 Section 3.2 presents the proposed Context-Aware Style Transfer (CAST) model with parallel data, and Section 3.3 further introduces how to augment the CAST model with non-parallel data in a hybrid approach.
3.1 OVERVIEW
Problem Definition The problem of contextual text style transfer is defined follows. Given style-labelled parallel dataset p {(x, 4)), (y,,4), c; J.M. where the i-th instance contains the original sentence x; in style 4. its corresponding rewritten sentence y; in another style 4. and the paragraph context C, and y: are expected to contain the same semantic content, but in different language styles (i.e., 4 * L.. The goal is to transform x, in style 4 to in style 4, while keeping the sentence y; semantically coherent with its context C, In practice, labelled parallel data may be difficult to garner. Therefore, we assume that additional non-paralle data l4 {(x;, 4)) can be leveraged to enhance overall model training.
Training Objective The overall architecture of the proposed CAST model is illustrated in Figure I The hybrid model training process consists of two paths, one for parallel data and the other for non-parallel data. In the parallel path, Seq2Seq loss and contextual coherence loss are defined, to learn the two encoders (sentence encoder and context encoder) and the sentence decoder with labeled parallel data. The non-poraal parallel path is designed to further enhance the sentence encoder and decoder with three additional losses: (1) a sefftreconttructiion loss; (ii) back-translation loss; and (iii) style classification loss. The overall training objective, taking both parallel and non-parallel paths into consideration, can be written as:
L P.M final L: + X L + Iy + >a trans + A At style
IP
(1)
where Å1, λ2, A and X4 are hyper-paranen parameters to balance different objectives. Each these loss terms will be explained in the following sub-sections.
3.2 CAST WITH PARALLEL DATA
In this subsection, we discuss the training objective associated with parallel data, consisting of (t) contextual Seq2Seq loss; and (ii) contextual coherence loss.
3

---

Under review as a conference paper at ICLR 2020
Contextual Seq2Sey Loss When parallel data available, Seq2Seq model can be directly learned for text style transfer. We denote Seq2Seq model as (E, D), where the semantic represen- tation of sentence X; is extracted by the encoder E' (i.e., (x,)), and the decoder D aims to learn conditional distribution of y; given the encoded feature (xx)) and style 4; (ygpp(y((Exx))
E
However, in such a saneence-t--snteenee style transfer setting, the context of the paragraph ig- nored, which if well utilized, could help improve the quality of generated text (such as paragraph- level topical coherence).
Thus, to take advantage of the paragraph context c; information, we use two separate encoders E., and Eg to encode the sentence and the context independently. The outputs the two encoders are combined via a linear layer, to obtain a context-aware sentence representation, which is used for generating the target sentence. The model trained to minimize the following loss:
1e-02
E
log E.())
(3)
Compared with Eqn. 2 the use of Ea(c,) makes the text style transfer process context-dependent. The generated sentence can be denoted as y; = ((E,(x)), Ec(,,
Contextual Coherence Loss To enforce contextual coherence (i.e., to make the generated sen- tence align with the surrounding context c,), we train a coherence classifier that aims to distin- guish whether c; the context of y,, by adopting a language model with an objective similar to next sentence prediction (Devlin et al. 2019).
Specifically, assume that y: is the (-th sentence of paragraph pa (i.e., Y: p.")), and c; {p.
} is its surrounding context. We first reconstruct the paragraph 3 by inserting y; into the proper position in c; denoted as yr). Based on this, we obtain paragraph representation u, via a language model encoder. Then, we apply linear layer to the representation, followed by a tanh function and softmax layer to predict a binary label 8;, which indicates whether c, is the context of y;:
(f+1) ..... P; (T)
P: = (P.
u, = LM((c,; f(y:))
(4)
(LM Ci, ys) softmax (tanh (Wu; + b))
(5)
where LM represents the language model encoder, and Note that since y; are discrete tokens which are non-differentiable, we use the continuous feature, denoted as ((..), that generates ý, as the input of the language model. We construct paired data {y:, Ci, )x)==1 for training the classifier, where the negative samples are generated by replacing a sentence in paragraph with another random sentence. After pre-training, the coherence classifier used to obtain the contextual coherence loss:
indicates that c; is the context of y:-
L Lockere x.,e.~P
logPLM(s 1|c, f(y:))
Intuitively, minimizing L encourages the y; to blend better to its context C,. Note that the coherence classifier is pre-trained, and fixed during the training of the CAST model. The above coherence loss can be used to update the parameters of Es, F,, and during model training.
3.3 CAST WITH NON-PARALLE DATA
For the contextual style transfer task, there are not many parallel datasets available with ltbeeed paragraph pairs. To overcome the data sparsity issue, we propose to further boost the CAST model by leveraging additional non-parallel data 2/ (xx, 4 which are less expensive collect. In order to fully exploit l/ to enhance the training of the sentence encoder and decoder (E,, D), we introduce three additional training losses, detailed below.
Reconstruction Loss The reconstruction loss aims to encourage E, and D to reconstruct the input sentence itself, if the desired style is the same the input. The corresponding objective similar to Eqn. (2)
Lif.co
IE loggo (x. x.<il
4

---

Under review as a conference paper at ICLR 2020
Dataset
# sent.
# rewritten sent.
words per sent.
# words per paragraph
# vocabulary
Reddit-Context
14,734
14,734
9.4
38.5
4,622
Enron-Conter
23,158
25,259
7.6
25.9
2,196
Table 1: Statistics on Enron-Context and Reddit-Context datasets.
Compared with Equ. 2 here we encourage the decoder to recover x; original stylistic property accurate as possible when given the style label 4. The self-reconstruuted sentence is denoted as & (x,,))),
Back- Translation Loss The back translationn loss requires the model reconstruct the input sen- tence after transformation loop. Specifically, the input sentence X; is first transferred into the target style, i.e., x, D(E, (x,), Then the generated target sentence is transferred back into its original style, i.c., X, (x),4) The tack-translation loss is defined as:
1 bacckttrone
S (E (x),)(x,1E(()).44)
E
(8)
where the source style is denoted 4, and the target style is denoted 7,.
Style Classification Loss To further boost the model, we use U to train classifier to predict the style of given sentence, and regularize the training of (E,, with the pre-trained style classifier. Specifically, the objective for training the style classifier is:
Latyle
E log (4:1.). x.-ld
where (-) denotes the style classifier. After the classifier is trained, we keep its parameters fixed, and apply it to update the parameters of (E,; D). Specifically, the style classification loss defined over the pre-traing pre-trained style classifier can be written as:
L
E ximld (F
E
log Pc(4)))) +
E
log PC ([,1x))
(10)
Ex
NEW DATASETS FOR CONTEXTUAL TEXT STYLE TRANSFER
Existing text style transfer datasets, matter parallel non-parallel, do not contain contextual information, thus are not suitable for our new task. Therefore, we introduce two new datasets:
and Reddit-Context, derived from two existing datasets Enron (Klimt & Yang 2004) and Reddit Politics (Serban et al. 2017). respectively.
Enron-Context To build a formality transfer dataset with paragraph contexts, we randomly sam- pled emails from the Enron corpus (Klimt & Yang 2004). After pre-processing and filtering with NLTK (Bird et al. 2009), we asked Amazon Mechanical Turk (AMT) annotators to identify infor- mal sentences within each email, and rewrite them in more formal style. Then, asked different annotators to verify if each rewritten sentence is more formal than the original sentence.
Reddit-Context We further collected a new offensive vs. non-offensive dataset from the Reddit Politics corpus (Serban et al. 2017). First, we performed classification the original dataset at sen- tence level, to identify offensive sentences from whole paragraphs. After filtering some extremely long/short sentences, natives. To provide robust datasets for benchmark, we randomly selected a subset of sentences to be rewritten into two references, which makes up of 10% of the whole dataset.
asked AMT annotators to rewrite the offensive ones to non-offensive alter-
After manually removing wrong or duplicated annotations, we obtained a total of 14,734 rewritten sentences for Ehron-Context and 23, for Reddit-Context. We also limited the vocabulary size by using words with frequency equal or larger than 20 (70) in Enron (Reddit). Table provides the statistics on the datasets.
Non-parallel Corpus Besides parallel datasets, also explore non-parallel datasets to enhance model training. For formality transfer, one choice is Grammarlys Yahoo Answers Formality Corpus (GYAFC) (Rao & Tetreault 2018), crawled and annotated from two domains in Yahoo Answers.
5

---

Under review as a conference paper at ICLR 2020
Formality Transfer
Non-parallel
Train
Style classifier
Parallel
Train
Dev
Test
Coherence classifier
GYAFC
58k
12k
Enron-Context
13k
0.5k
1k
2.5k
Offensiveness Transfer
Non-parallel
Train
Style classifier
Parallel
Train
Dev
Test
Coherence classifier
REDDIT
106k
15k
Contitt-Cotext
22k
0.5k
1k
3.5k
Table 2: Statisties of the parallel and non-parallel datasets on the two text style transfer tasks.
This corpus contains paired informal and formal sentences, without context. We randomly selected subset of sentences from the original dataset, and used it in a non-parallel manner. By the end, we collected 28,375/29,77, 7777 formal/informal sentences. The second dataset is the offensive/non- offensive Reddit dataset. Following dos Santos et al.] (2018), we used a pre-trained classifier to extract offiensive/non-offensiii sentences from Reddit posts. In total, we collected 53,028/53,711 offensiveemonnoofeeiivee sentences.
EXPERIMENTS
In this section, we compare our model with state-of-the-an baselines on the two new datasets, and provide both quantitative analysis and human evaluation to validate the effectiveness of our model.
5.1 EXPERIMENTAL. SETUP
Datasets Table 2 provides a summary of the parallel and non-parallel datasets used for the two style transfer tasks. For the non-parallel datasets, we split them into two: one for the proposed model training, and the other for the style classifier pre-training. Similarly, for the parallel datasets, the training sets are divided into two as well, for the training of CAST and the coherence classifier, respectively.
Evaluation Metrics The contextual style transfer task requires generating sentences to: (i) pre- serve the original content and structure in the source sentence; (ki) conform to the pre-specified style; and (iii) align with the surrounding context in the paragraph. Thus, we consider the following automatic metrics to evaluate the effectiveness different methods:
(i) Content Preservation. We assess the degree of content preservation based r-gram statistics, by measuring BLEU scores (Papineni et al. 2002) between generated sentences and human refer- ences. Following Rao & Tetreault (2018), also use GLEU for the formality transfer task, which was originally introduced for the grammatical error correction task (Napoles et al. (015).. For offen- siveness transfer, we include perplexity (PPL) as used in dos Santos al. (2018), which is computed by a word-level LSTM language model pre-trained on non-offensive sentences.
(ii) Style Accuracy. Similar prior work, we generate samples from the model, and measure style accuracy (i.c., Acc. of the pre-trained style classifier).
(iii) Context Coherence. As aforementioned, we use the pre-trained coherence classifier measure how the generated sentences match the surrounding context.
For formality transfer, the pre-trained style classifier and coherence classifier reaches 91.35% and 86.78% accuracy, respectively. For offenssivnness transfer, the accuracies are 93.47% and 84.96% respectively. Therefore, we consider them reliable serve evaluation metrics.
Baselines We compare our proposed model with several baselines: (i) Seq2Seq: Transformer- based Seq2Seq model (i.e., Eqn. (2)). taking only sentences as inputs, and trained only on parallel data; (ii) Contextual Seq2Seq:
Transformer-based contextual Seq2Seq model (i.e., Eqn. (.). laking both context and the sentence as input, and trained only on parallel data; (iii) Hybrid Seq2Seq (Xu et al. 2019); a Seq2Seq model leveraging both parallel and non-parallel data; (iv) ControlGen (Hu et al. 2017 2018): : state-of-the-an text transfer model using non- paralllll data.
Implementation Details The context encoder, sentence encoder and sentence decoder are all im- plemented as a lne-r Transformer with 4 heads. The hidden dimension of one head is 256, and

---

Under review as a conference paper at ICLR 2020
Formality Transfer
Offensiveness Transfer
Model
Acc.
Coherence
BLEU
GLEU
Acc.
Coherence
BLEU
PPL
Seq2Seq
64.05
78.09
24.16
10.46
83.05
80.28
17.22
140.39
Contextual Seq2Seq
64.28
81.25
23.72
10.37
83.42
81.69
18.74
138.42
Hybrid Seq2Seq
65.09
79.62
24.35
10.93
83.28
84.87
20.78
107.12
ControlGen
62,18
73.66
14.32
8.72
82.15
78.81
10.44
92.14
CAST
68.04
85.47
26.38
15.06
88.45
85.98
23.92
93.03
Table 3: Quantitative ovaluation results of different models on the two style transfer tasks
Task: informal formal transfer
Context
Input
I'm assuming that you't 'd set up be part of that meeting
I'll call him back to meeting, [Input]. asked him what sort of deals they're 're working on.
ControlGen
I' I'm guessing that you would be set that call ?
C-Seq2Seq
I am ussuming that you would part of that person..
H-SeqqSeq
I am assuming that you would be part of that party ?
CAST
Am I correct ussume that you would attend that meeting
Input
Do y all interface with C/P Do you compete with them ?
Thanks Can someone
ControlGen
the C/P know that the deals are good [Input]. ]. If not deal confirmations could but they need the deal details.
C-Seq2Seq
Do we interface with them?
H-Seq2Seq
Do we interface with them?
CAST
Do you all interface with C/P
Task: offensive to non-offensive transfer
Context
Input
are ugly..
ControlGen
You bad guy
With the glasses, [Input]. I don't need them because never read How do look
C-Seq2Seq
Have bad day
H-Seq2Seq
What a bad day
CAST
You look not good
Table 4: Examples from the two datasets, where orange denotes the sentence be transferred, and blue denotes content that also appears in the context. C-Seq2Seq: Contextual Seq2Seq; H-Seg2Seq: Hybrid Seq2Seq.
the hidden dimension of the feed-forward sub-layer is 1024. The context encoder is set take max- imum of 50 words from the surrounding context of the target sentence. For the style classifier, we use a standard CNN-based sentence classifier (Kim 2014).
Since the non-parallel corpus l/ contains more samples than the parallel corpus P, we down-sample to assign each mini-batch the same number of parallel and non-parallel samples to balance the training. alleviating the catastrophic forgetting problem described in Howard & Ruder (2018). We train the model using Adam optimizer with mini-batch size 64 and learning rate 0.0005, The vali- dation set is used to select the best apper-paramcters. Hard-samplin (Logeswaran et al. 2018) is used to back- pack-propagate the loss through discrete tokens from the pre-trained classifier to the model.
For the ControlGen (Hu al. 2017) baseline, their default model following the original paper.
the code provided by the authors, and use setting. For Hybrid Seq2Seq (Xu et al. 2019), we re-implement their
5.2 EXPERIMENTAL RESULTS
Formality Transfer Results on the formality transfer task are summarized in Table The CAST model. which leverages both context and non-parallel data, achieves the best performance over all the baselines. Particularly, CAST is able to boost GLEU and Coherence scores with a large margin. Hybrid Seq2Seq also achieves good performance by utilizing non-parallel data. By incorporating the context information, Contextual Seq2Seq also improves over the vanilla Seq2Seq model. As expected, ControlGen does not perform well, since only non-parallel data is used for training.
Offensiveness Transfer Results are summarized in Table CAST achieves the best performance over all the metrics except for the PPL. In terms of Coherence, both methods that leverage the context achieve better performance compared with the Seq2Seq baseline. Contextual Seq2Seq also improves BLEU, which is different from the observation in the formality transfer task. Our model produces slightly worse performance on PPL than ControlGen. We hypothesize that this is because our model tends to use the same non-offensive word replace an offensive word, producing some unusual sentences.
7

---

Under review as a conference paper at ICLR 2020
Formality Transfer
Offensiveness Transfer
Model
Acc.
Coherence
BLEU
GLEU
Acc.
Coherence
BLEU
PPL
CAST
68.04
85,47
26.38
15.06
88.45
85,98
23.92
93.03
w/o context encoder
65.35
82.9
23.98
14.17
84.15
80.96
20.54
127.02
w/o cohere. classifier
65.47
80.16
14.82
14.45
85.11
79.37
21.97
115.57
w/o both
62.19
74.47
15.88
10.46
72.69
78.15
13.14
147.31
w/o non-parallel data
60.19
75.49
13.5
9.88
70.84
78.72
10.53
151.08
Table 5: Ablation study of CAST on the style transfer tasks.
CAST
CAST
CAST vs. ControlGen
Task
Aspects
Contextual Seq2Sey
Hybrid Seq2Seq
win
lose
tic
win
lose
tic
win
lose
tic
Style Control
57.1
28.3
14.6
46.9
26.
28.0
72.1
12.6
25.3
Formality
Content Preservation
59.7
22.1
18.2
50.4
20.8
28.2
68.8
14,5
17.7
Transfer
Context Consistence
56.4
23.1
20.5
51.5
19.7
28.8
70.1
10.6
19.3
Style Control
58.6
25.3
16.1
50.1
29.2
20.3
54.8
19.9
25.3
Offensiveness
Content Preservation
62.3
26.5
11.2
54.0
17.5
28.5
53.1
30.2
16.7
Transfer
Context Consistence
60.1
32.4
17.5
55.3
24.9
20.8
58.1
35.8
16.7
Table 6: Results of pairwise human evaluation between CAST and three baselines on both tasks. Win/lose/tie indicate the percentage of texts generated by CAST are better/worseeeqqal to the compared model.
Qualitative Examples Table presents some qualitative examples. Generally, we observe that our model better at replacing informal words with formal ones (Example and C), and generate more context-aware sentences (Example A and C), possibly due the use of coherence and style classifiers. We also observe that the exploitation the context information can also help our model preserve the semantic content in the original sentence (Example B).
Ablation Study To investigate the effectiveness of individual components, we perform ablation studies by removing some components of the proposed model. Results on both tasks are provided in Table The context encoder and the coherence classifier play an important role in the proposed model. The context encoder able to improve content preservation and style transfer accuracy, while the coherence classifier can help improve the coherence score but not much for style accuracy. By using these two components, our model can find a proper balance between transferring to the correct style and maintaining the consistency with context. When both of them are removed (the 4th row), performance on all the metrics drops significantly. We also observe that without using non-parann non-parallel data, the model performs poorly, showing the importance of using hybrid method.
Human Evaluation Considering the subjective nature of this task, we conduct human evaluation based on content preservation, style control and context consistency, following Mir al. (2019). Given the original sentence and the transferred sentence with the corresponding context, the AMT crowd-source workers were asked to select the best one based on these three aspects. The evaluation interface also allows neutral option, if the worker considers both sentences as equally good in certain aspect. We randomly sampled 200 sentences from the corresponding test set, and collected three human responses for each pair. Table(orrporrts the pairwise comparison results on both tasks. Based on human judgment, the quality of transferred sentences by CAST significantly higher than the other methods on all three metrics. This is consistent with our observation in the experiments on automatic metrics.
6 CONCLUSION
In this paper, we present new task contextual text style transfer. To provide benchmarks for this new task, we introduce two new datasets, which contain annotated sentence pairs accompanied by paragraph contexts. We also propose new model, which can jointly capture content preservation and context coherence, and exploit additional abundant non-parallel data for boosting performance. In both quantitative and human evaluations, our approach significantly outperforms baseline meth- ods that do not rely on context information. Ablation study also demonstrates the effectiveness of different components in the model design. We believe current model takes first step towards modeling context information for text style transfer, and would like to explore more advanced solu- tions to integrating context.
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning align and translate. In ICLR, 2015.
Steven Bird, Ewan Klein, and Edward Loper. Natural Language Processing with Python. O'Reilly Media, 2009.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL, 2019.
Cicero Nogueira dos Santos, Igor Melnyk, and Inkit Padhi. Fighting offensive language on social media with unsupervised text style transfer. In ACL, 2018.
Zhenxin Fu, Xiaoye Tan, Nanyun Peng, Dongyan Zhao, and Rui Yan. Style transfer in text: Explo- ration and evaluation. In AAAI, 2018.
Ian Goodfellow, Jean Pouget-Abadie, Mchdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. NeurlPS. 2014.
Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text classification. In ACL, 2018.
Zhiting Hu, Zichao Yang. Xiaodan Liang, Ruslan Salakhutdinov, and Eric Xing. Toward controlled generation of text. In ICML, 2017.
Zhiting Hu, Haoran Shi, Zichao Yang, Bowen Tan, Tiancheng Zhao, Junxian He, Wentao Wang, Lianhui Qin, Di Wang, et al. Texar: modularized, versatile, and extensible toolkit for text generation. arXiv preprint arXiv:1809.00794 2018.
Harsh Jhamtani, Varun Gangal, Eduard Hovy, and Eric Nyberg. Shakespearizing modern language using copy-enriched sequence-to-sequeee models. arXiv preprint 1771011007.01161, 2017.
Yoon Kim. Convolutional neural networks for sentence classification. OrXiv:1408.5882 2014.
arXiv preprint
Bryan Klimt and Yiming Yang. Introducing the enron corpus. CEAS, 2004.
Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc' Aurelio Ranzato. Unsupervised machine translation using monolingual corpora only. In ICLR, 2018.
Juncen Li, Robin Jia, He He, and Percy Liang. Delete, retrieve, generate: simple approach to sentiment and style transfer. In NAACL, 2018.
Lajanugen Logeswaran, Honglak Lee, and Samy Bengio. Content preserving text generation with attribute controls. In NeurlPS, 2018.
Sourab Mangrulkar, Suhani Shrivastava, Veena Thenkanidiyoor, and Dileep Aroor Dinesh. A context-aware convolutional natural language generation model for dialogue systems. In SIG- DIAL, 2018.
Tomas Mikolov and Geoffrey Zweig. Context dependent recurrent neural network language model. IEEE Spoken Language Technology Workshop (SLT), 2012.
Remi Mir, Bjarke Felbo, Nick Obradovich, and Iyad Rahwan. Evaluating style transfer for text.
arXiv preprini arXiv: 100.002990002955 2019.
Courtney Napoles, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. Ground truth for grammatical error correction metrics. In ACL, 2015.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. In ACL, 2002.
Shrimai Prabhumoye, Yulia Tsvetkov, Ruslan Salakhutdinov, and Alan W Black. Style transfer through back-translation. arXiv preprint arXiv: 1804.09000, 2018.

---

Under review as a conference paper at ICLR 2020
Sudha Rao and Joel Tetreault. Dear sir madam, may i introduce the gyafe dataset: Corpus, benchmarks and metrics for formality style transfer. In NAACL, 2018.
Iulian Vlad Serban, Chinnadhurai Sankar, Mathieu Germain, Saizheng Zhang, Zhouhan Lin, Sandeep Subramanian, Taesup Kim, Michael Pieper, Sarath Chandar, Nan Rosemary Ke, Sai Mudumba, Alexandre de Brébisson, Jose Sotelo, Dendi Suhubdy, Vincent Michalski, Alexan- dre Nguyen. Joelle Pineau, and Yoshua Bengio. A deep reinforcement learning chatbot. arXiv preprint arXiv: /0t..//99990399 2017.
Tianxiao Shen, Tao Lei, Regina Barzilay, and Tommi Jaakkola. Style transfer from non-parallel text by cross-alignment. NeurIPS, 2017.
Alessandro Sordoni, Yoshua Bengio, Hossein Vahabi, Christina Lioma, Jakob Grue Simonsen, and Jian-Yun Nie. A hierarchical recurrent encoder-decoder for generative context-aware query sug- gestion. In CIKM, 201
Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jun Nie, Jianfeng Gao, and Bill Dolan. A neural network approach to context-sensitive generation of conversational responses. In NAACL, 2015b.
Sandeep Subramanian, Guillaume Lample, Eric Michael Smith, Ludovic Denoyer, Marc' Aurelio Ranzato, and Y-Lan Boureau. Multiple-attribute text style transfer. arXiv preprint 1rXiv:1811.000 2018.
Jian Tang, Yifan Yang. Samuel Carton, Ming Zhang, and Qiaozhu Mei. Context-aware natural language generation with recurrent neural networks. arXiv preprint 1rXiv:1611009900 2016.
Oriol Vinyals and Quoc V. Le. A neural conversational model. arXiv preprint 2015.
Tian Wang and Kyunghyun Cho. arXiv:1511.03729. 2015.
Larger-context language modelling. arXiv preprint
Wenlin Wang, Zhe Gan, Wenqi Wang. Dinghan Shen, Jiaji Huang, Wei Ping, Sanjeev Satheesh, and Lawrence Carin. Topic compositional neural language model. arXiv preprint arXi:11712.09783, 2017.
Tsung- Hsien Wen, Milica Gasic, Dongho Kim, Nikola Mrksic, Pei-Hao Su, David Vandyke, and Steve Young. Stochastic language generation in dialogue using recurrent neural networks with convolutional sentence reranking. In Proceedings of the 16th Annual Meeting of the Special Interest Group Discourse and Dialogue, 2015.
Jingjing Xu, Xu Sun, Qi Zeng, Xuancheng Ren, Xiaodong Zhang, Houfeng Wang, and Wenjie Li. translation: cycled reinforcement learning approach. arXiv preprint 11X5v:1805.0511 2018
Unpaired
Ruochen Xu, Tao Ge, and Furu Wei. Formality style transfer with hybrid textual annotations. arXiv preprint 1903.v9900.00633 2019.
Zichao Yang, Zhiting Hu, Chris Dyer, Eric P Xing, and Taylor Kirkpaiikp.trick. Unsupervised text style transfer using language models as discriminators. In NeurlPS, 2018.
Hongyu Zang and Xiaojun Wan. Towards automatic generation of product reviews from aspect- sentiment scores. In Proceedings of the 10th International Conference Natural Language Generation, 2017.
10

---

Under review as a conference paper at ICLR 2020
Thes Daik about Mestihiin "Blood Sentiments" the given persgrrpho. Glactintnation
missepara
Hight gef phrooe senten thot feel beired seno ments sub serect on, as arpler below!
Do rubert can wort has be aortince
This (uslatted) piraxctentencs as cllebrat aved Sentinema" destinanaion
poalcit arivtty moogger hornight
Lloeenee how sertmwnt
ELECTION
Musi be de phose the uame denement not comvtting not coroneed
START
righ jutament any grllos sourcie withi promotor recurt, Prodgety ho thngh other forne @botinmation)
Sinwog
Figure 2:
The AMT interfaces we used collect the bias/violence data. The top interface for turkers to select the bias/violence data from a paragraph. The bottom interface is for turkers to rewrite the sentence/phrase given the context.
Data Collection: We the offensive language and hate speech classifier from (Davidson al., 2017) to classify the offensive sentence. We perform the classification the sentence level for each Reddit post.
As shown in Figure the data collection are divided into two sub-tasks: first select the bias/violence data from paragraph and then rewrite the sentence/phrase given the context.
The rewritten sentences from The 23,158 Reddit-Context validated ourselves. Each rewritten sentence is reviewed by one volunteer to check if inoffensive while preserves the original content.
are validated by one of collaborators from company.
AMT Interface:
In Figure we show the AMT user interfaces to collect the bias/violence data.
User Study: In Table 7 present more qualitative examples both tasks.

---

Under review as a conference paper at ICLR 2020
Task: informal to formal transfer
Context
Input
I'll share what come up with later will share you what come up later on.
If anyone has some of their classics, please forward to with a full citation. [Input]. Bob
ControlGen
C-Seq2Seq
will call you later.
H-Seq2Seq
will share with you later.
CAST
will share my ideas later.
Input
you were pissed last night
ControlGen
you were born last night.
You still mad at me? [Input]. Sorry.
B
C-Seq2Seq
we were upset last night
H-Seq2Seq
you were upset last night.. You were angry last night.
CAST
Tass: offensive to non-offensive transfer
Context
Input ControlGen
My finking teinnn tooth!
Oh, it's my tooth!
My bad job
[Input].
C
C-Seq2Seq
What a big big tooth.
can't even think anymore
H-Seq2Seq
What a bad news.
because of it.
CAST
My broken tooth!
Table 7: Additional examples from the datasets, where orange denotes the sentence to he transferred, and blue denotes content that also appears context. C-Seq2Seq: Contextual Seq2Seq; H-Seq2Seq: Hybrid Seq2Seq.
12


Reviews:
[{"review_id": "rJlA-nW1qS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a new task for text style transfer, based on the idea that the the surrounding context of a sentence is important, whereas previous such tasks have only looked at sentences in isolation. Two new crowd-sourced datasets are created, and a combination of now fairly standard neural components is shown to outperform some strong baselines on the new datasets, on a variety of evaluation metrics. An ablation analysis of the components - including some auto-encoding auxiliary losses  - shows that all the various parts are helpful to performance.\n\nOverall I don't see a lot wrong with the technical content of the paper, and it is well-written and easy to understand. I also feel that the collection of the data is a contribution, and this could be a useful dataset to the community. But the main problems I have with the current version - and the reason I am unable to recommend acceptance as it stands - are twofold:\n\n1) the task doesn't make a lot of sense to me. If I have a paragraph written in a particular style, e.g. Shakespearean (an example in the paper), why would I want to rewrite a single sentence in e.g. modern English? Likewise, if I have a paragraph that's largely negative in sentiment (assuming for now this is a kind of style; see below), why would I want to change one of the sentences to a positive sentiment? I can understand why I might want to take a whole paragraph, or document, and rewrite it in another style, but not a sentence in isolation. Now it may be that the proposed method could be used consecutively to rewrite a larger unit, but it's not obvious how that would work and it needs some discussion.\n\n2) it's not clear to me what is meant by \"style\" in this paper. No definition is provided, and the authors rely on references to previous papers and various examples throughout the paper for the reader to grasp what is meant here. Examples include: informal/formal, offensive/inoffensive, Shakespearean, sentiment, political slant. The first three look fine as examples of style; the latter two I'm not sure, since they are likely to change the semantic content. A minor point related to this one is that in the Text Style Transfer related work subsection, there's even a reference to an MT paper as an example.\n\nHaving said all that, I think the new corpus could potentially be a useful resource (assuming the authors can utilise it for a meaningful task). I would have liked to see more details of how the corpus was collected. Putting the style guidelines in an Appendix would be a possibility. But more details could be given in section 4, for example: how are the offensive sentences identified? (which classifier?) How did you manually remove wrong annotations, and what does wrong mean here? (presumably not by checking all 14k examples?) \n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "r1lVfEwqYr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose the task of contextual text style transfer: transferring the style of one text into another (i.e., informal to formal, or offensive to non-offensive), when the text is present within some larger, provided context. The authors propose a model (CAST) which takes advantage of the additional context to perform the style transfer. CAST outperforms previous style transfer models according to several automatic metrics, as well as human evaluation.\n\nOverall, I am not convinced that the context is useful in the proposed tasks (Reddit-context and Enron-context), for 2 reasons:\n* In the examples shown in Table 4, it seems possible to complete the proposed contextual style transfer tasks without the additional context (i.e., \"Do y’all interface with C/P\" -> \"Do you all interface with C/P\" for formal->informal transfer).\n* CAST performs better across several metrics (including human evaluation), but I am not sure that the gain is from using the context in the manner that the authors suggest (increasing the generated sentence's coherence with the context). In particular, it is hard for me to see how some style transferred sentences can appear much more coherent than others, given that the the required changes are pretty local (i.e., word substitution). On the other hand, style transfer models seem to have some difficulty in maintaining semantic context, so it seems that some of the gains in e.g. BLEU could be from CAST using the context simply as a bag of words to draw from in generating a style-transferred sentence. Instead of including context, the issue of semantic correctness could be fixed in other, simpler ways, however, such as making a more extractive single-sentence style transfer model. I think some further analysis is necessary to conclude that the context makes the generations more coherent, rather than helping the model in some other way.\n\nOther general concerns/questions for the authors:\n* The baselines do not appear to have learned even the single-sentence style transfer task that well. It appears that the authors used the default hyper parameters for other style transfer models (Hybrid Seq2Seq and ControlGen), but wouldn't it be better to choose hyper parameters for those models based on the evaluation criteria (as is done for CAST)? I know that some style transfer models can be quite sensitive to hyperparameters, so I would be surprised if the models worked optimally out of the box. I also feel that a strong unsupervised machine translation model like XLM (or the related style transfer model from \"Multiple-attribute text style transfer\") could do better than the provided baselines, as these models seem to work notably better than their predecessors.\n* I have a general concern that CAST is quite complicated and has a lot of moving parts and parameters to tune. I don't think there are ablations on all components of the model (i.e., the style classifier). I also wouldn't be surprised if a simple state-of-the-art unsupervised or semi-supervised machine translation model like XLM outperformed CAST, without using any context. To add the use of the context, it seems possible to make a simple modification to an existing state-of-the-art transfer/translation model like XLM (adding the context to the input)\n* Why are the style and coherence classifiers fixed during training? Wouldn't it be better to update the classifiers while training the rest of the model (so they doesn't get exploited)?\n* Which of the evaluated models/baselines are trained using the style and/or coherence classifiers directly used for evaluation? For models that are not trained to optimize the style and/or coherence classifier predictions, I am not sure it is fair to compare those models to CAST on the style/coherence classifier accuracy, since CAST has been directly trained to perform well on that metric.\n* Are the evaluation classifiers identical to those used to train CAST? At the least, it seems better to use a different style/coherence classifier (i.e., trained with a different random seed or other hyperparameters) at test time that the one used during training (i.e., in case CAST has overfit the style/coherence of generations to the classifiers used during training).\n* In Eqn. 5, why is there a tanh activation immediately preceding the softmax? Wouldn't the tanh limit the range of the output softmax probabilities?\n\nThe proposed datasets do seem useful to the community, and the empirical results do support that CAST performs better than baselines - it would just be helpful to understand in what way the gains are coming from the additional context (to know if the task is an interesting task).", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}, {"review_id": "SkeIzu9tYB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "\n=== Summary ===\n\nThe paper considers the following task: given a sentence, its context paragraph, and the target style (formalness, inoffensiveness), generate a new sentence that still fits the original context. The evaluation metrics include content preservation (BLEU, GLEU, PPL), style accuracy (based on a pre-trained model), and context coherence (ditto).\n\nThe input context is the main highlight of this new task. The paper proposes a model that can use both parallel and non-parallel data. In addition to the existing non-parallel corpora, two datasets with crowdsourced parallel sentences were collected. Compared to the baselines, the proposed model produces sentences that are more relevant to the context.\n\n=== Review ===\n\nThe major contributions of this paper are the new task setting and the accompanying parallel corpora. The datasets are rather large, and the fact that they contain parallel sentences (e.g., informal - formal sentences with the same semantics) makes the dataset potentially useful for other scenarios as well.\n\nThe proposed method is a natural extension of previous style transfer methods. One new addition is the contextual coherence loss, which is a relatively straightforward application of BERT-based models. If I am not missing anything, the model and the experiments are likely correct. In addition to the surrogate metrics, the paper also presents human evaluation, which strengthens the results.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a new style transfer task, contextual style transfer, which hypothesises that the document context of the sentence is important, as opposed to previous work which only looked at sentence context. A major contribution of the paper is the creation of two new crowd-sourced datasets, Enron-Context and Reddit-Context focussed on formality and offensiveness. The reviewers are skeptical that it was context that has really improved results on the style transfer tasks. The authors responded to all the reviewers but there was no further discussion. I feel like this paper has not convinced me or the reviewers of the strength of its contribution and, although interesting, I recommend for it to be rejected. 
Decision: Reject