Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This work has proposed a new method to textual style transfer, which is based on the assumption that there exist some pseudo-parallal sentences pairs between two styles. It first construct synthetic parallel corpora by using two similarity measures: semantic similarity based on larger LMs and scene graph similarity. Then it trains the generation model via reinforced imitation learning. 
main_review: Strengths:
1. The scene graph similarity used to select pseudo parallel pairs is novel and makes sense to me.
2. The reinforced imitation learning shows better performance compared with baseline of MLE loss for one-to-many mapping.
3. The empirical results establish the effectiveness of this method.

Weaknesses:
1. This method is based on the assumption that there are parallel pairs in the original corpora of two styles so that sentences of the same content and different styles can be found out, which limits the use of the method to cases where abundant corpora are existing and there are parallel sentences there in the corpora. In those cases where only small unsupervised corpora are there and there are no sentences pairs sharing the same content, this method would not work.
2. In the current used three datasets, I would like to know how good are those paired sentences selected out. This kind of quantitative human evaluation of constructed pseudo-parallel corpora is very important to establish the effectiveness of this work, which cannot be lacking.
3. What is the BLEU score? Is it self-BLEU or ref-BLEU? If ref-BLEU, is it calculated on one human reference or four? I know at least for Sentiment transfer and formality transfer, the test sets contain four references. 
4. In Table 1, according to the Tabl1 of the original DualRL paper: https://arxiv.org/pdf/1905.10060.pdf, the BLEU scores can be 55.2 and 44.9 for Yelp and GYAFC datasets, while the scores reported in this work are significantly lower. I am not sure whether the references are different here. Please explain this gap.
5. In Table 1, the performance of Ours w/. RD also looks very high, especially for the Political Stance dataset, the difference between w/. RD and w/. S-Emb are very small for BLEU and PPL of w/. RD is even lower, which contradicts to my intuition. Using a pseudo-parallel corpora constructed by random selection should not produce any good performance, should it? This kind of RD baseline even outperforms many strong SOTA baselines, for example of DUAL RL, which is ironic. 
6. In Table 1, it shows that the SAS score is only helpful for one datasets out of three, then why is it so? Why is it only useful in the formality case?
7. This work is inspired by this work: https://aclanthology.org/D19-1306/, however, it has never been compared with it. 
summary_of_the_review: This work has merits, but its weaknesses are also salient as mentioned above. I am looking forward to author responses for addressing my concerns, or I vote for rejecting this work.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes an approach to addressing text style transfer with non-parallel data. The basic idea is to follow three steps: (1) for a given source sentence, mine some nearly parallel sentences from the target domain; (2) perform an MLE learning; and (3) augmented with an imitation learning. 

The proposed method was evaluated on three text style transfer tasks: (1) sentiment transfer; (2) formality transfer; and (3) a new task called political stance transfer. 
main_review: Strengths

- A step-wise way of training text style transfer models on non-parallel datasets
- A new dataset for style transfer, collected from the political domain. 

Weakness

- First, I am not sure about the novelty of the proposed method. Given the ideas of MLE and RL/IL (specifically, the INFORCE algorithm used in this work) has been used in text generation for a while, I am not sure how much novelty here by simply combining them together. 
- Second, the writing of this work needs to be improved.
  - In section 2.2, the Scene Alignment Score seems to be an important component of the proposed first step, however, I don't think this work ever (1) explain what is a scene graph and (2) justify the validity of using the Scene Alignment Score for alignment. The numbers in Table 1 may provide some empirical evidence for the second question, but it is not sufficient. 
  - I was confused by the terms used in section 2.4, including "Reinforced Imitation Learning", "Reinforced Policy Gradient". To be specific, based on the description, I didn't understand the difference between reinforced imitation learning and imitation learning. In addition, I am also not sure what reinforced policy gradient is. It looks like the description in section 2.4 mixes many terms together, without an appropriate explanation. 
  - In Algorithm 1, what is $J_{IL}^{safe}$? 
  - About table 1, I am not sure how the highlights were selected in this table. Apparently, not all the highlights are the best results. 
summary_of_the_review: My major concern of this work is the technical novelty. In addition, the writings of this paper also needs to be improved.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes to transform the popular unsupervised style transfer to a semi supervised task. First the paper proposes simple methods to mine parallel sentences from two unlabelled corpora. Further, they use the mined parallel sentences, in a MLE based training (sequence to sequence framework using BART) and refine it using Contrastive Learning based imitation learning. The paper achieves improvements on preserving the content between the transferred sentences. In addition, the authors also propose a new dataset for political style transfer in a semi-automated way which is a welcome addition to the paper.

main_review: **Strengths**

The paper tackles an important issue of text style transfer with novel ideas that aims to preserve content in an effective manner. 
The paper is relatively well written with justifications and reasons for the choices. The authors could have given examples 
The paper presents a simple idea to use reinforcement learning: imitation learning in particular  to better maintain the content between the two styles. 

**Weaknesses**

1. The authors assume that  parallel sentences inherently exist between the source and the target style corpus. However, the existing unsupervised text style transfer methods do not make such assumptions. Since the entire basis of the paper is on such an assumption, providing examples of the inherent parallel sentences is crucial. The paper can be made stronger by  providing  examples (either manually picked or mined).

2. **Scene graph alignment** - What are the scene entities? The authors mention that the use (Wu et al. 2019) paper on Unified Visual Semantic Embeddings. The reviewer skimmed through the other paper as well. However, which parser is used in the current paper?  Wu et al. use a syntactic parser to mine the objects in the sentence, adjectives, nouns, subjects of verbs etc and initialize an embedding for different entities and learn a joint embedding space. Here are my concerns with respect to this.
a) The authors need to be clear on the parser which is used to parse the sentence in this paper. Referring to Wu et al.’s paper without describing what the scene graph entities make it unclear on the reason to include this as a refinement step. 
b) If the above parser to extract objects is used, how is it important for style transfer? For example, the style transfer datasets used in the paper might not share any “visual” entity like a clock, plate etc. Instead they might share some abstract entities like the service provided, the cleanliness of the restaurant etc. 

     The reviewer understands that the sentences that talk about similar entities are important, the approach to obtain those entities is unconvincing. Seeing Figure S1 in the appendix, it is clear that the scene entities are words that are salient in the given style or words that carry content. Although the reviewer understands the idea, it would be a good idea to rewrite this to make it clear to the reader.


3. **Evaluation Measures**: The used evaluation measures in the paper have known problems. For example, using BLEU does not measure semantic similarity with the input sentence and certain sentences with unnatural structure can still have low perplexity scores. Refer to Kalpesh Krishna et al 2020(https://arxiv.org/abs/2010.05700), which is included as a baseline method to compare against. The adoption of these measures would have made the paper stronger. Although the reviewer agrees that the majority of the text style transfer papers use the metrics used in this paper, the reviewer would appreciate if the authors could evaluate based on the paper from Krishna et al 2020 and add it in the Appendix or supplementary material

4. **Evaluation**: The evaluation section should concentrate on the major contribution of the paper: maintaining the content between the two styles well. For example, with imitation learning we expect the BLEU metrics (and the i-PINC metric)  to do better because imitation learning aims to preserve the content in a better manner between the source and the target style which needs to be the highlight of the evaluation section. The other metrics might not be the focus of the paper and achieving high measures on them is an added bonus. The word “performance” is excessively used in the evaluation section. Instead, the specific measure that the others are talking about should be mentioned. 
	Also, Mean and Standard Deviation with multiple experiment runs are missing from the evaluation section.

5. **Conclusion**: The conclusion is a summary of the main findings of the paper. It would be ideal if the authors can provide implications of the current work going forward. Can this be extended to other works that try to control the fine-grained syntax of the sentence etc would be a welcome addition to the conclusion 

6. **New Dataset**: More details on the new dataset introduced in the paper is needed. Why is the new dataset more challenging compared to the other two datasets that are mentioned in the paper?

**Minor Typos, Grammatical Errors and Other Readability Issues**

1. There is an extra space on Page 2 Para 2 Line 2 after Figure till (b)
2. The authors introduce the term “scene graphs” in the introduction on Page 2 Para 2 and Line 4. Readers who are not aware of the computer vision literature might be left wondering on how scene graphs are relevant for Text Style Transfer 
3. The rows showing ablations in Table 1 are confusing “w/” is read as without or with. The authors could instead use “+” or “-” symbol to indicate with or without
4. Table  1 highlight boxes need to be explained to the reader.  Please mention that you are highlighting the best performing method for that metric. Also some of highlight boxes are missing in the Sentiment dataset

summary_of_the_review: 1. Overall, the paper is easy to read and the ideas to improve content preservation between the source and the target sentence is simple and effective. The ablation studies and parameter sensitivity studies are well performed.  
2. The reviewer would require more details on the entities that are preserved between the sentences, examples on the inherent parallel sentence between the two corpora in the introduction.
3.  Problematic evaluation measures used in the literature need to be revised 
4. The evaluation section needs to focus on the main contribution of the paper. Splitting Section 3.2 into more paragraphs might be a starter. 
5. The benchmark table needs to have multiple runs with mean and standard deviation reports.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper addresses a text-style transfer task based on non-parallel datasets.

The authors propose a three-step approach:
- First, to match each source sentence with several sentences from the target style dataset using sentence embedding similarity and scene graph matching.
- Next, to fine-tune a seq2seq network (BART) in an NMT-like setup with several references.
- Finally, to use imitation learning to enforce the loss contrast between the best target candidate and all others.

The authors propose a new dataset for the TST task, called "political stance transfer," and use it for evaluation among the previously known datasets. The results of such a combined approach are competitive with previous works.
main_review: Strengths:

- The paper uses a novel combination of existing approaches to achieving competitive results.

- The code provided with the paper is clean, structured, and can be useful for developing future models for the same task.

- The paper itself is well written and structured.

Weaknesses and questions:

- The authors claim "existing methods learn a mapping without considering the self-parallelism, ... they tend to learn the mapping between source and target style by randomly mapping sentence pairs." However, no proofs or references for such behavior were given. On the contrary, most of the works mentioned as related have no such problematic behavior by design.

- The first step of the proposed approach (matching sentences from the source and target corpora) implies that both corpora have a similar distribution. However, it is not the case for some definitions of "styles." This assumption possibly limits the method's applicability, but it wasn't addressed in the motivation or discussion sections.

- Moreover, the token-level scene preservation scheme implies that scene entities couldn't be style markers themselves. This assumption also limits possible applicability until the opposite is proven.

- Again, the part of the loss is based on the difference in the orders of scene entities, but in practice, it can be an important part of the style. Consider the example from the 'political stance' dataset presented with the paper:
Style1: Mulvaney tapped to lead Trump s budget office.
Style2: Trump picks debt warrior Mulvaney to lead White House budget office.

- There is a known inevitable trade-off between content-preservation and style-transfer metrics. Thus, to show the improvement of the new model, one needs to demonstrate Pareto-improvement both of them (in this case, this means the improvement both in terms of ACC and BLEU metrics). However, the reported results show the TSF-DelRetGenLM model is better in terms of one of these two metrics for both sentiment and formality datasets. Thus, the evaluation shows the comparability of the model with the baselines on two standard datasets (and the superiority only on the novel proposed dataset). At the same time, the authors claim "SOTA results according to many metrics on the three TST tasks," which doesn't sound fair. 





summary_of_the_review: I like the proposed approach and the clarity of the paper and code.
I'm slightly concerned with the paper's novelty since it presents a combination of previously known methods, but still, I think it can be useful for the community.
The main concern is somehow ambiguous motivation and definition of the scope of applicability of the proposed approach (check the questions above). 
I believe this should be clarified.

My score for the paper is "marginally above the acceptance threshold".
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: Style transfer is a text generation task where sentences need to be rewritten in such a way that a particular target attribute is introduced (like formality), but at the same time the content is approximately preserved. Style transfer is usually researched in unsupervised settings --- researchers assume no access to parallel data of sentences differing only in the target attribute. However, most prior works assume access to a corpus of unpaired sentences in each of the target attributes.

The main technical idea in this paper is searching for (mining) roughly parallel pairs in the unpaired corpora, to create a "pseudo-parallel dataset". The authors use two methods to extract pairs - (1) semantic similarity based on RoBERTa embeddings; (2) similarity of a scene-graph built by using an off-the-shelf parser. These pseudo-parallel data is used to fine-tune a large pretrained language model (BART). To improve performance, the authors utilize imitation learning via REINFORCE. The key idea is checking whether the current greedy sample is closer by a margin to the closest mined parallel sentence (positive example), compared to other mined parallel sentences (negative examples). If not, sampling is used to explore the space and the sampled sequence is rewarded / penalized according to the increase / decrease in margin using REINFORCE.

The authors test their approach on sentiment transfer, formality transfer and politcal stance transfer (which is a new dataset introduced by them), and note improvements over several popular baselines on automatic metrics and human evaluations. The authors also analyze their choice of hyperparameters and conducted several ablation studies to justify their design decisions.
main_review: *Strengths*

1. I liked the ideas in the paper. As far as I know, scene graphs have not been used previously in style transfer and are a nice way to ensure semantic preservation. I believe parallel data mining has been done before in unsupervised machine translation [1] and pretraining [2] (papers I suggest the authors cite), but not for style transfer. The part of the approach using imitation learning to refine the pseudo parallel data is also new and effective as confirmed by the ablation studies.

2. The authors collect a new dataset for political stance rewriting. This dataset is fairly large (56K pairs) and the task sounds harder than sentiment transfer. This is a valuable contribution and it could be a useful benchmark for future style transfer research. The authors have also carefully curated the dataset to remove hate speech, and used a filtering method (Appendix B) to ensure mined sentences are faithful to the attribute.

3. The authors show improvements in three different text rewriting tasks over several baselines on both automatic metrics and human evaluations. Additionally, the authors provide ablation studies and analyze the choice of their data mining hyperparameters.

*Weaknesses*

1. While I liked the ideas in the paper, I thought the main technical contribution of mining psuedo parallel data from an unpaired corpus is fundamentally limited. First, there's no guarantee the unpaired corpus will have a similar content distribution across attributes (which is required to retrieve roughly parallel sentences). The formality / political stance transfer benchmarks tested were originally parallel datasets, which ensures a shared content distribution. Second, you might need a very large unpaired corpus to ensure this mining is successful, which may not be practical in lower resource settings (this is why recent work is moving towards few-shot setups for style transfer [4, 5]). I suggest the authors try style transfer tasks where the content distributions will be very different across attributes, such as Shakespeare <---> Tweets from [6].

2. Not a very big concern (since your method outperforms baselines on all three metrics), but it will be nice to see an aggregated overall score for style transfer performance (especially on human evaluation). The three style transfer metrics (accuracy, similarity, fluency) often conflict with each other [7], so an overall score is generally more informative. You could do this for both automatic and human evaluations. For human evaluations, since you use 7-point Likert scales you could follow the approach in [8] who calculate the number of instances which got a Likert score of (let's say 5 or more) on all three metrics *simultaneously*. A more general alternative is presented in [4, 6] which can be used for both automatic and human evaluations.

**Minor**

1. The acronym LaMer is not specific to the contributions of this paper (pseudo parallel data, scene graphs, imitation learning). It is also a slang for "dull" in informal settings. I suggest changing this acronym to something else.

2. In 2.4, you could alternatively use unlikelihood training [3] on the amateur policies.

3. I suggest reducing focus on the details of the RL methodology and adding more details on the new benchmark dataset proposed in the main body of the paper.

4. Question about human evaluation ---> did you randomly shuffle the order of generations from different systems? That way you would avoid annotator bias from their previous annotations ("the last generation is always best")

5. Several links seem to be broken, especially those to the Appendix.

6. I suggest the authors add more qualitative outputs from their system into the Appendix part of the PDF, rather than in an external attachment.

[1] - https://aclanthology.org/P19-1178  
[2] - https://arxiv.org/abs/2006.15020  
[3] - https://arxiv.org/abs/1908.04319  
[4] - https://arxiv.org/abs/2110.07385  
[5] - https://aclanthology.org/2021.acl-long.293  
[6] - https://arxiv.org/abs/2010.05700  
[7] - https://arxiv.org/abs/1910.03747  
[8] - https://aclanthology.org/N18-1169
summary_of_the_review: Overall, I'm in favour of acceptance since the paper has several interesting new ideas, a large new dataset for political stance transfer, and automatic + human evaluation comparing the benefits of their approach against baselines. The weakness #1 prevents me from giving the next higher score (of 8).

-----------

**After author response**: I'm pleasantly surprised by the few-shot style transfer results, and commend the authors for their rigorous rebuttal to all reviewers. I think the paper will be stronger with results on a dataset with a different content distribution between the two styles (irrespective of size, such as Shakespeare <---> Tweets). My overall assessment is more like a 7 or 7.5, which means I'll increase my score to 8.
review_title: null

Meta-review: The paper proposes a new method for unsupervised text style transfer by assuming there exist some pseudo-parallal sentences pairs in the data. The method thus first mines and constructs a synthetic parallel corpus with certain similarity metrics, and then trains the model via imitation learning. Reviewers have found the method is sound and the empiricial results are decent. The assumption on pseudo-parallal pairs would limited the application of the methods in other settings where the source/target text distributions are very different. The authors have added discussion on this limitation during rebuttal.
Decision: Accept (Poster)