Original: N ON -P ARALLEL T EXT S TYLE T RANSFER WITH S ELF -P ARALLEL S UPERVISION
Revised: N ON -P ARALLEL T EXT S TYLE T RANSFER WITH S ELF -P ARALLEL S UPERVISION

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: The performance of existing text style transfer models is severely limited by the non-parallel datasets on which the models are trained.
Revised: The performance of existing text style transfer models is severely limited by the non-parallel datasets on which the models are trained.

Original: In non-parallel datasets, no direct mapping exists between sentences of the source and target style; the style transfer models thus only receive weak supervision of the target sentences during training, which often leads the model to discard too much style-independent information, or utterly fail to transfer the style.
Revised: In non-parallel datasets, no direct mapping exists between sentences of the source and target style; the style transfer models thus only receive weak supervision of the target sentences during training, which often leads the model to discard too much style-independent information, or utterly fail to transfer the style.

Original: In this work, we propose LaMer, a novel text style transfer framework based on large-scale language models.
Revised: In this work, we propose LaMer, a novel text style transfer framework based on large-scale language models.

Original: LaMer ﬁrst mines the roughly parallel expressions in the non-parallel datasets with scene graphs, and then employs MLE training, followed by imitation learning reﬁnement, to leverage the intrinsic parallelism within the data.
Revised: LaMer ﬁrst mines the roughly parallel expressions in the non-parallel datasets with scene graphs, and then employs MLE training, followed by imitation learning reﬁnement, to leverage the intrinsic parallelism within the data.

Original: On two benchmark tasks (sentiment & formality transfer) and a newly proposed challenging task (political stance transfer), our model achieves qualitative advances in transfer accuracy, content preservation, and ﬂuency.
Revised: On two benchmark tasks (sentiment & formality transfer) and a newly proposed challenging task (political stance transfer), our model achieves qualitative advances in transfer accuracy, content preservation, and ﬂuency.

Original: Further empirical and human evaluations demonstrate that our model not only makes training more efﬁcient, but also generates more readable and diverse expressions than previous models.
Revised: Further empirical and human evaluations demonstrate that our model not only makes training more efﬁcient, but also generates more readable and diverse expressions than previous models.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: [Figure]
Revised: [Figure]

Original: tent (John et al., 2019; Fu et al., 2018).
Revised: tent (John et al., 2019; Fu et al., 2018).

Original: Existing TST methods perform well when transferring simple styles, such as sentiment; however, they tend to do a poor job on more abstract and subtle styles, such as formality and political stance (Lee et al., 2021; Fu et al., 2019b).
Revised: Existing TST methods perform well when transferring simple styles, such as sentiment; however, they tend to do a poor job on more abstract and subtle styles, such as formality and political stance (Lee et al., 2021; Fu et al., 2019b).

Original: The lack of parallel datasets is one of the main bottlenecks for text style transfer tasks.
Revised: The lack of parallel datasets is one of the main bottlenecks for text style transfer tasks.

Original: Parallel datasets contain pairwise sentences that only differ in speciﬁc styles, while non-parallel datasets typically contain a large corpora of sentences with different styles, without direct one-to-one mapping between sentences (Fu et al., 2019a;
Revised: Parallel datasets contain pairwise sentences that only differ in speciﬁc styles, while non-parallel datasets typically contain a large corpora of sentences with different styles, without direct one-to-one mapping between sentences (Fu et al., 2019a;

Original: Krishna et al., 2020).
Revised: Krishna et al., 2020).

Original: In real-world settings, it is much easier to construct non-parallel datasets by collecting large quantities of non-parallel sentences with different styles; conversely, carefully-aligned parallel TST datasets are much more costly to construct, usually requiring human annotation, and are thus scarce.
Revised: In real-world settings, it is much easier to construct non-parallel datasets by collecting large quantities of non-parallel sentences with different styles; conversely, carefully-aligned parallel TST datasets are much more costly to construct, usually requiring human annotation, and are thus scarce.

Original: This discrepancy between the availability of parallel and non-parallel datasets for style transfer tasks prompts researchers to develop TST models for non-parallel datasets.
Revised: This discrepancy between the availability of parallel and non-parallel datasets for style transfer tasks prompts researchers to develop TST models for non-parallel datasets.

Original: However, these methods are less than ideal.
Revised: However, these methods are less than ideal.

Original: As demonstrated in Figure 1 (a), they tend to learn the mapping between source and target styles on the original randomly mapped sentence pairs, but fail to take into account the inherent parallelism within the data (Dai et al., 2019; Luo et al., 2019;
Revised: As demonstrated in Figure 1 (a), they tend to learn the mapping between source and target styles on the original randomly mapped sentence pairs, but fail to take into account the inherent parallelism within the data (Dai et al., 2019; Luo et al., 2019;

Original: Shen et al., 2017).
Revised: Shen et al., 2017).

Original: Current methods either rely on surface-level classiﬁcation of tokens as style- or content-bearing (Li et al.,
Revised: Current methods either rely on surface-level classiﬁcation of tokens as style- or content-bearing (Li et al.,

Original: 2018a;
Revised: 2018a;

Original: Castro et al., 2017; Karadzhov et al., 2017), or separate content and style vectors in latent space (John et al., 2019; Liu et al., 2020; Wang et al., 2019).
Revised: Castro et al., 2017; Karadzhov et al., 2017), or separate content and style vectors in latent space (John et al., 2019; Liu et al., 2020; Wang et al., 2019).

Original: Once style and content have been separated, they then swap styles leaving the content unchanged.
Revised: Once style and content have been separated, they then swap styles leaving the content unchanged.

Original: However, surface-level split is not reliable when the style cannot be extracted at the token level; latent-space disentanglement could also fail due to a lack of pairwise supervision and results in discarding too much style-independent content.
Revised: However, surface-level split is not reliable when the style cannot be extracted at the token level; latent-space disentanglement could also fail due to a lack of pairwise supervision and results in discarding too much style-independent content.

Original: More recent work pivot to use pseudo parallels as pairwise supervision.
Revised: More recent work pivot to use pseudo parallels as pairwise supervision.

Original: For example, Jin et al.
Revised: For example, Jin et al.

Original: (2019) inspired by parallel data construction in machine translation (Hu et al., 2019; Gr´egoire &
Revised: (2019) inspired by parallel data construction in machine translation (Hu et al., 2019; Gr´egoire &

Original: Langlais, 2018), construct pseudo parallels via iterative matching and back-translation.
Revised: Langlais, 2018), construct pseudo parallels via iterative matching and back-translation.

Original: The iterative generation procedure is rather time consuming; however, training additional translation systems for
Revised: The iterative generation procedure is rather time consuming; however, training additional translation systems for

Original: TST could also be costly, which may introduce unnecessary bias towards TST models training.
Revised: TST could also be costly, which may introduce unnecessary bias towards TST models training.

Original: This paper presents LaMer, a novel self-supervised text style transfer framework built upon recent text-to-text language models (LMs).
Revised: This paper presents LaMer, a novel self-supervised text style transfer framework built upon recent text-to-text language models (LMs).

Original: As illustrated in Figure 1 (b), rather than accepting the random mapping in the given non-parallel dataset, we ﬁrst align sentences from source and target styles based on their scene graphs, and then run conditional MLE training on LM to maximize the probability of generating (roughly) parallel style sentences.
Revised: As illustrated in Figure 1 (b), rather than accepting the random mapping in the given non-parallel dataset, we ﬁrst align sentences from source and target styles based on their scene graphs, and then run conditional MLE training on LM to maximize the probability of generating (roughly) parallel style sentences.

Original: As a ﬁnal reﬁnement step, we deploy reinforced imitation learning with contrastive loss on the aligned data to further reward the best transfer demonstration and penalize the others.
Revised: As a ﬁnal reﬁnement step, we deploy reinforced imitation learning with contrastive loss on the aligned data to further reward the best transfer demonstration and penalize the others.

Original: In comparison with existing methods, LaMer fully exploits the intrinsic parallelism within non-parallel data and innovatively applies the transfer learning power of large-scale LMs to boost text-style transfer performance.
Revised: In comparison with existing methods, LaMer fully exploits the intrinsic parallelism within non-parallel data and innovatively applies the transfer learning power of large-scale LMs to boost text-style transfer performance.

Original: Our contributions can be outlined as follows:
Revised: Our contributions can be outlined as follows:

Original: • We present a language model based text style transfer framework, L A M ER , that achieves signiﬁcant advances in two existing benchmark TST tasks and one newly proposed challenging TST task.
Revised: • We present a language model based text style transfer framework, L A M ER , that achieves signiﬁcant advances in two existing benchmark TST tasks and one newly proposed challenging TST task.

Original: Human evaluations further conﬁrm that LaMer generation is not only more readable but also richer in diverse expressions than previous methods.
Revised: Human evaluations further conﬁrm that LaMer generation is not only more readable but also richer in diverse expressions than previous methods.

Original: • We propose a new challenging text style transfer task, Political Stance Transfer , and release a benchmark dataset, which includes news sentence-pairs describing the same fact but from opposite political stances (i.e., liberal and conservative ).
Revised: • We propose a new challenging text style transfer task, Political Stance Transfer , and release a benchmark dataset, which includes news sentence-pairs describing the same fact but from opposite political stances (i.e., liberal and conservative ).

Original: The stances of the sentences are labeled by expert human editors.
Revised: The stances of the sentences are labeled by expert human editors.

Original: • As part of LaMer, we introduce a simple but powerful method to create parallel corpora from non-parallel data.
Revised: • As part of LaMer, we introduce a simple but powerful method to create parallel corpora from non-parallel data.

Original: We show that this method can be used as a general treatment to improve existing TST models.
Revised: We show that this method can be used as a general treatment to improve existing TST models.

Original: Given the abundance of non-parallel corpora for different TST tasks, this method has the potential to greatly impact the ﬁeld of TST.
Revised: Given the abundance of non-parallel corpora for different TST tasks, this method has the potential to greatly impact the ﬁeld of TST.

Original: Our code and data will be released after publication with the MIT license.
Revised: Our code and data will be released after publication with the MIT license.

Original: For reviewing purposes, we have included the code and data in the supplementary.
Revised: For reviewing purposes, we have included the code and data in the supplementary.

Original:  A PPROACH
Revised:  A PPROACH

Original: LaMer is comprised of three main steps.
Revised: LaMer is comprised of three main steps.

Original: First, we mine (roughly) parallels target style expressions for each of the source style sentences based on their LM-based sentence embeddings and scene graphs ( § 2.2).
Revised: First, we mine (roughly) parallels target style expressions for each of the source style sentences based on their LM-based sentence embeddings and scene graphs ( § 2.2).

Original: Second, we deploy MLE training on the text-to-text LM BART (Lewis et al., 2020b) to maximize the probability of generating such roughly parallel sentences in target style ( § 2.3).
Revised: Second, we deploy MLE training on the text-to-text LM BART (Lewis et al., 2020b) to maximize the probability of generating such roughly parallel sentences in target style ( § 2.3).

Original: Finally, we use imitation learning to further reﬁne the generation with the self-supervision from the mined parallel demonstrations ( § 2.4).
Revised: Finally, we use imitation learning to further reﬁne the generation with the self-supervision from the mined parallel demonstrations ( § 2.4).

Original: 2. N OTATION AND P ROBLEM S TATEMENT
Revised: 2. N OTATION AND P ROBLEM S TATEMENT

Original: For the task of binary text style transfer (src ↔ tgt), We denote the source and target style datasets as
Revised: For the task of binary text style transfer (src ↔ tgt), We denote the source and target style datasets as

Original: D src and D tgt , which contain M and N sentences expressing two different styles respectively (i.e.,
Revised: D src and D tgt , which contain M and N sentences expressing two different styles respectively (i.e.,

Original: D src = { s src 1 , s src 2 , ...s src M } , D tgt
Revised: D src = { s src 1 , s src 2 , ...s src M } , D tgt

Original: = { s tgt 1 , s tgt 2 , ...
Revised: = { s tgt 1 , s tgt 2 , ...

Original: s tgt N } ).
Revised: s tgt N } ).

Original: We also denote the tokens in source and target sentences as { x 1 , x 2 , ..., x t } ∈ s src i and { y 1 , y 2 , ..., y t } ∈ s tgt j , for given source and target sentences s src i and s tgt j .
Revised: We also denote the tokens in source and target sentences as { x 1 , x 2 , ..., x t } ∈ s src i and { y 1 , y 2 , ..., y t } ∈ s tgt j , for given source and target sentences s src i and s tgt j .

Original: Given an input sentence s src of the source style, an ideal style transfer model should be able to generate an appropriate, ﬂuent target sentence ˆ s tgt , which is of the target style while maintaining the style-independent content.
Revised: Given an input sentence s src of the source style, an ideal style transfer model should be able to generate an appropriate, ﬂuent target sentence ˆ s tgt , which is of the target style while maintaining the style-independent content.

Original: Existing methods learn a mapping f random : D src → D tgt without considering the self-parallelism between D src and D tgt ; LaMer, however, works on the assumption that m ( ≤ M ) sentences in D src have n parallels in D tgt .
Revised: Existing methods learn a mapping f random : D src → D tgt without considering the self-parallelism between D src and D tgt ; LaMer, however, works on the assumption that m ( ≤ M ) sentences in D src have n parallels in D tgt .

Original: The number of parallels n for each of the m sentences is variant but no more than N .
Revised: The number of parallels n for each of the m sentences is variant but no more than N .

Original: Therefore, the goal of LaMer can be framed as ﬁrst identifying a parallel subset of the original non-parallel dataset and learning a mapping on this dataset: f parallel : D src m → D tgt n ( m ≤ M, n ≤ N ) .
Revised: Therefore, the goal of LaMer can be framed as ﬁrst identifying a parallel subset of the original non-parallel dataset and learning a mapping on this dataset: f parallel : D src m → D tgt n ( m ≤ M, n ≤ N ) .

Original: [Figure]
Revised: [Figure]

Original: In this work, we argue that the non-parallel text style transfer datasets have inherent property of parallelism .
Revised: In this work, we argue that the non-parallel text style transfer datasets have inherent property of parallelism .

Original: We outline in the following three major strategies to mine roughly parallel sentences:
Revised: We outline in the following three major strategies to mine roughly parallel sentences:

Original: Random (RD).
Revised: Random (RD).

Original: For each sentence s src ∈ D src , randomly pick k sentences from D tgt as parallels.
Revised: For each sentence s src ∈ D src , randomly pick k sentences from D tgt as parallels.

Original: This method mainly serves as an ablation to show the effectiveness of other strategies.
Revised: This method mainly serves as an ablation to show the effectiveness of other strategies.

Original: Sentence Embedding (S-Emb.).
Revised: Sentence Embedding (S-Emb.).

Original: Sentences that only differ in linguistic style should have higher semantic similarity than other unrelated sentences.
Revised: Sentences that only differ in linguistic style should have higher semantic similarity than other unrelated sentences.

Original: As a more advanced strategy, we leverage off-theshelf LMs (e.g., RoBERTa (Liu et al., 2019)) to mine semantically similar sentences.
Revised: As a more advanced strategy, we leverage off-theshelf LMs (e.g., RoBERTa (Liu et al., 2019)) to mine semantically similar sentences.

Original: We ﬁrst encode every sentence in D src and D tgt with their embeddings extracted from an LM.
Revised: We ﬁrst encode every sentence in D src and D tgt with their embeddings extracted from an LM.

Original: Then, for each sentence s src ∈ D src , we pick k most similar sentences from D tgt through cosine similarity as parallels.
Revised: Then, for each sentence s src ∈ D src , we pick k most similar sentences from D tgt through cosine similarity as parallels.

Original: S-Emb.
Revised: S-Emb.

Original: + Scene Alignment Score (S-Emb.
Revised: + Scene Alignment Score (S-Emb.

Original: + SAS).
Revised: + SAS).

Original: We add a scene graph based alignment step to the sentence embedding method to leverage shared entities between source and target sentences.
Revised: We add a scene graph based alignment step to the sentence embedding method to leverage shared entities between source and target sentences.

Original: Scene graph extraction has been successfully used in computer vision (Venugopalan et al., 2015;
Revised: Scene graph extraction has been successfully used in computer vision (Venugopalan et al., 2015;

Original: Chen et al., 2020b) to guide (Yang et al., 2020; Pan et al., 2020) or judge (Anderson et al., 2016) diverse caption generation.
Revised: Chen et al., 2020b) to guide (Yang et al., 2020; Pan et al., 2020) or judge (Anderson et al., 2016) diverse caption generation.

Original: Inspired by the CV example and the observation that scene graphs of sentences with similar content but different styles tend to have overlapping nodes, our algorithm highlights the shared entities (nodes) of the sentence scene graph by running the scene graph parser released by Wu et al.
Revised: Inspired by the CV example and the observation that scene graphs of sentences with similar content but different styles tend to have overlapping nodes, our algorithm highlights the shared entities (nodes) of the sentence scene graph by running the scene graph parser released by Wu et al.

Original: (2019) 1 , which outputs mined scenes graphs in subject-relation-object triplets.
Revised: (2019) 1 , which outputs mined scenes graphs in subject-relation-object triplets.

Original: In our case, we only consider the end nodes of these triplets (which are subj.
Revised: In our case, we only consider the end nodes of these triplets (which are subj.

Original: and obj. )
Revised: and obj. )

Original: as the scenes entities .
Revised: as the scenes entities .

Original: Figure 2 shows an example of mined sentences and corresponding scenes entities.
Revised: Figure 2 shows an example of mined sentences and corresponding scenes entities.

Original: Let | e ( s src ) | and | e ( s tgt ) | be the number of scene entities extracted from the source and target style sentence, and | e ( s src ∩ tgt ) | be the number of their overlaps, we compute Scene Alignment Score (SAS) as:
Revised: Let | e ( s src ) | and | e ( s tgt ) | be the number of scene entities extracted from the source and target style sentence, and | e ( s src ∩ tgt ) | be the number of their overlaps, we compute Scene Alignment Score (SAS) as:

Original: [Equation]
Revised: [Equation]

Original: where precision ( = | e ( s
Revised: where precision ( = | e ( s

Original: [Equation]
Revised: [Equation]

Original: tgt ) | | e ( s tgt ) | ) and recall ( = | e ( s src ∩ tgt ) | | e ( s src ) | ) describe the portion of target and source entities that are part of the overlap, and positive real factor β is chosen such that recall is considered β times as important as precision in a speciﬁc domain.
Revised: tgt ) | | e ( s tgt ) | ) and recall ( = | e ( s src ∩ tgt ) | | e ( s src ) | ) describe the portion of target and source entities that are part of the overlap, and positive real factor β is chosen such that recall is considered β times as important as precision in a speciﬁc domain.

Original: We also normalize SAS by the length of s tgt , since longer target sentence candidates are more likely to have higher SAS.
Revised: We also normalize SAS by the length of s tgt , since longer target sentence candidates are more likely to have higher SAS.

Original: SAS serves as a complementary alignment metric to further ﬁlter the parallel sentences picked by the previous sentence-embedding-based metric.
Revised: SAS serves as a complementary alignment metric to further ﬁlter the parallel sentences picked by the previous sentence-embedding-based metric.

Original: The complete procedure of S-Emb.
Revised: The complete procedure of S-Emb.

Original: + SAS alignment is:
Revised: + SAS alignment is:

Original: Encode each sentence in D src and D tgt as sentence embeddings via a pre-trained LM.
Revised: Encode each sentence in D src and D tgt as sentence embeddings via a pre-trained LM.

Original: Compute cosine similarity between all sentence pairs ( s i , s j ) where s i ∈
Revised: Compute cosine similarity between all sentence pairs ( s i , s j ) where s i ∈

Original: D src and s j ∈ D tgt .
Revised: D src and s j ∈ D tgt .

Original: Pick the k most similar sentences s ( k ) j from D tgt as parallel candidates for s i .
Revised: Pick the k most similar sentences s ( k ) j from D tgt as parallel candidates for s i .

Original: For each sentence s i ∈ D src and its parallel candidates s ( k ) j , compute pairwise SAS by
Revised: For each sentence s i ∈ D src and its parallel candidates s ( k ) j , compute pairwise SAS by

Original: Equation 1.
Revised: Equation 1.

Original: Pick those s j whose SAS scores (with respect to s i ) are above a threshold p as the ﬁnal parallel sentences to s i .
Revised: Pick those s j whose SAS scores (with respect to s i ) are above a threshold p as the ﬁnal parallel sentences to s i .

Original: From empirical observations, we found that the selection of p and k is task-speciﬁc and has crucial effects on the performance of LaMer.
Revised: From empirical observations, we found that the selection of p and k is task-speciﬁc and has crucial effects on the performance of LaMer.

Original: We explore the outcomes of different choices of k and p in § 3.3.
Revised: We explore the outcomes of different choices of k and p in § 3.3.

Original: Though SAS alignment LaMer relies on entity alignment for mining parallel corpora, other alignment mechanisms, such as (multilingual) LM alignment (i.e., the S-Emb. method) can potentially be used for transfer cases where entities may not be shared across styles (such as ancient v.s. modern
Revised: Though SAS alignment LaMer relies on entity alignment for mining parallel corpora, other alignment mechanisms, such as (multilingual) LM alignment (i.e., the S-Emb. method) can potentially be used for transfer cases where entities may not be shared across styles (such as ancient v.s. modern

Original: English (Xu et al., 2012), or even multilingual text style transfer (Krishna et al., 2021)).
Revised: English (Xu et al., 2012), or even multilingual text style transfer (Krishna et al., 2021)).

Original: 2. C ONDITIONAL T OKEN - LEVEL MLE T RAINING
Revised: 2. C ONDITIONAL T OKEN - LEVEL MLE T RAINING

Original: After mining parallel sentences for a source sentence s src , from our corpus, we take the source sentence s src as the condition, and preform a conditional token-level MLE training on the mined parallels in the manner of standard autoregressive pre-training.
Revised: After mining parallel sentences for a source sentence s src , from our corpus, we take the source sentence s src as the condition, and preform a conditional token-level MLE training on the mined parallels in the manner of standard autoregressive pre-training.

Original: Assuming for a certain source sentence s src we have n target parallels mined in the previous step (i.e., s src → { s tgt 1 , s tgt 2 , ...s tgt n } , n ≤ N ), the
Revised: Assuming for a certain source sentence s src we have n target parallels mined in the previous step (i.e., s src → { s tgt 1 , s tgt 2 , ...s tgt n } , n ≤ N ), the

Original: MLE loss can be computed as:
Revised: MLE loss can be computed as:

Original: [Equation]
Revised: [Equation]

Original: where y it ∈ s tgt i denotes the i -th step ground truth token belonging to the i -th target parallel sentence.
Revised: where y it ∈ s tgt i denotes the i -th step ground truth token belonging to the i -th target parallel sentence.

Original: LM ( y 1: t − 1 | s src ) is the LM output at t -th step (a distribution on the vocabulary) given the source text and previous generation.
Revised: LM ( y 1: t − 1 | s src ) is the LM output at t -th step (a distribution on the vocabulary) given the source text and previous generation.

Original: We choose BART Lewis et al.
Revised: We choose BART Lewis et al.

Original: (2020b) as the LM, which is already pre-trained on several text-to-text tasks, and concatenate the source and target text with 
<s
Revised: (2020b) as the LM, which is already pre-trained on several text-to-text tasks, and concatenate the source and target text with 
<s

Original: > tokens pre-deﬁned by BART as input data.
Revised: > tokens pre-deﬁned by BART as input data.

Original: At each step, with the supervision of ground truth target tokens y ( i ) t , we minimize the MLE loss only at the target part.
Revised: At each step, with the supervision of ground truth target tokens y ( i ) t , we minimize the MLE loss only at the target part.

Original: The LM is thus trained to maximize the probability of generating tokens ˆ y t approaching to y ( i ) t .
Revised: The LM is thus trained to maximize the probability of generating tokens ˆ y t approaching to y ( i ) t .

Original: Though the results produced are reasonable, solely relying on MLE training has several defects.
Revised: Though the results produced are reasonable, solely relying on MLE training has several defects.

Original: First,
Revised: First,

Original: MLE treats all parallels of a certain source equally without penalizing low-quality demonstrations, which not only makes the model less robust on noisy data, but also possible to be misled by negative samples during local optimization.
Revised: MLE treats all parallels of a certain source equally without penalizing low-quality demonstrations, which not only makes the model less robust on noisy data, but also possible to be misled by negative samples during local optimization.

Original: Second, token-level MLE does not pay special attention to scene entities shared between the source and target, which we found take up the main portion in the human written references (as demonstrated in § A.3).
Revised: Second, token-level MLE does not pay special attention to scene entities shared between the source and target, which we found take up the main portion in the human written references (as demonstrated in § A.3).

Original: We are thus motivated to further improve the performance using the self-supervision from the parallels.
Revised: We are thus motivated to further improve the performance using the self-supervision from the parallels.

Original: 2. R EINFORCED I MITATION L EARNING BY C ONTRAST
Revised: 2. R EINFORCED I MITATION L EARNING BY C ONTRAST

Original: For all the mined target style parallels s tgt of s src , we denote the one that has the highest similarity score (with S-Emb.) or has the highest SAS (with S-Emb.
Revised: For all the mined target style parallels s tgt of s src , we denote the one that has the highest similarity score (with S-Emb.) or has the highest SAS (with S-Emb.

Original: + SAS)
Revised: + SAS)

Original: as the expert demonstration s tgt+ , while the others are set as s tgt- .
Revised: as the expert demonstration s tgt+ , while the others are set as s tgt- .

Original: In the formulation of reinforcement learning, at each step t , the policy π θ observes the state y 1: t −(partially generated text), takes an action a t =
Revised: In the formulation of reinforcement learning, at each step t , the policy π θ observes the state y 1: t −(partially generated text), takes an action a t =

Original: y t (pick a token), and transits to the next state y 1: t .
Revised: y t (pick a token), and transits to the next state y 1: t .

Original: In the context of TST tasks, the policy can be interpreted as the LM-based generation model: ˆ π ∼ LM ( a t | y 1: t − 1 , s src ) , and the set of s tgt+ can be seen as the trajectory derived from an expert policy π + , while s tgt- is produced by an amateur policy π − .
Revised: In the context of TST tasks, the policy can be interpreted as the LM-based generation model: ˆ π ∼ LM ( a t | y 1: t − 1 , s src ) , and the set of s tgt+ can be seen as the trajectory derived from an expert policy π + , while s tgt- is produced by an amateur policy π − .

Original: The goal of this step imitation learning can thus be framed as learning a policy that has more probability to behave like π + rather than π − , which can be achieved by minimizing the contrastive loss:
Revised: The goal of this step imitation learning can thus be framed as learning a policy that has more probability to behave like π + rather than π − , which can be achieved by minimizing the contrastive loss:

Original: [Equation]
Revised: [Equation]

Original: where ψ ∗ ( · ) is the distance measurement of two policies in terms of semantic coherence d S EM ( · ) and scene preservation d P SV ( · ) , such that ψ ∗ ( · )
Revised: where ψ ∗ ( · ) is the distance measurement of two policies in terms of semantic coherence d S EM ( · ) and scene preservation d P SV ( · ) , such that ψ ∗ ( · )

Original: =
Revised: =

Original: d S EM ( · ) + d P SV ( · ) ; and ∆ is a hyperparameter that controls the contrasting margin.
Revised: d S EM ( · ) + d P SV ( · ) ; and ∆ is a hyperparameter that controls the contrasting margin.

Original: The general form is inspired by the triplet loss (Schroff et al., 2015) and recent in-batch constrastive learning (Gao et al., 2021): we encourage the learned policy ˆ π to be more similar to the expert one π + than the amateur one π − by at least margin ∆ .
Revised: The general form is inspired by the triplet loss (Schroff et al., 2015) and recent in-batch constrastive learning (Gao et al., 2021): we encourage the learned policy ˆ π to be more similar to the expert one π + than the amateur one π − by at least margin ∆ .

Original: d S EM ( · ) and d P SV ( · ) are detailed below.
Revised: d S EM ( · ) and d P SV ( · ) are detailed below.

Original: Sequence-level Semantic Coherence For sequence
Revised: Sequence-level Semantic Coherence For sequence

Original: ˆ s generated under current learning policy ˆ π and a certain target style demonstration s tgt , we encode them using bi-directional LM RoBERTa
Revised: ˆ s generated under current learning policy ˆ π and a certain target style demonstration s tgt , we encode them using bi-directional LM RoBERTa

Original: (Liu et al., 2019), which is widely used as sentence encoder in related work (Reimers & Gurevych, 2019;
Revised: (Liu et al., 2019), which is widely used as sentence encoder in related work (Reimers & Gurevych, 2019;

Original: Li et al., 2020).
Revised: Li et al., 2020).

Original: Then, we take their negative cosine similarity as the distance for semantic coherence:
Revised: Then, we take their negative cosine similarity as the distance for semantic coherence:

Original: [Equation]
Revised: [Equation]

Original: Lower value in d S EM means the learned policy ˆ π can produce similar quality sequences in terms of semantics as target style demonstrations.
Revised: Lower value in d S EM means the learned policy ˆ π can produce similar quality sequences in terms of semantics as target style demonstrations.

Original: It is a sequence-level measurement since the encoding is based on the whole sentence.
Revised: It is a sequence-level measurement since the encoding is based on the whole sentence.

Original: Token-level Scene Preservation
Revised: Token-level Scene Preservation

Original: In our context, legitimate target style parallels should preserve most scene entities that appear in source style sentences.
Revised: In our context, legitimate target style parallels should preserve most scene entities that appear in source style sentences.

Original: To explicitly promote the content preservation performance of the current policy, we introduce a token-level distance d P SV .
Revised: To explicitly promote the content preservation performance of the current policy, we introduce a token-level distance d P SV .

Original: We ﬁrst extract scene entities from s tgt + and each of the s tgt − , and then measure the differences in both the order of appearance of the entities and the existence of the entities.
Revised: We ﬁrst extract scene entities from s tgt + and each of the s tgt − , and then measure the differences in both the order of appearance of the entities and the existence of the entities.

Original: Taking s tgt+ as an example (similar calculation is done on each of the s tgt- ), we compute: d P SV + = αd Order + + (1 − α )
Revised: Taking s tgt+ as an example (similar calculation is done on each of the s tgt- ), we compute: d P SV + = αd Order + + (1 − α )

Original: d Exist + = α (cid:107) (cid:126) e
Revised: d Exist + = α (cid:107) (cid:126) e

Original: ( s tgt + ) ,(cid:126) e ( s src ) (cid:107) Lev.
Revised: ( s tgt + ) ,(cid:126) e ( s src ) (cid:107) Lev.

Original: + (1 − α ) (cid:107) (cid:126) e
Revised: + (1 − α ) (cid:107) (cid:126) e

Original: ( s tgt+ ) − (cid:126) e ( s tgt+ ∪ src ) (cid:107) H 

Revised: ( s tgt+ ) − (cid:126) e ( s tgt+ ∪ src ) (cid:107) H

Original: where (cid:126) e ( · ) means the extracted scene entities, (cid:126) e ( · ) is its one-hot encoded vector, (cid:107) (cid:126)
Revised: where (cid:126) e ( · ) means the extracted scene entities, (cid:126) e ( · ) is its one-hot encoded vector, (cid:107) (cid:126)

Original: x , (cid:126) y (cid:107) Lev.
Revised: x , (cid:126) y (cid:107) Lev.

Original: measures Levenshtein distance (Yujian &
Revised: measures Levenshtein distance (Yujian &

Original: Bo, 2007), one type of edit distance, and for vectors (cid:126) e ( s src ) and (cid:126) e ( s tgt ) , we compute their hamming distance 3 (Norouzi et al., 2012).
Revised: Bo, 2007), one type of edit distance, and for vectors (cid:126) e ( s src ) and (cid:126) e ( s tgt ) , we compute their hamming distance 3 (Norouzi et al., 2012).

Original: Lower d Order + means the appearing order of (cid:126) e ( s src ) resemble (cid:126) e ( s tgt+ ) .
Revised: Lower d Order + means the appearing order of (cid:126) e ( s src ) resemble (cid:126) e ( s tgt+ ) .

Original: Similarly, lower d Exist + means (cid:126) e ( s src ) and (cid:126) e ( s tgt+ ) are mostly overlapped.
Revised: Similarly, lower d Exist + means (cid:126) e ( s src ) and (cid:126) e ( s tgt+ ) are mostly overlapped.

Original: Similar procedure applies to d P SV − as well.
Revised: Similar procedure applies to d P SV − as well.

Original: We present more details about how we compute these distances in § A.4.
Revised: We present more details about how we compute these distances in § A.4.

Original: 2.4. P OLICY G RADIENT BY REINFORCE
Revised: 2.4. P OLICY G RADIENT BY REINFORCE

Original: We choose the REINFORCE algorithm (Williams, 1992) to optimize the current policy π θ .
Revised: We choose the REINFORCE algorithm (Williams, 1992) to optimize the current policy π θ .

Original: As shown in Equation 4, we use greedy decoding results as the baseline (Rennie et al., 2017), and sampling decoding for the exploration step, which mimics the actor-critic manner in reinforcement learning (Fujimoto et al., 2018; Haarnoja et al., 2018).
Revised: As shown in Equation 4, we use greedy decoding results as the baseline (Rennie et al., 2017), and sampling decoding for the exploration step, which mimics the actor-critic manner in reinforcement learning (Fujimoto et al., 2018; Haarnoja et al., 2018).

Original: The policy gradient can thus be presented as:
Revised: The policy gradient can thus be presented as:

Original: [Equation]
Revised: [Equation]

Original: (4) where J sampleIL and J greedyIL can be both calculated by Equation 3.
Revised: (4) where J sampleIL and J greedyIL can be both calculated by Equation 3.

Original: We take the difference of two the mode rewards as the advantage, feeding into the REINFORCE algorithm to optimize our policy π θ .
Revised: We take the difference of two the mode rewards as the advantage, feeding into the REINFORCE algorithm to optimize our policy π θ .

Original: Finally, the output policy π θ is able to generate style transferred sequences resembling the expert demonstrations s tgt+ .
Revised: Finally, the output policy π θ is able to generate style transferred sequences resembling the expert demonstrations s tgt+ .

Original: More details about the complete loop of policy gradient can be found in § A.
Revised: More details about the complete loop of policy gradient can be found in § A.

Original: Discussion: Imitation Learning, Reward-based RL, and MLE.
Revised: Discussion: Imitation Learning, Reward-based RL, and MLE.

Original: Framing token-by-token text generation as a sequential decision-making process, many models choose reinforcement learning (RL) as the training strategy because of its superior performance in discrete action space optimization (Vinyals et al., 2019; Silver et al., 2017).
Revised: Framing token-by-token text generation as a sequential decision-making process, many models choose reinforcement learning (RL) as the training strategy because of its superior performance in discrete action space optimization (Vinyals et al., 2019; Silver et al., 2017).

Original: By directly setting indifferentiable evaluation metrics such as BLEU (Papineni et al., 2002) or ROUGE (Lin, 2004) as the reward function (Xu et al., 2018;
Revised: By directly setting indifferentiable evaluation metrics such as BLEU (Papineni et al., 2002) or ROUGE (Lin, 2004) as the reward function (Xu et al., 2018;

Original: Gong et al., 2019) or using domain-speciﬁc reward such as trained discriminator outputs (Lin et al., 2020; Scialom et al., 2020), existing RL-based methods show promising results in both generation quality (Chen et al., 2020a; Li et al., 2018b) and diversity (Li et al., 2016).
Revised: Gong et al., 2019) or using domain-speciﬁc reward such as trained discriminator outputs (Lin et al., 2020; Scialom et al., 2020), existing RL-based methods show promising results in both generation quality (Chen et al., 2020a; Li et al., 2018b) and diversity (Li et al., 2016).

Original: In the context of TST tasks, however, instead of reward-based RL, we argue imitation learning (IL) is a better choice, since: 1)
Revised: In the context of TST tasks, however, instead of reward-based RL, we argue imitation learning (IL) is a better choice, since: 1)

Original: Compared with ﬁxed environment settings like chess games, it is challenging to design a uniform reward ﬁtting a variety of source style text that can produce meaningful guidance towards contextually correct and semantically coherent direction during generation.
Revised: Compared with ﬁxed environment settings like chess games, it is challenging to design a uniform reward ﬁtting a variety of source style text that can produce meaningful guidance towards contextually correct and semantically coherent direction during generation.

Original: It is also unnecessary to calculate an elaborately designed reward.
Revised: It is also unnecessary to calculate an elaborately designed reward.

Original: The roughly parallel target style sentences mined in the alignment step can naturally serve as expert demonstrations.
Revised: The roughly parallel target style sentences mined in the alignment step can naturally serve as expert demonstrations.

Original: They are high-quality and cost-free, which can greatly improve the sampling and learning efﬁciency of RL optimization.
Revised: They are high-quality and cost-free, which can greatly improve the sampling and learning efﬁciency of RL optimization.

Original: A common obstacle of IL is known as covariate shift (Ross & Bagnell, 2010), which means IL can provide a relatively good performance for the test cases similar to the demonstrations (used for policy training), but it could still suffer from bad generalization for cases it never meets during training, as the number of demonstrations is ﬁnite.
Revised: A common obstacle of IL is known as covariate shift (Ross & Bagnell, 2010), which means IL can provide a relatively good performance for the test cases similar to the demonstrations (used for policy training), but it could still suffer from bad generalization for cases it never meets during training, as the number of demonstrations is ﬁnite.

Original: Similar problems called exposure bias (Bengio et al., 2015;
Revised: Similar problems called exposure bias (Bengio et al., 2015;

Original: Ranzato et al., 2016) also exists in MLE when the supervised data is scarce.
Revised: Ranzato et al., 2016) also exists in MLE when the supervised data is scarce.

Original: We claim our token-level
Revised: We claim our token-level

Original: MLE training on LMs can somewhat mitigate such problems, since the large-scale LMs are already pre-trained on huge quantities of data across many domains as a prior .
Revised: MLE training on LMs can somewhat mitigate such problems, since the large-scale LMs are already pre-trained on huge quantities of data across many domains as a prior .

Original:  E XPERIMENTS 3. D ATASETS AND E XPERIMENTAL S ETUP
Revised:  E XPERIMENTS 3. D ATASETS AND E XPERIMENTAL S ETUP

Original: Sentiment Transfer.
Revised: Sentiment Transfer.

Original: We use the Yelp reviews dataset collected by Shen et al.
Revised: We use the Yelp reviews dataset collected by Shen et al.

Original: (2017) which contains 250k negative sentences and 380k positive sentences, organized in non-parallel fashion.
Revised: (2017) which contains 250k negative sentences and 380k positive sentences, organized in non-parallel fashion.

Original: Li et al.
Revised: Li et al.

Original: (2018a) released a small test set that has 500 human written references for both target sentiments, and they are already one-on-one aligned with the source sentiment sentences.
Revised: (2018a) released a small test set that has 500 human written references for both target sentiments, and they are already one-on-one aligned with the source sentiment sentences.

Original: Formality Transfer.
Revised: Formality Transfer.

Original: A more challenging TST task is to modify the formality of a given sentence.
Revised: A more challenging TST task is to modify the formality of a given sentence.

Original: We use the GYAFC dataset (Rao & Tetreault, 2018), which contains formal and informal sentences from two domains.
Revised: We use the GYAFC dataset (Rao & Tetreault, 2018), which contains formal and informal sentences from two domains.

Original: In this paper, we choose the Family Relationship domain, which consists of about 52k training sentences, 5k development sentences, and 2.5k test sentences.
Revised: In this paper, we choose the Family Relationship domain, which consists of about 52k training sentences, 5k development sentences, and 2.5k test sentences.

Original: Different from the Yelp dataset, the test set of GYAFC has four human written references for each source style test case.
Revised: Different from the Yelp dataset, the test set of GYAFC has four human written references for each source style test case.

Original: Our evaluation results for this task are reported as the average against the four references.
Revised: Our evaluation results for this task are reported as the average against the four references.

Original: Political Stance Transfer.
Revised: Political Stance Transfer.

Original: News outlets reporting the same event naturally meet the requirement of style transfer data, which is aligned in content but different in style.
Revised: News outlets reporting the same event naturally meet the requirement of style transfer data, which is aligned in content but different in style.

Original: We leverage Allsides.com 4 as an aggregator, where outlets reporting the same events with different political stances are already organized and labeled by expert human editors.
Revised: We leverage Allsides.com 4 as an aggregator, where outlets reporting the same events with different political stances are already organized and labeled by expert human editors.

Original: In total, we collect 2,298 full-length news articles from 6/1/2012 to 4/1/2021 in pairs of liberal-conservative leaning.
Revised: In total, we collect 2,298 full-length news articles from 6/1/2012 to 4/1/2021 in pairs of liberal-conservative leaning.

Original: Though the data seems parallel at the article level, in this work we focus on sentence level TST.
Revised: Though the data seems parallel at the article level, in this work we focus on sentence level TST.

Original: We further split the articles into about 56k sentence pairs as a non-parallel dataset for training (detailed in § A.2).
Revised: We further split the articles into about 56k sentence pairs as a non-parallel dataset for training (detailed in § A.2).

Original: These titles are written by expert writers from liberal- or conservative-learning media outlets, and aligned by the
Revised: These titles are written by expert writers from liberal- or conservative-learning media outlets, and aligned by the

Original: Allsides editors.
Revised: Allsides editors.

Original: For ethical consideration, we manually examined the articles to removing those containing sensitive topics (such as hate speech).
Revised: For ethical consideration, we manually examined the articles to removing those containing sensitive topics (such as hate speech).

Original: To adhere to the outlets’ TOS, we will release our dataset as links to the articles (instead of the text), with the appropriate code to extract the content for training purposes.
Revised: To adhere to the outlets’ TOS, we will release our dataset as links to the articles (instead of the text), with the appropriate code to extract the content for training purposes.

Original: A closer comparison of our proposed novel dataset with another political stance
Revised: A closer comparison of our proposed novel dataset with another political stance

Original: TST dataset (Voigt et al., 2018) can be found in § A.1.
Revised: TST dataset (Voigt et al., 2018) can be found in § A.1.

Original: The detailed statistics of all three benchmark datasets and our experimental setup can be found in § A.3.
Revised: The detailed statistics of all three benchmark datasets and our experimental setup can be found in § A.3.

Original: We have discussed the hyperparameter selection for the best performing LaMer in § 3.3.
Revised: We have discussed the hyperparameter selection for the best performing LaMer in § 3.3.

Original: 3. M AIN R ESULTS ON T EXT S TYLE T RANSFER P ERFORMANCE
Revised: 3. M AIN R ESULTS ON T EXT S TYLE T RANSFER P ERFORMANCE

Original: TST models should be able to transfer source style sentence to a target style, while maintaining the style-independent content as much as possible.
Revised: TST models should be able to transfer source style sentence to a target style, while maintaining the style-independent content as much as possible.

Original: Most existing works evaluate their methods with transfer accuracy (ACC; classiﬁcation accuracy) and content-preservation (BLEU- ref ; token overlaps with reference).
Revised: Most existing works evaluate their methods with transfer accuracy (ACC; classiﬁcation accuracy) and content-preservation (BLEU- ref ; token overlaps with reference).

Original: In this work, besides using ACC and BLEU- ref , we also report more advanced metrics–SIM and FL introduced by Krishna et al.
Revised: In this work, besides using ACC and BLEU- ref , we also report more advanced metrics–SIM and FL introduced by Krishna et al.

Original: (2020)–to further evaluate our method.
Revised: (2020)–to further evaluate our method.

Original: LaMer has competitive performance.
Revised: LaMer has competitive performance.

Original: Table 1 shows the main results comparing LaMer with eight popular TST models on the three benchmark datasets.
Revised: Table 1 shows the main results comparing LaMer with eight popular TST models on the three benchmark datasets.

Original: Following prior art, in addition to the independent ACC and BLEU score 5 , we  report their geometric mean (GM)  as an overall metric of style transfer performance (based on prior work (Lee et al., 2021; Li et al., 2019; Xu et al., 2018)).
Revised: Following prior art, in addition to the independent ACC and BLEU score 5 , we also report their geometric mean (GM) for direct comparison with the DualRL baseline (Luo et al., 2019); however, it should be noted that GM has not been well-studied as an evaluation metric for TST and is only included for comparison purposes.

Original: We also consider inverse PINC scores (Chen & Dolan, 2011)— i -PINC—as a complementary metric of BLEU.
Revised: We also consider inverse PINC scores (Chen & Dolan, 2011)— i -PINC—as a complementary metric of BLEU.

Original: i -PINC counts the n-gram overlapsafter removing the tokens shared between source and target style
Revised: i -PINC counts the n-gram overlaps

Original: [Table]
Revised: [Table]

Original: Compared with existing methods
Revised: Compared with existing methods

Original: [Table]
Revised: [Table]

Original: LaMer performs better in terms of overall metric GM
Revised: LaMer performs better in terms of overall metric GM

Original: [Table]
Revised: [Table]

Original: and shows particularly strong net transfer ability (since i -PINC does not count duplicated tokens between s src and s tgt ).
Revised: and shows particularly strong net transfer ability (since i -PINC does not count duplicated tokens between s src and s tgt ).

Original: Among the three mining strategies we proposed in § 2.
Revised: Among the three mining strategies we proposed in § 2.

Original: [Table]
Revised: [Table]

Original: the S-Emb.
Revised: the S-Emb.

Original: + SAS seems to offer the best performance boosting in terms of content preservation
Revised: + SAS seems to offer the best performance boosting in terms of content preservation

Original: [Table]
Revised: [Table]

Original: but shows no clear advantage on style control accuracy.
Revised: but shows no clear advantage on style control accuracy.

Original: When comparing LaMer with the three LM-based models, we ﬁnd the advantage of LaMer mainly comes from the gain in content preservation (BLEU), which demonstrates the effectiveness of our scene graph alignment.
Revised: When comparing LaMer with the three LM-based models, we ﬁnd the advantage of LaMer mainly comes from the gain in content preservation (BLEU), which demonstrates the effectiveness of our scene graph alignment.

Original: We also ﬁnd some of the other TST models produce low-ﬂuency (i.e., high perplexity) transferred sentences (see Table 12 in § A.9), though we must note that recent studies have shown perplexity to be a poor measure of ﬂuency (Krishna et al., 2020; Mir et al., 2019; Lee et al., 2021).
Revised: We also ﬁnd some of the other TST models produce low-ﬂuency (i.e., high perplexity) transferred sentences (see Table 12 in § A.9), though we must note that recent studies have shown perplexity to be a poor measure of ﬂuency (Krishna et al., 2020; Mir et al., 2019; Lee et al., 2021).

Original: [Table] [Table]
Revised: [Table] [Table]

Original: LaMer is robust in few-shot settings.
Revised: LaMer is robust in few-shot settings.

Original: LaMer learns the inherent parallelism within the unsupervised data, which could be challenging in data scarce cases.
Revised: LaMer learns the inherent parallelism within the unsupervised data, which could be challenging in data scarce cases.

Original: Recent works have explored the limits of TST in few-shot settings (Riley et al., 2021; Krishna et al., 2021), we conduct similar studies by limiting the available training data (to 1%), aiming to answer whether or not LaMer can still provide comparable performance as others.
Revised: Recent works have explored the limits of TST in few-shot settings (Riley et al., 2021; Krishna et al., 2021), we conduct similar studies by limiting the available training data (to 1%), aiming to answer whether or not LaMer can still provide comparable performance as others.

Original: We consider LM-based baselines which show strong results in Table 1.
Revised: We consider LM-based baselines which show strong results in Table 1.

Original: We also compare with a zero-shot prompt-based TST baseline built upon GPT-3 (Reif et al., 2021) for reference.
Revised: We also compare with a zero-shot prompt-based TST baseline built upon GPT-3 (Reif et al., 2021) for reference.

Original: For evaluation metrics, we use the recently proposed J -score framework
Revised: For evaluation metrics, we use the recently proposed J -score framework

Original: [Figure]
Revised: [Figure]

Original: We also annotate (with a star) the recommended settings which reaches best balance between style control accuracy and content preservation BLEU
Revised: We also annotate (with a star) the recommended settings which reaches best balance between style control accuracy and content preservation BLEU

Original: (including ACCuracy, SIMilarity, and FLuency) (Krishna et al., 2020), which solves many of the issues in existing metrics 6 .
Revised: (including ACCuracy, SIMilarity, and FLuency) (Krishna et al., 2020), which solves many of the issues in existing metrics 6 .

Original: As shown in Table 2, we ﬁnd that LaMer achieves great performance ( J -score) in few-shot cases, partially because other baselines rely on external systems trained on ample data; 7 In extreme data hungry cases, the compromised external modules would propagate errors to following parts as they are coupled.
Revised: As shown in Table 2, we ﬁnd that LaMer achieves great performance ( J -score) in few-shot cases, partially because other baselines rely on external systems trained on ample data; 7 In extreme data hungry cases, the compromised external modules would propagate errors to following parts as they are coupled.

Original: LaMer, instead, requires neither extra data nor additional training on external systems (See Table 3), but fully leverages the self-supervision hidden in the unsupervised data, which achieves robust performance even in few-shot scenarios.
Revised: LaMer, instead, requires neither extra data nor additional training on external systems (See Table 3), but fully leverages the self-supervision hidden in the unsupervised data, which achieves robust performance even in few-shot scenarios.

Original: IL reﬁnement is crucial for LaMer content preservation.
Revised: IL reﬁnement is crucial for LaMer content preservation.

Original: As shown in Table 4, the BLEU score for LaMer drops substantially when the imitation learning reﬁnement stage is totally removed (No
Revised: As shown in Table 4, the BLEU score for LaMer drops substantially when the imitation learning reﬁnement stage is totally removed (No

Original: IL, MLE only), and we ﬁnd this drop is higher for challenging style tasks (such as political stance transfer), partially because the quality variance of their demonstrations is higher, i.e., a further guidance towards good demonstrations will make a big difference.
Revised: IL, MLE only), and we ﬁnd this drop is higher for challenging style tasks (such as political stance transfer), partially because the quality variance of their demonstrations is higher, i.e., a further guidance towards good demonstrations will make a big difference.

Original: We also conducted further ablation studies on components of IL:
Revised: We also conducted further ablation studies on components of IL:

Original: In general, removing d Order seems to cause greater performance drop in formality TST, while d Exist seems important to successful political stance TST.
Revised: In general, removing d Order seems to cause greater performance drop in formality TST, while d Exist seems important to successful political stance TST.

Original: Removing all scene preservation distance sub-components (i.e., IL ( d S EM )) leads to noticeable performance deterioration across all three TST tasks.
Revised: Removing all scene preservation distance sub-components (i.e., IL ( d S EM )) leads to noticeable performance deterioration across all three TST tasks.

Original: We take these ablation study results as evidence that the technologies we propose for LaMer are all crucial.
Revised: We take these ablation study results as evidence that the technologies we propose for LaMer are all crucial.

Original: [Table]
Revised: [Table]

Original: 3. C ONFIGURING L A M ER
Revised: 3. C ONFIGURING L A M ER

Original: The ﬁltering parameter p and k are hyperparameters that are crucial for the construction of roughly parallel datasets.
Revised: The ﬁltering parameter p and k are hyperparameters that are crucial for the construction of roughly parallel datasets.

Original: Thus, for all three TST tasks, we evaluate LaMer performance given different combination of p and k , as shown in Figure 3.
Revised: Thus, for all three TST tasks, we evaluate LaMer performance given different combination of p and k , as shown in Figure 3.

Original: We ﬁnd there is a approximate trade-off relationship between style control accuracy and content preservation BLEU score:
Revised: We ﬁnd there is a approximate trade-off relationship between style control accuracy and content preservation BLEU score:

Original: In general, higher k (more parallels) seems to improve accuracy but normally has lower BLEU, while for a ﬁxed k , a proper p can be reached empirically.
Revised: In general, higher k (more parallels) seems to improve accuracy but normally has lower BLEU, while for a ﬁxed k , a proper p can be reached empirically.

Original: We annotate the recommended settings of p and k for the three TST tasks we studied in this work.
Revised: We annotate the recommended settings of p and k for the three TST tasks we studied in this work.

Original: We provide some generation samples of LaMer (and other methods) in
Revised: We provide some generation samples of LaMer (and other methods) in

Original: Table 8 of § A.6.
Revised: Table 8 of § A.6.

Original: As an interesting observation, we ﬁnd the alignment procedure of LaMer can beneﬁt other methods (as shown in Table 11 of § A.8).
Revised: As an interesting observation, we ﬁnd the alignment procedure of LaMer can beneﬁt other methods (as shown in Table 11 of § A.8).

Original: 3. H UMAN E VALUATION
Revised: 3. H UMAN E VALUATION

Original: We conducted human evaluation to answer: 1) “How good is LaMer’s SAS alignment?” , and 2) “How is the quality of the style-transferred sentences?”
Revised: We conducted human evaluation to answer: 1) “How good is LaMer’s SAS alignment?” , and 2) “How is the quality of the style-transferred sentences?”

Original: (The exact questions and procedures are details in § A.7.).
Revised: (The exact questions and procedures are details in § A.7.).

Original: We recruited 212 participants from MTurk.
Revised: We recruited 212 participants from MTurk.

Original: Most annotators agreed that SAS-aligned pairs share similar style-independent content (Table 5).
Revised: Most annotators agreed that SAS-aligned pairs share similar style-independent content (Table 5).

Original: We conducted several one-way ANOVAs to test the difference among the seven baselines and LaMer (political stance TST results shown in Table 6; see Table 9 for sentiment TST and Table 10 for formality TST in § A.7).
Revised: We conducted several one-way ANOVAs to test the difference among the seven baselines and LaMer (political stance TST results shown in Table 6; see Table 9 for sentiment TST and Table 10 for formality TST in § A.7).

Original: LaMer was signiﬁcantly better than next-best in all three perspectives ( p < .05), and had the most ratings
Revised: LaMer was signiﬁcantly better than next-best in all three perspectives ( p < .05), and had the most ratings

Original: that where ≥ 5.
Revised: that where ≥ 5.

Original: Similarity: “How much the mined pairs are similar in styleindependent content?”
Revised: Similarity: “How much the mined pairs are similar in styleindependent content?”

Original: SAS Align: “How well SAS scores reﬂect such similarity?”
Revised: SAS Align: “How well SAS scores reﬂect such similarity?”

Original: Ratings are on a 7 point scale.
Revised: Ratings are on a 7 point scale.

Original: All ≥ 4 : The ratio for which the ratings are ≥ 4 for both questions
Revised: All ≥ 4 : The ratio for which the ratings are ≥ 4 for both questions

Original: [Table]
Revised: [Table]

Original: (on a 7 point scale)
Revised: (on a 7 point scale)

Original: We bold the highest average rating.
Revised: We bold the highest average rating.

Original: LaMer was rated signiﬁcantly higher than the next-best method ( p < .05).
Revised: LaMer was rated signiﬁcantly higher than the next-best method ( p < .05).

Original: All ≥
Revised: All ≥

Original: The ratio for which the ratings are ≥ 5 for all
Revised: The ratio for which the ratings are ≥ 5 for all

Original: [Table]
Revised: [Table]

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Other Text Style Transfer Methods.
Revised: Other Text Style Transfer Methods.

Original: Besides the baselines we have studies in the paper
Revised: Besides the baselines we have studies in the paper

Original: [Table]
Revised: [Table]

Original: we notice there are other TST methods focusing on either latent vector manipulation (Liu et al., 2020; Wang et al., 2019; John et al., 2019) or surface level editing (Castro et al., 2017; Karadzhov et al., 2017;
Revised: we notice there are other TST methods focusing on either latent vector manipulation (Liu et al., 2020; Wang et al., 2019; John et al., 2019) or surface level editing (Castro et al., 2017; Karadzhov et al., 2017;

Original: Mansoorizadeh et al.).
Revised: Mansoorizadeh et al.).

Original: LaMer differs from them as it is learning to transfer with self-supervision mined from the unsupervised data, without complicated manipulation or editing.
Revised: LaMer differs from them as it is learning to transfer with self-supervision mined from the unsupervised data, without complicated manipulation or editing.

Original: NLG by Searching for References.
Revised: NLG by Searching for References.

Original: To provide expert demonstrations to enhance LM generation, recent work attempt to retrieve related documents from external resources (e
Revised: To provide expert demonstrations to enhance LM generation, recent work attempt to retrieve related documents from external resources (e

Original: [Table]
Revised: [Table]

Original: g., Wikipedia) as training data augmentations (Lewis et al., 2020c; Guu et al., 2020; Lee et al., 2019; Lewis et al., 2020a).
Revised: g., Wikipedia) as training data augmentations (Lewis et al., 2020c; Guu et al., 2020; Lee et al., 2019; Lewis et al., 2020a).

Original: LaMer does not rely on any external resources
Revised: LaMer does not rely on any external resources

Original: [Table]
Revised: [Table]

Original: but mines parallel demonstrations from the training data itself
Revised: but mines parallel demonstrations from the training data itself

Original: [Table]
Revised: [Table]

Original: and thus is easier to deploy and alleviates potential bias.
Revised: and thus is easier to deploy and alleviates potential bias.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We have proposed LaMer, a self-supervised text style transfer method built on large-scale LMs.
Revised: We have proposed LaMer, a self-supervised text style transfer method built on large-scale LMs.

Original: LaMer demonstrates the potential of leveraging self-parallelism within large non-parallel corpora, which can guide efﬁcient TST models development or even LM design.
Revised: LaMer demonstrates the potential of leveraging self-parallelism within large non-parallel corpora, which can guide efﬁcient TST models development or even LM design.

Original: On three diverse TST tasks, we show that LaMer performs better than eight existing methods while being robust to data scarcity.
Revised: On three diverse TST tasks, we show that LaMer performs better than eight existing methods while being robust to data scarcity.

Original: Human evaluations further validate the advantage of our model compared to existing baselines.
Revised: Human evaluations further validate the advantage of our model compared to existing baselines.

Original: Future work of LaMer could study how to extend to multilingual scenarios, or transfer the style that essentially changes the form of language (e.g., ancient v.s. modern English).
Revised: Future work of LaMer could study how to extend to multilingual scenarios, or transfer the style that essentially changes the form of language (e.g., ancient v.s. modern English).

Original: One could also investigate combining the alignment module of LaMer with the text2text LM in one model, and jointly train to further boost the performance.
Revised: One could also investigate combining the alignment module of LaMer with the text2text LM in one model, and jointly train to further boost the performance.

Original:  E THICS AND R EPRODUCIBILITY S TATEMENT
Revised:  E THICS AND R EPRODUCIBILITY S TATEMENT

Original: The goal of LaMer is to provide a simple but efﬁcient general-purpose text style transfer framework by leveraging large-scale pre-trained LMs.
Revised: The goal of LaMer is to provide a simple but efﬁcient general-purpose text style transfer framework by leveraging large-scale pre-trained LMs.

Original: LaMer can generate text in a speciﬁed target style given source style input, which can serve as a useful tool for social good.
Revised: LaMer can generate text in a speciﬁed target style given source style input, which can serve as a useful tool for social good.

Original: For example, LaMer can assist language formalization (formality transfer), enable data anonymization (writing style transfer), or depolarize politically biased text (political stance transfer), via transferring the identiﬁable style of the input text.
Revised: For example, LaMer can assist language formalization (formality transfer), enable data anonymization (writing style transfer), or depolarize politically biased text (political stance transfer), via transferring the identiﬁable style of the input text.

Original: The performance of LaMer depends on the pre-trained LM, which means that the generation of
Revised: The performance of LaMer depends on the pre-trained LM, which means that the generation of

Original: LaMer can be affected by certain societal biases found in the pre-trained LM it is based on (e.g.,
Revised: LaMer can be affected by certain societal biases found in the pre-trained LM it is based on (e.g.,

Original: BART) (Blodgett et al., 2020).
Revised: BART) (Blodgett et al., 2020).

Original: Other negatives such as abusive language (Hancock et al., 2020), hate speech (Kennedy et al., 2018), and dehumanization (Mendelsohn et al., 2020) are also prevalent in the language data for LM pre-training.
Revised: Other negatives such as abusive language (Hancock et al., 2020), hate speech (Kennedy et al., 2018), and dehumanization (Mendelsohn et al., 2020) are also prevalent in the language data for LM pre-training.

Original: When deployed in real-world settings, LaMer may increase the risk of reinforcing and propagating polarizing and toxic language in public space through people’s dissemination (Waseem et al., 2017).
Revised: When deployed in real-world settings, LaMer may increase the risk of reinforcing and propagating polarizing and toxic language in public space through people’s dissemination (Waseem et al., 2017).

Original: Moreover, the generated text of LaMer could potentially be included in the next iteration of large-scale LM training data collection, which may amplify offensive stereotypes and overtly abusive views in the future LMs.
Revised: Moreover, the generated text of LaMer could potentially be included in the next iteration of large-scale LM training data collection, which may amplify offensive stereotypes and overtly abusive views in the future LMs.

Original: Though LaMer was developed as an “neutral” tool mainly for research purposes, it can also be used in malicious ways such as generating sophisticated spam in a particular style “disguise”, which we strongly discourage.
Revised: Though LaMer was developed as an “neutral” tool mainly for research purposes, it can also be used in malicious ways such as generating sophisticated spam in a particular style “disguise”, which we strongly discourage.

Original: Due to the lack of fact checking systems as a safety measure, LaMer is at the risk of generating seemingly coherent but counterfactual text in a certain target style, which could further undermine social trust in computer-mediated forms of communication [6].
Revised: Due to the lack of fact checking systems as a safety measure, LaMer is at the risk of generating seemingly coherent but counterfactual text in a certain target style, which could further undermine social trust in computer-mediated forms of communication [6].

Original: We thus recommend the users of LaMer to integrate necessary safety checks into the actual systems.
Revised: We thus recommend the users of LaMer to integrate necessary safety checks into the actual systems.

Original: In addition, our experiments and analysis are done when source and target style sentences are both in
Revised: In addition, our experiments and analysis are done when source and target style sentences are both in

Original: English, and therefore we do not claim that our ﬁndings will generalize across all languages, although our framework has the potential to be extended to other languages with necessary modiﬁcations.
Revised: English, and therefore we do not claim that our ﬁndings will generalize across all languages, although our framework has the potential to be extended to other languages with necessary modiﬁcations.

Original: Though it’s beyond the scope of this paper and our individual ability to address all above-mentioned complications, we strongly encourage users of our system (and other TST models and systems based on pre-trained language models) to acknowledge and beware of these issues.
Revised: Though it’s beyond the scope of this paper and our individual ability to address all above-mentioned complications, we strongly encourage users of our system (and other TST models and systems based on pre-trained language models) to acknowledge and beware of these issues.

Original: For reproducibility, we attach our code, evaluation scripts, and sample generation with our submission to aid reviewing.
Revised: For reproducibility, we attach our code, evaluation scripts, and sample generation with our submission to aid reviewing.

Original: The full political stance transfer dataset we proposed will be made publicly available after review process
Revised: The full political stance transfer dataset we proposed will be made publicly available after review process

Original: A A PPENDIX A. C OMPARING WITH A NOTHER P OLITICAL S TANCE D ATASET
Revised: A A PPENDIX A. C OMPARING WITH A NOTHER P OLITICAL S TANCE D ATASET

Original: Transferring political stance has also been studied in Voigt et al.
Revised: Transferring political stance has also been studied in Voigt et al.

Original: (2018), where the author also collected a dataset for political stance TST.
Revised: (2018), where the author also collected a dataset for political stance TST.

Original: Comparing with their dataset, we found ours has following differences:
Revised: Comparing with their dataset, we found ours has following differences:

Original: Language Formality : Their dataset is composed of comments on Facebook posts from members of the United States Senate and House, while our dataset consists of sentences from news articles written by professional reporters belonging to well-known media outlets (CNN, Fox News, etc.).
Revised: Language Formality : Their dataset is composed of comments on Facebook posts from members of the United States Senate and House, while our dataset consists of sentences from news articles written by professional reporters belonging to well-known media outlets (CNN, Fox News, etc.).

Original: The text in the dataset of Voigt et al.
Revised: The text in the dataset of Voigt et al.

Original: (2018) tends to be more casual and oral (e.g., “congrats , lee !” ).
Revised: (2018) tends to be more casual and oral (e.g., “congrats , lee !” ).

Original: Conversely, the text in our dataset is generally more formal and well-formed.
Revised: Conversely, the text in our dataset is generally more formal and well-formed.

Original: Completeness : Their dataset does not contain the original post to which the comments are responding, which causes some potential ambiguity on its political stance labels especially when the comment is generic (e.g., “happy birthday jerry : - )” ).
Revised: Completeness : Their dataset does not contain the original post to which the comments are responding, which causes some potential ambiguity on its political stance labels especially when the comment is generic (e.g., “happy birthday jerry : - )” ).

Original: Most of the sentences in our dataset are complete statements of certain facts.
Revised: Most of the sentences in our dataset are complete statements of certain facts.

Original: We also conducted an extra distillation step (described in Appendix A in the supplementary document) to only keep the sentences that are salient to the article’s stance label.
Revised: We also conducted an extra distillation step (described in Appendix A in the supplementary document) to only keep the sentences that are salient to the article’s stance label.

Original: Coverage :
Revised: Coverage :

Original: The comments in Voigt et al.
Revised: The comments in Voigt et al.

Original: (2018) are mainly from 2017-2018, while our dataset collects nearly 10-year of articles from 6/1/2012 to 4/1/2021, which covers a wider range of societal issues and events.
Revised: (2018) are mainly from 2017-2018, while our dataset collects nearly 10-year of articles from 6/1/2012 to 4/1/2021, which covers a wider range of societal issues and events.

Original: In the released version of our data we will add the time stamps to the sentences for customized ﬁltering.
Revised: In the released version of our data we will add the time stamps to the sentences for customized ﬁltering.

Original: A. S ENTENCE D ISTILLATION FOR C ONSTRUCTING P OLITICAL S TANCE D ATASET
Revised: A. S ENTENCE D ISTILLATION FOR C ONSTRUCTING P OLITICAL S TANCE D ATASET

Original: As mentioned in Section 3.1 of the paper, the articles collected from Allsides.com are already manually aligned (based on their political ideology) by Allsides editors.
Revised: As mentioned in Section 3.1 of the paper, the articles collected from Allsides.com are already manually aligned (based on their political ideology) by Allsides editors.

Original: We collected a total of 2,298 full-length news articles from opposite ideologies (i.e., liberal and conservative ) grouped by events.
Revised: We collected a total of 2,298 full-length news articles from opposite ideologies (i.e., liberal and conservative ) grouped by events.

Original: To create a sentence-level TST dataset for political stance transfer, we split the articles into sentences and took their article-level ideologies (as assigned by the editors) as their sentence-level stance labels.
Revised: To create a sentence-level TST dataset for political stance transfer, we split the articles into sentences and took their article-level ideologies (as assigned by the editors) as their sentence-level stance labels.

Original: However, we found that not all sentences explicitly exhibit the stance label inherited from their corresponding articles, so we implemented an additional distillation step to select those sentences that are salient with respect to their assigned political stance.
Revised: However, we found that not all sentences explicitly exhibit the stance label inherited from their corresponding articles, so we implemented an additional distillation step to select those sentences that are salient with respect to their assigned political stance.

Original: Assuming we have an article with N sentences
Revised: Assuming we have an article with N sentences

Original: X = { x 1 , x 2 , ..., x N } and the title of the article x title , we aim to ﬁnd the subset of sentences D distill ⊂ X which all explicitly exhibit the assigned political stance.
Revised: X = { x 1 , x 2 , ..., x N } and the title of the article x title , we aim to ﬁnd the subset of sentences D distill ⊂ X which all explicitly exhibit the assigned political stance.

Original: To better describe our approach, we ﬁrst introduce two concepts: (1) stance saliency, and (2) distillation cost.
Revised: To better describe our approach, we ﬁrst introduce two concepts: (1) stance saliency, and (2) distillation cost.

Original: Stance Saliency.
Revised: Stance Saliency.

Original: We compute the stance saliency v i for each sentence x i ∈
Revised: We compute the stance saliency v i for each sentence x i ∈

Original: X \ D distill , which is a measure of how well x i exhibits a certain political stance.
Revised: X \ D distill , which is a measure of how well x i exhibits a certain political stance.

Original: D distill is the set of sentences already picked our distilled set.
Revised: D distill is the set of sentences already picked our distilled set.

Original: We assume that the stance exhibited in x title matches that of the article (as assigned by the editors), Thus, we deﬁne stance saliency v i as the similarity between x i and the union of x title and D distill , which is:
Revised: We assume that the stance exhibited in x title matches that of the article (as assigned by the editors), Thus, we deﬁne stance saliency v i as the similarity between x i and the union of x title and D distill , which is:

Original: [Equation]
Revised: [Equation]

Original: where LCS ( · ) is longest common sequence between the two input sequences.
Revised: where LCS ( · ) is longest common sequence between the two input sequences.

Original: However, this metric is biased towards longer sentences as they can have more tokens matched with the title.
Revised: However, this metric is biased towards longer sentences as they can have more tokens matched with the title.

Original: We address this issue through the intriduction of a distilation cost, described below.
Revised: We address this issue through the intriduction of a distilation cost, described below.

Original: Distillation Cost.
Revised: Distillation Cost.

Original: We assign a distillation cost w i to each sentence x i in an article.
Revised: We assign a distillation cost w i to each sentence x i in an article.

Original: The cost is simply the length of the sentence:
Revised: The cost is simply the length of the sentence:

Original: [Equation]
Revised: [Equation]

Original: Note that for both concepts our computation is based on lower-cased and lemmatized sentence tokens.
Revised: Note that for both concepts our computation is based on lower-cased and lemmatized sentence tokens.

Original: We use λ to denote the ratio of the sentences to be picked as distilled sentences in an article consisting of N sentences, and deﬁne W max = λ · (cid:80) Ni =

Revised: We use λ to denote the ratio of the sentences to be picked as distilled sentences in an article consisting of N sentences, and deﬁne W max = λ · (cid:80) Ni =

Original: w i as the maximum distillation cost allowed.
Revised: w i as the maximum distillation cost allowed.

Original: Now that for each sentence we have a stance saliency and a distillation cost, we aim to choose a set of sentences as the distillation which have the highest possible sum of saliency for which the sum of distillation costs is not greater than W max .
Revised: Now that for each sentence we have a stance saliency and a distillation cost, we aim to choose a set of sentences as the distillation which have the highest possible sum of saliency for which the sum of distillation costs is not greater than W max .

Original: Given a function φ ( x i ) = { 1 , 0 } where 1 corresponds to sentence x i being picked for the distilled set and 0 means otherwise, the objective of distillation can be expressed as:
Revised: Given a function φ ( x i ) = { 1 , 0 } where 1 corresponds to sentence x i being picked for the distilled set and 0 means otherwise, the objective of distillation can be expressed as:

Original: [Equation]
Revised: [Equation]

Original: This is actually a NP-complete combinatorial optimization problem, called the Knapsack problem, which we solve using dynamic-programming (DP).
Revised: This is actually a NP-complete combinatorial optimization problem, called the Knapsack problem, which we solve using dynamic-programming (DP).

Original: In general, the distillation strategy aggressively picks sentences of high stance saliency while keeping the cost of distillation from exceeding the mask cost limitation W max .
Revised: In general, the distillation strategy aggressively picks sentences of high stance saliency while keeping the cost of distillation from exceeding the mask cost limitation W max .

Original: The procedure is demonstrated in Algorithm 1 below.
Revised: The procedure is demonstrated in Algorithm 1 below.

Original: Update Distillation Results Every Step
Revised: Update Distillation Results Every Step

Original: cost
Revised: cost

Original: Note that we initialize W distill with the length of title.
Revised: Note that we initialize W distill with the length of title.

Original: At every iteration of the loop we solve the DP problem with the limit set by current W distill , which is a variable that estimates how many sentences are already in the distillation set (no more than W max ).
Revised: At every iteration of the loop we solve the DP problem with the limit set by current W distill , which is a variable that estimates how many sentences are already in the distillation set (no more than W max ).

Original: The DP-based algorithm used in Algorithmfor ﬁnding the best distillation strategy is shown is Algorithm 2 below.
Revised: The DP-based algorithm used in Algorithmfor ﬁnding the best distillation strategy is shown is Algorithm 2 below.

Original: A. S TATISTICS OUR A LL T HREE D ATASETS , AND E XPERIMENTAL S ETUP
Revised: A. S TATISTICS OUR A LL T HREE D ATASETS , AND E XPERIMENTAL S ETUP

Original: We present more statistical details of all three datasets in Table 7.
Revised: We present more statistical details of all three datasets in Table 7.

Original: All of our experiments were run on a single RTX-2080 GPU, with batch size 4 and 2/3/2 epochs for LaMer in the above three TST tasks.
Revised: All of our experiments were run on a single RTX-2080 GPU, with batch size 4 and 2/3/2 epochs for LaMer in the above three TST tasks.

Original: Following previous work, we choose style control accuracy (ACC), BLEU score as our main metrics.
Revised: Following previous work, we choose style control accuracy (ACC), BLEU score as our main metrics.

Original: ACCs are judged by a pre-trained RoBERTa (Liu et al., 2019) classiﬁer with F1 = { 0.97, 0.86, 0.93 } using the training data of the three TST tasks.
Revised: ACCs are judged by a pre-trained RoBERTa (Liu et al., 2019) classiﬁer with F1 = { 0.97, 0.86, 0.93 } using the training data of the three TST tasks.

Original: About the J -score we calculate in few-shot experiments, we use the models released by the author 8 .
Revised: About the J -score we calculate in few-shot experiments, we use the models released by the author 8 .

Original: A. A DDITIONAL D ETAILS ABOUT T OKEN - LEVEL S CENE P RESERVATION
Revised: A. A DDITIONAL D ETAILS ABOUT T OKEN - LEVEL S CENE P RESERVATION

Original: In this section, we provide additional details regarding how we compute the token-level scene preservation d P SV .
Revised: In this section, we provide additional details regarding how we compute the token-level scene preservation d P SV .

Original: Figure 4 shows an example sentiment transfer task.
Revised: Figure 4 shows an example sentiment transfer task.

Original: For a given positive (source style) sentence s src and its corresponding negative (target style) parallels s tgt , we ﬁrst extract their
Revised: For a given positive (source style) sentence s src and its corresponding negative (target style) parallels s tgt , we ﬁrst extract their

Original: [Table]
Revised: [Table]

Original: datasets, we ﬁnd the political stance TST has the longest sentence length (see Avg.
Revised: datasets, we ﬁnd the political stance TST has the longest sentence length (see Avg.

Original: Sent Len), and more scene entities involved on average (see Avg.
Revised: Sent Len), and more scene entities involved on average (see Avg.

Original: # Scene Ent. / Sent).
Revised: # Scene Ent. / Sent).

Original: Human written references also reveal similar statistics (see the lower part of the table), which will potentially increase the difﬁculty for entity-matched transfer.
Revised: Human written references also reveal similar statistics (see the lower part of the table), which will potentially increase the difﬁculty for entity-matched transfer.

Original: We thus consider the political stance TST as a more challenging task than the other two.
Revised: We thus consider the political stance TST as a more challenging task than the other two.

Original: Algorithm
Revised: Algorithm

Original: Input: stance saliency v i , distill cost w i , and current cost limit W distill .
Revised: Input: stance saliency v i , distill cost w i , and current cost limit W distill .

Original: Record the number of stance saliency M = | v i | ;
Revised: Record the number of stance saliency M = | v i | ;

Original: Init DP-table T [ M + 1][ W distill + 1] with all 0; for i = 1 , 2 , . .
Revised: Init DP-table T [ M + 1][ W distill + 1] with all 0; for i = 1 , 2 , . .

Original: , M do for j = 1 , 2 , . . .
Revised: , M do for j = 1 , 2 , . . .

Original: , W distill do if j − w i − 1 < 0 then
Revised: , W distill do if j − w i − 1 < 0 then

Original: T [ i ][ j ] =
Revised: T [ i ][ j ] =

Original: T [ i − 1]
Revised: T [ i − 1]

Original: [ j ] ;
Revised: [ j ] ;

Original: Record distillation picking choice φ ( x i ) ; else
Revised: Record distillation picking choice φ ( x i ) ; else

Original: T [ i ][ j ] = max ( T [ i − 1]
Revised: T [ i ][ j ] = max ( T [ i − 1]

Original: [ j ] ,
Revised: [ j ] ,

Original: T [ i − 1]
Revised: T [ i − 1]

Original: [ j − w i − 1 ]
Revised: [ j − w i − 1 ]

Original: + v i − 1 ) ;
Revised: + v i − 1 ) ;

Original: Record distillation picking choice φ ( x i ) ; end end end { φ ( x i ) Mi =1 } ← backtracking via records; return best distillation strategy { φ ( x i )
Revised: Record distillation picking choice φ ( x i ) ; end end end { φ ( x i ) Mi =1 } ← backtracking via records; return best distillation strategy { φ ( x i )

Original: Mi =1 }
Revised: Mi =1 }

Original: ; scene entities using the method by Wu et al.
Revised: ; scene entities using the method by Wu et al.

Original: (Figure 4 (a)), and based on these entities we compute two distances d Order and d Exist as sub-components of d P SV .
Revised: (Figure 4 (a)), and based on these entities we compute two distances d Order and d Exist as sub-components of d P SV .

Original: For d Order , as shown in Figure 4 (b), we compare each of the s tgt entities with the s src entities, and measure how much s tgt needs to be edited to make the two sets of entities from opposite styles identical to each other.
Revised: For d Order , as shown in Figure 4 (b), we compare each of the s tgt entities with the s src entities, and measure how much s tgt needs to be edited to make the two sets of entities from opposite styles identical to each other.

Original: As mentioned in the paper, we use the Levenshtein distance as the edit distance; whose set of edits operation is: 1) insertion of a single token, 2) deletion of a single token, and
Revised: As mentioned in the paper, we use the Levenshtein distance as the edit distance; whose set of edits operation is: 1) insertion of a single token, 2) deletion of a single token, and

Original: 3) substitution of a single token.
Revised: 3) substitution of a single token.

Original: Each of these operations has unit cost, so the Levenshtein distance is equal to the minimum number of operations required to transform target style entities to source style entities.
Revised: Each of these operations has unit cost, so the Levenshtein distance is equal to the minimum number of operations required to transform target style entities to source style entities.

Original: [they, menu, beef] [0, 1, 1] [1, 1, 1] [they, menu, nothing, beef] [menu, beef] [nothing, menu] [they, menu, beef] [they, menu, beef] [0, 1, 1, 0] [1, 1, 0, 1] they had an extensive menu and the beef was delicious .
Revised: [they, menu, beef] [0, 1, 1] [1, 1, 1] [they, menu, nothing, beef] [menu, beef] [nothing, menu] [they, menu, beef] [they, menu, beef] [0, 1, 1, 0] [1, 1, 0, 1] they had an extensive menu and the beef was delicious .

Original: overpriced menu, and mediocre beef .
Revised: overpriced menu, and mediocre beef .

Original: nothing great on their menu .
Revised: nothing great on their menu .

Original: [Figure]
Revised: [Figure]

Original: As shown in Figure 4 (c), we calculate another distance component, d Exist , via hamming distance to emphasize the the difference between the sets of entities in source and target sentences.
Revised: As shown in Figure 4 (c), we calculate another distance component, d Exist , via hamming distance to emphasize the the difference between the sets of entities in source and target sentences.

Original: We ﬁrst get the union of the entities from a pair of source and target sentences; based on the union set, we can obtain the corresponding binary vector representations of both style entities: 1 means the entity exists in a certain style sentence, and 0 otherwise.
Revised: We ﬁrst get the union of the entities from a pair of source and target sentences; based on the union set, we can obtain the corresponding binary vector representations of both style entities: 1 means the entity exists in a certain style sentence, and 0 otherwise.

Original: Note the binary vectors for each of the sentences will be the same length (since it corresponds to the union of entities).
Revised: Note the binary vectors for each of the sentences will be the same length (since it corresponds to the union of entities).

Original: Hamming distance measures the difference between source and target sentences on the shared entities set (this was shown to be an upper bound on the Levenshtein distance).
Revised: Hamming distance measures the difference between source and target sentences on the shared entities set (this was shown to be an upper bound on the Levenshtein distance).

Original: Though there are overlaps between this metric and d Order , we use this metric to emphasize the difference in scene entities.
Revised: Though there are overlaps between this metric and d Order , we use this metric to emphasize the difference in scene entities.

Original: A. M ORE D ETAILS ABOUT P OLICY G RADIENT
Revised: A. M ORE D ETAILS ABOUT P OLICY G RADIENT

Original: We detail our policy gradient algorithm in Algorithm 3.
Revised: We detail our policy gradient algorithm in Algorithm 3.

Original: Note that J safeIL is a safe-reward threshold.
Revised: Note that J safeIL is a safe-reward threshold.

Original: If the greedy policy can already reach this threshold the policy will not be updated.
Revised: If the greedy policy can already reach this threshold the policy will not be updated.

Original: Empirically we set
Revised: Empirically we set

Original: J safeIL to { 0 . 8 , 0 . 6 , 0 . 4 } for the three TST tasks (sentiment, formality, and political stance).
Revised: J safeIL to { 0 . 8 , 0 . 6 , 0 . 4 } for the three TST tasks (sentiment, formality, and political stance).

Original: i ≤ max exploration steps do
Revised: i ≤ max exploration steps do

Original: [Equation]
Revised: [Equation]

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: We choose the REINFORCE algorithm (Williams, 1992) to optimize the current policy π θ .
Revised: We choose the REINFORCE algorithm (Williams, 1992) to optimize the current policy π θ .

Original: One known issue of the vanilla REINFORCE method is the instability of its optimization because of the high variance of the gradients.
Revised: One known issue of the vanilla REINFORCE method is the instability of its optimization because of the high variance of the gradients.

Original: A common remedy is to add a baseline function (i.e., a parameterized value-network whose purpose is to estimate the unbiased future behavior of current policy (Ranzato et al., 2016)).
Revised: A common remedy is to add a baseline function (i.e., a parameterized value-network whose purpose is to estimate the unbiased future behavior of current policy (Ranzato et al., 2016)).

Original: Rennie et al.
Revised: Rennie et al.

Original: (2017) introduce a better approach that estimates the baseline at test time.
Revised: (2017) introduce a better approach that estimates the baseline at test time.

Original: They use the greedy decoding results as the baseline in their self-critic sequence training algorithm.
Revised: They use the greedy decoding results as the baseline in their self-critic sequence training algorithm.

Original: We adopt their method to provide the baseline, and add an exploration step using sampling decoding.
Revised: We adopt their method to provide the baseline, and add an exploration step using sampling decoding.

Original: As shown in Algorithm 1, by iteratively switching between these two modes, we calculate corresponding contrastive losses deﬁned by Equation 3, and take the difference of two modes losses as the advantage, feeding into the REINFORCE algorithm to optimize our policy π θ .
Revised: As shown in Algorithm 1, by iteratively switching between these two modes, we calculate corresponding contrastive losses deﬁned by Equation 3, and take the difference of two modes losses as the advantage, feeding into the REINFORCE algorithm to optimize our policy π θ .

Original: Finally, the output policy π θ is able to generate style transferred sequences resemble the expert demonstrations s tgt+ .
Revised: Finally, the output policy π θ is able to generate style transferred sequences resemble the expert demonstrations s tgt+ .

Original: A. E RROR A NALYSIS , AND A DDITIONAL G ENERATION S AMPLES OF L A M ER
Revised: A. E RROR A NALYSIS , AND A DDITIONAL G ENERATION S AMPLES OF L A M ER

Original: In the introduction of the main paper
Revised: In the introduction of the main paper

Original: [Table]
Revised: [Table]

Original: we brieﬂy discuss the pitfalls of existing TST models.
Revised: we brieﬂy discuss the pitfalls of existing TST models.

Original: Tableshows actual generation samples from these methods to further highlight their weaknesses.
Revised: Tableshows actual generation samples from these methods to further highlight their weaknesses.

Original: We take the sentiment transfer task—which was used by all models in their evaluations—and pick a negative source style sentence carrying the style in an implicit way (i
Revised: We take the sentiment transfer task—which was used by all models in their evaluations—and pick a negative source style sentence carrying the style in an implicit way (i

Original: [Table]
Revised: [Table]

Original: e., there are no salient token-level cues exhibiting negative sentiment 9 ).
Revised: e., there are no salient token-level cues exhibiting negative sentiment 9 ).

Original: All TST models should generate a transferred positive sentence.
Revised: All TST models should generate a transferred positive sentence.

Original: [Table]
Revised: [Table]

Original: As can be seen in the table, CAE (Shen et al., 2017) suffers from discarding too much styleindependent content.
Revised: As can be seen in the table, CAE (Shen et al., 2017) suffers from discarding too much styleindependent content.

Original: When the style cannot be interpreted at the token level, DelRetrGen (Li et al., 2018a) results in generic generation since it is difﬁcult to transfer implicit styled sentence through token-editing.
Revised: When the style cannot be interpreted at the token level, DelRetrGen (Li et al., 2018a) results in generic generation since it is difﬁcult to transfer implicit styled sentence through token-editing.

Original: From our empirical observation, Style Transformer (Dai et al., 2019) typically generates sentences with low readability.
Revised: From our empirical observation, Style Transformer (Dai et al., 2019) typically generates sentences with low readability.

Original: Deep Latent Seq (He et al., 2020) tends to generate unrelated text (e.g., “...family appreciated...” ) and sometimes text with low readability (e.g., “...the ﬁrst time with the ﬁrst time...” ).
Revised: Deep Latent Seq (He et al., 2020) tends to generate unrelated text (e.g., “...family appreciated...” ) and sometimes text with low readability (e.g., “...the ﬁrst time with the ﬁrst time...” ).

Original: This is partially because their assumption that TST datasets can be aligned in latent space is too strong.
Revised: This is partially because their assumption that TST datasets can be aligned in latent space is too strong.

Original: We have included sample generations from LaMer for the test sets of all three tasks with our code submission.
Revised: We have included sample generations from LaMer for the test sets of all three tasks with our code submission.

Original: A. A DDITIONAL D ETAILS ABOUT H UMAN E VALUATION
Revised: A. A DDITIONAL D ETAILS ABOUT H UMAN E VALUATION

Original: We provide more details about our human evaluation in this section.
Revised: We provide more details about our human evaluation in this section.

Original: The results on sentiment TST are shown in Table 9 and those of formality TST are shown in Table 10.
Revised: The results on sentiment TST are shown in Table 9 and those of formality TST are shown in Table 10.

Original: Details About Participants and Compensation.
Revised: Details About Participants and Compensation.

Original: Participants ( N =212) were all from the United
Revised: Participants ( N =212) were all from the United

Original: States and above 18 years old.
Revised: States and above 18 years old.

Original: Participants were required to have a HIT approval rate greater than 95%.
Revised: Participants were required to have a HIT approval rate greater than 95%.

Original: The average age of the participants was 34.95 years-old (SD=9.90, Median=32).
Revised: The average age of the participants was 34.95 years-old (SD=9.90, Median=32).

Original: 141 of the participants (66.5%) were male, and 71 female (33.5%).
Revised: 141 of the participants (66.5%) were male, and 71 female (33.5%).

Original: Participants received 14.41 years of education on average (SD=4.46, Median=16).
Revised: Participants received 14.41 years of education on average (SD=4.46, Median=16).

Original: When asked to self-report their party afﬁliation, 52.4% participants self-reported as Democratic, 35.3% participants were self-reported Republicans, and 12.3% participants identiﬁed as independent.
Revised: When asked to self-report their party afﬁliation, 52.4% participants self-reported as Democratic, 35.3% participants were self-reported Republicans, and 12.3% participants identiﬁed as independent.

Original: Each participant was compensated 1 USD per task.
Revised: Each participant was compensated 1 USD per task.

Original: The average completion time per questionnaire was about 7.6 minutes as reported by the platform, which results in a $7.9/hr wage, above the regional and federal minimum wage.
Revised: The average completion time per questionnaire was about 7.6 minutes as reported by the platform, which results in a $7.9/hr wage, above the regional and federal minimum wage.

Original: Details About Human Evaluation Procedure.
Revised: Details About Human Evaluation Procedure.

Original: We evaluate the style transferred sentences generated by baselines and our LaMer.
Revised: We evaluate the style transferred sentences generated by baselines and our LaMer.

Original: The sentences are picked from the test set generations for the three tasks.
Revised: The sentences are picked from the test set generations for the three tasks.

Original: Each participant was asked to read two sets of sentences (one set containing a source style sentence, corresponding human-written target style sentence as ground-truth, and eight versions of target style sentences generated by seven existing methods and LaMer for comparison).
Revised: Each participant was asked to read two sets of sentences (one set containing a source style sentence, corresponding human-written target style sentence as ground-truth, and eight versions of target style sentences generated by seven existing methods and LaMer for comparison).

Original: After reading each set, they were asked to rate how well the generated sentences did in style control, readability, and content preservation on a 7-point Likert scales.
Revised: After reading each set, they were asked to rate how well the generated sentences did in style control, readability, and content preservation on a 7-point Likert scales.

Original: For example, for content preservation, they were asked “How much do you agree with the following statement (1- Strongly disagree to 7- Strongly agree): The machine-generated target style text preserves the context of the source style text well
Revised: For example, for content preservation, they were asked “How much do you agree with the following statement (1- Strongly disagree to 7- Strongly agree): The machine-generated target style text preserves the context of the source style text well

Original: (i
Revised: (i

Original: [Table]
Revised: [Table]

Original: e. they talk about the same thing).”.
Revised: e. they talk about the same thing).”.

Original: Higher ratings correspond to better performance.
Revised: Higher ratings correspond to better performance.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: A. D ISCUSSION ON L A M ER ’ S E FFICIENCY
Revised: A. D ISCUSSION ON L A M ER ’ S E FFICIENCY

Original: We are curious about whether existing methods that suffer from lack of direct supervision can take the advantage of aligned datasets.
Revised: We are curious about whether existing methods that suffer from lack of direct supervision can take the advantage of aligned datasets.

Original: In Table
Revised: In Table

Original: [Table]
Revised: [Table]

Original: we use the same aligned datasets used by LaMer (added aligned by RD as for ablation), to train three exiting systems by one epoch.
Revised: we use the same aligned datasets used by LaMer (added aligned by RD as for ablation), to train three exiting systems by one epoch.

Original: Of note, all these methods beneﬁt in different perspectives.
Revised: Of note, all these methods beneﬁt in different perspectives.

Original: For example, DelRetrGen (Li et al., 2018a) makes large improvements in style control accuracy, BLEU score, and ﬂuency (PPL).
Revised: For example, DelRetrGen (Li et al., 2018a) makes large improvements in style control accuracy, BLEU score, and ﬂuency (PPL).

Original: Most prior work emphasize their advances in performance, but do not consider learning efﬁciency.
Revised: Most prior work emphasize their advances in performance, but do not consider learning efﬁciency.

Original: In the same one epoch setting,
Revised: In the same one epoch setting,

Original: LaMer achieves the best performance compared with others.
Revised: LaMer achieves the best performance compared with others.

Original: Meanwhile, when further decreasing the training data size to 1/10 of the original, LaMer can still produce acceptable results, better than most of other methods trained with the whole epoch, mainly because LaMer is LM-based.
Revised: Meanwhile, when further decreasing the training data size to 1/10 of the original, LaMer can still produce acceptable results, better than most of other methods trained with the whole epoch, mainly because LaMer is LM-based.

Original: Combining all these key factors for efﬁcient learning, LaMer can serve as a general-purpose TST model for either data rich or data scarce cases.
Revised: Combining all these key factors for efﬁcient learning, LaMer can serve as a general-purpose TST model for either data rich or data scarce cases.

Original: A. P ERPLEXITY MEASURE OF L A M ER AND OTHER BASELINES
Revised: A. P ERPLEXITY MEASURE OF L A M ER AND OTHER BASELINES

Original: In Table 12 we present the perplexity measure of LaMer and other baselines as a proxy for ﬂuency.
Revised: In Table 12 we present the perplexity measure of LaMer and other baselines as a proxy for ﬂuency.

Original: As can be seen, certain methods, such as CAE and Style Transformer, have very high perplexity in more challenging TST tasks (e.g., Political Stance), potentially because the task requires longer generation and more entities involved in the sentences (see Table 7 for further discussions).
Revised: As can be seen, certain methods, such as CAE and Style Transformer, have very high perplexity in more challenging TST tasks (e.g., Political Stance), potentially because the task requires longer generation and more entities involved in the sentences (see Table 7 for further discussions).

Original: However, it should also be noted that prior work have shown perplexity to be a poor measure of ﬂuency Krishna et al.
Revised: However, it should also be noted that prior work have shown perplexity to be a poor measure of ﬂuency Krishna et al.

Original: ; Mir et al.
Revised: ; Mir et al.

Original: ; Lee et al.
Revised: ; Lee et al.

Original: [Table]
Revised: [Table]

Original: with one epoch, we study their learning efﬁciency compared with our method: LaMer obtains the best result on this sentiment transfer task, and is robust to data scarcity
Revised: with one epoch, we study their learning efﬁciency compared with our method: LaMer obtains the best result on this sentiment transfer task, and is robust to data scarcity

Original: [Table] [Table]
Revised: [Table] [Table]