Contextual Text Style Transfer (2020)

Abstract: We introduce a new task, Contextual Text Style Transfer - translating a sentence into a desired style with its surrounding context taken into account. This brings two key challenges to existing style transfer approaches: (I) how to preserve the semantic meaning of target sentence and its consistency with surrounding context during transfer; (ii) how to train a robust model with limited labeled data accompanied by context. To realize high-quality style transfer with natural context preservation, we propose a Context-Aware Style Transfer (CAST) model, which uses two separate encoders for each input sentence and its surrounding context. A classifier is further trained to ensure contextual consistency of the generated sentence. To compensate for the lack of parallel data, additional self-reconstruction and back-translation losses are introduced to leverage non-parallel data in a semi-supervised fashion. Two new benchmarks, Enron-Context and Reddit-Context, are introduced for formality and offensiveness style transfer. Experimental results on these datasets demonstrate the effectiveness of the proposed CAST model over state-of-the-art methods across style accuracy, content preservation and contextual consistency metrics.

## context-aware style transfer in this section, we first describe the problem definition and provide an overview of the model architecture in section 3.1. section 3.2 presents the proposed context-aware style transfer (cast) model with supervised training objectives, and section 3.3 further introduces how to augment the cast model with non-parallel data in a hybrid approach. ## experiments in this section, we compare our model with state-ofthe-art baselines on the two new benchmarks, and provide both quantitative analysis and human evaluation to validate the effectiveness of the proposed cast model. ## conclusion in this paper, we present a new task -contextual text style transfer. two new benchmark datasets are introduced for this task, which contain annotated sentence pairs accompanied by paragraph context. we also propose a new cast model, which can effectively enforce content preservation and context coherence, by exploiting abundant nonparallel data in a hybrid approach. quantitative and human evaluations demonstrate that cast model significantly outperforms baseline methods that do not consider context information. we believe our model takes a first step towards modeling context information for text style transfer, and will explore more advanced solutions e.g., using a better encoder/decoder like gpt-2 @xcite and bert @xcite , adversarial learning @xcite or knowledge distillation @xcite .