SC 2: Towards Enhancing Content Preservation and Style Consistency in Long Text Style Transfer (2024)

Abstract: Text style transfer (TST) aims to vary the style polarity of text while preserving the semantic content. Although recent advancements have demonstrated remarkable progress in short TST, it remains a relatively straightforward task with limited practical applications. The more comprehensive long TST task presents two challenges: (1) existing methods encounter difficulties in accurately evaluating content attributes in multiple words, leading to content degradation; (2) the conventional vanilla style classifier loss encounters obstacles in maintaining consistent style across multiple generated sentences.In this paper, we propose a novel method SC2, where a multilayer Joint Style-Content Weighed (JSCW) module and a Style Consistency loss are designed to address the two issues. The JSCW simultaneously assesses the amounts of style and content attributes within a token, aiming to acquire a lossless content representation and thereby enhancing content preservation. The multiple JSCW layers further progressively refine content representations. We design a style consistency loss to ensure the generated multiple sentences consistently reflect the target style polarity. Moreover, we incorporate a denoising non-autoregressive decoder to accelerate the training. We conduct plentiful experiments and the results show significant improvements of SC2 over competitive baselines. Our code: https://github.com/jiezhao6/SC2.

## introduction text style transfer (tst) aims to generate a text exhibiting a desired style based on the source text (e.g., @xmath0, while endeavoring to faithfully preserve the semantic content. the applications of tst cover a wide range of user-centric natural language generation tasks, such as personalized dialogue systems @xcite , educa- * corresponding author. given the scarcity of parallel data (i.e., text pairs conveying the same content but differing in styles) and the labor-intensive nature of annotating such pairs, existing research has predominantly focused on unsupervised tst. recent contributions in this domain, including studies by @xcite @xcite @xcite , have demonstrated significant progress. despite notable success, these works primarily concentrate on the transfer of a single sentence, which we call short tst. this is a relatively simple task and is difficult to apply to complex scenarios, i.e., long tst such as transferring news articles and novels. it is challenging to achieve desirable content preservation and style consistency across multiple sentences for these methods. in a very recent study, @xcite first concentrated on the long tst task and proposed storytrans, which learns content representations and fills stylistic tokens in separate stages. while somewhat effective, challenges persist in preserving content and ensuring style consistency. ## conclusion in this paper, we propose a novel method sc2 for long tst. we design multiple jscw layers to progressively refine content representations of tokens, thus enhancing content preservation. and we design a style consistency loss to ensure the generated sentences share similarities in style polarity. experimental results with automatic and human evaluations confirm the effectiveness of sc2 compared to competitive baselines.