Related Works
Style transfer in NLP has been studied in many variations. One line of research studied this problem as transferring to/from the style of popular novelists to/from modern English. Such as Boyd et al. ( 2020 ) used paraphrasing model for this purpose. Another variation is transferring style to a fictional movie/novel character’s style as studied by Han et al. ( 2022 ) . Other works studied style transfer by defining style attributes and transferring text style from one attribute to another (e.g., positive/negative, informal/formal) (Pavlick and Tetreault, 2016 ; Rao and Tetreault, 2018 ; Niu et al., 2018 ; Wang et al., 2019 ; Briakou et al., 2021 ; Zhang et al., 2018 ; Madaan et al., 2020 ; Reif et al., 2022 ) .

Existing style transfer approaches make different assumptions about data availability. Certain approaches assume the availability of a lot of training data in the target style and use either a sequence-to-sequence model (Rao and Tetreault, 2018 ; Niu et al., 2018 ; Riley et al., 2021 ) or a controlled text generation model guided by a schema (Tsai et al., 2021 ) or rules (Wang et al., 2019 ) . Other approaches assume the availability of zero or a small number of training examples and leverage either auto-encoders for controlled text generation such as sentiment polarity transfer and tense alteration (Shen et al., 2017 ; Mai et al., 2020 ; Shen et al., 2020 ; Montero et al., 2021 ; Shen et al., 2020 ) or in-context learning based on LLMs for specific attributes (Reif et al., 2022 ; Suzgun et al., 2022 ; Han et al., 2022 ) .

Another line of research studied style transfer by mapping texts with different style attributes in a common latent space that is independent of the style attributes, however, preserves the semantic meaning. This approach is conceptually similar to our idea of using style-free utterances as pivots. For example, Shen et al. ( 2017 ) assumed a shared latent content distribution across different text corpora, and proposed a method that aligns the latent representations to perform style transfer. They used an adversarial discriminator to align the latent spaces of different styles. Later Yang et al. ( 2018 ) extended this idea by using language models as discriminators by addressing the instability of the error signals provided by the GAN-based discriminators. Several works have been done along the line (Prabhumoye et al., 2018 ; Gao et al., 2019 ; Madaan et al., 2020 ) that utilized the concept of latent-space representation for style transfer. However, in a counter-study to such approaches that depend on the latent space representation for style transfer, Subramanian et al. ( 2018 ) showed that the assumptions related to the latent space are not necessary and are not always met in practice.

Existing works mostly ignore the context beyond a single sentence while transferring the style and rely on style attribute definitions. Recently, a few attempts have been made in the domain of contextual style transfer. For example, Cheng et al. ( 2020 ) studied style transfer of text in context where the context is defined as the paragraph where the input text appears. Han et al. ( 2022 ) studied style transfer in a contextualized setting where the LLMs are prompted to answer a question in the style of fictional characters. The question is used as context. However, the styles of the fictional characters are too evident and characterized by special words and other fictional characters involved in the novels or movies. In contrast, in this paper, we study style transfer in Task-Oriented Dialogues where (1) the context is the previous turns among the speakers, (2) there are only a few examples of the target style available, and (3) the style attributes are unknown and the conversation style may be a combination of many style attributes.

Recent surveys have emphasized applications of text style transfer in domain adaptation Jin et al. ( 2022 ) . In this paper, we take the first step towards applying style transfer to adapt training data for the downstream task of intent classification.