Audience-Centric Natural Language Generation via Style Infusion (2022)

Abstract: Adopting contextually appropriate, audience-tailored linguistic styles is critical to the success of user-centric language generation systems (e.g., chatbots, computer-aided writing, dialog systems). While existing approaches demonstrate text style transfer (TST) with large volumes of parallel or non-parallel data, we argue that grounding style on audience-independent external factors is innately limiting for two reasons. First, it is difficult to collect large volumes of audience-specific stylistic data. Second, some stylistic objectives (e.g., persuasiveness, memorability, empathy) are hard to define without audience feedback. In this paper, we propose the novel task of style infusion - infusing the stylistic preferences of audiences in pretrained language generation models. Since humans are better at pairwise comparisons than direct scoring - i.e., is Sample-A more persuasive/polite/empathic than Sample-B - we leverage limited pairwise human judgments to bootstrap a style analysis model and augment our seed set of judgments. We then infuse the learned textual style in a GPT-2 based text generator while balancing fluency and style adoption. With quantitative and qualitative assessments, we show that our infusion approach can generate compelling stylized examples with generic text prompts. We make the anonymized code and data accessible.

## introduction in this paper, we develop a novel approach to infuse audience-centric styles into pretrained language generation (nlg) models. learning to synthesize subjective styles is crucial to various applications. for instance, persuasion and memorability in computational advertising and marketing @xcite . user-centric applications of language generation, such as writing aids, chatbots, and dialog systems, often require these stylistic adjust-ments depending on both the audience and the task. prior work often defines textual style with large static sentence collections. however, stylistic objectives such as persuasiveness, memorability, and empathy are hard to define without a target audience @xcite due to non-uniform stylistic expectations across diverse user groups. thus, we suggest that subjective text styles and traits must be defined by the target audience instead of audienceindependent data. our work focuses on two resulting challenges -first, how to collect target audience feedback, and second, how to leverage the limited feedback efficiently for style infusion. textual styles -i.e., different linguistic presentations of the same conceptual content -play an integral role in persuasive/memorable communication. for instance, an informal style is less persuasive in formal settings @xcite . the style problem extends across diverse domains, from empathic styling in mental health @xcite to fact-driven, simplistic styling in tech support @xcite . existing work in textual style transfer (tst) takes two general approaches. the strictly supervised approaches leverage fixed parallel corpora, analogous to machine translation @xcite , while semisupervised and unsupervised techniques leverage non-parallel collections of stylized sentences @xcite . predefined metrics, heuristics, external oracles, and hybrid approaches have also been considered @xcite . constructing audience-centric or time-evolving / adaptive methods for style transfer remains an open challenge. existing approaches are guided by rigid modeling considerations and the distributions of fixed style-specific corpora. this is innately limiting for stylistic objectives such as persuasiveness, a trait with a widely disputed definition in existing literature, and multiple external confounds such as preexisting biases and independent features of the persuader (e.g., how many followers they have) @xcite @xcite @xcite . furthermore, it is infeasible to collect extensive annotated collections of text for each audience, style, and application @xcite . unlike prior work, we define and incorporate style grounded on our target audience. to address dynamic settings requiring audience-centric linguistic styles, we propose the novel style-infusion task. since human reviewers are better at pairwise style comparisons than direct scoring @xcite , we formulate style infusion as follows: how do we infuse the stylistic preferences of our audience, via pairwise sentence comparisons, in a generative language model (lm)? unlike conventional style transfer, our task leverages domain and audience-specific feedback instead of parallel non-parallel sentence collections rendered in any specific style. further, we adopt an incremental training approach rather than retraining models from scratch. we bootstrap an initial style analysis model to discriminate the positive and negative samples from audience feedback. our model then selects additional samples from a generic topical sentence collection to expand the seed set of audience judgments. by separating style analysis and text generation models, we create an adversarial setup to infuse the audience's stylistic feedback in any generative lm. we weight the noisy reward from the style analysis model (discriminator) with a reconstruction loss to balance style adoption and fluency. in summary, our contributions are as follows: ## related work prior work has explored "style transfer" in diverse settings ranging from "clickbait" headlines to formalizing text @xcite @xcite . while strictly supervised approaches show high fidelity to input samples @xcite , unsupervised and minimally supervised learning are widely applicable since parallel samples are unavailable @xcite . disentanglement, prototype editing, and pseudoparallel corpus creation are popular approaches. prototype editing applies stylistic markers to predefined sentence templates @xcite , disentanglement extracts style independent of the content @xcite . audience-centric feedback may not conform to these rigid hypotheses. first, unconstrained generation allows for freedom in sentence and paragraphlevel constructs to define the style @xcite . second, the separability of content and style is harder in specialized domains reliant on domainspecific jargon @xcite and expressions. our bootstrapping approach shares some commonalities with pseudo-parallel corpus creation (e.g. aligning sentences from two mono-style corpora) @xcite , but only utilizes a generic topical corpus to expand the audience-generated "seed set" of pairwise judg-ments. adversarial training has also been used to quantify style @xcite . our approach explicitly decouples the style discrimination and generation tasks for modularity and incremental training purposes. we pick two stylistic objectives that are highly audience-dependent and hard to define objectively -memorability and persuasiveness -to evaluate our approach. prior work in these styles has been limited to analysis but not generation. @xcite and @xcite find linguistic patterns, interaction dynamics, and discourse structure are strong identifiers of persuasive arguments, while convincingness @xcite , memorability @xcite have been better explained by linguistic feature correlation. however, there is a lack of work on unconstrained generation of persuasive and memorable text @xcite . our approach enables us to bridge some of these specific gaps while maintaining a generalized overall formulation. ## discriminative language model in this section, we train a bert-based style discriminator to provide feedback to our generator. ## style-aware language generation in this section, we infuse the stylistic preferences learned by our style discriminator in section 3 into a gpt-2 model @xcite pretrained on the causal language modeling (clm) objective foot_0 . the model takes in a textual prompt and generates text, y, that we want to infuse with the audience preferred style. for the persuasiveness task, the ukpconvarg1 dataset provides prompts for each argument pair. for the memorability task, we use the previous sentence as the prompt. during training, we feed the prompt and feedback pair to gpt-2 -the more preferred (styled) sample, y * s , and the corresponding less-preferred (non-styled) sample, y * ns . we use an adversarial training paradigm to enable the generator to learn from the discriminator, illustrated in figure 1 . ## experimental settings we compare our architecture against a few strong baselines: pretrained gpt-2 @xcite we use this pre-trained model as a representation of average text, allowing us to show shifts in style that occur due to training. fine-tuning we fine-tune the pre-trained gpt-2 model using the reconstruction objective on the style-specific corpus only (e.g. ukpconv1). fine-tuning + data augmentation we fine-tune the pre-trained gpt-2 model using the reconstruction objective on the augmented data. titlestylist @xcite we adapt the stylistic headline generation framework to generate stylistic text based on a prompt. @xcite utilize a denoising autoencoder with parameter sharing to disentangle style from content to control the style with a set of parameters. training settings for all gpt-2 based models, we use a base gpt-2 model from huggingface @xcite (1024 dimensions, adam optimizer, η = 5e -5). because of the length of our text and size of our models, we utilize deepspeed @xcite to distribute training over two 32gb v100s, and we train with fp16 mixed precision. we experiment with the loss parameters of c and β and discuss our findings in section 8. ## results on persuasiveness in this section, we analyze our results by showing a significant usage of linguistic features that resemble persuasive text, showing generated text, and with standard metrics. ## results on memorability in this section, we focus on memorability and show that our model can generate more robust, relevant, and memorable text than the baselines. ## empirical observations we analyze how the value of β affects generations, finding that generations from β = 0.1 suffer the same degeneration as fine-tuning while higher values avoid these issues. because α s is not always 1, the constant in front of the discriminator loss is less than β. consequently, the discriminator is not given enough weight, and the generator cannot learn as effectively from the discriminator. it is difficult to distinguish differences between β = 0.5 and β = 1.0, but aside from β = 0.5, β = 1.0 outperforms every other value of β. we also experiment with hard-coding the coefficients for the discriminator and reconstruction loss in ## conclusion in this paper, we introduced style infusion to motivate infusing audience-centric, stylistic preferences into unconstrained natural language generation models. we present a bootstrapped data augmentation method for limited pair-wise audience feedback and an adversarial training framework with a decoupling loss to train a style-infused gpt-2. through an automatic evaluation method for the transfer of audience-specific styles, we show that our approach generates compelling stylized examples with generic text prompts better than the baselines. synthesizing text with subjective styles, such as persuasion and memorability, remains a significant challenge in domains like computational advertising. our work takes the first few steps to address this problem. we plan to continue improving our work in many directions, such as incorporating long-document attention mechanisms @xcite to capture document-level style features and altering the discourse structure to convey information in a more interpretable manner.