Neural Stylistic Response Generation with Disentangled Latent Variables (2021)

Abstract: Generating open-domain conversational responses in the desired style usually suffers from the lack of parallel data in the style. Meanwhile, using monolingual stylistic data to increase style intensity often leads to the expense of decreasing content relevance. In this paper, we propose to disentangle the content and style in latent space by diluting sentence-level information in style representations. Combining the desired style representation and a response content representation will then obtain a stylistic response. Our approach achieves a higher BERT-based style intensity score and comparable BLEU scores, compared with baselines. Human evaluation results show that our approach significantly improves style intensity and maintains content relevance.

## introduction linguistic style is an essential aspect of natural language interaction and provides particular ways of using language to engage with the audiences @xcite . in human-bot conversations, it is crucial to generate stylistic responses for increasing user engagement to conversational systems @xcite . currently, most of the existing parallel datasets are not stylistically consistent. samples in these datasets are usually contributed by a variety of users, resulting in an averaging effect across style characteristics @xcite . meanwhile, constructing a parallel stylistic dataset for training the open-domain conversational agents is both labor-intensive and time-consuming. recent studies show the effect of stylizing responses using a monolingual dataset in the desired style and a conventional conversational dataset @xcite . however, increasing style intensity often leads to ours: i should prefer having a partner to being alone. s2s+lm: my friend had a considerable share in clearing the matter up. figure 1 : an example of responses generated by s2s, s2s+lm @xcite , style fusion @xcite , and our approach, targeting the holmes style, which is quite formal and polite. the expense of decreasing content relevance between dialogue history and response. as an example in figure 1 shows, @xcite independently train a response generation model and a stylistic language model and subsequently interpolates them in the inference phase. lacking the interaction between the stylistic language model and response generation encoder, it usually yields a trade-off between style intensity and content relevance. @xcite fuse a structured latent space where the direction denotes the diversity, and the distance denotes style intensity and content relevance. the main issue is that style intensity and content relevance are contradictory in measurement but are coupling to the same "distance" metric of the latent space. to sum up, the key issue of the above studies is the improper entanglement of style and content. to address the issue, we propose to disentangle the style and content of a response. the disentanglement is conducted on the structured latent space, where each sentence (dialogue history, response, and stylistic sentence) is projected into a vector representation. we further split the representation into two components: style and content representations. the former is a corpus-level feature since sentences within a dataset have the same style. in contrast, the content representation is a sentence-level feature decided by a sentence itself. we thus disentangle the content and style by diluting sentence-level information in the style representation. this encourages the encoding of content information into the content representation. otherwise, the content information will be corrupted in the style representation, making it hard to reconstruct the original content in the subsequent decoding process. we conduct experiments on dailydialogue conversational dataset @xcite and holmes monolingual stylistic dataset @xcite . experimental results show that our proposed approach improves style intensity and maintains content relevance. our contributions are listed below: ## conclusion we propose a uniform framework to simultaneously improve the style intensity and maintain the content relevance for neural stylistic response generation. in contrast to existing approaches, our approach disentangles the style and the content in the latent space by a diluting strategy. experiments show that our approach improves the style intensity of generated responses and maintains the content relevance at the same time, which demonstrates the effectiveness of this approach.