Controlling Dialogue Generation with Semantic Exemplars (2021)

Abstract: Dialogue systems pretrained with large language models generate locally coherent responses, but lack fine-grained control over responses necessary to achieve specific goals. A promising method to control response generation is exemplar-based generation, in which models edit exemplar responses that are retrieved from training data, or hand-written to strategically address discourse-level goals, to fit new dialogue contexts. We present an Exemplar-based Dialogue Generation model, EDGE, that uses the semantic frames present in exemplar responses to guide response generation. We show that controlling dialogue generation based on the semantic frames of exemplars improves the coherence of generated responses, while preserving semantic meaning and conversation goals present in exemplar responses.

## introduction large pre-trained language models @xcite currently used to power dialogue generation systems produce increasingly fluent and appropriate responses for novel dialogue contexts @xcite @xcite . however, the generated responses are often uninformative or inconsistent with high-level constraints of a dialogue system and the tasks it supports. prior work added high-level control for specific intents such as politeness @xcite , emotions @xcite and persona @xcite through a fixed set of coarse labels, but these methods require manually labelling data for each new intent. one approach for adding control over response intents is to use response exemplars that are handwritten or strategically curated to promote highlevel goals without explicit labels. by condition- ## frame semantics to achieve fluent and contextually-appropriate generated responses that adhere to the semantic structure of exemplars and capture their high-level goals, we use the frame semantics of the exemplars to guide the generation of responses. the central idea of frame semantics is frames, which are semantic abstractions describing universal categories of events, concepts, and relationships, based on the linguistic resource framenet @xcite . frame semantics provide a higher-level representation of individual tokens in the response exemplars based on the purpose of those tokens in the response. for instance, the tokens 'hear', 'say', 'see', 'smell', 'feel', all share a similar purpose of their semantic frame label 'perception', such that each frame can have many possible lexical surface forms. framenet defines more than 1200 frames such as 'perception'. representing response exemplars in terms of their semantic frames allows our model to reuse their semantic structure to adapt the low-level response tokens to fit novel dialogue contexts, and produce diverse response variations that fit within the semantic constraints. for example, in the use of frame semantics to represent exemplars in terms of their semantic meaning rather than their surface forms provides two additional benefits: (1) preserving the semantic structure of exemplars helps to preserve implicit constraints of dialogue systems present in exemplar responses including desired strategies, intents, and emotional tones, and (2) using frames rather than tokens helps the model to avoid overfitting. a model that uses exemplar tokens rather than frames during training can become over-relient on copying tokens, such that during generation the model copies inappropriate tokens from the exemplar response. for example, given the exemplar response "eggs are very beneficial for your body" ( in our experiments, we find that using frame semantics in exemplar-conditioned dialogue generation improves the coherency of responses, while preserving the semantic structure and underlying intents of the exemplar responses. ## model our model edge extends a dialogue generation model transfertransfo @xcite to control generation by including semantic frames from an exemplar response in addition to the dialogue history. transfertransfo is based on the transformer architecture and fine-tunes a generative pretrained model (gpt) @xcite with two objective functions: (1) a language modelling objective, and (2) a next-utterance classification objective. the language modelling objective function maximizes the likelihood for a given sequence of tokens, and the next-utterance classification objective distinguishes a correct response for an input dialogue context from a set of randomly selected distractor responses. we adapt the transfer-transfo model to our setting by first replacing gpt with gpt-2 @xcite as our base architecture. gpt-2 can be substituted with other language models such as transformer-xl @xcite or dialogue specific models such as di-alogpt @xcite . to incorporate semantic frames from exemplar responses in the trans-fertransfo architecture, we uniquely add tokens representing the semantic frames to the input context. specifically, we concatenate the input context, @xmath0, semantic frame tokens, @xmath1, and the response (figure 1 ). prior work also uses concatenation to add different signals to the input for training dialog systems @xcite . following transfertransfo model, we also add token, position, and speaker role embeddings. for frame extraction from exemplars, we use the open-sesame model @xcite and their open-sourced implementation foot_1 . we use the frame predicates and ignore the arguments. because there are no frames corresponding to wh-question words such as 'why' and 'how', 'yes' and 'no', question mark or pronouns, we add each of these tokens in the frame vocabulary. training during training, the model learns to generate the ground truth responses conditioned on the dialogue context tokens followed by the in-order predicted semantic frames for the ground truth response (figure 1 ). following transfertransfo, we mask the tokens of the context for the language modelling objective. to ensure that the model does not ignore the exemplar response, we use the frames of the ground truth response in input during training, instead of frames from a retrieved response. in pilot experiments, our model generated incoherent replies to the dialogue context when the semantic frames were incorrectly detected or irrelevant to the dialogue context. to make the model more robust to missing frames, frames changing order between the exemplar and the response, and irrelevant or inaccurate frames, we: (1) randomly drop 15% of semantic frames from the sequence, (2) randomly shuffle semantic frames sequences (over a length of 2 tokens) with a probability of 0.1, and (3) add random semantic frames in random positions to increase the sequence length by 30%. edge's ability to generate coherent responses despite inaccurate frame detection is important as the semantic frame prediction model that edge uses reports f1 scores of 73.25% for frame tar-get detection and 86.55% for frame identification. however, informal dialogue text can lead to lower performance. evaluating on 110 conversational sentences in the framenet 1.7 test set, the semantic frame prediction model achieves f1 scores of 71.78% for frame target detection and 74.58% for frame identification. we train edge by dropping, reordering and adding random frames so that edge learns to generate coherent responses in the presence of noisy frames from the exemplars. inference during inference, we either rely on predefined response exemplars, or perform retrieval by first using the state-of-the-art poly-encoder model @xcite to retrieve response candidates and then select the highest ranked response as the exemplar response. we add the semantic frame sequence from the exemplar response as the input along with the context of the conversation. the model then creates a response which is controlled by the semantic frames from the exemplar, and coherent with the context of the conversation. ## experimental setup we compared our model to existing generative and retrieval-based approaches in two settings: (1) open-domain dialogue generation using the dailydialog dataset @xcite , and (2) goaloriented anti-scam dialogue generation using a set of fraudulent emails @xcite as prompts and a small set of intent-specific anti-scam response exemplars to inform responses. for the anti-scam domain, we investigated exemplar conditioned responses in a case without domain-specific training (i.e. zero-shot generation). ## results and discussion in this section we report results for both opendomain and goal-oriented anti-scam domains. ## related work edge controls dialogue generation based on semantic frames of exemplars, building on prior retrieval-based, controllable and semantics-based language generation methods. retrieval-based generation has been applied in summarization, machine translation, and paraphrasing @xcite @xcite tasks to improve the quality of text generation or to incorporate knowledge from retrieved text @xcite . in dialogue generation, retrieval conditioned approaches have been proposed to address the lack of diversity in generated responses and the generation of short and dull responses, common in generative approaches. early approaches used lstm-based models @xcite @xcite and their ensembles @xcite to encode tokens of the retrieved responses to condition response generation. conditioning response generation directly on tokens of retrieved responses results in: (1) generating incoherent responses due to copying contextually irrelevant tokens, and (2) models learning to ignore retrieved responses due to a mismatch between retrieved responses and ground truth responses. prior work aimed to solve these problems by extracting only contextually relevant tokens from the retrieved response @xcite , and by replacing the retrieved response with a noisy version during training @xcite . by using semantic frames that represent an exemplar token's meaning rather than the low-level tokens themselves to guide generation, edge exerts better semantic control over the generated response. we additionally achieve higher coherence, fluency, and token-level diversity by reusing semantic frames rather than specific tokens. controllable text generation has been studied in tasks such as dialogue generation @xcite , summarization @xcite , paraphrasing @xcite , and other tasks @xcite , with the aim of controlling fixed attributes such as topic @xcite , emotion @xcite , politeness @xcite and style @xcite through coarse-level labels or control phrases @xcite . some traditional approaches used templates to control the generation of text @xcite . some recent approaches learn templates from the data and exemplars @xcite @xcite . we explore the common case of response exemplars instead of inflexible templates or coarse labels to guide the dialogue response generation. although state-of-the-art models pretrained on large dialogue corpus such as dialogpt @xcite , meena @xcite and blenderbot @xcite are capable of generating interesting and human-like responses, our focus is on controlling the response generation process by conditioning on exemplars. by using semantic frames from exemplar responses, our method flexibly captures intents implicitly present in the exemplar frames, and exercises fine-grained semantic control over generation of new responses based on these exemplars. semantics-based generation has reemerged for use in various tasks such as paraphrasing @xcite , machine translation @xcite and story generation @xcite . semantic representations such as semantic frames and semantic role labels provide abstractions that capture the underlying meanings of different surface realizations (e.g., paraphrases, other languages). we are the first to explicitly model frame semantic representations @xcite in dialogue generation. ## conclusion we present edge, an exemplar-based generative dialogue model. by generating responses that preserve semantic structures from exemplars, edge maintains desired qualities of dialogue systems including intents and strategies implicitly present in the curated exemplar sets, while achieving fluent and coherent responses. in future work, we plan to explore new mechanisms for incorporating semantic frames, experiment with other abstract representations of response exemplars, and apply our approach to other language generation tasks.