Imagination is All You Need! Curved Contrastive Learning for Abstract Sequence Modeling Utilized on Long Short-Term Dialogue Planning (2023)

Abstract: Inspired by the curvature of space-time, we introduce Curved Contrastive Learning (CCL), a novel representation learning technique for learning the relative turn distance between utterance pairs in multi-turn dialogues. The resulting bi-encoder models can guide transformers as a response ranking model towards a goal in a zero-shot fashion by projecting the goal utterance and the corresponding reply candidates into a latent space. Here the cosine similarity indicates the distance/reachability of a candidate utterance toward the corresponding goal. Furthermore, we explore how these forward-entailing language representations can be utilized for assessing the likelihood of sequences by the entailment strength i.e. through the cosine similarity of its individual members (encoded separately) as an emergent property in the curved space. These non-local properties allow us to imagine the likelihood of future patterns in dialogues, specifically by ordering/identifying future goal utterances that are multiple turns away, given a dialogue context. As part of our analysis, we investigate characteristics that make conversations (un)plannable and find strong evidence of planning capability over multiple turns (in 61.56% over 3 turns) in conversations from the DailyDialog dataset. Finally, we show how we achieve higher efficiency in sequence modeling tasks compared to previous work thanks to our relativistic approach, where only the last utterance needs to be encoded and computed during inference.

## introduction large scale transformers are becoming more and more popular in dialogue systems @xcite , @xcite ). though these models are very effective in generating human-like responses in a given context, based on their learning objective to minimize perplexity, they tend to have trouble generating engaging dialogues @xcite . @xcite have shown that human conversations usually do not sample from the most likelihood of words like transformers do. we argue that one reason for this is that natural conversations can be (always) considered goal-oriented (even chitchat) and motivate this claim based on literature from psychology. these have shown that "conversation is a goal-directed process" @xcite ) as humans shift conversation topics based on the social connection/audience and use it to shape social relations @xcite . the psychological literature also elaborates on how humans are able to plan and simulate dialogues by utilizing inner speech as part of verbal working memory @xcite . however, fluctuations between the speaker at the goal turn (saying the utterance later on) and their dialogue partner can be observed. as we see on the blue & red highlighted turns, the goal turn speaker has a greater similarity to the goal utterance than the dialogue partner. we filtered all samples causing these fluctuations and find that these transitive entailing properties are essential for guiding the conversation toward the given goal. regardless of whether the person had the intent to reach the target goal.we demonstrate in this paper how we can build upon this phenomenon to learn the relative distance between utterance pairs. in particular, by mixing the training objective of natural language inference (nli) for the semantic embedding space with a distance proportional and directional aware (through two special tokens [before] & [after]) cosine similarity-based loss of utterance pairs. the resulting curved contrastive learning (ccl) is presented on three tasks: (1) short-term planning, (2) next utterance selection, and (3) longterm planning. furthermore, we investigate two research questions: our work builds upon two major concepts, dialogue planning, and entailment. related publications from the stated fields are discussed below. ## methods in this section, we formally define the research questions (problem definition), our baselines for the evaluation, and the core of imaginary embeddings based on which advanced architectures are built in the following sections. ## next utterance selection with curving motivated by the curved property, the most suitable next utterance u @xmath0. we can assess a relative likelihood between all future utterances by measuring the entailment strength p e (i.e. imagining the closeness) of every u f to the history of utterances based on the cosine similarity as follows: in the ranking evaluation, we sort the results of ∀u @xmath1: p e (u f |his) to determine the rank of the true utterance. notably, we can observe the entailment strength (or activation) of individual utterances to a future one, which enables many other applications. during inference, while the dialogue partner is still speaking, we can precompute the entire context (apart from the new incoming utterance). furthermore, we can utilize the curved context for greedily selecting the next goal max @xmath2. we refer to this as greedy curving. ## long-term planning approaches in this section, we describe how imaginary embeddings can be used to order goals (a set of utterances) within dialogues for long-term planning. the models are evaluated with lstpe, a given set of goals g with |g| = 3, and an equal distance between each node. imaginary embeddings are perfectly suited for this task as they can be concatenated into cosine similarity chains by using the ([b] before and [a] after token) as illustrated in figure 4 . we mathematically define it as: ## experiments our experiments are conducted on two dialogue corpora, dailydialog @xcite and the microsoft dialogue challenge (mdc) corpus @xcite . we experiment with two transformer architectures bert @xcite and roberta @xcite to generate imaginary embeddings. in the short-term planning (transformer guidance) setting, we let our imaginary embeddings guide dialogpt @xcite for dailydialog and godel @xcite for the mdc corpus. for the next utterance selection, we use pre-trained checkpoints of dialo-grpt @xcite and convrt @xcite as baselines. furthermore, we add bm25 @xcite as well as an ablation study with the two special tokens (before and after) but without the curved learning objective that we explore in the appendix g. ## conclusion in this paper, we introduced curved contrastive learning, a novel technique for generating forwardentailing language embeddings. we demonstrated that these can be utilized on various sequence modeling tasks by only using the cosine similarity between the separately encoded sequence members in the curved space. in particular, for the next utterance selection by imagining the closeness of every context utterance to candidate utterances in the curved space (where dailydialog's true utterances are constantly in the top 10%), outperforming our pre-trained baselines dialogrpt and convrt on sequences longer than 2 turns while reducing encoding costs. furthermore, we have shown their pattern recognition ability on the ordering/identification of future representations (with an average rank of 1.87/6) even at longer distances and far apart. we also demonstrated that these embeddings can be applied to guiding dialogue transformers to approach a goal over multiple turns. in particular, by imagining the closeness of candidate utterances towards the goal through the transitive properties of the curved space. following up on our claim, that even chit-chat can be considered goal-oriented (rq1), we find strong evidence of planning capability in chit-chat conversations over