We Are What We Repeatedly Do: Inducing and Deploying Habitual Schemas in Persona-Based Responses
Authors: Benjamin Kane
Lenhart K. Schubert
Venue: EMNLP 2023 Main (2023)
Keywords: Dialogue
Response
Generation
Persona
Schema
LLM
TL;DR: We propose and evaluate a method for generating interesting persona-based responses using LLMs by first inducing symbolic event schemas.

Abstract: Many practical applications of dialogue technology require the generation of responses according to a particular developer-specified persona. While a variety of personas can be elicited from recent large language models, the opaqueness and unpredictability of these models make it desirable to be able to specify personas in an explicit form. In previous work, personas have typically been represented as sets of one-off pieces of self-knowledge that are retrieved by the dialogue system for use in generation. However, in realistic human conversations, personas are often revealed through story-like narratives that involve rich habitual knowledge -- knowledge about kinds of events that an agent often participates in (e.g., work activities, hobbies, sporting activities, favorite entertainments, etc.), including typical goals, sub-events, preconditions, and postconditions of those events. We capture such habitual knowledge using an explicit schema representation, and propose an approach to dialogue generation that retrieves relevant schemas to condition a large language model to generate persona-based responses. Furthermore, we demonstrate a method for bootstrapping the creation of such schemas by first generating generic passages from a set of simple facts, and then inducing schemas from the generated passages.

We Are What We Repeatedly Do:
Inducing and Deploying Habitual Schemas in Persona-Based Responses
Benjamin Kane
Lenhart Schubert
University of Rochester bkane2@ur. wwwhester.cco.. edu
University of Rochester
schuber rochester..... edu
Abstract
Many practical applications of dialogue tech- nology require the generation of responses according to particular developer-specinee persona. While variety of personas can be elicited from recent large language models, the opaqueness and unpredictability of these mod- els make desirable be able to specify per- sonas in an explicit form. In previous work, personas have typically been represented as of one-off pieces of self-knowledge that are retrieved by the dialogue system for use in generation. However, in realistic human con- versations, personas often revealed through story-like narratives that involve rich habitual knowledge- knowledge about kinds of events that agent often participates in (e.g.. g.. work ac- tivities, hobbies, sporting activities, favorite en- tertainments, etc.), including typical goals, sub- events, preconditions, and postconditions of those events. We capture such habitual knowl- edge using an explicit schema representation, and propose an approach to dialogue genera- tion that retrieves relevant schemas to condition a large language model to generate persona- based responses. Furthermore, we demonstrate a method for bootstrapping the creation of such schemas by first generating generic passages from a set of simple facts, and then inducing schemas from the generated passages.
Persona
Sample generic passages
induction
Schema
theader
Hey there! Recently Ivo been enjoying cooking. Do you oook?
grancioofi ofher uaud
upreconda
That's cooll profer baking.
gianorother for
Retrieval
repladdes
In-context knowledge
grandr pull oven"
"Mn insiiiibee anomo worts throupe my housw"
That sounde great! My grandmather usnd in like bake piez on Saturcays. She would gather the ingredients herseff. it would kd alwaye the house with an imserdittd aromes.
Figure 1: A diagram our approach. (1) Given an unstructured persona dataset, we first sample "generic passages" from the facts in the persona, and then induce structured event schemas from the sampled stories. (2) We condition an LLM to generate dialogue responses that are fluent with previous conversation yet that make use of the rich knowledge contained in the result- ing schemas - by first using a retrieval model to select a relevant schema, and then providing the schema to the LLM as in-context knowledge.
1 Introduction
Virtual conversational agents simulated humans that can engage in conversation with a human user present a major application of dialogue technol- ogy. Such systems have been deployed for diverse uses including conversational coaches, chatbots for entertainment, and customer service bots. A critical, yet challenging, problem in designing con- versational agents is endowing them with a specific persona, and generating responses that are both natural and consistent with this persona. Systems that are able to do this are both found to be more en- gaging by users (Zhang et al., 2018), 1)), and increase
the level of confidence and trust that users place in the system (Shum et al., 2018). Furthermore, in many practical applications beyond chit-chat, there is complementary need to control the flow of dialogue; for example, ensuring consistency of gen- erated responses with hand-engineered templates may help to improve a dialogue system's 's topical coherence (Grassi et al., 2021).
Seminal conversational systems such as ELIZA (Weizenbaum, 1966) operated on the basis of symbolic knowledge, allowing directly for per- sona development through the manipulation of ex-

---

plicit rules. However, this dependence on explic- itly coded knowledge also rendered such systems whowledgeiimpoveiishee
that conditions an LLM generate narrative-like responses consistent with these schemas through in-context prompting'. Furthermore, since it of- ten desirable for dialogue designers to be able to specify a persona using a small number of simple natural language facts, we propose a method for bootstrapping the creation of schemas from set of simple facts. This method involves leveraging LLMs first generate "generic passages" from the given facts, and then induce structured ha- bitual schemas from the passages - capturing both explicit steps from the passage and implicit knowl- edge associated with the event described by the passage. A high-level diagram of our approach is shown in Figure/ 1. We present evaluation results showing that the generated schemas are generally high quality, and can be used condition LLMs to generate responses that are more diverse and engaging, yet also controllable.
and unable to make ob- vious inferences. Decades of AI research, aimed at solving this problem, have culminated in the cre- ation of large language models (LLMs), and the emergence of in-context learning i.e., steering the LLM towards particular behavior by includ- ing knowledge examples in the natural language prompt (Brown et al., 2020). Recent work has found that systems that leverage in in-context learning with LLMs outperform those that fine-tune smaller language models on conversational data (Madotto et al., 2021; Zheng and Huang, 2022).
While a variety of personas be elicited from the information implicit in the weights of LLMs,, the resulting personas are often unpredictable, opaque to dialogue designers, and prone to hal- lucination (Lim et al., 2022). Therefore, much recent work has focused on representing personas and knowledge explicitly, in manner that can be leveraged by LLMs for generation using retrieval- in-the-loop methods (Shuster et al., 2021). Typi- cally, these upproaches represent personas using structured sets of natural language "facts" about an agent, possibly augmented with additional knowl- edge from a knowledge base.
2 Related Work
2.1 Persona-Based Dialogue Generation
Many past systems have attempted to integrate ex- plicit customizable persona profiles with statistical response generation techniques; one of the earliest such systems was NPCEditor (Leuski and Traum, 2010), which used information retrieval (IR) to re- trieve a hand-designed response from a persona, but was limited to question answering dialogues. Attempts to make persona based generation more general and robust were initially based on encoding personas as single vector in sequeence-to-sequeece architectures (Li al., 2016b; Kottur et al., 20.7).
In casual human-human dialoguc, however, per- sonas are often revealed through story-like narra- tives about experiences rather than one-off facts (Dunbar al., 1997). For example, if speaker mentions something involving sports, the interlocu- tor might respond by relating their typical experi- ences playing sport in the past. These types of marratives, typically taking the form of "generic passages" (Carlson and Spejewski, 1997), often capture habitual knowledge knowledge about the kinds of events that an agent participates in, or used to participate in. This knowledge includes the typical steps of habitual event, as well as the typi- cal goals. preconditions, and postconditions of the event. Originating from early research in artificial intelligence, event schemas have been proposed as a structured representation of the rich types of pro- totyical knowledge associated with generic and habitual events, such as causal and enabling rela- tions, temporal relations, etc. (Chambers, 2013; Lawley et al., 2021; Li et al., 1021).
More recently, efforts have focused on making use of personas more directly: Zhang et al. (2018) 18) crowd-sourced large dataset of persona-based dialogues in which personas unstructured sets of natural language facts, and created a Seq2Seq model that uses IR retrieve relevant persona facts as input. Subsequent stud- ics built upon this approach using different models and extended persona datasets (Mazaré et al., 2018; Qian et al., 2018: Madotto et al., 2019; Zheng et al., 2019; Su et al., 2019; Salemi et al., 2023). How- ever, while such approaches are effective at mak- ing responses conform to particular persona, the generated responses are often shallow due the simplicity of the persona representation.
represented
In this paper, we propose a novel approach to di- alogue generation that uses collection of explicit event schemas to augment an agent's persona, and
'Code can be found at https://gthhu.cco///aaaaahhaaanh

---

Some studies have shown that adopting richer persona representations that blend persona informa- tion with general world knowledge lead to more in- teresting and consistent responses (Majumder et al., 2020; Lim et al., 2022; Oh and Kim, 2022). Along these lines, Majumder et al. (2021) demonstrated that by sampling background stories relevant to re- trieved persona facts from external story corpus, conversational model could generate responses that are more diverse and engaging than by using the persona alone. However, since the story corpus in this work was unrelated to the personas, there is a risk that a selected story may not fully cohere with the given persona. Moreover, using the story directly may leave out knowledge that is implicit but not necessarily expressed in the story, such as the underlying goals of participants. We build on this work by considering each story latent step in inducing schema that contains both implicit and explicit knowledge associated with the story.
is relevant using multi-level information retrieval system to embed both the event schemas (treated as individual documents) and the knowledge con- tained within each event schema (treated as collec- tions of documents), and to rank the schemas S based on similarity to the embedding for un-1- Following (Zheng and Huang, 2022), we employ prompting-base approach in which a pre-trained LLM used to produce a response utterance, pro- vided a prompt that is dynamically constructed from the dialogue history and the selected schemas.
3.1 Schema Induction
Since structured event schemas for habitual activi- ties are typically expensive for dialogue designers to create, requiring reasoning about causal rela- tions and other implicit knowledge, we focus on the problem of automatically inducing event schemas from an unstructured persona P (pi, ). where pi are natural language "facts" such as "I like to play tennis." "2, Formally, we represent an event schema as a tuple (H, S,, G, is a schema header; a sentence characterizing the over- all schema event. Pr, S, and Po are sets containing schema preconditions, static conditions (conditions expected to hold throughout the overall event), and postconditions, respectively. G is set containing typical goals of participants of the event, and E is set containing typical episodes (i.e., substeps) of the event. We show an example of an event schema in Figure 2.
2.2 Deriving Symbolic Knowledge from Large Language Models
The framework of deriving explicit knowledge from LLMs has been explored in other work, though primarily in the context of IR and com- monsense reasoning systems rather than persona- based dialogue generation. West et al. (2022) show that implicit knowledge within an LLM can be dis- tilled into a symbolic commonsense knowledge graph using prompt engineering techniques. Other work focuses specifically on the problem of event schema induction using neuro-symbolic pipeline (Lawley and Schubert, 2022) or zero-shot incre- mental prompting techniques (Dror et al., 2023; Sha, 2020). However, these studies focused on the induction of generic event knowledge (e.g.. the steps typically taken to plan a wedding), rather than the habitual event knowledge implicit in a specific persona (c.g., a persona's typical experiences when attending weddings in the past).
In order to generate sufficiently interesting and accurate schemas, we employ the method of latent schema sampling (LSS) introduced in (Lawley and Schubert, 2022) this method regards an LLM, when conditioned on a schema header, as implicitly characterizing a distribution over stories sampled from that distribution. A full schema can then be induced from the sampled stories.
Thus, for each p; E P, we sample N, stories (specifically, generic passages (Carlson and Spe- jewski, 1997) describing the typical process of habitual event) using the GFI-3.5-TURBO LLM
3 Method
We use few-shot prompt in which the LLM is supplied with a short definition of a generic pas- sage, followed by K, examples. In contrast to the ncuro-symbolic pipeline in (Lawlcy and Schubert,
Given a dialogue context 21 = {ur, u2, .... un-1) containing system and user utterances, our goal is to generate response un that utilizes knowl- edge from relevant event schema S E S {S1, S2. ..., } this schema represents knowl- edge about a habitual activity that is part of the speaker's 's persona and that is relevant to the previ- ous turn 1. 1- We ensure that the selected schema
"These facts may hand-desi by dialogue designer, crowdsourced (as (Zhang et al.. 2018)), or generated by an LLM.
pattps://plttfoot httpp//tttoo.oppaaai....mm wwpeaai.co//oeeeeeeooooooo comm/docs/mddels/ overview

---

hhaaeer "I work in bookstore."
:precondi tions "My shift has started."
"static-conditions (
"The bookstore stocked with books."
"Customers visit the bookstore." "I am knowledgable about books customer service."
: :psstconditions (
"My shift at the bookstore is over." "Some customers have purchased books."
:goals
"My goal is to assist customers in finding the books they are looking for."
"The customers' goal is to find the books they want to purchase."
:episodss
"Customers come looking for new titles to add to their collection, to browse." "I welcome the customers and ask if they need any assistance."
"I help the customers find books by using knowledge of the store inventory." [...]
"I organize the bookshelves when the customers not in the store."
Figure 2: An example of an event schema for habitual "work bookstore" activity. Note that some episodes are omitted for brevity.
2022), we leverage the in-context learning capa- bilities of GPT-3.5-TURBO to directly induce an event schema from a set of Np passages, given an abstract schema template and K2 in-context exam- ples See Appendix B.2 for our specific prompts, and Appendix C.2 for additional examples of gen- erated schemas.
tice, the mode of generation may be mediated by dialogue manager that manages the conversation flow and provides "raw" utterances (which may, for instance, be programmed by dialogue designers) to be selected for paraphrasing. For the purposes of this paper, we assume that, in the case of paraphrase generation, we have raw utterances available.
3.2 Dialogue Generation
3.2.1 Schema Retrieval
We use the GPT-3.5-TURRO LIM to generate flu- ent responses, conditioned on prompt containing a subset the knowledge contained within re- trieved schema. Additionally, in order to allow controllable dialogue flow management which is necessary for usability in many applied domains (Grassi et al., 2021) allow for modes of generation:
As a first step in constructing prompt, we use a multi-level retrieval system that uses pre- trained Sentence Transformer model (Reimers and Gurevych, 2019) to embed and retrieve relevant schema knowledge. We pre-compute embeddings for each schema, well as for each fact within each schema. For each dialogue turn, we also com- pute embedding of the previous utterance U, 1
unconstrained generation, in which the LLM is prompted with the entire dialogue history and generates the next utterance without any con- straints (apart from the retrieved knowledge); and few-shot paraphrase generation, where the L.MM is prompted with a given sentence to paraphrase along with several in-context examples. In prac-
es. T(S)) eS..J, T(f))
VS, E S Vf, E S, S
C = ((un-))
'In practice, we found Np = 1, Kp = 2 and K, = he sufficient to produce accurate generations.
"httsss//huggingfaaeecc ceneeeeeeeeeeeeeeeeeeeneeeeeeeee www.enneeettaafforerrerrmmm vil

---

Here is Sentence Transformer model, S is the full of schemas, fy E St is the full set of facts contained within each section of schema S,. We then retrieve the single schema most rele- vant to Un using a cosine similarity measure, and score the facts within that schema based on com- puted similarity:
BASE
UNCS
PARA
Base Persona
Dialogue History
Event Schema
X
/
/
Raw Response
X
X /
Table I: A summary of the differences in the resources available to each method that we compare in our evalua- tions. Note that each method in the order presented has access to all resources available to the previous method.
score(J,) = sim .fr 1 S argmax sim:
ff E S
es, Cun-1)
S.ES
é1 e2
4 Experiments
sim e1, 62) = eei e2
We first evaluate our response generation method according to the following desiderata: (1) the gen- erated responses improve diversity of output; (2) the generated responses engaging, interesting, and relevant given the previous conversation, and (3) the generated responses are controllable; i.c., dialogue designer can ensure that the responses still correctly express an intended response. The specific hyperparameter values that we use for our experiment are shown in Appendix A.
The top N facts according to score are re- trieved to be used in the prompt.
3.2.2 Unconstrained Generation
In the case of unconstrained generation, we sample a response from the LLM by prompting it with the full dialogue history, after conditioning facts from the relevant habitual schema and the current dialogue schema:
Since important advantage of our approach is the reusability of the generated schemas for down- stream tasks (e.g., for inferring additional facts from dialogue agent's experiences), we also con- duct an evaluation of the quality of the generated schemas - specifically, whether the facts within the schema correctly represent typical knowledge associated with the event that the schema describes.
UA, LLM ++ FD + U),
where FR = {fI,...,Nyy C SR are the relevant facts retrieved in the previous step, FD SD E(SD) are all non-episodic facts from the current dialogue schema (i.e., preconditions, goals, etc.), and U [M1, "n-1} is the dialogue history.
4.0.1 Dataset
3.2.3 Few-shot Paraphrase Generation
We conduct our experiment using the PersonaChat dialog dataset? (Zhang et al., 2018). We gener- ate schemas and evaluate the performance of our response generation method using the test split, containing of 131,438 unique utterances. When evaluating paraphrase generation method, we use the gold response annotations from the Per- sonaChat dataset for the raw utterances that are input the model.
the case paraphrase generation, we employ a sew-shot prompting strategy to condition the LLM to paraphrase the given sentence in a manner that is interesting, appropriate, and that makes use of the relevant facts. Specifically, in addition to the inputs used in the unconstrained setting, we format several in-context paraphrase examples along with "raw" utterance to paraphrase, given the actual dialogue context:
4.0.2 Baselines
Un LFM + FD 8 u in),
We consider two baselines for evaluating the perfor- mance of our approach: First, use the GPT-3.5- TURBO LLM without schema retrieval, provided only with the base persona and dialogue history the prompt (BASE). The specific baseline prompt is shown in Appendix B.2. Second, we consider
where in is the sentence to paraphrase, and E is set of Ke in-context examples: E {(21) á] (..),.., "ukk,,,k ; =)
Examples of both types of prompts can be found in Appendix B.2.
"Excluding the schema header, which is always included in the prompt.
hhttps://hugginggfacs co/ /tdddaattt/baarrrd carsonachatt_truccsss

---

the human-generated gold utterances from the Per- sonaChat dataset themselves (GOLD) as a baseline for our diversity, engagement, and relevancy met- rics. Against these, we compare our two generation methods: unconstrained generation UNCS and para- phrase generation PARA. The differences between the three generation methods are summarized in Table I for reference.
the baseline methods that do not have access to the schemas, as measured by D-2 and ENTR (although D-1 tends to favor the methods that generate re- sponses that are shorter and therefore have a higher relative fraction of distinct uni-grams). Further- more, we observe that the paraphrase generation method achieves considerably higher similarity to the gold responses than both the baseline and un- constrained methods (which perform comparably well on this metrie).
4.1 Response Generation Evaluation
4.1.1 Automatic Evaluation
4.1.2 Human Evaluation
Following prior work (Majumder al., 2021; Li et al., 2016a), we use several methods to measure the diversity of the generated outputs, per desider- atum (1). First, we compute the mean percentage of unigrams and bigrams in the generated outputs that are distinct relative to the total number of gen- erated words, reported D-1 and D-2 respectively. We also report the mean lengths of the outputs as Length. Since the distinct n n-gram measures do not represent the actual frequency distributions of words (and will tend to be penalized with longer responses), we also report the mean ENTR score across outputs calculated as the geometric mean of entropy values of n-gram frequency distributions, for n é 21.
To assess desideratum (2), we conduct a human evaluation of 100 randomly sampled examples metrics associated with response quality, following prior work (Majumder et al., 2021) namely, whether the generated responses are en- gaging and relevant given the dialogue context. Annotators are tasked to make a pairwise compar- ison between responses from a pair of generation methods. We first collect annotations comparing the two baseline methods; under the assumption of transitive preferences, then use the "winning" baseline a comparison for each proposed method, We hired two Anglophone annotators for every sample; further details of our evaluation setup, in- cluding a screen capture of the task, are shown Appendix B.
In order to test the controllability of our para- phrase generation method against other baselines, per desideratum (3), we also report several text sim- ilarity methods computed between a generated out- put and the gold PersonaChat response. We report widely-used ggram-based similarity metrics such BLEU, ROUGE-L, and METEOR, as well as the cosine similarity between contextualized em- beddings produced by the ALL-DISTILROBERTA v1 Sentence Transformer model (Reimers and Gurevych, 2019) (ST). However, since not all sen- tences in a generated response may be directly related to the gold response (e.g., an acceptable paraphrase may consist of story followed by the intended response), it is difficult to interpret these metries the level of the full response. Hence, we compute the maximum pairwise similarity for each full sentence between the generated and gold responses, and report the average value across all responses.
Our results are shown in Table 2, with starred values indicating differences that are significant with p 0.05, using non-parametric bootstrap tests 2000 subsets of size 50. The collected annotations are fairly noisy, with inter-annotator agreement (Krippendorff"s alpha) being 0.21 and 0.23 for "engaging" and "relevant", respectively.
Despite this, we were able to observe moder- atc and statistically sigmificant preferences for both the paraphrase and unconstrained methods over the LLM baseline in terms of engagement, and for the paraphrase method over the baseline in terms of relevancy. The LLM baseline itself was, in turn, sig- nificantly preferred over the gold responses for both questions. We believe that this can be attributed to the relatively short length and low diversity of language of the gold responses (as indicated in Table 3), as well as the ability of LLMs to interpo- late smoothly with conversation history, even when constrained by our proposed methods.
These results are shown in Table 3. We observe that the methods that use event schemas for genera- tion generate responses with higher diversity than
We note, however, that many annotators were indifferent between the different generation meth- ods. This is plausibly due to the fact that, gener- ally, multiple response strategies are considered
"Split hased on "M, "?", and "I" punctuation, filtering out sentences less than 5 words in length.

---

PARA Vs.
UNCS
BASE
UNCS BASE
BASE GOLD
Metric win
loss
win
loss
win
loss
win
loss
Engaging
34.7
27.4
46.8*
21.1
39.5
24.2
43.0*
23.0
Relevant
33.2
23.7
44.7*
24.2
37.4
22.6
40.5*
25.0
Table 2: Pairwise comparisons between responses generated from each method (% win/loss, leaving ties out). Entries with are statistically significant with p < 0.05 using a non size 50. Additional details of the human evaluation in Appendix B.
non- parametric bootstrap test on 2000 subsets of
Method
GOLD
BASE
UNCS
PARA
Persona:
Diversity
I do not care about fashion.
LEnGTh
50.1
122
303
372
I work in bookstore.
D-1
97.0
93.8
81.7
78.9
D-2
88.9
94.2
96.0
96.7
ENTR
2.20
2.91
3.61
3,84
User: worked at Petsmart for two years and then got canned.
Controllability
BLEU
1.25
.843
8.60
GOLD: Ah too bad! work at a barnes and noble. Chemistry grad.
ROUGE-L 19.3
19.8
34.6
METEOR
14.6
16.5
33.2
BASE: Oh really? Why did you get fired?
ST
35.6
35.0
55.6
UNCS: Oh no, sorry to hear that. But hey, maybe your knowledge of pets could come in handy if you're interested finding some books on the subject. As someone who works at a bookstore, I'm pretty knowledgeable about different books [...]
Table 3: Diversity and controllability metries on the PersonaChat test set. D-1/2 are the % of distinct uni- and bi-grams; ENTR the geometric mean n- n-gram entropy. BLEU, ROUGE-L, and METEOR n-gram-based similarity metrics, and ST is the Sentence Transformer similarity measure. All similarities are calculated average maximum pairwise values across sentences in each response. Best scores are bolded.
standard
PARA: That's tough. I actually work at a barnes and noble bookstore. Although, didn't 't major in English or literature. gradu- ated from college with chemistry degree. When I'm not shelving books, I help cus- tomers find what they're looking for L...]
acceptable for the open-endee conversations in the PersonaChat dataset, and attests to the capability of LLMs to generate suitably engaging and relevant responses across prompting strategies.
Table 4: Sample generations by different methods, using the schema from Figure 2. Content in the gold utterance preserved by the paraphrase method are highlighted red. See Appendix C.2 for additional examples.
4.2 Schema Evaluation
We evaluate the quality of the schemas, in them- selves, through another human evaluation. We ran- domly select subset of 200 individual schema facts from all generated schemas, each paired with the header of the schema was taken from. An equal number of facts are selected for each type of schema relation. As a baseline, we select an- other 200 facts from the generated schemas, but ran- domly swap schema headers so that facts are paired with headers from unrelated schemas. For each type of schema relation, given a fact of that type and schema header, we hire two Anglophone an- notators rate, on a 5-point Likert scale, how typi- cal the fact of an event described by the schema header. For instance, for "iiaticcoondttion fact,
an annotator might be asked "How typical is it that Sentence 2 is true throughout the duration of the event in Sentence 1?". Our full list of questions. and additional details of our evaluation setup, shown in Appendix B.
The mean Likert ratings for the baseline and the generated schemas are shown in Table 5. All differences are significant with < 0.05 using a Mann Whitney U test. We observe that the gener- ated schemas are generally found contain facts that are typical of the described event, relative to the randomized baseline. The smallest typicality

---

Relation
BASE
SCHEMA
can be used produce more engaging responses, it sufficient often, conversations focus around more specific experiences and memories, and the knowledge captured by schemas generated with our approach can be somewhat banal. In future work, we aim to extend our approach to generate schemas that capture atypical aspects of an agent's experience with particular kind of event, as well as more ordinary memories knowledge. We also aim incorporate this response generation mechanism into a broader dialogue management framework that allows for a higher-level decision about whether responding using a habitual schema is appropriate given a particular context.
PRECONDITIONS
2.51
3.65
STATIC-CONDITIONS
2.89
3.74
POSTCONDITIONS
2.93
3.23
GOALS
2.99
3.55
EPISODES
2.80
3.36
ALL
2.82
3.50
Table 5: Mean Likert ratings for the baseline and gen- erated schemas; both the aggregate value and disaggre- gated values across schema relation types are shown. All differences are statistically significant with < 0.05.
differences were observed for the "postcondition" relation, suggesting that inferences of this type may be more complex than other schema relations.
Limitations
4.3 Qualitative Analysis
We acknowledge several limitations of our pro- posed approach. First, given that our approach relies the of LLMs in both schema induc- tion and response generation, it is limited by the inherent tendency of LLMs hallucinate informa- tion (Ji et al., 2023). Although in our qualitative analysis we did not encounter many instances the LLM fabricating wholly false information. this tendency presented itself in more subtle ways par- ticularly in the paraphrase model due to the com- plexity of the prompt. For example, if the sentence to paraphrase contains first person pronoun, the LLM occasionally might reverse the pronoun in the generated response, falsely attributing some fact to the user instead. In some cases may ignore the sentence to paraphrase altogether.
Table 4 shows generated responses from different methods for particular persona and context. Qual- itatively, we observe that the two models that are conditioned on the habitual schema from Figure 2 are able to generate longer and more detailed responses, making use of generic knowledge such as that people who work at bookstores can gener- ally help customers find books of interest. On the other hand, the baseline model tends to generate re- sponses that are fairly short and open-ended". Fur- thermore, we observe that the paraphrase method is more frequently able preserve the meaning of the intended raw utterance, as indicated.
5 Conclusion and Future Work
Second, although our method succeeds at gen- erating more diverse, and engaging responses, this can often be inappropriate in certain conversational contexts, such as in scenario that calls for a short affective response from the agent rather than a lengthy narrative-like response. Moreover, such responses may become repetitive over the course of a full conversation. Our approach would likely need to be integrated into broader dialogue man- ager architecture in order to be usable in practice.
In this work, we demonstrated that habitual knowl- edge in the form of explicit event schema represen- tations could be used to condition LLMs to gener- ate more diverse and engaging dialogue responses. We experimented with generation settings, one of which furthermore allows for greater degree of controllability by dialogue designer who may wish to provide intended utterances for the LLM to paraphrase. Morcover, to case the burden of schema design, we proposed a novel method of inducing schemas from a base persona using an LLM through sampling "generic passages" about habitual activities.
Third, the inference time and costs associated with LLMs (see Appendix B for estimates from our experiments) may make it difficult to use this approach at scale, or to generate schemas in an online manner.
Although the inclusion of habitual knowledge
"One important caveat is that this behavior is not necessar- ily undesirable; short open-ended questions can often be used in conversation to demonstrate interest or empathy towards the interlocutor, although gh in this paper we are focused on the challenge of generating more engaging responses.
Finally, we note that our experiments were lim- ited to the English language; the performance of our approach may degrade if applied to lower- resource languages.

---

Ethics Statement
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang. Andrea Madotto, and Pascale Fung. 2023. Survey of halluci- nation in natural language generation. ACM Comput. Surv, 551))).
Prior work has found that stories generated by LLMs can reinforce potentially harmful social bi- ases (Lucy and Bamman, 2021). Since our work involves the use of LLMs to generate story-like pas- sages intermediate step in deriving schemas, the resulting schemas would likely need to be care- fully vetted to ensure that they do not contain harm- ful information. Furthermore, due to the possi- bility of hallucination described in Section 5, our approach should not be used in high-impact appli- cations where failure to correctly paraphrase an intended response may incur heavy costs.
Satwik Kottur, Xiaoyu Wang, and Vitor Carvalho, 2017. Exploring personalized neural conversational mod- els. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pages 772843734.
Lane Lawley, Benjamin Kuehnert, and Lenhart Schu- bert. 2021. Learning general event schemas with episodic logic. In Proceedings of the 1st and 2nd Workshops on Natural Logic Meets Machine Learn- ing (NALOMA), pages 1-6, Groningen, the Nether- lands (online). Association for Computational Lin- guistics.
Acknowledgments
This work was supported by DARPA under grant 191N1111100022 and NSF Award 17503000
Lane Lawley and Lenhart Schubert. 2022. Mining logi- cal event schemas from pre- rre-trained language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, pages 332-345, Dublin, Ireland. Association for Computational Linguisties.
References
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler. Jeffrey Wu. Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz. Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Ad- vances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc.
Anton Leuski and David Traum. 2010. NPCEditor: A tool for building question-answering characters. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC' 10), Valletta, Malta. European Language Resources Asso- ciation (ELRA).
Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan, 2016a. jective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computa- rional Linguistics: Human Language Technologies, pages 110-119, San Diego, California. Association for Computational Linguistics.
ob-
Greg N. Carlson and Beverly Spejewski. 1997. Generic passages. Natural Language Semantics, 5(2):101- 165.
Jiwei Li, Michel Galley, Chris Brockett, Georgios Sp- ithourakis, Jianfeng Gao, and Bill Dolan. 2016b. A persona- based neural conversation model. In Pro- ceedings the 54th Annual Meeting the Associa- tion for Computational Linguistics (Volume Long Papers), pages 994-1003, Berlin, Germany. Associa- tion for Computational Linguistics.
Nathanael Chambers. 2013. Event schema induction with a probabilistic entity-driven model. In Proceed- ings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1797-1807.
Rotem Dror, Haoyu Wang, and Dan Roth. 2023. Zero- shot on-the-fly fly event schema induction. In Findings of the Association for Computational Linguistics: EACL. 2023, pages 705-725, Dubrovnik, Croatia. As- sociation for Computational Linguistics.
Manling Li, Sha Li, Zhenhailong Wang. Lifu Huang, Kyunghyun Cho, Heng Ji. Jiawei Han, and Clare Voss. 2021. The future is not one-dimensional: Com- plex event schema induction by graph modeling for event prediction. In Proceedings of the 2021 Confer- ence on Empirical Methods in Natural Language Pro- cessing, pages 5203-5215, 15, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Robin Dunbar, Anna Marriott, and Neill Duncan. 1997.
Human conversational behavior. Human nature (Hawthorne, N.Y.J. 8:231-246.
Lucrezia Grassi, Carmine Tommaso Recchiuto, and An- dia-
Jungwoo Lim, Myunghoon Kang, Yuna Hur, Se- ung Won Jeong, Jinsung Kim, Yoonna Jang, Dongyub Lee, Hyesung Ji, DongHoon Shin, Seun- gryong Kim, and Heuiseok Lim. 2022. You truly un- derstand what need: : Intellectual and friendly dialog
tonio Sgorbissa. 2021. Knowledge-groundee logue flow management for social robots and con- versational agents. International Journal of Social Robotics, 4:33 1293.

---

agents grounding persona and knowledge. In Find- ings the Association for Computational Linguisties: EMNLP 2022, pages 1053-1066, Abu Dhabi, United Arab Emirates, Association for Computational Lin- guistics.
Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. 2023. Lamp: When large lan- guage models meet personalization.
Lei Sha. 2020. Gradient-guided unsupervised lexically constrained text generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages Online. Association Computational Linguistics.
Li Lucy and David Bamman. 2021. Gender and repre- sentation bias in gpt-3 generated stories. In Proceed- ings of the Third Workshop on Narrative Understand- ing. pages 48-55.
Harry Shum, Xiaodong He, and Di Li. 2018. From cliza to xiaoice: challenges and opportunities with social chatbots. Frontiers Information Technology & Electronic Engineering, 19:10-26.
Andrea Madotto, Zhaojiang Lin, Genta Indra Winata, and Pascale Fung. 2021. Few-shot hot: Prompt-based learning for dialogue systems.
Andrea Madotto, Zhaojiang Lin, Chien-Sheng Wu, and Pascale Fung. 2019. Personalizing dialogue agents via meta-learning. In Proceedings of the 57th An- nual Meeting of the Association for Computational Linguistics, pages 5454-540 1444-,559,,5 Florence, Italy. Asso- ciation for Computational Linguistics.
Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston, 2021. Retrieval augmentation reduces hallucination in conversation. In Findings of the Association for Computational Linguistics: EMNLP 2021. pages 3784-3803, Punta Cana, Do- minican Republic. Association for Computational Linguisties.
Bodhisattwa Prasad Majumder, Taylor Berg- Kirkpatrick, Julian J. McAuley, and Harsh Jhamtani. 2021. Unsupervised enrichment of persona-grounded dialog with background stories. In ACI.
Teng-Guang Su, Aliyah Hsu, Yi-Lin Tuan, and Hung-yi Lec. 2019. Personalized dialogue response genera- tion learned from monologues. pages 4160-4164.
Joseph Weizenbaum. 1966. Eliza computer program for the study of natural language communication be- tween man and machine. Commun. ACM, 911:336 45.
Bodhisattwa Prasad Majumder, Harsh Jhamtani, Tay- lor Berg- Kirkpatrrk,, and Julian McAuley. 2020. Like hiking? you probably enjoy nature: Persona- grounded dialog with commonsense expansions. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pages 9194-9206, Online. Association for Computa- tional Linguistics.
Peter West, Chandra Bhagavatula, Jack Hessel, Jena Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. 2022. Symbolic knowledge distillation: from general language mod- els to commonsense models. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computationl Linguistics: Hu- man Language Technologies, pages 4602-4625, Seat- tle. United States. Association for Computational Linguistics.
Pierre Emmenuelmnn Mazaré, Samuel Humeau, Martin Rai- son, and Antoine Bordes. 2018 Training millions of personalized dialogue agents. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, pages 2775-2779, Brussels, Belgium. Association for Computational Linguistics.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Per- sonalizing dialogue agents: have dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics (Volume 1: Long Papers), pages 2204-20 221-2213. Melbourne, Australia. Association for Computational Linguistics.
Min Sik Oh and Min Sang Kim. 2022. Persona- knowledge dialogue multi-context retrieval and en- hanced decoding methods.
Qiao Qian, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. 2018. Assigning personality/pro- file to a chatting machine for coherent conversation generation. In Proceedings of the -eeentySeeenth International Joint Conference on Artificial Intel- ligence, IJCAI-18, pages 4279-4285. International Joint Conferences on Artificial Intelligence Organi- zation.
Chujic Zheng and Minlic Huang. 2022. Exploring prompt-based few-shot learning for grounded dialog generation.
Yinhe Zheng, Guan Yi Chen, Minlie Huang, Song Liu, and Xuan Zhu. 2019. Persona-aware dialogue gener- ation with enriched profile.
Nils Reimers and Iryna Gurevych. 2019. Sentence- BERT: Sentence embeddings using Siamese BERT- networks. In Proceedings of the 2019 Conference on Empirical Methods Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing (EMNLP-IJCNLP), pages 3982-3992 3992, Hong Kong. China, Association for Com- putational Linguistics.

---

A Implementation Details
fact/header pairs, randomly shuffled between HITss We asked the following questions for each relation type:
A.1 LLM Hyperparameters
We use the GPI-3.5-TURBO LLM for all genera- tion, with 2048 max tokens. We use the default hyperparameters, i.e., a temperature of 1, top 1. frequency penalty 0, and presence penalty 0. For the response generation prompts, we use stop se- quences corresponding to the agent names.
Preconditions: "How typical is that Sen- tence 2 is pre-condition of the event in Sen- tence 1
Static-condition:
"How typical is it that Sentence 2 is true throughout the duration of the event in Sentence 1?"
A.2 Experiment Hyperparameters
For schema induction, we use hp = 2 examples for each passage generation prompt, and k2 exam- ples for each schema induction prompt. We also set Np = i.e., we generate single passage for each fact/schema. For generation, we use li 3 examples for each paraphrase prompt, and retrieve N₂ facts from the selected habitual schema (excluding the header). These values were found to be sufficient through preliminary sensitivity analy- sis.
"How likely is it that Sen- tence result of the event in Sentence 1?"
Postconditions:
"How likely is it that Sentence 2 is goal of the agent of the event in Sentence 1
Goals:
Episodes: "How likely that Sentence 2 occurs as a step of the event in Sentence 1?"
B Experiment Details
We used a 5-point Likert scale with the following labels:
B.1 Experiment costs
We estimate that generation of schemas for every item in the PersonaChat test cost approximately $11, and took about 16 hours to complete (with OpenAI queries being sent sequence from sin- gular process). Generating responses for all three methods for every item in the dataset cost approxi- mately S8, and took about 14 hours to complete.
1. very non-typical
somewhat non-typical
3. neutral
somewhat typical
B.2 Human Evaluation Setup
For our human evaluation of the generated re- sponses, we used Amazon Mechanical Turk to hire two Anglophone (Lifetime HIT acceptance % > 98) annotators to rate batches of 5 pairwise com- parison between generated responses. Our study participants were limited to native English speakers within the United States. Participants were compen- sated rate of $8.4 per hour for each assignment, and on average took about minute to complete each assignment.
5. very typical
Figure shows sample HIT for annotating a schema fact.
C Language Model Prompts
We include examples of the prompts used in our method. For GPT-3.5- inputs, we use head- ers to distinguish inputs using system, user, and assistant roles. Variables to be filled in with spe- cific content are shown using angle brackets.
The comparisons were shuffled randomly be- tween Human Intelligence Tasks (HITs), and the A/B responses were also swapped randomly. Fig- ure 3 shows sample HIT for comparison between generated responses engagement and sensi- bility.
C.1 Schema Induction
In Figure 5 and Figure 6, we show prompts that are used to generate generie passages and to derive schemas from passages, respectively.
For our schema evaluation, we ask annotators (using the same qualifications) to rate batches of 10

---

colapsa
roquino len tinguege understancing
For rach nntancc you have to read the dislog hitory between two people B. and observe the two ahtmratvv responses We ropeet you to compase the two ullemaaviis on:
Engagingg Which rosporse you shink la more wwg.iigrrttootitnn...
histoyy?
Diviog Hildooy::
turn 9t:11;
Sjo12)
Ataemalivess for 5i's neot turre
Response R1: 8/11a)
Response R2: $//1b)
1.1 response do you think mmgogtn/ettrretnng?
1.2 Which resporse do you is better relevence hartory?
Both similar
The human evaluation interface that we use for collecting pairwise comparisons between response generation methods. Variable Cyy replaced with the jth turn for item while ria and rs are replaced with the response candidates for item i.
Figure 3:
C.2 Dialogue Generation
In Figure 7 and Figure 8, we show prompts that are used to generate responses in the un- constrained generation mode and the few-shot paraphrasing mode, respectively. In both cases, <background-user> and <background-syl sys> are replaced with basic facts about the user and sys- tem (in our evaluation, we use the basic personas from PersonaChar 10 while <habitual-facte is replaced with the selected facts from the retrieved habitual schema. The baseline prompt that we use in our evaluation is shown in Figure 9,
D Examples
We present additional examples (hand-sclected) of generated responses for each method on the Per- sonaChat dataset in Tables 6, 7, 8, 9, and 10. We show examples of both good responses, and re- sponses reflecting common issues that encoun- tered (discussed in Section 5). We also show addi- tional examples of generated schemas in Figures and 11.
"In practice, these may also contain facts controlled by the dialogue manager, such as the current goals and common ground in the conversation.

---

Instructions Clics co lazso)
This taok regurns busid English languago underslanding
sach issaano, be recurre sentercc
and another sentencc describing fact. You be sadedd whether
described first
Although suntarrcas may contan parson pronouma (frral, you should subject sunteeocss artilrary person.. For ecompii, the question were the to followings
go Bentence bag popcor.
How typical is that Bentence 2 ocours as stop of the event in Bentenne 17
"Very indioate
the question wore the following:
Sentence bathing
How typical io that Sentence pre oondotod of the event Sentence 1?
You "Very non-tyoical Indicate that
conditin
$5/1_0) Sentence 2: Sye
Siptect 0)
Ver non-typical non-typical
typical
Very typical
Figure 4: The human evaluation interface that for collecting schema quality annotations. Variables al_i and a2_ are replaced with the sentences for the schema header and fact for item i, respectively, while qtext_i is replaced with the corresponding question from the above list.
[System]
generii passage" is shart passage about recurring type evant, whare subsaquent sentence describes how that event would typically accur. Given a fact describing recurring event. generate passage.
[User]
Hy grandmother used to bake pies on Saturdays.
[Assistant]
Hy grandmother used to bake the nost wonderful pies Saturdays. She would go to the orchard on Shudy Lane peaches.. Then would into kitchen
early in marning. she and shoo everyone else away. About clack an irresistible arona wafted through the entire house.
pick basket
[User] <faer>
Figure 5: The GPT-3.5-TURBO prompt that is used to generate generic passage from given persona fact.

---

[System]
passage describing babitual avent, schena consisting the following sections:
hhaader
ppredicate argunents))
ppreconditions
[a 1ist of preconditions of the event
:static-conds
list conditions that held throughout the event]
ipostconditions
conditions hold after event]
:goals
la 1ist goals for all agents Involved the event]
:episades
[a list of steps that occurred during this event]
[Useer]
Hy grandauther used to bake the nost wonderful pies Saturdays. She would to the orchard Shady Lane early in morning. She used to pick basket apples and peaches Then would go into kitchen and shoo everyone else away. About o' o'clock irresistible aroma wafted through the entire house.
[Assistant]
:headar
Ny grandmother used to bake pies on Saturday.
:precanditions
it is Saturday
grandnother collected ingredients pie.
:static-cands
Ny grandmother has the equipment to bake pie.
My grandnother knows how to bake pies.
:osstconditions
Pies been baked.
hava received ples.
:goals
My grandrother's is goal is to bake pies for geal receive pies ny grandnother.
:episodes
My grandnother goes to the orchard on Shady Lane.
grandnother picks basket of apples and peaches.
L...]
[User] <passage>
Figure 6: The - prompt that is used to induce an event schema from a generic passage, using in-context examples. Note that some the example schema is omitted for brevity.
[System]
conversation <user> <sys>.
Background for <background-user>>
Background <sys>: <background-sys>
Use all the fallowing facts about <sys> in your response: <hubitel--aaets>
<histary>
Figure 7: The GPT-3.5-TURBS prompt that is used to generate response in unconstrained generation mode, given names the system user, as well inputs described Section 3.2.2.

---

[System]
cuser> is having conversation with <sys2.
Background for <user>:
<background-wer>>
Background For <background-sy1>
[ORIGINAL] [REWRITTEN]. responses, but Do NOT change the neaning the original sentence.
Rewrite
relevant your rewritten
Relavant
enjoy skiing
skiing
year. of activities do you like?
Parson What
Person A: I'n pretty into skiing.
Person B: Do like video ganes?
<sys>:: What sorts of activities do like?
<user>: pretty
skiing.
<sys>: That s awesome! 1 actually really into skiing as well. Just last year I went skiing in utah, which other types of activities like? do you like playing video gones?
lot of fun. what
[...]
Relevant facts: <habitual-facts>
<history>
Person [ORIGINAL J: <sentence>
<history>
<sys> [REMRITTEN]:
Figure 8: The GPT-3.5-TURBO prompt that is used to generate a response in paraphrase generation mode, given names for the system and user, well the inputs described in Section 3.2.3.
[System]
Mrite conversation between <user> and <sys>.
Background cuser>:
<backgrondd-sse>>
Background for <sys>;
<background-sys>
chistarys
Figure 9: The GPT-3.5-TURBO prompt that is used as a baseline in the evaluation in Section 4.

---

hheader "I bought my first home and I am unpacking and decorating."
ppreconditions
"I have completed the process of buying home."
"My belongings have been moved into my new home."
:static-condition
"I have unpacked boxes unpack." "
"I have furniture and decor to up."
:postconditions
"My belongings unpacked and organized,
"I enjoy living in my home."
ggoals
"My goal unpack and organize my belongings."
"My goal decorate and set up my home to my liking."
:episodes
"I take deep breath and pick up a box."
"I start looking for the perfect spot to set the box down."
"I begin unpacking boxes and organizing my belongings."
"I give extra attention to rooms like the kitchen and bathroom."
start setting up furniture and decor." [...]
Figure 10: Example of generated habitual schema, corresponding the example in Table 6.
header "I like to take w
pprccondiionss
"It is early morning late evening."
:static-condition
"I have walking shoes." "I have access different routes."
:postconditions
"I feel refreshed and clear-minded." " "I have exercised."
:goals
"My goal clear my mind."
"My goal is to receive exercise."
:episodes
"I put on my walking shoes.
"I head out the door.
"I walk for a few minutes, which are the toughest minutes of the walk."
"I feel the cool breeze on my face and the warmth of the sun on my skin." "Sometimes I venture into the woods or through the park on my walks."
[...]
Figure II: Example of generated habitual schema, corresponding the example in Table 7.

---

Persona:
am expecting twins in two months. A already have children. just bought my first home. work at bank. am an omnivore.
Persona:
I like to eat hamburgers and french fries.
I don't drink soda.
I live in a house.
I like to take walks.
Gosh. How exciting. And to think I was just looking forward to dancing tonight!
User:
No, I'm single. just graduated college. You?
User:
GOLD: Only dancing I do is down the hall to the bathroom in our new house.
GOLD:
I am married and we just bought
house.
BASE: Yes, it's definitely going to be an ad- venture! Enjoy your night of dancing though, sounds like fun.
BASE: I'm system, so don't really have a personal life like humans do. But it's cool that you just graduated! What did you study?
UNCS: Sounds like a lot of fun! I'm actually busy unpacking and decorating my new home. Once everything is in place, can finally relax and enjoy my new home. It's been quite the project to set up all of my furniture and decor, but now it's decorated and set up to my liking. How about you, do you have any exciting plans for the weekend?
UNCS: I am not capable of marriage, as I am a computer program. However, do live in a house and I like take walks. Sometimes I venture into the woods or through the park on my walks. always leaves me feeling re- freshed and glear-minded even have specific walking shoes for the job! Have you exercised today?
That does sound like a fun time! The only dancing I do lately is down the hall to the bathroom in our new house. just bought my first home and I'm in the process of unpack- ing and setting up furniture and decor. Once everything is in place, I can finally relax and enjoy my new home. It's definitely busy time, especially with twins on the way, but I'm excited have space that decorated and set up to my liking.
PARA:
PARA: That's great to hear! I'm married and we actually just bought a house. Speaking of walking, love taking walks myself. It's so refreshing and clears my mind. I make sure wear my walking shoes, and sometimes I'll venture into the woods or through the park. Do you feel the same way after taking a walk?
Table 7:
Example generation where the paraphrase method yields more sensible response due correctly paraphrasing the gold response.
Table 6: Example generation where methods yield reasonable responses.

---

Persona:
My pets name is charlic. produce music for artists.
Persona:
I love to go to Disney world every year. Mickey mouse is my favorite character. I play with my friends the playground. I'm in the third grade.
My favourite food is pizza. drive 2015 honda Civic.
User: love winter thought, nice and cold.
User: Hello, how are you doing today?
GOLD: do enjoy the driving in my Honda civic.
GOLD: Okay. go to school. like Disney world.
I'm glad you enjoy it! I prefer warmer weather, but everyone has their own prefer- ences. By the way, have you listened to any good music lately?
BASE:
BASE: I'm doing great, thank you for asking! I always love talking to new people. How about you?
UNCS: That's great! Charlie loves winter too, that means more walks for him. He's my faithful companion and provides me with emotional support. I take care of his grooming so that he remains clean and healthy. Being with Charlic gives me joy and I'm grateful for the love and companionship he provides.
Hello! I'm doing well, thank you for asking. I'm currently at school in the third grade, learning different subjects every day. Today I completed my spelling test and book report, and now I'm out for recess. It's week- day, so I'll be returning to class soon to con- tinue learning. How can I assist you?
UNCS:
PARA: I actually don't enjoy driving in my Honda Civic, but one thing that always bright- ens up my day is taking care of Charlie, my pet. He provides me with so much love and companionship, and even emotional support when need make sure to groom his fur to keep him clean and take him out for walks to get some exercise. It's really rewarding to be pet owner.
PARA: I'm doing well, thank you for asking. Currently, I'm at school learning different sub- jects every day in the third grade. Today, already completed my spelling test and book report. During recess, like to go out and play with my friends on the playground. What about you? know you love Disney world, is there anything else you enjoy doing?
Table 9: Example generation where the paraphrase method yields a poor response due to hallucination: the LLM attributes loving Disney world to the user, rather than to itself (per the gold response), even though this was never mentioned in the previous conversation.
Table 8: Example generation where the unconstrained method yields a more sensible response, due to the para- phrase model missing the subtle implicature in the gold response (i.c., that the speaker doesn't enjoy driving due to the winter conditions).

---

Persona:
My favorite kind movie is documentary. I'm pregnant with my first child. live with my hushand in Pennsylvania.
read a book every week.
User: That's exciting! Do you go to church?
GOLD: I do, me and my husband both go to Bible study every wednesday.
BASE: No, I don't fon'tgoo to church regularly. But do believe in having faith and spirituality in my life. What about you?
UNCS: No, I am focusing on preparing for the baby's arrival and visiting the doctor for regular check-ups ensure that my baby is healthy. 1 am also setting up nursery and gathering necessary baby supplies. Addition- ally, am staying active and taking care of my physical health so that am prepared for motherhood. cannot wait for the moment when can hold my baby in my arms. I have been reading books and gathering information about motherhood to prepare myself as well.
PARA: Yes, do go to church with my hus- band. I'm actually visiting the doctor regu- larly to make and preparing for the baby's arrival by setting up nursery and gathering necessary baby supplies. I'm also staying active and taking care of my physical health. can't 't wait un- til the moment when can hold my baby in my arms! I've also been reading books and gathering information about motherhood.
that my baby is healthy
Table 10: Example generation where both schema- based methods yield poor responses due to the retrieved schema not being relevant (although the paraphrase method is still able to convey the gold response).


Reviews:
[{"review_id": "Jj2T0LNptt", "reviewer": "Reviewer_x1KL", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the valuable feedback! We have refined the paper to address some of the presentation improvements suggested by your review. We provide responses below.\n\n---\n\n**1. Source of gold utterances**\n\nIn general, the \"gold utterances\" required for the paraphrase method may be supplied by an expert dialogue designer or crowdsourced within a particular domain (see lines 273-274). In our experiment, we make use of the existing gold annotations within the PersonaChat dataset.\n\nIn order to clarify the language around the paraphrase method in our paper, we have replaced mentions of \"gold utterances\" with the more generic term \"raw utterances\" throughout Section 3 -- making clear that such raw utterances must be supplied as domain-specific inputs to our paraphrase method -- and have added a clearer explanation of how we derive the raw utterances from the PersonaChat gold utterances for our experiments in Section 4.\n\n**2. Interpretation of comparatively poor human evaluations of gold responses in Table 2**\n\nWe believe that the crowdsourced PersonaChat gold responses were judged as less engaging than the LLM-based methods primarily due to their rather short length and relatively low diversity of language, as indicated by the metrics in Table 3. Furthermore, in a qualitative analysis of the gold responses, we observed that many were not fluent with the conversation history, but instead abruptly shifted the topic to one of the facts provided to the crowdsourced workers. Our assumption is that the LLM-based methods were judged as more relevant than the gold responses due to the ability of LLMs to interpolate smoothly with conversation history, even when constrained by our proposed methods. We have extended our discussion of the results in Table 2 in the paper.\n\n**3. Information on scale and wording for Table 5 Likert ratings**\n\nWe have added the scale and wordings used for our Likert ratings to the paper, reproduced below:\n\n1: very non-typical\n\n2: somewhat non-typical\n\n3: neutral\n\n4: somewhat typical\n\n5: very typical\n\n**4. Effects limited to English language**\n\nThe authors of this paper have previously focused primarily on English language dialogue systems for several application domains, and our current contribution is largely motivated by an effort to improve the performance of these systems. However, we agree that further investigation of LLM-based dialogue generation across languages remains an important direction for future work, as noted in our discussion of limitations.\n\nWe hope this clarifies the reviewer’s concerns. Please don’t hesitate to let us know if you have any further questions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Plausible approach to leverage persona-driven, event-related habitual knowledge in prompting LLMs for turn-level dialogue generation.  Well thought out experimental set up. Clear and concise presentation of the work done. ", "reasons_to_reject": "Effects are shown for English only. The experimentation results are ambiguous, and indicate the need for further investigation of LLM-based dialogue generation. ", "questions_to_authors": "", "paper_topic_and_main_contributions": "The presented work is based on the assumption that personas in realistic human dialogue are related to story-like narratives and involve rich habitual knowledge which can be induced from LLMs. These event schemas can then be used in dialogue generation by first calculating the cosine similarity between sentence embeddings of the previous utterance and the schema embeddings, and then by prompting a LLM (GPT-3.5 turbo) with the dialogue history, habitual facts from the previous dialogue step and the non-episodic facts from the current dialogue schema (unconstrained generation) as well as instructing the LLM to paraphrase a given sentence in a way that it is interesting and appropriate and uses relevant facts  (few-shot paraphrase generation). The approach is tested using the PersonaChat dialogue dataset. The evaluation experiments (using automatic methods and human evaluation) provide some evidence that generated schemas comprise facts that are considered more typical for the schema than a random combination of schema headers and facts. There is also some evidence that the use of event schemata in LLM prompting has positive effect on the diversity of generated dialogue utterances, and that utterances generated by the paraphrase generation method show higher similarity to human generated (gold) utterances from the PersonChat dataset than utterances generated by the other prompting methods.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Line 277, 325, 365: briefly say where the respective gold utterances come from; unclear whether “”old utterance” always refers to utterances from the PersonaChat dataset.\nTable 2: Any assumption why the gold response scores comparatively badly?\nTable 5 Likert ratings: add information on the scale and wordings you used, e.g. not very typical (5), typical (4), neutral (3) …\n"}}, {"review_id": "64kFhrNbmO", "reviewer": "Reviewer_gBoD", "summary": "", "questions": "1-\tHow many facts do we need when generating new schemas?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the valuable feedback! We have refined the paper to address the questions raised by your review. We provide responses below.\n\n---\n\n**1. Number of facts needed when generating new schemas**\n\nWe assume that each fact within a persona corresponds to exactly one schema, containing that fact as a header (i.e., the sentence characterizing the overall event; see lines 223-225). For example, the persona fact \"I work in a bookstore.\" would be used to generate the schema shown in Figure 2. Since our method of schema induction involves first sampling passages from an LLM, each generated schema requires only the single fact that corresponds to the schema header. We have added this clarification to Section 3.1 of the paper.\n\n**2. Clarification of other details needed for reproducibility**\n\nWe have clarified some points in the schema generation and evaluation sections with the help of the feedback in this review as well as the others; we refer to our rebuttals to the other reviews for additional details. In addition, we want to note that the parameters and data used in our experiment are detailed in the Appendices.\n\nIf there are any other clarifications or details that we can provide to improve reproducibility of our work, please do not hesitate to follow up with further questions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1-\tThe paper is well structured. It presents clearly the main contributions;\n2-\tThe method is well formulated;\n3-\tThe paper presents several rich appendices that detail the method and experiments;\n", "reasons_to_reject": "1-\tSome passages of the paper should be clarified, such as schema generation and evaluation.", "questions_to_authors": "1-\tHow many facts do we need when generating new schemas?", "paper_topic_and_main_contributions": "The paper presents an approach to dialogue generation that uses a collection of explicit event schemas to augment an agent's persona and conditions an LLM to generate narrative-like responses consistent with these schemas through in-context prompting.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Ov2FkZwLYf", "reviewer": "Reviewer_qnGv", "summary": "", "questions": "A: In l. 194, the paper mentions the use of dialogue context – Do you only observe response utterances of the system or from the user as well, as it is not clear in the paper?\nB: It is correctly pointed out that the approach leads to very long responses which may be inappropriate in certain situations. How do you plan to tackle this issue?\nC: How could you mitigate the issues by including a gold standard for response evaluation?\nD: How were the study participants compensated and could give more information about the demographics?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the valuable feedback! We have refined the paper to address some of the questions and concerns raised by your review. We provide responses below.\n\n---\n\n**1. Mentioning topical coherence in the introduction and dropping it in the remainder of the paper**\n\nOur intention with mentioning topical coherence in the introduction was primarily to motivate the need for allowing controllable dialogue flow, with the problem of topical coherence being given as a specific example where controllable dialogue flow may be desirable. We refer in our introduction to (Grassi et al., 2021) who, for instance, use an ontology of hand-engineered response templates to improve the topical coherence of their dialogue system. This need forms the motivation for our proposed paraphrase method, where a hand-engineered raw response is provided by a dialogue designer to be paraphrased into a more natural persona-based response.\n\nIn order to clarify this, we have amended lines 44-48 to the following:\n*\"Furthermore, in many practical applications beyond chit-chat, there is a complementary need to control the flow of dialogue; for example, ensuring consistency of generated responses with hand-engineered templates may help to improve a dialogue system's topical coherence (Grassi et al., 2021).\"*\n\n**2. Better illustrated examples in the Methods section**\n\nIf space in the camera-ready version allows, we can create an additional figure showing the architecture of our two response generation methods, using the same example from Figure 2 and Table 4.\n\n**3. Fairness of including the gold standard in response generation**\n\nWe believe that there has been some confusion over the purpose of our inclusion of the gold standard in our paraphrase method, as a result of unclear language used in our paper.\n\nAs mentioned in our response to the first point, the motivation for our paraphrase method is to allow conformity with an intended raw response, which may for instance be supplied by a dialogue designer. For our PersonaChat experiment, we make the assumption that we can regard the gold annotations from the dataset as raw responses to paraphrase -- in this sense, we do not actually view them as a true gold standard for responses (indeed, in our human evaluation, annotators tended to prefer even the LLM baseline to the gold annotations), but rather as purely an input to our paraphrase method.\n\nThe purpose of our similarity-based evaluation metrics, then, is not to measure the overall quality of the generated responses (we instead do this through a combination of automatic diversity-based metrics and human judgements of engagement/relevancy), but rather to investigate whether the paraphrase method is indeed able to conform to the given raw response. To test this hypothesis, we measure similarity between the raw response and the generated response for the paraphrase method in comparison to the other methods. Since it is an open question whether an LLM is in fact able to maintain consistency with the raw response, especially when conditioning on additional habitual knowledge and conversation history, we believe that our results on these metrics present an interesting finding.\n\nWe appreciate your feedback, as it prompted us to modify the paper to clarify the language used around the paraphrase method. Particularly, we have replaced mentions of \"gold responses\" with the more generic term \"raw responses\" throughout the paper, except for Section 4, where we have added an explicit statement that we use the gold annotations from PersonaChat as raw responses in our experiments.\n\n**4. Clarification questions**\n\n*A) Dialogue context*\n\nOur dialogue context contains both utterances from the system as well as utterances from the user (we have added this clarification to the paper as well).\n\n*B) The issue of lengthy responses being inappropriate in particular situations*\n\nWhile addressing this issue largely lies outside of the scope of our paper, we believe that our proposed response generation method should be situated within a dialogue management system that, at a minimum, should maintain some representation of the dialogue state, update the dialogue state based on interpretations of user input, and map the current dialogue state to suitable actions (cf. the minimal framework for dialogue management provided by [1]).\n\nWithin such a dialogue system, the method used for generation may be determined by the dialogue manager based on relevant features of the dialogue and domain. A simple approach, for instance, would be using a classifier to select between our persona-based response generation methods or an alternative response generation method that produces e.g. short empathetic responses, open-ended questions, etc.\n\n[1] David R. Traum and Staffan Larsson. “The Information State Approach to Dialogue Management.” In: Current and New Directions in Discourse and Dialogue. Ed. by Jan van Kuppevelt and Ronnie W. Smith. Dordrecht: Springer Netherlands, 2003, pp. 325–353.\n\n*C) Mitigating the issues by including a gold standard for response evaluation*\n\nWe hope that this has been adequately answered by our response to (3), but if not, please let us know and we will attempt to clarify further.\n\n*D) Compensation and demographics of participants*\n\nOur study participants were recruited using Amazon Mechanical Turk, and were limited to native English speakers within the United States. Participants were compensated at a rate of $8.4 per hour for each assignment, and on average took about 1 minute to complete each assignment. We have added the missing information to Appendix B.\n\nWe hope this clarifies the reviewer’s concerns. Please don’t hesitate to let us know if you have any further questions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Very interesting approach for generating more sophisticated personas for LLMs in a structured manner while relying on a strong theoretical background, i.e. the inclusion of habitual knowledge. Particularly, the use of event schemes seems to be a promising approach for generating more natural responses and making a system's persona more controllable\n- Good reproducibility\n- Rich evaluation including automatic and human evaluation\n- Results suggest that the approach creates more engaging and diverse responses-", "reasons_to_reject": "- At times lack of clarity:\n            Introduction: The paper mentions that there is a complementary need to control the follow of dialogue to ensure topically coherent \n            responses (l. 44 - 48), but then the topic is more or less dropped throughout the paper\n            Methods: Here, there are some unclarities due to the many concepts mentioned in this section. Better illustrated examples could \n            help to better understand the response generation process during dialogue.\n\n- Inclusion of the gold standard in response generation seems to be a bit unfair, especially when measuring BLEU score, etc., as the responses then often contained the gold sentence, which naturally results in more similarity to the gold standard.", "questions_to_authors": "A: In l. 194, the paper mentions the use of dialogue context – Do you only observe response utterances of the system or from the user as well, as it is not clear in the paper?\nB: It is correctly pointed out that the approach leads to very long responses which may be inappropriate in certain situations. How do you plan to tackle this issue?\nC: How could you mitigate the issues by including a gold standard for response evaluation?\nD: How were the study participants compensated and could give more information about the demographics?", "paper_topic_and_main_contributions": "The paper proposes a novel approach to dialogue generation for virtual conversational agents. The goal is to endow these agents with a specific persona and generate responses that are both natural and consistent with this persona. The approach leverages event schemas, representing habitual knowledge in narrative-like form, to augment the agent's persona. The authors introduce a method to bootstrap the creation of schemas from simple natural language facts. Through in-context prompting with large language models, the proposed approach generates more engaging and controllable responses, enhancing the performance of conversational agents. The main contributions include the use of event schemas to represent personas and knowledge explicitly, the bootstrapping method for creating schemas, and the improved performance of dialogue generation demonstrated through evaluation results.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors present a method to enable more consistent persona based dialogue generation for LLMs by incorporating situated, habitual knowledge in the form of schemas.

Most reviewers (at least the ones who have acknowledged the rebuttal) agree that this is a exciting approach to enable more complex personas in dialogue with LLMs, though there are some questions (partially answered) about human evaluation protocols and inclusion of gold responses, etc. especially as GPT 3.5 is used making the results somewhat irreproducable. Overall, I believe the pros of this approach outweighs the cons and merits presentation for further study.
Decision: Accept-Main