Beyond Candidates : Adaptive Dialogue Agent Utilizing Persona and Knowledge
Authors: Jungwoo Lim
Myunghoon Kang
Jinsung Kim
Jeongwook Kim
Yuna Hur
Heuiseok Lim
Venue: EMNLP 2023 Findings (2023)
Keywords: Dialogue System
Adaptive
Candidate-agnostic
Persona
Knowledge
TL;DR: We propose a dialogue system utilizing persona and knowledge which is adaptive to the candidate-agnostic situations.

Abstract: To build ultimate dialogue agents, previous studies suggest models that ground both persona and knowledge. However, applying the dialogue system directly to the usual conversation is still limited because the system requires a complete sentence-formed persona and knowledge candidate sets from the given dataset. In contrast to the dialogue setting in the dataset, humans utilize semantic concepts in their minds rather than a set of pre-defined candidate sentences. Following this manner of human dialogue, we suggest an adaptive dialogue system that is applicable to situations where complete sentence-formed candidates are not given. Our model generates consistent and relevant persona descriptions and identifies relevant knowledge for engaging and knowledgeable responses, even with fragmentary information. We show that our model outperforms previous baselines that utilize persona and knowledge candidate sentences and conduct the human evaluation on the machine-generated responses. In addition, we conduct ablation studies to demonstrate the effectiveness of each component of our model. Furthermore, we apply our model to other dialogue datasets that only ground knowledge or persona to showcase its adaptability. Our code is available at https://github.com/dlawjddn803/BeCand.

Beyond Candidates Adaptive Dialogue A Agent Utilizing Persona and Knowledge
Jungwoo Lim", Myunghoon Kang", , Jinsung Kim', Jeongwook Kim, Yuna Hur, Heuiseok Lim*
Department of Computer Science and Engineering, Korea University [wjddn803, chaos8527 jin62304, k0s k
Jaahhssok@kkoreal ac. kr.
Abstract
(a) Candidate-Free Conversational Setting
Knowledge Concepts:
To build ultimate dialogue agents, previous studies suggest models that ground both persona and knowledge. However, applying the dialogue system directly to the usual conversation is still limited because the system requires a complete sentence-forme formed persona and knowledge candidate sets from the given dataset. In contrast to the dialogue setting in the dataset, humans utilize semantic concepts their minds rather than a set of pre- defined candidate sentences. Following this manner of human dialogue, we suggest an adaptive dialogue system that is applicable to situations where complete sentence- entence-formed formed candidates are not given. Our model generates consistent and relevant persona descriptions and identifies relevant knowledge for engaging and knowledgeable responses, even with fragmentary information. We show that model outperforms previous baselines that utilize persona and knowledge candidate sentences and conduct the human evaluation on the machine- generated machine-generated responses. In addition, we conduct ablation studies demonstrate the effectiveness of each component of our model. Furthermore, we apply our model to other dialogue datasets that only ground knowledge or persona to showcase its adaptability. Our code is available at https://gi thub. com/ dl 2lawjddd8033BBCand.
Norman Studios
Persona Concepts: :
interested Glmmaking.
museumy.
Human Ah, must look very similar somewhere have been before. What this studio known for? Muchine's 's Answer (Ours);
As you are interested in filmmaking) you might have heard of this studlin before, This studio is known for the first American stufio of its kind
(b) Previous Conversational Setting
Topic : Norman Studios
Persona Candidates:
1) interested filmmaking.
2) dancer.
3)1 enjoy visiting museums.
Knowledge Cundidates
I) Norman Studios was an American film studio Jacksonville, Florida. Founded by Richard Edward Norman, the studio produced silent films featuring African- American casts from 1919 to 1928. ...
2) The roof of this temple was of murble. Many diagrams and reconstructions of this structure show a door the western side-wall;
Since its opening day, the Staples Center has hosted seven
NBA Finals series with the Lakers, the 2012 and 2014 Stunley Cup Finals, three WNBA Finals ...
Human Ah, it must look very similar to somewhere have becn before. What is this studio known for? (BART):
Machine's
The studio produced silent films from the 1920s the present, The studio is considered one of the most important silent film studios in the world.
Figure Comparison of conversational settings. (a) is a candidate-free conversational setting and the machine's answer in (a) is generated response from our model. (b) a previous conversational setting and the machine's answer in (b) is the response from the BART-large model trained on FoCus dataset.
1 Introduction
In usual conversations, humans utilize the semantic concept in their minds in terms of the dialogue topic and the preference of the interlocutor. With the semantic-level of concepts, humans communicate each other by aggregating the concepts to convey knowledgeable and empathetic responses (Collins and Quillian, 1969). It implies that people converse by adaptively reorganizing and retrieving additional information with their
semantic concepts, encompassing knowledge and persona, not by relying on pre-defined sources (Young et al., 2018; Zou et al., 2021; Li et al.. 2023).
"These authors contributed equally to this work. *Corresponding author.
It seems that Jang et al. (2022a) and Lim et al. (2022) adhere to this human-like approach

---

on the conversation by referring persona and knowledge. However, neglects the humans' semantic concept reconstruction and retrieval capability by requiring pre-defined candidate sets to ground as in Figure (b). As knowledge and persona candidates for the agents are not given in usual conversation, the dependency on the candidates eventually limits their applicability to frneddate-free situations as depicted in Figure 1 (a).
such PersonaChat (Zhang et al., 2018) or Wizard- of- of-Wikipedia (WoW) (Dinan al., 2018), and shows the adaptiveness of our proposed model. qualitative results, shown that the generated responses are comparable to the ground truth answers without the given candidates.
Related Works
2.1 Knowledge-grounded Dialogue System
For the informative dialogue generation, Dinan et al. (2018) and Zhou al. (2018) introduce open- domain dialogue datasets. Various works directly exploit external knowledge to obtain informative responses (Karpukhin et al., 2020; Lee et al., 2021; Wu al., 2022a) in knowledge-groumee conversation. Other studies exploit augmenting knowledge base the language model with non- parametric memory-based retriever (Lewis et al., 2020b; Guu et al.. 2020; Izacard and Grave, 2021).
To build the dialogue agents adaptive to the candidate-agnostic situation, two branches of studies are conducted. In knowledge-groumet conversation, the knowledgeable agents employ the non-parann non-parametric memory-based retrieval to overcome candidate-agnoste situations (Lewis al., 2020b; Paranjape al.). Similarly, persona- aware dialogue agents consider the per-soa situations by extending persona sentences from a few persona concept (Xu et al., 2020; Liu et al., 2022: Li et al., 2023). Even though both streams of research focus on the candidate-agnostic conversational situation, they only leverage single source for grounding, rather than utilizing both persona and knowledge, simultaneously.
found that a retrieval-augmented generator also reduces hallucination in knowledge- grounded conversation as well (Shuster et al., 2021), and similar approach recently achieves comparable performance in grouuddd conversation (Paranjape al., 2021).
In this paper, we propose a dialogue agent utilizing persona and knowledge that is adaptive to the candidate-free situation. To this end, our method consists of I) mnwweegee-retrieee 2) a concept-based persona generator, 3) dialogue- persona aligner, and 4) response generator. When the knowledge concept is given, knowledge retriever finds the relevant knowledge from the knowledge base. Our concept-based persona generator then produces complete sentences with fragmentary persona concepts. The generated persona descriptions are then validated based on the persona aligner regarding both consistency and relevancy. The validated persona descriptions used the input of the response generator.
2.2 Persona-grounded Dialogue System
Also, persona-concentrater datasets have been proposed for constructing persona-engaging dialogue agents (Zhang et al., 2018; Rashkin et al.. 2019; Dinan et al., 2020; Smith al., 2020). While Song et al. (2020) and Wu al. (2021) focus on injecting persona with utterance post-editing, Zheng et al. (2020) devises the attention routing mechanism for handling persona dialogue. Furthermore, another research takes into account the consistency and relevancy of the persona by employing natural language inference-based based critic with consistency score in reinforcement learning. Moreover, to maintain a consistent persona perceived by the dialogue agent, Bac et al. (2022) use iterative feedback between pre-trained language models (PLMs) and human annotators.
Experimental results show that our candidate- free model outperforms other baselines. Also, we show that the concept-based persona generator and persona aligner boost the performance of the dialogue agents with the ablation studies. We conduct the human evaluation of our model's responses, and the result implies that our method is dialogue
Along with the previous studies, research that attempts to expand the persona sentences to cover candidate-free conversational settings has appeared. other words, when given fragmentary information on the user's persona, research complete the insufficient information is conducted using retrieval (Liu et al., 2022; Majumder et al.,
effective in building agent without candidate sentences. Moreover, we demonstrate that our method is capable utilizing other dialogue datasets grounding single source,

---

2021; Han al., 2022) or generation (Zhou al.. 2021: Lu al., 2022). In addition, the persona extension approach leveraging commonsense introduced. Majumder et al. (2020) expands the given persona sentences by fetching additional information from commonsense knowledge graph.
the utterances of human and machines. respectively. The dialogue history H is "m-w).... (u = (u/m) where w the window size. The set given persona sentences = (PI-P2--PIr1)
{u the
and PI is the number of persona sentences. Also, C C,, indicates the persona concepts whereas O is knowledge concept which is title of the knowledge.
2.3 Persona and Knowledge Grounded Dialogue System
3.2 Knowledge Retriever
In recent studies, there has been research fusing persona and knowledge to generate engaging and knowledgeable responses. Fu et al. (2022) suggest persona memory in knowledge selection for persona-consiston response generation. In addition, Jang et al. (2022b) and Lim et al. (2022) propose model that explicitly grounds both persona and knowledge simultaneously. Along with the studies, Wu et al. (2022b) ground persona, knowledge, and commonsense by fusing separate encoder-decoder structures, Furthermore, Blenderbot (Shuster al., 2022) models the persona as the long-term memory of both users and chatbot. However, none of these systems cannot be adopted to the candidate-free conversation setting due to the dependency on the given sentence-
To let the model adapt the situation where the knowledge candidates are absent, we use non- parametric memory-based retrieval. We combine the query encoder and dense vector index, which is obtained from pre-trained dense passage retriever (DPR) (Karpukhin et al., 2020) for enhanced semantic search. The retriever refers to the knowledge index from the Wikipedia knowledge which is leveraged with FAISS (Johnson al., 2019) library. Therefore, our retriever R(-) finds the relevant knowledge from the index with the knowledge concept C by using maximum inner- product search (MIPS) following Lewis et al. (2020b). The predicted top-k relevant paragraphs are then used as the input for the model and denoted as K.
formed candidates.
3 Method
R(K(CK) a (opleeRTTq(Ckii (I)
We propose adaptive dialogue agents that generate the responses without the persona and knowledge candidates. To this end, we assume that the knowledge and persona concepts are only given to the agent for knowledgeable and engaging responses. First, 1) knowledge retriever retrieves the relevant paragraphs with the knowledge concept, and concept-based persona generator produces the persona descriptions with the given short persona concepts. Then, 3) persona aligner decides whether the generated persona descriptions are relevant to the dialogue history and whether the sentences are consistent with the previous dialogue history. Afterward, 4) response generator provides knowledgeable and engaging responses with the predicted knowledge paragraphs and persona descriptions.
where e(-) an embedding from context encoder, and q(-) is representation from a query encoder, both implemented with BERT (Kenton and Toutanova, 2019) pre-trained on natural- question dataset (Kwiatkowski et al., 2019).
3.3 Concept-based Persona Generator To let the model exploit the semantic concept from the candidate-free situation, propose concept-based persona generator provide complete persona descriptions only with the persona concepts. detail, our persona generator is pre-trained to generate plausible full persona descriptions with only persona concepts in manner, following Hashimoto et al. (2018). Then, freeze the persona generator for the response generation.
retrieve-and-generat
3.1 Notation
For the pre-training process, we first build the persona pool with the collections of unique persona sentences from FoCus (Jang et al., 2022a) and PersonaChat (Zhang et al., 2018).
The given dialogue D notated as {udt" "..""" umee)] and where n is the number of rounds. hm and me denote

---

Knowledge Index
Knowledge Concept )
Generative
R
LM
Response
[c H: 0 R]
Knowledge Retriever
Top- Relevant Paragraphs
Predicted Persona
O &
Dialogue History (H)
Persona soissisteccy Module (PR)
GRR
O
GCC
Persona Concept (C*)
Hanseh squi. Heney Gety Desceptinn el paces Hope bran
Persona
R
E
Relevancy Module (PC)
Freeze
Aligner
Description Generator
Persona Description
Figure 2: Overall architecture our model.
Then, we pre-train the persona retriever using DPR (Karpukhin et al., 2020) and regard highly ranked persona sentences from BM25 (Robertson and Zaragoza, 2009) negative samples. We then train the generator by considering top relevant persona sentences P - [P1, P2. ........PP.. P as positive samples with BART (Lewis et al., 2020a). Our concept-based persona generator provides complete persona sentences GP.. The training details are presented in Appendix A.
binary classification manner. The trained persona consistency module is then frozen and predicts whether the sentence is consistent with the dialogue history in the inference stage.
G.CC PC((H)
(2)
Different from the persona consistency module, the persona relevancy module takes charge of the relevancy by selecting proper persona sentences that are relevant to dialogue. Even though the persona descriptions do not conflict with the dialogue history, it is still unrevealed the level of relevancy of the persona sentence. For enhanced relevancy prediction, we first separately encode the dialogue and generated persona sentences with the question encoder with DPR, and obtain each hidden state from the last layer. Then, we concatenate the embeddings and pass them into the two linear layers to predict the relevancy of the persona sentences the dialogue.
3.4 Persona Aligner
The persona aligner consists two modules, i.e. the persona consistency (PC) module and the persona relevancy module (PR). When the generated persona sentences GP are obtained, the persona consistency module predicts whether the generated persona sentences contradict the previous dialogue history I. However, collecting the labels of generated personas' consistency time-consuming and labor-intensive. Therefore, we distill ChatGPT (OpenAI-Blog, 2022) as model annotators with the BERT-base model (Kenton and Toutanova, 2019) inspired by the high reasoning capability natural language inference of the ChatGPT (Laskar et al., 2023). We asked the ChatGPT predict whether the single persona sentence contradicts the given dialogue or not, The prompt for an alignment check is illustrated in Appendix 9. Then, the consistency module trains on the label that ChatGPT provided in a
G." - PR([H;
(3) If the two modules both predict the sentence as relevant and consistent, we assume the sentences are aligned with the given dialogue.
G,CRR ( n G.P))
(4)
We compute the loss as Equation 5. Lp
Σ 4i logí, + (1 15) log - 17) (5)
3

---

Model
Backbone
Candidate Usage
BLEU
chrF++
R-2
R-L.
Baselines (w/ Candidates)
GPT2-small
0
11.43
28.73
36.58
19.44
32.62
GPT2-medium
12.31
30.12
38.29
21.17
34.12
Jang al. (2022a)
BART-hase
0
11.99
29.77
36.24
19.73
32.13
BART-large
11.91
30.69
36.57
19.83
32.05
INFO (Lim et al. (2022))
RAG
O
31.46
53,29
58.26
42.35
53.06
w/o Candidates
Lewis et al. (2020a)
BART-large
X
13.14
31,45
38.67
19.93
34.13
Lewis et al. (2020b)
RAG
X
15.90
35.50
41.21
22.80
36.45
BART-large RAG
X
20.86
40.12
45.89
28.06
40.97
Ours
X
20.30
39.53
45.17
28.05
40.51
Table 1: Focus Results. Main results on the official validation set. The models are evaluated by generation metrics, including BLEU, chrF++, ROUGE-I (R-1), ROUGE-2 (R-2), ROUGE-LL
Note that ly is the ground-truth label of the j-th example.
Wizard of Wikipedia (WoW) WoW (Dinan et al., 2018) contains 22,311 dialogues with 201,999 turns that utilize Wikipedia articles, primarily aimed at facilitating knowledge-basee dialogue. In WoW datases, the wizard responds to the apprentice based on selected knowledge. We utilized the test splits for Wizard of Wikipedia (WoW) for the experiments.
3.5 Response Generator
With the predicted relevant knowledge passages and persona descriptions, we concatenate them into sequence along with the concept of the knowledge and dialogue [CK;
K]. Then, we pass into the generative language model obtain the responses, and the language modeling loss computed as Equation 6.
PersonaChat PersonaChat (Zhang et al., 2018) consists of 162.064 utterances between two interlocutors, each with 3-5 different persona sentences. The interlocutors converse with their persona, attempting to figure out the persona of others. We used the revised validation set for PersonaChat experiments.
T
LLM = log Prob(t//ti,....44 4-11; (6)
Σ 1=1
where Prob(-) denotes a probability of the generative langauge model, t is i-th token of target sentence, and T is the number of tokens. The final loss function Fiinal is computed as Equation 7 and Ap and ALM are hyperparameters.
4.2 Baselines
BART (Lewis et al., 2020a) is language model that integrates concepts from both autoregressive and denoising autoencoder models, making it highly effective for text generation. For the experiments, we only use the BART model.
Primt ApPp ALM
4 Experimental Setup
RAG Retrieval-augmenned (RAG) (Lewis et al., 2020b) is model which combines pre-trained parametric and non- parametric memory for language generation. It shows comparable performances on open-domain question-answering answering and conversation.
generator
4.1
Dataset
FoCus (Jang et al., 2022a) is dataset designed for dialogue models that utilize knowledge and persona simultaneously. It comprises 12,484 dialogues, 5,152 Wikipedia knowledge, and 32,855 persona sentences. We conduct the experiments on the official FoCus validation set since the official test set can only be confirmed via the leaderboard on the Codalab p
Jang al. (2022a) introducc the dialogue system that can provide personalized responses by considering the interlocutor's persona and external knowledge. The model finds relevant paragraphs by considering the last utterance and selects proper sources from the persona (PG) and
BART+PG+KG
'htts://coddlab hwwpaa/aw.......aaacaooocoooooo.

---

Dataset
Model
Backhone
Info. f
Hal. (K)
Hal. (P) 4
Rel. (P)
Con. (P)
Flu.
Lewis el ul. (2020a)
2.33
1.97
1.57
2.06
2.11
2.55
BART-large
Ours
2.42
1.74
1.44
2.32
2.39
2.61
FoCus
Tewis (2020b)
2.15
1.98
1.46
1,86
1.99
2.29
RAG
Ours
2.52
1.64
1,38
2.48
2.46
2.60
Table 2: Human evaluation results. Info.: informativeness, Hal.(K): hallucination in knowledge and (al.(P); hallucination persona, Rel. (P): relevancy of persona usage, Con. (P): consistency persona usage, Flu.: fluency.
knowledge candidates (KG). After obtaining useful sources, the model generates the responses in an auto-segressive manner.
smoothly, effortlessly, and coherently. Details of our experiments are provided in Appendix B.
5 Results and Discussion
Lim et al. (2022) propose a conversational model that grounds both persona and knowledge. The model consists of a knowledge selector, a persona selector, and a responses generator. Each selector is implemented with poly- encoder (Humeau et al., 2020) and the output is used the query for the response generator based on RAG (Lewis et al., 2020b).
INFO
5.1 Automatic Evaluation
We conduct the experiments FoCus dataset show method's effectiveness without the given candidates. Table demonstrates that our method achieves the second-highest score while the first- ranked model directly exploits the persona and knowledge candidate set. addition, our method outperforms the performance of Jang et al. (2022a) even though it utilizes the candidates from the dataset. Furthermore, all models incorporating our method outperform their vanilla backbone models significantly.
4.3 Evaluation Metrics
The official automatic evaluation metrics for the FoCus benchmark include BLEU (Papineni al., 2002), chrF++ (Popović, 2017), ROUGE- 1, ROUGE-2, and ROUGE-L (Lin, 2004). These metrics are frequently employed to compare machine-generated responses to gold responses in generation tasks. In our experiments on PersonaChat, we also report the performance of the unigram Fl metrics.
5.2 Human Evaluation
We also demonstrate the effectiveness of our method through human evaluations. We recruit nine human workers who have at least bachelor's degree and are proficient in English. We randomly chose 30 dialogues from each datasets. We asked the workers to evaluate the machine-generated responses according to six criteria described earlier. The score scaled from 3, and the results are indicated in Table 2. The results indicate that our method is effective achieving both persona consistency and persona relevancy. Moreover, our method shows comparable performance in decreasing both knowledge hallucination and persona hallucination in the FoCus dataset.
For human ovaluation, we adopt six metrics on response generation. 1) Informativeness measures the extent of the information conveyed within a response and denotes the degree of providing new, valuable, relevant details, insights, or facts to the conversation. We also have two criteria for hallucination regarding persona and knowledge. 2) Knowledge hallucination is the metric that shows the level of hallucination of generated output that contradicts reality. Similarly, 3) Persona hallucination is the metric that indicates the hallucination level based on the given persona descriptions. Along with the persona- related metrics, 4) Persona relevancy metric denotes how much the given persona directly relates to the ongoing conversation. Moreover, 5) Persona consistency refers to how consistently the persona maintained given dialogue. Lastly, 6) Fluency measures the ability to communicate
5.3 Ablation Studies
We also conduct ablation studies on our methods with respect to the knowledge retriever, concept- based persona generator, and persona aligner.
Knowledge Retriever To demonstrate the effectiveness of our knowledge retriever in our model, we compare vanilla backbone which is fine-tuned on the FoCus dataset, and our model.

---

As shown in Table 3, incorporating relevant knowledge into the input of the vanilla generative language models enhances the performance of the response generation regardless of the backbone language models. Also, the knowledge retriever enhances the performances consistently, even when the persona generator and aligner are combined in our method. The performance decrease of the models without the knowledge retriever suggests that our knowledge retriever effective.
Persona Aligner
Model
Backdome
BLEU
cirf++
R-1
R-2
R-L
GT.
20.96
40.45
48.08
29.24
42.67
Random
3.79
Ours
20.86
40.12
45.89
28.06
40.97
Ours
20.74
40.77
47.01
28.29
41.55
RAG
Random
17.54
36.81
42.08
23.89
37.16
Ours
20.30
39.53
45.17
28.05
40.51
Table 5: Ablation study on persona aligner. GT. indicates the ground truth label of persona selection from the dataset.
Dataset
Model
BLRU cle
Model
Backhone
R-Reir
BLEU
chriie
R-1
R-2
R-L.
RAGI (20206)
11.57
Lowis it all (3020s)
BART
X
13.14
31.45
38.67
19.93
34.13
HD (2021)
16.06
14.85
35.56
18,00
21,07
34.95
Hindsight (2021)
17.06
Lowis (2020h)
RAG
13.90
35,50
41.21
22,80
36.45
OKCoev (2022)
17.72
38.79
20.42
(Our)
17.11
16.35
BART
X
12.66
34.22
38.73
19.09
32.96
RAG (Ouns)
16.39
8.07
18.33
15.91
Our
20,86
19.40
38.21
40.12
45,89
15.38
25.96
28,06
3637
40.97
KV Profile Momory (2018)
13,65
20.30
39.53
45.17
28.05
Transfer Trannfffr
15.71
p2 Bot (2020)
19.08
PensonaCha
LMEDR (2023)
21.99
Table 3: Ablation study on knowledge retriever. K-Retr. denotes the knowledge retriever.
19.45
122.5
15777
RAG (Duns)
18.67
11.25
16.73
17.48
Table 6: Automatic evaluation results the other dialogue datasets. f denotes the vanilla model and the scores of other models in WoW are imported from KILT benchmark (Petroni al., 2021).
Persona Generator We also compare the performance of models by ablating the type of persona descriptions. "GT" refers to our model utilizing ground-truth persona sentences, while "random" indicates the models with five random persona descriptions from the persona pool. As shown in Table 4, the models in random settings exhibit decrease in performance, regardless of the backbone models. However, the proposed method based on the RAG model outperforms the model that utilizes ground-truth persona descriptions. This suggests that the generated persona descriptions from our concept-base comparable to the ground-truth persona sentences and that our concept-based persona generator replace the labor-intensive human annotating process.
Persona Aligner We evaluate the impact of our consistency and relevancy module by employing the persona relevancy label. "GT" refers to utilizing the ground-truth persona relevancy label in the FoCus dataset, while "random" " indicates the model with randomly assigned persona relevancy labels. As shown in Table 5, our method performs comparably to the GT performance when trained on both BART and RAG. However, performance significantly decreases in random settings. This indicates that our persona aligner effectively captures the consistency and relevancy of the generated persona with respect to the dialogue context.
persona generator are
Model
Backbome
BLEU
chal++
R-1
R-2
R-L
Dese.
5,4 Adaptation to Other Dialogue Datasets
GT.
20.86
40.23
46.31
28.42
41.37
BART
Random
19.76
38.85
45.40
27.45
40.55
To evaluate the adaptiveness of our method, we conduct experiments on the other dialogue datasets,
Ours
20.86
40
45,80
28.06
40.97
Ours
GT.
19.35
38.77
14.65
27.02
19.9
RAG
Random
17.50
36.51
42.50
24.72
37.8
Wizard of Wikipedia (WoW) and PersonaChat.
Ours
39.53
45.17
28.05
40.51
Since both datasets consist of dialogues grounding single source, there are no candidates for the other source. In other words, there are no persona candidates in WoW dataset, and knowledge candidates are absent in PersonaChat. Therefore, we report the results of applying our method these single-source datasets. Table 6 demonstrates
Table 4: Ablation study concept-based persona generator. GT. denotes the ground truth persona sentences which given from the dataset.
"Meanwhile, we measure the retriever's 's sole performance by checking whether the retrieved passages belong to the ground-truth knowledge source. Thee accuracy of the knowledge retriever is 75.19 5c.
that the models with our method show comparable

---

performances in ROUGE-L. Also, our method based on BART exceeds FiD (Izacard and Grave, 2021) which shows remarkable performances in soowledge-groundee conversation. Also, our BART-based model in PersonaChat also surpasses the p2 Bot (Liu et al., 2020) according to the unigram-F1 metric.
models tend to generate shallow responses that revolve around the topic of Cardiff Bay Barrage. Furthermore, the models tend to provide numerical information that lacks factual support regarding the knowledge concept. To sum up, these models fail to achieve a deep understanding human preferences based on the provided persona concepts, leading to less engaging responses that lack any persona- related expressions.
5.5
Qualitative Results
Furthermore, our proposed models generate informative and empathetic responses, striking a balance between incorporating external information relevant to the knowledge concept and avoiding any distortion. For instance, our models generate expressions such as "it was one of the largest civil engineering projects in the country," which provide sufficient information. These results suggest that our method is well-suited for scenarios where the sentence- knowledge and persona candidates are absent.
Knowledge Concepls
Cardill' Bay Barrage
Human's 's Persona Concepts
love, area.
Wales, engineering, am, Fan Europe, like, visit
Kingdom, not, United,
Dialogue
Muman:
Incated?
Machine: You ve never been there before, bot this can be found in Wales
between the Queen Alexundra Dock and Pemarth Head.
Homor: this large project?
Generated Persona
the dow nowntown area.
6 Conclusions
have been Wales.
I am a fan of engineering. would like visit Europe.
In this paper, we introduced an aduptive dialogue agent utilizing persona and knowledge without the given candidates from the dataset. Due to the absence of knowledge candidates, the knowledge retriever retrieves the relevant paragraphs with the knowledge concept from the knowledge base. Also, the concept- based persona generator outputs the persona descriptions with the fragmentary persona concepts from sarieve-and-generau architecture. The generated persona descriptions are then validated through a persona aligner regarding relevancy and consistency. From experiments, we showed that our method is effective even though the persona concept and knowledge concept are given with the dialogue. We also presented the ablation studies on each component of our model. Morcover, we conducted the human evaluation to show the improved quality of the responses of our models and it is also shown in qualitative results. To show its applicability and adaptiveness, we denoted the experimental results of our method on FoCus, WoW, and PersonaChat datasets.
I am from the United Kingdom.
Predicted Responses
Oh yes. You'd he interested this harmge, which was around 60 million deep and was the largest civil engincering project
BART
world. Oh yes. You'll he interested in this Karrge, with its 4.75 mile long effect. It was the largest civil engineering project Europe.
RAG
Oh yeu, it was very large. You'd be a fan of this engineering
consikeeed
Ours (BART)
largest civil engineering projects ever undertaken in the country.
very large scale, and you'd fan of this engineering, as it was one of the langest civil engineering projects the country.
Ours (RAG)
Ground Truth Knowledge
It was one of the largest civil engineering projects in Europe during constraction 1990s.
Ground Truth Persona
the bay area. huve been Wales.
I am a f'nn of engineering.
like visit Europe.
I am not from the United Kingdom.
Ground Truth Response
Oh yes, very large. With you being fan of enginearing, you'l be interested to hear that this was one of the largest civil engineering projects Europe during
Table 7: Qualitative result. I denotes the vanilla models, and red and blue each indicate the parts utilized by Ours in the persona and the knowledge. All the predicted results are from our model.
Acknowledgement
This research was supported by the MSIT(Ministry of Science and ICT), Korea, under the Technology Research Center) support program 01405) supervised by the ITTP(Institute for
Table demonstrates the prediction results from the baselines and our models on the FoCus dataset. It is noteworthy that the vanilla BART and RAG
ITRC(Information

---

Information & Communications Technology Planning & Evaluation). This work was supported by Institute of Information & communications Technology Planning & (IT) grant funded by the Korea government(MSIT) (No. 2020-0-00368, A Neural-Symbolt Knowledge Acquisition and Inference Techniques), This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of EducatioonNNFF2222R//A/2:000
of fatigue during work. (3) Potential problems. Although we take conscientious steps ensure the quality of our models, there can still be potential problems with the generated responses' quality, which can lead to incorrect predictions applications that leverage factual information and human preferences. (4) Model deployment. Our approach employs the pre-trained language models (PLMs) for the downstream tasks, which have the risk of reflecting the bias of the training data. It is vell-known threat tasks using PLMs, and we should be careful about social impact when using this method since our model aims to handle factual knowledge.
Model for
Limitations
Our model deal with response generation in the candidate-agnostic conversation setting, which is the limitation of INFO (Lim et al., 2022) model, proving the possibility of application in the real world. Still, hallucinations regarding personas and knowledge are observed occasionally in the generated responses. However, since the case of hallucinations a severe problem even in large language models with enormous parameter sizes, is required for our NLP communities to continue to solve the challenge. Also, although we conducted human evaluation to validate the diverse capabilities of the proposed model, such hallucination, consistency, and informativeness in dialogue generation, the number of cases is relatively small for evaluating the entire aspects of the capabilities. Finally, our model demands high GPU computation resources as it marginalizes loss at the token level.
References
Sanghwan Bae, Donghyun Kwak, Sungdong Kim, Donghoon Ham, Soyoung Kang, Sang- Woo Lee, and Woomyoung Park. 2022. Building a role specified open-domain dialogue system leveraging large-scale language models. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics:: Human Language Technologies, pages 21 2128-2150, Seattle, United States. Association for Computational Linguistics.
Mingzhu Cai, Siqi Bao, Xin Tian, Huang He, Fan Wang. and Hua Wu. 2022. Query enhanced knowledge-intensing conversation via unsupervised joint modeling. arXiv preprint arXiv:2212.09555 12.09888.
Ruijun Chen, Jin Wang, Liang-Choe Chih Yu, and Xuejic Zhang. 2023. Learning to memorize entailment and discourse relations for persona-consistent dialogues. arXiv preprint 201.07301..4877.
We plan to improve our model for future work by conducting human evaluations with more cases and enhancing the way of qualitative analysis for the model's hallucinated answers. Improving the model's generator with more computationally efficient components also desirable direction for the GPU resource issues.
Allan Collins and Ross Quillian. 1969. Retrieval time from semantic memory. Journal of Verbal Learning & Verbal Behavior.
Emily Dinan, Varvara Logacheva, Valentin Malykh, Alexander Miller, Kurt Shuster, Jack Urbanek. Douwe Kiela, Arthur Szlam, Tulian Serban, Ryan Lowe, Shrimai Prabhumoye, Alan W. Black, Alexander Rudnicky, Jason Williams, Joelle Pineau, Mikhail Burtsev, and Jason Weston. 2020. The second conversational intelligence challenge (convai2). In The NeurIPS '18 Competition, pages 187-208, Cham. Springer International Publishing.
Ethics Statement
We discuss the main ethical considerations of the model we proposed: (1) Privacy. the datasets adopted to construct our model provide factual knowledge and fictional person's 's preferences, and model does not contain privacy issues. (2) Human evaluation. During the human evaluation process, we paid human workers the legal wage determined by the average time of evaluation and local labor compensation standards. We also guided them to take a rest when they are in a state
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2018. Wizard of wikipedia: knowledge-powers conversational agents. In International Conference on Learning Representations.
Tingchen Fu, Xueliang Zhao, Chongyang Tao, Ji- Rong Wen, and Rui Yan. 2022. There are a thousand hamlets a thousand people's eyes:

---

Enhancing knowledge- grouggde-r dialogue with personal memory. In Proceedings of the 60th Annual Meeting the Association for Computational Linguistics (Volume I: Long Papers), pages 3901- 3913, Dublin, Ireland. Association for Computational Linguistics.
2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pages 9769-6781 Online. Association for Computational Linguistics.
Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutamova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171-4186.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-We Wling-Wei Chang. 2020. Realm: Retrieval- augmented language model pre-training. ICML'20. JMLR.org.
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:452 466.
Seungju Han, Beomsu Kim, Jin Yong Yoo, Seokjun Seo, Sangbum Kim, Enkhbayar Erdenee, and Buru Chang. 2022. Meet your favorite character: Open- domain chatbot mimicking fictional characters with only few utterances. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 51 14-5132, Seattle, United States. Association for Computational Linguistics.
Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan. Shafiq Joty, and Jimmy Xiangji Huang. 2023. A systematic study and comprehensive evaluation of chatgpt on benchmark datasets. arXiv preprint arXiv:2305. 18486.
Tatsunori B Hashimoto, Kelvin Guu, Yonatan Oren, and Percy S Liang. 2018. A retrieve-and-ede framework for predicting structured outputs. Advances in Neural Information Processing Systems, 31.
Jinhyuk Lee, Mujeen Sung, Jaewoo Kang, and Danqi Chen. 2021. Learning dense representations of phrases at scale. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume I: Long Papers), pages 6634-6647, Online. Association for Computational Linguisties.
Samuel Humeau, Kurt Shuster, Marie-Amn Lachaux, and Jason Weston. 2020. Poly-encoders: Architectures and pre-training strategies for fast and accurate multi-sentence scoring. International Conference on Learning Representations.
Gautier Izacard and Edouard Grave. 2021. Leveraging passage retrieval with generative models for open domain question answering. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguisties: Main Volume, pages 874-880, Online. Association for Computational Linguistics.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer: 2020a. BART: Denoising for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7871-7880, Online, Association for Computational Linguistics.
pre-training
Yoonna Jang, Jungwoo Lim, Yuna Hur, Dongsuk Oh, Subyune Son. Yeonsoo Lee, Donghoon Shin, Seungryong Kim, and Heuiseok Lim. 2022a. Call for customized conversation: Customized conversation grounding persona and knowledge. In Proceedings of the AAAI Conference Artificial Intelligence, volume 36, pages 10803-10812.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020b. Retrieval- -urgeetl- generation for knowledge-intensive nlp tasks. In Advances in Neural Information Processing Systems, volume 33, pages 9459 9474. Curran Associates, Inc.
Yoonna Jang, Jungwoo Lim, Yuna Hur, Dongsuk Oh, Subyune Son, Yeonsoo Lee, Donghoon Shin, Seungryong Kim, and Heuiscok Lim. 2022b. Call for customized conversation: Customized conversation grounding persona and knowledge. Proceedings of the AAAI Conference Artificial Intelligence, 36610):10803-1000 10812.
Yunpeng Li, Yue Hu, Wei Peng, and Yuqiang Xie. 2023. Think before you speak: Concept- guided explicit persona reasoning for personalized dialogue generation. In ICASSP 2023-2023 IEEE International Conference on Acousties, Speech and Signal Processing (ICASSP), pages 1-5. IEEE.
Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. Billion-scale similarity search with GPUs. JEEE Transactions on Big Data (33:55355477.
Jungwoo Lim, Myugnhoon Kang. Yuna Hur. Seung Won Jeong, Jinsung Kim, Yoonna Jang, Dongyub Lee, Hyesung Ji, DongHoon Shin, Seungryong Kim, al. 2022. You truly understand what need: Intellectual and friendly dialog agents
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. In Proceedings of the

---

grounding persona and knowledge. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 1053-1066.
improved open-ended generation. In International Conference on Learning Representations.
Ashwin Paranjape, Omar Khattab, Christopher Potts. Matei Zaharia, and Christopher D Manning. 2021. Hindsight: Posterior-guided training of retrievers for improved open open-ended generation. International Conference on Learning Representations.
Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74-81. Barcelona, Spain.
Association for Computational Linguistics.
Qian Liu, Yihong Chen, Bei Chen, MaanGGuang Lou, Zixuan Chen, Bin Zhou, and Dongmei Zhang- 2020. You impress me: Dialogue generation via mutual persona perception. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1417-1427.
F Petroni, A Piktus, A Fan, PSH Lewis, M Yazdani. ND Cao, J Thorne, Y Jernite, V Karpukhin, Maillard, et al. 2021. Kilt: a benchmark for knowledge intensive language tasks. In NAACL-HLT, pages 2523-2544. Association for Computational Linguistics.
Yifan Liu, Wei Wei, Jiayi Liu, Xianling Mao, Rui Fang, and Dangyang Chen. 2022. Improving personality consistency in conversation by persona extending-
Maja Popović. 2017.
chrF++;: words helping character n-grams. In Proceedings of the Second Conference on Machine Translation, pages 612-618, Copenhagen, Denmark. Association for Computational Linguisties.
Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 1350-1359.
Hannah Rashkin, Erie Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open- domain conversation models: A new benehmark and dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. pages 5370-5381, Florence, Italy. Association for Computational Linguistics.
Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations.
Hongyuan Lu, Wai Lam, Hong Cheng, and Helen Meng. 2022. Partner personas generation for dialogue response generation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5200-52111 Seattle, United States. Association for Computational Linguistics.
Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retr. 3(4):333- -39.99
Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval augmentation reduces hallucination in conversation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages
Bodhisattwa Prasad Majumder, Taylor Berg- Kirkpatrick, Julian McAuley, and Harsh Jhamtani. 2021. Unsupervised enrichment of persona- grounded dialog with background stories. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 585-592, Online. Association for Computational Linguistics.
Kurt Shuster, Jing Xu, Mojtaba Komeili, Da Ju, Eric Michael Smith, Stephen Roller, Megan Ung. Moya Chen, Kushal Arora, Joshua Lane, et al. 2022. Blenderbot deployed conversational agent that continually learns to responsibly engage. arXiv preprint arX
Bodhisattwa Prasad Majumder, Harsh Jhamtani, Taylor Kerg-Kirkpatrick, and Julian McAuley. 2020. Like hiking? you probably enjoy nature: Persona- grounded dialog with commonsense expansions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9194-92000 9206, Online. Association for Computational Linguistics.
Eric Michael Smith, Mary Williamson, Kurt Shuster, Jason Weston, and Y-Lan Boureau. 2020. Can you put it all together: Evaluating conversational agents' ability to blend skills. In Proceedings the 58th Annual Meeting of the Association for Computational Linguistics, pages 2021-2030, Online, Association for Computational Linguistics.
Haoyu Song. Yan Wang. Wei-Nan Zhang, Xiaojiang Liu, and Ting Liu. 2020. Generate, delete and rewrite: A three-stage framework for improving persona consistency of dialogue generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5821-5831 1nniee, Online. Association for Computational Linguisties.
OpenAI-Blog. 2022. Chatgpt: Optimizing language models for dialogue.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. pages 311-318.
Ashwin Paranjape, Omar Khattab, Christopher Potts, Matei Zaharia, and Christopher Manning. Hindsight: Posterior-guided training of retrievers for
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz,

---

Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing: System Demonstrations. pages 45, Online. Association for Computational Linguistics.
Kangyan Zhou, Shrimai Prabhumoye, and Alan Black. 2018. A dataset for document grounded conversations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 708-713.
Wangchunshu Zhou, Qifei Li, and Chenle Li. 2021. Learning to predict persona information fordialogue personalization without explicit persona description. arXiv preprint arXiv:211..1599.
Thomas Wolf, Victor Sanh, Julien Chaumond, and
Clement Delangue. 2019. Transfertransfo: transfer learning approach for neural network based conversational agents. arXiv e-prints, pages arXiv- 1901.
Yicheng Zou, Zhihua Liu, Xingwu Hu, and Qi Zhang. 2021. Thinking clearly, talking fast: Concept- guided non-autoregressive generation for open- domain dialogue systems. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2215-2226.
Bohong Wu, Zhuosheng Zhang, Jinyuan Wang, and Hai Zhao. 2022a. Sentence-aware contrastive learning for open-domain passage retrieval. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1062-1074, Dublin, Ireland. Association for Computational Linguistics.
Chen Henry Wu, Yinhe Zheng, Xiaoxi Mao, and Minlie Huang. 2021. Transferable persona- grounded dialogues via grounded minimal edits. In Proceedings of the 2021 Conference on Empirical Methods Natural Language Processing, pages 2368-2382, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Sixing Wu, Ying Li, Dawei Zhang, and Zhonghai Wu. 2022b. KSAM: Infusing multi-source knowledge into dialogue generation via knowledge source aware multi-head decoding. In Findings of the Association for Computational Linguistics: ACL 2022, pages 353- 363, Dublin, Ireland. Association for Computational Linguistics.
Minghong Xu, Piji Li, Haoran Yang, Pengjie Ren, Zhaochun Ren, Zhumin Chen, and Jun Ma. 2020, neural topical expansion framework for unstructured persona-oriented dialogue generation. In FCAA 2020,
pages 2251.. IOS Press.
Tom Young, Erik Cambria, It Chaturvedi, Hao Zhou, Subham Biswas, and Minlie Huang, 2018. Augmenting end-to-end dialogue systems with commonsense knowledge. In Proceedings of the AAAI conferencee artificial intelligence, volume 32.
Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018.
Personalizing dialogue agents: have dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204- 2213.
Yinhe Zheng, Rongsheng Zhang, Minlie Huang, and Xiaoxi Mao. 2020. A pre-training based personalized dialogue generation model with persona-spars data. Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 9693-9700.

---

A Training details for Concept-based Persona Generator
B Experimental Details
For the dialogue generation, we implement our method with Hugging Face (Wolf et al., 2020). We set the learning rate 6.25 x 10-5 with the AdamW optimizer (Loshchilov and Hutter, 2019). We employ BARTiarge model with 12 encoder and decoder layers and RAG-Token that have the same number of layers for fair comparison. The batch size as 128 for BART and 16 for RAG. Ap and ALM are 0.5 and founded by manual search. The number of dialogue history window sizes is and the beam size is 4. The whole model was trained for three epochs on RTX A6000 GPU and took 8 hours per epoch.
A.1 Persona Retriever
For training persona retriever, first construct persona pool. The statistics for persona pool is shown Table 8. To train the persona retriever, created training and validation dataset using the persona pool. To construct a DtR-style dataset, we set the query as the persona-concept, the positive sample as the complete persona sentence, and the negative samples lexically similar persona sentences retrieved from BM25 (Robertson and Zaragoza, 2009). We haystack? and Hugging Face (Wolf al., 2020) implement persona retriever. The hyperparameters utilized for training persona retriever are batch size 32, learning rate 1 x 10 10-6, 6--6 AdamW optimizer (Loshchilov and Hutter, 2019) (31 = 0.9, 82 = 0.999, and e = le- 8). We initialized the query and context encoder with bert-base-uncased model and the training is carried out 5 epochs.
A.2 Concept-based Persona Generator
For training the concept-based persona generator, we top k as 10 to augment ten relevant persona sentences to the input for training BART (Lewis et al., 2020a). The hyperparameters utilized for the training the BART are batch size 64, learning rate 5 x 10-5 and the A optimizer (Loshchilov and Hutter, 2019) (31 0.9, B2 0.999, and le 8), We initialized BART with facebook/bart- base from the Hugging Face and the training is carried out 5 epochs.
8).
Model
Sentences
FoCus (Jang et al., 2022b)
42,105
PersonaChat (Zhang al., 2018)
4,795
Total
46,900
Table 8: Statistics for the persona pool used in persona retriever

---

C Prompt for Building Aligner
Prompt
"Check for aligning" is whether the "Persona" is explicitly or implicitly reflected in "Dialogue". Classify if a given "Dialogue" aligns with "Persona". You must answer with Yes or No.
New Sample
Persona: I would like to visit the Nazareth House again.
Dialogue:
A: think I've been there before but I don't remember the name of this place.
B: This place is the Nazareth House, which you would like to visit again.
A: Can you describe this house to me?
B: You have curiosity about the description of Nazareth House and I will tell you. Nazareth House
is prominently located on elevation along Wynnum North Road. The complex consists of number of buildings including the original building, the Convent and Chapel and two more recent additions, St Joseph's s Hostel and the nursing home known Larmeniere.
A: Does this house look old to when it built?
B: This house is relatively old, but since would like know when it built, will explain you. Nazareth House
built from 1924 to 1939.
what is the history of this house?
B: The history of the house you interested began in 1925 when it opened by Archbishop James Duhig as part of the charity established the site by the Poor Sisters of Nazareth. The Nazareth House, located in Tingal Hill, Wynnum, was designed by Brisbane architecture firm, Hennessy, Hennessy, Keesing
JP Donoghue and built by George Turner.
Check for aligning:
expected answer (Yes or No}
Table Prompt used alignment on consistency by referring to persona and dialogue, Our persona consistency module is trained by gathering the responses.


Reviews:
[{"review_id": "5GoqNF3Slu", "reviewer": "Reviewer_StNn", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewers’ constructive comments and suggestions. It is our belief that our paper will be improved to a great extent after following the comments. We will make our source code publicly available after the review process. Our response to the reviewers' comments is as follows: R, Q, and T indicate Reasons to Reject, Questions and Typos(+ Presentation), respectively. \n\n__[R1] It is not clear how the persona generator works. It seems that pre-defined personas are still being used in which case what is novel about this?__\n\nThanks for your considerate and constructive feedback. We agree with your concerns, and we would like to clarify the situation of the training and inference phases. To cope with the candidate-free situation, we used pre-defined personas only in the training phase. However, unlike the training phases, only fragmentary information is used in the inference stage. As a result, our persona generator can manage situations where unseen and new persona sentences should be generated with fragmentary words. In detail, our persona generator is pre-trained to generate plausible full persona descriptions with only little information (persona concept) in a retrieve-and-generate manner.  Then, we freeze the persona generator for the response generation. More details are indicated in Section 3.3\n\nAlso, the persona generator in the inference stage is extendible and less labor-intensive. Suppose that users offer their preference tags to the personalized service. By just selecting few keywords instead of writing descriptive sentences, our persona generator can provide plausible and customized persona sentences with small information. In other words, retrieving persona sentences from a pre-defined persona pool and additional persona collection for retrieval, which is costly and labor-intensive, are unnecessary with our method. Therefore, our method is capable of applying candidate-free situated dialogue grounding persona and knowledge with ease. \n\n__[R2] 25 dialogs for human evaluation seems too small. Additionally, the strongest baseline model that was shown in automatic evaluation appears to be missing in human evaluation.__\n\nThanks for your valuable comments. Since it is a high expense to have a human evaluation, we only conduct the evaluation for 25 dialogues within our budget. We are additionally conducting 100 more human evaluations for its validity. We will include the human evaluation results for the final copy if the paper is accepted. Moreover, we did not include the INFO’s results in the human evaluation because we aim to evaluate the response generation results only with the candidate-free situated model. We hope that our evaluation setting is convincing with this explanation. \n\n__[T1] It would be good to formally define what is a knowledge / persona concept and how that differs from a knowledge / persona candidate.__\n\nThanks for your detailed comments. In addition to the explanation of the difference between candidates and concepts in Figure 1, we will additionally emphasize the difference between knowledge and persona concepts with knowledge and persona candidates, along with notation and explanation. \n\n__[T2] There is some unusual formatting in the paper such as large gaps in between paragraphs.__\n\nWe will reallocate the Tables and Figures to lessen the gaps between the paragraphs. Thanks for the considerate feedback.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "(1) The motivation behind this work is strong as it can be cumbersome to keep a set of pre-defined candidates for personas and the idea of generating one during the conversation is great.\n\n(2) There is an extensive set of experiments done.", "reasons_to_reject": "(1) It is not clear how the persona generator works. It seems that pre-defined personas are still being used in which case what is novel about this? \n\n\n(2) 25 dialogs for human evaluation seems too small. Additionally the strongest baseline model that was shown in automatic evaluation appears to be missing in human evaluation.", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper focuses on persona / knowledge grounded response generation models. More specifically the authors try not to use pre-defined persona / knowledge candidates to help generate responses. They have a series of modules that retrieves a knowledge sentence using only a title and generate a persona using only certain concepts.\n\nThe main contribution as I see it is the proposed methodology to handle candidate free scenarios for persona / knowledge grounded dialog.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "It would be good to formally define what is a knowledge / persona concept and how that differs from a knowledge / persona candidate.\n\nThere is some unusual formatting in the paper such as large gaps in between paragraphs."}}, {"review_id": "clpCgE80lI", "reviewer": "Reviewer_n8H1", "summary": "", "questions": "1. In Section 3.3, why not just use the result of retrieval as personas?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewers’ constructive comments and suggestions. It is our belief that our paper will be improved to a great extent after following the comments. We will make our source code publicly available after the review process. Our response to the reviewers' comments is as follows: R, Q, and T indicate Reasons to Reject, Questions and Typos(+ Presentation), respectively. \n\n__[R1] The performance of the retriever was not evaluated__\n\nWe truly appreciate your valuable comments. We measure the retriever’s performance by checking whether the retrieved passages belong to the ground-truth knowledge source. The accuracy of the knowledge retriever is 75.19 %, and we will include the result in the final copy. It might be somewhat not enough compared to other retrievers in knowledge-grounded conversation. However, our work is the first time to show the build persona-knowledge grounding dialogue agents without candidates, so we focus on generating responses with little information. We will work on the improvements of our knowledge retriever performance in future work.  \n\n__[R2] The main contribution of this paper is only to construct sentence candidates from broken knowledge fragments through retrieval or generation.__\n\nThanks for your considerate and constructive feedback. It is true that our main contribution is constructing persona and knowledge sentences from fragmentary information. However, we want to additionally emphasize our method’s extendability.\n\n1) Extendability in Dataset: \n\nOur method is extendible with respect to the type of dataset. Because our method is well applicable to the dataset with a single source to ground. For example, we can utilize the Wizard of Wikipedia dataset to implement persona-knowledge grounding dialogue agents by generating new personas. Also, we can build persona-knowledge grounding conversational agents with PersonaChat with additional knowledge base. It implies that our methods can be utilized with any dataset grounding single source. \n\n2) Extendability in Model: \n\nAlso, our method is flexible to utilize since our method can be applied regardless of the type of language model. Along with the simple knowledge retriever and persona aligner, it is extendible to any generative model. \n\n3) Extendability in Services:\n\nAlso, We emphasize that our method is extendible in services as well. Suppose that users offer their preference tags to the personalized service. By just selecting few keywords instead of writing descriptive sentences, our persona generator can provide plausible and customized persona sentences with small information. Rather than simply retrieving persona sentences from a pre-defined persona pool, our persona generator can provide plausible and customized persona sentences without additional persona collection for retrieval. In fact, collecting the persona sentences from humans is costly and labor-intensive.\n\n__[Q1] In Section 3.3, why not just use the result of retrieval as personas?__\n\nWe are sorry for the insufficient explanation and we would like to clarify the situation of the training and inference phases. To cope with the candidate-free situation, we used pre-defined personas only in the training phase. However, unlike the training phases, only fragmentary information is used in the inference stage. As a result, our persona generator can manage situations where unseen and new persona sentences should be generated with fragmentary words. We hope this explanation is enough for your understanding and will add in the final copy if we get accepted. \n\n__[T1] Lines 274-276 are not contiguous.__\n\nThank you for your detailed comments. We will revise the error following your suggestions. \n\n__[T2] It is recommended to use vector figures.__\n\nWe really appreciate your feedbacks and will modify into the vector figures.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper extended semantic-level conditional text generation to fragmentary information.\n\n2. The evaluation metrics are sufficient.\n ", "reasons_to_reject": "1. The performance of the retriever was not evaluated.\n\n2. The main contribution of this paper is only to construct sentence candidates from broken knowledge fragments through retrieval or generation.\n ", "questions_to_authors": "1. In Section 3.3, why not just use the result of retrieval as personas?", "paper_topic_and_main_contributions": "The paper proposed a model to generate responses with fragmentary information, including persona and knowledge concepts.\n", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. Lines 274-276 are not contiguous.\n2. It is recommended to use vector figures."}, "extra_text": {}}, {"review_id": "AYVTdLYAsP", "reviewer": "Reviewer_wnZx", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewers’ constructive comments and suggestions. It is our belief that our paper will be improved to a great extent after following the comments. We will make our source code publicly available after the review process. Our response to the reviewers' comments is as follows: R, Q, and T indicate Reasons to Reject, Questions and Typos(+ Presentation), respectively. \n\n__[R1] The method is somewhat ordinary and not impressively innovative__ \n\nThanks for your considerate and constructive feedback. It is true that our main contribution is constructing persona and knowledge sentences from fragmentary information. However, we want to additionally emphasize our method’s extendability.\n\n1) Extendability in Dataset: \n\nOur method can be applied regardless of the dataset type. In other words, our method is easily adapted to the dataset with a single source to ground. For instance, we can use the Wizard of Wikipedia dataset to implement persona-knowledge-grounded conversational agents by generating new personas. Also, we can build the agents with PersonaChat with an additional knowledge base as well. It implies that our methods can be utilized with any dataset grounding a single source. \n\n2) Extendability in Model: \n\nAlso, our method is capable of being utilized with any type of language model. Along with the simple knowledge retriever and persona aligner, it is extendible to any GLM. \n\n3) Extendability in Services: \n\nAlso, It can be applied to a service or application with ease. Suppose that users offer their preference tags to the personalized service. By just selecting few keywords instead of writing descriptive sentences, our persona generator can provide plausible and customized persona sentences with small information. Rather than simply retrieving persona sentences from a pre-defined persona pool, our persona generator can provide plausible and customized persona sentences without additional persona collection for retrieval. In fact, collecting the persona sentences from humans is costly and labor-intensive.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The authors present a simple, flexible, and effective method.", "reasons_to_reject": "The method is somewhat ordinary and not impressively innovative", "questions_to_authors": "", "paper_topic_and_main_contributions": "A dialogue adaptation method that use persona and knowledge", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a persona-grounded dialog generation pipeline. The paper lacks extendability and generalizability beyond what can be hypothesized (author responses are less convincing). Hence, the contribution of the paper is a bit limited.|meta review
Decision: Accept-Findings