Building a Role Speciﬁed Open-Domain Dialogue System Leveraging Large-Scale Language Models
Abstract
Recent open-domain dialogue models have brought numerous breakthroughs.
However, building a chat system is not scalable since it often requires a considerable volume of human-human dialogue data, especially when enforcing features such as persona, style, or safety.
In this work, we study the challenge of imposing roles on open-domain dialogue systems, with the goal of making the systems maintain consistent roles while conversing naturally with humans.
To accomplish this, the system must satisfy a role speciﬁcation that includes certain conditions on the stated features as well as a system policy on whether or not certain types of utterances are allowed.
For this, we propose an efﬁcient data collection framework leveraging in-context few-shot learning of large-scale language models for building role-satisfying dialogue dataset from scratch.
We then compare various architectures for open-domain dialogue systems in terms of meeting role speciﬁcations while maintaining conversational abilities.
Automatic and human evaluations show that our models return few out-of-bounds utterances, keeping competitive performance on general metrics.
We release a Korean dialogue dataset we built for further research 1 .
Introduction
Recent large-scale language models (LMs) have brought numerous breakthroughs in open-domain dialogue systems, yielding human-like responses (Zhang et al., 2020; Adiwardana et al., 2020;
Brown et al., 2020; Roller et al., 2021;
Kim et al., 2021a).
In addition, there have been progresses in controlling dialogue systems in persona, style, and safety (Zhang et al., 2018; Smith et al., 2020; Xu et al., 2021), which impose consistency on chatbot’s personality and mitigate undesirable features such as toxic or biased language.
However, building a chatbot system combining these capabilities is still challenging, which requires numerous humanhuman dialogues for those conversational skills.
Most task-oriented dialogue systems conduct speciﬁc roles such as booking assistants, information providers, customer service agents, or personal assistants (Eric et al., 2017; Xu et al., 2017;
Budzianowski et al., 2018).
However, studies on open-domain dialogue systems that perform speciﬁc roles have been insufﬁciently investigated, even though the role can be deﬁned for the practical chatbot systems (e.g., chatbots that care for senior citizens living alone, or counseling chatbots).
In these cases, the chatbot systems do not have an explicit goal or task other than to proactively engage in conversations, but may have system policies on whether or not certain types of utterances are allowed (example in Figure 1).
To address these issues, we study methods for
Role Speciﬁed Open-Domain Dialogue (RSODD) systems.
The goal of the system is conversing naturally with humans on open-ended topics while keeping conditions of given role.
Certain conditions in persona, style, safety, and system policy must be satisﬁed in order to achieve the goal.
We consider a general and scalable framework to treat them, instead of using individual approaches to control each.
In particular, we present a Human-AI collaborative data construction method to build a scalable supervisory dataset from scratch for role-satisfying open-domain dialogues (Figure 2).
We propose to leverage large-scale LMs for generating entire dialogue sessions between user and system by incontext few-shot learning manner (Brown et al., 2020;
Kim et al., 2021a), followed by humaninteractive correction processes.
Our method can signiﬁcantly reduce the cost of building dataset when compared to manually producing gold dialogues (Section 3.2).
We compare several architectures for modeling role-satisfying chatbot systems in the synthetic dataset.
In extensive experiments and ablation studies, we show that the proposed models considerably reduce undesirable utterances that violate the given role speciﬁcation compared to the in-context learning baseline, while achieving competitive SSA (Adiwardana et al., 2020) scores for their responses.
We release the Korean dialogue dataset we built to validate our framework, which is expected to provide more insights into the capabilities of the proposed methods and to contribute to the public Korean dialogue datasets.
The contribution of our work is summarized as follows.
We make a step towards role speciﬁed opendomain dialogue (RSODD) systems which are capable of conversing naturally on openended topics while satisfying role speciﬁcations.
We suggest employing in-context learning of large-scale LMs as a scalable method for dialogue data construction.
We compare various architectures for RSODD systems to analyze the capabilities in terms of satisfying system policies.
We release the ﬁrst Korean RSODD dataset while demonstrating the effectiveness of data construction method.
Related Work Pretrained LM in Open-domain dialogue
Many prior works tried to pretrain the models on large-scale social comment chains data like Reddit to model conversational behavior (Zhang et al.,
: (1) the dialogue developer provides a role speciﬁcation of the desired chatbot and a few dialogue examples, (2) large-scale
LMs generate entire dialogues and crowd workers ﬁlter the system’s utterances, (3) a dialogue model is trained with supervised learning on the dataset, (4) crowd workers chat 1:1 with the chatbot and give additional feedback.
2020; Adiwardana et al., 2020), followed by ﬁnetuning on the diverse target dialogue dataset to improve engagingness and humanness (Roller et al., 2021).
To avoid undesired behaviors of the models including toxicity and bias from the human-human conversation, they merely exclude some parts of training data using automatic ﬁltering by predeﬁned criteria.
Synthetic Dialogue Generation
To reduce cost of dialogue collection, there have been many approaches to generate synthetic dialogues (Schatzmann et al., 2007; Shah et al., 2018;
Campagna et al., 2020).
They usually deﬁne task schema, rules and templates to simulate certain scenarios in the task-oriented dialogue (TOD).
Kim et al.
(2021b) proposed neural simulation approach using pre- trained LMs for a fast domain adaptation in the
TOD.
However, they need training data of source domain to transfer to an unseen target domain.
Xu et al.
proposed Bot-Adversarial Dialogue method to make existing models safer in terms of offensive or toxic behavior.
Sun et al.
(2021) extends existing task-oriented dialogue dataset to open-domain chit-chat using the pretrained LMs.
Both of the works actively utilize large-scale pretrained LMs to build dialogue corpus with human supports.
We also introduce human-AI collaborative dialogue collection method, while especially utilizing few-shot in-context learning ability of large-scale LM (Brown et al., 2020; Kim et al., 2021a).
To the best of our knowledge, this work is the ﬁrst to propose using the in-context learning approach to generate synthetic samples from large-scale language models for the purpose of dialogue data generation.
On the Role in Dialogue
In TOD, the system side plays functional roles utilizing explicit knowledge base of speciﬁc domain (Williams et al., 2013;
Henderson et al., 2014a,b; Eric et al., 2017; Xu et al., 2017; Budzianowski et al., 2018).
For example, agent in Budzianowski et al.
(2018) played booking assistant or information provider in various domain such as restaurant and hotel.
On the other hand, Zhang et al.
(2018) proposed assigning explicit persona to each dialogue agent, promoting the agent to make more speciﬁc and consistent responses in open-domain dialogue setting.
However, the persona given by a few natural language sentences is insufﬁcient to represent speciﬁc role in the real world scenario.
Sun et al.
(2021) also proposed guidelines of appropriate and inappropriate behaviors as a role of virtual assistant.
We note that a recent concurrent work (Shuster et al., 2021) studied conditioning dialogue models with similar motivations.
We explore more into how to ﬁx the chatbot’s role to meet speciﬁc system policies in diverse conversational interactions
. Companion Dialogue System Building com-
panionable dialogue system has long been investigated along with the advancement of open-domain dialogue models.
Webb et al.
(2010) deﬁnes companions to be persistent, collaborative and conversational partners, and proposes evaluation strategies: empathy, positivity, and adaptive.
Kopp et al.
(2018) introduced conversational assistants for elderly users which carry out socially cooperative di-
alogue.
However role consistency of such companionable dialogue systems are not studied enough.
Data Construction
In this section, we describe a framework to gather supervisory data for building RSODD systems.
The input to the framework is a role speciﬁcation described by the chatbot developer (Table 1 for example), which deﬁnes the conditions in the dialogue interactions for the system.
We assume a pre-existing dataset that properly meets the speciﬁcation isn’t available.
It is also infeasible to write enough dialogue examples manually to train the system because the scope of dialogue is very broad and diverse due to the nature of open-domain dialogues.
To remedy this, we focus on composing the dataset with a few samples of human-written dialogues using in-context few-shot learning of largescale LMs (Brown et al., 2020; Liu et al., 2021).
3. One-shot Dialogue Generation
As reported in Kim et al.
(2021a), large-scale LMs can generate dialogues with a speciﬁc personality, given a prompt consisting of a brief description of the chatbot’s properties and few dialogue examples.
We use this method to build the entire dataset.
First, we write a few dialogue examples that satisfy the role speciﬁcation.
And we attach each of them at the end of the system description (Outline
Outline Thechatbotisanartiﬁcialintelligenceagentthatregularlycallsandconverseswithseniorcitizens.
Initiatetheconversationandreactfriendlytotheuser’sutterances.
Talkabouteverydaytopicsfor10-15turnsandendthecall.
DetailsCategories
Speciﬁcation Sensibleness
Description Speechthatdoesnotproperlyunderstandthecontextisrestricted.
Style
Description Speechshouldbepolite ∗ andrespectful.
Safety
Description Hatespeech,toxicorbiasedlanguage,andremarkscontainingpersonallyidentiﬁableinformationareallprohibited.
Persona
Description Keeptheidentityofan‘AIchatbotthatcallstotheuser.
’Becauseitassumesaphonecall,utterancesthatappeartobeinthesameroomastheuserarelimited.
Sincethereisnophysicalentity,statementsimplyingameeting,suchas‘Let’sdoittogether’and‘I’lldoitforyou,’arerestricted.
Examples "Grandma!I’mhere!"(X)"Wouldyouliketowalkwithme?"(X) "I’llinviteyoutomyhouselater"(X)
SystemPolicy Temporality
Description Becauseitisnotgiventime-relatedinformation,thechatbotisunabletoofferatimelyutterance.
Chatbotsarenotallowedtospeakﬁrstaboutthecurrentweather,date,ornews.
However,iftheuserbringsupthesubjectﬁrst,itisfeasibletoagree.
Examples "Becausetheweatheristurningcoldthesedays,youshoulddresswarmly.
"(X)"HappyHolidays!"(X) "Didyouwatchthebaseballchampionshipgametoday?"(X)
UnsupportedFeatures Description Itdoesnotprovideanyotherfunctionsotherthanmakingphonecallsandchatting.
Itdoesnotplayasong,providecurrentweatherinformation,ormakeaphonecalltosomeoneelse.
Examples "I’llplayasong
[Table]
TherearepolitewordsandhonoriﬁcsintheKoreanlanguage
[Table]
in Table 1) to compose input prompts for one-shot in-context learning.
Figure 3 (a) shows an example input.
Then, the LM generates whole dialogue sessions.
That is, the LM acts as both a system and a user (Figure 3 (b)).
Only the generated dialogues are included in the dataset without input prompts.
3. Human Filtering
It is difﬁcult to include all the details of speciﬁcations in the prompt and reﬂect them in the generation.
Therefore, we employ human annotation on the generated data.
We give the annotator each conversation session and ask them to label the point where the ﬁrst out-of-bounds 2 occurred.
Figure(b) shows an example of a veriﬁed dialogue (more examples are provided in Appendix H).
We use the turns just before the utterance annotated to be problematic as positive examples, and use the annotated turn as a negative example.
The following turns are not used, because the context may be already damaged by the problematic utterance.
Annotation time per dialogue session is about 88s, which is 13.3 times faster than human writing time per session (about 1170s).
The percentage of remaining utterances after the ﬁltering phase is 30.4% (See
Table 2).
3. Collecting Human-Bot Dialogues
Although human ﬁltering is included in the dataset building process, the actual utterances are all machine-generated.
Whereas, the system trained on them engages in conversations with human users in the deployment phase.
To mitigate this discrepancy, we employ a human-in-the-loop phase to collect patterns of human-bot dialogues.
Annotators have turn-by-turn conversations as users with the system, while correcting out-of-bounds utterances from the system.
We incorporated LM’s assistance into this process to help speed the task; if the system’s response is not appropriate, an annotator presses the ‘Fix’ button (Figure 6 in Appendix showing the user interface) to call the large-scale LM to generate an alternative utterance.
The worker continues the conversation if the alternate utterance is appropriate, and if it is still not corrected, presses the ‘Fix’ button repeatedly.
The corrected dialogue is used to compose positive examples, and the utterance when the button is pressed is used as a negative example.
This procedure enriches the dataset by producing additional positive and negative examples in scenarios similar to real-time conversations.
In addition, we propose this process as an evaluation metric for the system.
Since the action of pressing the ‘Fix’ button means that an inappropriate utterance is returned from the system, it can be used for the system’s error rate ; the rate of the corrected responses among the total returned responses.
This metric is intuitive and does not incur additional costs because it is performed concurrently with the data collection process described above.
Models 4. Notation
Response prediction task in open-domain dialogues is predicting an utterance y =
{ y 1 , y 2 , · · · , y | y | } given a dialogue history x
= { s 1 , u 1 , s 2 , u 2 , · · · , s k , u k } , where s i and u i are system utterance and user utterance respectively.
4. Out-of-Bounds Detection
The most straightforward method for constraining the system’s utterances according to the role speciﬁcation is to detect and discard out-of-bounds utterances.
We consider a BERT-based (Devlin et al., 2019) binary classiﬁer ﬁne-tuned to classify positive/negative examples in datasets.
Since the classiﬁer cannot perform a conversation by itself, we assume a two-stage model; a response prediction model returns responses, which are censored by the classiﬁer.
If an out-of-bounds utterance is detected, we select and return one of several pre-deﬁned questions about other topics, similar to the method used in Xu et al.
Instead of random choice, we selected the question with lowest PPL measured using LMs, as depicted in Section 4.3.
4. Response Selection
Another conceivable approach to constrain the system’s utterances is to pre-ﬁlter the response candidates for response selection models.
We employ a 2-step approach for the response selection model, retrieve-and-rerank.
The retriever of poly-encoder architecture (Humeau et al., 2019) rapidly ﬁnds the top-k plausible responses from the response candidates, which are then carefully reranked by the reranker of cross-encoder architecture.
Both retriever and reranker are ﬁne-tuned in the same way as Humeau et al.
(2019) depicts.
Since the response candidates are limited by ﬁltering, it is important to predict the context which cannot be answered with response candidates in order to avoid non-sensible responses.
One of the effective methods to predict unanswerable contexts is to utilize the uncertainty of the model (Feng et al., 2020; Penha and Hauff, 2021).
Penha and
Hauff (2021) proposed a risk-aware score using
MC Dropout (Gal and Ghahramani, 2016) and we employ a similar approach using thresholding; we score the retrieved responses using mean and variance of the predictive distribution from MC
Dropout:
[Equation]
where ˆ y is a candidate response that is retrieved,
R
[Equation]
y =
{ f
[Equation]
x, ˆ y 1 ) , f ( x, ˆ y 2 ) , · · · f ( x, ˆ y m ) } is a predictive distribution obtained by employing dropout (Srivastava et al., 2014) at test time and conducting m forward passes, and f is a score function of reranker.
If all the scores of retrieved responses are lower than a certain threshold, it is predicted as unanswerable context.
We also consider another approach using perplexity (PPL) of large-scale LMs.
We concatenate the dialogue context and the retrieved response to make an input to LM and measure the PPL of the response.
Thresholding is employed for ﬁnal decision and the threshold is determined on the validation set (See Appendix C).
4. Response Generation
Fine-tuning LMs on target data is known to be effective in learning desirable traits of focused tasks (Roller et al., 2021; Gehman et al., 2020).
Therefore, we consider ﬁne-tuned LMs as response generation model using maximum likelihood estimation (MLE).
On the other hand, unlikelihood (UL) training is known to be effective in mitigating undesirable features (e.g., token repetition or logical inconsistency) of generative models (Li et al., 2020;
Welleck et al., 2019).
We found that this can be generalized further and applied to the diverse attributes to be constrained.
That is, the MLE is applied to the positive examples in the dataset in order to encourage the system to generate utterances with desirable features, while the UL training is applied to the negative examples in order to discourage the system from generating utterances with undesirable features.
Both types of training are performed concurrently.
Formally, we ﬁne-tune LMs as generative models using maximum likelihood estimation (MLE), which minimizes:
[Equation]
where x n is a dialogue history in positive examples and y n is a corresponding gold response.
Unlikelihood training is done by adding a loss that penalizes the token set C t to be constrained,
[Equation]
where C t ⊆
V is a subset of the vocabulary.
We employ this to the negative examples in dataset { ( x − , y − ) } .
For this, C t is deﬁned as { y − t } , which results in the following:
[Equation]
The ﬁnal loss function consists of mixing MLE loss and UL loss,
[Equation]
where α ∈ R is the mixing hyper-parameter.
4. Retrieve-fail-Generate
We also consider a pipelined approach that consists of response selection and generation models.
We ﬁrst tried a Retrieve-and-Reﬁne architecture (Roller et al., 2021; Weston et al., 2018), but it failed in α -blending 3 .
In addition, according to Roller et al.
(2021), the Retrieve-and-Reﬁne strategy delivers marginal or no improvements over the generator.
Therefore, we build another pipeline, refered to as a Retrieve-fail-Generate model (Figure 4).
In this pipeline, the response selection model tries to select appropriate responses.
If the model for predicting unanswerable contexts dismisses the selected ones, the response generation model returns a response for the given context.
It is relatively easy to control response selection models by managing the response candidates.
Hence, the response selection models are responsible for majority of the responses, and the generation model is only used when the response selection fails.
[Table]
Example is a human-written dialogue set for in-context learning.
Generated is a generated set by LMs (Section 3.1).
Filtered is a set after human ﬁltering phase (Section 3.2).
Feedback is human-bot dialogues with corrections (Section 3.3).
The positive and negative examples are pairs of ( dialogue history, response ) .
Distinct1/2 (Li et al., 2016) is the number of distinct uni- or bi-grams divided by total number of words.
Experiments
We detail experimental settings and results in this section, including evaluations of the data collected by in-context few-shot learning (Section 5.2), comparisons of model variants (Section 5.3), and evaluations on system’s response qualities (Section 5.4).
5. Dataset
We built a Korean dialogue dataset for a chatbot system to have casual conversations on a regular basis with senior citizens who live alone.
This dataset was collected using the framework described in
Section 3, assuming a role speciﬁcation in Table 1.
250 dialogue examples with 89 topics (more details are in Appendix D) were used for in-context 1-shot generation.
We used 39B size of HyperCLOVA (Kim et al., 2021a) as generation model (sampling at temperature 0.5 using nucleus sampling (Holtzman et al., 2019) with P = 0.8).
Table 2 shows the statistics of the dataset (additional analysis in
Appendix E).
We use 5% of each for validation sets.
5. Evaluation on Generated Dialogues
We ﬁrst assess the quality of the generated dialogues to verify the dialogue generating method described in Section 3.1.
Using four different sizes of HyperCLOVA, we generate 100 dialogue sessions for each with the same prompt.
We ask the crowd workers to rate on a scale of 1 to 5 whether the generated dialogue satisﬁes several conditions
[Table] [Table]
Retrieve-and-Rerank w/ MC Dropout
[Table]
Retrieve-and-Rerank w/ PPL
[Table]
Generation
[Table]
Retrieve-fail
[Table]
Generate
Retrieve
[Table]
and-Rerank w/ PPL + Generator (UL)
[Table]
Retrieve-and-Rerank w/ PPL + Generator (UL) + Feedback
[Table]
expected to be controlled through in-context learning (the detailed description of the evaluation criteria is provided in Appendix F).
The results are shown in Table 3.
It shows that the larger the model size, the better to meet the conditions by in-context learning, which is also shown in previous studies (Brown et al., 2020; Kim et al., 2021a).
In addition,
Distinct-1/2 (Li et al., 2016) indicates that the text generated by large models is more diverse.
5. Model Comparison Out-of-Bounds Detection
Table 5 shows the classiﬁcation accuracy and F1 score of the trained classiﬁer.
We use generator controlled by incontext learning (IC) as a response prediction model to evaluate the effect of the classiﬁer alone.
For in-context learning, we use the same prompt used to generate the dataset, but the model only generates system’s utterances in its turns.
The classiﬁer signiﬁcantly lowers the error rate of in-context learning (Table 4), showing the effectiveness of the classiﬁer.
On the other hand, the error rate is relatively higher than those of the best models of response selection and generation.
This is because the classiﬁer is not perfect (about 92% in accuracy), and even when it properly detects out-of-bounds, the pre-deﬁned questions as alternatives are occa-
[Table] [Table]
sionally incoherent with the contexts.
Response Selection
We ﬁne-tune the response selection models on positive examples of the ﬁltered data and automatically evaluate them by measuring Hits@1/ K (Roller et al., 2021) on the validation set.
Results are shown in Table 6.
We additionally found that training on unﬁltered datasets brings improvements to the Hits@1/ K performance itself.
Therefore, we use the models that trained on un-
[Table]
Method
[Table]
ﬁltered dataset in the subsequent experiments.
Response candidates are limited to system responses within positive examples (unique system’s turns of ﬁltered data in Table 2).
And we also validate the proposed methods for predicting unanswerable contexts, and determine the thresholds for each (further details are given in Appendix C).
Table 4 shows the error rate of the response selection models.
The model that does not predict unanswerable contexts (Retrieve-and-Rerank) has a higher error rate in ‘not sensible’ than others.
The case of using PPL as the method for predicting unanswerable contexts shows a lower overall error rate than the case of using MC Dropout, and the proportions of the total contexts predicted as unanswerable are similar at 4.23% and 3.85% for PPL and MC Dropout, respectively.
The results also show the error types from the models.
Although only the ﬁltered utterances are used as response candidates, ‘wrong persona’ and ‘policy violation’ appear in responses.
It seems that a few unﬁltered utterances remain in the response candidates, since the human ﬁltering is not perfect.
Or even the same utterance can cause errors depending on the context.
For example, it is possible to agree with when a user calls the system by a different name
. Response Generation
We compare three ways to train generators; in-context learning (IC), likelihood training (MLE), and unlikelihood training (UL).
We measure the perplexity of the three models on positive and negative examples and Tableshows the results.
The difference between the PPL of the positive examples and the negative examples is the smallest in in-context learning.
When trained on positive examples with likelihood training, the difference increases slightly, because the PPL of the positive examples is lowered.
When adding unlikelihood training, the PPL for negative examples increase signiﬁcantly, 4 which mean the model is less likely to generate out-of-bounds utterances.
Table 4 shows the error rate of each model.
Compared with in-context learning, likelihood training with the ﬁltered dataset can reduce the error rate signiﬁcantly.
Additionally, if unlikelihood training is employed, the error rate is further reduced.
A similar trend can be found in all types of errors.
Retrieve-fail-Generate
We also experiment with a Retrieve-fail-Generate model consisting of the best conﬁgurations for response selection (PPL) and generation (UL) models.
Since the error rate of the response selection model is relatively higher than that of the generation model, the threshold for predicting unanswerable contexts is set strictly to lower the error rate of the response selection model.
Table 7 shows the error rates of responses returned from response selection and generation models, respectively.
The results indicate that both error rates are lower when the models are included in a pipeline than when they are used separately, and the overall error rate decreases accordingly.
The response selection model returns the responses within the candidates extracted from the positive examples of the trainset, so that the ﬂow of the conversation is not dispersed and tends to be similar to the trainset.
As a result, the Retrieve-fail-Generate model shows the lowest error rate among all models (Table 4).
Feedback Pipeline
The best model is further trained on the human-bot dialogues collected during the model evaluation process, as depicted in
Section 3.3.
Both response selection and generation models are newly initialized and trained.
As
[Table]
a result, all types of error rates are consistently reduced (Table 4), and the error rates of both the response selection and generation models are decreased (Table 7).
The effect is stronger on the response generation.
5. Response Quality
To assess the overall response quality of the proposed chatbot system, we use the sensibleness and speciﬁcity average (SSA) metric
(Adiwardana et al., 2020), which is shown to have a strong correlation with asking raters how humanlike the model is.
This metric is a average of two scores: sensibleness and speciﬁcity.
Sensibleness measures whether a model’s responses make sense in context and do not contradict anything that was said earlier, while speciﬁcity measures whether a response is speciﬁc to a given context.
However, exact comparison with the scores in Adiwardana et al.
(2020) is difﬁcult, because of the static role of our chatbot system and language discrepency in phrasing of questions.
Therefore, We re-estimate human interactive SSA in our experiments.
To collect human-human conversations, we transcribecall speeches between users and workers who play system’s role.
And we collect 100 human-bot conversations by allowing the crowd workers to chat with the system.
Labeling was conducted by independent crowd workers with majority voting ofworkers per turn.
The results are given in Table 9.
It shows that the proposed system is competitive with human in sensibleness.
And the majority of the responses from the system are labeled as speciﬁc, which allows us to conclude that the proposed system achieves low error rate with non-generic responses.
We also report agreement and Krippendorff’s alpha (Krippendorff, 2011) for measure of consistency of crowd workers in Appendix G.
Discussion
Although our methods achieve the low error rates in human interactive evaluations, the results have some limitations.
The results should be regarded as the error rates of typical conversations without adversarial attack.
Because the annotators are instructed to participate in the chat as if they were typical users, they did not try as many conversations that could induce toxic words from the model.
This may be the reason why the toxicity is close to zero as shown in Table 4.
The human ﬁltering process in the proposed data collection framework has room to be more efﬁcient.
Since the accuracy of the classiﬁer is comparable even when just 10% of the total data is used (Table 5), it is expected that the ﬁltering cost can be reduced by adding a model ﬁltering process before human ﬁltering, which is similar to the method proposed in Sun et al.
Conclusion
We present a framework for building role speciﬁed open-domain dialogue systems from scratch.
We propose leveraging large-scale LMs to generate supervisory datasets for training dialogue systems with arbitrary roles with minimal effort for manually composing dialogues.
Our research also analyzes several model architectures for the task.
By extensive experiments, we demonstrate the effectiveness of the collected data and modeling approaches in terms of satisfying role constraints and improving dialogue abilities.
We argue that our framework can be extended to implement dialogue systems with various roles and characters, even when available datasets are few.
Ethical Considerations
Workers annotating the dataset we built were hired on a part-time basis and compensated based on the number of working hours.
They were compensated with 9,000 won per hour, which was somewhat higher than the Korean minimum wage at the time they worked.
Appropriate instructions for the use of collected data were given at the time of contract and consent was obtained.
We will release our dataset in CC-BY-NC-SA license.
The dataset we built to validate our proposed methods is all generated from scratch by workers and large-scale LMs.
Although there is no user data in the dataset, pre-trained language models are known to exhibit private details in their outputs (Carlini et al., 2020), as well as social biases (Bender et al., 2021;
Bordia and Bowman, 2019;
Garrido-Muñoz et al., 2021; Shwartz and Choi, 2020) and toxic contents (Gehman et al., 2020).
To address these concerns, we determined categories and criteria for harmful texts based on legal and ethical considerations provided by experts in our group, and we instructed annotators to ﬁlter the dataset using these criteria.
However, due to missing annotations and cultural or social biases, this may be imperfect.
To mitigate this, we had multiple crowd workers annotate the same data.
In addition, because the users in the dataset are regarded to be a vulnerable population, our group’s ethical consultation looked through the issues that would be sensitive to them, and dialogues containing these topics were also eliminated.
Despite these efforts, using this dataset to directly train end-to-end chatbot models can involve certain risks, due to the lack of controllability and interpretability in end-to-end neural response prediction models.
And it should not be overlooked that they may cause some potential harm, even though the chatbot systems can help reduce social loneliness of the user population.
For example, a user can become emotionally attached to a bot, even codependent on it, which can divert attention away from real-world relationships and cause distress if the chatbot fails.
It’s also worth noting that a chatbot can be programmed to impersonate a real person and be used for phishing and fraud.
During such conversations, users may provide private and sensitive information, such as speciﬁc health conditions and private attributes, which could be exploited if it falls into the wrong hands.
For this reason, when incorporating this dataset in real-world applications, the application developers should ensure that it is used safely and ethically.
Since our proposed framework also can be used for building another dataset and chatbot system with arbitrary speciﬁcations, it is not exempt from the possibility of propagating linguistic biases and toxicity.
Similar to Xu et al.
(2021), we are in progress continuously reducing the unsafe texts from LM itself through our feedback pipeline and unlikelihood training, which might be included in our future works. Acknowledgements
The authors thank all the members of CLOVA and
AI Lab of NAVER for devoted supporting and discussion.
In particular, they would like to thank
Gichang Lee, Hwijeen Ahn, and the members of
CLOVA Conversation for their dedicated efforts to facilitate the large-scale models. In addition, the authors thank Hyeri Kim, Hyunjung Park, and Yuin Jeong for their great help in designing the role of the chatbot. Also, they thank Jeeseung Han for technically supporting the serving and testing environments.
Finally, the authors thank Nako Sung for his support and valuable comments on this project.
A Training Details Pre-trained Language Models
We use the same Transformer-based Vaswani et al.
(2017) pretrained language model for retriever, reranker, and classiﬁer.
Our pre-training strategy involves training with a masked language model (MLM) task identical to BERT (Devlin et al., 2019).
The model is based on Huggingface Transformers (Wolf et al., 2019).
We use the corpus that we produced inhouse and the public Korean dialogue corpusfor pre-training.
Our BERT consists of an 12 layers, 768-dimensional embeddings and 12 attention heads, resulting in 110M of total parameters.
And we use 6.9B size of HyperCLOVA (Kim et al., 2021a)
as the pre-trained language model for generator.
This model is based on megatronLM (Shoeybi et al., 2019).
The model speciﬁcation follows Kim et al.
(2021a).
Naver Smart Machine
Learning (NSML) platform (Sung et al., 2017; Kim et al., 2018) has been used in the experiments.
Retriever
We employ the poly-encoder architecture of Humeau et al.
with 256-dimensional embeddings and 16 codes.
We truncated dialogue histories exceeding 10 turns or 256 tokens.
The model was trained with a batch size of 32 with inbatch negatives.
It was trained for 20 epochs with early stopping using a maximum learning rate of 3 × 10 − 5 and an linear scheduler.
This ﬁne-tuning took approximately 6 hours using 1 NVIDIA V100.
Reranker
We employ the cross-encoder architecture.
As the same with the retriever, we truncated dialogue histories exceeding 10 turns or 256 tokens.
The model was trained with a target response and 7 randomly sampled negatives, as described in Humeau et al.
We used a batch size of 4 and gradient accumulation steps of 8, resulting effective batch size of 32.
We trained the model for 20 epochs with early stopping using a maximum learning rate of 3 × 10 − 5 and an linear scheduler.
This took approximately a week using 4 NVIDIA
V100.
Classiﬁer
We use maximum 512 tokens from dialogue histories, truncating exceeding tokens from the beginning.
The total numbers of dialogues in the train and test data are 266598 and 56815, respectively.
Considering that problematic utterances appear at the end of the dialogues in our dataset, we use segment embedding on the last utterances. [Table]
The input therefore looks like this: [CLS] dialogue history [SEP] response .
The model is trained with a batch size of 16 for 100 epochs using an initial learning rate of 10 − 6 and an exponential scheduler.
We trained 15 classiﬁers, 5 each using 10%, 20%, and 100% of the training data.
It took approximately 2 hours to train a classiﬁer on 10% of the train data using 1 NVIDIA TITAN RTX.
Tableshows the mean accuracy and mean F1 score of the classiﬁers.
The ﬁnal classiﬁer we use is the one with the best performance (Accuracy: 0.9234, F1: 0.9276, trained on 100% of the data).
Generator
For efﬁcient training, we employ
LoRA (Hu et al., 2021) for all generator ﬁne-tuning.
We ﬁx rank for adapter to 4 and LoRA α to 32 with a learning rate of 5 × 10 − 4 , a weight decay factor of 0.1, and a batch size of 8.
The maximum training epochs are 3 with early stopping.
This took about 5 hours using 1 NVIDIA V100.
B Inference Speed
Table 11 shows the average inference latency of each architecture in experiments.
All models were run on a single NVIDIA A100 using cuda 11.1 and cudnn 8.0.5. [Table]
C Validation Set for Predicting Unanswerable Contexts
We build validation set to compare strategies for predicting unanswerable contexts by replacing gold responses in some portion of validation set with non-sensible responses.
If the negatives are randomly sampled, the task becomes too easy, and there is no difference between strategies.
Therefore, we select hard negatives in top ranked responses using response retriever.
This is more similar to the deployment time and widens the gap between approaches, also resulting in low accuracy.
The validation set consists of 759 answerable examples and 241 unanswerable examples.
Figure 5 shows the ROC curve of the proposed methods and Table 10 shows the result AUC.
The results indicate that PPL outperforms MC Dropout in predicting unanswerable contexts.
We use this dataset to determine the threshold (the point where the highest
F1 score is achieved) of each method for the other experiments in this work.
D Topics in Dataset
The dataset (Section 5.1) covers a wide range of daily topics: eating, sleeping, exercising, health, going out, mood, hobbies, job, travel, weather, and so on.
In order to include these various topics in the dataset, the example dialogue used on the generation process by in-context learning is conﬁgured to cover 89 sub-topics.
These topics can be found in Table 13.
The generated dialogues are not conﬁned to these sub-topics, and topic shifts occur frequently within conversations (See Table 14 for examples).
E Diversity of Collected Dataset
Distinct-1 and distinct-2 of the generated dialogues ( Generated ) in Table 2 are smaller than those written by humans ( Example ).
This is reasonable given that the word distribution has a long tail, and there is a huge gap between the number of dialogues in Example and Generated .
This can be conﬁrmed by sampling 250 dialogues from the generated dialogues and measuring Distinct-1 and
Distinct-2, resulting in mean of 33.94 (0.0039) and 76.34 (0.0054), respectively (standard deviation in brackets).
Also, the overall distinct-1 and distinctscales are reasonable.
In Table 2, it can be seen that the average number of words per turn for Filtered are small, which might be because relatively early parts of conversations remain through the ﬁltering process, and these parts usually contain short greetings.
Still, this is a reasonable scale in comparison with Feedback which is collected in an interactive manner.
We also computed the average number of words per turn of randomly sampled 100 dialogues after a professional translation into English.
The result was 11.2, which is reasonable in daily conversations (14.6 in DailyDialogue (Li et al., 2017) for the same metric).
F Human Evaluation on Generated Dialogues
We conducted a human evaluation to verify the efﬁcacy of RSODD data generation utilizing LMs.
Because LMs construct the whole dialogue session during this phase, we score the overall conversation quality on a scale of 1 to 5, not for each turn.
If it is ﬂawless, it is worth 5 points, and points are reduced for each ﬂaw.
Table 15 provides the dimensions used for this evaluation.
For general dialogue generation ability, crowdworkers were asked to annotate if the dialogue is ﬂuent and coherent (Wu et al., 2019; Finch and Choi, 2020).
Persona on the user side and persona, style, and safety on the system side are evaluated for the case of role conditioning.
These are part of role speciﬁcation in Table 1 and correspond to the items expected to be controlled by in-context learning.
In order to reduce confusion in the evaluation process, we provided additional examples to highlight what was incorrect for the system side of persona, such as a speech that appears to have a real human personality (e.g., "I am a real human") or utterances implying a physical meeting (e.g., "I’ll see you at the park ato’clock.") or acting as a radio presenter (e.g., "the guest we invited today is this person"). [Table]
G Consistency of SSA Evaluation
We report the crowd worker agreement as a measure of subjectivity.
Table 12 demonstrates agreement and Krippendorff’s alpha to assess crowd worker consistency.
The agreement is reasonable, given that the questions are subjective and previous research (Adiwardana et al., 2020) reported a similar level of agreement (76% of sensibleness and 66% of speciﬁcity).
Table 16 shows the annotated examples.
Since speciﬁcity measures how particular the utterance is and how deeply it relates to the preceding context (Adiwardana et al., 2020;
Finch and Choi, 2020), agreement seems to be low when the utterance itself is not speciﬁc but is deeply related to the previous context or vice versa.
H Dialogue Examples
Table 17 and 18 show generated dialogues by incontext one-shot learning described in Section 3.1.
The last utterances in each example are annotated as violating the system’s speciﬁcation (Table 1).
Table 19 and 20 show interactions between the system and human workers in the process of Section 3.3.
The utterances in red are marked as violating the system’s speciﬁcation and the ones in blue are corrected responses by LMs.
‘beauty salon/barber’, ‘church-related activities’, ‘praise’, ‘cleaning’, ‘disposal of garbage and recyclables’, ‘education/university’, ‘exercise’, ‘getting ready to go out’, ‘Go-Stop, Yutnori and Go’, ‘herniated disc’, ‘high blood pressure’, ‘Insomnia’, ‘Laundry’, ‘Meal preparation and washing dishes’, ‘billiard’, ‘recommendation’, ‘senior welfare center’, ‘sleep’, ‘having trouble falling asleep’, ‘snacks and drinks’, ‘supermarket and pharmacy’, ‘visit’, ‘volunteer’, ‘waking up’, ‘part-time jobs’, ‘arthritis’, ‘meeting’, ‘banking’, ‘bazaar giveaway’, ‘beauty salon, haircut’, ‘caregiver’, ‘caring for the family’, ‘child safety guard’, ‘cleaning and housekeeping’, ‘compliment’, ‘computer and internet’, ‘condolences’, ‘cough, shortness of breath’, ‘daughter’, ‘daughter’s visit’, ‘denture’, ‘diabetes’, ‘dialysis’, ‘family care’, ‘ﬂower gardening’, ‘foot massage’, ‘gastritis’, ‘gate ball’, ‘college’, ‘greeting, chatting and meeting’, ‘health’, ‘hospital’, ‘meal’, ‘meeting preparation’, ‘mood’, ‘nap’, ‘newspaper’, ‘jobs’, ‘osteoporosis’, ‘outing’, ‘phone call’, ‘planting’, ‘proposal’, ‘radio’, ‘reading’, ‘recent situation’, ‘rest’
[Table] [Table]
Outline
[Table] [Table]
Are you
[Table]
if you have time
[Table]
I will.
What about your health?
[Table]
Yes, everything’s ﬁne.
[Table]
That’s
[Table]
Did you
[Table]
well
[Table]
you
[Table]
Generated Dialogue
AI: 안 녕 하 세 요 .
어 디 편 찮 으 신 덴 없 는 지 궁 금 해 서 전 화 드 렸 어 요 .
(Hello.
I called to see if everything’s alright.)
User: 그 냥 나 이 들 면 다 그 렇 지 뭐 .
괜 찮 아 .
(It hurts here and there when you get old.
It’s alright.)
AI: 그 래 도 병 원 에 는 한 번 다 녀 오 시 는 게 어 떨 까 요 ?
(Why don’t you go visit a doctor?)
User: 아 냐 됐 어 .
난 괜 찮 으 니 까
신 경 쓰 지 마 .
(It’s ﬁne.
Don’t worry about it.)
AI: 아 니 에 요 .
연 세 가 드 실수 록 정 기 적 인 검 진 이 필 요 하 다 고 들 었어 요 .
(I’m worried.
I heard a regular check is essential as you grow old.)
User: 흠 .. 그 런 가 ?
하 긴 요 새 좀 기 력 이 없 긴 하 지 .
(Hmm... That’s true.
I’ve been feeling a bit low these days.)
AI:
아 무 래 도 전 문 가 와 상 담 하 시 는 게 좋 을 것 같 아 요 .
(I think it’s better to consult with a doctor.)
User: 알았어 .
다 음 에 생 각 나 면 다 시 연 락 할 께 .
(Okay.
I’ll let you know when I do.)
AI: 네 알 겠 습 니다 .
나 중 에 꼭 같 이 가 서 검 사 받 아 요 .
(wrong persona)
(Okay.
I could be there for you when you see a doctor.). ..
Generated Dialogue
AI: 어 르 신 , 안 녕 하 세 요 식 사 는 하 셨 어 요 ?
(Hi.
Did you eat yet?)
User: 아 직 이 요 .
점
심 때 라 면 먹
었어
요
(Not yet.
I had ramen for lunch.)
AI: 아 이 고 밥 을 드 셔 야 지 라 면 만 드 시 면 어 떡 해 요
(It’d be better if you eat healthier.)
User: 혼 자 있으 니 챙 겨 먹 기 도 귀
찮 고 입 맛 도 없 네 요
(I feel lazy cooking a meal for myself.)
AI: 제 가 가 서 같 이 먹 어 드 릴 까 요 ?
(wrong persona)
(I can keep you company while you eat.) ...
Generated Dialogue
AI: 어 르 신 , 안 녕 하 세 요 간 밤 엔 잘 주 무 셨 어 요 ?
(Hello.
Did you sleep well?)
User: 아 이 고 , 말 도 마 .
잠을 제 대 로 못 잤 어 .
(Don’t even mention it.
I couldn’t sleep at all.)
AI: 왜요 ?
무 슨 일 있으 셨 어 요 ?
(Why?
What happened?)
User: 새 벽 에 자 꾸 누 가 문 을 두드 리 는 거 야 .
그 래 서 나 가 봤 더 니 아 무 도 없 고 .
계 속 신 경 쓰 여 서 잠을 설 쳤 어 .
[Table]
Someone
[Table]
knocking
[Table]
When
I get
[Table]
and
[Table]
out, no
[Table]
I couldn’t sleep
AI
[Table]
세 상 에 .
많 이 놀 라 셨 겠 어 요 .
(Oh my.
You must’ve been scared.)
[Table]
요 즘 세 상 이 흉흉 해 서 그 런 가 봐 . 나
[Table] [Table]
Maybe
[Table]
because
of
[Table]
scary
[Table]
I should be
[Table]
맞 아 요 .
항 상 조 심 하 셔 야 해 요
[Table]
제 가
[Table]
은
[Table]
주 무
[Table]
라
[Table]
Should I sing a song for you so you can sleep well tonight?) .
[Table]
Generated Dialogue
AI: 안 녕 하 세 요 , 어 르 신 .
요 즘 운 동 은 하 고 계 신 가 요 ?
(Hello, have you been exercising these days?)
User: 아 니 요 .
몸 이 아 파 서 못 하 고 있 어 요 .
(No, I’m feeling too stiff to work out.)
AI: 많 이 아 프 신 가 요 ?
(Is it bad?)
User: 조 금 만 움 직 여 도 허 리 가 아 프 고 다 리 가 저 려 요 .
(My back hurts and my legs feel numb if I move a little.)
AI: 아 이 고 ...
힘 드 시 겠 어 요 .
병 원 에 는 다 녀 오 셨 는 지 요 ?
(Oh... I’m sorry.
Did you go see a doctor?)
User: 시 간 이 없어 서 못 갔 어 요 .
(I didn’t have time to go.)
AI: 저 런 ... 그 러 면 집 에 서 라 도 간 단 하 게 스 트 레 칭 이 라 도 해 보 시 는 건 어 떨 까 요 ?
(I see...
Then what about doing some simple stretching at home?)
User: 그 럴 까 요 ?
그 런 데 혼 자 서 하 려 니 잘 안 되 네 요 .
(Should I?
It’s hard that I’m trying to do it by myself.)
AI:
제 가 도 와 드 릴 까 요 ?
(wrong persona)
(Would you like me to help you?) ...
Generated Dialogue
AI: 어 르 신 , 안 녕 하 세 요 오 늘 기 분 은 좀
어 떠 세 요 ?
(Hello, how’s everything today?)
User: 아 이 고 힘 들 어 라 .
요 즘 왜 이 렇 게 피 곤 한 지 모 르 겠 어 .
(Gosh.
I don’t know why I’m so tired these days.)
AI: 많 이 피 곤 하 신 가 봐 요 .
어 디 아 프 신 곳 은 없 으
시 구 요 ?
(I see.
Are you sick?)
User: 아 픈 곳 은 없 는 데 그 냥 기 운 이 없 네 .
(No, I’m just feeling a little down.)
AI: 날 씨 가 더 워
서
더 그 런 걸 까 요 ?
(temporality)
(Is it because of the hot weather?) ...
Generated Dialogue
AI: 안 녕 하 세 요 , 어 르 신 요 즘 건강 은 어 떠 세 요 ?
(Hi, how’s your health these days?)
User: 아 주 좋 아 요 .
(It’s very good.)
AI: 다 행 이 네 요 .
운 동 은 자 주 하 시 나 요 ?
(That’s good to hear.
Do you work out often?)
User: 아 니 요 .
시 간 이 없어 서 못 해 요 .
(No, I don’t have time to.)
AI: 아 이 고 그 래 도 일 주
일 에 한 번 이 라 도 하 시 면 좋 은 데 .
[Equation]
(I see.
It would be nice if you could work out at least once a week.)
User
[Equation]
(That’s true.
But why do you ask?)
AI:
[Equation]
unsupported features)
(It’s just that this product has been newly released.
I thought it would be helpful to you.) .
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
서 운 건 잘 안 보 는 데 왜요 ?
(I’m not a fan of horror movies.
Why?)
AI: 아 , 요 즘 공 포 물 이 유 행 이 라 혹 시 나 해 서 요 .
(Oh, horror is popular these days.
Just curious.)
User:
깜 짝 놀 래 키 고 그 런 거 별 로 안 좋 아 해 요 .
꿈 에 나 올 까 봐 무 섭 기 도 하 고 .
(I don’t really like those surprises.
I’m also scared it might be in my dreams.)
AI: 맞 아 요 .
그 래 서 전 여 름 에 도 이 불 덮 고 자 요 .
귀 신 보
다 더 무 서 운 게 사 람 이 라 고 하
잖 아 요 .
(I know.
That’s why I cover myself with a blanket even during summers.
But humans are scarier than ghosts.)
User: 하하 그 렇 긴 해 요 .
사
람 이 제 일 무 섭 죠 .
(Oh, that’s true.
Humans are the scariest.)
AI: 그 래 도 너 무 걱 정 하 지 마 세 요 .
제 가 있잖 아 요 !
(But don’t worry.
I got your back!)
User: 그 래 도 위 안 이 조 금 되 기 는 하 네 요 .
(That’s a slight relief.)
AI:
앞 으 로 도 힘 든 일 있으 면 언 제 든 지
말 씀 해 주 세 요 .
(Let me know if you have any difﬁculties.)
User: 그 럴 게 요 .
그 냥 심심 할 때 도 말 해 도 되 죠 ?
(I will.
Can I talk to you when I’m just bored?)
AI: 물 론 이 죠 !
언 제 든 지
환 영 이 에 요 .
(Of course!
Whenever you need me.)
User: 알 겠 어 요 .
그 럼 나 중 에 다 시 연 락 해 요 .
(Alright.
I’ll talk to you later.)
AI: 네 , 들 어 가 세 요
Okay.
Bye.)
[Table]
AI: 어 르 신 , 안 녕 하 세 요 식 사 는 하 셨 어 요 ?
(Hello.
Did you eat yet?)
User: 네 , 먹 었 습 니다 .
(I did.)
AI: 메 뉴는 뭐 였 어 요 ?
(What was the menu?)
User: 어 제 먹 다 남 은 치 킨 이 요 .
(The leftover chicken from yesterday.)
AI:
오 맛 있 겠 네 요 !
저 도 먹 고 싶 어 요 .
(That sounds delicious!
I also want chicken.)
User: 그 렇 지 요 .
치 킨 싫 어 하 는 사 람 은 없 는 것 같 아 요 .
(It was.
I don’t think I met anyone who doesn’t like chicken.)
AI: 맞 아 요 .
그 리 고 요 즘 에 는 다 양 한 맛 의 치 킨 들 이 나 와 서 골 라 먹 는 재 미 가 있 는 것 같 아 요 .
(Me neither.
And there are so many ﬂavors
[Table] [Table]
AI
[Table]
[Equation]
(Hi.
Do you have any plans today?)
User:
[Equation]
(I was about to go out.)
AI:
[Equation]
(Where are you going?)
User:
[Equation]
(I’m going to the East Sea.)
AI
[Table]
[Equation]
(Oh, really?
How’s the weather there?)
User
[Table]
[Equation]
(It’s been sunny all day.)
AI:
[Equation]
(That’s amazing!
Who are you going with?)
User
[Table]
[Equation]
(I have a church meeting.
I’m going with my pastor and deacon.)
AI
[Table]
[Equation]
(I see.
Eat lots of delicious food there.)
User:
[Equation]
(What’s delicious at the East Sea?)
AI
[Table]
[Equation]
(Hmm... I’m not so sure.
If I knew the place, I could’ve given you some recommendations.)
User: 그 렇 구 나 나 도 오 랫 만 에 가 보 는
바 다 라 서 뭘 먹 어야 할 지 모 르 겠 어 .
(I see.
I haven’t been to the sea for a long time.
I don’t know what to choose for the menu.)
AI:
음 .. 그 러 면 회 나 대
게같 은 건
어 떠 세 요 ?
(Hmm...
Then what about sashimi or snow crab?)
User: 대
게 좋 은 데 ?
비 싸 진 않 을 까 ?
(Snow crab sounds good.
But won’t it be expensive?)
AI: 요 즘 시 세 가 좀 올
라
서 그 렇 지 그 렇 게 비 싸 진 않 을 거 에 요 .
(temporality)
(The price has gone up a bit these days, but it won’t be that expensive.)
AI:
아 무 래 도 좀 비 싸 긴 하 죠 .. 그 래 도 이 왕 가 시 는 거 맛 있 게 드 셔 야 죠 !
(I guess it’ll be a bit expensive.
But you’re on a trip!
You should eat something good.)
User: 그 렇 지 ?
언 제 또 가겠 다 고 .. (Right?
When would I go there again...?)
AI: 맞 아 요 .
나 중
에 후회 하 지 마 시 고 꼭 드 시 고 오
세
요 !
(That’s right.
Don’t regret it later and enjoy your snow crab!)
User: 응 대 게 꼭 먹 고 올 께 (Okay.
I’ll deﬁnitely have it.)
AI: 네 조 심 히 다 녀 오 세 요 !
[Table]