Venue: NAACL 2022

# Less is More: Learning to Refine Dialogue History

for Personalized Dialogue Generation

Hanzun Zhong\({}^{1}\), Zhicheng Dou\({}^{2,}\) , Yutao Zhu\({}^{3}\), Hongjin Qian\({}^{2}\), Ji-Rong Wen\({}^{2}\)

\({}^{1}\)School of Information, Renmin University of China, Beijing, China

\({}^{2}\)Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China

\({}^{3}\)University of Montreal, Quebec, Canada

{hanxun_zhong,dou,ian}@ruc.edu.cn

{yutaozhu94,jirong.wen}@gmail.com

Corresponding author.

###### Abstract

Personalized dialogue systems explore the problem of generating responses that are consistent with the user's personality, which has raised much attention in recent years. Existing personalized dialogue systems have tried to extract user profiles from dialogue history to guide personalized response generation. Since the dialogue history is usually long and noisy, most existing methods truncate the dialogue history to model the user's personality. Such methods can generate some personalized responses, but a large part of dialogue history is wasted, leading to sub-optimal performance of personalized response generation. In this work, we propose to refine the user dialogue history on a large scale, based on which we can handle more dialogue history and obtain more abundant and accurate persona information. Specifically, we design an MSP model which consists of three personal information refiners and a personalized response generator. With these multi-level refiners, we can sparsely extract the most valuable information (tokens) from the dialogue history and leverage other similar users' data to enhance personalization. Experimental results on two real-world datasets demonstrate the superiority of our model in generating more informative and personalized responses.

+
Footnote †: July 10-15, 2022 ©2022 Association for Computational Linguistics

## 1 Introduction

Recent years have witnessed great progress in building personalized dialogue systems. In general, previous work explores building a personalized dialogue system via two pathways: (1) directly modeling user personality from predefined persona descriptions or user attribute Qian et al. (2018); Zhang et al. (2018); Song et al. (2019); and (2) implicitly modeling the user personality from the user's dialogue history Li et al. (2016c); Ma et al. (2021). The latter is considered superior as the dialogue history is easy to obtain and comprises rich personalized information. In this paper, we follow the second pathway that automatically learns implicit user profiles from the user's dialogue history to assist in personalized response generation.

It is challenging to model user personality directly from the dialogue history. The main reason is that a user's dialogue history may contain massive historical dialogues, which are too heavy to load in the model and likely to be noisy. A straightforward solution is to truncate the dialogue history, as done by existing work Ma et al. (2021); Qian et al. (2021). However, as tremendous information has been wasted, the model's performance is also influenced. On the other hand, we observe that the dialogue history from other users may also be helpful in generating a personalized response for the current user. For example, users with the same interest in "soccer" may talk about similar things on such a topic. This has been overlooked by existing methods. Intuitively, the problem of "data explosion" is even more severe in the latter case (when other similar users' dialogue history is also considered). To alleviate these problems, we propose using a hierarchical refiner structure to sparsely extract the most valuable query-aware persona information from both the current and other similar users' dialogue history. By this means, more dialogue history can be utilized for learning user personality and improving response generation.

Our model is called **MSP**, which stands for **M**odeling and **S**electing user **P**ersonality from the dialogue history for generating personalized responses. Instead of attending to all dialogue history, MSP refines the most valuable historical information that can well portray the user's personality and guide the response generation. Specifically, MSP consists of three personal information refiners working at different levels and a personalized response generator. At **first**, a user refiner is designed to select a group of users who have similar interests to the current user. By refining dialogue history at the user level, we can obtain similar data

to share information with similar users and avoid mutual interference with other users. **Then**, a topic refiner filters out the current and similar users' dialogue history that has different topics with the current query at the sentence level. **Next**, we design a token refiner to extract the most valuable query-aware user profiles from the remaining dialogue history at the token level. **Finally**, a personalized response generator combines user profiles and the current query to generate responses. Given that there is no explicit supervisory signal guiding the refiner to extract an exemplary user profile, we design a supplementary sentence matching task and a joint training method. The generator will construct a pseudo-label to guide the refiner's extraction.

Our contributions are three-fold:

(1) We design an MSP model to tackle the data noise problem. It can efficiently refine user profiles through dialogue history and generate personalized responses. By this means, our method can capture abundant user profiles while keeping away from noisy data.

(2) We design a refiner structure to extract the query-aware profile at three levels. Similar users' information is taken into account, which can help improve the personality of the response.

(3) We design a joint training method for the refiner and generator. The refiner provides the generator with user profiles to assist in generating responses, while the generator constructs a pseudo-label for the refiner to assist in selecting user profiles.

## 2 Related Work

Personalized Dialogue GenerationOpendomain dialogue generation has been extensively studied Koehn et al. (2003); Vinyals and Le (2015); Serban et al. (2016); Zhang et al. (2019a, b); Liu et al. (2020); Xiao et al. (2020); Zhu et al. (2020). Recently, personalized dialogue systems have attached more and more attention. Typical methods include: (1) explicitly using predefined persona descriptions or attributes as users' profile to generate personalized responses Qian et al. (2018); Zhang et al. (2018); Olabiyi et al. (2019); Song et al. (2019); (2) using user ID embeddings to enhance personalized dialogue generation Li et al. (2016c); Chan et al. (2019); and (3) extracting implicit user profile from users' dialogue history to generate personalized responses Al-Rfou et al. (2016); Bak and Oh (2019); Ma et al. (2021). Since manually collecting user profiles is impractical for large-scale datasets and the user ID embeddings perform badly, in this study, we focus on the last group of methods for personalized response generation.

DHAP Ma et al. (2021) is the state-of-the-art method in personalized dialogue generation. It uses a transformer-based structure to model the user's dialogue history and extract personal information for response generation. Unfortunately, this model can only handle a limited number of user dialogue histories, wasting a lot of valuable information. Our method has two main differences from DHAP: (1) We propose a refiner structure in our model so that more dialogue history can be handled and the most valuable information can be extracted for improving response generation; (2) With our proposed refiner, we can further incorporate more dialogue history from other users (having similar interests) to facilitate personalized dialogue generation.

Retrieval-guided Natural Language Generation Retrieval-based methods can collect relevant information for language generation Yang et al. (2019). It has been widely applied in many tasks such as text style transfer Li et al. (2018) and dialogue generation Wu et al. (2019); Cai et al. (2019). The idea of using a retrieval system to get useful information inspires our study. We use a refiner to automatically extract personal information from dialogue history and guide the personalized generation.

## 3 Methodology

In this section, we first formalize our problem and provide an overview of our proposed MSP model. Then, we describe the details of each component and the model optimization.

### Problem Statement and Overview

Considering a set of users \({\cal U}=\{u_{1},\cdots,u_{l}\}\), for any user \(u_{i}\), we have their dialogue history with others \(U^{i}=\{(q_{1}^{i},r_{1}^{i}),\cdots,(q_{t}^{i},r_{t}^{i})\}\), where \(q_{j}^{i}\) is a _query_ issued by others, while \(r_{j}^{i}\) is the corresponding _response_ given by \(u_{i}\).1 Our target is to generate a personalized response \(r^{i}\) for the user \(u_{i}\) to reply a new query \(q\). As we introduced earlier, the personalized information can be obtained from the dialogue history \(U^{i}\) of the user \(u_{i}\) and other dialogue history \(U^{j}\) from similar users \(u_{j}(j\neq i)\) 

The overview of our MSP model is shown in Figure 1. MSP consists of three personal information refiners working on different levels and a personalized response generator. Specifically, the first refiner is working at the user level. By comparing the dialogue history of the current user \(u_{i}\) with others, MSP can select a group of users having similar interests with \(u_{i}\). After obtaining a group of similar users, we further refine their dialogue history according to the relevance to the current query's topic. Moreover, we add the last refiner to extract several tokens so that the most fine-grained personal information can be extracted from the relevant utterances. Finally, the query and the extracted tokens are fed into the generator and construct a personalized response.

### User Refiner

The dialogue history of users with similar interests may share much personal information. Therefore, our first target is to select a group of users with similar interests to the current user. We design a user refiner to achieve this. Since the users' interest is usually contained in their dialogues with others, we consider both the queries and responses in the dialogue history to select similar users. Specifically, for the user \(u_{i}\)'s dialogue history \(U^{i}\), we apply a pre-trained BERT Devlin et al. (2019) and represent them by the embedding of [CLS] token:

\[\mathbf{U}_{q}^{i}=\sum_{j=1}^{t}\mathrm{BERT}(q_{j}^{i}),\quad\mathbf{U}_{r}^ {i}~{}~{}=\sum_{j=1}^{t}\mathrm{BERT}(r_{j}^{i}).\]

Then, we can select \(k_{u}\) users that have similar interest with the current user \(u_{i}\) as:

\[u^{\text{sim}} =\operatorname{TopK}(\mathbf{U}^{i}\cdot\mathbf{U}^{j},k_{u}), \tag{1}\] \[\mathbf{U}^{i} =[\mathbf{U}_{q}^{i};\mathbf{U}_{r}^{i}], \tag{2}\]

where \(\operatorname{TopK}(\cdot,\cdot)\) is the top-\(k\) selection operation.

After the user refiner, we can obtain the dialogue history of the similar users \(\{u_{j}\}_{j=1}^{k_{u}}\). It is worth noting that, since the number of users is large in the datasets, we choose to use the dot-product to compute the similarity of the users so that the whole process can be implemented by dense retrieval libraries, such as Faiss Johnson et al. (2021), which is very efficient.

### Topic Refiner

The users' dialogue history often contains many dialogues with others. These dialogues have various topics, which may be irrelevant to the current query. Therefore, we propose a topic refiner to select relevant dialogue history for personalized response generation. Specifically, we use a topic classifier to compute the topic distribution of the current query

Figure 1: The overview structure of the proposed model which consists of four modules: (1) user refiner, (2) topic refer, (3) token refiner, and (4) generator.

 

\(q\) and the queries \(q_{j}^{i}\) in the history dialogues:

\[t=\text{MLP}(\mathrm{mean}(\mathrm{BERT}(q)), \tag{3}\] \[t_{j}^{i}=\text{MLP}(\mathrm{mean}(\mathrm{BERT}(q_{j}^{i})), \tag{4}\]

where \(t,t_{j}^{i}\in\mathbb{R}^{d^{t}\times 1}\), and \(d^{t}\) is the number of topic. Then, we filter out the dialogue history \(<q_{j}^{i},r_{j}^{i}>\) that has different topics with the current query, _i.e._, \(\max(t_{j}^{i})\neq\max(t)\).

In the topic refining process, we compare the queries in the history dialogues with the current query to filter out topic-irrelevant dialogues. This process can further reduce the noise and make our model more lightweight. Both the dialogue history of the current user and that of the similar users (obtained in the former step) are refined. In the next step, we will use the **responses** in these selected history dialogues and extract the most valuable tokens for personalized response generation.

### Token Refiner

After the previous two refiners, we obtain a collection of historical responses. Though we can directly add them into the generation process, our preliminary experiments indicate that they perform poorly. A major reason is the noisy quality of the responses. Indeed, existing studies Xing et al. (2017); Zhu et al. (2020) have demonstrated the effectiveness of using informative tokens to improve the response generation. Inspired by these studies, we further devise a token refiner to extract the most fine-grained information (tokens) from the historical responses. Specifically, we compute an attention map \(\mathbf{A}\) between the query \(q\) and the historical responses \(r^{\text{sim}}\) and \(r^{\text{cur}}\) (they are from the similar users and the current user respectively) as:

\[\mathbf{A} =\operatorname*{softmax}\left(\frac{\mathbf{Q}\mathbf{K}^{T}}{ \sqrt{d}}\right), \tag{5}\] \[\mathbf{Q} =\text{TRM}_{\text{enc}}(q)\cdot\mathbf{W}_{Q},\] (6) \[\mathbf{K} =\text{TRM}_{\text{enc}}(r)\cdot\mathbf{W}_{K}, \tag{7}\]

where TRM\({}_{\text{enc}}(\cdot)\) is a transformer encoder. \(r\) refers to \(r^{\text{sim}}\) or \(r^{\text{cur}}\), and correspondingly, \(\mathbf{A}\) refers to the similar user matching map \(\mathbf{A}^{\text{sim}}\) and current user matching map \(\mathbf{A}^{\text{cur}}\). \(\mathbf{W}_{Q},\mathbf{W}_{K}\in\mathbb{R}^{d\times d}\) are parameters, and \(d\) is the dimension of the hidden state. After obtaining the attention matching map \(\mathbf{A}\), we select tokens to form the similar users' profile and current user's profile according to each token's attention weight:

\[\mathbf{c}^{sim} =\mathrm{TopK}(\mathrm{Max}(\mathbf{A}^{\text{sim}}),k_{p}), \tag{8}\] \[\mathbf{c}^{per} =\mathrm{TopK}(\mathrm{Max}(\mathbf{A}^{\text{cur}}),k_{p}), \tag{9}\]

where \(k_{p}\) is a hyper-parameter to control the number of profile tokens. Its influence will be investigated in our experiments.

### Generator

We use a transformer decoder as to generate a personalized response by using the similar users' profile \(\mathbf{c}^{\text{sim}}\), current user's profile \(\mathbf{c}^{\text{cur}}\), and query information \(\mathbf{q}\) as input. The decoding process can be defined as:

\[\hat{y} =\text{TRM}_{\text{dec}}(\mathbf{x}), \tag{10}\] \[\mathbf{x} =[\mathbf{c}^{\text{sim}};\mathbf{c}^{\text{per}};\mathbf{q}], \tag{11}\]

where \([;]\) is the concatenation operation and \(\hat{y}\) is the word generation probability.

### Training and Optimization

The generator is optimized by maximizing the generation probability of the ground-truth \(y\):

\[\mathcal{L}^{\texttt{g}}=-y\log\hat{y}. \tag{12}\]

In practice, we find that the token refiner is hard to train. We speculate the reason is a missing of direct supervision signals. In this case, it is difficult to tell whether the training errors stem from the generation process or the refining process. To tackle this problem, we propose a supplementary sentence matching task to assist the token selection.

Supplementary Sentence Matching TaskThe core idea of this task is to train the token refiner directly by introducing supervision signals so that it can automatically mine valuable tokens. Specifically, we design a sentence matching task to match the query with the dialogue history. The task's target is to find the history sentences that help generate personalized responses. We consider using the query-history cross attention weight \(\mathbf{A}\) to generate a matching representation and then use this representation to finish the task. In this way, once the matching task has been well-finished, we can use attention map \(\mathbf{A}\) to identify the most informative and valuable parts of a history sentence that are helpful to generate a personalized response.

To achieve our idea, we design a matching process. First, we calculate the matching representations \(\mathbf{H}\) by the cross-attention map \(\mathbf{A}\) and then 

apply a CNN with a max-pooling operation to aggregate token information:

\[{\bf S} =\mathrm{Maxpool}(\text{CNN}({\bf H})) \tag{13}\] \[{\bf H} ={\bf A}\cdot{\bf V},\] (14) \[{\bf V} =\text{TRMenc}(r)\cdot{\bf W}_{V}. \tag{15}\]

Next, we flatten \({\bf S}\) and applies a LSTM to aggregate the sentence information:

\[{\bf h}=\text{LSTM}(\operatorname{Flatten}({\bf S})). \tag{16}\]

Finally, we use the sentence matching vector \({\bf h}\) to compute the matching score:

\[\hat{g}=\mathrm{Sigmoid}(\text{MLP}({\bf h})). \tag{17}\]

For guiding the sentence matching task, we design a pseudo-label \(g\) to measure the matching goodness of each history sentence. We expect the history with more persona profile information can achieve a higher score. Therefore, we use the difference between the personalized ground-truth \(y\) and a non-personalized generated response \(\hat{y}^{\prime}\) to measure the persona profile and create the pseudo-label:

\[g =\begin{cases}1,&g_{\rm soft}\geq\alpha\\ 0,&g_{\rm soft}<\alpha,\end{cases} \tag{18}\] \[g_{\rm soft} =\mathrm{Sum}(\mathrm{Max}((y-\hat{y}^{\prime})\cdot r))/d_{ \text{y}}, \tag{19}\]

where \(d_{\text{y}}\) is the length of \(y\), and \(\alpha\) is a threshold. Finally, we minimize the binary cross entropy loss between \(g\) and \(\hat{g}\):

\[\mathcal{L}^{s}=g\log\hat{g}+(1-g)\log(1-\hat{g}). \tag{20}\]

Joint TrainingTo facilitate the learning with the above gradient approximation approach, we design a joint training process to train the refiner and generator in turn. Specifically, in each training iteration, we first sample a batch of query \(q\), response \(y\), similar and current users' dialogue history \(r^{\text{sim}}\) and \(r^{\text{cur}}\) from dataset \(D\). Then, we generate a non-personalized response \(\hat{y}^{\prime}\) and create the pseudo-label \(g\) (Eq. 18) through a non-personalized generator. This pseudo-label is used to train the token refiner by optimizing the loss \(\mathcal{L}^{s}\) (Eq. 20). Further, we sample another batch \(D^{p}\) from \(D\). After extracting similar user profile \(\mathbf{c}^{\text{sim}}\) and current user profile \(\mathbf{c}^{\text{cur}}\) (Eq. 8, Eq. 9), we generate the personalized response \(\hat{y}\) and update the generator by optimizing the loss \(\mathcal{L}^{\text{g}}\) (Eq. 12). To avoid misleading the generation (by poor profile) at the beginning of the training process, we pre-train the refiner for \(N_{f}\) steps before extracting the profile for the generator. The detailed training process is summarized in Algorithm 1.

```
Input:\(M\) dialogue triplets:\(D=\{(q,y_{i},r_{i})\}_{i=1}^{M}\) Output:A personalized dialogue model
1:Init the refiner and generator module
2:while not converge do
3: Sample \(n_{s}\) dialogue triplets \(D^{q}=\{q_{i},y_{i},r_{i}\}_{i=1}^{n_{s}}\)
4: Get \(\hat{Y}^{q}=\{\hat{y}^{\prime}_{i}\}_{i=1}^{n_{s}}\) on \(D^{q}\) from \(p_{g}\left(\hat{y}^{\prime}|q\right)\)
5: Get pseudo-label \(g\) on \(D^{q}\)
6: Train refiner by optimizing \(\mathcal{L}^{s}\) on \(D^{q}\)
7:if Current Step >\(N_{f}\)then
8: Sample \(n_{d}\) dialogue triplets \(D^{p}=\{q_{i},y_{i},r_{i}\}_{i=1}^{n_{d}}\)
9: Extract \(C^{p}=\{c_{i}\}_{i=1}^{n_{d}}\) on \(D^{p}\) from \(p_{h}\left(c|q,r\right)\)
10: Train generator by optimizing \(\mathcal{L}^{o}\) on \(D^{p}\cup C^{p}\)
11:endif
12:end while
```

**Algorithm 1** Joint Training Process

## 4 Experiments

### Datasets

To evaluate our model's performance, we conduct experiments on a Chinese Weibo dataset Qian et al. (2021) and an English Reddit dataset.2 Both are collected from open-domain social media platforms. On these platforms, users can post various topics, and other users can respond to them. We compare user-id and timestamps to associate the query with its corresponding response and the current user's dialogue history. As a result, each training sample contains a query, a response, and a sequence of dialogue history. Finally, the dataset is divided into training, validation, and test sets in chronological order. The statistics of the datasets are provided in Table 1.

Footnote 2: Weibo: [https://www.weibo.com/](https://www.weibo.com/); Reddit: [https://www.reddit.com/](https://www.reddit.com/)

### Baseline Methods

We compare our proposed model with eight highly correlated and strong baselines. They can be categorized into four groups:

Non-personalized Methods(1) Seq2SeqAttention Sutskever et al. (2014) is a vanilla sequence-to-sequence model with attention mechanism Luong et al. (2015). (2) MMI Li et al. (2017)

\begin{table}
\begin{tabular}{l c c} \hline \hline  & **Reddit** & **Welbo** \\ \hline \# Users & 78,031 & 46,973 \\ Avg. history length & 72.4 & 30.8 \\ Avg. \# words of query & 19.8 & 22.9 \\ Avg. \# words of response & 9.1 & 9.6 \\ \# Training samples & 5,734,129 & 1,495,149 \\ \# Validation samples & 10,000 & 10,000 \\ \# Testing samples & 10,000 & 10,000 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Statistics of Reddit and Weibo datasets.

 

2016a) is based on Seq2seq and use maximum mutual information as an extra loss to improve diversity. (3) DialoGPT Zhang et al. (2019b) is a variant of GPT-2 Radford et al. (2019) designed for dialogue generation.

Predefined Profile-based MethodsSince there is no persona description in the datasets, we test these methods by using the user's dialogue history as a simulation of predefined persona profiles. (4) GPMN Zhang et al. (2018) enhances the Seq2seq model with a memory module, which encodes and stores the persona profile as memory representations. (5) PerCVAE Zhao et al. (2017) encodes predefined personalized sentences as a conditional representation and uses CVAE to generate a personalized response.

User ID-based MethodsSpeaker Li et al. (2016c) is based on seq2seq while using user-ID embedding as user representation to facilitate the response generation. (7) Persona WAE Chan et al. (2019) uses MAE Wasserstein autoencoder for response generation. It maps user-ID embeddings to a personalized Gaussian mixture distribution and then samples the personalized vector to guide the response generation.

User Dialogue History-based MethodsDHAP Ma et al. (2021) uses history memory to store and construct the dynamic query-aware user profile from dialogue history and then uses a personalized decoder to generate a response. Since this model also learns the user profile directly from the dialogue history, it is the most relevant baseline of our method.

### Implementation Details

We experiment with multiple sets of hyperparameters to select the best model, and the final hyperparameters are listed as follows: The dimensions of the embeddings and Transformer hidden units are 768. The number of heads in the Transformer is 12. The number of layers in the Transformer is set as 2 and 12 respectively in the query encoder and decoder. The topic number is 15, and the similar user number is set as 10. The selected profile token number is 200 for the Weibo dataset and 30 for the Reddit dataset. The batch size is 128. Following Holtzman et al. (2020), we adopt nucleus sampling as our decoding strategy. We use the Adam Kingma and Ba (2015) optimizer for training the refiners and AdamW Loshchilov and Hutter (2019) with a warm-up method for the generator. Our code is publicly available.3

Footnote 3: [https://github.com/hanghanghanglang12315/MSP/tree/release](https://github.com/hanghanghanglang12315/MSP/tree/release).

### Evaluation

Metric-based EvaluationWe first evaluate all methods by several metrics with respect to different aspects. (1) BLEU-1/2 Papineni et al. (2002) and ROUGE-L Lin and Och (2004) are typical word overlap-based metrics for measuring the similarity between the generated response and the groundtruth.4\({}^{*}\) (2) Distinct-1/2 Li et al. (2016b) consider the number of uni- or bi-grams in the generated response, which is commonly used for evaluating the diversity. (3) The embedding-based metrics (_i.e._, average, extrema, and greedy) Chan et al. (2019) are introduced to measure the semantic similarity between the generated response and the groundtruth one. (4) As a personalized dialogue model, following previous studies Ma et al. (2021), two tailored metrics are adopted to measure how much information is included in the dialogue history that can be reflected in the response. Persona-F1 P-F1) Lian et al. (2019) calculates the F1 value to measure the uni-grams co-occurring in both the generated response and the dialogue history. Persona Coverage (P-Cover) Song et al. (2019) calculates the IDF-weighted word overlap between the generated response and the golden one so that the importance of different words can be taken into account.

Footnote 4: The results of BLEU-3/4 and ROUGE-1/2 are provided in Appendix A.

Human AnnotationDue to the variability of human language, a response that differs from the ground-truth may also be appropriate. Following previous studies Chan et al. (2019), we conduct a human evaluation of all methods. Concretely, we

\begin{table}
\begin{tabular}{l} \hline \hline
**Readability** \\
3: Fluent and easy to read \\
2: Grammatically formed \\
1: Not a complete sentence or hard to read \\ \hline
**Informativeness** \\
3: Have clear and specific meaning \\
2: Contain few informative words \\
1: Meaningfulness sentence \\ \hline
**Personalization** \\
1: Reflect personal information contained in user history \\
0: Does not resemble any user history \\ \hline \hline \end{tabular}
\end{table}
Table 2: Criteria of human annotation.

 

sample 100 (query, response, user dialogue history) triplets and hire three well-educated annotators to score the responses generated by different models. Three aspects, _i.e._, readability, informativeness, and personalization, are considered. The first two factors are scored on a scale of [1; 3] for their quality, while the third is assessed on a scale of [0, 1], indicating whether the response can accurately reflect the user's personality. The detailed scoring criteria are shown in Table 2.

### Experimental Results

Metric-based EvaluationTable 3 shows all models' performance under different metrics. On both datasets, it is clear to see that our MSP model outperforms baselines on all metrics. The improvement is statistically significant (t-test with \(p\)-value \(<\) 0.05). These findings indicate that our model is capable of generating more fluent, diverse, and personalized responses than all baselines. In particular, we can observe: (1) MSP achieves better performance on overlap-based metrics. This suggests that our model can provide responses that are more similar to the ground-truth with the help of the selected tokens. (2) For diversity metrics, the higher distinct values show that our generated responses are more diverse. Additionally, predefined profile-based methods and user dialogue history-based methods outperform others. This shows that incorporating external information can aid in the generation of more informative responses. (3) In addition to generating more overlapped words with the ground-truth response, the improvements in embedding metrics reflect that our model generates more semantically relevant responses. (4) Finally, the increase in personalized metrics implies that our approach can incorporate more user-specific information into the generation. Furthermore, the significant improvement over DHAP demonstrates that our model can extract more meaningful personalized information from the user dialogue history.

Human AnnotationThe result of human annotation on the Weibo dataset is shown in Table 4. The Fleiss Kappa is around 0.62, indicating a substantial agreement achieved by three annotators. In general, the results of human annotation are consistent with those of the metric-based evaluation. Both of them demonstrate our model's superiority in generating more fluent, informative, and personalized responses. Compared to non-personalized methods, user id-based methods can enhance personalization at the expense of readability. User dialogue history-based methods (_i.e._, DHAP and MSP) can largely improve the personalization of the response while retaining a high level of readability and informativeness. We attribute this to the abundant personal information contained in the user dialogue history.

\begin{table}
\begin{tabular}{l l c c c c c c c c c} \hline \hline  & & \multicolumn{3}{c}{Overlap-based Metric} & \multicolumn{3}{c}{Diversity} & \multicolumn{3}{c}{Embedding Metric} & \multicolumn{3}{c}{Persona Metric} \\ \cline{8-11}  & & BLEU-1 & BLEU-2 & ROUGE-L & Dist-1 & Dist-2 & Average & Extrema & Greedy & P-F1 & P-Cover \\ \hline \multirow{9}{*}{Searage} & Seq2Seq & 3.330† & 0.294� & 8.985† & 0.930† & 2.180† & 0.321† & 0.266† & 0.254† & 0.154† & 0.041† \\  & MMI & 3.631† & 0.095† & 5.264† & 10.710† & 43.458† & 0.477† & 0.696† & 0.305† & 0.325† & 0.054† \\  & DialoGPT & 6.068† & 0.741† & 8.459† & 15.322† & 55.536† & 0.557† & 0.793† & 0.324† & 0.522† & 0.061† \\  & GPMN & 4.899† & 0.696† & 7.785† & 11.724† & 32.730† & 0.353† & 0.391† & 0.301† & 0.542† & 0.084† \\  & PerCVAE & 5.114† & 0.299† & 7.380† & 14.098† & 49.733† & 0.469† & 0.657† & 0.299† & 0.903† & 0.086† \\  & Speaker & 4.994† & 0.113† & 7.868† & 6.035† & 19.007† & 0.492† & 0.712† & 0.311† & 0.225† & 0.082† \\  & PersonaWAE & 3.510† & 0.155† & 10.546† & 2.551† & 19.743† & 0.563† & 0.757† & 0.307† & 1.740† & 0.103† \\  & DHAP & 9.324† & 0.894† & 14.122† & 15.175† & 58.806† & 0.523† & 0.747† & 0.313† & 1.791† & 0.144† \\  & MSP (Ours) & **11.875** & **5.108** & **15.563** & **24.203** & **73.196** & **0.605** & **0.883** & **0.331** & **2.170** & **0.297** \\ \hline \multirow{9}{*}{Searage} & Seq2Seq & 1.820† & 0.023† & 4.069† & 5.203† & 19.485† & 0.545† & 0.554† & 0.470† & 0.051† & 0.029† \\  & MMI & 2.065† & 0.011† & 3.784† & 5.914† & 31.093† & 0.543† & 0.607† & 0.454† & 0.085† & 0.038† \\  & DialoGPT & 4.735† & 0.397† & 8.943† & 6.353† & 29.106† & 0.604† & 0.733† & 0.448† & 0.137† & 0.040† \\  & GPMN & 2.686† & 0.376 † & 4.776† & 12.325† & 35.762† & 0.406† & 0.331† & 0.358† & 0.189† & 0.037† \\  & PerCVAE & 5.933† & 0.576† & 8.112† & 9.631† & 40.213† & 0.637† & 0.649† & 0.499† & 0.212† & 0.040† \\  & Speaker & 2.642† & 0.054† & 4.469† & 8.951† & 34.187† & 0.538† & 0.606† & 0.457† & 0.115† & 0.031† \\  & PersonaWAE & 2.637† & 0.113† & 8.199† & 1.758† & 25.915† & 0.629† & 0.685† & 0.442† & 0.206† & 0.032† \\  & DHAP & 6.858† & 0.737† & 11.720† & 18.707† & 66.932 & 0.709 & 0.721† & 0.539 & 0.227† & 0.111† \\  & MSP (Ours) & **7.174** & **0.883** & **12.171** & **21.247** & **68.897** & **0.716** & **0.764** & **0.545** & **0.276** & **0.137** \\ \hline \hline \end{tabular}
\end{table}
Table 3: Metric-based evaluation result on Weibo and Reddit dataset. The best results are in **bold**. “I” indicates that our model achieves significant improvement in t-test with \(p\)-value \(<\) 0.05.

 



## 5 Further Analysis

We further conduct a series of analyses to elaborate our model. All analyses here are based on the result of the Weibo dataset, while similar results can be observed on the Reddit dataset.

Ablation StudyTo investigate the impact of different modules in MSP, we conduct an ablation study by removing or using different strategies in each module.

We first study the influence of the refiners at three levels: (1) We remove the user refiner and train our model using randomly sampled users. We can see the performance of all metrics drops. This illustrates that our MSP model can select users that share the same interests as the current user and thereby improving response quality. (2) We remove the topic refiner and supply the token refiner with full dialogue history. The performance degradation demonstrates that various topics in dialogue history introduce lots of noise, misleading the token refiner on extracting valuable tokens, thus impairing the personalized response generation. (3) We eliminate the token refiner and feed all dialogue history sentences directly into the generator.5 The decline in performance implies the effectiveness and necessity of token selection. It is worth noting that, as compared to using the complete history, our selection strategy can reduce training time by 41.6%, considerably increasing efficiency. All of the aforementioned experimental results suggest that MSP's advantage stems from high-quality personalized information extraction rather than simply introducing additional information.

Footnote 5: Due to the length limitation of GPT-2, history with more than 512 tokens will be truncated.

We then explore the impact of personalized information from two sources, _i.e._, the current user's profile and the similar users' profile. Removing either of them results in decreased performance. This exemplifies their usefulness. Specifically, compared with similar users' profiles, eliminating the current user's profile will hurt the personalization effect heavily. This result shows that, for personalization, the current user's profile is more essential than that of similar users, which is quite intuitive. However, the similar users' profile has a significant effect on BLEU-1/2, implying that such a profile can provide abundant information for response generation. Consequently, integrating both types of profiles significantly improves response generation.

Finally, we conduct an experiment to validate our proposed joint training for the token refiner. The declining performance indicates that the token refiner is unable to extract useful information in the absence of additional supervision signals. Indeed, when the sentence matching task is removed, the token refiner extracts tokens that are relevant to the current query, which is less useful for generating a personalized response.

Influence of Selection MechanismTo validate the effectiveness of our proposed selection mechanism, we replace the refiner with a traditional retrieval method (_i.e._, BM25 Robertson and Walker, 1994)). Specifically, we use the query to retrieve

\begin{table}
\begin{tabular}{l c r r} \hline \hline Model & Readability & Informativeness & Personality \\ \hline Seq2Seq & 1.76\({}^{\dagger}\) & 1.37\({}^{\dagger}\) & 0.11\({}^{\dagger}\) \\ MMI & 1.96\({}^{\dagger}\) & 1.88\({}^{\dagger}\) & 0.19\({}^{\dagger}\) \\ DialogPT & 2.33 & 2.10\({}^{\dagger}\) & 0.32\({}^{\dagger}\) \\ GPMN & 2.01\({}^{\dagger}\) & 2.16\({}^{\dagger}\) & 0.35\({}^{\dagger}\) \\ PerCVAE & 2.10\({}^{\dagger}\) & 2.01\({}^{\dagger}\) & 0.39\({}^{\dagger}\) \\ Speaker & 1.89\({}^{\dagger}\) & 1.44\({}^{\dagger}\) & 0.24\({}^{\dagger}\) \\ PersonaWAE & 1.81\({}^{\dagger}\) & 2.01\({}^{\dagger}\) & 0.32\({}^{\dagger}\) \\ DHAP & 2.29\({}^{\dagger}\) & 2.19\({}^{\dagger}\) & 0.55\({}^{\dagger}\) \\ MSP (Ours) & **2.37** & **2.39** & **0.67** \\ \hline Ground-Truth & 2.71 & 2.66 & 0.76 \\ \hline \hline \end{tabular}
\end{table}
Table 4: The result of human evaluation on Weibo dataset. “\({}^{\dagger}\)” indicates that our model achieves significant improvement in t-test with \(p\)-value \(<\) 0.05.

Figure 2: Comparison with the retrieval-based model on the Weibo dataset.



15 relevant responses and feed them into our model for training. The experimental results are shown in Figure 2. We can observe that the retrieval strategy achieves comparable performance with our model on word-overlap and embedding-based metrics. This suggests that the relevant dialogue history for the query can provide valuable information for response generation. However, the retrieval strategy performs poorly on diversity and personalization metrics. This demonstrates that, without careful selection, the retrieved information is too generic and thus less helpful for personalized response generation.

Influence of Personalized Tokens AmountIn MSP, three refiners are designed to extract personalized tokens for response generation. Intuitively, the amount of the tokens will influence the refiner's performance. We report this influence in Figure 3. As we can see in the left part, the quality of response generation improves with more tokens used. This is because fewer tokens are incapable of covering sufficient personalized information for response generation. Our MSP model performs optimally when about 200 personalized tokens are selected. When more tokens are introduced, the performance degrades. The potential reason is that more tokens would bring the noise to the generation. This is consistent with our speculation that the dialogue history is noisy and the information selection is both effective and necessary.

To provide a more qualitative view of our method, we conduct a case study in Appendix B.

## 6 Conclusion

In this work, we propose an MSP model for personalized response generation. Unlike previous related work, we utilize a refiner structure to extract query-aware persona information from large-scale dialogue history. The multi-level refiners can sparsely extract valuable information from dialogue history and leverage similar users' information to enhance the current user's personalization. Experimental results confirm the effectiveness of our model in generating informative and personalized responses.

## Acknowledgement

Zhicheng Dou is the corresponding author. This work was supported by National Natural Science Foundation of China No. 61872370 and No. 61832017, Beijing Outstanding Young Scientist Program NO. BJJWZYJH012019100020098, and Intelligent Social Governance Platform, Major Innovation & Planning Interdisciplinary Platform for the "Double-First Class" Initiative, Renmin University of China. We also acknowledge the support provided and contribution made by Public Policy and Decision-making Research Lab of RUC.

## References

* Al-Rfou et al. (2016) Rami Al-Rfou, Marc Pickett, Javier Snader, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2016. Conversational contextual cues: The case of personalization and history for response ranking. _CoRR_, abs/1606.00372.
* Bak and Oh (2019) JinYeong Bak and Alice Oh. 2019. Variational hierarchical user-based conversation model. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_, pages 1941-1950. Association for Computational Linguistics.
* Cai et al. (2019) Deng Cai, Yan Wang, Wei Bi, Zhaopeng Tu, Xiaojiang Liu, and Shuming Shi. 2019. Retrieval-guided dialogue response generation via a matching-to-generation framework. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_, pages 1866-1875. Association for Computational Linguistics.
* Chan et al. (2019) Zhangming Chan, Juntao Li, Xiaopeng Yang, Xiuying Chen, Wenpeng Hu, Dongyan Zhao, and Rui Yan. 2019. Modeling personalization in continuous space for response generation via augmented Wasserstein autoencoders. In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_, pages

Figure 3: Experiments with the different number of user profiles on the Weibo dataset. To keep the dimension consistent, P-Cover is multiplied by a factor of 100.

 

1931-1940. Association for Computational Linguistics.
* Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)_, pages 4171-4186. Association for Computational Linguistics.
* Holtzman et al. (2020) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
* Johnson et al. (2021) Jeff Johnson, Matthijs Douze, and Herve Jegou. 2021. Billion-scale similarity search with gpus. _IEEE Trans. Big Data_, 7(3):535-547.
* Kingma and Ba (2015) Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
* June 1, 2003_. The Association for Computational Linguistics.
* Li et al. (2016a) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016a. A diversity-promoting objective function for neural conversation models. In _ANACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016_, pages 110-119. The Association for Computational Linguistics.
* Li et al. (2016b) Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016b. A diversity-promoting objective function for neural conversation models. In _NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016_, pages 110-119. The Association for Computational Linguistics.
* Li et al. (2016c) Jiwei Li, Michel Galley, Chris Brockett, Georgios P. Spithourakis, Jianfeng Gao, and William B. Dolan. 2016c. A persona-based neural conversation model. In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers_. The Association for Computer Linguistics.
* Li et al. (2018) Juncen Li, Robin Jia, He He, and Percy Liang. 2018. Delete, retrieve, generate: a simple approach to sentiment and style transfer. In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume 1 (Long Papers)_, pages 1865-1874. Association for Computational Linguistics.
* Lian et al. (2019) Rongzhong Lian, Min Xie, Fan Wang, Jinhua Peng, and Hua Wu. 2019. Learning to select knowledge for response generation in dialog systems. In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019_, pages 5081-5087. ijcai.org.
* Lin and Och (2004) Chin-Yew Lin and Franz Josef Och. 2004. Automatic evaluation of machine translation quality using longest common subsequence and skip-bigram statistics. In _Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics, 21-26 July, 2004, Barcelona, Spain_, pages 605-612. ACL.
* Liu et al. (2020) Ya-Fei Liu, Hongjin Qian, Hengpeng Xu, and Jinmao Wei. 2020. Speaker or listener? the role of a dialogue agent. In _Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020_, volume EMNLP 2020 of _Findings of ACL_, pages 4861-4869. Association for Computational Linguistics.
* Lohschilov and Hutter (2019) Ilya Lohschilov and Frank Hutter. 2019. Decoupled weight decay regularization. In _7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019_. OpenReview.net.
* Luong et al. (2015) Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015_, pages 1412-1421. The Association for Computational Linguistics.
* Ma et al. (2021) Zhengyi Ma, Zhicheng Dou, Yutao Zhu, Hanxun Zhong, and Ji-Rong Wen. 2021. One chatbot per person: Creating personalized chatdots based on implicit user profiles. In _SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021_, pages 555-564. ACM.
* Olabiyi et al. (2019) Oluwatobi Olabiyi, Anish Khazane, Alan Salimov, and Erik T. Mueller. 2019. An adversarial learning framework for A persona-based multi-turn dialogue model. _CoRR_, abs/1905.01992.
* Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Weihg Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In _Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA_, pages 311-318. ACL 

Hongjin Qian, Zhicheng Dou, Yutao Zhu, Yueyuan Ma, and Ji-Rong Wen. 2021a. Learning implicit user profiles for personalized retrieval-based chatbot. _CoRR_, abs/2108.07935.
* Qian et al. (2021b) Hongjin Qian, Xiaohe Li, Hanxun Zhong, Yu Guo, Yueyuan Ma, Yutao Zhu, Zhanliang Liu, Zhicheng Dou, and Ji-Rong Wen. 2021b. Pchatbot: A large-scale dataset for personalized chatbot. In _Proceedings of the SIGIR 2021_. ACM.
* Qian et al. (2018) Qiao Qian, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. 2018. Assigning personality/profile to a chatting machine for coherent conversation generation. In _Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden_, pages 4279-4285. ijcai.org.
* Radford et al. (2019) Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. _OpenAI blog_, 1(8):9.
* Robertson and Walker (1994) Stephen E. Robertson and Steve Walker. 1994. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In _Proceedings of the 17th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval. Dublin, Ireland, 3-6 July 1994 (Special Issue of the SIGIR Forum)_, pages 232-241. ACM/Springer.
* Serban et al. (2016) Iulian Vlad Serban, Alessandro Sordoni, Yoshua Bengio, Aaron C. Courville, and Joelle Pineau. 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In _Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA_, pages 3776-3784. AAAI Press.
* Song et al. (2019) Haoyu Song, Weinan Zhang, Yiming Cui, Dong Wang, and Ting Liu. 2019. Exploiting persona information for diverse generation of conversational responses. In _Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019_, pages 5190-5196. ijcai.org.
* Sutskever et al. (2014) Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In _Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada_, pages 3104-3112.
* Vinyals and Le (2015) Oriol Vinyals and Quoc V. Le. 2015. A neural conversational model. _CoRR_, abs/1506.05869.
* February 1, 2019_, pages 7281-7288. AAAI Press.
* Xiao et al. (2020) Ziang Xiao, Michelle X. Zhou, Wenxi Chen, Huahai Yang, and Chang Yan Chi. 2020. If I hear you correctly: Building and evaluating interview chatbots with active listening skills. _CoRR_, abs/2002.01862.
* Xing et al. (2017) Chen Xing, Wei Wu, Yu Wu, Jie Liu, Ya Lou Huang, Ming Zhou, and Wei-Ying Ma. 2017. Topic aware neural response generation. In _Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, February 4-9, 2017, San Francisco, California, USA_, pages 3351-3357. AAAI Press.
* Yang et al. (2019) Liu Yang, Junjie Hu, Minghui Qiu, Chen Qu, Jianfeng Gao, W Bruce Croft, Xiaodong Liu, Yelong Shen, and Jingjing Liu. 2019. A hybrid retreiveal-generation neural conversation model. In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management, CIKM 2019, Beijing, China, November 3-7, 2019_, pages 1341-1350. ACM.
* Zhang et al. (2019a) Hainan Zhang, Yanyan Lan, Liang Pang, Jiafeng Guo, and Xueqi Cheng. 2019a. Recosa: Detecting the relevant contexts with self-attention for multi-turn dialogue generation. In _Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28-August 2, 2019, Volume 1: Long Papers_, pages 3721-3730. Association for Computational Linguistics.
* Zhang et al. (2018) Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers_, pages 2204-2213. Association for Computational Linguistics.
* Zhang et al. (2019b) Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2019b. Dialogpt: Large-scale generative pre-training for conversational response generation. _CoRR_, abs/1911.00536.
* August 4, Volume 1: Long Papers_, pages 654-664. Association for Computational Linguistics.
* Zhu et al. (2020) Yutao Zhu, Zhicheng Dou, Jian-Yun Nie, and Ji-Rong Wen. 2020. Reboost: a retrieval-boosted sequence-to-sequence model for neural response generation. _Inf. Retr. J._, 23(1):27-48 



## Appendix A Additional Experimental Results

As n-gram word overlap metrics can reflect user speaking style more accurately, we evaluate the BLEU-3/4 (Papineni et al., 2002), and the result is shown in Table 6. It is consistent with other evaluations that our model outperforms every indicator. This demonstrates that user profiles also contain speaking style information, and our model can use the information to achieve a personalized response.

## Appendix B Case Study

To show the effect of our model more concretely, we adopt a case study, and the results are shown in Table 7. It shows that our model can extract profiles from both current and similar users and generate informative and personalized responses. Specifically, in his dialogue history, he mentioned sports **H1, H2** and music **H3, H4** topics. Firstly, we can select similar users who also talk about sports and music, using the user refiner. Then, as the query is related to the music topic, we extract the current and sim users' dialogue history responses about the music topic **R1, R2** by the topic refiner. Furthermore, the token refiner selects some meaningful and personalized words from long sentences of reference. In this case, we can find that the token refiner extracts some compliment words (like, beautiful, enjoying) from the current user's history sentence since the current user likes listening to music. And the token refiner captures more concrete tokens from sim users' history sentences, such as "Angela Leung" and "Ember". By combing two profiles, our personalized generator gets an informative and personalized response close to ground-truth. In contrast, DialoGPT generates a fluent but meaningless response to the query.

\begin{table}
\begin{tabular}{c c r r r r} \hline \hline  & \multicolumn{2}{c}{BLEU-3} & \multicolumn{2}{c}{BLEU-4} & \multicolumn{2}{c}{ROUGE-1} & \multicolumn{2}{c}{ROUGE-2} \\ \hline Seq2Seq & 0.011† & 0.001† & 8.740† & 0.373† \\ MMI & 0.046† & 0.007† & 5.316† & 0.105� \\ DialoGPT & 0.114† & 0.027† & 9.414† & 0.632† \\ GPMN & 0.359† & 0.066† & 8.086† & 0.753† \\ PerCVAE & 0.466† & 0.089† & 7.946† & 0.485† \\ Speaker & 0.107† & 0.041† & 7.997† & 0.155† \\ PersonaWAE & 0.889† & 0.155† & 11.34†† & 0.358† \\ DHAP & 1.170† & 0.401† & 14.131† & 3.608† \\
**MSP** (Ours) & **3.973** & **3.522** & **16.249** & **5.812** \\ \hline Seq2Seq & 0.007† & 0.001† & 3.989† & 0.233† \\ MMI & 0.007† & 0.003† & 3.960† & 0.245† \\ DialoGPT & 0.054† & 0.010† & 8.977† & 0.610† \\ GPMN & 0.039† & 0.006† & 4.896† & 0.330† \\ PerCVAE & 0.068† & 0.009† & 8.004† & 0.540† \\ Speaker & 0.021† & 0.005† & 4.017† & 0.245† \\ PersonaWAE & 0.029† & 0.007† & 8.247† & 0.517† \\ DHAP & 0.079† & 0.013† & 10.680† & 0.697† \\
**MSP** (Ours) & **0.106** & **0.019** & **11.078** & **0.745** \\ \hline \hline \end{tabular}
\end{table}
Table 6: The results of other evaluation metrics on Weibo and Reddit dataset. “† indicates that our model achieves significant improvement in t-test with \(p\)-value \(<\) 0.05. The best results are in **bold**.





\begin{table}
\begin{tabular}{l l} \hline \multirow{2}{*}{History} & **H1:** Liverpool’s configuration has the life of a champion, support it! \\  & **H2:** Champion Liverpool! \\  & **H3:** I like to listen to music. \\  & **H4:** The songs on this album are really beautiful, so worth enjoying. \\ \hline Query & New album ”How Am I? -The sun rises as it always does”, the first single “Ember” heals the system, go listen! \\ \hline Persona Reference & **R1:** I like to listen to music. \\  & **R2:** The songs on this album are really beautiful, so worth enjoying. \\ \hline Sim Reference & **R1:** Quite like this type of song. \\  & **R2:** Angela Leung’s ”Ember” is really good. \\ \hline Persona Profile & like, song, album, beautiful, enjoying \\ \hline Sim Profile & like, song, Angela Leung, good, Ember \\ \hline \multirow{2}{*}{Response} & **DialoGPT**: My little heart has flown. \\  & **MSP(Ours)**: Angela Leung’s song is very good. \\  & **Golden**: Angela Leung’s songs must be listened to. \\ \hline \end{tabular}
\end{table}
Table 7: A case study. Due to space limitation, we omit some user history responses and references.