SlowGuess/ACL_2020_OCR huggingface dataset:
Bowen Wu $^{1}$ , Mengyuan Li $^{2*}$ , Zongsheng Wang $^{1}$ , Yifu Chen $^{3*}$ , Derek F. Wong $^{4}$ , Qihang Feng $^{1}$ , Junhong Huang $^{1}$ , Baoxun Wang $^{1}$
$^{1}$ Platform and Content Group, Tencent
$^{2}$ Peking University, Beijing, China
$^{4}$ NLP $^{2}$ CT Lab / Department of Computer and Information Science, University of Macau
{jasonbwwu, jasoawang, careyfeng, vincenthuang, asulewang}@tencent.com
limengyuan@pku.edu.cn, chenyifu17@mails.ucas.ac.cn, derekfw@um.edu.mo
Leveraging persona information of users in Neural Response Generators (NRG) to perform personalized conversations has been considered as an attractive and important topic in the research of conversational agents over the past few years. Despite of the promising progress achieved by recent studies in this field, persona information tends to be incorporated into neural networks in the form of user embeddings, with the expectation that the persona can be involved via End-to-End learning. This paper proposes to adopt the personality-related characteristics of human conversations into variational response generators, by designing a specific conditional variational autoencoder based deep model with two new regularization terms employed to the loss function, so as to guide the optimization towards the direction of generating both persona-aware and relevant responses. Besides, to reasonably evaluate the performances of various persona modeling approaches, this paper further presents three direct persona-oriented metrics from different perspectives. The experimental results have shown that our proposed methodology can notably improve the performance of persona-aware response generation, and the metrics are reasonable to evaluate the results.
As an essential research topic in generative conversational agents (a.k.a., chat-bots), Persona Modeling is of great importance for such deep neural network based intelligent interactive systems (Li et al., 2016b; Kottur et al., 2017; Wang et al., 2017). Apparently, user-personality-dependent responses provided by a chat-bot are able to significantly improve the consistency of its conversations, meanwhile, it is possible for users
to flexibly customize the persona of a chat-bot based on some existent dialogues. As for the studies on this topic, with no doubt, incorporating persona factors into End-to-End generative models is an attractive topic with great challenges.
The current studies mainly focus on adopting the explicit meta-data of user profiles (Qian et al., 2018; Chu et al., 2018) or character descriptions (Zhang et al., 2018; Mazare et al., 2018; Song et al., 2019) to generate persona-aware responses. However, on one hand, user profiles are usually highly privacy-related and thus it is difficult to obtain such information from users practically. On the other hand, little correlation can be explicitly observed between such meta-data profiles and persona characteristics of users. Especially, those character descriptions, tailor-made for the persona-aware response generation with the great cost of manual work, are only a variant of user profile innately in terms of different natural language forms.
One of the reasonable and practically executable methodologies for introducing persona factors into conversation models is to adopt the real-valued user representation as a medium (Li et al., 2016b; Kottur et al., 2017; Liu et al., 2018; Al-Rfou et al., 2016). In particular, such user representations can be derived from users' historical dialog utterances with rich linguistic and personality information involved. Taking persona representations as the guidance for generating customized responses becomes a widely accepted methodology due to the recent development of deep latent variable models (Zhao et al., 2017; Shen et al., 2017; Zhou and Wang, 2018). However, for current models, without the explicit learning objectives or constraints, the user representation is adopted in a passive way to reduce the model loss and KL divergence via end-to-end learning. In this case, it is highly possible that the
* Contribution during the internship at Tencent.
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 53-65 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
Figure 1: The architecture of the Persona-Aware Variational Response Generator (PAGenerator) described in this paper.  $\oplus$  represents the concatenation of inputs and CE denotes the cross-entropy of predictions. The dotted arrow line indicates the connection is optional, and the default model named PAGenerator decodes with the user embedding.
employed embeddings will not work as effectively as expected.
Consequently, it is necessary to employ explicit guidance to help variational response generators sense persona. From observations upon persona-contained dialogs, there exist intuitive characteristics for directing the optimization of the persona-aware variational response generation. Obviously, for a given user, the appropriately modeled and leveraged persona information can help to generate hidden variables semantically relevant with corresponding responses. Besides, since users may have their own linguistic style, the adoption of personal information in NRG aims to have direct influence on the degree of linguistic (e.g. lexical and syntactic) convergence for a specific user.
This paper aims at exploring the explicit guidance to help the variational response generator exploit persona information hidden in the non-structured contents produced by the users, by utilizing intuitive characteristics of personalized conversations for model training. The contributions of this paper can be summarized as follows:
- A persona-aware variational response generator is proposed to exploit persona while modeling the conversations.
- Based on the model, two regularization terms are presented to guide the model in encoding user information into the latent variables and converging to user-specific responses.
- Three discriminative metrics are further introduced to evaluate the capabilities of persona-aware response generators.
Based on the current progress on the development of latent variable models, we propose a persona-aware variational response generator to automatically exploit persona from conversations, and utilize such personal information to model the future conversation. Besides, given that personal information can be exploited as optimization guidance to better modeling persona, we further introduce two regularization terms to guide the model learning. In the following section, we first describe the general structure of PAGenerator, and then explain the two additional regularization terms.
2.1 Persona-Aware Variational Response Generator
Utilizing latent variables in response generation has become a widely accepted methodology in NRG due to their Bayesian essence. It helps to deal with external knowledge efficiently, e.g. Persona. Therefore, our proposed model is built based on the generation model with latent variables. The overall architecture of the single turn persona-aware variational response generator proposed in this paper is illustrated in Figure 1.
Let  $q, r, u$  stand for the query, the reply and the corresponding user of  $r$ , respectively, and  $e_u$  stands for the embedding of user  $u$ . A bidirectional LSTM is first employed to encode the query and reply into fixed size vectors  $h_q$  and  $h_r$ . After that, the prior network (parametrized by  $\theta$ ) takes  $u_e$ ,  $h_q$  as inputs to generate the distribution  $p_\theta(z|q, u)$  of latent variable  $z$ . Meanwhile,  $h_q$ ,  $h_r$  are fed into a posterior network (parameterized by  $\phi$ ) to compute  $q_\phi(z|q, r)$ . As we adopt the assumption that  $z$  follows isotropic Gaussian distribution,  $p_\theta(z|q, u)$  and  $q_\phi(z|q, r)$  are also normally
distributed, such that:
$$
p _ {\theta} (z | q, u) \sim \mathcal {N} \left(\mu_ {p}, \sigma_ {p} ^ {2} \mathbf {I}\right) \tag {1}
$$
$$
q _ {\phi} (z | q, r) \sim \mathcal {N} (\mu_ {q}, \sigma_ {q} ^ {2} \mathbf {I})
$$
where the means and variances are computed as follows:
$$
\left[ \begin{array}{c} \mu_ {p} \\ \log (\sigma_ {p} ^ {2}) \end{array} \right] = W _ {p} \left[ \begin{array}{c} q \\ u \end{array} \right] + b _ {p} \tag {2}
$$
$$
\left[ \begin{array}{c} \mu_ {q} \\ \log \left(\sigma_ {q} ^ {2}\right) \end{array} \right] = W _ {q} \left[ \begin{array}{c} q \\ r \end{array} \right] + b _ {q} \tag {3}
$$
where  $W_{p}, W_{q}, b_{p}$  and  $b_{q}$  are the trainable parameters. A sample of  $z$  using the reparametrization trick (Kingma and Welling, 2013) is then fed into the decoder as a part of input at each time step.
In addition, the bag-of-word (BOW) loss (Zhao et al., 2017) is employed to tackle the latent variable vanishing problem, and PAGenerator is trained to maximize the variational lower-bound (Chung et al., 2015; Serban et al., 2017):
$$
\begin{array}{l} \mathcal {L} (\theta , \phi ; q, r, u) = \mathrm {E} _ {q _ {\phi} (z | q, r)} [ \log p _ {\theta} (r | z, q, u) ] \\ - K L \left(q _ {\phi} (z | q, r) \| p _ {\theta} (z | q, u)\right) \tag {4} \\ + \mathrm {E} _ {q _ {\phi} (z | q, r)} [ \log p (r _ {b o w} | z, q, u) ] \\ \end{array}
$$
2.2 User Information Enhancing Regularization
Ideally, we expect that the introduction of user embedding is fully utilized during model training. However, due to the KL vanishing problem, the training of PAGenerator suffers from the hazard that the rapid decrease of  $\mathcal{L}$  in Equation 4 might be attributed to the strong fitting capability of the decoder on the training data, rather than the involvement of user embedding. Thus, we introduce a regularization term to promote the usage of user's hidden information in latent variables.
At the beginning, as illustrated in Figure 1, a general  $unk_u$  is introduced to represent the case for user unspecified. Subsequently, taking the default user embedding  $e_{unk_u}$  as input, we obtain the KL divergence as  $KL(q_{\phi}(z|q,r)\| p_{\theta}(z|q,unk_u))$  from the network. In this case, once the real user  $u$  is introduced, a regularization term  $\mathcal{R}_1(\theta ,\phi ;q,r,u)$  can be constructed as follows:
$$
\begin{array}{l} \mathcal {R} _ {1} (\theta , \phi ; q, r, u) = \max  (- \gamma_ {1}, \\ K L \left(q _ {\phi} (z | q, r) \| p _ {\theta} (z | q, u)\right) \tag {5} \\ - K L \left(q _ {\phi} (z | q, r) \| p _ {\theta} (z | q, u n k _ {-} u)\right)) \\ \end{array}
$$
$$
\begin{array}{l} \text {w h e r e} \gamma_ {1} \in \mathbb {R}, \gamma_ {1} > 0, \text {a n d} p _ {\theta} (z | q, u n k _ {-} u) \sim \\ \mathcal {N} (\mu_ {p} ^ {\prime}, \sigma_ {p} ^ {\prime 2} \mathbf {I}). \end{array}
$$
It should be noted that, according to the equation above, the two prior distributions are generated from the same network with partially different inputs  $(u\text{VS.} unk\_ u)$ , and the regularization constrains the prior distribution with specified user to be closer to the posterior distribution. Thus, the optimization encourages the utilization of user information and correspondingly inhibits the generated results from ignoring the user information. Meanwhile,  $\mathcal{R}_1$  in our proposed model also alleviates the KL vanishing problem.
2.3 Variance Controlling Regularization
The BOW loss forces the latent variables to predict the bag-of-words in the response. Therefore, the semantic distribution of  $z$  is required to be capable of representing the topics and wording of the target response. Besides, for a given query, the possible replies from a specific user should be more convergent to each other than those from an unknown user, due to each user's unique preference on the topics and wording. Correspondingly, under the assumption that the distribution of  $z$  represents the user's language preference, the specification of user information is expected to reduce the entropy of the isotropic Gaussian distribution of  $z$ , reflected by a lower standard deviation  $\sigma_{p}$ . On this basis, we introduce another regularization term  $\mathcal{R}_2(\theta, \phi; q, r, u)$  to control the variance:
$$
\mathcal {R} _ {2} (\theta , \phi ; q, r, u) = \max  (- \gamma_ {2}, \sigma_ {p} ^ {2} - \sigma_ {p} ^ {\prime 2}) \tag {6}
$$
where  $\gamma_{2}\in \mathbb{R}$  and  $\gamma_{2} > 0$ .  $\mathcal{R}_2$  prefers those  $z$  with decrease  $\geq \gamma_{2}$  in standard deviation  $\sigma_{p}$  after specifying users, and such decrease indicates the latent variables are more semantically convergent.
On this basis, we update the new training objective of PAGenerator as follows:
$$
\begin{array}{l} \mathcal {L} ^ {\prime} (\theta , \phi ; q, r, u) = \mathcal {L} (\theta , \phi ; q, r, u) \\ - \mathcal {R} _ {1} (\theta , \phi ; q, r, u) - \mathcal {R} _ {2} (\theta , \phi ; q, r, u) \\ \end{array}
$$
By employing the two regularization terms to constrain the model training,  $\mathcal{L}'(\theta, \phi; q, r, u)$  now also pays attention to the utilization of user information and language preference.
3 Specified Evaluation Metrics of Persona NRG
In the previous section, two regularization terms are proposed to guide the model in the persona exploration. However, we still lack effective persona-focused metrics to quantify how well one
model is on learning persona. The currently applied metrics for persona-aware NRG evaluation, such as perplexity and BLEU, are used to evaluate the plain NRG models (Li et al., 2016b; Kottur et al., 2017). Apparently, such metrics are inadequate to evaluate the capacity of a response generator on capturing persona.
Innately, an effective persona-aware response generator should be able to successfully identify and generate responses for users according to their language styles. Besides, the generated responses from different users should be diversified to each other in wording. Considering these properties, we propose the following metrics to measure the level of persona-aware in response generators.
3.1 Language Style Detection
It is important for a persona-aware response generator to identify a user's response from other user-irrelevant ones, by detecting the user's language style in responses. In this subsection, we propose User-Relative-Rank (uRank) to measure such capability. Given a query-response-user triple  $\{q,r,u\}$ , a pre-trained seq2seq model  $S2S$  and a model  $M$  to be evaluated, we first generate  $n$  user-irrelevant responses  $\{r_i'|i\in [1,n]\}$  from  $S2S$  using beam search. For a desired persona-aware model  $M$ , it is expected to assign the ground truth response  $r$  with a higher probability than other user-irrelevant ones  $\{r_i'|i\in [1,n]\}$ . Thus, taking  $S2S$  as reference, we set  $uRank$  to be 1 if  $M$  scores  $r$  a higher ranking position among  $r_i'$  than  $S2S$ , specifically:
$$
\begin{array}{l} r a n k _ {M} = \left| \left\{i \mid P _ {M} \left(r _ {i} ^ {\prime}\right) > P _ {M} (r) \right\} \right| \\ \operatorname {r a n k} _ {S 2 S} = \left| \left\{i \mid P _ {S 2 S} \left(r _ {i} ^ {\prime}\right) > P _ {S 2 S} (r) \right\} \right| \tag {8} \\ \end{array}
$$
$$
u R a n k = \left\{ \begin{array}{l l} 1 & \text {i f r a n k _ {M} <   r a n k _ {S 2 S}} \\ 0 & \text {o t h e r w i s e} \end{array} \right.
$$
where  $P_{m}(r)$  and  $P_{s2s}(r)$  are the probabilities of  $\{q, r, u\}$  given by  $M$  and  $s2s$  respectively,  $|X|$  presents the cardinal number of a set  $X$ , and the lower score of either  $\text{rank}_M$  or  $\text{rank}_{S2S}$  indicates a better ranking position. Overall, for model  $M$ , its average  $uRank$  for different queries denotes the rate of rank-promoted ground-truth replies.
3.2 Language Style Imitation
Apart from perceiving users' language styles, an effective persona-aware model should also be able to imitate language styles by generating responses satisfying users' language behaviors.
User-Language-Perplexity  $(uPPL)$  is proposed to measure this property.
Given a user  $u_{i}$ , to conduct such metric, a statistical language model  $LM_{i}$  is first trained using the user's utterances. After that, for a generated response  $r'$ , its corresponding  $uPPL$  is defined as the perplexity of  $r'$  given by  $LM_{i}$ .  $uPPL$  quantifies the power of a persona-aware model on generating responses similar to users' history utterances.
3.3 Diversity between Users
Finally yet importantly, due to the introduction of user information, given a query, we expect that responses for different users from a persona-aware model should be also diversified. Therefore, Users-Distinct (uDistant) is proposed in this paper to capture such property. Given a query  $q_{i}$  and  $m$  different users  $\{u_{j} | j \in [1,m]\}$ , we generate different responses  $\{r_j' | j \in [1,m]\}$  for each user using  $M$ . On this basis, Distinct-1 and Distinct-2 (Li et al., 2016a) of the response set  $\{r_j' | j \in [1,m]\}$  are utilized to measure the in-group diversity of responses generated by  $M$  within users. Li et al. (2016b) also compare models through the case studies from the similar perspective.
To evaluate the performance of our proposed method, we implement experiments on a Chinese Social Networking Service (SNS) corpus and the Cornell Movie Dialogues corpus (Danescu-Niculescu-Mizil and Lee, 2011). The Chinese SNS corpus is crawled from a Chinese social network service Douban, containing totally 1,022,592 single-turn dialogues from 12,857 users; while the Cornell Movie Dialogues corpus consists of conversations from movie scripts. By cleaning up the Cornell corpus with the open-source script, we obtain 109,952 single-turn dialogues from 9,035 movie characters. The training/test ratios for the two corpora are around 200:1 and 50:1, respectively. Besides, for the Douban corpus, the mean, maximum, minimum, and the standard deviation values of the number of utterances for each user are 80, 1190, 33, and 49, respectively. Meanwhile, these statistics values are 14, 237, 4, and 22, correspondingly.
<sup>1</sup>https://www.douban.com/group
$^{2}$ https://github.com/suriyadeepan/datasets/
There are two main differences between the two datasets: 1) The scenes of conversations are different. The dialogues in Douban are crawled from an open domain social media. By contrast, since the characters in Cornell movie corpus are assigned with fixed personas, the language styles and habits of users are more templatized. Besides, the language style in Cornell is more oral-like, with many personal pronouns. 2) The average number of utterances for each user of the Douban corpus is around 10 times more than that of Cornell.
4.2 Model Variations
S2SA Vanilla sequence-to-sequence model with attention (Sordoni et al., 2015).
fact.bias S2SA with fact bias for persona modeling (Michel and Neubig, 2018). fact.bias is originally proposed in NMT, it models user information as an additional bias vector learned through a factored model in the softmax layer.
Speaker Model Framework proposed by Li et al. (2016b). This model is similar to S2SA + fact.bias, except that the user information is added as a part of decoder input rather than bias in the softmax layer.
VAE Standard Variational AutoEncoder for response generation (Serban et al., 2017). In our experiment, we replace the utterance with the query only and apply the auxiliary BOW loss (Zhao et al., 2017) in training.
CVAE Conditional Variational AutoEncoder with user information as prior knowledge for modeling persona (Zhao et al., 2017). Similar to VAE, bag-of-words loss is applied in CVAE.
For a fair comparison, we use the same configuration for all models. The size of word embedding and user embedding are respectively set to 300 and 128. All the user embeddings, including that of the unknown user, are initialized randomly and trained during the optimizing. We employ a bi-directional LSTM of hidden size  $= 256$  for encoding, and a LSTM of hidden size  $= 512$  for decoding. For latent models, the dimension of  $z$  is set as 128.
All models are optimized using Adam (Kingma and Ba, 2014) with learning rate  $= 2e - 4$  and batch size  $= 128$ . For latent models, we also use KL annealing (Bowman et al., 2016) (400,000 batches for Douban corpus and 100,000 batches for Cornell Movie corpus) to achieve better performance.
4.3 Automatic Evaluation Metrics
To thoroughly evaluate our systems, both standard and persona-focused metrics are employed in our experiments. For standard metrics, we adopt unigram BLEU (BLEU-1) (Papineni et al., 2002) and Word Embedding metrics (Liu et al., 2016) including Embedding Average (Average), Vector Extrema (Extrema) and Greedy Matching (Greedy) to evaluate the semantics of generated responses with regards to ground truths. We use the pretrained word embeddings from (Song et al., 2018) for the Douban corpus and embeddings from (Pennington et al., 2014) for the Cornell movie corpus.
The three proposed metrics (uRank, uPPL and uDistinct) are adopted to measure the performance of capturing persona. For uPPL, we use a bi-gram language model for perplexity computation. Since the effectiveness of uPPL relies on the quality of constructed user language models, we pretrain the SLM with the whole training data and afterwards finetune it using each user's utterances. Besides, we drop the users with utterances less than 100 in Douban and 30 in Cornell. The value of uRank, which depends on the rankings of predicted probabilities of responses, is not stable for latent models due to the randomness on sampling  $z$ . Therefore, uRank for each latent model is computed by running 10 rounds, so that we obtain 10 ranking results and their corresponding uRank. Then we average the obtained 10 uRank as the final uRank for each latent enhanced model. The later experimental results show that uRank for any latent model varies slightly around  $\pm 0.005$  for each round.
4.4 The Human Evaluation Criterion
For further comparisons, we also use the crowdsourcing labeling resources of our organization to manually evaluate the relevance and the persona of generated responses. Since the degree of persona reflected in the response is even more difficult to be judged by humans, we simplify the annotation into a "yes or no" task, that is, annotators are only asked to decide whether the response can reflect persona for the given user. Before that, the annotators have to read all the utterances of each user to learn the persona for judging. Moreover, in practice, we limit the number of each user's sample utterances to 100. However, the judgment is inevitably much more subjective. Thus, for each sample, we recruit 11 annotators to label and make the final determination by voting. The evaluation
of relevance is relatively easy. For the evaluation of relevance, each query-response pair is cross-evaluated by 3 annotators, following the labeling criterion used in (Xing et al., 2017; Wang et al., 2018). The details of data sampling and labeling are given in the Supplementary Material.
5 Results & Analysis
5.1 Results on the Douban Corpus
We first report the performance on the Douban corpus. The results of automatic evaluating metrics are illustrated in Table 1, numbers in bold mean that the improvement on that metric is statistically significant over other methods (p-value  $\leq$  0.01). It is observed that the BLEU-1 scores of various models are relatively low and close to each other. We attribute this to the fact that the semantics of possible responses for one query is highly diversified in terms of speaking styles and topics, there might be the situation that only a small portion of words share among the responses except those of high-frequency words (Mou et al., 2016; Liu et al., 2016). However, user enhanced models achieve higher BLEU-1 scores due to their capability in considering the preference of a user.
Furthermore, by comparing the performances on embedding metrics, we find that all models obtain decent scores, but none of the models outperform the others significantly. Such phenomenons can also be observed in previous studies (Serban et al., 2017; Wang et al., 2019), since all the models generate responses semantically similar to the ground truths. Despite this, PAGenerator achieves the highest score on average, which suggests the responses generated by PAGenerator are more semantically relevant to the ground truths.