SlowGuess/ACL_2020_OCR huggingface dataset:
Haoyu Song $^{1}$ , Yan Wang $^{2}$ , Wei-Nan Zhang $^{1*}$ , Xiaojiang Liu $^{2}$ , Ting Liu $^{1}$ $^{1}$ Research Center for Social Computing and Information Retrieval  
Harbin Institute of Technology, Heilongjiang, China  
 $^{2}$ Tencent AI Lab, Shenzhen, China  
{hysong, wnzhang, tliu}@ir.hit.edu.cn  
{brandenwang, kieranliu}@tencent.com
Maintaining a consistent personality in conversations is quite natural for human beings, but is still a non-trivial task for machines. The persona-based dialogue generation task is thus introduced to tackle the personality-inconsistent problem by incorporating explicit persona text into dialogue generation models. Despite the success of existing persona-based models on generating human-like responses, their one-stage decoding framework can hardly avoid the generation of inconsistent persona words. In this work, we introduce a three-stage framework that employs a generate-delete-rewrite mechanism to delete inconsistent words from a generated response prototype and further rewrite it to a personality-consistent one. We carry out evaluations by both human and automatic metrics. Experiments on the Persona-Chat dataset show that our approach achieves good performance.
In an open-domain conversation scenario, two speakers conduct open-ended chit-chat from the initial greetings and usually come to focus on their characteristics, such as hobbies, pets, and occupations, etc., in the course of the conversation. For humans, they can easily carry out conversations according to their personalities (Song et al., 2019a), but fulfilling this task is still a challenge for recent neural dialogue models (Welleck et al., 2019).
One main issue is that these models are typically trained over millions of dialogues from different speakers, and the neural dialogue models have a propensity to mimic the response with the maximum likelihood in the training corpus (Li et al., 2016b), which results in the frequent inconsistency in responses (Zhang et al., 2018). Another issue
Figure 1: A common problem for persona-based dialogue models is that they can hardly avoid the generation of inconsistent persona words. Although the model generates a response which looks good, it is an inconsistent one. With further rewriting, the model can focus more on improving persona consistency.
is the user-sparsity problem (Qian et al., 2017) in conventional dialogue corpora (Serban et al., 2015). Some users have very few dialogue data, which makes it difficult for neural models to learn meaningful user representations (Li et al., 2016b).
To alleviate the above issues, Zhang et al. (2018) introduced the Persona-Chat dataset to build more consistent dialogue models. Different from conventional dialogue corpora, this dataset endows dialogue models with predefined personas, which is in the form of textually described profile (as shown in the first line of Figure 1). The persona-based dialogue models also adopt an encoder-decoder architecture and are enhanced with persona encoding components, such as memory network (Sukhbaatar et al., 2015) and latent variable (Kingma and Welling, 2013). These models turn out to produce more consistent responses than the persona-free ones (Zhang et al., 2018; Song et al., 2019a).
Despite the successful application of the encoder-decoder framework in persona-based dialogue models, one concern is that they lack extra attention to the key persona information. The model will learn
*This work was done when the first author was an intern at Tencent AI Lab. Wei-Nan Zhang is the corresponding author.
5821
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5821-5831 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
to minimize the overall loss of every decoded word, but this may lead to the neglect of the key personas: change of one persona-related word may not significantly affect the overall loss, but could turn a good response into a totally inconsistent one. As shown in Stage 1 of Figure 1, only one improper word "Colorado" leads the response to be inconsistent.
A desirable solution should be able to capture personas and automatically learn to avoid and refine inconsistent words before the response. In this paper, we present a Generate-Delete-Rewrite framework, GDR, to mitigate the generation of inconsistent personas. We design three stages specifically for the goal of generating persona consistent dialogues: The first Generate stage adopts a transformer-based generator to produce a persona-based response prototype. The second Delete stage employs a consistency matching model to identify inconsistencies and delete (by masking) the inconsistent words from the prototype. Finally, in the Rewrite stage, a rewriting polishes the masked prototype to be more persona consistent. To examine the effectiveness of our GDR model, we carried out experiments on the public available Persona-Chat dataset (Zhang et al., 2018).
We summarize the main contributions as follows:
- A three-stage end-to-end generative framework, GDR, was proposed for the generation of persona consistent dialogues.
- A matching model was integrated into the generation framework to detect and delete inconsistent words in response prototype.
- Experimental results show the proposed approach outperforms competitive baselines on both human and automatic metrics.
End-to-end dialogue generation approaches are a class of models for building open-domain dialogue systems, which have seen growing interests in recent years (Vinyals and Le, 2015; Shang et al., 2015; Serban et al., 2016; Li et al., 2016c; Zhao et al., 2017; Li et al., 2017). These dialogue models adopted recurrent units in a sequence to sequence (seq2seq) fashion (Sutskever et al., 2014). Since the transformer has been shown to be on par with or superior to the recurrent units (Vaswani et al., 2017), some dialogue models began to take advantage of
this architecture for better dialogue modeling (Dinan et al., 2018; Su et al., 2019).
Besides the advancements in dialogue models, the emergence of new dialogue corpus has also contributed to the research field. Zhang et al. (2018) introduced the Persona-Chat dataset, with explicit persona texts to each dialogue. Based on seq2seq model and memory network, they further proposed a model named Generative Profile Memory Network for this dataset. Following this line, Yavuz et al. (2019) designed the DeepCopy model, which leverages copy mechanism to incorporate persona texts. Song et al. (2019a) integrated persona texts into the Per-CVAE model for generating diverse responses. However, the persona-based models still face the inconsistency issue (Welleck et al., 2019). To model the persona consistency, Welleck et al. (2019) annotated the Persona-Chat dataset and introduced the Dialogue Natural Language Inference (DNLI) dataset. This dataset converts the detection of dialogue consistency into a natural language inference task (Bowman et al., 2015).
Personalized dialogue generation is an active research field (Li et al., 2016b; Qian et al., 2017; Zhang et al., 2018; Zheng et al., 2019a,b; Zhang et al., 2019). In parallel with this work, Song et al. (2019b) leveraged adversarial training to enhance the quality of personalized responses. Liu et al. (2020) incorporated mutual persona perception to build a more explainable (Liu et al., 2019) dialogue agent. Other relevant work lies in the area of multi-stage dialogue models (Lei et al., 2020). Some retrieval-guided dialogue models (Weston et al., 2018; Wu et al., 2019; Cai et al., 2019a,b; Su et al., 2020) also adopted a multi-stage framework, but the difference from our work is obvious: we generate the prototype rather than retrieve one. A high-quality retrieved response is not always available, especially under the persona-based setting.
In this work, we consider learning a generative dialogue model to ground the response with explicit persona. We focus on the persona consistency of single-turn responses, and we leave the modeling of multi-turn persona consistency as future work.
Formally, we use uppercase letters to represent sentences and lowercase letters to represent words. Let  $Q = q_{1}, q_{2}, \ldots, q_{n}$  denotes the input query with  $n$  words, and let  $P = \{P^{(1)}, P^{(2)}, \ldots, P^{(k)}\}$ .
5822
(1) Generate
(2) Delete
Figure 2: The overall architecture of our three-stage GDR model, including a prototype generator (Generate stage), a consistency matching model (Delete stage), and a masked prototype rewriter (Rewrite stage). The italics denote the inputs of each stage, and the boldfaces denote the outputs. All the attentions (attn) here refer to the multi-head attention. For the sake of brevity, we omitted some layers of the transformer in this figure.
(3) Rewrite
be the  $k$  different persona texts, where  $P^{(i)} = p_1^{(i)},p_2^{(i)},\ldots ,p_{m_i}^{(i)}$  is the  $i$ -th persona text with  $m_{i}$  words. Our goal is to learn a dialogue model  $\mathbb{M}$  to generate a response  $\hat{Y} = y_{1},y_{2},\dots,y_{k}$ , which is consistent with the persona, based on both query  $Q$  and persona  $P$ . In abbreviation,  $\hat{Y} = \mathbb{M}(Q,P)$ .
More concretely, as shown in Figure 2, the proposed model  $\mathbb{M}$  consists of three parts:
1) Prototype generator G. This component takes persona texts and query as input and generates a response prototype for further editing. It adopts an encoder-decoder architecture (Sutskever et al., 2014), with the transformer (Vaswani et al., 2017) applied in both the encoder and the decoder.
2) Consistency matching model D. This model is designed to detect and delete those words in the prototype that could lead to inconsistency. We train this model in a natural language inference fashion on the DNLI (Welleck et al., 2019) dataset.
3) Masked prototype rewritten R. The rewritten learns to rewrite the response prototype to a more consistent one. It is also a transformer decoder, which adopts a similar architecture as the decoder of G. The difference lies in that it takes the masked prototype, rather than the query, as input.
3.2 Generate: Prototype Generator
We apply the encoder-decoder structure to build our prototype generator G. For the encoder, we use the self-attentive encoder in the transformer. For the decoder, built upon the transformer decoder, we propose a tuple-interaction mechanism to model the relations among persona, query, and response.
Self-Attentive Encoder
As the persona  $P$  is composed of several sentences, we unfold all words in  $P$  into a sequence  $p_1^{(1)}, p_2^{(1)}, \ldots, p_{m_j}^{(i)}, \ldots, p_{m_k}^{(k)}$ .
Then we use the self-attentive encoder (Vaswani et al., 2017) to compute the representations of the persona texts and query separately. The multi-head attention is defined as MultiHead  $(Q, K, V)$ , where  $Q, K, V$  are query, key, and value, respectively. The encoder is composed of a stack of  $N_G$  identical layers. Take the first stack encoding of  $P$  for example:
$$
\mathbf {V} _ {p} ^ {(1)} = \operatorname {M u l t i H e a d} (\mathbf {I} (P), \mathbf {I} (P), \mathbf {I} (P)), \quad (1)
$$
$$
\mathrm {O} _ {p} ^ {(1)} = \operatorname {F F N} \left(\mathrm {V} _ {p} ^ {(1)}\right), \tag {2}
$$
$$
\operatorname {F F N} (x) = \max  \left(0, x W _ {1} + b _ {1}\right) W _ {2} + b _ {2}, \tag {3}
$$
where  $\mathbf{V}^{(1)}$  is the first layer result of the multi-head self-attention and  $\mathrm{I}(\cdot)$  is the embedding function of the input. The input embedding for word  $w_{i}$  is the sum of its word embedding and position embedding.  $\mathrm{O}^{(1)}$  denotes the output of the first layer feed-forward network. For other layers:
$$
\mathrm {V} _ {p} ^ {(n)} = \text {M u l t i H e a d} \left(\mathrm {O} _ {p} ^ {(n - 1)}, \mathrm {O} _ {p} ^ {(n - 1)}\right), \mathrm {O} _ {p} ^ {(n - 1)} \tag {4}
$$
$$
\mathrm {O} _ {p} ^ {(n)} = \operatorname {F F N} \left(\mathrm {V} _ {p} ^ {(n)}\right), \tag {5}
$$
where  $\mathrm{n} = 2,\dots,N_G$ . We applied layer normalization to each sublayer by LayerNorm  $(x + \mathrm{Sublayer}(x))$ .  $Q$  is encoded in the same way. After  $N_G$  identical layers, we can get the final representations  $\mathrm{O}_p^{(N_G)}$  and  $\mathrm{O}_q^{(N_G)}$ , where  $\mathrm{O}_p^{(N_G)}$  and  $\mathrm{O}_q^{(N_G)}$  are the encoded persona and encoded query, respectively.
5823
Tuple-Interaction Decoder
In the decoding phase, there are three types of information, persona  $P$ , query  $Q$ , and response  $Y$ , which make up a tuple  $(P,Q,Y)$ . Accordingly, three inter-sentence relations need to be considered: (1) The alignment between  $Q$  and  $Y$  is beneficial to yield better results (Bahdanau et al., 2014). (2) As the persona is composed of several sentences and describes different aspects, we need to find the most relevant persona information according to the relations between P and Y. (3) We also want to know whether the query needs to be answered with the given persona. Thus we should take the relations between  $P$  and  $Q$  into account.
Considering the above factors, we design a two-layer tuple-interaction mechanism in the decoder, as shown in the first part of Figure 2. There are three attentions in two layers: query attention (Q-Attn) and persona attention (P-Attn) in the first layer, and persona-query attention (PQ-Attn) in the second layer.  $N_{G}$  such identical layers compose of the decoder. For the first layer:
$$
\mathbf {V} _ {y} ^ {(1)} = \operatorname {M u l t i H e a d} (\mathbf {I} (Y), \mathbf {I} (Y), \mathbf {I} (Y)), \quad (6)
$$
$$
\mathrm {E} ^ {(1)} = \operatorname {M u l t i H e a d} \left(\mathrm {V} _ {y} ^ {(1)}, \mathrm {O} _ {p} ^ {\left(N _ {G}\right)}, \mathrm {O} _ {p} ^ {\left(N _ {G}\right)}\right), \tag {7}
$$
$$
\mathbf {F} ^ {(1)} = \operatorname {M u l t i H e a d} \left(\mathrm {V} _ {y} ^ {(1)}, \mathrm {O} _ {q} ^ {\left(N _ {G}\right)}, \mathrm {O} _ {q} ^ {\left(N _ {G}\right)}\right), \tag {8}
$$
$$
\mathrm {T} ^ {(1)} = \operatorname {M u l t i H e a d} \left(\mathrm {E} ^ {(1)}, \mathrm {F} ^ {(1)}, \mathrm {F} ^ {(1)}\right), \tag {9}
$$
$$
\mathrm {O} _ {d e c} ^ {(1)} = \operatorname {F N N} (\operatorname {m e a n} \left(\mathrm {E} ^ {(1)}, \mathrm {F} ^ {(1)}, \mathrm {T} ^ {(1)}\right)), \tag {10}
$$
where  $\mathbf{E}^{(1)}$  and  $\mathbf{F}^{(1)}$  are the results of the first layer P-Attn and Q-Attn.  $\mathbf{T}^{(1)}$  is the result of the first layer PQ-Attn.  $\mathrm{O}_{dec}^{(1)}$  denotes the first layer output. Note that the  $Y$  here is masked to ensure depending only on the known words (Vaswani et al., 2017). Repeatedly, for other layers:
$$
\mathrm {V} _ {y} ^ {(n)} = \text {M u l t i H e a d} \left(\mathrm {O} _ {\text {d e c}} ^ {(n - 1)}\right), \left. \mathrm {O} _ {\text {d e c}} ^ {(n - 1)}\right), \left. \mathrm {O} _ {\text {d e c}} ^ {(n - 1)}\right), \tag {11}
$$
$$
\mathrm {O} _ {d e c} ^ {(n)} = \operatorname {F N N} \left(\operatorname {m e a n} \left(\mathrm {E} ^ {(n)}, \mathrm {F} ^ {(n)}, \mathrm {T} ^ {(n)}\right)\right), \tag {12}
$$
where  $n = 2, \dots, N_G$ . After  $N_G$  layers, the decoder output  $\mathrm{O}_{dec}^{(N_G)}$  is projected from hidden size to vocabulary size, then followed up by a softmax function to get the words' probabilities:
$$
\operatorname {P r o b} ^ {(1)} = \operatorname {S o f t M a x} \left(\mathrm {O} _ {d e c} ^ {\left(N _ {G}\right)} W _ {3} + b _ {3}\right), \tag {13}
$$
where  $W_{3}$  is a hidden size  $\times$  vocabulary size weight matrix and  $b_{3}$  is the bias term with vocabulary size dimension. And  $\mathrm{Prob}^{(1)}$  denotes the output distribution of the first stage. Now we can get the response prototype  $\hat{Y}^{(1)}$  from the  $\mathrm{Prob}^{(1)}$ .
Figure 3: The architecture of our consistency matching model. “·” and “—” denote element-wise product and difference. The dotted line shows inference process, including consistency matching and word deleting.
3.3 Delete: Consistency Matching Model
The goal of the consistency matching model D is to reveal word-level consistency between the persona texts and the response prototype, thus the inappropriate words can be deleted from the prototype.
This model is trained to estimate the sentence-level entailment category (Bowman et al., 2015) of a response for the given persona texts, which includes entailment, neutral and contradiction. The key is that if the category is not entailment, we can delete the most contributing words by replacing them with a special mask token, thus giving the model a chance to rephrase. The attention weights can measure each word's contribution.
The architecture of our consistency matching model is shown in Figure 3. From bottom to top are the self-attentive encoding layer, cross attention layer, and consistency matching layer.
As described in section 3.2, the self-attentive encoder  $(\mathrm{SAE}(\cdot))$  performs self-attention over the input to get sentence representations. Because the task of consistency matching is quite different from dialogue generation, we did not share the encoders between the generator G and matching model  $D$ :
$$
\bar {\mathrm {A}} = \operatorname {S A E} _ {D} (P), \tag {14}
$$
$$
\bar {\mathbf {B}} = \operatorname {S A E} _ {D} \left(\hat {Y} ^ {(1)}\right), \tag {15}
$$
where  $\bar{\mathbf{A}}$  is a hidden size  $\times \mathfrak{n}$  matrix.  $\bar{\mathbf{A}} = [\bar{a}_1,\bar{a}_2,\dots,\bar{a}_n]$  and  $\bar{\mathbf{B}} = [\bar{b}_1,\bar{b}_2,\dots,\bar{b}_m]$ . The  $n$  and  $m$  are the number of words in persona  $P$  and response prototype  $\hat{Y}^{(1)}$ . Here we applied average pooling stragety (Liu et al., 2016; Chen et al., 2017)
5824
to get the summary representations:
$$
\bar {a} _ {0} = \sum_ {i = 1} ^ {n} \frac {\bar {a} _ {i}}{n}, \tag {16}
$$
and we can get the response attention weights and attentive response representations by:
$$
\mathrm {W} _ {b} = \bar {\mathrm {a}} _ {0} ^ {\top} \bar {\mathrm {B}}, \tag {17}
$$
$$
\widetilde {\mathbf {B}} = \mathbf {W} _ {b} \bar {\mathbf {B}} ^ {\top}, \tag {18}
$$
where  $\mathbf{W}_b$  is attention weights and  $\widetilde{\mathbf{B}}$  is response representations. Similarly, we can get  $\mathbf{W}_a$  and  $\widetilde{\mathbf{A}}$ .
Once A and B are generated, three matching methods (Chen et al., 2017) are applied to extract relations: concatenation, element-wise product, element-wise difference. The results of these matching methods are concatenated to feed into a multi-layer perceptron, which has three layers and tanh activation in between. The output is followed up by a SoftMax function to produce probabilities.
In the inference process, as shown in Figure 3, the response attention weights  $\mathbf{W}_b$  is leveraged to illustrate the inconsistent words, which will be deleted<sup>1</sup>. In practice, we use a simple heuristic rule for deleting words: as long as the category is not entailment, we will delete  $10\%$  of the words (at least one word)<sup>2</sup>, with the highest attention weight, in the prototype  $\hat{Y}^{(1)}$ . In this way, we get the masked prototype  $\hat{Y}^{(2)}$ .
3.4 Rewrite: Masked Prototype Rewriter
The rewriter R takes the masked prototype and persona texts as input and outputs the final response.
R is also a transformer decoder, which is similar to the decoder of G in section 3.2, but with a minor difference: the masked prototype is close to the target response, thus the direct attention between the prototype and target response is needless. The architecture of R can be seen in the third part of Figure 2, which can be formalized as:
$$
\mathrm {O} _ {m p} ^ {(N _ {G})} = \operatorname {S A E} _ {G} (\hat {Y} ^ {(2)}), \tag {19}
$$
$$
\mathrm {V} ^ {(n)} = \operatorname {M u l t i H e a d} \left(\mathrm {O} _ {r w} ^ {(n - 1)}, \mathrm {O} _ {r w} ^ {(n - 1)}\right), \left. \mathrm {O} _ {r w} ^ {(n - 1)}\right), \tag {20}
$$
$$
\mathbf {S} ^ {(n)} = \operatorname {M u l t i H e a d} \left(\mathrm {V} ^ {(n)}, \mathrm {O} _ {p} ^ {\left(N _ {G}\right)}, \mathrm {O} _ {p} ^ {\left(N _ {G}\right)}\right), \tag {21}
$$
$$
\mathrm {K} ^ {(n)} = \operatorname {M u l t i H e a d} \left(\mathrm {S} ^ {(n)}, \mathrm {O} _ {m p} ^ {\left(N _ {G}\right)}, \mathrm {O} _ {m p} ^ {\left(N _ {G}\right)}\right), \tag {22}
$$
$$
\mathrm {O} _ {r w} ^ {(n)} = \operatorname {F N N} \left(\operatorname {m e a n} \left(\mathrm {S} ^ {(n)}, \mathrm {K} ^ {(n)}\right)\right), \tag {23}
$$
Data	Train	Valid	Test
Persona Texts	74,522	5,843	4,483
Q-R Pairs	121,880	9,558	7,801
Table 1: Some statistics of Persona-Chat dataset. Valid denotes Validate and Q-R denotes Query-Response.
Label	Train	Valid	Test
Entailment	100,000	5,500	5,400
Neutral	100,000	5,500	5,400
Contradiction	110,110	5,500	5,700
Table 2: Key statistics of DNLI dataset.
where  $\mathrm{O}_{mp}^{(N_G)}$  is the encoded masked prototype and  $\mathrm{SAE}_G$  is the self-attentive encoder of G.  $\mathrm{O}_p^{(N_G)}$  is the encoded persona. After  $N_{R}$  identical layers, the same generation process as in G is applied to the  $\mathrm{O}_{rw}^{(N_R)}$ , and we can get the final response  $\hat{Y}^{(3)}$ .
The consistency matching model D is trained separately from the prototype generator G and rewriter R. As forementioned, the matching model D is trained in a natural language inference fashion on the DNLI dataset (Welleck et al., 2019), which has been well defined by the previous studies (Bowman et al., 2015; Chen et al., 2017; Gong et al., 2018). We minimize the CrossEntropy loss between the outputs of D and the ground truth labels.
The G and R share the same training targets. We trained them by the standard maximum likelihood estimate. Notice that there are two different deleting strategies in training: (1) In the warm-up phase, because the G can hardly generate high-quality prototypes at this period, we randomly delete each word, with a  $10\%$  probability, from the prototype. (2) After that, the trained consistency matching model D is leveraged to delete words.
We carried out the persona-based dialogue generation experiments on the public available PersonaChat dataset (Zhang et al., 2018). Furthermore, we trained the consistency matching model on the recently released Dialogue Natural Language Inference (DNLI) dataset (Welleck et al., 2019).
We show the statistics of the Persona-Chat
<sup>1</sup>In this paper, "delete" a word means replacing this word with a special mask token.
In our experiments, we found that deleting more words made it difficult for rewriter R to learn.
dataset in Table 1. The DNLI dataset (Welleck et al., 2019) is an enhancement to the Persona-Chat. It is composed of persona-utterance pairs from the Persona-Chat, and these pairs are further labeled as entailment, neutral, and contradiction. Some statistics of this dataset are given in Table 2.
4.2 Compared Models
To the best of our knowledge, this is an early work in modeling explicit persona consistency. To show the effectiveness of our models, we mainly compare it with the persona-based dialogue models:
- S2SA S2SA is an RNN-based attentive seq2seq model (Bahdanau et al., 2014). It only takes the query as input.
- Per-S2SA This is a seq2seq model that prepends all persona texts to the query as input (Zhang et al., 2018).
- GPMN Generative Profile Memory Network is an RNN-based model that encodes persona texts as individual memory representations in a memory network (Zhang et al., 2018).
- DeepCopy An RNN-based hierarchical pointer network, which leverages copy mechanism to integrate persona (Yavuz et al., 2019).
- Per-CVAE This is a memory augmented CVAE model to exploit persona texts for diverse response generation (Song et al., 2019a).
- Transformer Different from the RNN-based models, transformer is a self-attention based sequence transduction model (Vaswani et al., 2017). The persona texts are concatenated to the query to serve as its input.
For all the RNN-based baseline models, they are implemented by two-layer LSTM networks with a hidden size 512. For the Transformer, the hidden size is also set to 512, and the layers of both encoder and decoder are 3. The number of heads in multi-head attention is 8, and the inner-layer size of the feedforward network is 2048. The word embeddings are randomly initialized, and the embedding dimension of all models is set to 512.