SlowGuess/NAACL_2022_OCR huggingface dataset:
Xueliang Zhao $^{1,2}$ , Tingchen Fu $^{3}$ , Chongyang Tao $^{4}$ , Wei Wu $^{5}$ , Dongyan Zhao $^{1,2*}$ , Rui Yan $^{3*}$
$^{2}$ Center for Data Science, AAIS, Peking University
$^{3}$ Gaoling School of Artificial Intelligence, Renmin University of China
4Microsoft 5Meituan, Beijing, China
{x1.zhao,zhaody}@pku.edu.cn ruiyan@ruc.edu.cn
{lucas.futingchen,chongyangtao,wuwei19850318}@gmail.com
Grounding dialogue generation by extra knowledge has shown great potentials towards building a system capable of replying with knowledgeable and engaging responses. Existing studies focus on how to synthesize a response with proper knowledge, yet neglect that the same knowledge could be expressed differently by speakers even under the same context. In this work, we mainly consider two aspects of knowledge expression, namely the structure of the response and style of the content in each part. We therefore introduce two sequential latent variables to represent the structure and the content style respectively. We propose a segmentation-based generation model and optimize the model by a variational approach to discover the underlying pattern of knowledge expression in a response. Evaluation results on two benchmarks indicate that our model can learn the structure style defined by a few examples and generate responses in desired content style.
Building an open domain dialogue system has attracted increasing attention from the community of AI and NLP. Despite the impressive progress, existing models are notorious for replying with generic and bland responses. To bridge the gap, researchers resort to ground dialogue generation by extra knowledge such as unstructured documents (Zhou et al., 2018c; Dinan et al., 2019). By this means, the documents serve as content sources and make a dialogue system knowledgeable regarding various concepts in a discussion.
However, existing studies focus on how to synthesize a response with proper knowledge (Dinan et al., 2019; Kim et al., 2020; Zhao et al., 2020b), but pay little attention to the fact that the same knowledge could be expressed differently even under the same context. These models usually em
Knowledge
• MovieName: Frozen
• Year: 2013
• Rating: Rotten Tomatoes: 89%, Metacritics: 74/100, CinemaScore: A+
• Cast: Kristen Bell as Anna, the 18-year-old Princess of Arendelle and Elsa's younger sister, Livvy Stubenrauch as 5-year-old Anna, Katie Lopez as 5-year-old Anna (singing), Agatha Lee Monn as 9-year-old Anna ...
...
Conversations
User:	I was really surprised that disney chose Kristen Bell to be the voice of Anna in Frozen	I was really surprised that disney chose Kristen Bell to be the voice of Anna in Frozen
User2:	Yes, I didn't imagine it'd be her!	Yes, I didn't imagine it'd be her!
User2:	What do you think about the rating?	What do you think about the rating?
User:	74 in Metacritics. I believe it deserves, indeed.	The rating is 74 in Metacritics. Let me say, high enough for a Disney move
User:	And I'd give credit to three different voice actors for anna. I'm really impressed. What about you?	And I do think it was overkill to use three different voice actors for anna. Do you agree?
Table 1: A case from CMU.DoG. Given the same knowledge and context, the last two turns in left and right conversations exhibit positive and negative sentiments, respectively. Each utterance can be decomposed into knowledge-related and knowledge-irrelevant segments.
ploy a regular decoder to generate the response in an auto-regressive manner given the contextual representations of knowledge and dialogue context, which makes the generation process less explainable and controllable.
In general, the expression style of response is composed of two aspects: the structure of the response and the style of the content in each part. As the example shown in Table 1, knowledge-related phrases and clauses tend to be long, like "And I'd give credit to three different voice actors for anna.", or short, like "74 in Metacritics". Besides, they may appear at the beginning of the sentence, or at the end. For the sake of description, we decompose a response into a sequence of non-overlapping segments, each is either related to certain background knowledge and diverse in content style, or almost irrelevant to the knowledge but simply playing the role of stitching the context and carrying on the conversation. We therefore define the structure style as the distribution and number of two kinds of segments. The structure style itself is far from dominant in the sentence expression, since different speakers could convey converse attitude even if the
*Corresponding authors: Dongyan Zhao and Rui Yan.
Human Language Technologies, pages 2258 - 2273
context and the knowledge are exactly the same. So it is necessary to introduce the content style as the expression fashion within each knowledge-related segment. We further introduce two latent variables to facilitate end-to-end training, one for predicting the start and end positions of a segment, the other for deciding the category of each segment. Since the human annotations for sentence segmentation are absent and enumerating over all possibilities to maximize the likelihood of the response is time-consuming, we propose a variational framework for segmentation-based generation and induce an evidence lower bound of the likelihood.
Formally, our model follows an encoder-decoder architecture. The encoder is to obtain the contextual representation of conversational context and knowledge in a regular way. The decoder consists of three types of modules: (1) context module, for response only based on context without knowledge; (2) plain-knowledge module, for response referring knowledge but without particular style; and (3) stylized-knowledge module, for response referring knowledge and with a specific style. The context module is the only module not relying on knowledge, but simply paying attention to contextual information. Compared with plain-knowledge module, stylized-knowledge module has unique adapters, which is their primary discrepancy. When decoding, the decoder first predicts the segmentation of the response and then makes a choice in three kinds of modules to generate a single segment. Both the segmentation and the module selection are instructed under sequential latent variables.
We train our model on the Reddit Corpus published by Li et al. (2020) and evaluate our model on two benchmarks of knowledge-grounded conversation: Wizard of Wikipedia (Wizard) (Dinan et al., 2019) and CMU Document Grounded Conversation (CMU_DOG) (Zhou et al., 2018c). Evaluation results indicate that our model can significantly outperform state-of-the-art methods in the zero-resource setting (i.e., only trained on the Reddit Corpus). In addition, the performance of our model improves significantly on Wizard and CMU_DOG with the presence of only  $10\%$  training data and the segment distributions after fine-tuning are consistent with our prior knowledge about the two datasets, indicating that our model can learn the structure style with little cost. Finally, our model outperforms previous state-of-the-art models on the accuracy of performing sentiment classification us
ing generated responses, which indicates that the model can be controlled to express knowledge with the desired content style.
Contributions in this work are three-fold: (1) exploration of the knowledge expression in knowledge-grounded conversation; (2) proposal of a variational segmentation-based generation model to discover the underlying expression style in a response; (3) empirical verification of the effectiveness of the proposed model on two benchmarks of knowledge-grounded conversation.
On the vanilla encoder-decoder architecture (Shang et al., 2015; Vinyals and Le, 2015), various extensions have been made to model the structure of dialogue contexts (Serban et al., 2016, 2017; Zhang et al., 2019a); to improve diversity of responses (Li et al., 2015; Xing et al., 2017; Zhao et al., 2017; Tao et al., 2018); to control attributes of responses (Xu et al., 2019; Zhou et al., 2018a; Wang et al., 2018; See et al., 2019); and to bias responses to some specific personas (Li et al., 2016; Zhang et al., 2018). Recently, grounding dialogue generation by extra knowledge has seemed promising to bridge the gap between conversation with existing systems and conversation with humans, and the knowledge could be obtained from knowledge graphs (Zhou et al., 2018b; Moon et al., 2019; Tuan et al., 2019), retrieved from unstructured documents (Dinan et al., 2019; Lian et al., 2019; Zhao et al., 2019, 2020a; Kim et al., 2020; Li et al., 2020; Fu et al., 2022) or visual background (Mostafazadeh et al., 2017; Shuster et al., 2018; Huber et al., 2018). In this work, we study document-grounded dialogue generation. Rather than selecting knowledge relevant to dialogue context and directly exploiting pretrained language models to generate the response, we focus on expressing knowledge in this task.
The idea of sequence modeling via segmentation (Wang et al., 2017; Kim et al., 2019) has attracted widespread attention in several natural language processing tasks. In text segmentation task, Wang et al. (2017) propose a probabilistic model for sequence modeling via their segmentation and a "Sleep-WAke Network" (SWAN) method. In machine translation, Huang et al. (2017) propose a neural phrase-based machine translation system that models phrase structures in the target language using SWAN. In data-to-text generation, Wiseman et al. (2018) develop a neural template-like genera
tion model with a HSMM decoder, which is learned tractably by backpropagating through a dynamic program; to tackle the problem of weak Markov assumption for the segment transition probability, Shen et al. (2020) propose to explicitly segment target text into fragments and align them with their data correspondences, and jointly learn the segmentation and correspondence via dynamic programming.
We start by providing the problem formalization and overview of the proposed model in Sec.3.1. Then in Sec.3.2 we describe the design for each components. Lastly, we elaborate how to optimize the components with variational inference and weak supervision in Sec.3.3.
Suppose that we have a dataset  $\mathcal{D} = \{(U_i, K_i, R_i)\}_{i=1}^N$ , where  $\forall i \in \{1, \ldots, N\}$ ,  $K_i$  serves as background knowledge of the dialogue  $(U_i, R_i)$  with  $K_{i,j}$  being the  $j$ -th sentence,  $U_i$  is the context of the dialogue with  $U_{i,j}$  the  $j$ -th utterance, and  $R_i$  is the response. To bias the expression to a specific structure style, we further assume that there are a few examples  $\mathcal{D}_{sty} = \{(U_i, K_i, R_i)\}_{i=1}^M$  provided by users depicting the required style for knowledge expression. Note that we have  $N \gg M$ , since corpus in a specific expression style is rare and difficult to acquire. The goal is to learn a generation model  $p_\theta(R|U, K)$  ( $\theta$  denotes the parameters of the model) from  $\mathcal{D}$ , to generate a response  $R$  following  $p_\theta(R|U, K)$  given a new dialogue context  $U$  and the associated knowledge  $K$ . Different from previous KGC generation model, we allow users to either (1) bias the structure style of  $P_\theta(R|U, K)$  to  $\mathcal{D}_{sty}$  with little cost; or (2) switch the content style of knowledge expression in  $R$ .
Figure 1 gives an overview of the proposed model, which is based on the encoder-decoder architecture. The encoder generates the contextual representations of the dialogue and knowledge, while the decoder generates the segments one after another.  $\mathbf{h}_t^N$  encodes the dialogue context up to timestep  $t - 1$  with  $N$  denoting the number of decoder layers. Given  $R = (r_1,\dots ,r_t,\dots ,r_{l_r})$  with  $r_t$  referring the  $t$ -th token of  $R$  whose length is supposed to be  $l_{r}$ , the variable  $Z = \{z_{t}\}_{t = 1}^{l_{r}}$  is utilized to control the choice of module of each
segment (Module Indicator), and its historical information is encoded by  $\{\mathbf{c}_t\}_{t=0}^{l_r}$ .  $M = \{m_t\}_{t=1}^{l_r}$  is a sequence of binary variables and used to determine the boundary of each segment (Boundary Indicator). Specifically,  $m_t = 1$  indicating that the current segment is already completed and a new segment should be created at the next timestep. Otherwise,  $m_t = 0$  and the current segment remains unfinished. The generative process is disassembled into two steps: (1) determine the type of a new segment based on previously generated text and previous segment types; (2) generate within the current segment until the binary variable  $m_t = 1$ .
Context and Knowledge Encoding. We exploit the pre-trained BART (Lewis et al., 2020) as the backbone of our architecture, which is pretrained using a variety of denoising objectives and achieves state-of-the-art results on a range of text generation tasks. Given the dialogue context  $U = (U_{1},\dots ,U_{n})$ , we simply concatenate them as  $(u_{1},\dots ,u_{l_{u}})$ . Similarly, we concatenate the associated knowledge  $K = (K_{1},\dots ,K_{m})$  as  $(k_{1},\dots ,k_{l_{k}})$ .  $l_{u}$  and  $l_{k}$  are the length of dialogue context and background knowledge respectively. The input of the encoder is then defined as:
$$
I = [ \mathrm {B O S} ] k _ {1} \dots k _ {l _ {k}} [ \mathrm {E O S} ] u _ {1} \dots u _ {l _ {u}} [ \mathrm {E O S} ]. \tag {1}
$$
The input  $I$  is truncated or padded to the maximum capacity and then passes through the stacked self-attention layers and results in a knowledge-aware context representation  $\mathbf{C}$ , and a context-aware knowledge representation  $\mathbf{K}$ . Specifically, the context-aware knowledge representation is defined as  $\mathbf{K} = [\mathbf{h}_1^{enc},\dots ,\mathbf{h}_{l_k + 1}^{enc}]$  where  $\mathbf{h}_t^{enc}$  is the last layer of BART encoder at time  $t$ . Similarly, the knowledge-aware context representation is defined as  $\mathbf{C} = [\mathbf{h}_{l_k + 2}^{enc},\dots ,\mathbf{h}_{l_k + l_u + 2}^{enc}]$ .
Prior of Module Indicator. We use the sequential discrete latent variable  $Z = \{z_{t}\}_{t=1}^{l_{r}}$  to decide which module to invoke at each timestep. The transition of  $z_{t}$  occurs only when a segment is completed, which is decided by the binary boundary variable  $M$ . The prior quantifies the distribution of  $z_{t}$  before we observe the segment, and it is reasonable to assume that the prior of  $z_{t}$  depends on previous module choices  $z_{<t}$  and previously generated text. Then the transition of  $Z$  is defined as:
Figure 1: Architecture of the proposed model. "Copy" means  $m_t = 0$  and the state of the module indicator  $c_{t+1}$  remains unchanged. "Update" indicates that  $m_t = 1$  and the state has been updated to include the information from the previous segment.
$$
\begin{array}{l} p _ {\theta_ {z}} \left(z _ {t} \mid r _ {<   t}, z _ {<   t}, m _ {t - 1}\right) = m _ {t - 1} \cdot \tilde {p} \left(z _ {t} \mid \mathbf {c} _ {t}\right) \\ + (1 - m _ {t - 1}) \cdot \delta \left(z _ {t} = z _ {t - 1}\right), \tag {2} \\ \end{array}
$$
where  $\delta$  is a Kronecker delta function.  $\mathbf{c}_t$  encodes all previous latent states  $z_{< t}$  and generated text  $r_{< t}$  as follows:
$$
\mathbf {c} _ {t} = m _ {t - 1} \cdot f _ {z - \operatorname {r n n}} \left(\tilde {\mathbf {z}} _ {t - 1}, \mathbf {c} _ {t - 1}\right) + (1 - m _ {t - 1}) \cdot \mathbf {c} _ {t - 1}. \tag {3}
$$
$\tilde{\mathbf{z}}_{t - 1} = [\mathbf{e}_{t - 1};\mathbf{h}_{t - 1}^{N,dec}]$  with  $\mathbf{e}_{t - 1}$  the embedding of  $z_{t - 1}$  and  $\mathbf{h}_{t - 1}^{N,dec}$  the representation of last generated token. Specifically,  $m_{t - 1} = 0$  means that the next timestep  $t$  is still in the same segment as the previous timestep  $t - 1$  and thus the latent variable  $z_{t}$  should not be updated. Otherwise, it means that current segment is completed and  $z_{t}$  is updated with the transition function  $\tilde{p} (z_t|\mathbf{c}_t)$ . Because we only have  $N_{sty} + 2$  options when choosing a module, where  $N_{sty}$  is the number of different user-defined styles in addition to 2 default styles, so in this model, the latent variable  $z_{t}$  ranges in natural integer to denote corresponding style type. Specifically,  $z_{t} = 0$  denotes choosing the context expression module to generate a knowledge-irrelevant segment;  $z_{t} = 1$  tells the model to choose the knowledge expression module without specially customized style; we leave the  $z_{t}\geq 2$  to be user-defined so as to select the knowledge expression module combined with customized style. The transition function  $\tilde{p} (z_t|\mathbf{c}_t)$  is
then implemented as a multinomial distribution parameterized by  $\operatorname{Softmax}(f_{z - \mathrm{mlp}}(\mathbf{c}_t))^1$ .
Prior of Boundary Indicator. The boundary indicator  $M = \{m_t\}_{t=1}^{l_r}$  depicts the segmental structure of the response, with  $m_t = 1$  indicates that a new segment should start at time  $t + 1$ . Presumably, the prior of  $m_t$  could be inferred from  $r_{\leq t}$  and  $z_t$ . We model the distribution  $p_{\theta_m}(m_t | r_{\leq t}, z_t)$  by a Bernoulli distribution parameterized by  $\sigma(f_{m - \mathrm{mlp}}([e_t; h_t^{N,dec}]))$ , where  $\sigma$  denotes the sigmoid function.
Stylized Generation. As mentioned above, the generation process involves scheduling different modules according to  $z_{t}$ . Here we give a systematic description of the generation process. The decoder accepts the token generated last timestep  $r_{t-1}$  as input, performs transformation in  $N$  decoder layers, finally obtains a dense representation.
We use  $\mathbf{h}_t^l$  to denote the hidden state after the  $l$ -th layer at timestep  $t$ , which is a shorthand for  $\mathbf{h}_t^{l,dec}$  for brevity. Specially,  $\mathbf{h}_t^0$  is the output of the embedding layer. When  $z_{t} = 0$ , it implies that knowledge encoding is unnecessary for current segment so  $\mathbf{h}_t^l$  is defined as:
$$
\mathbf {h} _ {t} ^ {l} = \operatorname {D e c o d e r L a y e r} \left(\mathbf {h} _ {t} ^ {l - 1}, \mathbf {H} _ {t - 1} ^ {l - 1}, \mathbf {C}\right), \tag {4}
$$
where  $\mathbf{H}_{t - 1}^l = [\mathbf{h}_1^l,\dots ,\mathbf{h}_{t - 1}^l ]$  is a sequence of decoder hidden states in previous timesteps, and
<sup>1</sup>We use  $f_{* - \mathrm{mlp}}$  to denote a multi-layer perceptron network in this paper.
$\mathbf{C}$  is the context representation mentioned above. DecoderLayer  $(\cdot ,\cdot ,\cdot)$  is implemented as pre-trained BART decoder layer where  $\mathbf{h}_t^{l - 1}$  first plays selfattention on  $\mathbf{H}_{t - 1}^{l - 1}$  then performs cross-attention on  $\mathbf{C}$ . The probability  $p(r_t|r_{< t},z_t = 0)$  is defined as a multinomial distribution parameterized by Softmax  $(f_{r - \mathrm{mlp}}(\mathbf{h}_t^N))$  ,where  $\mathbf{h}_t^N$  encodes the generated tokens up to timestep  $t - 1$ . When  $z_{t} = 1$  the implementation of decoder layer is analogous to the  $z_{t} = 0$  case except that we replace  $\mathbf{C}$  with  $\mathbf{K}$ , since knowledge is needed:
$$
\mathbf {h} _ {t} ^ {l} = \operatorname {D e c o d e r L a y e r} \left(\mathbf {h} _ {t} ^ {l - 1}, \mathbf {H} _ {t - 1} ^ {l - 1}, \mathbf {K}\right). \quad (5)
$$
To generate a segment with a particular customized style when  $z_{t} \geq 2$ , we introduce some adapters to bias the generation efficiently following Houlsby et al. (2019). Specifically, the hidden state  $\mathbf{h}_t^l$  is defined as:
$$
\mathbf {h} _ {t} ^ {l} = \operatorname {D e c o d e r L a y e r} _ {a d p} (\mathbf {h} _ {t} ^ {l - 1}, \mathbf {H} _ {t - 1} ^ {l - 1}, \mathbf {K}), \quad (6)
$$
where  $\mathrm{DecoderLayer}_{\mathrm{adp}}(\cdot, \cdot, \cdot)$  denotes the transformer decoder layer with adapters inserted. To make the style fine-grained and adjustable, each style has a unique set of adapters. Different styles have no adapter in common. In addition, our model has the ability to learn to express in any style, as long as a discriminator for the desired style is provided.
3.3 Learning Details
We introduce auxiliary distributions  $q_{\phi_m}(M|R) = \prod_{t=1}^{l_r} q_{\phi_m}(m_t|R)$  and  $q_{\phi_z}(Z|M,R) = \prod_{t=1}^{l_r} q_{\phi_z}(z_t|M,R)$ , which serve as an approximation to the intractable posterior of the boundary indicator  $M$  and the module indicator  $Z$ . We then apply variational approximation which gives the following evidence lower bound objective (ELBO) (Hoffman et al., 2013):
$$
\begin{array}{l} \log p _ {\theta} (R | U, K) \\ \geq \mathbb {E} _ {q _ {\phi_ {m}} (M | R)} \left(\mathbb {E} _ {q _ {\phi_ {z}} (Z | M, R)} \sum_ {t = 1} ^ {l _ {r}} \log p _ {\theta} \left(r _ {t} \mid r _ {<   t}, z _ {t}\right) \right. \\ \left. - \sum_ {t = 1} ^ {l _ {r}} m _ {t - 1} \cdot D _ {\mathrm {K L}} \left(q _ {\phi_ {z}} \left(z _ {t} \mid M, R\right) \| p _ {\theta_ {z}} \left(z _ {t}\right)\right)\right) \tag {7} \\ - \sum_ {t = 1} ^ {l _ {r}} D _ {\mathrm {K L}} \left(q _ {\phi_ {m}} (m _ {t} | R) \| p _ {\theta_ {m}} (m _ {t})\right), \\ \end{array}
$$
where  $p_{\theta_z}(z_t)$  and  $p_{\theta_m}(m_t)$  stand for  $p_{\theta_z}(z_t|r_{< t},z_{< t},m_{t - 1})$  and  $p_{\theta_m}(m_t|r_{\leq t},z_t)$  respectively, and  $D_{\mathrm{KL}}(\cdot \| \cdot))$  refers to Kullback-Leibler divergence. Detailed derivations are presented in the appendix.
Based on the intuition that the response provides hints about the segmentation, we construct the posterior distribution  $q_{\phi_m}(m_t|R)$  as a Bernoulli distribution parameterized by  $\sigma (f_{m - \mathrm{mlp}}^{\prime}(\psi_t))$  .  $\psi_t$  is a feature extracted from a bi-directional LSTM  $\psi (R)$  . Since the module indicator is kept unchanged within a segment, the posterior distribution  $q_{\phi_z}(z_t|M,R)$  is conditioned on the boundary indicator  $m_{t - 1}$  and defined as:
$$
\begin{array}{l} q _ {\phi_ {z}} \left(z _ {t} | M, R\right) = m _ {t - 1} \cdot \tilde {q} \left(z _ {t} | \psi_ {t}\right) \\ + \left(1 - m _ {t - 1}\right) \cdot \delta \left(z _ {t} = z _ {t - 1}\right), \tag {8} \\ \end{array}
$$
where  $\delta (\cdot)$  denotes Dirac delta function and the transition function  $\tilde{q} (z_t|\psi_t)$  is implemented as a multinomial distribution parameterized by Softmax  $(f_{z - \mathrm{mlp}}^{\prime}(\psi_t))$ . Once we have the posterior distribution, we apply Gumbel-Softmax (Jang et al., 2016) to take samples of  $m_{t}$  and  $z_{t}$ .
Weak Supervision on M and Z. We first use StanfordNLP toolkit (Manning et al., 2014) to parse every response in the training set as a sequence of segments, and use  $\tilde{M} = \{\tilde{m}_t\}_{t=1}^{l_r}$  to denote the results of segmentation labeling. The pseudo label of module choice  $\tilde{Z} = \{z_t\}_{t=1}^{l_r}$  is tagged in a similar way to multiclass classification, determined by (1) the similarity between each segment and knowledge and (2) the classification confidence of the style discriminator. More details about the construction of  $\tilde{Z}$  and  $\tilde{M}$  are provided in the appendix.
With  $\tilde{Z}$  and  $\tilde{M}$ , the loss function of weak supervision is defined as:
$$
\begin{array}{l} \mathcal {L} _ {m} = - \sum_ {t = 1} ^ {l _ {r}} \log p _ {\theta_ {m}} (\tilde {m} _ {t} | r _ {\le t}, \tilde {z} _ {t}), \\ \mathcal {L} _ {z} = - \sum_ {t = 1} ^ {l _ {r}} \tilde {m} _ {t - 1} \cdot \log p _ {\theta_ {z}} \left(\tilde {z} _ {t} \mid r _ {<   t}, \tilde {z} _ {<   t}, \tilde {m} _ {t - 1}\right). \tag {9} \\ \end{array}
$$
The learning algorithm is summarized and provided in the appendix.
We test our model on benchmarks of knowledge-grounded dialogue generation, including Wizard of
2The proposed method is also able to control the content style of knowledge-irrelevant segmentation by introducing extra adapters. But we focus on knowledge expression in this work.
3We always have  $m_0 = 1$