SlowGuess/NAACL_2022_OCR huggingface dataset:
Mao Yan Chen* and Siheng Li* and Yujiu Yang†  
Shenzhen International Graduate School, Tsinghua University  
{chenmaoy19, lisiheng21}  $@$  mails.tsinghua.edu.cn  
yang.yujiu@sz.tsinghua.edu.cn
In empathetic conversations, humans express their empathy to others with empathetic intents. However, most existing empathetic conversational methods suffer from a lack of empathetic intents, which leads to monotonous empathy. To address the bias of the empathetic intents distribution between empathetic dialogue models and humans, we propose a novel model to generate empathetic responses with human-consistent empathetic intents, EmpHi for short. Precisely, EmpHi learns the distribution of potential empathetic intents with a discrete latent variable, then combines both implicit and explicit intent representation to generate responses with various empathetic intents. Experiments show that EmpHi outperforms state-of-the-art models in terms of empathy, relevance, and diversity on both automatic and human evaluation. Moreover, the case studies demonstrate the high interpretability and outstanding performance of our model. Our code are available at https://github.com/mattc95/EmpHi.
Empathy is a basic yet essential human ability in our daily life. It is a capacity to show one's caring and understanding to others. Many types of research have been conducted on empathetic expression to enhance the empathy ability of Artificial Intelligence, e.g., computational approach for empathy measurement (Sharma et al., 2020), empathetic expression understanding in newswire (Buechel et al., 2018), online mental health support (Sharma et al., 2021), etc. In this work, we focus on the task of generating empathetic responses (Rashkin et al., 2019; Lin et al., 2019; Majumder et al., 2020) in open-domain conversation.
Existing empathetic dialogue models pay more attention to the emotion-dependent response generation (Lin et al., 2019; Majumder et al., 2020).
Figure 1: EmpHi generates empathetic responses with human-like empathetic intents (text in blue box), while existing empathetic dialogue models generate responses with contextually irrelevant and monotonous empathy (text in orange box).
However, using emotion alone to generate responses is coarse-grained, and the model needs to incorporate empathetic intent information. On the one hand, the speaker often talks with a particular emotion while the listener shows their empathy with specific empathetic intents, e.g., Acknowledging, Agreeing, Consoling and Questioning etc (Welivita and Pu, 2020). On the other hand, see in Figure 1, when the user expresses sadness, existing models tend to generate sympathetic responses like "I'm sorry to hear that." However, empathy is not the same as sympathy, so the models should not only generate responses with Sympathizing intent. We demonstrate this phenomenon elaborately with a quantitative evaluation in Section 2. In real life situation, humans could reply with various empathetic intents to the same context which depends on personal preference. For example, given a context, "I just failed my exam", an individual may respond "Oh no, what happened?" with Questioning intent to explore the experience of the user, or "I understand this feeling, know how you feel" with
* Equal contribution. † Corresponding author(yang.yujiu@sz.tsinghua.edu.cn).
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1063 - 1074 July 10-15, 2022 ©2022 Association for Computational Linguistics
Agreeing intent. These types of empathy are more relevant, interactive, and diverse.
To address the above issues, we propose a new framework to generate empathetic responses with human-like empathetic intents (EmpHi) which could generate responses with various empathetic intents, see examples in Figure 1. Specifically, EmpHi learns the empathetic intent distribution with a discrete latent variable and adopts intent representation learning in the training stage. During the generation process, EmpHi first predicts a potential empathetic intent and then combines both implicit and explicit intent representation to generate a response corresponding to the predicted intent. Our main contributions are:
- We discover and quantify the severe bias of empathetic intents between existing empathetic dialogue models and humans. This finding will lead subsequent researchers to pay more attention to fine-grained empathetic intents.
- To address the above problem, we propose EmpHi, which generates responses with human-like empathetic intents. Experiments have proved the effectiveness of our model through the significant improvement in both automatic and human evaluation.
- According to the quantitative evaluation and analysis, EmpHi successfully captures humans' empathetic intent distribution, and shows high interpretability in generation process.
Empathetic Response Generation. Providing dialogue agents the ability to recognize speaker feelings and reply according to the context is challenging and meaningful. Rashkin et al. (2019) propose the EmpatheticDialogues for empathetic response generation research. Most subsequent empathetic conversation researches are evaluated on this dataset, including ours. They also propose Multitask-Transformer, which is jointly trained with context emotion classification and response generation. To further capture the fine-grained emotion information, Lin et al. (2019) introduce MoEL, a transformer with a multi-decoder. Each of them is responsible for the response generation of one specific emotion. Majumder et al. (2020) propose
Figure 2: Empathetic intent distribution of human in empathetic conversation.
MIME, which mimics the speaker emotion to a varying degree.
All these models focus on emotion-dependent empathetic response generation, whereas we pay more attention to the empathetic intents and propose to generate a response that is not only emotionally appropriate but also empathetically human-like.
One-to-many Response Generation. Given dialogue history, there could be various responses depends on personal preference. Zhao et al. (2017) propose to learn the potential responses with continuous latent variable and maximize the log-likelihood using Stochastic Gradient Variational Bayes (SGVB) (Kingma and Welling, 2014). To further improve the interpretability of response generation, Zhao et al. (2018) propose to capture potential sentence-level representations with discrete latent variables. MIME (Majumder et al., 2020) introduces stochasticity into the emotion mixture for various empathetic responses generation.
Different from the previous works, we propose a discrete latent variable to control the empathetic intent of response and achieve intent-level diversity.
3 Empathetic Expression Bias
Although existing empathetic conversational methods have shown promising progress, we reveal there is a severe bias of empathetic expression between them and humans according to quantitative evaluation.
Empathy plays a vital role in human conversation, Welivita and Pu (2020) make a taxonomy of empathetic intents and calculate the frequency of each intent in EmpatheticDialogues (Rashkin et al., 2019). As shown in Figure 2, humans show
Figure 3: The architecture of EmpHi, which consists of a context encoder, an emotion classifier, a prior network (intent predictor), a recognition network, and a response decoder with copy mechanism.
Figure 4: Empathetic intent distribution of human and MIME (sad emotion), the intent index represents the same intent as in Figure 2.
their empathy naturally by Questioning, Acknowledging, and Agreeing intents etc.
However, there are no empirical experiments have shown how empathetic dialogue models express their empathy? To further study, we finetune a BERT classifier (Devlin et al., 2019) on the released EmpatheticIntents $^{1}$  dataset (Welivita and Pu, 2020). Our classifier achieves  $87.75\%$  accuracy in intent classification and we apply it to identify the empathetic intents of responses generated by the state-of-the-art empathetic dialogue model MIME (Majumder et al., 2020). As shown in Figure 4, the severe empathetic intent distribution bias emerges while comparing humans to MIME. Given context with sad emotion, existing models usually generate "I am sorry to hear that" with Sympathiz-
ing intent, which is not human-like and contextually relevant. In addition, we can tell that the empathetic expression of MIME is monotonous. We also quantify the intent distribution of other empathetic dialogue models in the Appendix A. The results are similar to Figure 4.
We believe this phenomenon is caused by that existing models only generate responses according to context emotion and lack fine-grained empathetic intent modeling. Therefore, we propose EmpHi, which generates empathetic responses with humanlike empathetic intents.
4 EmpHi Method
4.1 Task Definition and Overview
Given the context,  $C = [c_{1}, c_{2}, \dots, c_{m}]$ , which consists of  $m$  words for single or multiple utterances. We aim to generate empathetic response,  $X = [x_{1}, x_{2}, \dots, x_{n}]$ , with human-like empathetic intent. The whole model architecture is shown in Figure 3.
EmpHi learns the potential empathetic intent distribution with a latent variable  $z$ , which could be seen in Figure 5. Conditional Variational AutoEncoder (CVAE) (Yan et al., 2016; Zhao et al., 2017; Gu et al., 2019) is trained to maximize the conditional log likelihood,  $\log p(X|C)$ , which involves an intractable marginalization over  $z$ . We train the CVAE efficiently with Stochastic Gradient Variational Bayes (SGVB) (Kingma and Welling, 2014) by maximizing the variational lower bound of the log likelihood:
$$
\log p (X | C) \geq \mathbf {E} _ {q (z | X, C)} [ \log p (X | C, z) ] \tag {1}
$$
<https://github.com/anuradhal992/ EmpatheticIntent>
(a)
Figure 5: An illustration of the difference between existing empathetic dialogue models (a) and EmpHi (b).
$p(X|C,z)$  denotes response reconstruction probability,  $q(z|X,C)$  is recognition probability and  $p(z|C)$  is prior probability. Our method mainly consists of three aspects:
- To capture the explicit relationship between the latent variable and the intent, we propose an intent representation learning approach to learn the intent embeddings.
- We construct an intent predictor to predict potential response intent using contextual information and then use this intent for guiding the response generation.
- During the generation process, EmpHi combines both implicit intent embedding and explicit intent keywords to generate responses corresponding to the given intents.
4.2 Learning Intent Representation
To achieve more interpretability, we choose a discrete latent variable that obeys categorical distribution with nine categories, each corresponding to one empathetic intent. Directly maximizing Eq.1 would cause two serious problems: the relation between the latent variable and intent is intractable; the vanishing latent problem results in insufficient information provided by the latent variable during generation. (Bowman et al., 2016; Zhao et al., 2017; Gu et al., 2019).
To solve the above issues, we separately train a recognition network  $q_{r}(z|X)$  to encourage intent variable  $z$  to capture context-independent semantics, which is essential for  $z$  to be interpretable (Zhao et al., 2018). The task of the recognition network is to provide the accurate intent label of the
response, which corresponds to an intent embedding. Then, by maximizing likelihood  $p(X|C,z)$ , the embedding captures corresponding intent representation automatically. The recognition network  $q_{r}(z|X)$  does not need additional training. We utilize the BERT intent classifier mentioned above, which achieves  $87.75\%$  accuracy in intent classification. In addition, as the sample operation easily brings noise for the intent representation learning when sampling a wrong intent, we use argmax operation to avoid the noise, the response reconstruction loss is:
$$
\mathcal {L} _ {1} = - \log p (X | C, z _ {k}) \tag {2}
$$
$$
z _ {k} = \underset {z _ {k}} {\arg \max } q _ {r} \left(z _ {k} | X\right) \tag {3}
$$
$k\in \{0,1,2,\dots ,8\}$  , each integer corresponds to a specific empathetic intent as in Figure 2.
4.3 Intent Predictor and Emotion Classifier
The intent predictor is based on the prior network  $p_i(z|C)$ , which predicts the distribution of response intent by the given context. During inference, we sample potential intents from this distribution in order to generate human-like empathetic responses. Specifically, the context is encoded with gated recurrent units (GRU) (Chung et al., 2014):
$$
h _ {t} = \mathbf {G R U} \left(h _ {t - 1}, E \left(c _ {t}\right)\right), \tag {4}
$$
where  $h_t$  is the hidden state of GRU encoder,  $E(c_t)$  denotes the word embedding of the  $t$ -th word in context, we use  $h_m$  as context embedding, then the prior network is:
$$
p _ {i} (z \mid C) = \operatorname {S o f t m a x} \left(\mathbf {F F N} _ {\mathbf {z}} \left(h _ {m}\right)\right), \tag {5}
$$
where FFN represents Feed-Forward Network with two layers. The prior intent distribution is supervised by recognition distribution with KL-divergence in Eq.1:
$$
\begin{array}{l} \mathcal {L} _ {2} = \mathbf {K L} \left(q _ {r} (z | X) | | p _ {i} (z | C)\right) \\ = \sum_ {k = 1} ^ {K} q _ {r} \left(z _ {k} \mid X\right) \log \frac {q _ {r} \left(z _ {k} \mid X\right)}{p _ {i} \left(z _ {k} \mid C\right)}. \tag {6} \\ \end{array}
$$
Since the context emotion is proved to be beneficial to empathetic dialogue generation (Rashkin et al., 2019; Lin et al., 2019; Majumder et al., 2020), we also employ an emotion classifier to classify the emotion of context:
$$
\begin{array}{l} \mathcal {P} = \operatorname {S o f t m a x} _ {\mathbf {e} ^ {[ 1 ]}} \left(\mathbf {F F N} _ {\mathbf {e}} \left(h _ {m}\right)\right), \tag {7} \\ p _ {e _ {i}} = \mathcal {P} [ i ] \\ \end{array}
$$
Given the ground truth emotion label  $\mathbf{e}_{\mathbf{t}}$ , the emotion classifier is trained with cross-entropy loss:
$$
\mathcal {L} _ {3} = - \log p _ {\mathbf {e} _ {\mathrm {t}}}. \tag {8}
$$
4.4 Response Generator
As for the response generation  $p(X|C,z)$ , we consider implicit intent embedding for the high-level abstraction of an intent. In addition, we also introduce intent keywords for explicitly utilizing intent knowledge during the generation process.
Implicit. To generate response with an empathetic intent, the most intuitive approach is taking the intent embedding as additional input to decoder during the generation process. We also consider emotion embedding as traditional empathetic dialogue models:
$$
s _ {t} = \mathbf {G R U} \left(s _ {t - 1}, \left[ E \left(x _ {t - 1}\right); v (z); v (e); c _ {a t t} \right]\right), \tag {9}
$$
where  $s_t$  is the state of GRU decoder,  $c_{att}$  denotes the context attention value which contains key information of context (Bahdanau et al., 2015).  $v(z)$  is intent embedding and  $v(e)$  is emotion embedding, both will not change during the generation process. However, this may sacrifice grammatical correctness (Zhou et al., 2018; Ghosh et al., 2017). Therefore we add a gate operation to capture intent and emotion dynamically:
$$
\operatorname {I n p u t} = \mathbf {F F N} _ {\mathbf {i}} ([ E (x _ {t}); c _ {a t t}; s _ {t} ]),
$$
$$
\text {G a t e} = \text {S i g m o i d (I n p u t)}, \tag {10}
$$
$$
\bar {v} (z) = \operatorname {G a t e} \odot v (z),
$$
where  $\odot$  represents element-wise product. Each time step, the intent representation is used appropriately according to current word, state, and context value. The gate operation for emotion is the same as above.
Explicit. The empathetic expression is quite distinct over vocabularies, e.g., 'know', 'understand', 'agree', are indicative of the empathetic intent Agreeing. Therefore, we employ the copy mechanism to explicitly utilize intent keywords for intent conditional generation. See in Appendix B for more details about intent keywords.
$$
\alpha_ {t} = \operatorname {S i g m o i d} \left(v _ {s} ^ {\top} s _ {t}\right),
$$
$$
p (x _ {t} = w _ {g}) = \operatorname {S o f t m a x} (W _ {g} s _ {t}),
$$
$$
p \left(x _ {t} = w _ {i}\right) = \operatorname {S o f t m a x} \left(W _ {i} s _ {t}\right),
$$
$$
p \left(x _ {t}\right) = \left(1 - \alpha_ {t}\right) \cdot p \left(w _ {g}\right) + \alpha_ {t} \cdot p \left(w _ {i}\right), \tag {11}
$$
where  $\{s_t,v_s\} \in \mathcal{R}^{d\times 1},\{W_g,W_i\} \in \mathcal{R}^{V\times d},d$  is hidden size and  $V$  denotes the vocabulary size. The copy rate  $\alpha_{t}$  is used to balance the choice between intent keywords and generic words, it is trained with binary cross entropy loss:
$$
\mathcal {L} _ {4} = \sum_ {t = 1} ^ {n} q _ {t} \cdot \log \alpha_ {t} + (1 - q _ {t}) \cdot \log (1 - \alpha_ {t}), \tag {12}
$$
$n$  is the amount of words in response,  $q_{t}\in \{0,1\}$  indicates that whether  $x_{t}$  is a intent keyword.
4.5 Loss Function
To summarize, the total loss is:
$$
\mathcal {L} = \lambda_ {1} \mathcal {L} _ {1} + \lambda_ {2} \mathcal {L} _ {2} + \lambda_ {3} \mathcal {L} _ {3} + \lambda_ {4} \mathcal {L} _ {4}, \tag {13}
$$
In order to join all losses with weighting method, we add 4 hyperparameters in total loss,  $\lambda_{i}$ , where each  $\lambda_{i}$  is corresponding to  $L_{i}$ .  $\mathcal{L}_1,\mathcal{L}_2,\mathcal{L}_3,\mathcal{L}_4$  denote the losses of response reconstruction, intent prediction, emotion classification and copy rate prediction respectively.
We evaluate our method and compare with others on EmpatheticDialogues² (Rashkin et al., 2019) which contains  $25k$  open domain dialogues. Follow the same setting as the authors of this dataset, the proportion of train/validation/test data is  $8:1:1$ . Each dialogue consists of at least two utterances between a speaker and listener. There are 32 emotion situations in total, which are uniformly distributed.
We compare our model with the three latest empathetic conversational models:
- Multitask Transformer (Multi-TRS). A transformer model trained by the response generation task and the context emotion classification task (Rashkin et al., 2019).
- Mixture of Empathetic Listeners (MoEL). An enhanced transformer model with 32 emotion-specific decoders to respond appropriately for each emotion (Lin et al., 2019).
$^{2}$ https://github.com/facebookresearch/patheticDialogues
- MIMicking Emotions for Empathetic Response Generation (MIME). The state-of-the-art empathetic dialogue model allows the generator to mimic the context emotion to a varying degree based on its positivity, negativity, and content. Furthermore, they introduce stochasticity into the emotion mixture and achieves one-to-many generation (Majumder et al., 2020).
5.3.1 Automatic Metrics
- BLEU. We choose BLEU (Papineni et al., 2002) for relevance evaluation which measures the  $n$ -gram overlaps with reference and compute BLEU scores for  $n \leq 4$  using smoothing techniques (Chen and Cherry, 2014). Since the state-of-art model MIME and ours are both one-to-many generators, we calculate BLEU recall and BLEU precision (Zhao et al., 2017; Gu et al., 2019). For each test case, we sample 5 responses from latent space and use greedy search for MIME and EmpHi, use beam search for MoEL and Multitask-Transformer.
- Distinct. Distinct (Li et al., 2016) is a widely used metric for diversity evaluation. Specifically, we compute the number of distinct unigrams (Distinct-1) and bigrams (Distinct-2), then scale them by the total number of unigrams and bigrams.
5.3.2 Human Ratings
First, we randomly sample 100 dialogues and their corresponding generations from the three baseline models and EmpHi. Then, we invite five volunteers with master degrees to do the human evaluation. The annotators mark each response from 1 to 5 for empathy, relevance, and fluency.
To clarify the marking criteria, we provide an explanation for each metric:
- Empathy. Whether the response shows that the listener understands and shares the speaker's feeling. Can the listener imagine what it would be like in the speaker's situation?
- Relevance. Whether the response is relevant to the context.
- Fluency. Whether the response is easy to read and grammatically correct.
5.3.3 Human A/B Test
Following (Lin et al., 2019; Majumder et al., 2020), we construct this evaluation task to directly compare our model with each baseline. We randomly sample 100 dialogue responses from EmpHi vs {Multitask-Trans, MoEL, MIME}. Given randomly ordered responses from above models, four annotators select the better response, or tie if they think the two responses have the same quality. The average score of four results is calculated, and shown in Table 6.
5.4 Implement Detail
For  $\mathrm{MIME}^3$  (Majumder et al., 2020) and  $\mathrm{MoEL}^4$  (Lin et al., 2019), we reproduce their results using their open-source codes and their default hyperparameters. According to the log-likelihood in the validation dataset for Multitask-Transformer, we use grid search for the best head number, layer number, and feed-forward neural network size. The best set is 2, 10, and 256, respectively. EmpHi uses a two-layer Bi-GRU as the encoder and a two-layer GRU as the decoder,  $\lambda$  is set as [1, 0.5, 0.5, 1] respectively. All the feed-forward neural networks in EmpHi have two layers, 300 hidden units and ReLU activations. For the sake of fairness, we use pretrained Glove vectors (Pennington et al., 2014) with 300 dimensions as the word embedding for all models, the batch size is 16, and the learning rate is set to  $1e^{-4}$ .
6.1 Results Analysis
In this section, we mainly testify:
- human-like empathetic intent boost EmpHi's performance in terms of empathy, relevance, and diversity.
- EmpHi successfully captures the empathetic intent distribution of humans.
6.1.1 Human Evaluation
As shown in Table 1, EmpHi outperforms all baselines in terms of empathy, relevance, and fluency. The most distinct improvement is  $15.1\%$  on relevance because our model does not only depend on the speakers' emotion, but also makes use of the empathetic intents, which are contextually relevant. It is worth noting that empathy is the primary
<sup>3</sup>https://github.com/declare-lab/MIME
<sup>4</sup>https://github.com/HLTCHKUST/MoEL
Methods	#Params.	Empathy	Relevance	Fluency	BLEU	Distinct
P	R	F1	D-1	D-2
Multitask-Trans	20M	2.68	2.58	3.60	0.3072	0.4137	0.3526	0.4123	1.1390
MoEL	21M	3.18	3.18	3.95	0.3032	0.3614	0.3298	0.8473	4.4698
MIME	18M	2.89	2.90	3.77	0.3202	0.3278	0.3240	0.3952	1.3299
EmpHi	11M	3.48	3.66	4.34	0.3207	0.4723	0.3820	1.1188	5.3332
Human	-	4.04	4.40	4.56	-	-	-	7.0356	43.2174
Table 1: Automatic evaluation between EmpHi and other models. All results are the mean of 5 runs for fair comparison. D-1.&2. are magnified 100 times for each model.
Methods	Win	Loss	Tie
EmpHi vs Multitask-trans	56.5%	21.5%	22.0%
EmpHi vs MoEL	45.0%	28.5%	26.5%
EmpHi vs MIME	53.0%	27.0%	20.0%
Figure 6: Empathetic intent distribution of human and EmpHi (sad emotion), the intent index represents the same intent as in Figure 2.
metric in empathetic dialogue generation. EmpHi outperforms the previous SOTA on empathy by  $9.43\%$ , which directly indicates that human-like empathetic intents are beneficial to the empathy ability of the dialogue model.
Last but not least, a decent fluency score proves that our generated response could be understood by humans easily, where our model has an improvement of  $9.87\%$  from MoEL. In addition, the human A/B test results in Table 2 also confirm that the responses from our model are preferable to baselines. Overall, EmpHi successfully generates empathetic, relevant, and fluent responses.
6.1.2 Automatic Evaluation
As seen in Table 1, the automatic evaluation is consistent with human evaluation. The BLEU recall and F1 score are improved by  $14.2\%$  and  $8.34\%$ , respectively. However, we only have a slight im
Table 2: Results of Human A/B test.
Methods	BLEU	ACC
P	R	F1
EmpHi	0.3207	0.4723	0.3820	26.8%
EmpHi w/o intent	0.3105	0.4049	0.3515	21.9%
EmpHi w/o gate	0.3138	0.4634	0.3742	25.3%
EmpHi w/o copy	0.3215	0.4704	0.3820	25.9%
Table 3: Results of ablation study.