Empathy Intent Drives Empathy Detection
Authors: Liting Jiang
Di Wu
Bohui Mao
Yanbing Li
Wushour Slamu
Venue: EMNLP 2023 Main (2023)
Keywords: Empathy detection
Empathy intent recognition
Cascaded interactive attention
Label signal enhancement

Abstract: Empathy plays an important role in the human dialogue. Detecting the empathetic direction expressed by the user is necessary for empathetic dialogue systems because it is highly relevant to understanding the user's needs. Several studies have shown that empathy intent information improves the ability to response capacity of empathetic dialogue. However, the interaction between empathy detection and empathy intent recognition has not been explored. To this end, we invite 3 experts to manually annotate the healthy empathy detection datasets IEMPATHIZE and TwittEmp with 8 empathy intent labels, and perform joint training for the two tasks. Empirical study has shown that the introduction of empathy intent recognition task can improve the accuracy of empathy detection task, and we analyze possible reasons for this improvement. To make joint training of the two tasks more challenging, we propose a novel framework, Cascaded Label Signal Network, which uses the cascaded interactive attention module and the label signal enhancement module to capture feature exchange information between empathy and empathy intent representations. Experimental results show that our framework outperforms all baselines under both settings on the two datasets.

Empathy Intent Drives Empathy Detection
Liting Jiang 123 Di Wu Bohui Mao 1.2.3 Yanbing Li Wushour Slamu 1,2,3 'School of Computer Science and Technology. Xinjiang University, Urumqi, China "Resource Monitoring and Research Center on Minority Languages, Urumqi, China 3 "Xinjiang Laboratory of Multi-language Information Technology, Urumqi, China (107556521210, wd11, 107552204050 xju.edu.cc edu. cn (liyb, mushour)exjuu edu.
Abstract
(a)
Empathy Representation
Empathy Decoder
Enroder
Empathy plays an important role in the human dialogue. Detecting the empathetic direction expressed by the user is necessary for empa- thetic dialogue systems because it highly rel- evant to understanding the user's needs. Sev- eral studies have shown that empathy intent information improves the ability to response capacity of empathetic dialogue. However, the interaction between empathy detection and em- pathy intent recognition has been explored. To this end, we invite 3 experts to manually an- notate the healthy empathy detection datasets IEMPATHIZE and TwittEmp with 8 empathy intent labels, and perform joint training for the two tasks. Empirical study has shown that introduction of empathy intent recognition task can improve the accuracy of empathy detec- task, we analyze possible reasons this improvement. To make joint training of the two tasks more challenging, we propose novel framework, Cascaded Label Signal Net- work, which uses the cascaded interactive atten- tion module and the label signal enhancement module capture feature exchange informa- tion between empathy and empathy intent rep- resentations. Experimental results show that framework outperforms all baselines under both settings on the two datasets.
(b)
Empathy Decoder
Encoder
M Miked
Representation
Empathy
(c)
Empathy Representation
Empathy Empathy Intent Decoder
E
Encoder
Interaction
Empathy Intent Representatin
Figure Different modeling approaches.
Jurgens, 2020), empathetic expression understand-
ing in newswire (Buechel al., 2018), medical and healthcare (Khanpour et al., 2017; Hosscini and Caragea, 202 Chen et al., 2020; Wijaya et al., 2023), human-computer interaction (Virvou and Katsionis, 2004; Xie and Pu, 2021; Gao et al., 2021; Samad et al., 2022), etc.
Currently, millions of people seek psychologi- cal support by expressing their emotions in online health communities and look forward to receiving support from peers who may have had similar ex- periences and can understand their feelings. There- fore, some researchers have performed studies on the direction of empathy expression in people's 's ut- terances. For example, the utterance: "I lost my mom to cancer in April and just miss her so much. There are so many pieces to work on and I find it so hard to work on be my grief is strong". The model needs to detect the direction of empathy pressed by the user, 'Seek'. Hosseini and Caragea (2021a) provided online healthy dataset and baseline for empathy detection (ED), but their meth- ods are monotonous, as shown in Figure (a). They did not consider that humans have potential empa-
1 Introduction
Empathy is essential in human social interaction. In the process of human dialogue, empathy en- ables listeners to establish rapport with speakers by understanding their emotional and cognitive states, arousing their interest, and comforting them (Kim et al., 2022). Therefore, it is worthwhile to detect the empathetic direction of dialogue utter- ances. In recent years, researchers have studied empathy detection in various fields, such men- tal health support (Sharma et al., 2020; Zhou and
*Conresponding author. 'https: h/gpt//b///ttub.... hwwptthuu..comJ/aaaggiei Jinn/iiagT77/CSS

---

thy intent information while expressing empathy, Chen et al. (2022) learned the distribution of poten- tial empathy intent and then combined implicit and explicit representations of empathy intent to gener- ate responses with empathy intent. However, they do not explore the interaction between empathy and empathy intent representations.
2 Related Work
2.1 Empathy Detection
Empathy detection, one of a series of empathy tasks, has been widely studied by many researchers (Hosseini and Caragea, 2021a; Chen et al., 2020). Currently, the task involves two types of research: analyzing empathy from text (Yang et al., 2019;
In this paper, we invite 3 experts to manually an- notate the empathy intent of each utterance in two empathy detection datasets IEMPATHIZE (Hos- seini and Caragea, 2021b) and TwittEmp (Hosseini and Caragea, 2021a) according to the lexicon of empathy intent example utterance provided by We- livita and Pu (2020). On this basis, we use a simple joint training method to test the feasibility of em- pathy intent recognition (EIC) as an auxiliary task, as shown in Figure (b). The experimental results show that the joint training of ED and EIC leads higher accuracy of ED, and we find that the cat- egories of ED are easier distinguish after the introduction of EIC due to the obvious correspon- dence between empathy and empathy intent labels. To make joint training tasks more challenging, we propose a novel framework, called Cascaded Label Signal Network (CLSN). First, it uses the BERT (Kenton and Toutanova, 2019) model to obtain the semantic features of utterances and extracts differ- ent representations through two linear layers. Then, the cascaded interactive attention module is used to implement feature interaction and control knowl- edge flow. Finally, the label signal enhancement module is used to further extract interactive fea- tures from the label information of the two tasks and feed them to different decoders to complete ED and EIC, We model the interaction information flow between both tasks, as shown in Figure 1(c).
Buechel et al., 2018; Sedoe et al., 2020; Ghosh et al., 2022), and from spoken dialogues (Fung et al., 2016; Kim et al., 2022: Alam et al., 2018; Pérez-Rosas et al., 2017: Ayshabi and Idicula, 2021). Empathy plays an important role in online health communities (Medeiros and Bosse, 2016) as can facilitate the healing process by reduc- ing psychological distress and increasing optimism through empathetic dialogue (Yalcin and DiPaola, 2018; Williams and Cano, 2005). Sharma et al. (2020) used RoBERTa- bi-encoder model to identify empathy in conversations on online mental health platforms. Khanpour et al. (2017) proposed model based on Convolutional Networks and Long Short-Term Memory to identify empathetic messages in online health communities. Alam et al. (2018) proposed an Italian spoken empathetic di- alogue system consisting only of paired conver- sations between patient and therapist in different audio. Bi et al. (2023) defined an empathy planner to capture and reason about multi-source informa- tion that considers cognition and affection. They also introduced a dynamic integrator module that allows the model dynamically select the appropri- ate information to generate empathetic responses.
2.2 Empathy Intent Recognition
Some empathetic dialogue generation studies in- corporate empathy intent information, such as, We- livita and Pu (2020) have manually labeled 500 re- sponse intents. Using lexical and machine learning methods, they automatically analyzed utterances of the entire dataset with identified response intents and 32 emotion categories, and the information vi- sualization method is used to summarize the emo- tional dialogue exchange model and its temporal evolution. Welivita et al. (2021) curated a novel large-scale silver dialogue dataset, EDOS (Emo- tional Dialogues in OpenSubtitles). It contains million movie subtitles for emotional dialogue, with 32 fine-grained emotions, 8 intent categories. and a neutral category. Saha and Ananiadou (2022) proposed a fusion model of the Transformer model and the Hierarchical Encoder Decoder, called the
Overall, our contributions are as follows: (1) We invite 3 experts to manually annotate empathy in- tent labels on the ED datasets IEMPATHIZE and TwittEmpp provide the necessary datasets for the joint ED and EIC study; (2) To the best of our knowledge, we are the first to attempt joint training of the ED and EIC tasks to improve the accuracy of the ED task. We also explore the possible reasons for the accuracy improvement; (3) To make joint training of the two tasks more challenging, we pro- pose a novel framework, CLSN, which explicitly controls the knowledge transfer between the two tasks. Experimental results show that our frame- work outperforms all baselines under both settings in the two datasets.

---

Hierarchical Transformer Network, capture the
(b)
Negatius 23%
Acknowledging
Acknoweddiig 275
speaker's emotion and dialogue context. To gen- erate intent controlled empathetic responses, they used the results of Reinforcement Learning to im- plicitly optimize rewards.
Posiivve 11%
Negative 32%
Console 9%
9%
22%
Neuttal 40%
Wishing
Postive 7%
Quetioning
3 Empirical Study
Sympathicing
Questoonngg
5%
Sympathizing
18%
this section, we introduce EIC as an auxiliary task in the ED task to verify its effectiveness. In addition, we explore the reasons for the influence of the EIC task on ED.
Figure 2: Distribution of empathy intent labels in the IEMPATHIZE dataset (a) and TwittEmp dataset (b).
seek-classifier. The multi-class setting considers three classes, In experiments, we split the dataset, keeping 60% of the data for the training set, 20% for the validation set, and 20% for the test set. See Appendix A for detailed statistics.
3.1 Problem Definition
Both ED and EIC can be regarded as classification tasks. Given an utterance U = (u1, "2, Ug). the ED task detects the direction of empathy based on the U, and the EIC task identifies the potential empathy intent of the user based on the U.
3.3 Empirical Study Results
Following the work of (Hosseini and Caragea, 2021b), BERT is used as a shared encoder to train the two tasks together and decode the hidden states separately to get the results of both tasks. We pre- experiment with IEMPATHIZE under both settings, Table shows significant improvement in ED accuracy when the EIC task is introduced. The FI- score of the ED improved by 1.74%, 2.79%, and 5.61 % under the both settings, demonstrating that implementing joint learning between EIC and ED can improve the performance of the ED task.
3.2 Dataset and Annotation
IEMPATHIZE? containing sentences from online cancer survivors. It contains 5007 sentences, empathy labels: 'Seek', 'provide' and 'None'. In total, 1046 sentences are annotated as 'Seek', 966 as 'provide', and 2995 as "None". To introduce the EIC task, we annotate the intent labels on the empathy detection dataset. Inspired by Welivita and Pu (2020); Chen et al. (2022), we identify 7 empathy intent labels: 'Acknowledging', 'Con- soling', "Questioning'. 'Sympathizing', 'Wishing'. 'Positive', 'Negative' and 'Neutral' (all other un- mentioned intents) by observing the IEMPATHIZE dataset. We recruit experts to assign an empathy intent label to each utterance in the IEMPATHIZE dataset in combination with the lexicon of intent la- bels example utterance provided by Welivita and Pu (2020). The first round of annotation is completed by 2 experts, and Cohen's 's kappa coefficient is 88.4% for the empathy intent of the IEMPATHIZE dataset. For samples with inconsistent annotation, the 3rd expert decided which category the empathy intent belonged to. The distribution of the dataset is shown in Figure 2 (a).
Settings
P(%)
R(%)
Fi(%)
Seek +2.11
+8.50
+2.79
Provide
+3.36
+6.63
+5.61
Multi-class
+1.06
+3.98
+1.74
Table 1: Experimental results of the empirical study. Multi-class denotes multi-clas setting, seek and provide denote binary settings.
3.4 Analysis of Reasons for Improvement
The accuracy of the ED task has improved sub- stantially, and two questions are asked explore the reasons for the improvement. Question (I); After the introduction of the EIC task, why did the accuracy of the ED task improve significantly?
Following previous work (Hosseini and Caragea, la,b,, use both settings to create classifiers. The binary setting is used to identify utterances that seek empathy or provide empathy. For example, To create the clek-classifier, we set 'Seek" as positive samples, 'None' and 'Provide' as negative samples. The provide-classifiers is created in similar way as
Question (2): How does the introduction of the EIC task help to improve the accuracy of ED?
For Question (1): In terms of whether the introduc- tion of the EIC task improves the performance of the model, we performed visualization on the test set the IEMPATHIZE dataset (multi-class). The T-SNE ( (Van der Maaten and Hinton, 2008) results are shown in Figure 3.
Phttps: http://tttttttttt/e com/ htwp:/tuub.com/aaaah/eeeeeeee

---

(b)
there are possible empathy intent labels for an utterance and only possible empathy labels, it harder to accurately identify the empathy in- tent label of an utterance than it is to identify the empathy label. Once the model correctly pre- diets the empathy intent label of the utterance, the model is able to more accurately identify the empathy label of the utterance based on the apparent correspondence of the two labels co- occurrence frequency in Table 2. Figure 4 shows some cases, and we can see that the simultaneous occurrence the two labels is consistent with the co-occurrence frequency in Table 2. The empiri- cal study is able to correctly predict the empathy labels of the three utterances, demonstrating that the introduction of the EIC task can help the model to improve the performance of ED tasks.
None
Seek
Provide
Figure 3: T-SNE visualization of the label representa- tions on the IEMPATHIZE dataset.
The T-SNE results for the introduced EIC task are shown in Figure 3 (b). The categories 'Provides" and 'Seek' are more discriminable. In addition, the three categories are more aggregated. This demonstrates the ability of the model to learn better representation of the hidden state after the introduction of the EIC task, which is one of the important reasons for the accuracy gains achieved.
(a) So he is basically stuck at home watching TV. Empirical Study: None, Neutral
BERT : Seek;
Sadly, it never filled in again and it has been years now.
BERT None;
Empirical Seek, Negative
J ust wanted send best wishes for bone scan too!
For Question (2): We count the frequency of co- occurrence of different labels in the IEMPATHIZE training dataset, as shown in Table 2.
BERT None;
Empirical S Study Provide, Wishing
Figure 4: Case study. The red color indicates the wrong label, the green color indicates the correct label
Labels
None
Seek
Provide
Achnowledging
141
50
We use the same data annotation approach to annotate TwitlEmp'. The Cohen's kappa coeffi- cient is 80.0%, The distribution of the dataset is shown in Figure 2 (b). TwittEmp was collected from Twitter cancer topics and there are a to- tal of 3000 sentences. The category labels are the same as IEMPATHIZE, where 1000 sentences are annotated as "Seek", 1000 as "Provide", and 1000 as "None".
Consoling
42
202
Neutral
1126
38
49
Quextioning
92
26
Sympathizing
39
Wishing
14
10
104
Positive
189
26
103
Negative
187
463
29
Table 2: Empathy and empathy intent labels occurrences frequency.
From Table 2 we can see that when the empathy intent label is 'Acknowledging', 'Neutral', 'Ques- tioning' 'Positive', the empathy label is more likely be 'None', the empathy intent label NNeg- ative', the empathy label is more likely to be 'Seek', the empathy intent label is 'Consoling'. 'Wishing". 'ympatthziig', or 'Positive", the empathy label is more likely to be 'Provide". For example: "I am sorry to hear about your mother-in-lane troubles after her diagnosis, I am sympathetic", the word 'sympatheticc' reflects the empathy intent of 'Sym- pathizing' and expresses the direction of empathy 'Provide'. The EIC channel shares the captured semanties with the ED channel which can help the ED channel to better perceive the empathy informa- tion contained in the utterance. Intuitively, since
Although the joint training method can improve the accuracy of the ED task, the modeling approach does not explicitly control the information flow between ED and EIC. To address this shorteoming and to make the joint training of the two tasks more challenging, we propose novel framework in section 4.
4 Methodology
In this section, we provide detailed description of the CLSN, which can effectively model the inter- action between empathy and empathy intent repre- sentations. It consists of encoding layer ($4.1), cascaded interactive attention module (54.2), la-
Phttps: httptttcccc////aaaa///eataaa

---

bel signal enhancement module (64.3), and sep- arate classification decoders ($4.4). An overview of our framework shown in Figure 5.
where ò E {I, denotes empathy intent and denotes empathy, respectively. FC denotes a fully connected layer, Hg and H, are empathy and empathy intent representations, respectively.
Empathy Decoder
Intent Decoder
4.2
Cascaded Interactive Attention Module
Label gnal Enhancement. Module
The approach of simply sharing the hidden state is not sufficient to achieve explicit information trans- fer between two tasks. Therefore, we design a cascade of interactive attention modules. It can explicitly model the interaction between ED and EIC tasks. By this method, interactive information continuously extracted features between tasks and finally returns to the original task to complete the cascade operation.
Label enhancement eature
Label Enhencement Feature
Kic
Signal
Interactvv Ameentonnn Module
Cascade Feature (Emppillyinnten-EEpaaly
Cascade Faature (*YerttEimmelhh-iineet
To make the two tasks learn features from each other, we use the attention mechanism (Vaswani et al., 2017) to connect the two tasks. First, we map the matrices HE and H, by linear projection to obtain the corresponding Q K, V, respectively. Then the attention mechanism is used to obtain the corresponding empathy representation HÆ-/ with empathy intent features and empathy intent representation H, with empathy features, re- spectively. At the same time, we use the empa- thy and empathy intent representations combined with H and HE- respectively to construct the attention layer obtain the cascaded interaction features H HE-I-E and HI-E-I.
Kic
Empathy-Innee Fealure
Intenti Empathy Feature
Q
Kt
Vt
Empathy Feature
Intent Festure
BERT
A
A
uz
Us
Figure 5: The overall architecture of the CLSN.
H = Attention (H,, H5, Hs),
(3)
4.1 Encoding Layer
H6-1-8 Attention (H& H Hy-s), (4)
In our framework, ED shares an encoder with EIC. Given an utterance U, we first mark [CLS]] and [SEP] at the beginning and end of the utterance and then use the pre-trained language model BERT beert-baseunnasedl to encode the semantic utter- ance capture contextual semantic information.
where and 8 E EJ 3 if denotes E, then 6 denotes I, and vice versa.
4.3 Label Signal Enhancement Module
To make the extracted features more effective, we constructed label signal enhancement module. It combines the empathy labels EM and the empathy intent labels In as label signals. Specifically, con- catenating all label encodings of the two tasks as the query vectors of the module with the empathy and empathy intent representations for the attention computation can effectively capture, the semantics of the two information with respect to labels. Tak- ing the empathy detection task example, the computation with label encoding can not only cap- ture the semantics about empathy, but also capture the semantics about empathy intent for assisting the decoding of empathy, and the same is true for the empathy intent recognition task. The label signal
= BERT (CCLS) + U + [SEP)) (1)
where H Rd, and d denotes the hidden dimen- sion. During the empirical study, have found that the model can learn the task representation by itself, even without the additional label repre- sentation as an auxiliary feature guide. Therefore, before modeling the interaction between the two tasks of ED and EIC, we obtain different features of text encoding through two fully connected layers to allow the model to learn empathy and empathy intent representations by itself.
H = FC (H)
(2)

---

and the interaction features can obtain the corre- sponding representation of label perception. We do not explicitly use the predicted results to guide another task, as this could lead to problems with the error cascade. Formally, before decoding, we use the Icarnable label signal embedding LS, as input to establish an association with the represen- tations Hg-I-E and H5-E-I, respectivlly, which can explicitly enhance the importance of features that are easier to classify in feature information. We also add H for enhanced capture contextual semantic information.
where > represents the weight of the 1-th class
sample, of denotes Ps. 7 hyper-paramelers and E [0, 5].
The final joint objective is formulated as:
L - a + I.
(12)
where a and / are hyper-parameters.
Experiments
5.1 Experimental Settings
We use the BERT pre-trained model to extract vec- tors as the initialization embedding. The batch size is 32 and 64 on IEMPATHIZE and TwittEmp, re- spectively. The epoch is set to 100. The learning rate is set to 0.0001 and the dropout ratio is set to 0.2. The output dimension of the cascaded inter- action module and the label signal enhancement module is 128. We use Adam (Kingma and Ba, 2015) optimize the parameters in the model. In the loss function, 17 is set to 5, À is set to 1. All ex- periments are conducted at Gelfrree RTX 20801i.
LSg- = IN t EM
①
H8 7
(5)
LH, Attention (LS,, H H8 H8-
H LH H,
where LHg g and ,H, denote the empathy and the empathy intent representations after label signal en- hancement. H E and H' denote the final empathy and the empathy intent representations.
4.4 Decoder
5.2 Baselines
Finally, we apply the max-pooling operation to HG and H to obtain the representations H НEм and HIM, respectively. Two separate decoders are used for ED and EIC.
To validate the performance of our framework, we compare our framework to some baselines, includ- ing: (1) BERT (Hosseini and Caragea, 202 1b): The BERT model was fine-tuned the IEMPATHIZE dataset; (2) KD (Hosseini and Caragea, 202 la::
n² softmax W H EM + b²), HEM
(8)
They used the idea of knowledge distillation combine emotions and sentiments complete the empathy detection; (3) Empirical Study: BERT models as a shared encoder to jointly train the empathy detection task and the empathy intent recognition task; (4) Joint ID and SF (Zhang and Wang, 2016): The model was used for the spo- ken language understanding task, and we use it for the joint training of ED and EIC; (5) DCR- Net(Qin al., 2020): They propose the DCR-NET to explicitly consider the cross-influence between tasks. The model captures mutual knowledge by stacking of relational layers within itself; (6) Co- GAT (Qin et al., 2021): The core module of the model is the co- co-interactive graph interaction layer, in which cross utteeant connections and cross- task connections are constructed and iteratively updated with each other establish information transfer between two tasks. Since this study fo- cuses only on a single-turn dialogue task, we re- place the cross-utterance connections in the model with cross- word connections.
y = softmax (w H + b')
where y" y are the predicted distributions for ED and EIC, respectively. W and 7 are trainable parameters, =o and b' are bias.
4.5 Joint Training
We use a joint training scheme consider both ED and EIC and update the parameters by joint optimization. Lin et al. (2018) proposed to add a modulation factor (1 px)" to the cross-entropy loss to discriminate between easy/hard examples. Pi represents the probability value of the t-th class from softmax output. It can alleviate the problem of unbalanced category distribution in our data. Focal loss is used for ED and EIC, respectively:
Emp
x (1 v log (v
(10)
CIntent -26 (1 vt log (vt) (11)

---

IEMPATHIZE
TwitEmp
Models
ED
EIC
ED
EIC
P(S)
R(%)
FI(S)
P(S)
R(%)
(1())
P(S)
R(%)
(%)
P(S6)
R(%)
FI(%)
Multi-class
Multi-class
BERT
75.88"
73.42"
74.42"
Empirical Study
76.94
77.40
76.16
67.36
66.90
66.15
75.96
75.83
75.78
61.55
60.67
60.75
Joint ID and SF
77.05
76.20
76.24
69.05
68,90
68.38
77.06
76.83
76.82
55.53
57.17
55.94
DCR-NET
77.22
77.70
76.99
68. 12
68,20
67.90
76.80
77.00
76.81
58.43
58.17
57.16
Co-GAT
65.21
64.50
64.73
55.61
55.85
55.73
69.64
69.71
69.35
34.23
32.22
30.20
CLSNN
79.70
78.80
79.13
69.81
70,40
69.69
78.32
78.33
78.31
61.90
61.50
61.01
Seek
Seek
BERT
78.37*
73.40"
76.37"
KD EmoNNet
.7.
61.09"
68.57
Empirical Study
80.48
81.90
79.16
68.05
67.50
66.83
80.45
80.67
80.52
56.84
58.33
56.97
Joint ID and SF
83.08
84.10
83.15
68.30
68.60
68.42
82.99
83.17
82.71
55.99
60.00
57.16
DCR-NET
82.58
83.70
82.61
67.97
67.40
67.11
80.86
80.17
80.39
57.67
58.50
57.44
Co-GAT
70.59
70.19
70.38
55.81
55.45
55.63
78.80
70.12
71.21
27.81
32.26
29.46
CLSNN
84.84
85,40
85.02
69.55
69.40
69.31
84.14
84.33
84.17
60.37
60.50
59.50
Provide
Provide
BERT
86.87
83.37*
84.49"
Emeoett
77 27.21
82.57
79.48
Empirical Study
90.23
90.00
90.10
67.45
67.50
67.83
87.88
87.67
87.74
57.40
58.17
57.19
Joint ID and SF
91.29
90.60
90.85
68.14
68.20
67.17
89.68
89.67
89.67
57.80
59.33
57.82
DCR-NET
90.71
90.90
90.79
67.93
67.00
63.50
90.31
90.33
90.32
56.03
57.67
56.34
Co-GAT
78.38
73.64
75.61
58.20
57.88
58.04
83.21
83.53
83.37
27.18
31.52
28.97
CLSSN
91.64
91.60
91.62
70.92
70.70
70.65
90.82
90.83
90.83
61.19
62.00
61.11
Table 3: ED and EIC results on the IEMPATHIZE and TwitEEmp datasets. "4" and denote original paper results and unreported results, respectively.
5.3 Main Results and Analysis
our framework establishes a deeper information transfer between two tasks. Specifically, after the initial information interaction is completed, the cur- rent task can be used query to search for single- level interaction features, thus achieving deep fea- ture extraction from the current task another task. The information flow continuously interacts between two tasks, eventually bringing feature in- formation from another task back to the current task. It not only enriches the feature information of the current task, but also avoids the lack of fea- tures due to insufficient interaction. In addition, the label signal enhancement module in frame- work uses the label encoding information of the tasks to help the model automatically detect the category features the model, while avoiding the problem of error cascades due to mutual guidance between tasks. The clear bidirectional flow mod- eling information in the two modules enables our framework to achieve competitive results.
For both ED and EIC tasks on the two datasets, we use Precision (P), Recall (R), and Fi-score (F1) as evaluation metries. The experimental results are shown in Table and Appendix B. We can see that the performance of CLSN on both datasets is significantly improved compared with all baselines.
The Joint ID and SF uses the bi-directional GRU to extract features and classify them directly, with- out considering the feature interactions between empathy and empathy intent representations, there- fore, the model achieves lower accuracy. The DCR- NET uses the stacked co-interactive relationship layer implement the interaction between the two tasks. Since the method models the information interaction between two tasks, the accuracy of the model is better than the joint ID and SF. How- ever, DCR-NET has limited information interac- tion capabilities, resulting in lower accuracy than framework. It be seen that the accuracy of Co-GAT is lower, which shows that adopting cross-word connections instead of cross-utterances connections can not fully show the modeling per- formance Co-GAT. Compared with all basclines, the cascaded interaction attention mechanism in
5,4 Ablation Study
This subsection aims to demonstrate the effective- ness of different components our framework, including the cascaded interactive attention mod-

---

BERT
will handle whatever comes way
in terms of treatment None
liways hated the
dreaded scans and mammo's too.
None
Empirical study
you will handle whatever your way
terms of treatment, Provide
always hated dreaded
mammo's
CLSN
you
will
handle
whatever
comes
your
way
in
terms
of
treatment.
Provide
always
hated
the
dreaded
scans
and mammo's
too.
Seek
Figure 6: On the empathy detection task, the CLSN and other baselines visualize the distribution of label contribu- tions at the same network layer. Note: Red background color indicates how much the word contributes to 'Seek', green background color indicates how much the word contributes to 'Provide'. Words with darker background colors indicate higher contribution values the corresponding label.
IEMPATHIZE
dataset are degraded after removing this compo- nent. Therefore, it is effective for the label signal enhancement component to guide the two task rep- resentations. Moreover,
Model
Multi-class
Sock
Provide
ED
EIC
ED
EIC
FD
EIC
CLSN
79.13
69.69
85.02
69.31
91.62
70.65
framework achieves optimal results only when both the cascaded in- teractive attention module and the label signal hancement module are applied to CLSN, proving that these two modules are complementary.
w/o CIA
77.26
69.42
82.99
69.44
90.93
68.98 [
w/o LSE
78.29
1 69.13
83.491
68.24|
90.68
68.80
rp.
CEL
78.33 68.724
82.57J 67.62
91.06]
67.64|
Table 4: Fl-score of ablation experiments on the IEM- PATHIZE dataset.
Effectiveness of Focal Loss. We verify the effec- tiveness of the focal loss by replacing it with the cross-entropy loss (r.p. CEL). The experimental results show that the use of the focal loss to alle- viate the problem of unbalanced distribution of category labels is effective. We can observe that the use of the cross-entropy loss function lower than our proposed full framework under both clas- sifier settings, and there is significant reduction in EIC. This shows that focal loss can indeed help improve the quality of ED by improving the unbal- anced samples of empathy intent during training. Furthermore, can be seen from Tables 3 and 4, when framework uses cross-entropy the loss function under both settings, the F1-scores still out- perform all baselines in most cases, indicating that the accuracy improvement achieved by CLSN is not only the use of the focal loss function, but the superiority of the model is also a factor.
ule, the label signal enhancement module, and the focal loss.
Effectiveness of Cascaded Interactive Atten- tion Module. We remove the cascaded interactive attention module from our framework (w/o CIA). observe that the Fi-score of ED decreases by 1.87% after removing the cas- caded interaction attention module under the multi- class setting in the IEMPATHIZE dataset. The F1- score of EIC decreased by 0.27%. The F1-score of the two tasks also shows decreasing trend in the binary classification setting. These results indicate that the cascaded interactive attention module is essential in the proposed CLSN due to its ability effectively establish the information interaction between the two tasks.
From Table 4,
Effectiveness of Label Signal Enhancement Module. We remove this part of the framework (w/o LSE). From Table 4, we can see that the Fi-score of both settings in the TEMPATHIZE
5.5 Visualisation Analysis
To explore the impact of different modeling ap- proaches on words in utterances, we use the Cap-

---

tumt to visualize the contribution of different words to empathy labels (Kokhlikyan et al., 2020; Chefer et al., 2021). As shown in Figure 6, the BERT does not accurately identify the empathy la- bels for two cases. The empirical study introduced the EIC task and accurately identifies the categories for the two cases, but the model could not focus well on the words associated with the correct label. In contrast, the CLSN not only accurately identifies the empathy labels in both cases, but is also able focus more reasonably on the words associated with the empathy labels in the utterance, demon- strating the superiority of our framework.
correspondence could be direction of exploration for our future work.
Ethics Statement
The dataset studied in this paper does not involve ethical issues.
Acknowledgements
We thank the anonymous reviewers for their help- ful comments and suggestions. This work was supported by the National Natural Science Foun- dation of China (61433012) and the National Key R & D Program of China (No. 2014CB340506) and the Excellent Doctoral Student Research Innovation Project of Xinjiang University (No. XJU2022BS077).
6 Conclusion
In this paper, to investigate whether the perfor- mance of the ED task can be improved by the in- troduction of the EIC task, we invite 3 experts to manually label the empathy intent labels on two em- pathy detection datasets, JEMPATHIZE and Twit- tEmp, as datasets for joint training of ED and EIC. The empirical study shows that the introduction of EIC task is effective in improving the accuracy of the ED task, and we also explore possible reasons for this improvement. In addition, we propose the CLSN, which explicitly models the information flow interaction between the two representations through cascaded interactive attention module and labeled signal enhancement module. Experi- mental results show that the CLSN achieves better accuracy than all baselines under both settings in the two datasets.
References
Firoj Alam, Morena Danieli, and Giuseppe Riccardi. 2018. Annotating and modeling empathy in spo- ken conversations. Computer Speech and Language. 50(C):40-61.
MK Ayshabi and Sumam Mary Idicula. 2021. A multi- resolution mechanism with multiple decoders for em- pathetic dialogue generation. In 2021 8th Interna- tional Conference on Smart Computing and Commu- nicutions (ICSCC), pages 240-245. IEEE.
Guanqun Bi, Yanan Cao, Piji Li, Yugiang Xie, Fang Fang, and Zheng Lin. 2023. Seri: Sketching- reasoning-intedd integratiegg progressive workflow for em- pathetic response generation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1-5, IEEE.
Limitations
Sven Buechel, Anneke Buffone, Barry Slaff, Lyle Un- gar, and João Sedoc. 2018. Modeling empathy and distress in reaction to news stories. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4758-4766
The limitations of this paper are mainly twofold: (1) Although the introduction of the EIC task into the ED task can significantly improve the accuracy of the ED task, this requires manual annotation and is costly, and in future studies we will explore few-shot learning approaches to mitigate this cost; (2) In exploring the reasons for this improvement, we count the frequency co-occurrence matrices of the two labels, and find that there is a clear cor- respondence between the empathy and empathy intent labels. Although there is an obvious corre- spondence between the labels, but the proposed CLSN only uses the two implicit states for interac- tion when modeling information transfer, without explicitly modeling information transfer based on the above label correspondence. How to use this
Hila Chefer, Shir Gur, and Lior Wolf. 2021. Trans- former interpretability beyond attention visualization. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 782-791.
Mao Yan Chen, Siheng Li, and Yujiu Yang. 2022. Em- phi: Generating empathetic responses with human- like intents. In Proceedings of the 2022 Conference of the North American Chapler of the Association for Computational Linguistiex: Human Language Technologies, pages 1063-1074.
Zhuohao Chen, James Gibson, King-Chang Chiu, Qiao- hong Hu, Tara K Knight, Daniella Meeker, James A Tulsky, Kathryn Pollak, and Shrikanth Narayanan. 2020. Automated empathy detection for oncology

---

encounters. 2020 IEEE International Conference on Healtheare Informatics (ICHI), pages 1-8. LEEE Computer Society.
Araya, Siqi Yan, et al. 2020. Captum: A unified and generic model interpretability library for pytorch. arXiv preprint
Pascale Fung, Anik Dey, Farhad Bin Siddique, Ruixi Lin, Yang Yang. Yan Wan, and Ho Yin Ricky Chan. 2016. Zara the supergirl: An empathetic personal- ity recognition system. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demon- strations, pages 87-91.
Tsung- Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2018. Focal loss for dense object detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(2):318-327.
Medeiros and T Bosse. 2016. Empirical analysis of social support provided via social media. In Proceed- ings of the 8th International Conference on Social Informatics, SocInfo' 16, pages 439-453, Springer Verlag.
Jun Gao. Yuhan Liu, Haolin Deng, Wei Wang, Yu Cao, Jiachen Du, and Ruifeng Xu. 2021. Improving em- pathetic response generation by recognizing emotion cause in conversations. In Findings of the Associa- tion for Computational Linguisties: EMNLP 2021, pages 807-819.
Verónica Pérez-Rosas Rada Mihalcea, Kenneth Resni- cow, Satinder Singh, and Lawrence An. 2017. Under- standing and predicting empathic behavior in coun- seling therapy. In Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guistics (Volume I: Long Papers), pages 1426-1435.
Soumitra Ghosh, Dhirendra Maurya, Asif Ekbal, and Pushpak Bhattacharyya. 2022. Team iitp-ainlpml at wassa 2022: Empathy detection, emotion classifica- tion and personality detection. In Proceedings of the 12th Workshop on Computational Approaches to Sub- jectivity, Sentiment & Social Media Analysis, pages 255-260.
Libo Qin. Wanxiang Che, Yangming Li, Mingheng Ni and Ting Liu. 2020. Der-net: A deep co-interactive relation network for joint dialog act recognition and sentiment classification. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8665-8672.
Mahshid Hosseini and Cornelia Caragea. 2021a. Dis- tilling knowledge for empathy detection. In Find- ings of the Association for Computational Linguis- ties: EMNLP 2021. pages 3713-3724.
Libo Qin, Zhouyang Li, Wanxiang Che, Minheng Ni, and Ting Liu. 2021. Co-gat: co-interactive graph attention network for joint dialog act recognition and sentiment classification. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 13709-13717.
Mahshid Hosseini and Comeela Caragea. 2021b.
takes two to empathize: One to seek and one to pro- vide. In Proceedings of the AAAI Conference on Ar- rificial Intelligence, volume 35, pages 13018-13026.
Tulika Saha and Sophia Ananiadou. 2022. Emotion- aware and intent-controlled empathetic response gen- eration using hierarchical transformer network. In 2022 International Joint Conference Neural Net- works (IJCNN), pages 1-8. IEEE.
Jacob Devlin Ming-Woe Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-train tre-training of deep bidirec- tional transformers for language understanding. In Proceedings of NAT, pages 4171-4186
Hamed Khanpour, Comella Caragea, and Prakhar Biyani. 2017. Identifying empathetic messages in online health communities. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers). pages 246-251.
Azlaan Mustafa Samad, Kshitij Mishra, Mauajama Fir- daus, and Asif Ekbal. 2022. Empathetic persuasion: Reinforcing empathy and persuasiveness in dialogue systems. In Findings of the Association for Compu- tational Linguistics: NAACL 2022, pages 844 856.
Wongyu Kim, Youbin Ahn, Donghyun Kim, and Kyong- Ho Lee. 2022. Emp-rft: Empathetic response gen- eration via recognizing feature transitions between utterances. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computtaiional Linguisticx: Human Language Technologies, pages
João Sedoc, Sven Buechel, Yehonathan Nachmany, An- neke Buffone, and Lyle Ungar. 2020. Learning word ratings for empathy and distress from document-leve user responses. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 1664- 1673.
Ashish Sharma, Adam Miner, David Atkins, and Tim Al- thotf. 2020. A computational approach to understand- ing empathy expressed in text-based mental health support. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 5263-5276.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations ICIR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
Narine Kokhlikyan, Vivek Miglani, Miguel Martin, Edward Wang, Bilal Alsallakh, Jonathan Reynolds, Alexander Melnikov, Natalia Kliushkina, Carlos
Laurens Van der Maaten and Geoffrey Hinton. 2008
Visualizing data using t-sne. Journal of machine learning research, 9(1)).

---

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processsin systems, 30.
A
Dataset Statistics
Split
IEMPATHIZE
TwittFmp
None
Seek
Provide
None
Seek
Provide
Train
1794
630
580
616
614
570
M. Virvou and G. Katsionis. 2004. Relating error di- agnosis and performance characteristics for affect perception and empathy in educational software application. pages 22-27.
Dev
609
191
203
203
180
217
Test
592
225
183
181
206
213
Total
2995
1046
966
1000
1000
1000
Table 5: Dataset Statistics for IEMPATHIZE and Twit-
Anuradha Welivita and Pearl Pu. 2020. A taxonomy of empathetic response intents in human social conversa- tions. Proceedings of the 28th International Con- ference on Computational Linguistics, pages 4886- 4899.
tEmp on utterance (Ut)) and empathy label (None, Seek, Provide) counts multi-class setting.
Split
IEMPATHIZE
TwittEmp
Anuradha Welivita, Yubo Xie, and Pearl Pu. 2021. A large-scale dataset empathetic response gener- ation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1251-1264.
None
Seek
None
Seek
Train
2374
630
1186
614
Dev
812
191
420
180
Test
775
225
394
206
Total
3961
1046
2000
1000
Akmal Setiawan Wijaya, Dhomas Hatta Fudholi, and Ahmad R Pratama. 2023. A computational approach analyzing the empathy online donations during covid-19, 19. MATRIK: Jurnal Manajemen, Teknik In-
Table 6: Dataset Statistics for IEMPATHIZE and Twit- tEmp on utterance (Utt) and empathy label (None, Seek, Provide) counts in binary setting (seck)
formatika dan Rekayasa Komputer, 22(2):185 194.
AC de C Williams and A Cano. 2005. Facing others in pain: the effects of empathy. Pain, 118(3):285-285 288.
Split
IEMPATHIZE
TwittEmp
None
Provide
None
Provide
Train
2424
580
1230
570
Yubo Xie and Pearl Pu. 2021. Empathetic dialog gen-
Dev
800
203
383
cration with fine-grained intents. In Proceedings of
Test
817
183
387
213
the 25th Conference on Computational Natural Lan- guage Learning, pages 133-147.
Total
4041
966
2000
1000
Özge Nilay Yalcin and Steve DiPaola. 2018. com- putational model empathy for interactive agents. Biologically inspired cognitive architectures, 26:20- 25.
Table 7: Dataset Statistics for IEMPATHIZE and Twit-
mmpp on utterance (Utt) and empathy label (None, Seek, Provide) counts in binary setting (provide)
Diyi Yang. Robert Kraut, Tenbroeck Smith, Eli- jah Mayfield, and Dan Jurafsky. 2019. Seekers,
B class-wise Performance
providers, welcomers, and storytellers: Modeling so- cial roles in online health communities. In Proceed- ings of the 2019 CHI conference on human factors in computing systems, pages 1-14.
Figure 7, Figure 8, and Figure 9 show the results of the CLSN for each category in different settings in the ED and EIC tasks, respectively.
Xiaodong Zhang and Houfeng Wang. 2016. A joint model of intent determination and slot filling for spo- ken language understanding. In Proceedings of the Fienty-Fiftll International Joint Conference on Arti- ficial Intelligence, pages 2993-2999.
Naitian Zhou and David Jurgens. 2020. Condolence and empathy in online communities. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 609 626.

---

100
(a)
100
(b)
80
80
60
60
40
40
20
20
None
Seek
Provide
Acknowledging
Consoling
Neutral
Questioning
Sympathizing
Wishing
Positive
Negative
None
Seek
Provide
Acknowledging
Consoling
Neutral
Questioning
Sympathizing
Wishing
Positive
Negative
EIC
ED
EIC
Figure 7: Results of the CLSN model the ED and EIC tasks in the multi-class setting of the IEMPATHIZE (a) and TwittEmp (b) datasets.
(a)
(b)
100
80
80
60
60
40
40
20
20
None
Acknovledging
Consoling
Neutral
Questioning
Sympathizing
Wishing
Positive
Negative
None
Acknoowledggng
Consoling
Neutral
Questioning
Sympsthizing
Wishing
Positive
Negative
ED
EIC
ED
EIC
Figure 8: Results of the CLSN model on the ED and EIC tasks in the binary setting (seek) of the TEMPATHIZE (a) and TwittEmp (b) datasets.
100
(a)
100
(b)
80
80
60
60
40
40
20
20
None
Acknovleedging
Consoling
Noutral
Questioning
Sympathizing
Wishing
Positive
Negative
None
Provide
Consoling
Neutral
Questioning
Sympathizing
Wishing
Positive
Negative
Acknovledgingg
ED
EIC
ED
EIC
Figure 9: Results of CLSN model on ED and EIC tasks the binary setting (provide) of the IEMPATHIZE
(a) and TwittEmp (b) datasets.


Reviews:
[{"review_id": "4WFiKf3l9q", "reviewer": "Reviewer_D1Wg", "summary": "", "questions": "1. How exactly does the label signal enhancement module extract relevant features from the label information? A more detailed explanation would be beneficial.\n\n2. Can you elaborate on the potential relationship between empathy detection and empathy intent recognition? How does one task affect the other?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks very much for your review comments!\n\n**Reasons To Reject:**\n\n**Q1: Explanation of Label Signal Enhancement: While the paper introduces the label signal enhancement module, it could benefit from a more detailed explanation of how the module effectively extracts informative features from the label information.**\n\n**Response 1**: Concatenating all label encodings of the two tasks as the query vectors of the module with the empathy and empathy intent representations for the attention computation can effectively capture, the semantics of the two information with respect to labels. Taking the empathy detection task as an example, the computation with label encoding can not only capture the semantics about empathy, but also capture the semantics about empathy intent for assisting the decoding of empathy, and the same is true for the empathy intent recognition task.\n\n**Q2: Relationship between ED and EIC: the paper does not delve deep into the potential relationship between empathy detection and empathy intent recognition, which could be addressed to enhance the understanding of the proposed approach.**\n\n**Response 2:** The potential relationship between the two tasks ED and EIC is reflected in the semantics, due to the fact that some key semantics in the utterance can provide relevant information for both tasks. For example: \"I am sorry to hear about your mother-in-law’s troubles after her diagnosis, I am sympathetic\", the word 'sympathetic' reflects the empathy intent of 'Sympathizing' and expresses the direction of empathy as 'Provide'. The EIC channel shares the captured semantics with the ED channel which can help the ED channel to better perceive the empathy information contained in the utterance. In addition, the correlation between the two tasks is also reflected in the effect of label correspondence, as detailed in Response 3.\n\n**Q3: Impact of Label Correspondence: The correspondence between empathy labels and empathy intent labels is mentioned, but the paper does not explicitly elaborate on how this correspondence influences the model's performance.**\n\n**Response 3:** As can be seen from Table 1, the accuracy of empathy detection increases after the introduction of the empathy intent recognition task, possibly the reason being that the correspondence between the two kinds of labels makes the model more likely to accurately perceive the empathy labels of the utterance. The correspondence between the empathy detection labels and the empathy intent recognition labels is reflected in the co-occurrence matrix of the two kinds of labels (see Table 2 for details). When the model perceives the utterance's empathy intent label as 'Consoling', 'Sympathizing', or 'Wishing', then it is more likely that the empathy label of the utterance is 'Provide'. Similarly, if the empathy intent label is 'Negative', the empathy label is likely to be 'Seek', whereas if the empathy intent label is  'Acknowledging', 'Neutral', 'Questioning', or 'Positive', the empathy label is likely to be 'None'. This correspondence makes the model less difficult to detect empathy.\n\n**Questions For The Authors:**\n\n**Q1: How exactly does the label signal enhancement module extract relevant features from the label information? A more detailed explanation would be beneficial.**\n\n**Answer 1:** Please refer to **Response 1**.\n\n**Q2: Can you elaborate on the potential relationship between empathy detection and empathy intent recognition? How does one task affect the other?**\n\n**Answer 2:** Please refer to **Response 2**.\n\nWe will revise the paper based on the above comments! Thanks again for your valuable comments!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Novel Approach: The paper addresses an underexplored area by jointly considering empathy detection and empathy intent recognition. This approach adds to the understanding of how empathy is expressed and perceived in dialogue.\n\n2. Cascaded Interaction Module: The proposed CLSN effectively models interaction between ED and EIC representations using cascaded attention modules, providing a structured approach to feature exchange.\n\n3. Label Signal Enhancement Module: The label signal enhancement module adds an innovative aspect to the framework, which helps the model effectively capture features from both tasks.\n\n4. Experimental Evaluation: Comprehensive experiments with comparisons to multiple baselines on different datasets demonstrate the superiority of the CLSN.", "reasons_to_reject": "1. Explanation of Label Signal Enhancement: While the paper introduces the label signal enhancement module, it could benefit from a more detailed explanation of how the module effectively extracts informative features from the label information.\n\n2. Relationship Between ED and EIC: The paper does not delve deep into the potential relationship between empathy detection and empathy intent recognition, which could be addressed to enhance the understanding of the proposed approach.\n\n3. Impact of Label Correspondence: The correspondence between empathy labels and empathy intent labels is mentioned, but the paper does not explicitly elaborate on how this correspondence influences the model's performance.", "questions_to_authors": "1. How exactly does the label signal enhancement module extract relevant features from the label information? A more detailed explanation would be beneficial.\n\n2. Can you elaborate on the potential relationship between empathy detection and empathy intent recognition? How does one task affect the other?", "paper_topic_and_main_contributions": "The paper explores the interaction between empathy detection (ED) and empathy intent recognition (EIC) in dialogue systems, an area with limited prior research. The authors introduce a novel framework called the Cascaded Label Signal Network (CLSN) to jointly train ED and EIC tasks. They manually annotate empathy intent labels on two datasets and demonstrate that the introduction of EIC improves ED accuracy. The proposed CLSN outperforms various baselines on both datasets, showcasing its effectiveness.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "JGXZcHIvSj", "reviewer": "Reviewer_vC87", "summary": "", "questions": "1. Why are some results missing in Table 3?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks very much for your review comments!\n\n**Reasons To Reject:**\n\n**Q1: There are some missing results in Table 3. No numbers are reported for the BERT baseline in TwittEmp. Results for some baselines are also missing in IEMPATHIZE. I wonder why the authors did not report the numbers.**\n\n**Response 1:** The IEMPATHIZE and TwittEmp datasets were proposed for the ED task study, so there were no results from the EIC task. BERT[1] and KDEmoNet[2] have no data on the ED task because the original paper did not perform experiments in the corresponding setting, and we used the results from the original paper. \n\n[1] Hosseini M, Caragea C. It takes two to empathize: One to seek and one to provide[C]//Proceedings of the AAAI conference on artificial intelligence. 2021, 35(14): 13018-13026.\n\n[2] Hosseini M, Caragea C. Distilling knowledge for empathy detection[C]//Findings of the Association for Computational Linguistics: EMNLP 2021. 2021: 3713-3724.\n\n**Q2: Although the problem is well-defined, the scope is narrow at the moment. I believe empathetic response generation is the main challenge, for which empathy detection would be a sub-problem. If the empathy intent recognition module could somehow improve empathetic response generation, then that would make the paper much stronger.**\n\n**Response 2:** Empathy intent has been proven to improve the quality of empathetic response generation [3], however, no one has explored the relationship between empathy intent recognition and empathy detection tasks. Based on this, we jointly trained the two tasks, and the experimental results demonstrated a mutual promotion between an empathy detection task and an empathy intent recognition task. As soon as the paper is accepted, we will make the dataset and source code publicly available to advance the development of empathetic dialogue systems. In future work, we plan to further improve the quality of empathetic response generation based on our existing conclusions. We look forward to receiving your valuable comments at that time!\n\n[3] Chen M Y, Li S, Yang Y. EmpHi: Generating Empathetic Responses with Human-like Intents[C]//Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022: 1063-1074.\n\n**Questions For The Authors:**\n\n**Q1: Why are some results missing in Table 3?**\n\n**Answer 1:** Please refer to **Response 1**.\n\n**Typos Grammar Style And Presentation Improvements:**\n\n1. **The bolded texts in Section 3.4 are quite distracting. Emphasis should be written in italics.**\n\n2. **Opening quotes are backwards. You can use the ``` symbol in latex for opening quotes: ’Seek’ -> `Seek’**\n\n**Response:** We will carefully read and scrutinize all the details in the paper, and revise both the errors pointed out by the reviewers and any potential detail inaccuracies.\n\nWe will revise the paper based on the above comments! Thanks again for your valuable comments!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Well-executed preliminary analysis to show why empathy intent recognition is helpful.\n2. The dataset consisting of the empathy intent labels for IEMPATHIZE and TwittEmp would be helpful to the community if released.\n3. The proposed methodology of joint modeling of empathy and intent seems novel for the problem of empathy understanding. The ablations and analysis provided are helpful.", "reasons_to_reject": "1. There are some missing results in Table 3. No numbers are reported for the BERT baseline in TwittEmp. Results for some baselines are also missing in IEMPATHIZE. I wonder why the authors did not report the numbers.\n\n2. Although the problem is well-defined, the scope is narrow at the moment. I believe empathetic response generation is the main challenge, for which empathy detection would be a sub-problem. If the empathy intent recognition module could somehow improve empathetic response generation, then that would make the paper much stronger.", "questions_to_authors": "1. Why are some results missing in Table 3?", "paper_topic_and_main_contributions": "The paper presents a method for empathy detection with empathy intent recognition. The authors first provide motivation as to why this is necessary and then present some preliminary experiments to show that empathy intent can indeed help empathy detection. Finally, an interactive model with joint cross-attention in BERT is proposed, which helps in further improving the performance. The method is benchmarked on the IEMPATHIZE and TwittEmp datasets, where the proposed model shows improvement over several baselines in various classification metrics.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. The bolded texts in Section 3.4 are quite distracting. Emphasis should be written in italics.\n2. Opening quotes are backwards. You can use the `` ` `` symbol in latex for opening quotes: ’Seek’ -> `Seek’"}}, {"review_id": "KlEGDUtm5Y", "reviewer": "Reviewer_Xn3k", "summary": "", "questions": "I found a paper published by Libo Qin on AAAI in 2021 titled “Co-GAT: A Co-Interactive Graph Attention Network for Joint Dialog Act Recognition and Sentiment Classification” . This paper is also related to the theme of empathy detection and Libo Qin is also the author of the baseline model DCR-Net, but I don’ t see the comparison between CLSN framework and Co-GAT model in the baseline model. May I ask if the experimental effect is not as good as the Co-GAT model, or what other reason?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks very much for your review comments!\n\n**Reasons To Reject:**\n\n**Q1: The paper might have limited algorithmic novelty.** \n\n**Response 1:** We understand your concern about novelty. But we believe that our study has a certain degree of novelty and extensibility. The reasons are summarized as follows: \n\n**(1)** Previous studies on empathy detection have focused only on the detection of empathy features, ignoring empathy intent features. To the best of our knowledge, this is the first attempt to jointly train the empathy detection task and the empathy intent recognition task;\n\n**(2)** Our framework uses two kinds of label embeddings as query vectors to use attention to capture semantics related to empathy and empathy intent. The empathy detection channel perceives not only the empathy label embedding information, but also the empathy intent label embedding information, which facilitates the capture of shared information between tasks，the empathy intent recognition task is the same;\n\n**(3)** The findings of this paper are somewhat expandable. Based on previous research[1], it is possible that the joint training method in this paper can be used to improve the quality of empathetic response generation. We look forward to your valuable comments at that time.\n\n[1] Chen M Y, Li S, Yang Y. EmpHi: Generating Empathetic Responses with Human-like Intents[C]//Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022:1063-1074.\n\n**Q2: The baseline model compared in this paper is not novel enough.**\n\n**Response 2:** We are the first to attempt a joint training task for empathy detection and empathy intent recognition in a single utterance, so the number of baseline models is limited. We will make the dataset and source code publicly available on GitHub as soon as possible, and look forward to more interested researchers to improve the performance of empathy detection and empathy intent recognition tasks!\n\n**Questions For The Authors:**\n\n**Q1: I found a paper published by Libo Qin on AAAI in 2021 titled “Co-GAT: A Co-Interactive Graph Attention Network for Joint Dialog Act Recognition and Sentiment Classification” . This paper is also related to the theme of empathy detection and Libo Qin is also the author of the baseline model DCR-Net, but I don’ t see the comparison between CLSN framework and Co-GAT model in the baseline model. May I ask if the experimental effect is not as good as the Co-GAT model, or what other reason?**\n\n**Answer 1:** The IEMPATHIZE and TwittEmp datasets are both constructed based on a single utterance. We observed that Co-GAT performs well in a multi-turn dialogue task and there are cross-utterances connection modules in the model, but this setup was not compatible with our study, so we did not include it in the baseline of the experiment. Because of the differences in the data, it was not possible to directly migrate the Co-GAT model to the dataset in this paper, so we replaced the cross-utterances connections in that model with word-to-word connections. The experimental results of Co-GAT in different settings of the IEMPATHIZE dataset are as follows:\n\n|             |       |  ED   |       |       |  EIC  |       |\n| :---------- | :---: | :---: | :---: | :---: | :---: | :---: |\n|             | P(%)  | R(%)  | F1(%) | P(%)  | R(%)  | F1(%) |\n| Multi-class | 65.21 | 64.50 | 64.73 | 55.61 | 55.85 | 55.73 |\n| Seek        | 70.59 | 70.19 | 70.38 | 55.81 | 55.45 | 55.63 |\n| Provide     | 78.38 | 73.64 | 75.61 | 58.20 | 57.88 | 58.04 |\n\nComparing the above experimental results with the results of different models in Table 3, it can be seen that the accuracy of Co-GAT is lower, which shows that adopting word-to-word connections instead of cross-utterances connections in Co-GAT can not fully show the modeling performance of Co-GAT, and we will add the results of Co-GAT to the experimental part. In addition, we will continue to expand our existing research into complex multi-turn empathetic dialogue, at which time Co-GAT will still be used as a baseline model for comparison.\n\nWe will revise the paper based on the above comments! Thanks again for your valuable comments!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1.The paper addresses the issue of the interaction between empathy detection and empathy intent recognition.\n.The experimental results of this paper are well.\n3. The motivations are clear.", "reasons_to_reject": "1.The paper might have limited algorithmic novelty.\n2.The baseline model compared in this paper is not novel enough.", "questions_to_authors": "I found a paper published by Libo Qin on AAAI in 2021 titled “Co-GAT: A Co-Interactive Graph Attention Network for Joint Dialog Act Recognition and Sentiment Classification” . This paper is also related to the theme of empathy detection and Libo Qin is also the author of the baseline model DCR-Net, but I don’ t see the comparison between CLSN framework and Co-GAT model in the baseline model. May I ask if the experimental effect is not as good as the Co-GAT model, or what other reason?", "paper_topic_and_main_contributions": "This work mainly focuses on empathtic dialogue systems, finding that empathy intent information improves the ability to response capacity of empathetic dialogue. This paper summarize the main contributions as follows: (1) The authors invite 3 experts to manually annotate empathy intent labels on the empathy detection datasets IEMPATHIZE and TwittEmp and provide the necessary 105 data for the joint ED and EIC study; (2) The authors are the first to attempt joint training of the ED and EIC tasks to improve the accuracy of the ED task and also explore the possible reasons for the accuracy improvement; (3) To make joint training of the two tasks more challenging, the authors propose a novel framework, CLSN, which explicitly controls the knowledge transfer between the two tasks. Experimental results show that the framework outperforms all baselines under both settings in the two datasets.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "Qin, L., Li, Z., Che, W., Ni, M., & Liu, T. (2021). Co-GAT: A Co-Interactive Graph Attention Network for Joint Dialog Act Recognition and Sentiment Classification. Proceedings of the AAAI Conference on Artificial Intelligence, 35(15), 13709-13717."}}, {"review_id": "rm55nX2NbS", "reviewer": "Reviewer_v6LG", "summary": "", "questions": "The limitation with respect to work is not clearly defined in the limitation section. The statistics of the dataset is not included.  The relationship between None empathy and Neutral Intent should be clearly represented as some none samples are in other categories. Justify it.  Table 3 is not explained in the content.  Provide a class-wise performance in EIC in order to see which class is performing well or average. \nJustify the text \"The accuracy of the ED task can be significantly improved by introducing an EIC task into the ED task.\" with respect to results.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks very much for your review comments!\n\n**Reasons To Reject:**\n\n**Q1: The dataset labeling can be made available for more samples to get well-proof ness of method.**\n\n**Response1:** Yes, the available dataset is indeed small. In future studies, we will keep expanding the datasets for joint training of empathy detection and empathy intent for research use.\n\n**Questions For The Authors:**\n\n**Q1: The limitation with respect to work is not clearly defined in the limitation section.** \n\n**Answer 1:** We will modify our limitations section to the following description: The limitations of this paper are mainly twofold: 1. Although the introduction of the EIC task into the ED task can significantly improve the accuracy of the ED task, this requires manual annotation and is costly, and in future studies we will explore few-shot learning approaches to mitigate this cost; 2. In exploring the reasons for this improvement, we count the frequency co-occurrence matrices of the two labels, and find that there is a clear correspondence between the empathy and empathy intent labels. Although there is an obvious correspondence between the labels, but the proposed CLSN only uses the two implicit states for interaction when modeling information transfer, without explicitly modeling information transfer based on the above label correspondence. How to use this correspondence could be a direction of exploration for our future work.\n\n**Q2: The statistics of the dataset is not included.**\n\n**Answer 2:** We will add an appendix section that reports in detail the statistics for both datasets.\n\n**Q3: The relationship between None empathy and Neutral Intent should be clearly represented as some none samples are in other categories. Justify it.**\n\n**Answer 3:** As can be seen from Table 2, None and Neutral co-occurrence as often as 63%, indicating that they have a high correspondence in most cases. However, there are other cases, for example, some utterances simply state objective states, or questions, without any requirement for empathy. The following are two simple examples：\n\nEmpathy intent label: 'Acknowledging', utterance: \"This board is a wonderful place to get info and support.\"\n\nEmpathy intent label: 'Questioning'，utterance: \"Anyone else have low-med onco score and not doing chemo?\"\n\n**Q4: Table 3 is not explained in the content. Provide a class-wise performance in EIC in order to see which class is performing well or average.** \n\n**Answer 4:** We will modify the explanation and analysis of Table 3 in 5.3. In addition, we will report bar charts of the accuracy of CLSN for class-wise from the baseline model in the appendix for the reader's reference. The following shows CLSN's results for each class in the ED and EIC for the IEMPATHIZE dataset under the multi-class setting:\n\n| task |     label     |  P(%)  |  R(%)  | F1(%)  |\n| :--: | :-----------: | :----: | :----: | :----: |\n|  ED  |     none      | 87.33 | 82.46 | 84.82 |\n|      |     seek      | 60.44 | 70.47 | 65.07 |\n|      |    provide    | 73.77 | 75.00 |74.38 |\n| EIC  | Acknowledging | 41.67 | 24.69 | 31.01 |\n|      |   Consoling   | 67.68 | 70.53 | 69.07 |\n|      |    Neutral    | 72.77 | 80.10  | 76.26 |\n|      |  Questioning  | 96.97 | 91.43 | 94.12 |\n|      | Sympathizing  | 83.33 |35.71 | 50.00   |\n|      |    Wishing    | 78.38 | 70.73 | 74.36 |\n|      |   Positive    | 53.45 | 59.62 | 56.36 |\n|      |   Negative    | 76.17 | 73.66 | 74.90  |\n\n**Q5: Justify the text \"The accuracy of the ED task can be significantly improved by introducing an EIC task into the ED task.\" with respect to results.**\n\n**Answer 5:** The results in Table 1 show that after jointly training the ED and EIC tasks, the results of the ED task are significantly improved in all the different settings. The reason is that after the introduction of the EIC task, the data of 'Seek' and 'Provide' in the ED task become more discriminable, and there is a certain correlation between the two labels ED and EIC (see Table 2 for details), and this correlation increases the discriminability of the 'Seek' utterance and the 'Provide' utterance. In empirical study, the frequency of correct classification on the test set of the IEMPATHIZE dataset using the joint training approach in the multi-class setting is as follows:\n\n|     Label     | None | Seek | Provide |\n| :-----------: | :--: | :--: | :-----: |\n| Acknowledging |  13  |  0   |    5    |\n|   Consoling   |  0   |  2   |   51    |\n|    Neutral    | 290  |  0   |    4    |\n|  Questioning  |  25  |  1   |    0    |\n| Sympathizing  |  0   |  0   |    7    |\n|    Wishing    |  1   |  0   |   29    |\n|   Positive    |  26  |  1   |   13    |\n|   Negative    |  9   |  82  |    0    |\n\nThe frequency distribution of correctly identifying the two tasks in the table is consistent with the label co-occurrence distribution in Table 2, reflecting the rationality of our analysis.\n\n**Typos Grammar Style And Presentation Improvements:**\n\n**Explain I and E with respect to line 326 section 4.6. Table 4 is not discussed in the text. A grammar check is recommended.**\n\n**Response:** I and E are used as subscripts of H. I denotes empathy intent and E denotes empathy, respectively. We will read the full text carefully and correct any errors in detail that have been pointed out and potentially made by the reviewers.\n\nWe will revise the paper based on the above comments! Thanks again for your valuable comments!", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "A novel approach is proposed in this work to combine ED and EIC tasks. The paper has shown performance on IEMPATHIZE and TwittEmp datasets. The contribution lies in 1. combining ED and EIC tasks 2. Creating/updating a dataset with 3 expert opinions 3. Novel Architecture 4. Thorough results", "reasons_to_reject": "The dataset labelling can be made available for more samples to get well-proof ness of method. ", "questions_to_authors": "The limitation with respect to work is not clearly defined in the limitation section. The statistics of the dataset is not included.  The relationship between None empathy and Neutral Intent should be clearly represented as some none samples are in other categories. Justify it.  Table 3 is not explained in the content.  Provide a class-wise performance in EIC in order to see which class is performing well or average. \nJustify the text \"The accuracy of the ED task can be significantly improved by introducing an EIC task into the ED task.\" with respect to results.", "paper_topic_and_main_contributions": "The major contribution of the paper is an exploration of interaction of ED and EIC tasks. The BERT model is used to obtain the semantic features of utterances and extracts different representations through two linear layers. Then, the cascaded interactive attention module is used\nto implement feature interaction and control knowledge flow. Finally, the label signal enhancement module is used to further extract interactive features from the label information of the two tasks and feed them to different decoders to complete ED and EIC.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "Explain I and E with respect to line 326 section 4.6. Table 4 is not discussed in the text. A grammar check is recommended. "}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes an interesting idea such as jointly learning empathy intent with empathy for achieving better results for empathy classification in utterances. 

The paper is well-written and the results are solid, however, it is not clear how this model is effective in helping the empathic generation. The results of jointly learning two well-balanced tags on the same text content are always better than single training. 

The annotation is well-designed and the results of kappa between two annotators are high. It would be great if the authors provided the count for each category.

If accepted I suggest the reviewers' comments should be considered.
Decision: Accept-Main