Modeling User Satisfaction Dynamics in Dialogue via Hawkes Process
Year: 2023

Abstract:
Dialogue systems have received increasing attention while automatically evaluating their performance remains challenging. User satisfaction estimation (USE) has been proposed as an alternative. It assumes that the performance of a dialogue system can be measured by user satisfaction and uses an estimator to simulate users. The effectiveness of USE depends heavily on the estimator. Existing estimators independently predict user satisfaction at each turn and ignore satisfaction dynamics across turns within a dialogue. In order to fully simulate users, it is crucial to take satisfaction dynamics into account. To fill this gap, we propose a new estimator ASAP (sAtisfaction eStimation via HAwkes Process) that treats user satisfaction across turns as an event sequence and employs a Hawkes process to effectively model the dynamics in this sequence. Experimental results on four benchmark dialogue datasets demonstrate that ASAP can substantially outperform state-of-the-art baseline estimators.

heading: 1 Introduction
text: Dialogue systems are playing an increasingly important role in our daily lives. They can serve as intelligent assistants to help users accomplish tasks and answer questions or as social companion bots to converse with users for entertainment (Ni et al., 2022; Fu et al., 2022). In recent years, the research and development of dialogue systems has made remarkable progress. However, due to the complexity of human communication, the latest dialogue systems may still fail to understand users’ intents and generate inappropriate responses (Liang et al., 2021; Deng and Lin, 2022; Pan et al., 2022). These deficiencies pose huge challenges to deploying dialogue systems to real-life applications, especially high-stakes ones such as finance and health. In light of this, it is crucial to evaluate the performance of dialogue systems adequately in their development phase (Sun et al., 2021; Deriu et al., 2021).
Generally speaking, there are two types of evaluation methods, human evaluation and automatic evaluation (Deriu et al., 2021). Human evaluation is fairly effective, but costly and hard to scale up. By contrast, automatic evaluation is more scalable. However, due to the ambiguity of what constitutes a high-quality dialogue, there are currently no universally accepted evaluation metrics. Existing commonly used metrics such as BLEU (Papineni et al., 2002) usually do not agree with human judgment. Nonetheless, user satisfaction estimation (USE) has been proposed as an alternative (Bodigutla et al., 2019; Park et al., 2020; Kachuee et al., 2021; Sun et al., 2021). USE assumes that the performance of a dialogue system can be approximated by the satisfaction of its users and simulates users’ satisfaction with an estimator. In this regard, USE performs automatic evaluation and is thus scalable.
Aside from helping developers find the defects of a dialogue system, USE also makes it possible to carry out timely human intervention for dissatisfied users and continuously optimize the system from human feedback (Hancock et al., 2019; Bodigutla et al., 2020; Deriu et al., 2021; Deng et al., 2022). In essence, USE is a multi-class classification problem and the goal is to predict user satisfaction at each turn. Take the dialogue shown in Figure 1 as
8875
an example, where user satisfaction is measured on a three-point scale. At the first two turns, the system responds appropriately. However, at the third turn, even though the response seems to be reasonable, the system asks for information that the user has already provided at the first turn, which may lead to dissatisfaction.
As a model-based metric, the evaluation quality of USE relies heavily on the satisfaction estimator used. In order to train a robust estimator, different approaches have been proposed (Sun et al., 2021; Liang et al., 2021; Kachuee et al., 2021; Pan et al., 2022; Deng et al., 2022). Despite the effectiveness of these approaches, they estimate user satisfaction at each turn independently and ignore the dynamics of user satisfaction across turns within a dialogue. Given that a user’s satisfaction is not only related to the current dialogue context, but may also be related to the satisfaction states at previous turns, we argue that modeling user satisfaction dynamics is valuable for training a more powerful estimator.
To achieve this, we propose ASAP (sAtisfaction eStimation via HAwkes Process), a novel approach that leverages Hawkes process (Hawkes, 2018) to capture the dynamics of user satisfaction. Hawkes process is a self-exciting point process and it has been widely adopted to model sequential data such as financial transactions (Bacry et al., 2015) and healthcare records (Wang et al., 2018). In particular, we make the following contributions:
• We first propose a base estimator to predict user satisfaction based solely on the dialogue context. We then incorporate a Hawkes process module to model user satisfaction dynamics by treating the satisfaction scores across turns within a dialogue as an event sequence.
• We propose a discrete version of the continuous Hawkes process to adapt it to the USE task and implement this module with a Transformer architecture (Vaswani et al., 2017).
• We conduct extensive experiments on four dialogue datasets. The results show that ASAP substantially outperforms baseline methods.

heading: 2 Problem Statement
text: Suppose that we are provided with a dialogue session X containing T interaction turns, denoted as X = {(R1, U1), (R2, U2), . . . , (RT , UT )}. Each interaction turn t (1 ≤ t ≤ T ) consists of a responseRt by the system and an utterance Ut by the
user. The goal of USE is to predict the user satisfaction score st at each turn t based on the dialogue context Xt = {(R1, U1), (R2, U2), . . . , (Rt, Ut)}. Hence, our task is to learn an estimator E : Xt → st that can accurately estimate the user’s satisfaction throughout the entire dialogue session.
Previous studies have shown that adding user action recognition (UAR) as an auxiliary task can facilitate the training of a stronger satisfaction estimator (Sun et al., 2021; Deng et al., 2022). When user action labels are available, our task shifts to learning an estimator E ′ : Xt → (st, at) that predicts user satisfaction and user action simultaneously. Here, at denotes the user action at turn t.

heading: 3 Method
text: In this section, we first describe how to build a base USE model leveraging only the dialogue context and without modeling the dynamics of user satisfaction. Then, we extend this model by integrating the Hawkes process to capture the dynamic changes of user satisfaction across dialogue turns. The overall model architecture is illustrated in Figure 2.

heading: 3.1 Base Satisfaction Estimator
text: Similar to Deng et al. (2022), we utilize a hierarchical transformer architecture to encode the dialogue context Xt into contextual semantic representations. A hierarchical architecture enables us to handle long dialogues. This architecture consists of a token-level encoder and a turn-level encoder.

heading: 3.1.1 Token-Level Encoder
text: The token-level encoder takes as input the concatenation of the system responseRt and user utterance Ut at each turn t and yields a single vector ht as their semantic vector representation. To be specific, we adopt the pre-trained language model BERT (Devlin et al., 2019) to encode each (Rt, Ut) pair:
ht = BERT([CLS]Rt[SEP ]Ut[SEP ]). (1)

heading: 3.1.2 Turn-Level Encoder
text: The token-level encoder can only capture the contextual information within each turn. In order to capture the global contextual information across turns, we develop a turn-level encoder that takes the semantic representations {h1,h2, . . . ,ht} of all turns in the dialogue context Xt as input. We implement this encoder as a unidirectional Transformer encoder with L layers. Similar to the standard Transformer encoder layer (Vaswani et al.,

heading: 4(5674#567 4'567
text: 2017), each layer includes two sub-layers. The first sub-layer is a masked multi-head attention module (MultiHead). The second sub-layer is a positionwise feed-forward network which is composed of two linear transformations with a ReLU activation in between (FFN).
Formally, each layer of the turn-level encoder operates as follows:
H(0) = [h1 + pe(1), . . . ,ht + pe(t)], (2) H∗ = MultiHead(H(l),H(l),H(l)), (3)
H(l+1) = FFN(H∗ +H(l)) +H∗ +H(l), (4)
where H(0) is the input of the first layer, in which we add positional encodings pe(·) to retain the turn order information. We calculate pe(·) in the same way as Vaswani et al. (2017). H(L) = [c1, . . . , ct] is the output of the last layer with ct denoting the final contextualized representation of the t-th turn. Notice that layer normalization (Ba et al., 2016) is omitted in the formulae above for simplicity.

heading: 3.1.3 Satisfaction Estimation
text: After acquiring the contextual representation ct, we can readily compute the probability distribution of user satisfaction at turn t by applying an MLP network (Rumelhart et al., 1986) with softmax normalization to ct, as shown below:
pUSEt = softmax(MLP(ct)), (5)
where pUSEt ∈ RK , and K is the number of satisfaction classes. The class with the highest probability is selected as the prediction.

heading: 3.2 Hawkes Process Integration

heading: 3.2.1 Preliminaries on Hawkes Process
text: The Hawkes process is a self-exciting point process. It models the self-excitation of events having the same type and the mutual excitation of events with different types in an additive way. A Hawkes process is characterized by its conditional intensity function, which is defined as:
λ(t) = µ(t) + ∑
ti:ti<t
ψ(t− ti). (6)
Here, ti denotes the occurrence time of a past event, µ(t) > 0 is the background intensity or base intensity, and ψ(·) ≥ 0 is a pre-specified triggering kernel function. Typically, ψ(·) is chosen to be a time-decaying function (e.g., the exponential function exp(−t)), indicating that the impacts of past events on the current event decrease through time.
While being able to model the influence of past events, the formulation in Eq. (6) is too simple to capture the complicated dynamics of many real-life event sequences. For example, it assumes that each of the past events has a positive effect on the occurrence of the current event, which can be unrealistic in numerous complex scenarios. To improve its capability, neural Hawkes process models have been devised (Mei and Eisner, 2017; Xiao et al., 2017). These models generalize the standard Hawkes process by parameterizing its intensity function with recurrent neural networks (RNNs) such as LSTM (Hochreiter and Schmidhuber, 1996). More concretely, the new intensity function is calculated in the following way:
λ(t) = M∑
m=1
λm(t) = M∑
m=1
fm(w T mxt), (7)
where M is the total number of event types, xt is the hidden state of the event sequence, and wm is a parameter vector that converts xt to a scalar. fm(·) is the softplus function with a “softness” parameter βm, i.e., fm(y) = βm log(1 + exp(y/βm)). It guarantees that the intensity λ(t) is always positive. In addition to the stronger expressiveness, this formulation of the intensity function has another advantage in that the probability of each event type m can be simply calculated as λm(t)/λ(t).
The RNNs-based Hawkes process models inherit the intrinsic weaknesses of RNNs. Inspired by the superiority of Transformers over RNNs in dealing with sequential data, several Transformer Hawkes
process models have been proposed recently (Zuo et al., 2020; Zhang et al., 2020; Zhou et al., 2022). For these models, one representative definition of the type-specific intensity function λm(t) takes the form (Zuo et al., 2020):
λm(t) = fm ( αm
t− ti ti +wTmxti + bm
) . (8)
In Eq. (8), bm represents the base intensity and αm is introduced to modulate the importance of time interpolation. This interpolation enables λm(t) to be continuous over time. The overall intensity function λ(t) is still defined as λ(t) = ∑M m=1 λm(t).

heading: 3.2.2 Adapting Hawkes Process for Satisfaction Estimation
text: Intuitively, the user satisfaction scores across turns within a dialogue can be regarded as an event sequence and each score corresponds to one type of event. Therefore, it is a natural fit to adopt Hawkes process to model the dynamics of user satisfaction. However, it is infeasible to apply the standard Hawkes process or its neural variants mentioned above directly. This is because these Hawkes processes are continuous in time, i.e., the domain of their intensity function λ(t) is the interval (0, T ]. A continuous Hawkes process models both what the next event type will be and when the next event will happen. By comparison, the satisfaction score sequence in our case is discrete in time. We only need to predict the next event type (i.e., the satisfaction score) and there is no need to predict when it will happen as we estimate user satisfaction at every turn. This difference inspires us to design a discrete version of the Hawkes process.
It is worth emphasizing that one satisfaction prediction is supposed to be made at every dialogue turn, meaning that one event regardless of its type will certainly happen at each turn. To achieve this, we constrain the intensity function λ(t) to always take the value 1. Furthermore, following Eq. (7), λ(t) is decomposed into:
λ(t) =
K∑
k=1
λk(t) = 1, t ∈ {1, 2, . . . , T},
s.t. λk(t) > 0, ∀k = 1, 2, . . . ,K. (9)
Recall that K represents the number of satisfaction classes. Due to λ(t) = 1, λk(t) can be regarded as the probability that event type k happens (i.e., the satisfaction score is k). In Eq. (9), λ(t) is defined
on the discrete domain {1, 2, . . . , T} rather than the continuous interval (0, T ].
We propose to calculate each λk(t) by the following formula:
λk(t) = exp
( fk(MLPk(ct) + MLPk(xt)) ) ∑K
j=1 exp ( fj(MLPj(ct) + MLPj(xt)) ) ,
(10) where the term associated with ct characterizes the contribution of the dialogue context Xt to the intensity (i.e., base intensity) and the term corresponding to xt reveals the contribution of the satisfaction sequence. Different from Eqs. (7) and (8), we perform non-linear rather than linear transformations to convert both ct and xt into scalars using MLP networks. Note that fk(·) is the softplus function.
Next, we describe how to compute xt, the hidden state of the satisfaction score sequence. Given the strong capability of Transformer Hawkes process models, we choose to employ a Transformer architecture (named score-level encoder) to compute xt. In particular, we adopt a unidirectional Transformer with N layers. Same as the turn-level encoder (refer to §3.1.2), each layer contains two sub-layers, the multi-head attention sub-layer and the position-wise feed-forward sub-layer.
The input to its first layer is the satisfaction score sequence. To convert this sequence into vector representations, we introduce an embedding matrix Z ∈ Rd×K whose k-th column is a d-dimensional embedding for satisfaction class k. In principle, if we have the ground-truth score st for turn t, we can calculate the embedding vector of this turn as Zest , where est is the one-hot encoding of score st. In practice, however, we need to predict the satisfaction scores for all turns. Let ŝt be the predicted score of turn t and Zeŝt the corresponding embedding vector. Then, we can feed [Zeŝ1 , . . . ,Zeŝt ] to the score-level encoder to learn the dynamics of user satisfaction up to turn t and to obtain xt. This approach, albeit straightforward, has a severe limitation that there is no feedback from the score-level encoder to help train the base model because the gradients from the score-level encoder cannot be back-propagated to the base model. To overcome this limitation, we take the probability distribution of satisfaction classes pUSEt , as shown in Eq. (5), as the predicted “soft” score. Then, the embedding vector of turn t is computed by:
vt = Zp USE t . (11)
It can be seen that vt is a weighted sum of the em-
beddings of all satisfaction scores and the weights are the predicted probability by the base model.
Based on vt, the score-level encoder functions as follows to yield xt:
V (0) = [v1 + pe(1), . . . ,vt + pe(t)], (12) V ∗ = MultiHead(V (n),V (n),V (n)), (13)
V (n+1) = FFN(V ∗ + V (n)) + V ∗ + V (n). (14)
Similar to the turn-level encoder, we add positional encodings into the input of the first layer V (0) to retain the temporal information. The output of the last layer is symbolized as V (N) = [x1, . . . ,xt].

heading: 3.3 Training Objective
text: We employ the cross-entropy loss as our training objective. Recall that λk(t) represents the probability of the satisfaction score being k at turn t. Thus, the training objective of USE is defined as:
LUSE = −log p(st|Xt) = −log λst(t), (15)
where st is the ground-truth satisfaction label. As stated in §2, adding UAR as an auxiliary task has the potential to help us train a more powerful satisfaction estimator. Even though the proposed Transformer Hawkes process model is expected to improve the performance of USE significantly, it is still meaningful to study if adding this auxiliary task can further improve the performance. To this end, we leverage an MLP network with softmax normalization on top of the turn-level encoder to calculate the probability distribution of user action when the ground-truth labels are provided:
pUARt = softmax(MLP(ct)). (16)
Let pUARt,at be the probability corresponding to the ground-truth action label at at turn t. The training objective of UAR is then defined as:
LUAR = −log p(at|Xt) = −log pUARt,at . (17)
We jointly optimize USE and UAR by minimizing the following loss:
Ljoint = LUSE + γLUAR. (18)
Here, γ is a hyper-parameter that controls the contribution of the UAR task.

heading: 4 Experimental Setup
text: In what follows, we detail the experimental setup.

heading: 4.1 Datasets & Evaluation Metrics
text: We conduct our experiments on four publicly available dialogue datasets, including MultiWOZ 2.1 (MWOZ) (Eric et al., 2020), Schema Guided Dialogue (SGD) (Rastogi et al., 2020), JDDC (Chen et al., 2020), and Recommendation Dialogues (ReDial) (Li et al., 2018). In particular, we perform evaluations on the subsets of these datasets with user satisfaction annotations, which are provided on a five-point scale by Sun et al. (2021). Following existing works (Deng et al., 2022; Pan et al., 2022), the satisfaction annotations are mapped into three-class labels {dissatisfied, neutral, satisfied}. MWOZ, SGD, and ReDial are in English and all contain 1000 dialogues. While JDDC is a Chinese dataset and has 3300 dialogues. Except for ReDial, all the other three datasets have user action labels. The number of action types in MWOZ, SGD, and JDDC is 21, 12, and 236, respectively. For more details about these datasets, refer to Sun et al. (2021).
Following previous studies (Cai and Chen, 2020; Song et al., 2019; Choi et al., 2019; Deng et al., 2022), we use Accuracy (Acc) and Macro-averaged Precision (P), Recall (R), and F1 score (F1) as the evaluation metrics in our experiments.

heading: 4.2 Baseline Methods
text: We compare our proposed method ASAP with several state-of-the-art baseline methods in both singletask learning and multi-task learning settings.1
In the single-task learning setting, we only consider the USE task and the selected baselines are:
HiGRU (Jiao et al., 2019), which utilizes a hierarchical GRU structure (Cho et al., 2014) to encode the dialogue context.
HAN (Yang et al., 2016), which adds a two-level attention mechanism to HiGRU.
BERT (Devlin et al., 2019), which concatenates all the utterances in the dialogue context as a flat sequence. In addition, long sequences with more than 512 tokens are truncated automatically.
USDA (Deng et al., 2022), which leverages a hierarchical Transformer architecture to encode the dialogue context.
In the multi-task learning setting, we consider both the USE task and UAR task. And we compare ASAP to the following baseline methods:
1The implementation of ASAP is available at https:// github.com/smartyfh/ASAP.
JointDAS (Cerisara et al., 2018), which jointly performs UAR and sentiment classification. We replace sentiment classification with the USE task.
Co-GAT (Qin et al., 2021), which leverages graph attention networks (Veličković et al., 2017) to perform UAR and sentiment classification. We also replace sentiment classification with the USE task.
JointUSE (Bodigutla et al., 2020), which adopts LSTM (Hochreiter and Schmidhuber, 1996) for learning temporal dependencies across turns.
USDA (Deng et al., 2022), which uses CRF (Lafferty et al., 2001) to model the sequential dynamics of user actions to facilitate USE.
Our method ASAP is closely related to USDA. The main difference is that USDA focuses on modeling user action dynamics while ASAP focuses on modeling user satisfaction dynamics. Given that user action labels may not be available in practice, our method is more applicable.

heading: 5 Experimental Results

heading: 5.1 Baseline Comparison
text: Single-Task Learning. The results of single-task learning are summarized in Table 1 and Table 3. It
can be observed that our proposed method ASAP consistently outperforms all baseline methods on all datasets. Notably, ASAP shows substantially higher performance than USDA over all four metrics even though USDA conducts in-domain pretraining to strengthen its capability of representation learning. For example, ASAP achieves 9.6%, 3.9%, 4.3%, and 6.9% F1 score improvements on MWOZ, SGD, JDDC, and ReDial, respectively. Multi-Task Learning. The results of USE in the multi-task learning setting are reported in Table 2. For Co-GAT and JointUSE, we include results when the BERT model is leveraged. It can also be observed that the performance of ASAP is consistently higher than all baseline methods over all four metrics. For example, when compared to USDA, we observe that ASAP achieves 7.7%, 3.8%, and
3.2% absolute point improvements in terms of F1 score on MWOZ, SGD, and JDDC, respectively. Single-Task Learning vs. Multi-Task Learning. From Tables 1 and 2, we can find that ASAP tends to perform better in the multi-task learning setting on MWOZ and SGD. This indicates that adding UAR as an auxiliary task is beneficial for improving performance. However, it is worth noting that the performance gain is relatively low. To be specific, the improvements of F1 score on MWOZ and SGD are merely 0.6% and 0.5%, respectively. Besides, on the JDDC dataset, ASAP even performs worse in the multi-task learning setting due to the large number (i.e., 236) of action types. The strong performance of ASAP in the single-task learning setting verifies the significance of modeling user satisfaction dynamics, especially considering that it is costly to collect user action labels.
In summary, our proposed method ASAP is able to outperform baseline methods in both the singletask learning setting and multi-task learning setting. Most importantly, it can achieve highly competitive performance in the single-task learning setting.

heading: 5.2 Effectiveness of Hawkes Process Integration
text: The above results have demonstrated the effectiveness of our method ASAP as a whole. However, it is unclear how much the Hawkes process integration module (i.e., the satisfaction dynamics modeling module) contributes to the overall performance. To better understand the effectiveness of this module, we conduct an ablation study where we compare the performance of ASAP with that of the base satisfaction estimator (refer to §3.1). Recall that the base estimator leverages only the dialogue context for USE. The results on SGD and ReDial are shown in Figure 3. For SGD, we report the results of both single-task learning and multi-task learning. From Figure 3, it can be observed that ASAP consistently outperforms the base estimator over
all four metrics on both datasets. This observation validates the effectiveness of the Hawkes process integration module.

heading: 5.3 Contribution of Satisfaction Sequence to Intensity Function
text: As shown in Eq. (10), the dialogue context and satisfaction sequence both contribute to the intensity function of the Hawkes process. Here, we explore how much contribution should be attributed to the satisfaction sequence. This study is a supplement to the analysis in the previous section and can provide more insights into the effectiveness of satisfaction dynamics modeling. Considering that the softplus function is monotonically increasing, we can measure the importance of the satisfaction sequence by the value exp(MLPst(xt))/(exp(MLPst(xt)) + exp(MLPst(ct))). The larger this value is, the more the satisfaction sequence contributes. We calculate this value for all samples in the test set and employ a box plot to show the distribution of these values. The detailed results are provided in Figure 4, where the triangle marker indicates the mean value. We see that the importance of the satisfaction sequence depends on the dataset. For MWOZ and SGD, the dialogue context tends to contribute more than the satisfaction sequence. In contrast, for JDDC and ReDial, the satisfaction sequence tends to be more important. Despite the variance across datasets, we can conclude that the satisfaction sequence generally plays a critical role.

heading: 5.4 Performance over Dialogue Turn
text: Given that longer dialogues tend to be more challenging, we further investigate the relationship between the depth of dialogue and the performance of our method. Specifically, we study how the performance changes over dialogue turn. The results of ASAP on ReDial are illustrated in Figure 5, where we also report the results of the base estimator for comparison. We omit the results of the first three
turns because of their short dialogue context. From Figure 5, it can be seen that ASAP outperforms the base estimator in most turns, which again verifies the effectiveness of the Hawkes process integration module. However, we observe that the performance of ASAP and the base estimator degrades when the dialogue is deep. Nonetheless, the performance of ASAP is more robust to the increase of dialogue depth, which should be attributed to the modeling of user satisfaction dynamics.

heading: 5.5 Effects of Parameter γ
text: Figure 6 shows the impacts of the parameter γ on the performance of our method in the multi-task learning setting. Note that γ is used to adjust the weight of the UAR task. From Figure 6, we observe that when γ takes small values, the performance is relatively stable. However, the performance drops drastically when γ becomes large. This is because when γ takes large values, the training objective is dominated by the UAR task. As a consequence, our method fails to optimize the satisfaction estimator.

heading: 6 Related Work
text: We briefly review related work on user satisfaction estimation and Hawkes process. User Satisfaction Estimation. Evaluation is crucial for the development of dialogue systems (Sun et al., 2021). However, evaluating a dialogue system comprehensively can prove to be challenging due to the lack of a clear definition of what constitutes a high-quality dialogue (Deriu et al., 2021). Typically, a user study is carried out to collect feedback from end users. However, human evaluation is costly and time-intensive.
Another line of approaches is to perform evaluation from the language point of view. The main objective is to measure how natural and syntactically and semantically correct the system responses are (Kachuee et al., 2021). For example, several machine translation metrics such as BLEU (Pap-
ineni et al., 2002) and ROUGE (Lin, 2004) can be used to measure if system responses are consistent with a set of provided answers. These approaches, albeit efficient, suffer from misalignment with human judgment (Novikova et al., 2017).
More recently, user satisfaction estimation has been proposed as an alternative (Liang et al., 2021; Bodigutla et al., 2019; Sun et al., 2021; Deng et al., 2022; Pan et al., 2022). It leverages human annotations regarding turn-level satisfaction to train an estimator. The estimator is then utilized to perform automatic evaluation by simulating users. Due to this, the evaluation quality depends heavily on the performance of the estimator. In the literature, different approaches have been proposed to train robust estimators (Jiang et al., 2015; Choi et al., 2019; Park et al., 2020; Deriu et al., 2021; Deng et al., 2022). However, none of them considered satisfaction dynamics, which we have shown is a severe deficiency in fully simulating users. Hawkes Process. Hawkes process (Hawkes, 2018) is a self-exciting process and has been widely used to model sequential data (Salehi et al., 2019). To enhance the capacity of the standard Hawkes process, several RNNs-based and Transformer-based variants have been proposed (Xiao et al., 2017; Zhang et al., 2020; Zuo et al., 2020). All these Hawkes processes are continuous over time. There are also studies on discrete Hawkes processes (Seol, 2015; Browning et al., 2021). However, these discrete versions still predict when the next event happens.

heading: 7 Conclusion
text: In this paper, we proposed a new estimator ASAP that adopts the Hawkes process to efficiently capture user satisfaction dynamics across turns within a dialogue. Specifically, we devised a discrete version of the continuous Hawkes process to adapt it to the USE task and implemented this discrete version with a Transformer architecture. Extensive experiments on four benchmark datasets demonstrated
the superiority of ASAP over baseline USE methods and the effectiveness of the Hawkes process module in modeling user satisfaction dynamics.
Limitations
Although our proposed method ASAP is able to outperform baseline estimators, an important factor it ignores is the subjectivity of user satisfaction. In practice, different users may have different degrees of satisfaction with the same dialogue. This implies that ASAP may be effective for some users, but it may also fail to predict true satisfaction for others. In order to adequately simulate a user, it is essential to take the issue of subjectivity into account. Given this, we would like to extend ASAP for personalized satisfaction estimation by incorporating user profile information in the future.

heading: Acknowledgements
text: This work was funded by the EPSRC Fellowship titled “Task Based Information Retrieval” (grant reference number EP/P024289/1) and the Alan Turing Institute.

heading: B Performance of User Action Recognition
text: Recall that in the multi-task learning setting, our method ASAP is trained to predict user satisfaction and user action simultaneously. We have presented the results on USE. In this part, we further investigate the performance on the UAR task. The results on MWOZ, SGD, and JDDC are summarized in Table 4, from which we can see that while ASAP slightly underperforms USDA on MWOZ and SGD according to the official USDA results, its performance is on par with that of USDA based on our reproduced results. Compared to other baselines, ASAP consistently achieves better results on both MWOZ and SGD. However, on the JDDC dataset, we find that the performance of ASAP is relatively low. This is because we have used a small value of 0.1 for γ on this dataset. Because of this, during the training phase, ASAP is mainly optimized for the USE task rather than the UAR task. It is worth emphasizing that our focus is on improving the performance of USE instead of UAR in this work. Thus, the reported UAR results are based on the checkpoints which achieve the best USE performance. These checkpoints may not fully demonstrate the capabilities of ASAP on the UAR task. In fact, we empirically found that by setting γ to larger values, ASAP can achieve much higher performance on action recognition. But this sacrifices the performance on satisfaction estimation.

heading: C Effects of Number of Layers N in the Score-Level Encoder
text: Given that the score-level encoder (i.e., the Transformer Hawkes process module) consists of N layers, it is worth studying the impacts of N on performance by varying its value. For this purpose, we conduct another experiment on the SGD dataset and choose the value of N from {2, 4, 6, 8, 10, 12}. We carry out this experiment in both the single-task
learning setting and the multi-task learning setting. The results are shown in Figure 7. It can be observed that although different values of N lead to different results, the performance is relatively stable. Even so, the performance tends to be higher when N takes smaller values. When N is larger, it is harder to optimize the model because there are more parameters. Additionally, the model is also more prone to overfitting the data.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Limitations
7 A2. Did you discuss any potential risks of your work? Except for the limitations, we do not yet find any other risks the proposed model would have.

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Abstract and Section 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Sections 3, 4, and Appendix A

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 4 and Appendix A
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? The datasets we used are publicly available. As for our implementation, we will be happy to share the code upon acceptance and will create a license accordingly.
7 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? The datasets we used are publicly available and do not contain a license. For our contribution, we will add a license upon acceptance.
7 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? The data we used do not have any sensitive information.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 4.1
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 4.1
C 3 Did you run computational experiments? Section 5 and Appendices B and C

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Appendix A
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Appendix A
7 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? We use a fixed random seed in our experiments.

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Section 4.1 and Appendix A
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Given this, we would like to extend ASAP for personalized satisfaction estimation by incorporating user profile information in the future