Efficient Cross-Task Prompt Tuning for Few-Shot Conversational Emotion Recognition
Authors: Yige Xu
Zhiwei Zeng
Zhiqi Shen
Venue: EMNLP 2023 Findings (2023)
Keywords: Prompt Tuning
Emotion Recognition in Conversation
Large Language Models

Abstract: Emotion Recognition in Conversation (ERC) has been widely studied due to its importance in developing emotion-aware empathetic machines. The rise of pre-trained language models (PLMs) has further pushed the limit of ERC performance. However, most recent works on ERC using PLMs are heavily data-driven, and requires fine-tuning the entire PLMs. To improve both sample and computational efficiency, we propose a derivative-free optimization method called Cross-Task Prompt Tuning (CTPT) for few-shot conversational emotion recognition. Unlike existing methods that learn independent knowledge from individual tasks, CTPT leverages sharable cross-task knowledge by exploiting external knowledge from other source tasks to improve learning performance under the few-shot setting. Moreover, CTPT only needs to optimize a vector under the low intrinsic dimensionality without gradient, which is highly parameter-efficient compared with existing approaches. Experiments on five different contextual conversation datasets demonstrate that our CTPT method has superior results on both few-shot scenarios and zero-shot transfers.

Efficient Cross-Task Prompt Tuning for Few-Shot Conversational Emotion Recognition
Yige Xu Zhiwei Zeng 1,2, Zhiqi Shen
'Jinnt NTU-UBC Research Centre of Excellence in Active Living for the Elderly
Schooo of Computer Science and Engineering
Nanyang Technological University, Singapore
yige0020e sg, {zhiwei zeng, zqshen Jentu. edu.
Abstract
and computational efficiency as they often involve large number of training examples and millions of trainable parameters, which potentially prevents current PLM-based models from achieving their best performance in low-resource scenarios.
Emotion Recognition in Conversation (ERC) has been widely studied due to its impor- lance in developing emotion-aware empathetic machines. The rise of pre-trained language models (PLMs) has further pushed the limit of ERC performance. However, most recent works on ERC using PLMs are heavily data- driven and require fine-tuning the entire PLMs. To improve both sample and computational efficiency, we propose derivative-free opti- mization method called Cross-Task Prompt Tuning (CTPT) for few-shot conversational emotion recognition. Unlike existing methods that learn independent knowledge from indi- vidual tasks, CTPT leverages sharable cross- task knowledge by exploiting external knowl- edge from other source tasks to improve learn- ing performance under the few-shot setting. Morcover, CTPT only needs to optimize vec- tor under the low intrinsic dimensionality with- out gradient, which highly training-efficn efficient compared with existing approaches. Experi- ments on five different contextual conversation datasets demonstrate that our CTPT method has superior results on both few-shot scenarios and zero-shot transfers.
Few-shot learning techniques (Motiian et al., 2017; Wang et al., 2021) hold the promise to im- prove both sample and computation efficiency for deploying PLMs in new scenarios where data can be limited. Recently, prompt tuming (Li and Liang, 2021; Lester et al., 2021), which trains set of discrete continuous prompt embeddings condi- tioned frozen PLM, has shown promising re- sults few-shot learning settings (Gao et al., 2021; Gu et al., 2022; Guo et al., 2022). The prompt can be regarded as a way to retrieve the knowledge already memorized in the PLM. The effectiveness of prompts lies in their capability to adapt to new tasks while preserving the knowledge embedded in PLMs, without causing overfitting issues that arise from full-model fine-tuning (Liu et al., 2021).
However, most recent works on ERC are large- scale data-driven that focus the full dataset set- ting (Lee and Choi, 2021; Song et al., 2022a). Gui- bon al. (2022) firstly explore the few-shot ERC task, but their setting not strictly few-shot, which may lead to variety of examples for each label. For example, their training set contains more than k examples for each label under the k-shot setting,
1 Introduction
Emotion Recognition in Conversation (ERC) de- tects emotion categories (e.g., netural, happiness, sadness) of each utterance in given textual conversation. As pre-trained language models (PLMs) (Devlin et al., 2019; Liu et al., 2019) have brought a huge breakthrough to natural language processing (NLP), PLMs are also increasingly em- ployed by ERC models as encoders to improve recognition performance (Zhong al., 2019; Kim and Vossen, 2021: Chudasama et al., 2022). How- ever, the performance gain from PLMs is often achieved at the exorbitant cost of expensive train- ing and fine-tuning processes. PLM-based ERC models tend to suffer from poor sample efficiency
To this end, we strictly define the ERC task un- der the few-shot setting and propose Cross-Tace Prompt Tuning (CTPT) solution. Existing prompt tuning methods independently learn task-specific knowledge from each task, yet such knowledge is often very limited in the few-shot setting. Our proposed CTPT leverages cross-task knowledge by exploiting external knowledge from other source tasks improve learning performance under the few-shot setting. The cross-task knowledge from other source tasks can be divided into two parts: external task-specific knowledge and emotional

---

knowledge. For external task-specific knowledge, we utilize multi-head attention module (Vaswani et al., 2017) to learn knowledge from source tasks. For emotional knowledge, we combine the same emotion within different textual categories from different tasks and then reformulate the verbalizer that decodes the output to the label distribution.
shot setting, which is sample-efficiett CTPT can also obtain a better experimental result zero-shot transfer, which can be deployed in new scenarios with limited training examples.
2 Related Works
2.1 Emotion Recognition in Conversation
One limitation of prompt tuning is that it in- volves backpropagating the loss through all the Transformer layers of a PLM for every batch even though we freeze the PLM, which can lead to com- putational inefficiency. To further improve the com- putational efficiency of PLM-based ERC models, we optimize vector with intrinsic dimensional- ity (Li et al., 2018) instead of the whole continuous prompt, which reduces the number of parameters from hundreds of thousands to about 1,000. Follow- ing Sun et al. (2022), we use a Covariance Matrix Adaptation Evolution Strategy (CMA-ES) (Hansen and Ostermeier, 2001; Hansen et al., 2003) to opti- mize the parameters, which is derivative-free freee. With the derivative-free optimization, we separate our approach from the PLM and do not require back- propagation for parameter learning.
Early studies on ERC mainly utilized audio-based features (Lee and Narayanan, 2005) or lexicon- based features (Devillers and Vidrascu, 2006). Re- cently, there are a series of deep learning ap- proaches focused on emotion recognition in con- versational videos or multi-turn Tweets (Hazarika et al., 2018; Zahiri and Choi, 2018; Zhong et al., 2019; Ishiwatari et al., 2020). In recent years, PLM has been increasingly applied in ERC mod- els (Lee and Choi, 2021; Shen et al., 2021; Song et al., 2022a). commonality among these prior approuches is their shared approach on the inte- gration of various forms of external knowledge to enhance emotion detection, including knowledge from knowledge base (Zhong et al., 2019), knowl- edge from commonsense (Ghosal et al., 2020; Yi et al., 2022), knowledge from multi-mode (Li al., 2022), and inherent knowledge within PLM (Kim and Vossen, 2021). Unlike existing methods that fo- cus on enriching -ask-fpecific knowledge only, we also explore sharable cross-task knowledge from other source tasks.
Compared with single-task prompt tuning, our proposed CTPT method can utilize external knowl- edge from other tasks to boost the performance of the target task. Experiments under the few- shot scenarios and the zero-shot transfer show that CTPT can obtain a better result. In addition to this, our proposed CTPT is derivative-free, which does not need backpropagation. Compared with derivative-based backpropagation, the experiment result shows that CTPT can obtain comparable re- sults without derivative information.
2.2 Prompt Tuning
Despite the success of GPT-3 (Brown al., 2020) with 175 billion parameters, it has become increas- ingly difficult and expensive to utilize such big lan- guage models. One possible solution to leverage large pre-trained models is parameter-efficient tun- ing methods, such as prompt-tuning (Lester et al.. 2021; Li and Liang, 2021). )). In prompt tuning, down- stream tasks are reformulated as language mod- elling task with the help of a textual prompt. For ex- ample, a classification task that aims to predict the emotion category of a given sentence can be refor- mulated as: "I felt so [MASK]. [X]". Here [X] is the given sentence, [MASK] is the mask token that PLM needs to predict, and "I felt so [MASK]" is the tem- plate of prompting. The aforementioned prompt consists of discrete tokens, which are also known as a hard prompt. There another prompt named soft prompt (Qin and Eisner, 2021), which consists of continuous embeddings. Recently, prompt tun- ing has been proven successful in both few-shot
The main contributions of this paper are summa- rized as follows:
(1) To the best of our knowledge, we are the first to strictly define and tackle the few-shot setting for the ERC task. We propose Cross-Tase Prompt Tuning (CTPT) method that can efficiently learn and utilize cross-task knowledge.
(2) To improve the training efficiency, we use the derivative-free optimization algorithm to optimize the parameter. It skips the backpropagation stage and does not require gradient information.
(3) Our proposed CTPT only needs to opti- mize about 1,000 parameters, which is much more training- efficceent than any other existing PLM- based ERC method.
(4) Our proposed CTPT is trained under the few-

---

scenarios (Gu al., 2022: Vu al., 2022) and zero-shot transfer (Guo et al., 2022).
category for each utterance to maximum match the ground emotion category. In the i-th conver- sation, o = [rr), "/22 :. , s.lug]
Although prompt tuning has brought success in many NLP domains such as text classification (Gao
indicates the j-th utterance, o3.e indicates the k-th token in the j-th utterance, |mr] indicates the sequence length of the j-th utterance, and |x,| is the number of utterances in the i-th conversation.
et al., 2021), question answering ( (Yang et al., 2022), and commonsense reasoning (Liu et al., 2022), Yi et al. (2022) first practising prompt tuning on the ERC task that utilizes learnable continuous prompt model the relationship between contextual in- formation and commonsense knowledge. In this paper, we utilize learnable prompts to model the relationship between emotional categories among different tasks under the few-shot setting.
Dataset under the few-shot setting is subset that under the full dataset setting, The new dataset under the k-shot setting is marked as {(r, y) | e R.y y E Vhk, where X and y indicate the input sequence as well as the emotion category for the new training set. Correspondingly, the predicted emotion category is É. Here k-shot indicates that there are l training examples for each emotion cate- gory. We randomly select the new dataset and keep the same in the following experiments. Similar the full dataset setting, the aims under the few-shot setting is to predict the emotion category è.
2.3 Derivative-Free Optimization
Different from many neural networks that re- quire gradient information for backpropagation, derivative-free optimization (DFO) algorithms aim obtain optimal solutions without derivative infor- mation. Most DFO algorithms (Hansen et al., 2003; Shahriari et al., 2016) are under the sampling-and updating structure, which firstly samples solution x and then optimize the parameters via the function values / (x:). In recent years, DFO algorithms have been applied to many downstream areas, such as automatic machine learning (Snoek et al., 2012), and reinforcement learning (Salimans al., 2017).
Under the few-shot setting, the training set as well as the development set, are sampled randomly from the vanilla dataset of the full dataset setting, while the testing set keeps unchanged. At the be- ginning of the training stage, first randomly select some training examples under the following rules: for each given emotion category (e-g.. netu- ral, happiness, sadness, etc.), we randomly select utterances from the vanilla training set. In other words, we keep the textual conversational content but retain only one emotion category for one con- versation. Therefore, for each training example, the input content remains the conversation content X; [=t .².,. _xxs/l and the ground-truth be- comes y; v/ that j is randomly selected before the training stage.
More recently, Sun al. (2022) proposed a DFO method to optimize continuous prompts without gradient information. In this paper, we further ex- tend the DFO method to optimize not only the con- tinuous prompt but also the parameters for cross- task learning.
3 Methodology
3.1 Problem Definition and Notations
All experiments are conducted on few-shot set- tings (k = 16). For each dataset, we sample the subset of the training set and the development set and keep the testing set unchanged. For a fair com- parison, all baselines and CTPT are trained by the same training set.
In this section, we will briefly define the emotion recognition in conversation (ERC) task and the ERC task under the few-shot setting.
The full dataset setting contains the conversa- tion set X' = {xI, X2,--- }n} with n differ- ent conversations as well as the emotion cate- gory set y (y1,y2, yn). The target is to predict the corresponding emotion category set ع = { ...... e..).
3.2 Overview of the Model
In this section, we will briefly introduce the overview of the whole model. The input of CTPT is textual sequence that contains the conversational context, and the output of CTPT an emotion label. CTPT can be mainly divided into three parts: task- specific prompt tuning (TSPT), cross-task prompt learning (CTPL), and cross-task prompt observa- tion (CTPO). The overall architecture is shown in
More specifically, the input of the task is con- versational content x; = [o] ....... , -fxx|] The output of the task is an emotional category set e; [c], cg. 1- The ground-truth is also an emotional category set y; = je4 v/.--. -....,y/... 3/xx,1 The target of the task to predict the emotional

---

Derivative-Free Optimization
Output
((alaa
Label Scura Bippiiess
Emetion Mapping
p(a
Word Nazz
Language Model
Input
Predict
Joy
OK
Ibat
ureat
Neutral
Alleniin
Cross- Task Prompt
Task-Specific Prompt Tuning
Project
W
W
pd
Prompts
P
V
PC
Cross- Task Prompt Learning
Figure I: Overall architecture of our proposed CTPT model.
Figure 1.
the target task, named task-specific prompt tuning (TSPT). Similar to the existing prompt tuning meth- ods, we use soft prompt (Qin and Eisner, 2021) as the template, which can be formulated as:
The first part of CTPT is TSPT. In this part, we learn task-specific knowledge from different source tasks which is harnessed later to the target task" Then, we have CTPL that employs an attention module learn the external task-specific knowl- edge Iearned by TSPT from source tasks and the emotional knowledge from commonsense. Lastly, we have CTPO that utilizes gate-like mecha- nism summarize pertinent cross-task knowledge learned by CTPL. summary, we have "TSPT + CTPL + CTPO = CTPT".
p(yl>) - ((PLM()()))) P(x) = concat [p::)]
(1)
(2)
where concat|::1 - is the concatenation, PLM(-) in- dicates a pre-trained language model, P(-) is the pattern projective function that converts the input sequence into phrased sequence with [MASK] token. Here v(.) ) is the verbalizer injective function that decodes the label by the predicted distribution of the [MASK] token, which can be formulated
We concatenate the prompt with summarized cross-task knowledge p' as well as the input se- quence x, and then pass it into the PLM. After obtain the logits of the [MASK] token from PLM, we first decode the logits word distribution, then map the word distribution to the emotion label, which is the output the whole model.
vhh) = {p(([MASK] = hh]b E v.).
(3)
where h PLM(P(xr) is the hidden states out- puted from a PL.M, V/ the verbalizer for tar- get task i, and g(-) is a function trasnsforming the probability of v to the probability of the label. Here different task has different verbalizer.
In the derivative-free optimization, learnable pa- rameters are contained a vector z with intrinsic di- mensionality. The parameters in the neural network are computed by a linear projection from the vector We the cross-entropy function compute the loss between logits and the ground label and then use Covariance Matrix Adaptation Evolu- tion Strategy (CMA-ES) (Hansen and Ostermeicr, 2001; Hansen et al., 2003) to optimize z.
The soft prompt able matrix, and the objective function is:
Radd in Eq (2) is a learn-
pd = arg min E(),E), pERox
(4)
where L is the cross-entropy loss function, ) and & are defined in Section 3.1, and is the number of prompt tokens. The soft prompt can be regarded as a task-specific embedding that contains latent knowledge from the specific task.
3.3 Task-Specific Prompt Tuning
To address the computational efficiency problem, prompt tuning (Li and Liang, 2021; Lester al., 2021) is a promising solution. Before CTPT, use prompt tuning methods to learn knowledge for
3.4 Cross-Task Prompt Learning
With TSPT, we obtain a task specifc prompt p for the i-th task, which contains the task-fpecific
Source tasks indicate tasks exclude target task "Target task indicutes the task for evaluation.

---

knowledge of the target task. The independently learned lask-specific knowledge usually limited under the few-shot setting. One promising solution to address this problem is to introduce abundant sharable knowledge from other source tasks. The sharable knowledge includes external task task-specific knowledge from source tasks and prior emotional knowledge learned from commonsense.
collected from commonsense in addition exter- nal task-specific knowledge. Across different ERC datasets, varying labels might be used to denote identical emotional states. For example, DailyDi- alog uses "happiness" while MELD uses "joy" to represent the state of being happy. Given this under- standing, we can learn cross-task emotional knowl- edge from disparate tasks encompassing the same emotional state, notwithstanding the divergence in emotion labels, by modifying the verbalizer. There- fore, Eq (3) can be modified as:
External Task-Specific Knowledge Since task- specific knowledge is often very limited in the few- shot setting, we introduce external task-specific knowledge from other source tasks. As afore- mentioned, the external task-specific knowledge is stored in the prompt learned by TSPT. There- fore, we modify the Equation (2) for the i-th task as follows:
D = =uulVe E v,i== 1,2, ,n), (8)
h: V + V, v vh) =9 (P([MASK v oib)lu EV):
where h is mapping function that maps from different task-specific verbalizers to union verbal- izer, and n is the number of tasks. With the new verbalizer, the model can learn knowledge from source tasks under the same emotion.
P(x) concatif(p pi); x],
(5)
where Pº indicates the cross-task prompt for the i-th task, and f()) indicates the combination of task-specific prompt and cross-task prompt.
3.5 Cross-Task Prompt Observation
Inspired by the success of the attention mecha- nism, we utilize a multi-head attention module to decide what kind of knowledge should be collected from the source tasks. In multi-head attention, the query term is the task spec-ipe prompt from the tar- get task. The key term, as well as the value term, are the task-specifie prompt from each source task. The whole module is formulated as:
In cross-task prompt learning, we oblain a prompt P that contains the external task-specific knowl- edge collected from other source tasks and emo- tional knowledge collected from commonsense. Similarly to us, Asai et al. (2022) also utilizes an input-attention module to combine multiple source prompts. However, Asai et al. (2022) only consid- how to learn prompts from source tasks. We empirically notice that part of the learned knowl- edge is beneficial the target task while the other part is useless. To address this problem, we propose an extra stage: cross-task prompt observation.
D
Σ
MHA(pI p/),
(6)
3571
QKT
MHA MHA p/)
Σ head
softmax
Vd
In the cross-task prompt observation stage, more knowledge from the source task will be observed if it is helpful to improve the validation perfor- mance of the target task, while less in contrast. Formulatedly,
where MHA(;)) indicates the multi-head attention, d indicates the dimension of hidden state. Here Q, K, V are:
optimize vector g gate- like controller via the derivative-free optimization mentioned in Section 3.6. Thus, the final prompt becomes:
Q Wopi p
(7)
K = W p/:
V V p/-
p' f(P =
gi) P
In this module, WQ, WK, and VV are learn- able parameters projected by a learnable vector z (More details about optimization are shown in Scction 3.6).
where & indicates the token-level element-wise multiple, I is an one vector, and gi are learnable parameters for i-th task learned by the following objective function:
Emotional Knowledge In order to facilitate the ERC task, we also introduce emotional knowledge
g = arg min C((p(ul)) E &))
(10)
g:ez

---

3.6 Derivative-Free Optimization
4 Experiments
According to Li et al. (2018), the intrinsic dimen- sionality is the minimum number of parameters needed to obtain comparable results. Sun et al. (2022) also shows the efficiency of derivative-free optimization for intrinsic dimensionality vector in prompt tuning. To improve the computational effi ciency, instead of the derivative- backpropa- gation, we utilize Covariance Matrix Adaptation Evolution Strategy (CMA-ES) (Hansen and Oster- meier, 2001; Hansen et al., 2003) to optimize a vector with intrinsic dimensionality. In each opti- mization step, the optimizer will first sample some solutions of the learnable vector z. Then we can calculate the loss of each solution that is used by the optimizer to suggest new
4.1 Datasets
We conduct experiments on five widely-used pub- lic datasets to show the efficiency of CTPT, in- cluding: EC (Chatterjee et al., 2019), DailyDia- log (Li et al., 2017), MELD (Poria et al., 2019), EmoryNLP (Zahiri and Choi, 2018), and IEMO- CAP (Busso et al., 2008). Detailed statisties are shown in Table 1. Though some of the datasets are multi-modality,
only utilize the textual in- formation as the input for a fair comparison with baselines.
4.2 Baselines
For a comprehensive performance evaluation, we select the following four baselines for comparison:
To adapt our proposed CTPT, we modify the optimization step as follows:
(Zhong et al., 2019) The KET is a knowledge-enriched transformer model specifically designed for Emotion Recognition in Conversation (ERC). It employs knowledge base to infuse exter- nal knowledge, which is representative baseline model before the PLM decade.
KET
Task-Specific Prompt Tuning In this step, follow Sun et al. (2022) to compute the task-
specific prompt De by learnable vector z:
Az + Po:
(11)
where A is randomly initialized and fixed, and Po is the initialized prompts from most widely-used tokens.
(Lee and Choi, 2021) The TUCORE-GCN model is turn-context aware graph convolutional network designed for ERC. It incorporates both PLM encoder and graph convolutional network, making an exemplary rep- resentation of --adeddd baseline models.
TUCORE-GCN
Cross-Task Prompt Learning In this step, optimize vector z' instead of the parameters in the multi-head attention module. Similar the optimization of TSPT, we firstly project z' to the parameters space and then separate the parameter space by:
EmotionFlow (Song et al., 2022b) The Emo- tionFlow is PLM-based model with additional CRF layer to capture the emotion transition proba- bility among different utterances.
W- concat (WQ, WK W"),
(12)
SPCL
(Song et al., 2022a) The SPCL a PLM- based model using supervised prototypical con- trastive learning loss, focusing primarily on im- balanced classification problems. It has achieved state-of-the-and results on MELD and Emory
where W A'z' that A' is also randomly initial- ized and fixed. Then, we reshape the parameters and add a randomly initialized and fixed term:
we_ we + W.O.
(13)
4.3 Implementation Details
W W + w.k
In this paper, we use a soft prompt extended to the input and freeze the PLM. We use CMA- ES (Hansen and Ostermeier, 2001; Hansen et al., 2003) algorithm to optimize the parameters. We choose T5 (Raffel et al., 2020) our backbone model. All few-shot settings share the same train- ing and development set with = 16 following the settings of fhw-shot prompt tuning (Gao et al., 2021; Sun et al., 2022).
WV = V + w.y
Cross-Task Prompt Observation we optimize the vector z" in the same way as the vector z being optimized in TSPT:
this step,
g A + go.
(14)
where A and go are fixed.

---

Dataset
Domain
Emotions
# Conv.
Utter.
BC (Chatterjee al., 2019)
Tweet
4
30,160/2.755/55500 755/5,509
DailyDialog (li al., 2017)
Daily Chat
7
11,1 11/1 h0011000/1,000
MELD (Poria al., 2019)
TV Show Scripts
7
1,038/1 14/280
9,989/1 I, 109/2,610
EmoryNLP (Zahiri and Choi, 2018)
TV Show Scripts
7
659/89/79
1/551/954/984
JEMOCAP (Busso et al., 2008)
Daily Chat
6
100/20/31
4,758/ 1213/1,000 1,00/1,622
Table I: Statisties of five ERC datasets. a/b/c indicates the number of examples the training set, development set, and testing set, respectively.
Following Sun et al. (2022) and Lester et al. (2021), we use the soft prompt template and only train the continuous prompts extended to the in- put texts while freezing the PLM parameters. We utilize a Covariance Matrix Adaptation Evolution Strategy (CMA-ES) (Hansen and Ostermeier, 2001; Hansen al., 2003) to optimize the parameters without gradient information.
0.32
0.4
mize-FF
0.28
0.26
JEMOCAP
We use the soft template extended to the input sequence with length of 50. The last token of the template set as <unk> so that the model can better predict the masked token. Since the task is reformulated as generalization task with the text- to-text format, we choose T5 (Raffel et al., 2020) our backbone model.
Number of Extrenal Souncr Thoks
Figure 2: Impacts of removing external source tasks for MELD and IEMOCAP.
with TSPT. Specifically, in addition EC that TSPT has already obtained high performance under the few-shot setting, CTPT brings signif- icant improvement compared with TSPT, which demonstrates the effectiveness of utilizing cross- task knowledge. Since DailyDialog and MELD share the same emotion labels, CTPT obtains the most gain on these two datasets, which shows that our cross-task prompt tuning learn emotional knowledge from the labels.
4.4 Evaluation Metric
For EC and DailyDialog, due to the imbalance dis- tribution of categories (more than 80% examples are neutral emotion), we use micro-averaged Fi score excluding neutral category following Chat- terjee et al. (2019). For the rest three datasets, following Majumder et al. (2019), we use weighted macro-Fi Fi score. The overall evaluation setting is the same as Zhong et al. (2019).
5.2 Model Analysis
To gain deeper insights into CTPT from diverse angles, we undertake analytical experiments in a few-shot setting, where k = 16, in this section.
5 Results and A Analysis
5.1 Main Results
We compare the performance of CTPT against the baselines aforementioned in Section 4.2. We first re-implement the baseline models and achieve the similar performance reported in the original paper. Then we modify the preprocessing code to all baselines be trained under the same few-shot set- ting. The result under the few-shot setting is shown in Table 2.
Analysis in Training Stage Since the training stage of CTPT is different from other approaches, is worthwhile exploring the training stage.
First, to prevent the validation performance degradation brought by DFO algorithms, we train CTPT with backpropagation methods. As shown in Table 2, CTPT optimized by DFO algorithms (CTPT w/o. BP) has comparable result with that optimized by backpropagation methods (CTPT w. BP). In some tasks such as MELD, DFO algorithms perform better than backpropagation methods. The experiment result shows that CTPT obtains compa- rable results without derivative information, which can be deployed in non-GPU devices.
Compared with the baseline models, task- specific prompt tuning (TSPT) outperforms fine- tuning PLMS on most tasks under the few-shot setting. Meanwhile, benefiting from the cross-task knowledge, our proposed cross-task prompt tun- ing (CTPT) obtains an improvement compared

---

Model
BC
DailyDialog
MELD
EmoryNLP
IEMOCAP
KET (Zhong al., 2019)
0.1296
0.0909
0.0897
0,1312
0.1646
Baselines
TUCORE-GON (ee and Choi, 2021)
0.1918
0.2029
0.2596
0.1311
0.1527
EmotionFlow (Song et al., 20226)
0,4084
0.3749
0.2934
0,1465
0.1699
SPCL (Song al., 2022a)
0,4269
0.3699
0.2941
0.1499
0.1873
TSPT
0.6274
0,4996
0.2521
0.1613
0,2877
TSPT + CTPL
0.6226
0,5193
0,2732
0.1724
0,2829
Ours
CTPT (w/o. BP)
0.6394
0,5571
0,3212
0,1902
0,3124
CTPT (w. BP)
0.6405
0,5588
0.3128
0.2057
0,3182
Table 2: Performance of different ERC datasets under the few-shot settings (k 16). "TSPT" indicates task-
specific prompt tuning, "CTPT" indicates cross-task prompt tuning. The result of BBC and DailyDialog are micro- averaged Fi, and the result of other datasets are weighted macro-/i. We bolded the best result and underline the second best.
Training Time
GPU Memory Usage
Analysis in Source Data To explore the impact of external source data, we remove some exter- nal source tasks for MELD and IEMOCAP. For fair comparisons, we sample all the possible com- binations of external source tasks 3 and report the average score. As shown in Figure 2, both MELD and IEMOCAP perform better when in- creasing the number of external source tasks. How- ever, though different combinations bring different improvements, the average score improvement is likely linear.
Miero- -i
KET 0.0909
4.5 mins
1.2 GB
EmotionFlow
0.3749
8.7 GB
SPCI.
0.3699
7.2 GB
CTPT
0.5571
2.8
Table 3: Comparison resources requirements on Dai- lyDialog.
DailyDialog
IEMOCAP
TSPT
0.4996
0.2877
CTPT
Analysis in Pipeline Component In this section, examine the influence of various components within CTPT framework. Initially, carried out ablation experiments investigate the effect of integrating emotional knowledge into CTPL. As depicted Table 4, the incorporation of emotional knowledge enhances the performance of the down- stream task, highlighting its importance.
w/o. EK 0.5481 0.5571
0.3076
FK
0..124
Table Ablations of emotional knowledge. Here "EK" indicates "Emotional Knowledge".
Second, to explore the training efficiency of CTPT, we compare CTPT and other baseline mod- els in terms of training resource requirements. Sim- ply, we compare the two main factors: training time and GPU memory usage. All the methods are implemented with Py PoTch and experimented single Tesla V-100 GPU. We keep the batch size as one for fair comparison. The experiment results show that CTPT is training efficiency compared to EmotionFlow and SPCL. As shown in Table 3, CTPT requires less training time less GPU mem- ory than EmotionFlow and SPCL while offering better validation performance.
Further, to understand the contributions of CTPL and CTPO, we performed ablation experiments by omitting these components. Comparing the result "TSPT CTPL" with "CTPT" in Table 2, we can conclude that CTPO important to CTPT since the validation performance will be significantly degraded without CTPO. Though the experiment result shows that CTPL is negative to the validation performance in some tasks like EC and IEMOCAP, adding CTPO will be positive. In summary, while CTPL's cross-task knowledge might not consis- tently enhance validation performance, due the potential inclusion of redundant information, its combination with CTPO proves advantageous.
T5
RoBERTa
TSPT TSPT + CTPL
0.4996
0.4884
0,5193
0,5110
Analysis in Different Backbones As high- lighted in Section 4.3, we selected T5, one of the
CTPT (w/a. BP)
0.5571
0,5239
Table 5: Results of using different backbone models in DailyDialog.
For example, when the numher of external source tasks is two, there are C# 6 possible combinations.

---

Target Task
EC
DailyDialog
MELD
EmoryNLP
IEMOCAP
Source Task
FC
0.5119
0.2438
0.0307
0.1684
DailyDialog
0.5276
0.2400
0.0308
0.2204
MELD EmoryNLP IEMOCAP
0.4579
0.4834
0.0245
0.2313
0.3642
0.1804
0.1315
0.2658
0.3870
0.2192
0.1104
0.0599
Table Performance of zero-shot transfers. The task-specific prompt the target task excluded during the training stage. We bolded the best zero-shot transfer result for each target task.
most emblematic text-to-texi generative language models, as our primary backbone model. To fur- ther validate the generalizability of CTPT, we con- ducted additional experiments using encoder- only backbone language model, RoBERTa.
and vice versa. other words, the more similarity the two tasks have, the better zero-shot transfer per- formance they obtain. In summary, the experiment result shows that CTPT has good generalization ability in zero-shot transfer.
As evidenced in Table when employing RoBERTa as the backbone model, CTPT achieves results that surpass the TSPT baseline, compara- ble to those obtained with T5. This underscores the robust generalizability of our proposed CTPT across various backbone language models. It also suggests the potential for consistent enhancement in downstream task performance with the adoption of even more advanced backhone models.
Conclusion
In this paper, we strictly define the task of the few-shot setting for ERC and propose eross-task prompt tuning (CTPT) method to tackle this prob- lem utilizing the cross-task knowledge. CTPT learns from external task-specific knowledge from other tasks and emotional knowledge from com- monsense and then summarizes the learned cross- task knowledge to improve the validation perfor- mance. We use derivative-free optimization method to optimize the parameters without gradi- information, which skips the backpropagation stage. Experiments on ERC benchmarks show that CTPT can outperform baseline models in the few- shot setting and obtain a surprising result in the zero-shot transfer. In summary, CTPT is training- efficient that includes: (1) sample-efficinncy it is trained by few-shot training examples, and (2) computationalleffiienees it tunes only about 1,000 parameters with derivative-free algorithms that skip the backpropagation.
5.3 Zero-Shot Transfer
In real-world scenarios, annotated training exam- ples are not always available. Therefore, it is worth- while exploring the zero-shot generalization ability of CTPT. In this subsection, we conduct exper- iments under the zero-shot setting that train the prompt by source task and evaluate the target task while excluding the external task-specific prompt from the target task.
As shown in Table 6, CTPT performs surpris- ingly under the zero-shot transfer. It outperforms bascline methods under the few-shot setting EC, DailyDialog and IEMOCAP. Compared with the few-shot result of TSPT, CTPT zero-shot obtains better performance in DailyDialog and sightly degradation in MELD and IEMOCAP.
A Acknowledgements
This research is supported, in part, by the Joint NTU-WeBank Research Centre on Fintech (Award No. NWI-2020-007), Nanyang Technological Uni- versity, Singapore. This research is also sup- ported, in part, by the National Research Founda- tion, Prime Minister's Office, Singapore under its NRF Investigatorship Programme (NRFI Award No. NRF-NRFF05-2019-000 findings and conclusions or recommendations ex- pressed in this material are those of the authors and do not reflect the views of National Research Foundation, Singapore.
Due to the similarity among the first three tasks (EC, DailyDialog, and MELD), the prompt trained by these three tasks can be easily transferred with each other, which almost achieves the result of TSPT under 16-shot. Meanwhile, since the con- versations in EmoryNLP contain fine-grained and more complex emotions, prompts learned from other tasks can hardly be transferred EmoryNLP. Specifically, the results of zero-shot transfer from the first three tasks to the rest two tasks are poor,
Any opinions,

---

Limitations
Amodei. 2020. Language models are few-shot learn ers. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Informa- tion Processing Systems 2020, NeurIPS 2020, De- cember 6-12, 2020, virtual.
Though our proposed CTPT works well in source- limited scenarios, it has two main limitations:
The DFO algorithm we use is under the sampling-andduplating structure so we need to compute the logits of all sampled can- didate solutions to select the most optimal one. Mcanwhile, the convergence speed of DFO algorithms slower than backpropaga- tion. Therefore, CTPT requires more forward passes than seriivatvve-basee methods due to the aforementioned limitations.
Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jean- nette N. Chang. Sungbok Lee, and Shrikanth S. Narayanan. 2008. IEMOCAP: interactive emotional dyadic motion capture database. Lang, Resour. Eval- uation, 42(44)335-335)
Ankush Chatterjee, Umang Gupta, Manoj Kumar Chin- nakotla, Radhakrishnan Srikanth, Michel Galley, and Puneet Agrawal. 2019. Understanding emotions in text using deep learning and big data. Comput. Hum. Behav, 93:309-317.
In this paper, we use T5 as our backbone model. However, many large language models have been proven successful in other scenar- ios. It is worthwhile to explore how to utilize larger language model under source-limited scenarios in future.
Vishal Chudasama, Purbayan Kar, Ashish Gudmalwar, Nirmesh Shah, Pankaj Wasnik, and Naoyuki Onoe. 2022. M2FNet: Multi-modal fusion network for emotion recognition conversation.
IEEE/CVF Conference on Computer Vision and Pattern Recog- nition Workshops, CVPR Workshops 2022, New Or- leans, LA, USA, June 19-20, 2022, pages 4651 4660. IEEE.
Ethics Statement
this paper, we do not involve extra ethical con- siderations:
Laurence Devillers and Laurence Vidraseu. 2006. Real- life emotions detection with lexical and paralinguis- tic cues on human-human call center dialogs. In INTERSPEECH 2006 ICSLP, Ninth International Conference on Spoken Language Processing. Pitts- burgh, PA, USA, September 17-21, 2006. ISCA.
In this paper, we do not release any new data. All datasets we used are either public datasets or licensed for academic usage.
In this paper. the source codes of baselines and other artefacts are open- soun-ed or licensed for academic usage.
Jacob Devlin, Ming-Wei Wei Chang. Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume I (Long and Short Pa- pers), pages 4171 Association for Computa- tional Linguistics.
Our paper does not use demographic or iden- tity characteristics information, and it does not harm anyone.
Tianyu Gao, Adam Fisch. and Danqi Chen. 2021. Making pre-trained language models better few-shot learners. In Proceedings of the 59th Annual Meet- ing of the Association for Computational Linguis- tics and the 11th International Joint Conference on Natural Language Processing, ACIJJCNLP 2021, (Volume I: Long Papers), Virtual Event, August 1-6, 2021, pages 3816-3830. Association for Computa- tional Linguistics.
References
Akari Asai, Mohammadreza Salehi, Matthew E Peters. and Hannaneh Hajishirzi. 2022. Attentional mix- tures of soft prompt tuning for parameter-efficient multi-task knowledge sharing. arXiv preprint arXiv:2205.1 7791..
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu. Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam Mc- Candlish, Alec Radford, Ilya Sutskever, and Dario
Deepanway Ghosal, Navonil Majumder, Alexander F. Gelbukh, Rada Mihalcea, and Soujanya Poria. 2020. COSMIC: commonsense knowledge for emotion identification in conversations.
Findings of the
Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020, volume
EMNLP 2020 of Findings of ACL, pages 2470- 24811. Association for Computational Linguistics.

---

Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang. 2022. PPT: pre-trainn trained prompt tuning for few-shot learning. In Proceedings of the 60th Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Pupers), ACL 2022. Dublin, Ire- land, May 22-27, 2022, pages 8420-8423. Associa- tion for Computational Linguisties.
Chul Min Lee and Shrikanth S. Narayanan. 2005. To- ward detecting emotions in spoken dialogs. /EEE Trans. Speech Audio Process., 13(2):293-333
Brian Lester, Rami AI-Rfou, and Noah Constant. 2021.
The power of scale for parameter-efficien prompt tuming. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNIP 2021, Virtual Event / Punta Cana, Domini- can Republic, 7-11 November, 2021, pages 3045- 3059. Association for Computational Linguistics.
Gaël Guibon, Matthieu Labeau, Luce Lefeuvre, and Chlos Clavel. 2022. Few-shot emotion recogni- tion in conversation with sequential prototypical net- works, Softw: Impacts, 12:100237.
Chunyuan Li, Heerad Farkhoor, Rosanne Liu, and Ja- son Yosinski. 2018. Measuring the intrinsic di- mension of objective landscapes. In 6th Inter- national Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30- May 3. 2018, Conference Track Proceedings. OpenRe- view.net.
Xu Guo, Boyang Li, and Han Yu. 2022. Improving the sample efficiency of prompt tuning with domain adaptation. In Findings of the Association for Com- putational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates. December 7-11, 2022. pages 3523-3537, Association for Computational Linguis- tics.
Xiang Lisa Li and Percy Liang. 2021. Prefix-Tuning Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan- guage Processing, ACL/JJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 4582-4597. Association for Computational Linguistics.
Nikolaus Hansen, Sibylle Müller, and Petros Koumoutsakos. 2003. Reducing the time complex- ity of the derandomized evolution strategy with co- variance matrix adaptation (CMA-ES). Evol. Com- put., 11
Nikolaus Hansen and Andreas Ostermeier. 2001. Com- pletely derandomized self-adaptation in evolution strategies. Evol. Comput.. 9(2):199-195
Yanran Li, Hui Su. Xiaoyu Shen, Wenjie Li, Ziqiang Cao, and Shuzi Niu. 2017. Dailydialog: A manu- ally labelled multi-turn dialogue dataset. In Proceed- ings of the Eighth International Joint Conference on Natural Language Processing, IJCNLP 2017, Taipei, Taiwan, November 27 December 1, 2017 Volume 1: Long Papers, pages 986-9986-9 0666995, Asian Federation of Natural Language Processing.
Devamanyu Hazarika, Soujanya Poria, Amir Zadeh, Faik Cambria, Louis-Philiipe Morency, and Roger Zimmermann. 2.18. Conversational memory net- work for emotion recognition in dyadic dialogue videos. In Proceedings of the 2018 Conference of the North American Chapter of the Associa- tion for Computational Linguisties: Human Lan- guage Technologies, NAACL-HLT 2018, New Or- leans, Louisiana, USA, June 1-6, 2018, Volume I (Long Papers). pages 2122-2132. Association for Computational Linguistics.
Zaijing Li, Fengxiao Tang, Ming Zhao, and Yusen Zhu. 2022. EmoCaps: Emotion capsule based model for conversational emotion recognition. In Findings of the Association for Computational Linguistics: ACI 2022, Dublin, Ireland, May 22-27, 2022, pages 1610-1618. Association for Computational Linguis- tics.
Taichi Ishiwatari, Yuki Yasuda, Taro Miyazaki, and Jun Goto. 2020. Relation-awee -ware graph attention net- works with relational position encodings for emo-
Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Pe- ter West, Ronan Le Bras. Yejin Choi, and Hannaneh Hajishirzi. 2022. Generated knowledge prompting for commonsense reasoning. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers). ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 3154-3169. Association for Computational Linguis-
tion recognition in conversations. In Proceedings of the 2020 Conference Empirical Methods in Nat- ural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pages 7360-7370. Associ- ation for Computational Linguistics.
Taewoon Kim and Piek Vossen. 2021. EmoBERTa: Speaker-aware emotion recognition in conversation with roberta. arXiv preprint 2iv: 12009.
tics.
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang. Hiroaki Hayashi, and Graham Neubig. 2021. Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language processing. arXiv preprint 107.
Bongscok Lee and Yong Suk Choi. 2021. Graph based network with contextualized representations of turns dialogue. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing, EMNLP 2021, Virtual Event/ / Punta Cana, Do- minican Republic, 7-11 November, 2021, pages 443- 455. Association for Computational Linguistics.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis,

---

Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized bert pretraining ap- proach. arXiv preprint arXiv: 1907. / 1692.
2021, The Eleventh Symposium on Educational Ad- vances in Artificial Intelligence, EAAI 2021, Vir- tual Event, February 2-9, 2021, pages 13789-13797. AAAI Press.
Navonil Majumder, Soujanya Poria, Devamanyu Haz- arika, Rada Mihalcea, Alexander F. Gelbukh, and Erik Cambria. 2019. DialogueRNN: An attentive RNN for emotion detection in conversations, In The Thirty-Third AAAI Conference on Artificial Intelli- gence, AAAI 2019, The Thirty-First Innovative Ap- plications of Artificial Intelligence Conference, IAAI 2019, The Ninth AAAI Symposium on Educational Advances in Artificial Intelligence. EAAI 2019, Hon- olulu, Hawaii USA, January 27- February I, 2019. pages 0818-6825, AAAI Press.
Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. 2012. Practical bayesian optimization of machine learning algorithms. In Advances in Neural Infor- mation Processing Systems 25: 26th Annual Con- ference on Neural Information Processing Systems 2012. Proceedings of meeting held December 3- 6, 2012, Lake Tahoe, Nevada, United States, pages 2960-2968.
Xiaohui Song. Longtao Huang, Hui Xue, and Songlin Hu. 2022a. Supervised prototypical contrastive learning for emotion recognition in conversation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5197-5206, Abu Dhabi, United Arab Emirates. As- sociation for Computational Linguistics.
Saeid Motiian, Quinn Jones, Seyed Mehdi Iranmanesh, and Gianfranco Doretto. 2017. Few-shot adversaria domain adaptation. In Advances in Neural Informa- tion Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, De- cember 4-9, 2017, Long Beach, CA, USA, pages 6670-6680
Xiaohui Song, Liangjun Zang. Rong Zhang. Songlin Hu, and Longtao Huang. 2022b. Emotionflow: Cap- ture the dialogue level emotion transitions. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2022, Virtual and Singa- pore, 23-27 May 2022, pages 8542 2546. IEEE.
Soujanya Poria, Devamanyu Hazarika, Navonil Ma- jumder, Gautam Naik, Erik Cambria, and Rada Mi- halcea. 2019. MELD; A multimodal multi-party dataset for emotion recognition in conversations. In Proceedings of the 57th Conference of the Associa- tion for Computational Linguistics, ACI. 2019, Flo- rence, Italy, July 28- August 2, 2019, Volume I: Long Papers, pages 527-536. Association for Com- putational Linguistics.
Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, and Xipeng Qiu. 2022. Black-box tuning for manguagemmodel-as---sevvve Conference on Machine Learning. ICML 2022, 17- 23 July 2022, Baltimore, Maryland, USA, volume 162 of Proceedings of Machine Learning Research. pages 20841-20855. PMLR.
International
Guanghui Qin and Jason Eisner. 2021. Learning how to ask: Querying ims with mixtures of soft prompts. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computa- rional Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pages 5203-5212. Association for Computational Linguis- tics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4- 9, 2017, Long Beach, CA, USA, pages 5998-6008.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with unified text-to-text trans- former. J. Mach. Learn, Rex., 21:140: 1-40:667.
Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou', and Daniel Cer. 2022. SPoT: Better frozen model adaptation through soft prompt transfer. In Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume I: Long Pa- pers), pages 5039-5033 5039, Dublin, Ireland. Associa- tion for Computational Linguistics.
Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever. 2017. Evolution strategies as scalable alternative to reinforcement learning. arXiv preprint 10t.::770300884...
Yaqing Wang, Quamming Yao, James T. Kwok, and Li- onel M. Ni. 2021. Generalizing from a few exam- ples: A survey on few-shot learning. ACM Comput. Surv. 53(3):63:1-6333
Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P. Adams, and Nando de Freitas. 2016. Taking the hu- man out of the loop: A review of bayesian optimiza- tion. Proc. IEEE, 104(1):148-11 -175.
Antoine Yang. Antoine Miech, Josef Sivic, Ivan Laptev, and Cordelia Schmid. 2022. Zero-shot video ques- tion answering via frozen bidirectional language models. arXiv preprint arXiv:2206.08155.
Weizhou Shen, Junqing Chen, Xiaojun Quan, and Zhixian Xie, 2021. Dialogxl: All-in-one xlnet for multi-party conversation emotion recognition. In Thirty-Fifth AAAI Conference Artificial Intelli- gence, AAAI 2021, Thirty- Third Conference on In- novative Applications of Artificial Intelligence, IAAI
Jingjie Yi, Deqing Yang, Siyu Yuan, Caiyan Cao, Zhiyao Zhang, and Yanghua Xiao. 2022. Contex- tual information and commonsense based prompt for

---

emotion recognition in conversation. arXiv preprint arXiv:2207.13254
Sayyed M. Zahiri and Jinho D. Choi. 2018. Emo- tion detection on TV show transcripts with sequence- based convolutional neural networks. In The Work- shops of the The Thirly-Second AAAI Conference on Artificial Intelligence, New Orleans, Louisiana, USA, February 2-7, 2018, volume WS-18 of AAAI Technical Report, pages 44.2.. AAAI Press.
Peixiang Zhong, Di Wang, and Chunyan Miao. 2019. Knowledge-enrichhe transformer for emotion detec- tion textual conversations. In Proceedings of the 2019 Conference on Empirical Methods Nat- ural Language Processing and the 9th International Joint Conference on Natural Language Processing. EMNLP-IJCNE 2019, Hong Kong, China, Novem- ber 3-7, 2019, pages 165-176. Association for Com- putational Linguistics.


Reviews:
[{"review_id": "qjEtMEpkAG", "reviewer": "Reviewer_iGEX", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to read and review our work.  \n\nWe concur that TSPT has already been employed in different NLP approaches. Our main contributions lie in greatly improving the ERC performance in few-shot settings and under low-resource scenarios. We believe these contributions can be important in closing the gap between existing LLM models and real-world applications which are often faced with sample- and resource-limited scenarios.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The usage of emotion-task related specific prompt in emotion task is interesting, it requires much less training resources to turn the pre-trained models into a specific emotion recognition task, and the evaluation results have stated the effectiveness of the proposed approch.", "reasons_to_reject": "The main contribution of the proposed paper is the usage of TSPT on a pretrained LM (T5 indeed) for emotion recognition task. The idea is interesting and result is good, however, the main optimization approach TSPT is not novel: it is already employed in different NLP approaches. ", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposed few-shot conversational emotion recognition approach based on pre-trained language models.\n1, it is training-efficient for its partial refine design.\n2, it employs no BP in refinement.\n3, it has enhanced transferring effcient.\nIn evaluation, the proposed system has achieved close recognition quality comparing with SOTA TSPT even without BP refinement.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "sfaQSlHvps", "reviewer": "Reviewer_ESEW", "summary": "", "questions": "Question A: Why should we engage in few-shot learning for ERC when there is no lack of data and annotation is not particularly challenging?\n\nQuestion B: ERC which solely focuses on cross-task applications seems to have limited significance as it can be viewed as a truncated form of data augmentation. Since datasets have inherent limitations while dialogue data is vast, wouldn't semi-supervised learning be more crucial in comparison to cross-task applications for data augmentation?\n\nQuestion C: Since the performance of this approach is inferior to that of zero-shot chat GPT, what is the significance of this work?\n\n==============================================================================================\n\nI acknowledge the intriguing perspective of performing few-shot learning across tasks. However, if the achieved performance remains significantly lower than that of LLM in zero-shot scenarios, it becomes challenging for me to convince myself to embrace such an approach. For instance, observing a 30% performance gap on the MELD dataset and a 20% gap on the IEMOCAP dataset, it seems that the essence of few-shot learning might be compromised.  \n**Thus, I lean to keep my rating.**", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review and important queries. We address your concerns as follows: \n\n \n\n### Regarding Question A - The Motivation for Studying Few-Shot Setting in ERC: \n\nWe acknowledge that from the standpoint of general NLP research, there is an abundance of well-labelled conversational datasets. However, the relevance of the few-shot setting should not be underestimated, particularly in the context of real-world applications. One illustrative example is the live chat customer service scenario explored in [1]. In such a scenario, it is highly critical to detect customer emotions. However, conversations with the same person often have very limited turns and are also highly domain-specific. The utterances are often very sparse in emotions, and most of the labels associated with utterances are neutral. In addition, many real application data are private and confidential, so it might be hard to collect large labelled conversation corpus for some highly domain-specific tasks. This challenge was also highlighted in [1]. Hence, we believe studying few-shot settings is an important step in translating the current ERC works into more real-world applications. \n\nMost existing studies concentrate on full dataset settings. [1] is among the first ones to investigate the few-shot setting. We go a step further by adopting a more stringent few-shot definition; specifically, we utilize only $k$ training examples per emotional category, whereas [1] employs substantially more than $k$. We compared our CTPT framework with existing ERC techniques under identical few-shot conditions, successfully demonstrating CTPT's effectiveness. \n\n \n\n### Regarding Question B - Regarding the comparison with data augmentation and semi-supervised learning \n\nDifferent from data augmentation methods that usually generate some “virtual data”, our cross-task approach utilizes “real data” from other in-domain tasks. Hereby we do not need to consider the accuracy of the supervised signals. While semi-supervised learning may be a viable alternative, it still requires a large amount of unlabelled data for training. In contrast, our CTPT can deal with situations even when unlabelled data is limited.      \n\n \n\n### Regarding Question C - Comparison of the proposed CTPT and ChatGPT \n\nWe acknowledge the impressive performance of newly released LLMs including ChatGPT and GTP-4. However, it's important to note that our framework operates orthogonally to the backbone LLMs. Our proposed CTPT is a prompt tuning framework that can be applied to various backbone LLMs, including the backbone of ChatGPT. We could not run experiments with ChatGPT backbone as it requires enormous computing capacity. However, we have conducted our experiments on different backbone models, not only T5 but also other language models like RoBERTa. Across these diverse models, our framework consistently yields performance gains relative to their respective baselines. Thus, we have a strong rationale to hypothesize that our framework could also be beneficial when applied to newer LLMs like ChatGPT. \n\nAdditionally, the computational and financial burdens associated with tuning and deploying LLMs like ChatGPT underscore the critical importance of few-shot learning, further motivating our research in this direction. \n\n \n### Others\n\nWe will also revise to improve the figure quality and add significance tests and error analysis. \n\n### Reference\n\n[1] Guibon et al. Few-Shot Emotion Recognition in Conversation with Sequential Prototypical Networks. In EMNLP 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Reasonable method.\n- The proposed method outperforms the baselines.", "reasons_to_reject": "- The abstract looks a little unconvincing. For example, “ To improve both sample and computational efficiency, we propose a derivative-free optimization method called Cross-Task Prompt Tuning (CTPT) for few-shot conversational emotion recognition”. The conversational data is abundant, and few-shot learning may not be an appropriate choice. The author should clearly explain the rationale behind selecting few-shot learning in the paper and provide a more comprehensive justification for this choice.\n- The motivation is rather ambiguous and lacks sufficient theoretical support. For example, “Existing prompt tuning methods independently learn task-specific knowledge from each task, yet such knowledge is often very limited in the few-shot setting ...”. It is recommended to clearly state the research objectives and significance in the introduction. Providing relevant literature and data to support the rationale behind the motivation is essential. \n- Very difficult to follow the contribution of this paper. For example, “To the best of our knowledge, we are the first to strictly define and tackle the few-shot setting for the ERC task. We propose a Cross-Task Prompt Tuning (CTPT) method that can efficiently learn and utilize cross-task knowledge”. The author is not the first to propose few-shot learning in ERC. The author should provide a detailed explanation in the paper for its advantages compared to other methods.\n- The significance of few-shot learning seems limited for the ERC task that has an abundant supply of data and labeling is not particularly challenging.\n- Purely cross-task ERC has very limited utility, as it amounts to a truncated form of data augmentation. Dialogue data is infinite compared to the finite dataset, making semi-supervised approaches more crucial than cross-task augmentation. Additionally, it would be more meaningful to explore ERC research that spans different domains rather than just across tasks.\n- Performance is inferior to that of zero-shot chatGPT.\n- Table 2 and Table 4 need some significance tests to further verify the reliability of the experimental results..\n- The quality of Figure 1 is not very clear. The author should incorporate vector graphics where appropriate to enhance the quality and readability of the paper.\n- There is a lack of error analysis.", "questions_to_authors": "Question A: Why should we engage in few-shot learning for ERC when there is no lack of data and annotation is not particularly challenging?\n\nQuestion B: ERC which solely focuses on cross-task applications seems to have limited significance as it can be viewed as a truncated form of data augmentation. Since datasets have inherent limitations while dialogue data is vast, wouldn't semi-supervised learning be more crucial in comparison to cross-task applications for data augmentation?\n\nQuestion C: Since the performance of this approach is inferior to that of zero-shot chat GPT, what is the significance of this work?\n\n==============================================================================================\n\nI acknowledge the intriguing perspective of performing few-shot learning across tasks. However, if the achieved performance remains significantly lower than that of LLM in zero-shot scenarios, it becomes challenging for me to convince myself to embrace such an approach. For instance, observing a 30% performance gap on the MELD dataset and a 20% gap on the IEMOCAP dataset, it seems that the essence of few-shot learning might be compromised.  \n**Thus, I lean to keep my rating.**", "paper_topic_and_main_contributions": "This paper presents a derivative-free optimization method called Cross-Task Prompt Tuning (CTPT) for few-shot conversational emotion recognition. Unlike existing methods that learn independent knowledge from individual tasks, CTPT leverages sharable cross-task knowledge by exploiting external knowledge from other source tasks to improve learning performance under the few-shot setting. Moreover, CTPT only needs to optimize a vector under the low intrinsic dimensionality without gradient, which is highly training-efficient compared with existing approaches.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "8pBO8C5z6G", "reviewer": "Reviewer_q8Be", "summary": "", "questions": "See Reasons to reject.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and constructive feedback on our manuscript. \n\n \n\n1. On the Significance of Emotional Knowledge versus the Proposed Cross-Task Prompt Tuning Framework (CTPT) in Performance Enhancement: \n\nOur proposed CTPT utilizes cross-task knowledge, which includes both External Task-Specific Knowledge and Emotional Knowledge. We have conducted an ablation study to analyse and compare the performance improvement induced by external task-specific knowledge and emotional knowledge. While both types of knowledge contribute to performance improvement, external task-specific knowledge shows a more substantial impact (see the following Table). These results suggest that emotional knowledge does offer additional performance gains. Nonetheless, even in the absence of emotional knowledge, the CTPT framework itself, using only external task-specific knowledge, significantly enhances performance, indicating its primary role in the observed improvements. \n\n|  Method  |  Performance on DailyDialog  |\n|  ----  | ----  |\n| TSPT  | 0.4996 |\n| CTPT w/o. Emotional Knowledge  | 0.5481 |\n| CTPT w. Emotional Knowledge  | 0.5571 |\n\n\n2. On the Sensitivity of Optimization Results to Hyperparameters $A$ and $p_0$ in Equation (11): \n\nThese two hyperparameters are randomly initialized and kept constant during the optimization. Our experiments with varying random seeds demonstrate that the optimization outcomes are not very sensitive to these parameters.  \n\n \n\n3. On Figure Quality: \n\nThank you for pointing this out. We will enhance the clarity and quality of the figures in the revised manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Derivative-free, and few parameters to optimize.\n2. Extensive convincing results supporting the claimed contributions.", "reasons_to_reject": "1. How significant is the function of emotional knowledge in the model? Is the impressive performance caused mainly by the model framework or this external knowledge?\n\n2. Figure quality can be improved. \n\n3. How sensitive is the optimization result to \"A\" and \"p_0\" in Equation (11)?\n\n", "questions_to_authors": "See Reasons to reject.", "paper_topic_and_main_contributions": "The manuscript proposes a derivative-free optimization approach CTPT to improve the sample and computational efficiency of ERC approaches. The main contributions are the derivative-free approaches, extensive experiment results, and convincing experimental results. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper introduces a few-shot conversational emotion recognition approach using PLMs. The idea is based on a partial refinement design that is training-efficient, avoids backpropagation in refinement, and enhances transferring efficiency. The proposed system achieves high recognition quality, comparable to the existing TSPT method. Additionally, the paper presents Cross-Task Prompt Tuning (CTPT) which is a derivative-free optimization technique for few-shot conversational emotion recognition. CTPT leverages cross-task knowledge, improving learning performance by incorporating external knowledge from other source tasks.  


The majority of the reviewers engaged in post-rebuttal discussions and all agreed that the soundness of this work is mostly sufficient with an acceptable level of excitement and generally reproducible results.
Decision: Accept-Findings