MISCA: A Joint Model for Multiple Intent Detection and Slot Filling with Intent-Slot Co-Attention
Authors: Thinh Pham
Tran Chi
Dat Quoc Nguyen
Venue: EMNLP 2023 Findings (2023)
Keywords: Multiple Intent Detection
Slot Filling
Intent-Slot Co-Attention
Label Attention
TL;DR: MISCA: A Joint Model for Multiple Intent Detection and Slot Filling with Intent-Slot Co-Attention

Abstract: The research study of detecting multiple intents and filling slots is becoming more popular because of its relevance to complicated real-world situations. Recent advanced approaches, which are joint models based on graphs, might still face two potential issues:  (i) the uncertainty introduced by constructing graphs based on preliminary intents and slots, which may transfer intent-slot correlation information to incorrect label node destinations, and (ii) direct incorporation of multiple intent labels for each token w.r.t. token-level intent voting might potentially lead to incorrect slot predictions, thereby hurting the overall performance. To address these two issues, we propose a joint model named MISCA. Our MISCA introduces an intent-slot co-attention mechanism and an underlying layer of label attention mechanism. These mechanisms enable MISCA to effectively capture correlations between intents and slot labels, eliminating the need for graph construction. They also facilitate the transfer of correlation information in both directions: from intents to slots and from slots to intents, through multiple levels of label-specific representations, without relying on token-level intent information.  Experimental results show that MISCA outperforms previous models, achieving new state-of-the-art overall accuracy performances on two benchmark datasets MixATIS and MixSNIPS. This highlights the effectiveness of our attention mechanisms.

MISCA: A Joint Model for Multiple Intent Detection and Slot Filling with Intent-Slot Co-Attention
Thinh Pham and Chi Tran and Dat Quoe Nguyen VinAI Research, Vietnam
(v. thinhphp1, v.chib, v. v.datng9jévinai. iio
Abstract
slot filling. Recent research on this topic, recogniz- ing the high correlation between intents and slots, shows that a joint model can improve overall per- formance by leveraging the inherent dependencies between the two tasks (Louvan and Magnini, 2020; Zhang et al., 2019a; Weld et al., 2022). A number of joint models have been proposed to exploit the correlations between single-intent detection and slot filling tasks, primarily by incorporating atten- tion mechanisms (Goo al., 2018; Li et al., 2018; et al., 2019; Qin et al., 2019; Zhang et al., 2019b; Chen et al., 2019; Dao et al., 2021).
The research study of detecting multiple in- tents and filling slots becoming more popular because of its relevance complicated real- world situations. Recent advanced approaches, which are joint models based on graphs, might still face potential issues: (i) the uncer- tainty introduced by constructing graphs based on preliminary intents and slots, which may transfer intent-slot correlation information to incorrect label node destinations, and (ii) di- rect incorporation of multiple intent labels for each token w.r.t. token-level intent voting might potentially lead to incorrect slot predictions, thereby hurting the overall performance. To address these two issues, we propose a joint model named MISCA. Our MISCA introduces intent-slot co-attention mechanism and an underlying layer of label attention mechanism. These mechanisms enable MISCA to effec- tively capture correlations between intents and slot labels, eliminating the need for graph con- struction. They also facilitate the transfer of correlation information in both directions: from intents to slots and from slots to intents, through multiple levels of label-specific representations, without relying on token-level intent informa- tion. Experimental results show that MISCA outperforms previous models, achieving new state-of-the-at overall accuracy performances on two benchmark datasets MixATIS and MixS- NIPS. This highlights the effectivvness of our attention mechanisms.
However, real-world scenarios, users may of- ten express utterances with multiple intents, as il- lustrated in Figure 1. This poses a challenge for single-intent systems, potentially resulting in poor performance. Recognizing this challenge, Kim et al. (2017) is the first explore the detection of multiple intents in SLU system, followed by Gangadharaiah and Narayanaswamy (2019) who first propose a joint framework for multiple intent detection and slot filling. Qin et al. (2020) and Qin et al. (2021) have further explored the utilization of graph attention network (Veličković et al., 2018) to explicitly model the interaction between predicted intents and slot mentions. Recent state-of-the-art models Co-guiding (Xing and Tsang, 2022a) and Rela-Net (Xing and Tsang, 2022b) further incor- porate the guidance from slot information aid in intent prediction, and design heterogencous graphs to facilitate more effective interactions between intents and slots.
1 Introduction
Spoken language understanding (SLU) is a funda- mental component in various applications, ranging from virtual assistants to chatbots and intelligent systems. In general, SLU involves two tasks: intent detection to classify the intent of user utterances, and slot filling extract useful semantic concepts (Tur and De Mori, 201 1). A common approach to tackling these tasks is through sequence classifica- tion for intent detection and sequence labeling for
We find that two potential issues might still per- sist within these existing multi-intent models: (1) The lack of "gold" graphs that accurately capture the underlying relationships and dependencies be- tween intent and slot labels in an utterance. Previ- ous graph-based methods construct a graph for each utterance by predicting preliminary intents and slots (Qin et al., 2021; Xing and Tsang, 2022a,b). However, utilizing such graphs to update represen-

---

Utteaase
box
thet
Rath
352
lankly
Duchy
Ediy Ear
D-flye wanee
Ricke
Balget
Figure I: An example of utterance with multiple intents and slots.
tations of intents and slots might introduce uncer- tainty as intent-slot correlation information could be transferred to incorrect label node destinations. (2) The direct incorporation of multiple intent la- bels for each word token to facilitate token-level intent voting (Qin et al., 2021; Xing and Tsang, 2022a,b). To be more specific, due to the absence of token-level gold intent labels, the models trained to predict multiple intent labels for each word token the utterance. Utterance-leve intents depend on these token-level intents and require at least half of all tokens to support an intent. In Figure 1, for instance, a minimum of 12 tokens is needed to support the "atis grrrissgrund____ ground_faree" fare" intent, while only 8 tokens (enclosed by the left rectangle) support it. Note that each token within the utterance context is associated with specific intent. Thus, incorporating irrelevant intent representations from each token might potentially lead to incorrect slot predictions, thereby hurting the overall accuracy.
and fine-grat one-grained information about the semantic nuances associated with different intent and slot labels. This, in turn, ultimately helps improve the overall results of intent detection and slot filling. Our contributions are summarized as follows: (I) We introduce novel joint model called MISCA for multiple intent detection and slot filling tasks, which incorporates label attention and intent-slot co-altention mechanisms. (II) MISCA effectively captures correlations between intents and slot la- bels and facilitates the transfer of correlation infor- mation in both the intent-to-slot and slot-to-intent directions through multiple levels of label-specific representations. (III) Experimental results show that our MISCA outperforms previous strong base- lines, achieving new state-of-Ue lfte-offthheart overall accura- cies on two benchmark datasets.
2 Problem Definition and Related Work
Given an input utterance consisting of n word to- kens W1, W2, Wn, the multiple intent detection task a multi-label classification problem that pre- dicts multiple intents of the input utterance. Mean- while, the slot filling task can be viewed as a sequence labeling problem that predicts slot label for each token of the input utterance.
To overcome the above issues, in this paper, we propose new joint model with an intent-slot co- attention mechanism, which we name MISCA, for multi-intent detection and slot filling. Equivalently, our novel co-attention mechanism serves as an ef- fective replacement for the graph-based interac- tion module employed in previous works, climinat- ing the need for explicit graph construction. By enabling seamless intent-to-slot and slot-to-intent information transfer, our co-attention mechanism facilitates the exchange of relevant information be- tween intents and slots. This novel mechanism not only simplifies the model architecture, but also maintains the crucial interactions between intent and slot representations, thereby enhancing the overall performance.
Kim et al. (2017) show the significance of the multiple intents setting in SLU. Gangadharaiah and warayanaswamy (2019) then introduce joint ap- proach for multiple intent detection and slot filling, which models relationships between slots and in- tents via a slot-gated mechanism. However, this slot-gated mechanism represents multiple intents using only one feature vector, and thus incorporat- ing this feature vector to guide slot filling could lead to incorrect slot predictions.
In addition, MISCA also presents a label atten- tion mechanism as underlying layer for the co- attention mechanism. This label attention mecha- nism operates independently of token-level intent information and is designed specifically enhance the extraction of slot label- and intent label-specific representations. By capturing the characteristics of each intent/slot label, the label attention mech- anism helps MISCA obtain deep understanding
To generate fine-grained intents information for slot label prediction, Qin et al. (2020) introduce an adaptive interaction framework based on graph attention networks. However, the autoregressive nature of the framework restricts its ability to use bidirectional information for slot filling. To over- come this limitation, Qin al. (2021) proposes
'Our MISCA implementation is publicly available at: https:///itthb cot./VinnAARessearrr//MSSS.

---

Taek-Spaniffc Fmonner
Intene Docodar
BLLSTTV
Trak-Sturn Stannaa Dlteranod Ensonka
Into--Sirtt Co-Attemiiss
Sloe Docoker
>0
Attentinn
>0
Figure 2: Illustration of the architecture of our joint model MISCA.
global-locally graph interaction network that incor- porates both a global graph to model interactions between intents and slots, and a local graph to cap- ture relationships among slots.
filling through multiple intermediate layers. The output vectors generated by this co-attention com- ponent are used to construct input vectors for the intent and slot decoders which predict multiple in- tents and slot labels, respectively.
More recently, Xing and Tsang (2022a) propose two heterogeneous graphs, namely slot-to-intent and intent-to-slot, establish mutual guidance be- tween the tasks based preliminarily pre- dicted intent and slot labels. Meanwhile, Xing and Tsang (2022b) propose a heterogencous label graph that incorporates statistical dependencies and hier- archies among labels to generate label embeddings. They leverage both the label embeddings and the hidden states from a label-aware inter-dependent decoding mechanism construct decoding pro- cesses between the tasks. See discussion on potential issues of these models in the Introduction. Please find more related work in Section 4.3.
3.1 Utterance encoders
Following previous work (Qin et al., 2020, 2021; Song et al., 2022), we employ task-shared en- coder and a task-specific encoder.
Task-shared encoder: Given an input utterance consisting of word tokens W1, U22 ..., Wn, our sask-shared encoder creates a vector e; to repre- sent the 7th word token by concatenating con- textual word embeddings e; BiLSTM.anl and eçAA, and character-level word embedding eus
BiLSTMakrr;
e; = BiLSTMannl S
e
BiLSTMa my
(1)
3 Our MISCA model
Here, we feed a sequence e of real-valued word embeddings e ewz.... Sw,, into single directional LSTM (BiLSTMaord) layer (Hochreiter and Schmidhuber, 1997) and single self-attention layer (Vaswani et al., 2017) to produce the con- textual feature vectors e, BiLSTMaon and eçA respec- tively. In addition, the character-level word embed- ding e BiLSTMau is derived by applying another sin- gle BiL.STM (BiL the sequence real- valued embedding representations of characters in each word we; as done in Lample et al. (2016).
Figure 2 illustrates the architecture of our MISCA, which consists of four main components: (i) Task- shared and task-specific utterance encoders, (ii) Label attention, (iii) Intent-slot co-attention, and (iv) Intent and slot decoders.
The encoders component aims to generate intent- aware and slot-aware ssse-fpcc feature vectors for intent detection and slot filling, respectively. The label attention component takes these task- specific vectors input and outputs label-specific feature vectors. The intent-slot co-attention com- ponent utilizes the label-specific vectors and the slot-aware task-specific vectors to simultancously learn correlations between intent detection and slot
Task-specific encoder: Our task-specific en- coder passes the sequence vectors eim as input to two different single BiLSTM layers to produce

---

task-specific latent vectors el (1L and eî (e. i) E Rd for intent de- tection and slot filling, respectively. These task- specific vectors are concatenated to formulate task- specific matrices E' and ES as follows:
Here, the jth columns vf from V B4x(L and v from VS. Rd are referred to vector representations of the imput utterance w.r.t. the jth label L' and L respectively.
To capture slot label hierarchy information, at 2 2, taking vj - we compute the probabil-
E' [el, 62 ...e/.e] x
(2)
5.k of the jth slot label at the (k 1)th ity Pj
E = (ef e e E Rdexn
(3)
hierarchy level given the utterance, using a cor- responding weight vector w; S Rd and the sigmoid function. We project the vector p.* of label probabilities p) 5.k-1 using a projection matrix 2 S E IR x|IS&="1 and then concatenate the projected vector output with each slot label-specific vector of the 1tth hierarchy level:
3.2 Label attention
The word tokens in the input utterance might make different contributions to each of the intent and slot labels (Xiao et al., 2019; Song et al., 2022), motivating extraction of label-specific vec- tors representing intent and slot labels. In addi- tion, most previous works show that the slot labels might share some semantics of hierarchical rela- tionships (Weld et al., 2022), e.g. fineeggainee" la- bels toloc. _iay_name, toloc. state_ttteaaae _ame and toloc. country_name can be grouped into a more "coarse-grained" label type toloc. We thus in- troduce a hierarchical label attention mechanism, adapting the attention mechanism from Vu al. (2020), to take such slot label hierarchy informa- tion into extracting the -apei-s vectors.
Pj k sigmoid ww) 5,6 S.k
)
(8)
Dsk [p] S. S,k-1
P2 ... PLAAA
S,k-1
T
(9)
vith + v, @ Z x S
(10)
V.S. [viA V2
S. S.k
(11)
|LA.
The slot label-specific matrix vS. at 2 2 is now updated with more "coarse-grained" label information from the (k- - 1)th hierarchy level.
Formally, our label attention mechanism takes the task-specific matrix (here, El from Equation and ES from Equation 3) as input and computes a label-specific attention weight matrix (here, A' E R x and A S.k IR LS*|xnn at the ttth hierarchy level of slot labels) follows:
3.3 Intent-slot co-attention
Given that intents and slots presented in the same utterance share correlation information (Louvan and Magnini, 2020; Weld et al., 2022), it is in- tuitive to consider modeling interactions between them. For instance, utilizing intent context tors could enhance slot filling, while slot context vectors could improve intent prediction. We thus introduce a novel intent-slot co-attention mecha- nism that extends the parallel co-attention from Lu et al. (2016). Our mechanism allows for simultanc- ous attention to intents and slots through multiple intermediate layers.
A' softmax B' x DD' x E))
(4)
A S, softmax (BS tanh SD
S.k. (5)
where softmax is performed at the row level to make sure that the summation of weights in each row is equal to 1; and B' E IR //////dd D' E xde B. E RRI5. and DS Rdd, in which L' and L S.S are the intent label set and the set of slot label types at the tthh hierarchy level, respectively. Here, A E {1, 2,..,. e} where is the number of hi- erarchy levels of slot labels, and thus I,S. is the set of "fine-grained" slot label types (i.e. all original slot labels in the training data).
Our co-attention mechanism creates matrix S dd whose each column represents a "soft" slot label embedding for each input word token, based on its task saecific feature vector:
S - W Desotmaa(USpp
(12)
After that, label-specific representation matri- ces V and S.k are computed by multiplying the task-specific matrices El and ES with the attention weight matrices A l and A S.A. respectively, as:
where WS Rd. 24x UUS (1)xd
and 2| + 1 is the number of BIO-based slot tag labels (including the "0" label) as we formu- late the slot filling task as a BIO-based sequence labeling problem. Recall that L is the set "fine- -me-ggranedd" slot label types without "B-" and
V E (A!)* SSA 1 = x (AS.

---

"1-" prefixes, not including the "O" label. Here, softmax performed at the column level.
as in Equation 6) and H (computed following Equation 15) to create an intent label-specific ma- trix H c (de d)x[L') where its jth column vector uf E Rd is referred to as the final vector repre- sentation of the input utterance w.r.t. the 7th intent label in L'' Taking v'y, we compute the probability p} of the 5th intent label given the utterance by us- ing corresponding weight vector and the sigmoid function, following Equation 8.
Our mechanism takes a sequence of + in- put feature matrices V V8,1 V 2... VSA (computed as in Equations 6, 7, 11, 12) to perform intent-slot co-attention.
For notation simplification, the input feature ma- trices of our mechanism orderly referred to as Q1. Q2. .... Qe+2, where Q Vi. Q2 VS.1 Qe-1 f and Qe+2 S; and d; x m, is the size of the corresponding matrix Q whose each column is referred to as a label-specific vector: dj = de m1 = IL'; d2 = de m2 = |
We also follow previous works to incorpo- rate an auxiliary task of predicting the num- ber of intents given the input utterance (Chen et al., 2022b; Cheng et al., 2022; Zhu et al., 2023). In particular, ber y of intents for the input utterance y argmax (softmax (V!) WN)).. where WINP, E <Rex[L' and w E Be are weight matrix and vector, respectively, and 2 is the maxi- mum number of gold intents for utterance in the training data. We then select the top INP highest probabilities pJ and consider their corresponding intent labels as the final intent outputs.
d3 = de + dp , m3 = | ...; de+1 = de + dp me = [T, de 12 = dg meia 12
compute the num-
As each intermediate layer's matrix Q has dif- ferent interactions with the previous layer's matrix Q.1 and the next layer's matrix Q4+1, we project Q into two vector spaces to ensure that all label- specific column vectors have the same dimension:
-.Q. ;
W.
(13)
where We and We E Radx are projection weight matrices; and thus Q and Q E Rdxmr.
Our intent detection object loss LID is computed as the sum of the binary cross entropy loss based on the probabilities p/ for multiple intent prediction and the multi-class cross entropy loss for predicting the number INP of intents.
We also compute a bilinear attention between matrices Q and Q; to measure the correla- tion between their corresponding label types:
We formulate the slot filling task as a sequence labeling problem based on the BIO scheme. We concatenate ES (computed as in Equa- tion 3) and He+2 (computed following Equation 16) to create a slot filling-specitn matrix H E IR (de ddxn where its 1thh column vector vy E dR. is referred to as the final vector representation of the 7th input word w.r.t. slot filling. We project each aa into the IR 2 vector space by using a project matrix X E (1)))))) to obtain output vector h = . We then feed the out- put vectors h into linear-chain CRF predictor (Lafferty et al., 2001) for slot label prediction.
Slot decoder:
C, = Q,,.Q
(14)
where X, E Rdd-1xd, and thus C, 18"
Our co-attention mechanism allows the intent- to-slot and slot-to-intent information transfer by computing attentive label-specific representation matrices follows:
[aanh Q C + Q.) if = (15)
H.
tanh C
t1 + Q.) otherwise
(anh (tan(( Q2,1C, Q.), if - (16)
H.
tanh(H, I + Q.) otherwise
A cross-entropy loss LsF is calculated for slot filling during training while the Viterbi algorithm used for inference.
We use H, E Rdx[Ll and Hep2 E dx as computed following Equations 15 and 16 as the matrix outputs representing intents and slot men- tions, respectively.
3.5 Joint training
The final training objective loss L of our model MISCA is a weighted sum of the intent detection loss LiD and the slot filling loss LSF:
3.4 Decoders
Multiple intent decoder: We formulate the mul- tiple intent detection task as a multi-label classi- fication problem. We concatenate y (computed
f = ALID + (1- 2)
(17)

---

4 Experimental setup
perform grid search to select the word embed- ding size E {64, 128] and the loss mixture weight À 0.25, 0.5, 0.75, 0.9).
4.1 Datasets and evaluation metrics
We conduct experiments using the "clean" bench- marks: MiXATIS² (Hemphill et al., 1990; Qin et al., 2020) and MiXSNIPS' (Coucke et al., 2018; Qin et al., 2020). MixATIS contains 13,162, 756 and 828 utterances for training, validation and test, while MiXSNIPS contains 39,776, 2,198, and 2,199 utterances for training, validation and test, respec- tively. We employ evaluation metrics, including the intent accuracy for multiple intent detection, the Fi score for slot filling, and the overall accuracy which represents the percentage of utterances whose both intents and slots are all correctly predicted (reflect- ing real-world scenarios). Overall accuracy thus is referred to as the main metric for comparison.
Following previous works (Qin et al., 2020, 2021; Xing and Tsang, 2022a,b), also exper- iment with another setting of employing a pre- trained language model (PLM). Here, we replace our task-shared encoder with the RoBERTabas model (Liu et al., 2019). That is, e from Equation 1 is now computed as = RoBERTaause (W1m i). For this setting, we perform grid search to find the A initial learning rate E { {1e-6, 5e-6, le-5) and the weight À E {0.1, 0.25, 0.5, 0.75, 0.9).
For both the original (i.e. without PLM) and with-PLM settings, we train for 100 epochs and calculate the overall accuracy on the validation set after each training epoch. We select the model checkpoint that achieves the highest overall accu- racy on the validation set and use it for evaluation on the test set.
4.2 Implementation details
The l value is 1 for MixSNIPS because its slot labels do not share any semantics of hierarchical relationships. On the other hand, in MiXATIS, we construct hierarchy with f = 2 levels, which include "coarse- "coarse-grained" and "fine-grained" slot labels. The "coarse- -raiinedd labels, placed at the first level of the hierarchy, are label type prefixes that are shared by the "fine-grained" slot labels at the second level of the hierarchy (illustrated by the example in the first paragraph in Section 3.2).
4.3 Baselines
For the first setting without PLM, we compare our MISCA against the following strong baselines: (1) AGIF (Qin et al., 2020): an adaptive graph interac- tive framework that facilitates fine-grained intent information transfer for slot prediction; (2) GL- GIN (Qin et al., 2021): a non-autoregressive global- local graph interaction network; (3) SDJN (Chen et al., 2022a): a weakly supervised approach that utilizes multiple instance learning to formulate mul- tiple intent detection, along with self-distillation techniques; (4) GISCo (Song et al., 2022): an inte- gration of global intent-slot co-occurrence across the entire corpus; (5) SSRAN (Cheng et al., 2022): a scope-sensitive model that focuses on the intent scope and utilizes the interaction between the two intent detection and slot filling tasks; (6) Rela-Net (Xing and Tsang, 2022b): a model that exploits label typologies and relations through a heteroge- neous label graph to represent statistical depen- dencies and hierarchies in rich relations; and (7) Co-guiding (Xing and Tsang, 2022a): two-stage graph- framework that enables the two tasks to guide each other using the predicted labels.
In the encoders component, we set the dimen- sionality of the self-attention layer output to 256 for both datasets. For the BiLSTMword- the di- mensionality of the LSTT hidden states is fixed 64 for MiXATIS and 128 for MiXSNIPS, Ad- ditionally, in BiLSTM the LSTM hidden di- mensionality is set to 32, while in BiLSTM and BiLSTMS, it is set to 128 for both datasets (i.e., de is 128 * 2- = 256). In the label attention and intent- slot co-attention components, we set the following dimensional hyperparameters: dg 256, dp 32, d, 128, and d 128.
To optimize L, we utilize the AdamW opti- mizer (Loshchilov and Hutter, 2019) and set its initial learning rate to 1e-3, with a batch size of 32. Following previous work, we randomly ini- tialize the word embeddings and character em- beddings in the encoders component. The size of character vector embeddings is set to 32. We
For the second setting with PLM, compare MISCA with the PL.M-enhacced variant of the models AGIF, GL-GIN, SSRAN, Rela-Net and Co- guiding. We also compare MISCA with the fol- lowing PLM-based models: (1) DGIF (Zhu et al., 2023), which leverages the semantic information
data/MLxATIS_cleen data/MixSNIPS_Clle

---

MiXATIS
MixSNIPS
Model
Intent
Slot
Overall
Intent
Slot
Overall
(Acc.)
(F1)
(Acc.)
(Acc.)
(F1)
(Acc.)
AGIF (Qin et al., 2020)
74.4
86.7
40.8
95.1
94.2
74.2
GL-GIN (Qin et al., 2021)
76.3
88.3
43.5
95.6
94.9
75.4
SDJN (Chen et al., 2022a)
77.1
88.2
44.6
96.5
94.4
75.7
GISCo (Song et al., 2022)
75.0
88.5
48.2
95.5
95.0
75.9
SSRAN (Cheng et al., 2022)
77.9
89.4
48.9
98.4
95.8
77.5
Rela-Net (Xing and Tsang, 2022b)
78.5
90.1
52.2
97.6
94.7
76.1
Co Co-guiding (Xing and Tsang, 2022a)
79.1
89.8
51.3
97.7
95.1
77.5
Our MISCA
76.7
90.5
53.0
97.3
95.2
77.9
Table 1: Obtained results without PLM. The best score is in bold, while the second best score is in underline.
of labels: (2) SLIM (Cai al., 2022), which intro-
Model
MixA
MixS
duces an explicit map of slots to the corresponding intent; (3) UGEN (Wu al., 2022), a Unified Gen-
AGIF + RoBERTTan Thase
50.0
80.7
SLIM (Cai et al., 2022) + BERT
47.6
84.0
erative framework that formulates the joint task as question-answering problem; and (4) TFMN (Chen et al., 2022b), threshold-free intent de- tection approach without using threshold. Here, DGIF, SLIM and TFMN are based on BERT (De- vlin al., 2019; Chen et al., 2019), while UGEN is based on T5 (Raffel al., 2020).
UGEN (Wu et al., 2022) T5
55.3
78.8
DGIF (Zhu et al., 2023) + BERT
50.7
84.3
TFMN (Chen et al., 2022b) BERT
50.2
84.7
GL-GIN RoBERTapas abaac
53.6
82.6
SSRAN + RoBERTa Tabase
54.4
83.1
Rela-Net + RoBERTE Labuse
58.4
83.8
Co-guiding RoBERTaaaae Labuse
57.5
85.3
MISCA RoBERTapase
59.1
86.2
5 Experimental results
Table 2: The overall accuracy with PLM. "MixA" and "MixS" denote MiXATIS and MixSNIPS, respectively.
5.1 Main results
Results without PLM: Table 1 reports the ob- tained results without PLM on the test set, clearly showing that in general, our MISCA outperforms the previous strong baselines, achieving the highest overall accuracies both datasets.
overall accuracy across the two datasets, compared to both Rela-Net and Co-guiding.
We find that MISCA produces a higher improve- ment in overall accuracy on MixATIS compared to MiXSNIPS. One possible reason is that MISCA leverages the hierarchical structure of slot labels in MiXATIS, which is not present in MixSNIPS. For example, semantically similar "fine-grained" slot labels, e.g. fromloc. city city_name, city_name and toloc. cimmy_nae,, might cause ambiguity for some baselines in predicting the correct slot la- bels. However, these ffineggrained labels belong to different "coarse- grarseed-raiedd" types in the slot label hierarchy. Our model could distinguish these "fine- grained" labels at a certain intent-to-slot informa- tion transfer layer (from the intent-slot co-attention in Section 3.3), thus enhancing the performance.
In general, aligning the correct predictions be- tween intent and slot labels is challenging, result- ing in the overall accuracy being much lower than the intent accuracy and the Fi score for slot fill- ing. Compared to the baselines, MISCA achieves better alignment between the two tasks due to effective co-attention mechanism, while maintain- ing competitive intent accuracy and slot filling F! scores (here, MISCA also achieves the highest and second highest Fi scores for slot filling on Mix- ATIS and MiXSNIPS, respectively). Compared to the previous model Rela-Net, our MISCA obtains a 0.8% and 1.8% absolute improvement in overall accuracy on MiXATIS and MixSNIPS, respectively. In addition, MISCA also outperforms the previous model Co-guiding by 1.7% and 0.4% in overall accuracy on AiXATIS and MixSNIPS, respectively. The consistent improvements on both datasets re- sult in a substantial gain of 1. 1.1+% in the average
results with PLM: Following previous works, also report the overall accu- racy with PLM on the test set. Table 2 presents obtained results comparing our MISCA+RoBERTH with various strong baselines. We find that the PLM notably helps improve the performance

---

MiXATIS
MixSNIPS
Models
Intent (Acc.)
Slot (FI)
Overall (Ace.)
Intent (Acc.)
Slot (FI)
Overall (Acc.)
MISCA
76.7
90.5
53.0
97.3
95.2
77.9
w/o "slot" label attention
75.1 ()1.6)
89.3 (J1.2)
49.5 (13.5)
96.3 (11.0)
94.4 (10.8)
73.5 (14.4)
(ii) w/o co-attention
75.3 (11.4)
86.8 (U3.7)
44.0 (19.0)
95.5 (11.8)
94.2 (11.0)
72.7 (15.2)
Table 3: Ablation results.
Right
sit quankiyy
(a)
Umerance
how
rights does
busbank B-aidline name baidiis sane
slades
artines
hove
Lights 0
1
Slot
B-usocasky Juanc
O B-uilanc mane
Faidine sanz
Laaiiin uane
B-sircaall sode
Inscet
atis actine
Iiss quanity
Utterance
how
man
fight does
slada
luve to
busbaakk
and also how many comadin
airinnes
Interorionaal
Lights
use
131 Bevinesft
Stot
0
5
Laiidine,
Forriing
Figure 3: Case study between (a) MISCA and (b) MISCA w/o co-attention. Red color denotes prediction errors.
the baselines well as our MISCA. For example. RoBERTa helps produce an 6% accuracy increase MiXATIS and and an 8% accuracy increase on MiXSNIPS for Rela-Net, Co-guiding and MISCA. Here, MISCA+ROBERTa also consistently outper- forms all baselines, producing new statc-of-the-an overall accuracies on both datasets: 59.1% % on Mix- ATIS and 86.2% on MiXSNIPS.
co-attention mechanism is omitted (i.e. w/o co- attention). Here, the overall accuracy drops 9% on MixATIS and 5.2% MiXSNIPS. Both findings strongly indicate the crucial role of the intent-slot co-attention mechanism in capturing correlations and transferring intent-to-slot and slot-to-intenl in- formation between intent and slot labels, leading to notable improvements in the overall accuracy. Figure 3 showcases case study to demonstrate the effectiveness of our co-attention mechanism. The baseline MISCA w/o co-attention fails to rec- ognize the slot ai airline_name for "alaska airlines" and produces an incorrect intent atis_air sit_sairllll line. However, by implementing the intent-slot co- attention mechanism, MISCA accurately predicts both the intent and slot. leverages information from the slot toloc. city_name to enhance the probability of the intent atis_flight, while uti- lizing intent label-specific vectors to incorporate information about airline_nan __me__ This improve- ment is achievable due the effective co-attention mechanism that simultaneously updates intent and slot information without relying on preliminary re- sults from one task to guide the other task.
5.2 Ablation study
We conduct an ablation study with two ablated models: (i) w/o "slot" label attention This is variant where we remove all the slot label-specific representation matrices V.S. That is, our intent- slot co-attention component now only takes 2 input matrices of Q V and Q2 = S. (ii) w/o co- attention This is a variant where we remove the mechanism component of intent-slot co-attention. That is, without utilizing H1 and H4+2, we only use ES from the task-specific encoder for the slot decoder, and employ V from the label attention component for the multiple intent decoder (i.e. this can be regarded as direct adoption of the prior multiple-label decoding approach (Vu et al., 2020)). For each ablated model, also select the model checkpoint that obtains the highest overall accuracy on the validation set to apply the test.
6 Conclusion
In this paper, we propose novel joint model MISCA for multiple intent detection and slot filling tasks. Our MISCA captures correlations between intents and slot labels and transfers the correlation information in both forward and backward direc- tions through multiple levels of label-specific repre- sentations. Experimental results datasets demonstrate the effectiveness of MISCA, which outperforms previous models in both set- tings: with and without using a pre-trained lan- guage model encoder.
Table 3 presents results obtained for both ab- lated model variants. We find that the model per- forms substantially poorer when it does not use the slot label-specific matrices in the intent-slot co-attention mechanism (i.c. w/o "slot" label atten- tion). In this case, the model only considers corre- lations between intent labels and input word tokens, lacking slot label information necessary capture intent-slot co-occurrences. We also find that the largest decrease is observed when the intent-slot
benchmark

---

Limitations
Jacob Devlin, Ming-Wo Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Un- derstanding. In Proceedings of NAACL, pages 4171- 4186.
It should also be emphasized that our intent-slot co-attention mechanism functions independently of token-level intent information. This mechanism generates |r.|| vectors for multiple intent detection (i.e. multi-label classification). In contrast, the token-level intent decoding strategy bases on to- ken classification, using n vector representations. Recall that L' is the intent label set, and n is the number of input word tokens. Therefore, integrat- ing the token-level intent decoding strategy into the intent-slot co-attention mechanism is not feasible.
Haihong E, Peiqing Niu, Zhongfu Chen, and Meina Song. 2019. A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling. In Proceedings of ACL, pages 5467-5471.
Rashmi Gangadharaiah and Balakrishnan Narayanaswamy. 2019. Joint Multiple Intent Detection and Slot Labeling for Goal-Oriented Dialog. In Proceedings of NAACL, pages 564-569.
Chih-Wen Goo, Guang Gao, Yun-Kai Hsu, Chih-Li Huo, Ssung-Chiell Chen, Keng-Wee Wei Hsu, and Yun- Nung Chen. 2018. Slot-Gated Modeling for Joint Slot Filling and Intent Prediction. In Proceedings of NAACL, pages 753-757.
This work primarily focuses on modeling the interactions between intents and slots using inter- mediate attentive layers. We do not specifically em- phasize leveraging label semantics or the meaning of labels in natural language. However, although MISCA consistently outperforms previous models, its performance may be further enhanced by esti- mating the semantic similarity between words in an utterance and in the labels.
Charles T. Hemphill, John J. Godfrey, and George R. Doddington. 1990. The ATIS Spoken Language Systems Pilot Corpus, In Proceedings of the Third DARPA Speech and Natural Language Workshop. pages 96-101
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Temn Memory, Neural Computation, 98)) 1780.
References
Byeongchang Kim, Seonghan Ryu, and Gary Geunbae Lee. 2017. Two-stage multi-intent detection for spo- ken language understanding. Multimedia Tools and Applications, 76:113772 1377-11139..
Fengyu Cai. WanHao Zhou, Fei Mi, and Boi Faltings. 2022. SLIM: Explicit Slot-Intent Mapping with BERT Joint Multi-Intemt Detection and Slot Fill- ing. In Proceedings of /CASSP, pages 7607-7611.
John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In Proceedings of ICML., page 282-289.
Lisong Chen, Peilin Zhou, and Yuexian Zou. 2022a. Joint multiple intent detection and slot filling via self-distillation. In Proceedings of ICASSP, pages 7612-7616.
Lisung Chen, Nuo Chen, Yuexian Zou, Yong Wang. and Xinzhong Sun. 20226. A Transformer- Threshold-Fre Free Framework for Multi-Intent NLU. In Proceedings of COLING, pages 7187-7192.
Guillaume Lample, Miguel Ballesteros, Sandeep Sub- ramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural Architectures for Named Entity Recognition. In Proceedings of NAACL, pages 260-270.
Qian Chen, Zhu Zhuo, and Wen Wang. 2019. BERT for Joint Intent Classification and Slot Filling, arXiv preprint, arXiv:1902.100 10909.
Changliang Li, Liang Li, and Ji Qi. 2018 A Self- Attentive Model with Gate Mechanism for Spoken Language Understanding. In Proceedings of EMNLP, pages 3824-3833.
Lizhi Cheng, Wenmian Yang, and Weijia Jia. 2022. A Scope Sensitive and Result Attentive Model for Iulti-Intent Spoken Language Understanding. arXiv preprint, arXiv:2211.1222: 12220..
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretrain- ing Approach. arXiv preprint, arXiv: 1907.19799999
Alice Coucke et al. 2018. Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-desige voice interfaces. arXiv preprint arXiv:1805.1001 10190.
Ilya Loshchilov and Frank Hutter, 2019. Decoupled Weight Decay Regularization. Proceedings of ICLR.
Mai Hoang Dao, Thinh Hung Truong, and Dat Quoc Nguyen. 2021. Intent Detection and Slot Filling for Vietnamese. In Proceedings of INTERSPEECH, pages 4702.
Samuel Louvan and Bernardo Magnini. 2020. Recent Neural Methods Slot Filling and Intent Classifica- tion for Task-Oriente Dialogue Systems: Survey. In Proceedings of COLING, pages 480-496.

---

Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. 2016. Hierarchical dneestion-Imaee Co-Attention for Visual Question Answering. In Proceedings of NIPS.
Lin Xiao, Xin Huang, Boli Chen, and Liping Jing. 2019. Label-Specific Document Representation for Multi- Label Text Classification. In Proceedings of EMNLP- IJCNLP, pages 466-475.
Libo Qin, Wanxiang Che, Yangming Li, Haoyang Wen, and Ting Liu. 2019. A Stack- PropaaProngntioon Framework with Token-Level Intent Detection for Spoken Lan- guage Understanding. In Proceedings of EMNLP- JCNLP, pages
Bowen Xing and Ivor Tsang. 2022a. Co-guiding Net: Achieving Mutual Guidances between Multiple In- tent Detection and Slot Filling via Heterogeneous Semantics-Laber Graphs. In Proceedings of EMNLP. pages 159-169.
Libo Qin, Fuxuan Wei, Tianbao Xie, Xiao Xu, Wanx- iang Che, and Ting Liu. 2021. GL-GIN: Fast and Accurate -oo-Auutreereesive Model for Joint Multi- ple Intent Detection and Slot Filling. In Proceedings of ACL-IJCNLP, pages 188-188.
Bowen Xing and Ivor Tsang. 2022b. Group is better than individual: Exploiting Label Topologies and Label Relations for Joint Multiple Intent Detection and Slot Filling. In Proceedings of EMNLP, pages 3964-3975.
Libo Qin, Xiao Xu, Wanxiang Che, and Ting Liu. 2020. AGIF: An Adaptive Graph-Interactive Framework for Joint Multiple Intent Detection and Slot Filling. In Findings of the ACL: EMNLP, pages 1807-1816.
Chenwei Zhang, Yaliang Li, Nan Du Wei Fan, and Philip Yu. 2019a. Joint Slot Filling and Intent Detec- tion via Capsule Neural Networks. In Proceedings of ACL. pages 5259-5267.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. 2020. Exploring the Lim- its of Transfer Learning with a Unified Text-to-Text Transformer. The Journal of Machine Learning Re- search, 21(5)55485-551.
Zhichang Zhang. Zhenwen Zhang, Haoyuan Chen, and Zhiman Zhang. 2019b. Joint Learning Framework With BERT for Spoken Language Understanding. Ieee Access, 7:168849- 168858.
Zhihong Zhu, Weiyuan Xu, Xuxin Cheng, Tengtao Song, and Yuexian Zou. 2023. A Dynamic Graph teractive Framework with Label -emaanti Injection for Spoken Language Understanding. In Proceedings of ICASSP, pages 1-5.
Mengxiao Song, Bowen Yu, Li Quangang, Wang Yubin, Tingwen Liu, and Hongbo Xu. 2022. Enhancing Joint Multiple Intent Detection and Slot Filling with Global Intent-Slot Co-occurrence. In Proceedings of EMNLP, pages 7967-7977.
Gokhan Tur and Renato De Mori. 2011. Spoken Lan- guage Understanding: Systems for Extracting Seman- tic Information from Speech. John Wiley & Sons, Inc.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention you Need. In Proceedings of NIPS.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2018. Graph Attention Networks. In Proceedings of ICLR.
Thanh Vu, Dat Quoc Nguyen, and Anthony Nguyen. 2020. A Label Attention Model for ICD Coding from Clinical Text. In Proceedings of IJCAI-20, pages 3335-3341.
Henry Weld, Xiaoqi Huang, Siqu Long, Josiah Poon, and Soycon Caren Han. 2022. A Survey of Joint Intent Detection and Slot Filling Models in Natural Language Understanding. ACM Computing Surveys, 55(8):1-38.
Yangjun Wu, Han Wang. Dongxiang Zhang, Gang Chen, and Hao Zhang. 2022. Incorporating Instructional Prompts into Unified Generative Framework for Joint Multiple Intent Detection and Slot Filling. In Proceedings of COLING, pages 7203-7208.


Reviews:
[{"review_id": "MlFrvNNAWt", "reviewer": "Reviewer_eHBx", "summary": "", "questions": "1.The two datasets used in the paper have different data sources, and Table 1 uses the overall acc measures from both datasets to find the mean. For what purpose?\n\n2.Is it appropriate to use overall accuracy as the main indicator of comparison when each of the three evaluation indicators has its own focus?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review.\n\n\"1. Figure 2 is too simple, and the details of each module in the proposed framework cannot be seen from the figure, so it is recommended to improve it\"\n\n- We will work on improving the illustration of the overall architecture in an updated version of the paper. We are open to suggestions for improving the figure.\n\n\"2. The issue 2 presented in this paper draws directly on the number of intentions prediction approach used in existing research to help models perform the task of detecting multiple intentions. This approach has been proposed and applied by previous studies to multi-intent spoken language understanding tasks and has been shown to achieve better results. It is not appropriate to consider this issue as the problem addressed in this work.\"\n\n- For example (as enclosed by 2 rectangles in Figure 1), the utterance in Figure 1 has 8 tokens supporting the \"atis_ground_fare\" intent, and has 15 tokens supporting the \"atis_meals\" intent. Previous works attempted to directly incorporate both the \"atis_ground_fare\" and \"atis_meals\" intents for all 23 tokens, and then predict token-level intents. Utterance-level intents are determined by these token-level intents, requiring at least 23 / 2 = 12 tokens to support an intent (i.e. at least half of all tokens). Although archiving good results, that token-level strategy might not be intuitive, as only 8 tokens support the \"atis_ground_fare\" intent.\n- Our approach is designed to be more intuitive and does not require token-level intent information.  We will incorporate the above example in the Introduction section in an updated version of our paper to better illustrate the issue 2.\n\n\"3. This work draws directly on previous intent decoding approach, but does not replace such decoding approach in ablation experiments. Despite the better accuracy achieved by the MISCA framework, the persuasiveness of the framework's superiority is very limited.\"\n\n- Incorporating token-level intent decoding strategy into our main intent-slot co-attention mechanism is not feasible.  Our  intent-slot co-attention mechanism operates independently of token-level intent information and is designed specifically to enhance the extraction of slot label- and intent label-specific representations. As a result, we have not conducted an ablation study involving the integration of token-level intent information into our MISCA framework.\n\n\"4. The similarity between words mentioned in Limitations shows that there is still room to improve the performance of the framework, which is not supported by experiments and corresponding analyses. If the number of pages is limited, this can also be explained in the appendix.\"\n\n- There may be room for performance improvement further; however, this should not diminish the contributions of our paper and model, which have already surpassed all previous state-of-the-art results. As indicated in the policy, the \"Limitations\" discussion should not be used to undermine the contributions of our paper.\n\n\"The two datasets used in the paper have different data sources, and Table 1 uses the overall acc measures from both datasets to find the mean. For what purpose?\"\n\n- We achieve a new state-of-the-art (SOTA) overall accuracy on each dataset, as shown in Table 1. The additional average score of overall accuracies is just to demonstrate the consistent improvement on both datasets, thus validating the robustness of our model across different data sources. It's worth noting that previous strong baseline models such as SSRAN, Rela-Net, and Co-guiding only yield competitive results on a single dataset—either MixATIS or MixSNIPS—but not on both.\n\n\"Is it appropriate to use overall accuracy as the main indicator of comparison when each of the three evaluation indicators has its own focus?\"\n\n- Yes, it's. The overall accuracy represents the percentage of utterances in which both intents and slots are predicted correctly. This aligns well with real-world scenarios where both intents and slots need to be predicted accurately. As a result, the overall accuracy is considered the primary metric for comparison.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The problem-driven writing is very clear and easy to read;\n2. I found the abstract and introduction very exciting, although the subsequent presentation of the model and the experiments do not provide strong evidence for the superiority of the proposed framework.", "reasons_to_reject": "1. Figure 2 is too simple, and the details of each module in the proposed framework cannot be seen from the figure, so it is recommended to improve it;\n2.The issue 2 presented in this paper draws directly on the number of intentions prediction approach used in existing research to help models perform the task of detecting multiple intentions. This approach has been proposed and applied by previous studies to multi-intent spoken language understanding tasks and has been shown to achieve better results. It is not appropriate to consider this issue as the problem addressed in this work;\n3.This work draws directly on previous intent decoding approach, but does not replace such decoding approach in ablation experiments. Despite the better accuracy achieved by the MISCA framework, the persuasiveness of the framework's superiority is very limited;\n4.The similarity between words mentioned in Limitations shows that there is still room to improve the performance of the framework, which is not supported by experiments and corresponding analyses. If the number of pages is limited, this can also be explained in the appendix;\n5. The analysis of the validity of the model is weak and does not make a strong case that the work is a good approach to issues 1 and 2.", "questions_to_authors": "1.The two datasets used in the paper have different data sources, and Table 1 uses the overall acc measures from both datasets to find the mean. For what purpose?\n\n2.Is it appropriate to use overall accuracy as the main indicator of comparison when each of the three evaluation indicators has its own focus?", "paper_topic_and_main_contributions": "Previous research on multi-intent spoken language understanding suffers from two problems:1. Pre-recognition results may lead to erroneous cascading of subsequent interactions, and 2. Decoding of intentions by voting is flawed. This work proposes a novel framework, MISCA, to address these two issues and achieves state-of-the-art results on two publicly available datasets.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "cgLuqW4Jjk", "reviewer": "Reviewer_beYN", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review.\n\n\"1. Limited technical novelty. Compare with the two mentioned papers (Xing and Tsang, 2022a, b), although the previous papers focus on graph-based approaches, the idea, co-attention mechanism, and architecture of this paper are quite similar to the previous.\"\n\n- We do not comprehend the phrase \"quite similar to the previous\" in this comment. Our novel intent-slot co-attention design (equations 12-16) and model architecture differ from prior works which mainly depend on the Graph Attention Network (Xing and Tsang, 2022a, b). Our novel attention design is simple yet impactful, resulting in new state-of-the-art results on both experimental datasets. Both Xing and Tsang (2022a, b) only yield competitive results on a single dataset—either MixATIS or MixSNIPS—but not on both datasets.\n\n\"2. Calculating and presenting the averaged overall accuracies over two datasets (the Avg columns) in table 1 and table 2 seems kind of unfounded.\"\n\n- We have achieved a new state-of-the-art (SOTA) overall accuracy on each dataset, as shown in Table 1. The \"additional\" average score of overall accuracies is included to demonstrate the consistent improvement on both datasets, thereby validating the robustness of our model across different data sources. This averaging column, if not applicable, can be easily removed, so it should not be a reason for rejection.\n\n\"3. In the part of Task-shared encoder, a character-level word embedding has been concatenated to the vector. It might be better to briefly explain the purpose of the concatenation.\"\n\n- The purpose of concatenating character-level embeddings is to handle out-of-vocab tokens, a technique commonly employed in numerous previous systems.\n\n\"4. In the PLM part, It might be better to add an experiment with MISCA + BERT, since several strong baselines have not been applied to RoBERTa. Only one experiment with PLM seems to be quite few.\"\n\n- As shown in Table 2, the \"previous\" SOTA baselines including Rela-Net and Co-guiding \"only\" employ RoBERTa. We follow them to employ RoBERTa as our encoder in our experiments, and show that our model still outperforms both Rela-Net and Co-guiding.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Formulas and process the whole architecture with multiple approaches are well elaborated.\nA considerable improvement in the overall score of the Mixatis dataset.", "reasons_to_reject": "1. Limited technical novelty. Compare with the two mentioned papers (Xing and Tsang, 2022a, b), although the previous papers focus on graph-based approaches, the idea, co-attention mechanism, and architecture of this paper are quite similar to the previous.  \n\n2. Calculating and presenting the averaged overall accuracies over two datasets (the Avg columns) in table 1 and table 2 seems kind of unfounded.\n\n3. In the part of Task-shared encoder, a character-level word embedding has been concatenated to the vector. It might be better to briefly explain the purpose of the concatenation.\n\n4. In the PLM part, It might be better to add an experiment with MISCA + BERT, since several strong baselines have not been applied to RoBERTa. Only one experiment with PLM seems to be quite few. \n. ", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposed a novel architecture to enhance the joint process of multi-intent detection and slot filling tasks while solving the fine-grained label problem. Which adopts special hierarchical label attention and intent-slot co-attention mechanism, as well as BiLSTM and self-attention-based encoders. ", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "N/A", "Typos_Grammar_Style_and_Presentation_Improvements": "N/A"}, "extra_text": {}}, {"review_id": "TckZXQhDGr", "reviewer": "Reviewer_wfy7", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review.\n\n\"(1) The improvement is not significant, especially on MixSNIPS dataset (only 0.2% improvement is obtained)\"\n\n- Our model achieves similar absolute improvements compared to the previous SOTA baselines Rela-Net and Co-guiding, as these baselines achieved against other competitive models. It's important to highlight that our model establishes new SOTA results on both experimental datasets. In contrast, Rela-Net and Co-guiding only deliver competitive results on a single dataset—either MixATIS or MixSNIPS—but not on both datasets.\n\n\"- (2) It seems the label attention only works for dataset with hierarchical labels, which is limited. And the authors should provide evidence that the label attention can truly capture the “coarse-grained” label information.\"\n\n- Our label attention also contributes to achieving the new state-of-the-art (SOTA) results on the MixSNIPS dataset, which lacks label hierarchical information. Please refer to the results presented in Tables 1 and 2, as well as the ablation study outcomes in Table 3. So it is not limited.\n- In lines 521-540 in our paper, we have already presented evidence of the label attention capturing the \"coarse-grained\" label information.\n\n\"(3) The illustration of co-attention mechanism on Figure 2 and Section 3.3 is not clear enough.\"\n- We will work on improving the illustration of the overall architecture in an updated version of the paper. We are open to suggestions for improving the figure 2. \n- Please provide more details: which part of the Section 3.3 you find unclear?", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "(1)\tThe motivation is clear, and experiments are extensive.\n(2)\tThe performance is good (achieving stoa on two benchmarks).", "reasons_to_reject": "(1) The improvement is not significant, especially on MixSNIPS dataset (only 0.2% improvement is obtained)\n(2)\tIt seems the label attention only works for dataset with hierarchical labels, which is limited. And the authors should provide evidence that the label attention can truly capture the “coarse-grained” label information.\n(3)\tThe illustration of co-attention mechanism on Figure 2 and Section 3.3 is not clear enough.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This work explores a new framework for jointly tackle multiple intent detection and slot filling tasks. They find that existing joint models are disturbed by two reasons: (i) the uncertainty posed by constructing graphs based on preliminary intents and slots; (ii) the potential harm to overall accuracy caused by token-level intent voting, thereby limiting the performance. \nTo solve the problem, this work introduces a new joint model with an intent-slot co attention mechanism (MISCA). Specifically, they propose co-attention mechanism to replace the previous graph-based interaction module, which avoids constructing graph based on predicting preliminary intents and slots. Additionally, they propose label attention mechanism as an underlying for capturing the characteristics of each intent/slot label.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposed a novel architecture to enhance the joint process of multi-intent detection and slot filling. Experiments demonstrate that this approach, which is comparatively simpler that related works, achieves state-of-the-art results on both MixATIS and MixSNIPS.

Initial presentation of the work left reviewers unclear as to the significance of the contribution, especially in relation to existing approaches. This was clarified through the rebuttal period. In the final draft, the authors should aim to improve the presentation of the work along the lines indicated by the detailed rebuttal discussion with one of the reviewers.|meta review
Decision: Accept-Findings