MRRL: Modifying the Reference via Reinforcement Learning for Non-Autoregressive Joint Multiple Intent Detection and Slot Filling
Authors: Xuxin Cheng
Zhihong Zhu
Bowen Cao
Qichen Ye
Yuexian Zou
Venue: EMNLP 2023 Findings (2023)
Keywords: Non-Autoregressive
Intent Detection and Slot Filling
Reinforcement Learning

Abstract: With the rise of non-autoregressive approach, some non-autoregressive models for joint multiple intent detection and slot filling have obtained the promising inference speed. However, most existing SLU models (1) suffer from the multi-modality problem that leads to reference intents and slots may not be suitable for training; (2) lack of alignment between the correct predictions of the two tasks, which extremely limits the overall accuracy. Therefore, in this paper, we propose $\textbf{M}$odifying the $\textbf{R}$eference via $\textbf{R}$einforcement $\textbf{L}$earning (MRRL), a novel method for multiple intent detection and slot filling, which introduces a modifier module and employs reinforcement learning. Specifically, we try to provide the better training target for the non-autoregressive SLU model via modifying the reference based on the output of the non-autoregressive SLU model, and propose a suitability reward to ensure that the output of the modifier module could fit well with the output of the non-autoregressive SLU model and does not deviate too far from the reference. In addition, we also propose a compromise reward to realize a flexible trade-off between the two subtasks. Experiments on two multi-intent datasets and non-autoregressive baselines demonstrate that our MRRL could consistently improve the performance of baselines. More encouragingly, our best variant achieves new state-of-the-art results, outperforming the previous best approach by 3.6 overall accuracy on MixATIS dataset.

MRRL: Modifying the Reference via Reinforcement Learning for Non-Autoregressivee Joint Multiple Intent Detection and Slot Filling
Xuxin Cheng', Zhihong Zhu", Bowen Cao, Qichen Ye, Yuexian Zou'
School of ECE, Peking University, China {chengxx, zhihongzhu, cbw2021) } @ stu.p.u.edu.cn [yeeeqichen, zouyx @ pku.edu.cn
Abstract
classification task and slot filling could be regarded as a sequence labeling task (He and Gamner, 2023a; Cheng et al., 2023a).
With the rise of non-autoregressivve approach, some non-autoregressive models for joint mul- tiple intent detection and slot filling have ob- tained the promising inference speed. However, most existing SLU models (I) suffer from the multi-modality problem that leads to reference intents and slots may not be suitable for train- ing: (2) lack of alignment between the correct predictions of the two tasks, which extremely limits the overall accuracy. Therefore, this paper, propose Modifying the Reference via Reinforcement Learning (MRRL), a novel method for multiple intent detection and slot filling. which introduces modifier module and employs reinforcement learning. Specifically, we try to provide the better training target for the non-autoregressive SLU model via modify- ing the reference based on the output of the non- autoregressive SLU model, and propose suit- ability reward to ensure that the output of the modifier module could fit well with the output of the non- -o-auuoorerrssivvee SLU model and does not deviate too far from the reference. In addi- tion, we also propose compromise reward to realize flexible trade-off between the two sub- tasks. Experiments on two multi-intent datasets and aon-eevererressivve basclines demonstrate that our MRRL could consistently improve the performance of baselines. More encouragingly, best variant achieves new state-of of the-ar art re- sults, outperforming the previous best approach by 3.6 overall accuracy on MiXATIS dataset.
However, an utterance often contains more than a just single intent in the real scenarios (Zhu et al., 2023a; Xing and Tsang, 2023; Cheng et al., 2023f). With this in mind, multi-intent SLU has received more and more attention (Xu and Sarikaya, 2013; Kim et al., 2017; Shet et al., 2019). Gangadharaiah and Narayanaswamy (2019) proposes a multi-task framework to jointly model intent detection and slot filling. Qin al. (2020) introduces graph atten- tion networks (GAT) (Velickovic et al., 2018) to de- velop a fine-grained multi-intent prediction frame- work called AGIF, which aims to incorporate intent information into the slot filling decoding process in adaptive manner. Qin et al. (2021b) proposes novel global-locally graph interaction network GL-GIN, which appplies the non-autoregressive modeling techniques parallelize the decoding process, resulting in significant speedup compared to the traditional autoregressive SLU models. Xing and Tsang (2022a) further designs a two-stage SLU framework and achieves the mutual guidance be- tween intent and slot, which enhances the overall accuracy of SLU. Song et al. (2022) explores atten- tion mechanisms extract the relevant information from the independent utterance contexts and cap- ture shared speei-specific features across all utter- ances in the training set. Xing and Tsang (2022b) proposes ReLa-Net, -et, which utilizes a heterogeneous label graph to represent the statistical dependencies and hierarchies. Cheng et al. (2023d) applies con- trastive learning to explore and leverage the inher- ent relationships in multi-intent SLU. Cheng et al. (2023a) proposes scope-sensitive SLU model SS- RAN to reduce the distraction of the out-of-scope tokens and mitigate the error propagation problem caused by the bidirectional interaction.
1 Introduction
As crucial task in dialogue systems, spoken lan- guage understanding (SLU) aims to understand the user's current goal through constructing semantic frames (Tur and De Mori, 2011; Young et al., 2013; Zhu et al., 2023b; He and Garner, 2023b). Intent detection and slot filling are two common subtasks of SLU, where intent detection is an utterance-level
Equua contribution. Corresponding author.
Though existing non- autoregressive multi-intent SLU models have made the promising progress, we

---

Tokens
Passibility
Reference
Siat(F1) Intent(Acc)
intent.alis, quantiyy zis fliht atic distance, day name
D-fromloc.cityaamme
Bicity_pam pame
guardia
slot:
intent:atis sirport, aiis sirline
R-fromloc.aing airport name
stis distance, aiis day name
Lairport nume
Jowntown
slat:
intent:
B-citt_name
city, sirfare
distance, day name
Table I: The examples of the modti-modality problem. The gold intent label of the utterance is utilized as the intent of each token in the utterance.
find that most of them still face two issues:
(1) Suffer from the multi-modality problem. Although non-autoregressive models have proven the effectiveness in terms of high inference speed, they still suffer from the multi-modalle problem, which has been pointed out in several prior works in other tasks (Ran et al., 2020; Zhang et al., 2022a). However, this serious problem is still ignored in non-autoregressive SLU task. As shown in Table 1, for each token, there may be multiple possible cor- rect slots. Besides, due to the widespread use the teken-level intent detection decoder, this problem also occurs in intent detection. Despite the utiliza- tion of GAT (Velickovic et al., 2018), these models still have little prior knowledge about the reference during the inference progress, which leads some errors not commonly seen in autoregressive SLU models. For example, an I- slot might erroneously appear before its corresponding B- slot the slot sequence output by a non-ootooone -oo--uttoregessive model. As a result, the original reference intents and slots are not very suitable for model training.
20
40
60
80 100
epoch
Figure I: An example of the misalignment between the correct predictions of intent detction and slot filling. As the FI score of slot filling (red) increases, the accuracy of intent detection (green) may decrease, which limits overall accuracy of the utterance.
ing to provide better training target for the non- autoregressive model. Since the reference intents and slots may not be suitable for training, we mod- ify the reference intents and slots according to the output the non- non-autoreegessive SLU model. For the first problem, we propose suitability reward to ensure that the output of the modifier fits well with the output of the non-autoregressive model and still maintain the original information. The first part of suitability reward is related to the training loss the output of the modifier, and the second part is the similarity between the reference and the output of the modifier. For the second problem, we propose a compromise reward to improve the overall accu- racy. We directly utilize the overall accuracy as the compromise reward to achieve a flexible trade-off between the accuracy of intent detection and the Fl score of slot filling. Experiment results show that MRRL consistently improves the performance of baselines on two benchmark datasets MiXATIS and MiXSNIPS (Hemphill al., 1990; Coucke et al., 2018; Qin et al., 2020). Further analysis also veri- fies the advantages of our method.
(2) Lack of the alignment between the cor- rect predictions of the two subtasks. Most of existing SLU models decode the hidden stats of the two subtasks independently without leverag- ing the correlations between them, which leads to the misalignment of the correct predictions of the two subtasks. As shown in Figure I, the score of slot filling and the accuracy of intent detection might increase or decrease asynchronously. Over- all accuracy is an important metric and it denotes the ratio of the utterances for which both intents and slots are predicted correctly. Due the lack of alignment, overall accuracy utterance-level semantic frame parsing much worse than these two subtasks, which not conducive to deploying the SLU model actual scenarios.
The contributions of our work are three-fold:
To the best of our knowledge, our work is the first attempt to employ reinforcement learning to solve the multi-modality problem and the misalignment problem non- non-autoreggrssive multi-intent SLU models.
In this paper, we propose novel model termed MRRL to tackle the above two issues, which intro- duces a modifier and applies reinforcement learn-
We propose a novel method termed MRRL for non-autoregressive multi-intent SLU, which introduces a modifier and applies reinforce-

---

ment learning to modify the reference utter- ance into form suitable for model training. Experimental results demonstrate that MRRL can efficiently improve the performance of the baselines and the best variant achieves the new state-of-the-at results.
The final joint objective is formulated as:
-= == BLs
(4)
where a and B are
2.2 AGIF
2 Background
AGIF (Qin et al., 2020) is token-level adaptive interaction network that implements fine-grained integration of multi-intent information. An intent- slot graph interaction layer is used to model the strong correlation between slots and intents. Such an interaction layer is applied adaptively to each to- ken of an utterance, with the advantage that relevant intent information can be automatically extracted. Restricted by the autoregressive paradigm, during the inference, the previously predicted tokens must be fed the decoder generate the next token step by step, which leads slower inference speed.
this section, we first introduce the problem defi- nition of the multi-intent SLU task and then intro- duce several multi-intent SLU models.
Given an input utterance = (#1, #2, Rn), where n denotes the length of the utterance mul- tiple intent detection is a multi-label classification task and the token-leved predicted intent sequence is denoted as y (,(1,,,, y(2.f") ..... y (n.)") The final utterance-level predicted intent sequence y obtained by the token-level intent voting strat- egy (Qin et al., 2021b; Xing and Tsang, 2022a). The reference intent sequence is denoted as g² (g(1.1), g(..)................... ..... g(m where m denotes the number of intents in æ. Slot filling is sequence labeling task (Qin al., 2022; Cheng et al., 2023e; Zhu et al., 2023c). The predicted slot sequence is denoted as S the reference slot sequence is denoted as ÿ (g(1.5), (((2.5)...... g(n..)). For simplicity, we use ÿ to denote the union of g' and g
2.3 GL-GIN
GL-GIN (Qin et al., 2021b) is a global-locall graph-interaction network, including local slot- aware graph layer and global intent-slot interac- tion layer. Owing the non-autoregressive archi- tecture, GL-GIN achieves generate intents and slots sequence simultaneouslyy, thus increasing the inference speed. However, we find that it is not enough to just rely on the local graph interaction layer to model the slot dependencies, which limits the performance (see Sec.4.5 for more details).
()) and
2.1 Multi-Intent Spoken Language Understanding
Due to the interaction between the two subtasks of multiple intent detection and slot filling, joint mod- els are widely used to consider the two tasks and update parameters. The multiple intent detection objective is defined as:
2.4 Co-guiding Net
Co-guiding Net (Xing and Tsang, 2022a) is a two- stage framework that allows intent detection and slot filling to learn from each other. The first stage produces initial estimated labels for the two tasks and the second stage leverages estimated labels prior label information. Two heterogencous graph attention networks proposed to work on the two aforementioned graphs for modeling the guidance between intent and slot.
CE(0. v) = i log (v) + (1 () log (1 1- v)
(1)
Nr
CE .9. (2))
Li
(2)
where N, denotes the number of single intent la- bels, g(5,0) denotes the reference intent, and y; (i/)) denotes its corresponding predicted intent. Similarly, the slot filling objective defined as:
2.5 ReLa-Net
ReLa-Net (Xing and Tsang, 2022b) improves joint multiple intent detection and slot filling from a new perspective, which exploits the label typologies and relations through heterogeneous label graph and recurrent heterogeneous label matching network. The heterogeneous label graph includes both the global statistical dependencies and slot label hicrar-
G.9) 0
log (1.5))
Ls
(3)
where Nş denotes the number of slot labels, (1.8))
denotes the reference slot, and y, (1.5) denotes its corresponding predicted slot.
chies, which is proposed to represent the statistical

---

Intent Decoder
share
Intent Modifier
MT
y
R
Rfitt
simn
Reoo
Shared
M
Encoder
y
y
RSs fith
R sim
Slot
share
Slot Modifier
y MS
Decoder
Figure 2: The main architecture of MRRL. Two modifiers trained to optimize the suitability reward and the compromise reward, where suitability reward the weighted sum of fit reward and similarity reward. Fit reward relates to the joint training loss of the modifiers and similarity reward relates the similarity with the reference. Compromise reward is related to the overall accuracy.
dependencies and hierarchies in rich relations. And a recurrent heterogeneous label matching network is proposed to end-to-end capture the beneficial in- formation from the heterogeneous label graph and use them for tackling the joint task.
the modifier module has the potential to optimize the reference, making it more aligned with the train- ing objectives and enhancing the overall training process. Thus, we utilize two non-autoregressive decoders as the architecture of the modifier mod- ule. Similarly, the token-level predicted intent se- quence of the intent modifier is denoted MM" (n.M2))) the output of the
3 Approach
(y(LM (22.M intent modifier y A is also obtained by the voting technique. The output of the slot modifier is de- noted as y MS (y ((2, ,.... (S)) and the union of MI and MS is denoted as y We use Pas to denote the probability distribution of the modifier module:
this section, we first present the overview of our model ($3.1), which introduces the modifier mod- ule including intent modifier and a slot modifier into the original architecture. Then, we introduce the pretrain- finittnn-i paradigm ($3.2), where pre- training could establish better initial state for the fine-tuning of the modifier module. Finally, in- troduce the reinforcement learning module, which is applied to make the output the modifier more suitable for model training ($3.3). Figure 2 illus- trates the overview of our proposed method.
MT
MS x. 7
II PM (e (tMM)
(5)
(RMM))
3.1 Model Overview
Diverging from conventional SLU models, the non-autoregressive SLU model is trained with the output of the modifier module y MI and MS rather than the original reference ğ and gs. Then the multiple intent detection objective LM and slot filling objective LMs are formulated as:
To alleviate the problem of multi-modalin and mis- alignment, in addition to the conventional encoder- decoder architecture, introduce modifier mod- ule including intent modifier and slot modifier modify the reference according to the output of the non-autoregressive model. Note that the mod- ifier module only affect the training stage, so the inference cost does not change.
a Nr
LMJ =
CE
(.D)
(LMM))
(6)
The modifier module is designed as key com- ponent capable of transforming the reference into more suitable form for training. The transformation process takes into account both the reference itself and the output generated by the non- autoregressive model. Through leveraging this dual information,
N₂
(15))
log (v.. (...
LMs =
1
(7)
where (j.II) denotes the predicted intent of the in- tent modifier and y, (.M denotes the predicted slot

---

of the slot modifier. Then the final joint objective of the modifier module LM defined as:
Rs; and the final fit reward R
1
R/u.
LMI
(9)
M aL + BLMS
(8)
R fit LMS
(10)
3.2 Pretrain-Finetum Paradigm
R aR + BR
(11)
When directly applying reinforcement learning to train the modifier. a common problem arises where the modifier become trapped in a non-optimal state, resulting in the generation of sequences with mean- ingless intents and slots. The issue is inherent to reinforcement learning due its sensitivity to ini- tial states (Mihatsch and Neuncier, 2002; Fei et al., 2020; Shao et al., 2023).
where m is the length of the output of the intent modifier y and n is the length of the output of
the slot modifier y yMS.
Secondly, the output of the modifier should not deviate too far from the reference, so utilize similarity reward R measure the similarity between the reference and the output of the modi- fier module. We use the accuracy of multiple intent detection and Fl score of slot filling as the simi- larity function to measure the similarity between y1 and y2. which are denoted as (y y2) and Ss(y), 222, respectively. Then the intent similarity reward R sim the slot similarity reward R sim and the final similarity reward Rgim formulated
To address this challenge, we utilize pretrain- finetune paradigm. During the pre- mr--trinnng phase, we employ a joint objective C as defined in E to establish a better initial state for the modifier. This pre-training strategy aims to provide the modifier with a more favorable starting point, making subse- quent fine-tuning easier and reducing the likelihood of falling into local optimization.
Rdim = =Si )
(12)
Following the pre-training phase, we proceed to fine-tune the modifier using reinforcement learning, incorporating suitability reward and compromise reward to further guide the learning process. This line-tuning approach enables the designed modifier to refine its behavior and adjust its outputs based on the reinforcement signals received.
R Sz MS)
(13)
Ruim = R RR sim
(14)
where ÿ' is the reference intent and g is and the reference slot. The suitability reward Ruuit is the weighted sum of Riú and Reim
R = R fa + AgR
(15)
3.3 Reinforcement Learning
where Ay and A are two syyper-parametens
Another common problem in many multi-intent SLU models is that the F1 score slot filling and the accuracy of intent detection might increase or decrease asynchronously. As the F1 score of slot filling increases, the accuracy of intent detection might begin decrease. Overall accuracy denotes the ratio of utterances whose intents and slots are all correctly predicted. As result, is crucial achieve a trade-off between the accuracy of intent detection and the FI score of slot filling.
The primary objective of the modifier module is to generate a training target which is better suited for non-autoregressive models. Motivated by Wu et al. (2018); Rao et al. (2021); )) Lu et al. (2022); Shao et al. (2023), we quantify the requirements for the modifier module into two reward functions and op- timize them via reinforcement learning. The level of appropriateness of this target could be quantified using two reward functions.
Firstly, it is crucial for the output of the modifier module to align closely with the output of the SLU model. We employ a fitting reward Rax that shares a formal resemblance to the loss function LMs aim- ing to incentivize the reduction of the training loss.
Intuitively, we apply the overall accuracy to mea- sure the similarity between 3) and y2. and denote it SA A1(11, U2). We directly apply SA as the com- promise reward Reom to improve the overall accu- racy. The compromise reward Reom is:
For the output of the intent modifier Af and the output of the slot modifier MS we apply the length normalization to keep the scale of reward sta- ble and combine the normalized LMI and LMs to obtain the intent fit reward R/uu- the slot fit reward
Room SA M
(16)
The final reward R. for the modifier module is:
R = Rovit + R.Room
(17)

---

MixATIS
MixSNIPS
Model
Overal((Ac)
SlouFI)
(ncent(Acc)
Overall(Acc)
Slo((F)
Incent(Acc)
Atlention BiRNN (Liu and Lane, 2016)
74.6
59.5
95.4
Slot-Gated (Goo et al., 2018)
35.5
87.7
63.9
55.4
87.9
94.6
Bi-Model (Wang al., 2018h)
34.4
83.9
70.3
63.4
90.7
95.6
SF-ID al., 2019)
34.9
66.2
59.9
95.0
Stack- Mack-Propagation (Qin et al., 2019)
40.1
87.8
72.1
72,9
94.2
96.0
AGIF (Qin 2020)
40.8
86.7
74.4
74.2
94.2
95.1
LR- Transformer (Cheng et al., 2021b,a)
43.3
88.0
76.1
74,9
94.4
96.6
GISCo (Song et al., 2022)
48.2
88.5
75.0
75.9
95.0
95.5
SSRAN (Cheng et al., 2023a)
48.9
77.9
77.5
95.8
98.4
GL-GIN (Qin et al., 2021b)
43.0
88.2
76.3
73.7
94.0
95.7
MRRL
47.2
88.71
78.4
75.8"
95.21
96.51
Co-guiding Net (Xing and Tsang, 2022a)
51.3
89.8
79.1
77.5
95.1
97.7
w/ MRRL
54.8*
90.6
79.5
79.0'
96.41
98.6
ReLa-Net (Xing and Tsang, 2022b)
52.2
90.1
78.5
76.1
94.7
97.6
w/ MRRL
55.87
92.41
79.8
79.3
96.81
99.11
Table 2: Results comparison. denotes our model significantly outperforms baselines with p < 0.01 under t-test.
where de is hyper-parameter. We utilize the RE- INFORCE algorithm (Williams, 1992; Zhang et al., 2021) to optimize the reward R:
and fine-tuning and Ba, 2015) with 31 0.9, 82 0.98, and 4k warm-up updates optimize parameters in our model. For all the experiments, we select the model which works the best on the dev set and then evalu- ate it on the test set. a is set to 0.9, 3 is set to 1, xy is set to 0.3, As is set to 0.2, and X is set to 0.5. All parameters are obtained by the annealing strat- egy (Ahn et al., 2019). Experiments are conducted at GelFrre RTX 2080Ti and TITAN Xp.
Adam optimizer (Kingma
VJ
Pas (y
y MS xr R)
yM
(18)
V log PM A y MS R)
E MM
4 Experiments
4.1
Datasets and Metrics
We conduct our experiments on two public multi- intent datasets cleaned version of MiXATIS and MiXSNIPS (Qin et al., 2020). MiXATIS dataset is collected from ATIS dataset (Hemphill et al., 1990) and MiXSNIPS dataset is collected from SNIPS dataset (Coucke et al., 2018). MiXATIS includes 13,162 utterances for training, 756 utter- ances for validation and 828 utterances for testing. MiXSNIPS includes 39,776 utterances for training, 2, 199 utterances for validation and 2, 199 utterances for testing. Compared to single-domain MiXATIS dataset, MixSNIPS dataset is more complicated be- cause of the intent diversity and large vocabulary. Following Goo et al. (2018) 8); Qin et al. (2021b), we evaluate accuracy (Acc) for multiple intent de- tection, Fl score for slot filling, and overall accu- racy for the utterance-level semantic frame parsing. Overall accuracy denotes the ratio the utterances whose intents and slots are all correctly predicted.
4.3 Main Results
We introduce MRRL many baselines. The results on the test sets is listed in Table 2, from which we have the following observations:
(1) Our MRRL consistently improves the perfor- mance of several basclines on all tasks and datasets. More encouragingly, our best variant (i.e. ReLa- Nel w/ MRRL) achieves new state-of-the-arr results. Specifically speaking, on MiXATIS dataset, it over- passes the previous state-of-the-art model ReLa- Net by 3.6 and 2.3 on overall accuracy and slot filling, and overpasses the previous state-of-the-ard model Co-guiding Net by 0.7 on multiple intent detection; on MiXSNIPS dataset, it overpasses SS- RAN by 1.8, 1.0 and 0.7 on utterance-level seman- tic frame parsing, slot filling and multiple intent detection, respectively. This because our meth- ods provide a better training target for the non- autoregressive model by modifying the reference. which might be not be suitable for training due to the multi-modality problem.
4.2 Implementation Details
We pre-train the model for 5K steps with batch size 16 on each dataset. During both pre-training
(2) It is worth noting that the improvement on the MiXATIS dataset is more obvious than that on
htttps: //tp:hbb..tttttte/e httppp///ggttuub..moooppr//////
cww/LooperXX//CC.C

---

MiXATIS Slot(I)
MixSNIPS
Model
Avee
Intent(Ace)
Overall(Ace)
SlouF])
Intent(Ace)
Rel Ne- W/ MRRI.
55.8
92.4
79.8
79.3
96.8
99.1
w/o Rauit
53.6 (U1.8)
91.3 (U1.1)
78.8 (1T0)
(Q1.2)
(Q1.5)
98.2 (J0.9)
w/o RSs
54.4 (L1.4)
91.6 (L0.8)
79.2 (1006)
78.4 (110.)
(U1.2
98.5 (J0.6)
w/o Rasm
54.2 (L1.6)
91.8 (10.6)
79.4 (10.4)
78.6 (10.7)
95.8 (U1.0)
98.7 (J0.4)
w/o Reom
53.0 (J2.8)
(J0.4)
(10.3)
77.2 (12.1)
96.5 (L0.3)
98.8 (J0.3)
w/o Rauit + More Parameters
53.8 (12.0)
91.4 (L110)
79.2 (J0.6)
78.5 (10.8)
95.7 (L1.1)
98.4 (J0.7)
w/o R.com + More Purameters
53.4 (12.4)
92.2 (J0.2)
79.6
78.8 (10.5)
96.6 (J0.2)
98.9 (J0.2)
Table 3: Results of ablation experiments ReLa-Net on MiXATIS dataset and MixSNIPS dataset.
the MixSNIPS dataset. We suspect that this is be- cause MixSNIPS dataset is more complicated than MixATIS dataset. MixATIS dataset has a smaller vocabulary and fewer kinds of intentions and slots, where is easier to propose a more suitable training target for non-autonn aatt-uttorrresivve multi-intent models. As a result, the gain is greater on AiXATIS dataset. (3) The improvements in terms of overall accu- racy are much sharper. This is because our method includes the compromise reward to overcome the misalignment problem. In this way, the correct pre- dictions of the two tasks can be better aligned. As result, more test samples get the correct utterance- level semantic frame parsing results, and then the overall accuracy is improved more significantly.
worse than ReLa-Net w/ MRRL, which suggests that the improvements come from the proposed suitability reward rather than involved parameters.
4,4.2 Effectiveness of Compromise Reward
To verify the effectiveness of compromise reward, remove it and refer it to w/o Reom- We can find that the performance is decreased all tasks and datasets. Moreover, the drop in overall accu- racy is more pronounced than that of slot Fl and intent accuracy on the two datasets, which suggests that compromise reward can efficiently improve the overall accuracy. It is worth noting that slot Fl and intent accuracy do not drop when compromise reward is introduced. We believe that the reason is that compromise reward can further achieve the mutual guidance between intent and slot indirectly when realizing the flexible trade-off between the two subtasks. Like Sec.4.4.1, we also increase the layers of intent decoder and slot decoder to verify that the compromise reward rather than the added parameters works, which is named as w/o Reom + More Parameters. The result also suggests that the improvements come from the compromise reward.
4.4 Analysis
We conduct a set of ablation experiments on ReLaa Net w/ MRRL to verify the advantages of MRRL from different perspectives, and the experimental results are shown in Table 3.
4.4.1 Effectiveness of Suitability Reward
Suitability reward is one of the key contributions of our MRRL, which is dedicated to solving the problem of unsuitability of the reference caused by the multi-modality problem. To verify this, we remove the suitability reward and refer it to w/o Rusit in Tabel 3. We can clearly observe that over- all accuracy drops by 1.8 on MiXATIS and 1.2 on MiXSNIPS, the slot Fl drops by 1.1 on MiXATIS and 1.5 on MiXSNIPS, and intent accuracy drops by 1.0 on MiXATIS and 0.. on MiXSNIPS. When we only remove a component of Reuit (i.e. w/o Rfit and w/o Rimm) the performance also degrades in varying degrees. Following previous works (Qin et al., 2020, 2021b), to verify that the proposed suitability reward rather than the added parameters works, we increase the layers of intent decoder and slot decoder when Rusit is removed and refer it to w/o Reuit + More Parameters. We could observe that despite the added parameters, it still performs
4.5 Case Study
To further demonstrate how our approach allevi- ates the multi- problem, we provide sev- eral cases generated from GL-GIN, ReLa-Net and ReLa-Net MRRL in Figure 3.
It is obvious that despite the utilization of GAT, GL-GIN is still impacted by multi-modality issue, where B-airport aepoor_nnan raportnammeeeeeeeeeeeeee __nae is incorrectly predicted as B-fromloc. ai airport_nnaeee We believe this is be- cause GL-GIN has little prior knowledge about the reference during the inference progress, so milwau- kee is incorrectly predicted as the departure place when there is destination in the utterance. Com- pared to GL-GIN, ReLa-Net performs a little better, where the predicted intent is right but there are still some mistakes in the predicted slots. When MRRL is introduced to Rela-Net, the prediction is abso-

---

Models
Utferance: what ground transportation is available between
milwaukee
dirport
and
Slot:
B-airport_name
I-dirport_name
Ref.
Intent:
att__ground_eervcee
Slot:
B- Fromiloc..irport__amm I-fromloc.co wat.mmooc..ipprr__ama.
GL-GIN
Intent:
atis_distance
Stot:
B- rromloc.dirport_nomm wwwrooooo.iirpor__noom
ReLa-Net
Intent:
atis_grounn_seevies
ReLa-Net
Slot:
0
B-dirport_name
I-dirport_name
MRRL
Intent:
Figure 3: Cases that generated from GL-GIN (Qin et al., 2021b), ReLa-Net (Xing and Tsang, 2022b) and ReLa-Net w/ MRRL. The red text indicates the incorrect predictions.
lutely right, which indicates that MRRL can indeed alleviate the multi-modality problem.
mance the SLU model. Therefore, we introduce a modifier and propose a suitability reward to over- come the multi-modality problem and a compro- mise reward to overcome the misalignment prob- lem and improve the overall accuracy.
5 Related Work
5.1 Intent Detection and Slot Filling
5.2 Reinforcement Learning
As deep learning obtains impressive performance on various tasks (Li al., 2021, 2022: Zhang et al., 2022b; Yu et al., 2023; Zhang et al., 2023b; Li et al., 2023; Zhang et al., 2023a), more and more studies utilize deep learning to SLU and achieve notable achievements (Hakkani- Tür et al.. 2016; Xia et al., 2018; Liu et al., 2019; Huang et al., 2020; Wu et al., 2020; Qin et al., 2021a,b; Huang al., 2021, 2022; Chen al., 2022a,b; Cheng al., 2023e,b). Re- cently, the multi-intent SLU problem has garnered the significant attention, leading to the emergence several graph-based models which have demon- strated promising results. AGIF (Qin et al., 2020) applies graph attention to directly connect the slot nodes of each token with all predicted intent nodes. GL-GIN (Qin et al., 2021b) further introduces global-local graph interaction network specifically and leverages graph- techniques to capture interactions between different parts of the input ut- terance. More recently, Xing and Tsang (2022a) proposes Co-guiding Net to enhance the overall performance via enabling slot and intent to guide and influence each other during the training pro- cess. Xing and Tsang (2022b) proposes ReLa-Net to further exploit label typologies and relations.
Several NLP tasks have been solved through rein- forcement learning techniques, such as dialogue generation (Li et al., 2016, 2017), question answer- ing (Xiong et al., 2018; 8; Lu et al., 2022), machine translation (Wu et al.. 2018; Shao et al., 2023), sentiment transfer (Xu et al., 201 8), and essay scor- ing (Wang et al., 2018c), In SLU task, Wang et al. (201 8a) applies reinforcement learning learn the wrong labeled slots with without user's feedback, Rao al. (2021) proposes reinforce framework enhance automatic speech recognition robustness in SLU. In our work, we apply reinforcement learn- ing to alleviate the multi-modality problem and the misalignment problem in non-autoregreesive SLU.
6 Conclusion
In this paper, we propose MRRL, simple yet effec- tive method to alleviate the multi- multi-modality problem and misalignment problem in non-autoregressive multi-inted SLU. We introduce modifier to pro- vide a more suitable training target for the model, and apply reinforcement learning with the suitabil- ity reward and compromise reward. Experiments and analysis demonstrate the effectiveness of our proposed method, which can consistently improve the performance of baselines and the best variant achieves new state-of-the of-the-art performance. Future
However, most of the previous models neglect the multi-modality problem and the misalignment problem, which are both detrimental to the perfor-

---

work will focus on how to further alleviate the two problems for non-autoreereesivve multi-intent SLU.
Learning Framework for Cross- Lingsl Spoken Lan- guage Understanding, In Proc. of Interspeech.
Xuxin Cheng, Wanshi Xu, Zhihong Zhu, Hongxiang Li, and Yuexian Zou, 2023d. Towards spoken lan- guage understanding via multi-level multi-graine contrastive learning. In Proc. of CIKM.
Limitations
Although our MRRL consistently improve the per- formance of the basclines, and the best variant (i.e. ReLa-Net + MRRL) achieves new state-of-the-art results, it does not change the inherent structure of the model. In fact, the BiLSTM used is relatively simple, which limits the performance of SLU. In the future, we will pay more attention to these de- ficiencies and try to design better frameworks for non-autoregressive multi-intent SLU.
Xuxin Cheng, Ziyu Yao, Zhihong Zhu, Yaowei Li,
Hongxiang Li, and Yuexian Zou. 2023e. C2 A-SU:: Cross and Contrastive Attention for Improving ASR Robustness in Spoken Language Understanding. In Proc. of Interspeech.
Xuxin Cheng, Zhihong Zhu, Wanshi Xu, Yaowei Li,
Hongxiang Li, and Yuexian Zou. 20231. Accelerating multiple intent detection and slot filling via targeted knowledge distillation. In Proc. of EMNLP Findings.
Acknowledgements
Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Calta- girone, Thibaut Lavril, al. 2018. Snips voice plat- form: an embedded spoken language understanding system for private-by tewrateby-desipe be-den voice interfaces. ArXiv preprint.
We thank all anonymous reviewers for their con- structive comments. This paper was partially sup- ported by Shenzhen Science & Technology Re- search Program (No: CXWD20201231 0920081411300100 and NSFC (No: 62176008).
Haihong Peiqing Niu, Zhongfu Chen, and Meina Song. 2019. A novel bi-directional interrelated model for joint intent detection and slot filling. In Proc. of ACI.
References
Byung Hoon Ahn, Prannoy Pilligundla, and Hadi Es- maeilzadeh, 2019. Reinforcement learning and adap- tive sampling for optimized dnn compilation. ArXiv prepriat.
Yingjie Fei, Zhuoran Yang. Yudong Chen, Zhaoran Wang, and Qiaomin Xie. 2020. Risk-sensitive rein- forcement learning: Near-optimal risk- trade- off in regret. In Proc. of NeurIPS.
Dongsheng Chen, Zhiqi Huang, Xian Wa, Shen Ge, and Yuexian Zou. 2022a. Towards joint intent detection and slot filling via higher-order attention. In Proc. of
Rashmi
Gangadharaiah and Balakrishnan Narayanaswamy. 2019. Joint multiple intent detection and slot labeling for goal-oriented dialog. In Proc. of NAACL.
IJCAI.
Dongsheng Chen, Zhiqi Huang, and Yuexian Zou. 2022b. Leveraging bilinear attention to improve spo- ken language understanding. In Proc. of ICASSP.
Chih- Hiih-WeenC Goo, Guang Gao. Yun-Kai Hsu, Chih-Li Huo, Tsung- Ching Chen, Keng- Wei Hsu, and Yun-Nung Chen. 2018. Slot-gated modeling for joint slot filling and intent prediction. In Proc. of NAACI.
Lizhi Cheng, Weijia Jia, and Wenmian Yang. 202 An effective non-autoren aon-euvoreeressvve model for spoken language understanding. In Proc. of CIKM.
Dilek Hakkani-Tür, Gökhan Tür, Asli Celikyilmaz, Yun- Nung Chen, Jianfeng Gao, Li Deng, and Ye-Ye Yi Wang. 2016. Multi-domain joint semantic frame parsing using bi-directional RNN-LSTM. In Proc. of INTER- SPEECH.
Lizhi Cheng, Wenmian Yang, and Weijia Jia. 2021b. A result based portable framework for spoken language understanding. In Proc. of ICME.
Lizhi Cheng, Wenmian Yang, and Weijia Jia. 2023a. scope sensitive and result attentive model for multi- intent spoken language understanding. In Proc. of
Mutian He and Philip N. Garner. 2023a. Can ChalGPT Detect Intent? Evaluating Large Language Models for Spoken Language Understanding. In Proc. of Interspeech.
AAAL
Xuxin Cheng, Bowen Cao, Qichen Ye, Zhihong Zhu, Hongxiang Li. and Yuexian Zou. 2023b. MI-Imel: contrastive learn-
Mutian He and Philip N Garner. 2023b. The inter- preter understands your meaning: End-to-end spoken language understanding aided by speech translation. ArXiv preprint.
Mutual learning and ing for improving asr robustness in spoken language understanding. In Proc. of ACL Findings.
Xuxin Cheng, Wanshi Xu, Ziyu Yao, Zhihong Zhu, Yaowei Li, Hongxiang Li, and Yuexian Zou. 2023c. FC-MTLF: A Fine- and Coarse-grained Multi-Task
Charles T. Hemphill, John J. Godfrey, and George R. Doddington. 1990. The ATIS spoken language sys- tems pilot corpus. In Speech and Natural Language:

---

Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24- 1990.
Oliver Mihatsch and Ralph Neuneier. 2002. Risk- sensitive reinforcement learning. Machine learning.
Zhiqi Huang, Fenglin Liu, Peilin Zhou, and Yuex- ian Zou. 2021. Sentiment injected iteratively co- interactive network for spoken language understand- ing. In Proc. of ICASSP.
Libo Qin, Wanxiang Che, Yangming Li, Haoyang Wen, and Ting Liu. 2019. A propapprongtttoo framework with token-level intent detection for spoken language understanding. In Proc. of EMNLP.
Zhiqi Huang, Fenglin Liu, and Yuexian Zou, 2020. Fed- erated learning for spoken language understanding. Proc. of COLING.
Libo Qin, Qiguang Chen, Tianbao Xie. Qixin Li, Jian- Guang Lou, Wanxiang Che, and Min-Yes Kan. 2022.
GL-CLeF: A global-local contrastive learning frame- work for cross-lingual spoken language understand- ing. In Proc. of ACI.
Zhiqi Huang, Milind Rao. Anirudh Raju, Zhe Zhang, Bach Bui, and Chul Lee. 2022. MTI-SLT: Multi- task learning for spoken language tasks. In Proceed- ings of the 4th Workshop on NLP for Conversational AL.
Libo Qin, Tailu Liu, Wanxiang Che, Bingbing Kang, Sendong Zhao, and Ting Liu. 2021a. A co-interactive transformer for joint slot filling and intent detection. In Proc. of ICASSP.
Byeongchang Kim, Seonghan Ryu, and Gary Geunbac Lee. 2017. Two-stage multi- inteit detection for spo- ken language understanding. Multimedia Tooly and Applications.
Libo Qin, Fuxuan Wei, Tianbao Xie, Xiao Xu, Wanx- iang Che, and Ting Liu. 2021b. GL-GIN: Fast and accurate -anaaoorrerresive model for joint multiple intent detection and slot filling. In Proc. of ACL.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proc. of ICLR.
Libo Qin, Xiao Xu, Wanxiang Che, and Ting Liu. 2020. AGIF: An adaptive graph-interace framework for joint multiple intent detection and slot filling. In Proc. of EMNLP Findings.
Jinpeng Li, Yingce Xia, Rui Yan, Hongda Sun, Dongyan Zhao, and Tie- Liu. 2021. Stylized dialogue gen- eration with multi-pass dual learning. In Proc. of NeurIPS.
Qiu Ran, Yankai Lin, Peng Li. and Jie Zhou. 2020. Learning to recover from multi molti-modality errors for non-autoregressive neural machine translation. In Proc. of ACL.
Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016. Deep rein- forcement learning for dialogue generation. Proc. of EMNLP.
Milind Rao, Pranav Dheram, Gautam Tiwari, Anirudh Raju, Jasha Droppo, Ariya Rastrow, and Andreas Stoleke. 2021. Do as mean, not as say: Sequence loss training for spoken language understanding. In Proc. of ICASSP.
Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. 2017. Adversarial learning for neural dialogue generation. In Proc. of EMNLP.
Chenze Shao, Jinchao Zhang, Jie Zhou, and Yang Feng. 2023. Rephrasing the reference for non- autoregressive machine translation. In Proc. of AAAI.
Yinghui Li, Haojing Huang. Shirong Ma, Yong Jiang, Yangning Li, Feng Zhou, Hai-Tao Zheng, and Qingyu Zhou. 2023. On the (in)effectiveness of large lan- guage models for chinese text correction. ArXiv preprint.
Rohan Shet, Elena Davcheva, and Christian Uhle. 2019. Segmenting multi-intent queries for spoken language understanding. Studientexte zur Sprachkommunika- tion: Elektronische Sprachsignulvevareeiuugg 2019.
Yinghui Li. Shirong Ma. Qingyu Zhou, Zhongli Li, Li Yangning, Shulin Huang, Ruiyang Liu, Chao Li, Yunbo Cao, and Haitao Zheng. 2022. Learning from the dictionary: Heterogeneous knowledge guided fine-tuning for Chinese spell checking. In Proc. of EMNLP Findings.
Mengxiao Song, Bowen Yu, Li Quangang, Wang Yubin, Tingwen Liu, and Hongbo Xu. 2022. Enhancing joint multiple intent detection and slot filling with global intent-slot co-occurrence. In Proc. of EMNLP.
Bing Liu and lan R. Lane. 2016. Attention -baedd recur- rent neural network models for joint intent detection and slot filling. In Proc. of INTERSPEECH.
Gokhan Tur and Renato De Mori. 2011. Spoken lan- guage understanding: Systems for extracting seman- tic information from speech. John Wiley & Sons.
Yijin Liu, Fandong Meng. Jinchao Zhang, Jie Zhou, Yufeng Chen, and Jinan Xu. 2019. CM-net: A novel collaborative memory network for spoken language understanding. In Proc. of EMNLP.
Petar Velickovic, Guillem Cucurull. Arantxa Casanova, Adriana Romero, Pietro Lið, and Yoshua Bengio. 2018. Graph attention networks. In Proc. of ICLR.
Yu Wang. Abhishek Patel. Yilin Shen, and Hongxia Jin. 2018a. A deep reinforcement learning based multimodal coaching model (DCM) for slot filling in spoken language understanding(slu) In Proc. of INTERSPEECH.
Jiaying Lu, Xin Ye, Yi Ren, and Yezhou Yang. 2022. Good, better, best: Textual distractors generation for multiple-choice visual question answering via reinforcement learning. In Proc. CVPR.

---

Yu Wang, Yilin Shen, and Hongxia Jin. 2018b. bi- model based RNN semantic frame parsing model for intent detection and slot filling. In Proc. of NAACI.
Dong Zhang, Rong Ye. Tom Ko. Mingxuan Wang. and Yaqian Zhou. 2023a. DUB: Discrete unit back- translation for speech translation. In Proc. of ACI. Findings.
Yucheng Wang, Zhongyu Wei, Yaqian Zhou, and Xuan- jing Huang. 2018c. Automatic essay scoring incorpo- rating rating schema via reinforcement learning. In Proc. of EMNLP.
Junzi Zhang, Jongho Kim, Brendan O' O'Donoguu,, and Stephen P. Boyd. 2021. Sample efficient reinforce- ment learning with REINFORCE. In Proc. of AAAI
Ronald Williams. 1992. Simple statistical gradient- following algorithms for connectionist reinforcement learning. Machine learning.
Kexun Zhang, Rui Wang, Xu Tan, Junliang Guo, Yi Ren, Tao Qin, and Tie- Yan Liu. 2022a. A study of syn- tactic multi-modality in non-autoregressive machine translation. In Proc. of NAACL.
Wu, Liang Ding, Fan Lu, and Jian Xie, 2020. SlotRe- fine: A fast non-aueoreeeessive model for joint intent detection and slot filling. In Proc. of EMNLP.
Xin Zhang, Dong Zhang, Shimin Li. Yaqian Zhou, and Xipeng Qiu. 2023b. Speechtokenizer: Unified speech tokenizer for speech large language models. ArXiv preprint.
Lijun Wu, Fei Tian, Tao Qin, Jianhuang Lai, and Tie- Yan Liu. 2018. A study of reinforcement learning for neural machine translation. Proc. of EMNLP.
Xuanyu Zhang, Yongbing Zhang, Ruiqin Xiong, Qilin Sun, Jian Zhang. 2022b. Herosnet: Hyperspec- tral explicable reconstruction and optimal sampling deep network for snapshot compressive imaging. In Proc. of CVPR.
Congying Xia, Chenwei Zhang, Xiaohui Yan, Yi Chang, and Philip Yu. 2018. Zero-shot user intent detection via capsule neural networks. In Proc. of EMNLP.
Zhihong Zhu, Xuxin Cheng, Zhiqi Huang, Dongsheng Chen, and Yuexian Zou. 2023a. Enhancing code- switching for cross-lingual slu: A unified view of semantic and grammatical coherence. Proc. of EMNLP.
Bowen Xing and Ivor Tsang, 2022a. Co Co-guiding net: Achieving mutual guidances between multiple intent detection and slot filling via heterogeneous semantics- label graphs. In Proc. of EMNLP.
Bowen Xing and Ivor Tsang. 2022b. Group is better than individual: Exploiting label topologies and label relations for joint multiple intent detection and slot filling. In Proc. of EMNLP.
Zhihong Zhu, Xuxin Cheng, Zhiqi Huang, Dongsheng Chen, and Yuexian Zou. 2023b. Towards unified spoken language understanding decoding via label- aware compact linguistics representations. In Proc. of ACI. Findings.
Bowen Xing and Ivor W Tsang. 2023, Relational tempo- ral graph reasoning for dual-task dialogue language understanding. IEEE Transactions on Pattern Analy- sis and Machine Intelligence.
Zhihong Zhu, Weiyuan Xu, Xuxin Cheng, Tengtao Song, and Yuexian Zou. 2023c. A dynamic graph in- teractive framework with sebel-semantic injection for spoken language understanding. In Proc. of ICASSP.
Caiming Xiong, Victor Zhong, and Richard Socher. 2018. DCN+: mixed objective and deep residual coattention for question answering. In Proc. ICLR.
Jingjing Xu, Xu Sun, Qi Zeng, Xiaodong Zhang, Xu- ancheng Ren, Houfeng Wang, and Wenjie Li. 2018. Unpaired sentiment-to-sentimenn translation: A cy- cled reinforcement learning approach. In Proc. of ACL.
Puyang Xu and Ruhi Sarikaya. 2013. Convolutional neural network based triangular crf for joint intent detection and slot filling. In 2013 ieee workshop on automatic speech recognition and understanding.
Steve Young, Milica Gašié, Blaise Thomson, and Ja- son Williams. 2013. Pomdp-based statistical spo- ken dialog systems: A review. Proceedings of the IEEE.
Jiwen Yu, Xuanyu Zhang, Youmin Xu, and Jian Zhang. 2023. Cross: Diffusion model makes controllable, robust and secure image steganography. ArXiy preprint.


Reviews:
[{"review_id": "RgKiIwugCO", "reviewer": "Reviewer_6FMz", "summary": "", "questions": "1, Writing should be improved: you spend quite much content on related work and backgroud such as discussing the existing models in Section 2. \n2, Table 2: the baselines, the proposed models, and the other models should be separated more clearly to follow easily\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the insightful comments. The following are our responses to concerns raised:\n\nQ1: More analyses should be conducted to clearly show how the issues (misalignment, and multi-modality) have been solved by the proposed method.\n\nA1: Thanks for your advice. We will add more analyses and provide more examples in the final version to show how the issues (misalignment, and multi-modality) have been solved by the proposed method.\n\nQ2: Quite much content on related work and background.\n\nA2: Thanks for your advice. We introduce the details of related work and background to help the readers  better understand the task. We take your advice and will move some of the details of Section 2 to the appendix in the final version.\n\nQ3: Table 2: the baselines, the proposed models, and the other models should be separated more clearly to follow easily.\n\nA3: Thanks for your advice. We will separate them more clearly to follow easily in the final version.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1, The proposed models show the improvement in comparison with the baselines.\n2, The proposed method is useful to improve the tasks of dialogue models.\n3, The proposed reward is quite interesting", "reasons_to_reject": "1, More analyses should be conducted to clearly show how the issues (misalignment, and multi-modality) have been solved by the proposed method.\n", "questions_to_authors": "1, Writing should be improved: you spend quite much content on related work and backgroud such as discussing the existing models in Section 2. \n2, Table 2: the baselines, the proposed models, and the other models should be separated more clearly to follow easily\n", "paper_topic_and_main_contributions": "This paper proposes a method for multi-intent detection and slot filling, which aims at tackling the two issues of non-autoregressive models: multi-modality and misalignment. The proposed method includes a modifier to modify the reference according to the output of non-autoregressive model, and a reward for applying reinforcement learning.\nThe proposed method is combined with several baselines, and the results show the improvement in comparison with the baselines.\n", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "AefLXg9DQK", "reviewer": "Reviewer_cooG", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful comments. The following are our responses to concerns raised:\n\nQ1: Why not use the modifier as the final model?\n\nA1: Thanks for your question. Do you mean we only use the modifier as the final model? As we mentioned in $\\textbf{line 294 to line 308}$. If we directly apply reinforcement learning to train the modifier, the modifier will be trapped in a non-optimal state where only sequences of meaningless intentions and slots are output, consisting of some of the most frequent intentions and slots, respectively. Therefore, we do not use the modifier as the final model. We first apply a pre-training strategy to establish a better initial state for the modifier so that the modifier can learn more easily in fine-tuning and avoid falling into local optimization. Then we fine-tune the modifier with two rewards in reinforcement learning. \n\nQ2: What if you use reinforcement learning directly on base models?\n\nA2: Thanks for your question. As shown in Figure 2, the base models and the modifiers share the parameters. As we mentioned in $\\textbf{line 294 to line 302}$. If we directly apply reinforcement learning to train the modifier, the modifier will be trapped in a non-optimal state where only sequences of meaningless intentions and slots are output, consisting of some of the most frequent intentions and slots, respectively. In pre-training, we train the base models to establish a better initial state. And in fine-tuning, we fine-tune the modifier with two rewards in reinforcement learning. In the final version, we will further emphasize that they share the parameters.\n\nQ3: How does RL help balance the performance of two modules?\n\nA3: Thanks for your advice. We will add more quantitative and qualitative analysis in the final version to verify that RL could balance the performance of two modules. We will show the learning curve like Figure 1 to help readers to better understand the training process.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper identified two major challenges in non-autoregressive slot filling and intent detection. The authors proposed a \"knowledge distillation\" style approach to narrow the gap between model output and reference, which eases the training process. The result is great, boosting the performance of a series of baseline models.\n- The architecture is easy to follow and the writing is clear. ", "reasons_to_reject": "- Further analysis is needed to demonstrate the necessicity of a modifier module. Why not use it as the final model? What if you use reinforcement learning directly on base models?\n- More quantitative and qualitative analysis is needed to study the convergence and performance of RL-aided models. How does RL help balance the performance of two modules? For example, a similar learning curve like Figure 1 may help to better understand the training process.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This research applies reinforcement learning to address the multi-modality and misalignment problems in non-autoregressive multi-intent SLU. They introduce a new method called MRRL, which modifies the reference utterance for model training. MRRL shows promise, improving the performance of the baselines and achieving new state-of-the-art results.\n\n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "v9k8H8WS6v", "reviewer": "Reviewer_ejsG", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the insightful comments. The following are our responses to concerns raised:\n\nQ1: What is the different between the golden labels and modifier’s outputs? Why is their output the better target?\n\nA1: The golden labels are the ground truth and the modifier's outputs are the predictions. As we mentioned in $\\textbf{line 319 - line 329}$, we design the fit reward to help the output of modifier fit well with the output of the non-autoregressive model and the similarity reward to help the output of the modifier module not deviate too far from the reference. Therefore, we believe the output of the modifier is the better target. Besides, we also conduct the ablation studies to verify the superiority of the modifier, as shown in Table 3. We will also add more evidence in the final version.\n\nQ2: More evidence to prove that their compromise reward can exactly help the alignment between the correct predictions of the two tasks.\n\nA2: Thanks for your advice. We will add more evidence in the final version to verify that the compromise reward could help the alignment between the correct predictions of the two tasks. We will show the learning curve like Figure 1 to help readers to better understand the training process.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "(1)\tThe writing is clear, and experiments are extensive.\n(2)\tThe introduced method can easily applied on many existing approaches.\n(3)\tThe performance is good and significant (achieving stoa on two benchmarks).\n", "reasons_to_reject": "(1) From the motivation, the intent modifier and slot modifier can provide a better training target, what is the different between the golden labels and modifier’s outputs? And why their output is the better target? The authors should provide more evidence.\n\n(2)\tThe authors should provide more evidence to prove that their compromise reward can exactly help the alignment between the correct predictions of the two tasks. (like Figure 1)\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This work explore a novel method to non-autoregressively solve multi-intent joint multiple intent detection and slot filling in a better manner. They find that existing non-autoregressive approaches still suffer from two issues: (1) the multi-modality problem that lead to reference intents and slots may not be suitable for training; (2) lack of alignment between the correct 008 predictions of the two tasks ,thereby limiting the performance. To solve the problem, this work introduce propose Modifying the Reference via Reinforcement Learning (MRRL) for intent detection and slot filling. Specifically, they provide a better training target for the non-autoregressive model by modifying the reference according to the output of the non-autoregressive model. Additionally, they also propose a compromise reward to realize a flexible trade-off between the two tasks.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work applies reinforcement learning (RL) to address the multi-modality and misalignment problems in non-autoregressive multi-intent SLU. It introduces a new method called Modifying the Reference via Reinforcement Learning (MRRL).

The paper identifies two interesting challenges in non-autoregressive slot filling and intent detection, demonstrates good performance and proposes an interesting reward. However, the reviewers considered that additional analysis would be helpful in determining how the issues are being addressed, and the stability of the proposed RL approach.
Decision: Accept-Findings