Towards Unified Spoken Language Understanding Decoding via Label-aware Compact Linguistics Representations (2023)

Abstract: Joint intent detection and slot filling models have shown promising success in recent years due to the high correlations between the two tasks. However, previous works independently decode the two tasks, which could result in misaligned predictions for both tasks. To address this shortcoming, we propose a novel method named Label-aware Compact Linguistics Representation (LCLR), which leverages label embeddings to jointly guide the decoding process. Concretely, LCLR projects both task-specific hidden states into a joint label latent space, where both task-specific hidden states could be concisely represented as linear combinations of label embeddings. Such feature decomposition of task-specific hidden states increases the representing power for the linguistics of utterance. Extensive experiments on two single- and multi-intent SLU benchmarks prove that LCLR can learn more discriminative label information than previous separate decoders, and consistently outperform previous state-of-the-art methods across all metrics. More encouragingly, LCLR can be applied to boost the performance of existing approaches, making it easy to be incorporated into any existing SLU models.

## introduction spoken language understanding (slu) plays a critical role in the task-oriented dialogue system @xcite . a typical slu task mainly includes two subtasks, i.e., intent detection (id) and slot filling (sf). given by an utterance expressed in natural language from the user, id aims to identify the intent of the user (e.g., getweather), and sf aims to fill the slot for each token in the utterance (e.g., location, time). recent studies @xcite ✓denotes that the label is selected as the prediction. recent works have provided some first insights into jointly decoding the two tasks. @xcite extracted features through cnn layers and model the dependencies between intent labels and slot tokens. xing and tsang (2022b) combined task-specific hidden states with label information using linear layers and dot products for enhancing decoding. however, their methods introduce additional parameters and still attempt to perform decoding in different task hidden spaces, which severs correlations between the two tasks. to effectively and efficiently address the two tasks' gap, we propose to learn a joint label latent space based on label embeddings to jointly guide the slu decoding process. for this purpose, we propose a novel method named label-aware compact linguistics representation mechanism (lclr), which uses the same parametric model to project and reformulate both task-specific hidden states. in detail, lclr projects the task-specific hidden states into a joint label latent space in best approximation algorithm (del @xcite , where the task-specific hidden states could be concisely represented as linear combinations of label embeddings. such feature decomposition of task-specific hidden states increases representing power for the linguistics of utterance. in this manner, both intent-specific and slot-specific hidden states are represented with the distributions over the same sets of label hidden variables, which can be guided by the dual-task inter-dependencies conveyed in the learned label embeddings. we conduct extensive experiments on both single-intent and multi-intent slu benchmarks. the results show it can empower the different slu models to consistently achieve better performance. further analysis also demonstrates the advantages of our proposed lclr. overall, our contributions are three-fold: ## conclusion we propose a novel method called label-aware compact linguistics representation (lclr) to jointly guide the decoding process. in the joint label latent space, both task-specific hidden states are concisely represented as the linear combinations of label embeddings, enhancing representing power for the linguistics of utterance. this approach allows the decoding process to be guided by the dual-task inter-dependencies conveyed in the learned label embeddings. experimental results on both single-and multi-intent slu benchmarks demonstrate lclr can consistently empower various slu models to achieve better performance.