Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection (2020)

Abstract: Few-shot Intent Detection is challenging due to the scarcity of available annotated utterances. Although recent works demonstrate that multi-level matching plays an important role in transferring learned knowledge from seen training classes to novel testing classes, they rely on a static similarity measure and overly fine-grained matching components. These limitations inhibit generalizing capability towards Generalized Few-shot Learning settings where both seen and novel classes are co-existent. In this paper, we propose a novel Semantic Matching and Aggregation Network where semantic components are distilled from utterances via multi-head self-attention with additional dynamic regularization constraints. These semantic components capture high-level information, resulting in more effective matching between instances. Our multi-perspective matching method provides a comprehensive matching measure to enhance representations of both labeled and unlabeled instances. We also propose a more challenging evaluation setting that considers classification on the joint all-class label space. Extensive experimental results demonstrate the effectiveness of our method. Our code and data are publicly available.

## introduction intent detection (id) is a crucial task in natural language understanding, whose objective is to extract underlying intents behind the given utterances. the extracted intents could provide further contexts for further downstream natural language processing tasks such as dialogue state tracking or question answering. unlike traditional text classification, id is challenging for two main reasons (1) utterances are usually short and diversely expressed, (2) emerging intents occur continuously, especially across different domains @xcite . despite recent advances, state-of-the-art id methods @xcite require a large amount of annotated data to achieve competitive performance. this requirement inhibits models' capability in generalizing to newly emerging intents with no or limited annotations during inference. re-training or fine-tuning large models on few samples of emerging classes could easily lead to overfitting problems. motivated by human capability in correctly categorizing new classes with only a few examples @xcite , few-shot learning (fsl) paradigms are adopted to tackle the scarcity problems of emerging classes. fsl methods take advantage of a small set of labeled examples (support set) to learn how to discriminate unlabeled samples (query samples) between classes, even those not seen during training. recent works in fsl @xcite focus on learning the matching information between the labeled samples (support) and the unlabeled samples (query) to provide additional contextual information for instance-level representations, leading to effective prototype representation. however, these methods only extract similarity based on fine-grained word semantics, failing to capture the diverse expressions of users' utterances. this problem could further lead to overfitting either to seen intents or novel intents, especially in the challenging generalized few-shot intent detection (gfsid) setting where both seen and novel intents are existent in a joint label space during inference. instead, matching support and query samples on coarser-grained semantic components could provide additional informative contexts beyond word levels. for instance, two utterances "i need to get a table at a pub with southeastern cuisine" and "book a spot for six friends" share a sim-ilar intent label "book restaurant". while wordlevel semantics might find similar action words as "get" and "book", these words do not necessarily contribute to the correct intent findings. instead, coarser-grained semantics such as "get a table" and "book a spot" could provide further hints to identify "book restaurant" intent. as semantic components (sc) could be effectively extracted from multi-head self-attention, matching these sc between support and query can enhance both query and support representations, leading to improvements in generalization from seen training classes to unseen testing classes. to further enhance the dynamics of extracted sc across various domains and diversely expressed utterances, we introduce additional head regularizations. in addition, to overcome the insufficiency of a single similarity measure for matching sentences with diverse semantics, a more comprehensive matching method is further explored. our main contribution is summarized as follows: ## related work few-shot learning few-shot learning refers to problems where classifiers are required to generalize to unseen classes with only a few training examples per class @xcite . to overcome challenges of potential overfitting, most fsl methods adopt meta-learning approach where knowledge is extracted and transferred across multiple tasks. there are two major approaches towards fsl: (1) metric-based approach whose goal is to learn feature extractor that extract and generalize to emerging classes @xcite @xcite , and (2) optimizationbased approach that aims to optimize model parameters from few samples @xcite @xcite . in this work, we focus mostly on metric-based learning approach. specifically, we extend prototypical network (pn) @xcite in which prototypes are not only represented by support samples but also matching information between support and query samples. traditionally, fsl methods are evaluated in episodic procedure due to the major principle that test and train conditions must match @xcite . each episode represents a meta-learning task in which the models explicitly "learn to learn" minimize the loss on an unlabeled/ query set given the support/ labeled set. however, we claim that this evaluation is lack of practicality for two main reasons. first, evaluation on random samples could not help us understand the strengths or weaknesses of the model. for instance, if the trained model overfits a subset of novel classes, it is impossible to pinpoint the overfitting classes with episodic evaluation. secondly, in realistic applications, there is a need to categorize unlabeled samples into one of the novel/joint classes, rather than a set of sampled classes. episodic testing does not provide an end-to-end systematic evaluation. therefore, in our work, we propose a more challenging but realistic non-episodic evaluation setting where unlabeled samples are only inferred once with a probablility distribution over a fixed set of classes in novel or joint label space. sentence matching recent fsl works adopt multi-level matching and aggregation methods to improve fsl performance @xcite @xcite . instead of constructing prototypes purely from support samples, recent works integrate matching information between support and query samples on multiple levels. @xcite introduces feature-level and instance-level attention. @xcite introduces additional word-level attention and proposes more advanced multi-cross attention on instancelevel. on the other hand, @xcite adopts soft matching between support and query samples to build local context representation for both support and query samples. these methods have been proven effective in few-shot relation classification tasks. however, they rely on overly fine-grained level matching which potentially causes overfitting problems towards either seen or unseen set of classes. our work mainly differs in two aspects: (1) comprehensive multi-perspective matching for information matching and (2) matching on coarser-grained semantic-component levels that are extracted dynamically for effective knowledge transfer, especially in gfsl settings. ## problem formulation in this section, we provide definitions for both fewshot intent detection (fsid) and gfsid task. traditional fsl task is defined as c-way k-shot classification task in which classifier performs a series of tasks during both training and inference, which involves c randomly chosen classes with only k labeled samples from each class (@xmath0. these c • k samples are named as support samples. this series of tasks are repeated via episodes @xcite . in each episode, the objective is to correctly classify unlabeled samples (query samples) by using only the support samples. we denote seen label space as y s , novel label space as y n , and given the seen labels (y s ), we define d @xmath1, y 1 ), (x 2 , y 2 ), ...(x ns , y ns )}, where n s denotes the total number of seen samples and (x, y) denotes a pair of utterance and intent label. similarly, d @xmath2, y 1 ), (x 2 , y 2 ), ...(x nn , y nn )}. given an unlabeled utterance x, the objective of fsid is to maximize correct prediction for x within the novel label subspace y n as summarized in (1). for gfsid, there exists an additional joint label space y @xmath3. unlike fsid, gfsid is more challenging as the test samples could come from either seen or novel sample space. the objective function is modified as follows. in this section, we introduce our proposed architecture. specifically, we divide the framework into 3 main components: semantic encoder, semantic matching & aggregation, instance aggregation & class matching as illustrated in figure 1 . ## conclusions in this paper, we propose an effective semantic matching and aggregation network for few-shot intent detection. semantic components extracted from multi-head self-attention capture higher level contextual information beyond the word level, enhancing model's generalizability towards both seen and novel intents, especially when utterances are diversely expressed. comprehensive multiperspective matching method thoroughly exploits the similarity between query and support samples for further robust representations. in this work, we also propose a more challenging but realistic nonepisodic evaluation for both fsl and gfsl beyond traditional setting. our model achieves the stateof-the-art performance in both evaluation settings for snips and nlue benchmark datasets. further studies of more dynamic semantic extraction and effectively synthesized matching techniques are our desired future work.