C oach: A Coarse-to-Fine Approach for Cross-domain Slot Filling (2020)

Abstract: As an essential task in task-oriented dialog systems, slot filling requires extensive training data in a certain domain. However, such data are not always available. Hence, cross-domain slot filling has naturally arisen to cope with this data scarcity problem. In this paper, we propose a Coarse-to-fine approach (Coach) for cross-domain slot filling. Our model first learns the general pattern of slot entities by detecting whether the tokens are slot entities or not. It then predicts the specific types for the slot entities. In addition, we propose a template regularization approach to improve the adaptation robustness by regularizing the representation of utterances based on utterance templates. Experimental results show that our model significantly outperforms state-of-the-art approaches in slot filling. Furthermore, our model can also be applied to the cross-domain named entity recognition task, and it achieves better adaptation performance than other existing baselines. The code is available at https://github.com/zliucr/coach .

## introduction slot filling models identify task-related slot types in certain domains for user utterances, and are an indispensable part of task-oriented dialog systems. supervised approaches have made great achievements in the slot filling task @xcite , where substantial labeled training samples are needed. however, collecting large numbers of training samples is not only expensive but also time-consuming. to cope with the data scarcity issue, we are motivated to investigate cross-domain slot filling methods, which leverage knowledge learned in the source domains and adapt the models to the target domain with a minimum number of target domain labeled training samples. a challenge in cross-domain slot filling is to handle unseen slot types, which prevents general classification models from adapting to the target domain without any target domain supervision signals. recently, @xcite proposed a cross-domain slot filling framework, which enables zero-shot adaptation. as illustrated in figure 1a , their model conducts slot filling individually for each slot type. it first generates word-level representations, which are then concatenated with the representation of each slot type description, and the predictions are based on the concatenated features for each slot type. due to the inherent variance of slot entities across different domains, it is difficult for this framework to capture the whole slot entity (e.g., "latin dance cardio" in figure 1a ) in the target domain. there also exists a multiple prediction problem. for example, "tune" in figure 1a could be predicted as "b" for both "music item" and "playlist", which would cause additional trouble for the final prediction. we emphasize that in order to capture the whole slot entity, it is pivotal for the model to share its parameters for all slot types in the source domains and learn the general pattern of slot entities. therefore, as depicted in figure 1b , we propose a new cross-domain slot filling framework called coach, can you put this tune onto latin dance cardio can you put this restaurant type onto artist template generation encoder conditional random field (crf) o o b 3-way (b-i-o) classification per token ... ... utterance correct b i i ... ... latin dance cardio ... ... tune encoder encoder representation representation slot type description matrix ... ... similarity comparison similarity comparison music item playlist utterance representation template representations regularization loss step one step two can you put this music item onto playlist incorrect ... can you put this object name onto city a coarse-to-fine approach. it first coarsely learns the slot entity pattern by predicting whether the tokens are slot entities or not. then, it combines the features for each slot entity and predicts the specific (fine) slot type based on the similarity with the representation of each slot type description. in this way, our framework is able to avoid the multiple predictions problem. additionally, we introduce a template regularization method that delexicalizes slot entity tokens in utterances into different slot labels and produces both correct and incorrect utterance templates to regularize the utterance representations. by doing so, the model learns to cluster the representations of semantically similar utterances (i.e., in the same or similar templates) into a similar vector space, which further improves the adaptation robustness. experimental results show that our model surpasses the state-of-the-art methods by a large margin in both zero-shot and few-shot scenarios. in addition, further experiments show that our framework can be applied to cross-domain named entity recognition, and achieves better adaptation performance than other existing frameworks. ## related work coarse-to-fine methods in nlp are best known for syntactic parsing @xcite . @xcite reduced the search space of semantic parsers by using coarse macro grammars. different from the previous work, we apply the idea of coarse-to-fine into cross-domain slot filling to handle unseen slot types by separating the slot filling task into two steps @xcite . coping with low-resource problems where there are zero or few existing training samples has always been an interesting and challenging task @xcite @xcite . cross-domain adaptation addresses the data scarcity problem in low-resource target domains @xcite @xcite @xcite . however, most research studying the cross-domain aspect has not focused on predicting unseen label types in the target domain since both source and target domains have the same label types in the considered tasks @xcite . in another line of work, to bypass unseen label types, @xcite and @xcite utilized target domain training samples, so that there was no unseen label type in the target domain. recently, based on the framework proposed by @xcite (discussed in section 1), @xcite added an attention layer to produce slot-aware representations, and @xcite leveraged slot examples to increase the robustness of cross-domain slot filling adaptation. ## conclusion we introduce a new cross-domain slot filling framework to handle the unseen slot type issue. our model shares its parameters across all slot types and learns to predict whether input tokens are slot entities or not. then, it detects concrete slot types for these slot entity tokens based on the slot type descriptions. moreover, template regularization is proposed to improve the adaptation robustness further. experiments show that our model significantly outperforms existing cross-domain slot filling approaches, and it also achieves better performance for the cross-domain ner task, where there is no unseen label type in the target domain.