Bridge to Target Domain by Prototypical Contrastive Learning and Label Confusion: Re-explore Zero-Shot Learning for Slot Filling (2021)

Abstract: Zero-shot cross-domain slot filling alleviates the data dependence in the case of data scarcity in the target domain, which has aroused extensive research. However, as most of the existing methods do not achieve effective knowledge transfer to the target domain, they just fit the distribution of the seen slot and show poor performance on unseen slot in the target domain. To solve this, we propose a novel approach based on prototypical contrastive learning with a dynamic label confusion strategy for zero-shot slot filling. The prototypical contrastive learning aims to reconstruct the semantic constraints of labels, and we introduce the label confusion strategy to establish the label dependence between the source domains and the target domain on-the-fly. Experimental results show that our model achieves significant improvement on the unseen slots, while also set new state-of-the-arts on slot filling task.

## introduction slot filling, as an important part of the task-oriented dialogue system, is mainly used to extract specific information in user utterances. traditional supervised methods have shown remarkable performance in slot filling tasks @xcite @xcite @xcite @xcite ), but they require a large amount of domain-specific labeled data. recently, there has been increasing interest in the zero-shot cross-domain slot filling task, whose goal is to identify unseen slots in the target domain without any labeled data. previous methods can be classified into two types: one-stage and two-stage. in the one-stage framework, (bapna et al., 2017; shah * the first two authors contribute equally. weiran xu is the corresponding author. ## conclusion & discussion in this paper, we propose a novel method based on prototypical contrastive learning and label confusion strategy (pclc) for cross-domain slot filling. our main contribution was to improve the domain adaptability of the model. the proposed method conducts a refinement process for label semantic space to re-establish the constraint relationship between different slots. experiments show the effectiveness of our method, especially for recognizing the unseen slots. as there is still a huge gap in model performance between unseen and seen slots, in future work, we will focus on improving the performance on unseen slots while maintaining the performance on seen slots. representation disentanglement @xcite can be used to disentangle domain-specific and domain-shared knowledge in the source domain, then we can preserve domainshared knowledge and focus on establishing the relation of domain-specific knowledge between the source and the target domain.