Disentangling Task Relations for Few-shot Text Classification via Self-Supervised Hierarchical Task Clustering (2022)

Abstract: Few-Shot Text Classification (FSTC) imitates humans to learn a new text classifier efficiently with only few examples, by leveraging prior knowledge from historical tasks. However, most prior works assume that all the tasks are sampled from a single data source, which cannot adapt to real-world scenarios where tasks are heterogeneous and lie in different distributions. As such, existing methods may suffer from their globally knowledge-shared mechanisms to handle the task heterogeneity. On the other hand, inherent task relationships are not explicitly captured, making task knowledge unorganized and hard to transfer to new tasks. Thus, we explore a new FSTC setting where tasks can come from a diverse range of data sources. To address the task heterogeneity, we propose a self-supervised hierarchical task clustering (SS-HTC) method. SS-HTC not only customizes the cluster-specific knowledge by dynamically organizing heterogeneous tasks into different clusters in hierarchical levels but also disentangles the underlying relations between tasks to improve the interpretability. Empirically, extensive experiments on five public FSTC benchmark datasets demonstrate the effectiveness of SS-HTC.

## introduction recent advances in deep learning highly rely on massive human annotations. this reliance increases the burden of data collection and meanwhile hinders its potentials to the low-data regime, where the labeled data is scarce and difficult to obtain. inspired by human beings' capabilities that can quickly learn with a few examples, few-shot learning (fsl) @xcite , which aims to learn a classifier that generalizes well even with a few training instances per class, has recently attracted much attention. in the nlp domain, few-shot text classification (fstc) @xcite has been actively investigated in data-sparsity scenarios, e.g., relation classification @xcite , event classification @xcite , and intent classification @xcite , where new categories such as relations, events, or intent types tend to emerge frequently and lack sufficient annotations. meta-learning (a.k.a. learning to learn) approaches @xcite , which transfer prior knowledge from previous tasks to improve the effectiveness in learning new tasks, have achieved superior performance for fstc @xcite @xcite . the prior knowledge can be instantiated as a transferable metric space for retrieving nearest prototypes in @xcite , dynamic capsules in @xcite , and distributional signatures in @xcite , etc. despite their early success, such approaches have two main drawbacks: (1) they assume that all previous tasks are sampled from a single data source or domain, leading to tasks with low intertask variance. as a consequence, these methods globally share the prior knowledge across all tasks but fail to handle real-world applications where the historical tasks that potentially contribute may come from diverse data sources in different distri-butions (a.k.a. task heterogeneity @xcite ). for example, the knowledge learned from categorizing different types of products or services may be hardly transferred to classify public comments or opinions among different topics, or to determine users' intent in dialogues with chatbot services, with only limited labeled data; (2) handling heterogeneous tasks for the better generalization ability exactly requires reliable knowledge organization, which highly relies on disentangling underlying task relations that are ignored by prior works. motivated by those, we study a new fstc setting where tasks come from a diverse range of data sources with possibly different data distributions as shown in figure 1 . to embrace the skills learned from multiple task sources to improve the generalization ability, we propose a novel metalearning framework named self-supervised hierarchical task clustering (ss-htc), which groups tasks into different clusters based on inherent task relations in multiple levels. when a new task arrives, it can quickly take advantage of the historical knowledge learned within the cluster it belongs to. specifically, learning a superior task embedding is the cornerstone to disentangle underlying relations among tasks and group them into different clusters. however, a fsl task is hard to represent, as labeled training data in each fsl task are insufficient. to tackle this issue, we propose a labeloriented masked language modeling to recover the corresponding label texts using each training sample itself from the task. such a self-supervised manner, considering informative label text semantics, encourages the model to generate more discriminative task embedding to discover reasonable task relationships even with limited label information. after that, each task embedding is passed to a hierarchical task tree to dynamically perform soft task clustering in multiple levels, so that the knowledge is shared among highly related tasks in the same cluster but differentiated between different clusters of tasks. then, the updated task embedding outputted by the task tree encodes the representation of the cluster it belongs to. this cluster representation is finally passed to modulate the prior knowledge, a metric space for finding nearest prototypes following @xcite , to be cluster-specific. in a nutshell, ss-htc not only quickly accesses the most relevant cluster and tailors the prior knowledge to address the challenge of task heterogeneity, but also increases the model traditional deep learning models would severely overfit on fsl tasks since only a few labeled samples cannot accurately represent the true data distribution, which will result in learning classifiers with high variance and generalizing poorly to new data. in order to solve the overfitting problem in fsl, @xcite proposed an effective episodic meta-learning strategy that learns a generic classifier from diverse few-shot classification tasks and then employs the classifier to a new task. the purpose of episodic training is to mimic the real testing environment where tasks contain insufficient support sets and unlabeled query sets. the consistency between training and testing environments alleviates the shift gap and boosts the generalization. specifically, using the episodic strategy, the whole process of meta-learning can be divided into three parts: meta-training with training tasks ## conclusion in this paper, we propose the self-supervised hierarchical task clustering (ss-htc) method to tackle the task heterogeneity for fstc by dynamically organizing the tasks into hierarchical clusters and customize the cluster-specific knowledge. extensive experiments on various fstc benchmark datasets quantitatively and qualitatively demonstrate the effectiveness of ss-htc. in the future, the proposed ss-htc can be potentially generalized to the multilingual few-shot setting @xcite .