ICA -Proto: Iterative Cross Alignment Prototypical Network for Incremental Few-Shot Relation Classification (2023)

Abstract: In the task of incremental few-shot relation classification, model performance is always limited by the incompatibility between the base feature embedding space and the novel feature embedding space. To tackle the issue, we propose a novel model named ICA-Proto: Iterative Cross Alignment prototypical network. Specifically, we incorporate the query representation into the encoding of novel prototypes and utilize the query-aware prototypes to update the query representation at the same time. Further, we implement the above process iteratively to achieve more interaction. In addition, a novel prototype quadruplet loss is designed to regulate the spatial distributions of embedding space, so as to make it easier for the relation classification. Experimental results on two benchmark datasets demonstrate that ICA-Proto significantly outperforms the state-of-the-art baseline model.

## introduction relation classification (rc) is an important subtask of relation extraction (re), aims at classifying the relation between two marked entities in a given sentence. for example, the instance " @xcite e1 served as the president of @xcite e2 " expresses the relation member_of between the two entities newton and the royal society. some conventional methods @xcite @xcite for relation classification adopt supervised training and usually suffer from the scarcity of manually annotated data. to alleviate this problem, distant supervision (ds) is adopted to automatically label abundant training instances by heuristically aligning knowledge graphs (kgs) with texts @xcite . however, existing ds-based methods fail to deal with the problem of long-tail relations in kgs and still suffer from data deficiency @xcite . figure 1 : visualization of the representations of the query instances and prototypes of bert-increproto. we randomly sampled three base relations and three novel relations from the real-world dataset fewrel 1.0, each relation with its corresponding prototype (triangles for base relations and stars for novel relations) and eight query instances (points). to address the above long-tail problem, few-shot rc was proposed, which formulates rc in a fewshot learning scenario. this task requires the models trained with base relations to generalize well to novel relations with only few labeled instances. base relations are those relations that contain adequate instances and can be utilized effectively in the training phase to mimic the test phase on novel relations with few samples. fine-tuning pre-trained models @xcite is straightforward while suffering from the overfitting problem. therefore, metric-based methods @xcite @xcite were proposed to grasp the fast-learning ability from previous experiences and then quickly generalize to new concept. these methods have been experimentally proven to be effective. taking a step further, incremental few-shot rc @xcite considers a more realistic scenario, where the model is required to dynamically recognize the novel relations with a few samples, without reducing the base relation identification capability learned on the large-scale data of base relations. hence in the test phase, the query set consists of instances of not only base relations but also novel relations, which is more challenging. several related works @xcite @xcite have been proposed in the field of computer vision, focusing on image classification task. as for the task of incremental few-shot rc, increprotonet @xcite is the first work, which proposes a two-phase prototypical network model. specifically, increprotonet contains two separate prototypical networks @xcite . one is pre-trained in the first phase to acquire the base prototypes and base feature extractor, and the other obtains the novel prototypes and novel feature encoder with few-shot episode training in the second phase. however, increprotonet suffers from insufficient interaction between the class prototypes and the query instances. therefore, in the embedding space, the novel relations often overlap significantly with the base relations, and the query representations are scattered, as shown in figure 1 . in addition, the triplet loss used by increprotonet may be affected by noise samples, and its effectiveness decreases on tasks with domain shift. as a result, a low accuracy in the recognition of novel relationships has been observed. to alleviate the above problem, we propose a novel model named ica-proto that contains a specially-designed ica module. ica module consists of two sub-modules, i.e., cross alignment (ca) and iterative alignment (ia). specifically, ca is built to to dynamically and interactively encode the novel prototypes and query instances. on the one hand, the obtaining of novel prototypes is query-aware, namely that the query-related support instances contribute more to the final prototypes. on the other hand, the encoding of query instances is prototype-aware, since the query-related prototypes have more influence on the query representations. furthermore, to achieve sufficient interaction and alignment, we construct ia, which is to implement the above ca iteratively. in addition, prototype quadruplet (pq) loss is proposed to enlarge the distance between different types of prototypes, while making the distance between query and prototype of the same class as close as possible. the contributions of this paper can be summarized below: in the task of incremental few-shot rc, first we are given a large dataset containing n base base relations: is its i-th query instance. therefore, the model is required to dynamically recognize the novel relations based on a few novel support instances while keeping the base relation identification capability learned on the large base dataset. ## method in this section, we elaborate on the details of our proposed ica-proto model for incremental fewshot rc. first, we give a brief introduction to the increprotonet in section 3.1. then, we introduce the overall framework of our model in section 3.2. next, we present the proposed ica module with ca and ia sub-modules in section 3.3. moreover, the proposed pq loss is discussed in section 3.4. ## conclusion in this paper, we presented a novel and effective approach with iterative cross alignment module and prototype quadruplet loss for the task of incremental few-shot learning. benefit from the extensive interaction offered by the iterative cross alignment and the feature space regulation brought by the prototype quadruplet loss, our method outperformed the state-of-the-art baseline method significantly, as verified in our extensive experiments. in future work, we aim to further improve the performance of our model under the one-shot task setting, as well as accelerate the training process.