Insert or Attach: Taxonomy Completion via Box Embedding (2024)

Abstract: Taxonomy completion, enriching existing taxonomies by inserting new concepts as parents or attaching them as children, has gained significant interest. Previous approaches embed concepts as vectors in Euclidean space, which makes it difficult to model asymmetric relations in taxonomy. In addition, they introduce pseudo-leaves to convert attachment cases into insertion cases, leading to an incorrect bias in network learning dominated by numerous pseudo-leaves. Addressing these, our framework, TaxBox, leverages box containment and center closeness to design two specialized geometric scorers within the box embedding space. These scorers are tailored for insertion and attachment operations and can effectively capture intrinsic relationships between concepts by optimizing on a granular box constraint loss. We employ a dynamic ranking loss mechanism to balance the scores from these scorers, allowing adaptive adjustments of insertion and attachment scores. Experiments on four real-world datasets show that TaxBox significantly outperforms previous methods, yielding substantial improvements over prior methods in real-world datasets, with average performance boosts of 6.7%, 34.9%, and 51.4% in MRR, Hit@1, and Prec@1, respectively.

## introduction taxonomy, a critical knowledge graph with an "isa" relationship, plays a vital role in information retrieval, recommendation systems, and question answering @xcite @xcite @xcite @xcite . however, manual taxonomy enrichment is inefficient and costly due to the constant emergence of new concepts. to address the challenge of incorporating new concepts, taxonomy completion has been † corresponding author. query concepts original taxonomy completed taxonomy . . . insert ? attach ? animal vertebrate domestic animal siamese cat aquarium fish tiger crocodile cat insect animal cat insect siamese cat aquarium fish tiger crocodile vertebrate domestic animal @xmath0, cat, @xmath1, cat, @xmath2, cat, siamese @xmath3, insect, @xmath4, insect, @xmath5. . . introduced, with new concepts either inserted as both parents and children or attached only as children @xcite @xcite . this task goes beyond taxonomy expansion, which primarily treats new concepts as leaf nodes and tends to have limitations in downstream applications @xcite @xcite @xcite . taxonomy completion entails a more comprehensive incorporation of new concepts with two operations: insertion and attachment. for instance, in figure 1 , new query concepts such as cat and insect are added to the existing animal taxonomy. the process requires enumerating all possible candidate positions within the original taxonomy, including existing edges @xmath6, @xmath7, tiger>. each candidate position is then paired with the query concept, and a confidence score is calculated. finally, insect is attached as a child of animal and cat is inserted as a parent of siamese cat and children of domestic animal and vertebrate according to their confidences. recent research on taxonomy enrichment has examined various practical methods @xcite @xcite . nevertheless, all of these approaches embed concepts as vectors in euclidean space, which makes them less capable of modeling the asymmetric relationship ("is-a") in taxonomy. box-taxo @xcite tried to employ box embedding, a representation method that can capture more prosperous and asymmetric relationships like inclusion, disjoint, and proximity among concepts through its geometric properties. however, this method is limited in real-world applications for its reliance only on the volume property, rendering it suitable only for the taxonomy expansion and even incapable of discerning optimal ancestor concepts and handling multiple parents during inference. moreover, methods for taxonomy completion @xcite suffer from using a "pseudo-leaf" as a child node in attachment cases, leading to confusion in the matching. it is attributed that attachment cases often predominate due to leaf nodes' prevalence in real taxonomies. therefore, learning too much about the pseudo-leaf in the attachment cases may reduce the network's perceptual ability for child nodes in the insertion cases. to overcome these limitations, we present a novel framework for taxonomy completion called taxbox, which is the first to apply box embedding to taxonomy completion. this approach adopts a structurally enhanced box decoder, representing concepts as box embeddings @xcite encompassing the information of children, furnishing richer semantics. most importantly, taxbox combines two probabilistic scorers to unify the process of insertion and attachment in the box embedding space and incorporates both the volume and center closeness properties of box embedding. such a design effectively exploits the fine-grained geometric attributes of box embeddings, circumventing the need for a pseudo-leaf and yielding optimal, feasible results during the ranking process. additionally, we propose two novel training objectives, optimizing both box volume and position, and rectifying scorer numerical imbalances. the specific contributions of this paper are outlined as follows: ## related work taxonomy expansion and completion. taxonomy expansion, the process of attaching novel concepts into an existing taxonomy, has evolved over time with various approaches @xcite @xcite @xcite @xcite . although effective, these methods have limitations in addressing real-world applications. thus, zhang et al. (2021) introduced taxonomy completion, a generalization that allows for the insertion of a concept as a parent to existing nodes, generating wider-reaching solutions. subsequent research @xcite @xcite sought to tackle this more challenging version of taxonomy expansion. jiang et al. (2022) incorporated contextual embeddings into input embeddings, leveraging dual lstms to encode ancestor and descendant information (staudemeyer and morris, 2019). meanwhile, zeng et al. (2021) devised a generative strategy that concurrently generates concept names and classifies valid candidate positions. wang et al. (2022) introduced the quadruple evaluation network (qen), which utilized pretrained language models (plm) (devlin et al., 2018; sanh et al., 2019) to augment initial embeddings with semantically rich term representations. arous et al. (2023) learns a positionenhanced node representation through anchor sets to better find the candidate. box embedding. box embedding represents a mapping technique that embeds concepts or objects insertion score box embedding space taxonomy in vector space candidate child subtree candidate parent subtree candidate parent subtree candidate child subtree query node query node graph aggre--gation module mapping from vector space to box embedding space attachment score query box candidate child box candidate parent box insertion score: pr( | ) * pr( | ) * nrd( , , ) attachment score: pr( | ) * nrd( , ) n1 n3 q n5 n6 n2 n4 n0 n1 q n3 n2 n4 (a) (b) (c) (d) n0 ranking loss box constraint loss classi--fication loss structurally enhanced box decoder query box projector candidate box projector x x ranking process where to place among ? ## preliminary box embedding @xcite refers to a mapping that represents a concept or object as a hyperplane box. a box @xmath8, x m ] is a hyperrectangle such that x @xmath9,@xmath10,i holds for each axis @xmath11, 2, ..., d}. the center of box embedding is formulated as: there are two important operations: intersection and volume which are required for the calculation of the conditional probability of boxes' containment. given two box embedding @xmath12, x m ], @xmath13, y m ], the intersection of them is defined as follows: in this section, we elaborate on the proposed taxbox framework, as shown in figure 2 . we begin by defining the problem in section 4.1. then, in section 4.2, we introduce the structurally enhanced box decoder, which maps concepts into box embeddings with hierarchical information enhanced. section 4.3 focuses on the discussion of two probabilistic scorers that evaluate the query and candidate boxes, providing attachment and insertion scores. finally, in section 4.4, we elucidate the learning objectives that contribute to improved optimization of box decoding and scorer balancing. ## conclusion in this study, we present taxbox, a novel framework for taxonomy completion using box embeddings. incorporating restricted box constraint loss, dynamic ranking loss, and two probabilistic scorers for attachment and insertion, taxbox employs a structurally enhanced box decoder, mitigating the need for pseudo leaves. experiments on six realworld datasets demonstrate its effectiveness and performance. future research could refine scorers without numerical imbalance and explore postprocessing measures like reranking with llm.