V ee A lign: Multifaceted Context Representation Using Dual Attention for Ontology Alignment (2021)

Abstract: Ontology Alignment is an important research problem applied to various fields such as data integration, data transfer, data preparation, etc. State-of-the-art (SOTA) Ontology Alignment systems typically use naive domain-dependent approaches with handcrafted rules or domain-specific architectures, making them unscalable and inefficient. In this work, we propose VeeAlign, a Deep Learning based model that uses a novel dual-attention mechanism to compute the contextualized representation of a concept which, in turn, is used to discover alignments. By doing this, not only is our approach able to exploit both syntactic and semantic information encoded in ontologies, it is also, by design, flexible and scalable to different domains with minimal effort. We evaluate our model on four different datasets from different domains and languages, and establish its superiority through these results as well as detailed ablation studies. The code and datasets used are available at https://github.com/Remorax/VeeAlign .

## introduction ontology alignment is the task of establishing correspondences between semantically related elements (i.e. classes and properties) from different ontologies. it is useful for many applications, particularly for data integration and data migration. the problem has been extensively studied in the past decade, and the solutions have ranged from simple rule based systems @xcite to the ones incorporating external knowledge @xcite , as well as the most recent ones that use sophisticated deep learning based systems @xcite @xcite . a common limitation of these systems is their inability to generalize to new data -rule based systems are based on handcrafted rules which may not cover all data scenarios in all datasets. on the other hand, the deep learning (dl) based systems @xcite proposed so far, often have strong dependencies on domain-specific external knowledge bases such as lexicons and thesauri. moreover, they also underperform as compared to their rule-based counterparts. one of the primary reasons for this and in fact, also the dependency of dl architectures on external background knowledge is the lack of sufficient, usable training data. ground truth alignments are typically very scarce in number, especially for smaller ontologies, making supervised training difficult. moreover, the small number of "positive" alignment pairs as compared to the "negative" ones (i.e entity pairs that contain an alignment versus those that do not) leads to a class imbalance and further adds to the difficulties of supervised learning based solely on the reference alignments. for example, the conference dataset @xcite , used for experimentation in this paper, has 305 positive and 122588 negative alignments. as a result of both sparsity and class imbalance, even moderately complex dl architectures (that contain only a few parameters) overfit and therefore, perform poorly. given these challenges and the weaknesses of the previous approaches, our goal in this paper is two fold: a) to build a generic, domain-independent model that leverages the intrinsic semantic and structural information encoded in ontologies with no requirement of external, domain-specific knowledge and b) a model that uses a parametrically-light architecture to strike the right balance between the model expressivity (uses training data well) and model complexity (does not overfit). despite significant research, ontology alignment still remains a challenging task. figure 1 provides an illustration highlighting this challenge. the task is to determine alignment between the concept attendee in ontology-1 and the concept listener in ontology-2. current approaches that com-figure 1 : an example illustrating concept alignment and dependency on the surrounding context pute concept similarity or context similarity at the label level will fail to capture the alignment between these two concepts, since these concepts do not have high lexical or even semantic similarity. while there is a common concept (i.e. person) in the neighbouring context of both concepts, there are several other concepts in their respective contexts that are not similar. this example shows that not only is it important to consider neighbouring concepts, it is also important to model them in such a way that while computing similarity, the relevant neighbouring concepts have higher weights than the irrelevant ones. in this particular example, ancestor nodes should be given higher weights than children and concepts connected by object and datatype properties respectively. driven by this intuition and the need to address the limitations of the current approaches, this paper presents a novel ontology alignment system, referred to as veealign foot_0 , that exploits both syntactic (context surrounding the concepts) as well as semantic (label-based similarity) aspects of an ontology to compute alignment. we present a novel way of modelling context, where the context is split into multiple facets based on the type of contextual concepts it contains foot_1 . more specifically, we split the context into four facets, where each facet contains contextual concepts that have a different relationship with the central concept. this includes facets containing ancestors, children, contextual concepts connected by object properties and those connected by datatype properties respectively. such a multi-faceted context, however, poses a new challenge because not all of these facets contribute equally in the relatedness of the concepts. more-oever, each of these facets consist of paths which, in turn, are composed of nodes. therefore, the challenge here lies in developing a mechanism that weighs the relevant parts of the context (be it facets, or paths, or nodes, in increasing order of granularity), more than the less relevant ones. in order to deal with this challenge, we propose a novel dual attention mechanism comprising of a) path-level attention and b) node-level attention. the path-level attention combines all the path representations in a facet based on their importance (to the alignment of the central concept) to compute an aggregated path representation. similarly, the node-level attention combines all node representations in the aggregated path based on their importance to return an aggregated node representation. we apply these attention mechanisms sequentially on each facet to obtain its representation. the final context representation is obtained using a weighted sum of these facet representations, where the weights are proportional to their importance. we also note that the term "dual attention" is rather overloaded in prior literature, so we distinguish its usage in this work from that of prior works @xcite @xcite . in these works, dual attention has either referred to the parallel application of separate attention mechanisms or simultaneous application on two or more different features. our usage of dual attention lies in the sequential application of attention on the same features, first at the path level and then at the node level. the main contributions of this paper are, therefore, as follows: related work mainly spans two broad areas: ontology and knowledge graph (kg) alignment. ontology alignment there is a large body of work on ontology alignment @xcite @xcite , primarily driven by the ontology alignment evaluation initiative (oaei). oaei has been organising ontology alignment challenges since 2004 where multiple datasets belonging to different domains are released along with a public evaluation platform to evaluate different systems. among all systems submitted to the challenge, agreementmakerlight (aml) @xcite and logmap2 (jiménez-ruiz et al., 2020) have consistently outperformed other systems for the past several years. despite their high performance on oaei datasets, these systems have two identifiable weaknesses: a) for comparing concept labels, string similarity measures are used which do not address semantic similarity, and b) while these systems use neighbour alignment as a guiding factor for calculating alignments, they do not attempt to compute context similarity using a rigorous neighbourhood representation. in addition, while they have been engineered over the years to give the best performance on oaei datasets, their performance is less impressive on non-oaei datasets (refer section 4), indicating poor scalability. wiktionary @xcite is another top performing ontology alignment system, especially in the multilingual space. however, it relies heavily on the wiktionary knowledge base which again presents scalability issues. the few systems proposed outside oaei, particularly deep learning based systems, have strong dependencies on background knowledge sources, thus limiting their use. a good example of this is ontoemma @xcite , a neural network based ontology alignment system for the biomedical domain. it enriches ontological entities with aliases from the ontology, definitions from wikipedia, and usage contexts from domainspecific medical papers; and uses this additional information for ontology alignment. similarly, deepalign @xcite also requires synonyms and antonyms extracted from external sources such as wordnet & ppdb in order to refine word vectors using synonymy and antonymy constraints, which are then used for alignment. fi-nally, the recently proposed snn-om technique @xcite uses, among others, wordnetdependent similarity features for aligning sensor ontologies. a recent work attempting to combine the traditional and modern dl-based approaches proposed a machine learning-based extension to traditional ontology alignment systems, using distant supervision, ontology embeddings and siamese neural networks @xcite . while this was useful in incorporating richer semantics and outperforming the traditional systems, it was still reported to underperform as compared to veealign. ## approach in this section, we describe our system veealign including its underlying dual attention mechanism. ## experiments this section provides experiment details, i.e. the datasets used, baseline models, experimental setup, results, and their analysis including ablation study. ## conclusion this paper presents a general purpose ontology alignment system that does not require any external or background knowledge. it is based on a light and robust dl architecture that utilises a novel multifaceted context representation approach for exploiting the structural aspects of an ontology. a novel dual attention mechanism is proposed for focusing on the parts of the context that are most crucial for alignment. our experiments on 4 different datasets from 2 different languages and 3 different domains show that the proposed method outperforms the sota methods by a significant margin. ablation study examines the effect of context splitting and dual attention, and validate them as the right factors behind the performance improvement. bag_of_neighbours is what determines how we represent the concept's one-hop neighbours. we have two options for this, either consider them as a path of length one, or bag them together. in the case of the former, we further experiment over the optimal number of paths by using a wide range of values from 1 to the maximum number of paths possible, which is 38 for conference, 16 for lebensmittel, 12 for freizeit and 49 for the web directory datasets respectively. then, to determine optimal length of path, we experiment from 1 to maximum possible length of the path, which is 8 for conference, 7 for lebensmittel, 6 for freizeit and 8 for web-directory. overall, our model has approximately 307k trainable parameters. if we consider one-hop neighbours (children, datatype property neighbours and object property neighbours) as a bag, we get paths that are longer in length but fewer in number. in this case, the maximum number of paths are 2 for conference and 1 for the other 3 datasets. we take all the paths available as maximum number of paths, while for path length, we experiment from 1 to maximum length available which is 38 for conference, 16 for lebensmittel, 12 for freizeit and 49 for webdirectory. another parameter denotes how we aggregate different path representations. there are two ways of doing this aggregation, by either applying a weighted sum or passing the path weights through a max-pool layer.