# DAEOM: A Deep Attentional Embedding Approach for Biomedical Ontology Matching
## 2 Method
### Ontology Attentional Encoder  
The architecture of the ontology attentional encoder is shown in Figure 2. We first assign each sequence of terminological description associated with concept from source and target ontologies (\(O_{\text{s}}\) and \(O_{\text{t}}\)) to low-dimensional representations (\(X_{\text{s}}\) and \(X_{\text{t}}\)) such that semantically similar sequences are close. Then we develop a siamese graph attentional autoencoder, which effectively integrates both network structure (\(A_{\text{s}}\) and \(A_{\text{t}}\)) and terminological description (\(X_{\text{s}}\) and \(X_{\text{t}}\)) to learn latent representations \(Z_{\text{s}}\) and \(Z_{\text{t}}\). The encoder exploits both network structure and terminological description with a graph attention network, and multiple layers of encoders are stacked to build a deep architecture for  
Figure 1: The conceptual framework of Deep Attentional Embedded Ontology Matching (DAEOM). embedding learning. On the other side, the decoder reconstructs the topological network information and manipulates the latent graph representation. **Terminological Description Embedding. ** In this paper, we present Terminological-BERT (TBERT), a modification of the Bert [31] network using siamese networks that is able to derive semantically meaningful terminological description embeddings. TBERT adds a pooling operation to the output of BERT to derive a fixed sized terminological description embedding. The pooling strategy is computing the mean of all output vectors (MEAN- strategy). In order to fine-tune BERT, we create siamese networks to update the weights such that the produced terminological description embeddings (\(X_{s}\) and \(X_{t}\)) are semantically meaningful and can be compared with a similarity metric. We employ a similarity metric, described in [17] (listed as Weizhuo Li), to guide the optimizing procedure. Given input \(\{u,v\}\), where \(u\) and \(v\) are the mapping pair. TBERT seek value of the parameter such that the symmetric similarity metric is large if \(u\) and \(v\) have the same semantic, and small if they mean different semantics. The similarity metric function \(sim_{tb}\left(u,v\right)\) that measures the semantic relatedness between mapping pair \(\left(u,v\right)\) can be defined as:  
\[sim_{tb}\left(u,v\right)=2\cdot\frac{1}{1+e^{\left(\|f\left(u\right)-f\left(v \right)\|\right)_{2}}}, \tag{1}\]  
where \(||\cdot||_{2}\) is the L2-norm, and \(f\left(u\right)\) and \(f\left(v\right)\) are the generated terminological description embeddings. We define a loss function based on cross-entropy to optimize the vector representations of terminological description. The loss function is defined as follows:  
\[L_{tb}=-\sum_{\left(u,v\right)\in M_{intra}}\log sim_{tb}\left(u,v\right)- \sum_{\left(u,v\right)\in M^{{}^{\prime}}_{intra}}\log\left(1-sim_{tb}\left( u,v\right)\right), \tag{2}\]  
where \(M_{intra}\) is a set of positive mappings and \(M^{{}^{\prime}}_{intra}\) is a set of negative mappings (see Section 3.2 for details). **Jointly Embedding. ** After producing all the terminological description embeddings (\(X_{s}\) and \(X_{t}\)) of concepts from source and target ontologies, we present structural-GAT (SGAT), a variant of the graph attention network (GAT) [28] using siamese networks that is able to represent both network structure (\(A_{s}\) and \(A_{t}\)) and terminological description (\(X_{s}\) and \(X_{t}\)) in a unified framework. The idea is to learn hidden representations of each concept that allow for assigning different importance to different concepts within a neighborhood while dealing with different sized neighborhoods to aggregate feature  
Figure 2: Architecture of ontology attentional encoder. information.