# Schema First! Learn Versatile Knowledge Graph Embeddings by Capturing Semantics with MASCHInE

Nicolas Hubert, Heiko Paulheim, Pierre Monnin, Armelle Brun, Davy Monticolo

## ► To cite this version:

Nicolas Hubert, Heiko Paulheim, Pierre Monnin, Armelle Brun, Davy Monticolo. Schema First! Learn Versatile Knowledge Graph Embeddings by Capturing Semantics with MASCHInE. K-CAP 2023 - Knowledge Capture Conference 2023, Dec 2023, Pensacola, United States. pp.188-196, 10.1145/3587259.3627550 . hal-04344873

HAL Id: hal-04344873

https://inria.hal.science/hal-04344873v1

Submitted on 14 Dec 2023

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Schema First! Learn Versatile Knowledge Graph Embeddings by
Capturing Semantics with MASCHInE

Nicolas Hubert

Université de Lorraine, ERPI,
Université de Lorraine, CNRS, LORIA

Nancy, France

nicolas.hubert@univ-lorraine.fr

Heiko Paulheim

University of Mannheim
Mannheim, Germany

heiko.paulheim@uni-mannheim.de

Pierre Monnin

Univ. Côte d'Azur, Inria, CNRS, I3S
Sophia Antipolis, France

pierre.monnin@inria.fr

Armelle Brun

Université de Lorraine, CNRS, LORIA

Nancy, France

armelle.brun@loria.fr

Davy Monticolo

Université de Lorraine, ERPI

Nancy, France

davy.monticolo@univ-lorraine.fr

## ABSTRACT

Knowledge graph embedding models (KGEMs) have gained considerable traction in recent years. These models learn a vector representation of knowledge graph entities and relations, a.k.a. knowledge graph embeddings (KGEs). Learning versatile KGEs is desirable as it makes them useful for a broad range of tasks. However, KGEMs are usually trained for a specific task, which makes their embeddings task-dependent. In parallel, the widespread assumption that KGEMs actually create a semantic representation of the underlying entities and relations (e.g., project similar entities closer than dissimilar ones) has been challenged. In this work, we design heuristics for generating *protographs* – small, modified versions of a KG that leverage RDF/S information. The learnt protograph-based embeddings are meant to encapsulate the semantics of a KG, and can be leveraged in learning KGEs that, in turn, also better capture semantics. Extensive experiments on various evaluation benchmarks demonstrate the soundness of this approach, which we call *Modular and Agnostic SCHema-based Integration of protograph Embeddings (MASCHInE)*. In particular, MASCHInE helps produce more versatile KGEs that yield substantially better performance for entity clustering and node classification tasks. For link prediction, using MASCHInE substantially increases the number of semantically valid predictions with equivalent rank-based performance.

## CCS CONCEPTS

* **Computing methodologies** → *Artificial intelligence; Knowledge representation and reasoning;**
* **Information systems** → *World Wide Web*.

## KEYWORDS

Knowledge Graph Embeddings, Schema-based Representation Learning, Link Prediction, Entity Clustering, Node Classification

## 1 INTRODUCTION

Knowledge Graphs (KGs) such as DBpedia and YAGO represent facts as triples (*h*, *r*, *t*) formed by a head *h* and a tail *t* linked by a semantic relation *r* which qualifies the nature of their relationship. Typical learning problems with KGs are Link Prediction (LP), Entity Clustering (EC), and Node Classification (NC).

In most cases, these problems are addressed using Knowledge graph embedding models (KGEMs), which generate vector representations for entities and relations of a KG, a.k.a. Knowledge Graph Embeddings (KGEs). While generating dense and numerical vectors for entities and relations, KGEMs are expected to produce KGEs that retain the underlying semantics of the KG [21]. This widespread assumption that KGEMs create a semantic representation of the underlying entities and relations (i.e., project similar entities closer than dissimilar ones) has been challenged recently [14]. Their semantic capabilities presumably inherited from the rich information contained in the KG is neither fully satisfying nor consistent for tasks such as concept clustering [14]. In this work, we claim that this is because schemas (e.g. RDF/S and OWL) are often overlooked as an interesting source of information for improving embeddings.

Injecting schema-based information into the training phase could help enhancing the semantic awareness of KGEMs [13]. This way, better performance can be reached using available ontological information, without introducing major overhead in models' complexity. A few works incorporate such schemas when training KGEM, whether it is in the loss function [7, 13], in the negative sampling procedure [15, 16], or in the model representations [6, 23]. All of those methods are usually bound to just one KGEM and therefore not universally applicable.

Another line of research that is less explored focuses on preprocessing graphs prior to encoding their components [5, 18], rather than focusing on the representation capability of the KGEM itself. This idea of summarizing [4, 9] or coarsening [18] RDF<sup>1</sup> graphs proves useful in many circumstances. The intended goals and expected benefits are multiple: scaling up training [10], initializing embeddings with more robust vector representations learnt on the coarsened graph [18], and improving predictive performance in KG-based downstream tasks [22]. It is worth a reminder that several heuristics exist for coarsening or summarizing graphs [4]. All these approaches, however, use statistics over the graphs, instead of considering schema information. In contrast, our work focuses on using RDF/S information to generate an abstraction of the KG that encapsulates general knowledge about how entities and relations interact based on entity types, and relation domains and ranges. We call such an abstraction a *protograph* (see left part of Figure 2).

<footnote n="1">¹Resource Description Framework. See https://www.w3.org/RDF</footnote>
If considering schema-based information is usually done with the goal of improving results w.r.t. a given task such as LP [7, 13], it would be worth investigating whether the learnt embeddings actually have stronger semantic capabilities [13, 14]. If so, they are expected to provide better performance with regards to other tasks such as entity classification and clustering. In this work, we propose an approach named MASCHInE that leverages RDF/S information in a model-agnostic way, making it applicable to arbitrary KGEMs. Specifically, we create and embed *protographs* from a KG schema. The protograph embeddings are then used to initialize the actual KGEs, and training starts on these pre-trained KGEs. Finally, we assess the versatility of these KGEs by comparing results with a vanilla, traditional learning approach.

The main contributions of our work are summarized as follows:

* We devise two heuristics for building protographs that aim at encapsulating concise and meaningful information derived from the schema that underpins a KG.

* To the best of our knowledge, we propose the first work that studies the potential of pre-training embeddings with a protograph-assisted approach based on a schema, and subsequently use these embeddings for multiple tasks, e.g. link prediction, entity clustering and node classification.

* Through extensive experiments on several benchmarks, we demonstrate the value of using schema-based protographs as a means to generate versatile, multi-purpose embeddings.

The remainder of the paper is structured as follows. Related work is presented in Section 2. In Section 3, we detail our approach for building protographs and how they fit into our approach MASCHInE. In Section 4, the potential usefulness of embeddings learnt with MASCHInE is discussed, and experiments w.r.t. several benchmarks are carried out. Lastly, Section 5 sums up the main findings and outlines promising future research directions.

## 2 RELATED WORK

**Knowledge graph embeddings.** KGEMs have gained significant attention in recent years due to their ability to represent structured knowledge in a continuous vector space. The seminal translational model TransE [3] represents entities and relations as low-dimensional vectors and defines the relationship between a head, a relation, and a tail using a translation operation in the embedding space. Most of the KGEMs subsequently proposed aim at addressing its limitations and improve the representation expressiveness of KGEs [21], such as DistMult [24], ComplEx [20], ConvE [8], and TuckER [1]. Embeddings learnt by these KGEMs demonstrated potential applicability in tasks such as LP [21].

**Schema-based representation learning.** Many knowledge graphs are backed by a more or less expressive schema defining which classes of entities exist and by which relations they are allowed to interact, among others. Recent approaches in KG-based learning leverage the schema as a supplementary source of information to learn higher-quality embeddings for the task at hand. A common approach is to alter the negative sampling procedure in order to produce more realistic negative triples. For instance, type-constrained negative sampling (TCNS) [16] replaces the head or the tail with a random entity belonging to the same type as the ground-truth

entity. The model itself can be adapted to take schema-based information into account. In TaRP [6], type and entity-level information are simultaneously considered and encoded as prior probabilities and likelihoods of relations, respectively. Altering the loss function with schema-based information is another line of research. In [7], background ontological information is injected in the pairwise hinge loss function as constraints. Hubert et al. [13] leverage domains and ranges of relations, and propose semantic-driven loss functions that extend the most frequently used loss functions in LP.

## 3 PROPOSED APPROACH

In this section, we detail the two heuristics we propose for building protographs using schema-based information. Next, we provide some perspective on the overall proposed approach MASCHInE.

### 3.1 Protographs

Most KGs are underpinned by a schema, e.g., DBpedia, Wikidata, and YAGO. Generating a protograph using schema-based information to improve KGEM performance with respect to a given task is an under-explored avenue.

The protographs we build leverage RDF/S information and contain an entity $p_C$ for each class $C$, and the same set of relations as the KG. The first design principle consists in adding an edge for each relation with domain and range restrictions, i.e., given the two axioms $domain(r, C_i)$, $range(r, C_j)$, we add a triple $(p_{C_i}, r, p_{C_j})$ to the protograph<fnref n="2" />. We refer to this strategy as **P1**.

The protograph P1 contains as many triples as there are relations with both a domain and range<fnref n="3" /> in the KG. This may lead to relatively small-sized protographs from which there is little data for the KGEM to learn from. In addition, relation domains and ranges can refer to generic classes. If entities are typed in a fine-grained way – i.e. with more specific classes – such classes will be absent from the protograph. As such, we propose a second design principle. Assuming we observe the following axioms $domain(r, C_i)$, $range(r, C_j)$, $subclassOf(C'_i, C_i)$, $subclassOf(C'_j, C_j)$, we add the following triples to the protograph: $(p_{C_i}, r, p_{C_j})$, $(p_{C'_i}, r, p_{C'_j})$, $(p_{C_i}, r, p_{C'_i})$. In other words: for each subclass of the class appearing in the domain or range of a relation, an additional triple is created.

We refer to this protograph as **P2**. Figure 1 illustrates this process for a triple extracted from DBpedia.

P2 usually contains more triples than P1, at the possible expense of containing dubious ones. To illustrate, the DBpedia ontology contains the triples: $domain(currentWorldChampion, Sport)$, $range(currentWorldChampion, Agent)$, and $subclassOf(Family, Agent)$. They lead to the triple $(P_{Sport}, currentWorldChampion, P_{Family})$ for which the relevancy and correctness are questionable. Conceptually, with P2, we accept some noise to be introduced in the protograph for the benefit of more triples being generated. However, the generation of dubious triples is limited by two factors: firstly, when adding triples that derive from an original triple (i.e. containing the domain and range of the relation), we only consider direct subclasses – in contrast to transitive subclasses. Secondly, we fix one side of the triple and generate as many new triples as there

<footnote n="2">²Note that $C_i$ and $C_j$ might be the same, so there can be cycles in the protograph.</footnote>

<footnote n="3">³In the KGs used in this paper, all relations have an explicit domain and range.</footnote>
<figure><img src="image_2.png" /></figure>

<table><caption>Figure 1: Excerpt from DBpedia class hierarchy (left) and the generated prototriples (right) according to P2. The full URI names have been shortened due to space limitations. The prototriple highlighted in green (right) comes from the schema, whereas the following triples are generated by fixing the domain (resp. range) of `db0:headquarter` and replacing the class in the range (resp. domain) with each of its direct subclasses.</caption><thead><tr><th></th><th></th><th></th><th></th></tr></thead><tbody><tr><td>h</td><td>r</td><td>t</td></tr><tr><td>Organization</td><td>headquarter</td><td>PopulatedPlace</td></tr><tr><td>Organization</td><td>headquarter</td><td>Continent</td></tr><tr><td>Organization</td><td>headquarter</td><td>Country</td></tr><tr><td>Organization</td><td>headquarter</td><td>Street</td></tr><tr><td>Company</td><td>headquarter</td><td>PopulatedPlace</td></tr><tr><td>SportsClub</td><td>headquarter</td><td>PopulatedPlace</td></tr><tr><td>PoliticalParty</td><td>headquarter</td><td>PopulatedPlace</td></tr><tr><td></td><td colspan="2">Prototriples</td></tr></tbody></table>

are direct subclasses of the class of the other side, i.e., we avoid generating triples $(p_{C'_i} r, p_{C'_j})$.

## 3.2 MASCHInE

The proposed MASCHInE approach allows for generating pre-trained embeddings based on a protograph, to be further fine-tuned on the initial KG. MASCHInE is intended to produce more versatile KGEs that can be successfully used in various tasks. Conceptually, MASCHInE is made of four different modules that are depicted in Figure 2 and further detailed below.

(a) **Protograph building** is the preliminary step that generates a second source of data to learn from. In particular, schema-based information is leveraged to create a protograph that encapsulates more general knowledge about how entities and relations interact based on ontological constraints such as entity types, relation domains and ranges, etc. A mapping dictionary is created to pair entities between the KG and the protograph (the set of relations in the KG and the protograph are identical). In this work, each KG entity maps to its (potentially multiple) most specific classes – represented as entities in the protograph. In Section 3.1, we presented two heuristics for building protographs with such constraints. Other possibilities exist and we leave them for future work.

(b) **Protograph KGE learning** is the task of learning KGEs on the protograph itself. This can be achieved regardless of the KGEM at hand and is therefore model-agnostic<fnref n="4" />.

(c) **Instance KGE initialization** consists in transferring protograph embeddings to the corresponding entity and relation embeddings of the KG. To do so, we assign each entity in the KG the embedding vector of its corresponding protograph entity using the dictionary created in step (a) (i.e., its most specific class). When an entity maps to multiple protograph entities (i.e. when there exist multiple most specific classes for a given entity), the protograph embeddings of its most specific classes are averaged and the resulting embedding is transferred to the entity.

(d) **Fine-tuning** is performed on the entity KGEs. At this stage, embeddings learnt on the protograph could still intervene in the

fine-tuning procedure. In this work, however, we stick to plain entity KGE initialization, i.e. protograph embeddings are frozen after being transferred, and do not interact with entity KGEs anymore.

## 4 EXPERIMENTS

In our experiments, we apply the MASCHInE approach presented in Section 3. We experiment with five popular KGEMs. We first train on protographs, and then use the protograph embeddings to initialize entity and relation embeddings in the KG. Training starts on the KG and the fine-tuned KGEs are ultimately assessed in three tasks, namely LP, EC, and NC. For each task, we compare the vanilla setting (V), i.e., directly training the respective KGEM on the KG, with the MASCHInE approach based on P1 (resp. P2). Datasets, optimal hyperparameters, scripts, and pre-trained embeddings are publicly released<fnref n="5" />. In the following, protographs, KG entities and relations are encoded using five mainstream KGEMs: TransE [3], DistMult [24], ComplEx [20], ConvE [8], and TuckER [1].

### 4.1 Link Prediction

<table><caption>Table 1: KG and protograph characteristics. Column headers from left to right: number of entities, relations, and triples.</caption><thead><tr><th>Dataset</th><th></th><th>|E|</th><th>|R|</th><th>|T|</th></tr></thead><tbody><tr><td rowspan="3">YAGO14k</td><td>KG</td><td>14, 178</td><td>37</td><td>19, 183</td></tr><tr><td>P1</td><td>22</td><td>37</td><td>37</td></tr><tr><td>P2</td><td>590</td><td>37</td><td>4, 959</td></tr><tr><td rowspan="3">FB15k187</td><td>KG</td><td>14, 305</td><td>187</td><td>278, 436</td></tr><tr><td>P1</td><td>138</td><td>187</td><td>187</td></tr><tr><td>P2</td><td>138</td><td>187</td><td>187</td></tr><tr><td rowspan="3">DBpedia77k</td><td>KG</td><td>76, 651</td><td>150</td><td>190, 028</td></tr><tr><td>P1</td><td>55</td><td>150</td><td>150</td></tr><tr><td>P2</td><td>186</td><td>150</td><td>3, 210</td></tr></tbody></table>

**Datasets.** Due to the need for schema-defined KGs, the following three benchmark datasets are considered: YAGO14k, FB15k187, and DBpedia77k [13]. In particular, all entities are typed and relations have both a defined domain and range. Table 1 provides statistics for

<footnote n="4">⁴In our experiments, the same KGEM is used for both pre-training over the protograph and fine-tuning embeddings on the KG.</footnote>

<footnote n="5">⁵https://github.com/nicolas-hbt/versatile-embeddings</footnote>
<figure><img src="image_3.png" /><figcaption>Figure 2: Overview of MASCHInE. The overall training approach is based on several modules that each features different possibilities. First, the protograph is built following predefined design principles (a). Protograph embeddings are learnt (b) before being transferred to the corresponding entities and relations of the KG (c). Next, training starts on the KG (d). During this step, embeddings learnt on the protograph can still interact in the fine-tuning procedure (purple dotted arrow). Finally, the fine-tuned KGEs can be used in several tasks.</figcaption></figure>

<table><caption>Table 2: Link prediction results on YAGO14k, FB15k187, and DBpedia77k using KGEMs trained without protograph (V), with P1, and P2. For a given dataset and KGEM, comparisons are made between the three settings, and best results are in bold fonts. Hits@K and Sem@K are abbreviated as H@K and S@K, respectively.</caption><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="5">YAGO14k</th><th colspan="5">FB15k187</th><th colspan="5">DBpedia77k</th></tr><tr><th>MRR</th><th>H@3</th><th>H@10</th><th>S@3</th><th>S@10</th><th>MRR</th><th>H@3</th><th>H@10</th><th>S@3</th><th>S@10</th><th>MRR</th><th>H@3</th><th>H@10</th><th>S@3</th><th>S@10</th></tr></thead><tbody><tr><td rowspan="3">TransE</td><td>V</td><td><strong>0.766</strong></td><td><strong>0.806</strong></td><td><strong>0.885</strong></td><td>0.987</td><td>0.971</td><td><strong>0.232</strong></td><td><strong>0.259</strong></td><td><strong>0.430</strong></td><td><strong>0.970</strong></td><td>0.958</td><td><strong>0.236</strong></td><td><strong>0.285</strong></td><td><strong>0.405</strong></td><td>0.983</td><td>0.976</td></tr><tr><td>P1</td><td>0.746</td><td>0.791</td><td>0.871</td><td>0.994</td><td>0.993</td><td>0.227</td><td>0.254</td><td>0.425</td><td><strong>0.970</strong></td><td><strong>0.959</strong></td><td>0.217</td><td>0.260</td><td>0.382</td><td>0.985</td><td>0.980</td></tr><tr><td>P2</td><td>0.700</td><td>0.762</td><td>0.854</td><td><strong>1.000</strong></td><td><strong>0.999</strong></td><td>0.227</td><td>0.254</td><td>0.425</td><td><strong>0.970</strong></td><td><strong>0.959</strong></td><td>0.216</td><td>0.260</td><td>0.380</td><td><strong>0.987</strong></td><td><strong>0.982</strong></td></tr><tr><td rowspan="3">DistMult</td><td>V</td><td><strong>0.899</strong></td><td><strong>0.916</strong></td><td><strong>0.926</strong></td><td>0.658</td><td>0.490</td><td><strong>0.228</strong></td><td><strong>0.256</strong></td><td><strong>0.412</strong></td><td>0.985</td><td>0.969</td><td><strong>0.282</strong></td><td><strong>0.308</strong></td><td><strong>0.391</strong></td><td>0.845</td><td>0.804</td></tr><tr><td>P1</td><td>0.164</td><td>0.169</td><td>0.314</td><td><strong>1.000</strong></td><td><strong>1.000</strong></td><td>0.206</td><td>0.228</td><td>0.394</td><td><strong>0.988</strong></td><td><strong>0.980</strong></td><td>0.194</td><td>0.219</td><td>0.303</td><td>0.928</td><td>0.939</td></tr><tr><td>P2</td><td>0.289</td><td>0.325</td><td>0.511</td><td>0.999</td><td>0.999</td><td>0.206</td><td>0.228</td><td>0.394</td><td><strong>0.988</strong></td><td><strong>0.980</strong></td><td>0.190</td><td>0.207</td><td>0.287</td><td><strong>0.937</strong></td><td><strong>0.947</strong></td></tr><tr><td rowspan="3">ComplEx</td><td>V</td><td><strong>0.923</strong></td><td><strong>0.930</strong></td><td>0.933</td><td>0.723</td><td>0.587</td><td><strong>0.203</strong></td><td><strong>0.223</strong></td><td><strong>0.399</strong></td><td>0.942</td><td>0.923</td><td><strong>0.286</strong></td><td><strong>0.317</strong></td><td><strong>0.394</strong></td><td>0.823</td><td>0.754</td></tr><tr><td>P1</td><td>0.897</td><td>0.916</td><td>0.932</td><td>0.907</td><td>0.848</td><td>0.185</td><td>0.201</td><td>0.371</td><td>0.922</td><td>0.900</td><td>0.209</td><td>0.226</td><td>0.302</td><td>0.856</td><td>0.839</td></tr><tr><td>P2</td><td>0.914</td><td><strong>0.930</strong></td><td><strong>0.934</strong></td><td><strong>0.914</strong></td><td><strong>0.854</strong></td><td>0.185</td><td>0.201</td><td>0.371</td><td>0.922</td><td>0.900</td><td>0.209</td><td>0.225</td><td>0.306</td><td><strong>0.913</strong></td><td><strong>0.890</strong></td></tr><tr><td rowspan="3">ConvE</td><td>V</td><td><strong>0.934</strong></td><td><strong>0.936</strong></td><td><strong>0.940</strong></td><td>0.858</td><td>0.824</td><td>0.265</td><td>0.297</td><td>0.463</td><td><strong>0.979</strong></td><td><strong>0.971</strong></td><td>0.227</td><td><strong>0.259</strong></td><td>0.352</td><td>0.897</td><td>0.889</td></tr><tr><td>P1</td><td>0.927</td><td>0.930</td><td>0.935</td><td>0.882</td><td>0.845</td><td>0.272</td><td>0.301</td><td><strong>0.470</strong></td><td>0.977</td><td>0.970</td><td>0.227</td><td>0.256</td><td>0.352</td><td>0.932</td><td>0.932</td></tr><tr><td>P2</td><td>0.931</td><td>0.931</td><td>0.938</td><td><strong>0.935</strong></td><td><strong>0.922</strong></td><td><strong>0.272</strong></td><td><strong>0.301</strong></td><td><strong>0.470</strong></td><td>0.977</td><td><strong>0.971</strong></td><td><strong>0.229</strong></td><td><strong>0.259</strong></td><td><strong>0.356</strong></td><td><strong>0.943</strong></td><td><strong>0.942</strong></td></tr><tr><td rowspan="3">TuckER</td><td>V</td><td>0.919</td><td>0.931</td><td><strong>0.942</strong></td><td>0.899</td><td>0.821</td><td>0.282</td><td>0.311</td><td>0.467</td><td><strong>0.996</strong></td><td><strong>0.991</strong></td><td>0.274</td><td>0.305</td><td>0.401</td><td>0.988</td><td>0.985</td></tr><tr><td>P1</td><td><strong>0.922</strong></td><td><strong>0.933</strong></td><td>0.941</td><td>0.951</td><td>0.925</td><td><strong>0.286</strong></td><td><strong>0.317</strong></td><td><strong>0.472</strong></td><td>0.995</td><td>0.985</td><td>0.277</td><td><strong>0.308</strong></td><td>0.402</td><td>0.987</td><td>0.984</td></tr><tr><td>P2</td><td>0.918</td><td>0.930</td><td>0.941</td><td><strong>0.971</strong></td><td><strong>0.939</strong></td><td><strong>0.286</strong></td><td><strong>0.317</strong></td><td><strong>0.472</strong></td><td>0.995</td><td>0.985</td><td>0.275</td><td>0.304</td><td><strong>0.403</strong></td><td><strong>0.989</strong></td><td><strong>0.986</strong></td></tr></tbody></table>

these datasets and their respective protographs P1 and P2. Note that protograph entities are classes from the original KG. Full dataset description is provided at: https://github.com/nicolas-hbt/versatile-embeddings#datasets.

**Evaluation metrics.** This section is motivated by evaluating the fine-tuned KGEs for LP. Performance is assessed w.r.t. Mean Reciprocal Rank (MRR), Hits@K, and Sem@K [11, 12]. The latter

accounts for the proportion of triples whose predicted head (resp. tail) belongs to the domain (resp. range) of the relation.

**Implementation details.** KGEMs were implemented in PyTorch. In accordance with [13], KGEMs are trained during 400 epochs. For TransE, DistMult, and ComplEx, uniform random negative sampling [3] was used to pair each train triple with one negative counterpart. ConvE and TuckER are trained using 1-N scoring [8]. In this work, we stick with the loss function originally used for
each KGEM. Evaluation is performed every 10 epochs and the best model w.r.t. MRR on the validation set is saved for performance assessment on the test set. The aforementioned procedure prevails for the vanilla training mode. For P1 and P2, we use the same procedure as for training on the KG, except that 200 epochs of training are previously performed on the protograph, for a total of 600 training epochs.

**Experimental results.** LP results are reported in Table 2. On YAGO14k and DBpedia77k, learning embeddings with P1 or P2 provides better Sem@K values compared to V. No significant change is noticed on FB15k187. This is likely to be an artefact of the Freebase class hierarchy and how entities are typed in FB15k187: the class hierarchy only has 2 levels, all relation domains and ranges are level-2 classes (therefore P1 and P2 are the same, see Table 1) and entities share many classes together. In this situation, the protograph may not contain discriminative enough information for pre-training embeddings, which is reflected in the similar results achieved regardless of the training setting. In addition, we observe that some KGEMs seem to benefit more from the MASCHInE approach. In particular, ConvE and TuckER tend to perform better w.r.t. both rank-based and semantic-oriented metrics. Overall, we notice that including protographs in the training phase helps making semantically valid predictions, as evidenced by the better Sem@K under P1 and P2. Protographs may help organize the embedding space in such a way that geometric distances between embeddings better reflect their semantic similarities. Regarding rank-based metrics, we notice equivalent performance, which could be explained since LP relies less on semantics and more on relational patterns [2] compared to other tasks such as entity clustering (see below).

## 4.2 Entity Clustering

Jain et al. [14] demonstrate that embeddings do not consistently capture the semantics of the KG (in particular, they only considered schema-agnostic approaches). In this section, we investigate whether embeddings learnt for the LP task under P1 and P2 can actually prove useful in EC.

### 4.2.1 *Experiments on the original datasets.*
We investigate whether the embeddings learnt for the LP task (see Section 4.1) under P1 and P2 allow a better class separability for the task of EC.

**Datasets.** We reuse YAGO14k, FB15k187, and DBpedia77k as presented in Section 4.1. The ground-truth labels are the most-generic classes an entity belongs to. It should be noted that entities belonging to several most-generic classes are filtered out as they would be assigned multiple ground-truth labels.

**Implementation details.** Entity embeddings are retrieved at the best epoch on the validation sets of YAGO14k, FB15k187, and DBpedia77k, for all the KGEMs and under the three settings V, P1, and P2. Then, following the evaluation protocol in Jain et al. [14], the k-means clustering algorithm is run using default parameters of scikit-learn<fnref n="6" />.

**Evaluation metrics.** Clustering results are evaluated using the Adjusted Rand Index (ARI) – which measures the similarity between the cluster assignments by making pairwise comparisons –

and Normalized Mutual Information (NMI) – which measures the agreement between the cluster assignments. It should be noted that these metrics are applicable as we have the ground-truth labels for each entity. In particular, we compare clusters output by k-means with ground-truth labels that are the classes entities belong to. ARI and NMI results are reported in Figure 3. Lowest and highest scores are 0 (–1 for ARI) and 1, respectively.

**Experimental results.** Except for TuckER on FB15k187, both settings P1 and P2 provide substantial improvements. It seems that the entity embeddings learnt by TuckER are not able to cluster entities based on their classes, which diminishes the relevancy of using this model for drawing comparisons. That being said, the benefit of P1 and P2 is consistent across datasets. This is even more striking for YAGO14k, as evidenced in Figure 4: entities sharing the same ground-truth class tend to cluster more under P2. Jain et al. [14] pointed out the limitations of using KGEs for semantic representation of the underlying entities and relations. The relatively low performance of embeddings for EC under V (Figure 3 and Figure 4) indeed suggests that the semantic similarity between entities is not reflected through geometric similarities in their embeddings. However, P1 and P2 generate entity latent representations whose geometric similarities are more representative of their classes.

### 4.2.2 *Experiments on GEval clustering problems.*
In this follow-up entity clustering experiment, the GEval evaluation framework [17] is used. GEval provides gold standard datasets based on DBpedia and suggest clustering models, configurations and evaluation metrics<fnref n="7" />.

**Datasets.** GEval proposes 4 datasets based on DBpedia. The first dataset contain entities of classes `db0:AmericanFootballTeam` and `db0:BasketballTeam`. We do not use this dataset as DBpedia77k does not contain any of these entities. Instead, we use the other three datasets for which DBpedia77k has a biggest coverage. For clarity, we name these datasets as CC, CCB, and CAMAP. The first two ones contain different numbers of DBpedia entities of classes `db0:Cities` and `db0:Countries`, while the third dataset contains five types of entities. Therefore, there are two ground-truth clusters in the first two datasets and five clusters in the third one.

**Implementation details.** We rely on GEval guidelines and perform the preliminary filtering step described as follows. First, we select entity embeddings learnt on DBpedia77k for the LP task (see Section 4.1). Embeddings of entities that do not appear in the GEval benchmark datasets are filtered out. Clustering with all the algorithms suggested in the GEval framework is performed on the remaining entities and evaluated w.r.t. several metrics.

**Evaluation metrics.** As in the previous experiment, we use ARI. Following the GEval guidelines, we additionally use Adjusted Mutual Information Score (AMI), V-Measure (VM), Fowlkes-Mallow Score (FM), Homogeneity (H), and Completeness (C). V-Measure is the harmonic mean of homogeneity and completeness measure, while the Fowlkes-Mallow Score is the geometric mean of pairwise precision and recall. All these metrics measure the correctness of the cluster assignments based on ground-truth labels. For each of these datasets, and under each setting, results achieved with the

<footnote n="6">⁶https://scikit-learn.org/stable/index.html</footnote>

<footnote n="7">⁷https://github.com/mariaangelapellegrino/Evaluation-Framework</footnote>
<figure><img src="image_4.png" /><figcaption>(a) YAGO14k</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>(b) FB15k187</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>(c) DBpedia77k</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>(d) YAGO14k</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>(e) FB15k187</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>(f) DBpedia77k</figcaption></figure>

<caption>Figure 3: Entity clustering results on YAGO14k, FB15k187, and DBpedia77k.</caption>

<table><caption>Table 3: Entity clustering results on the GEval datasets. Bold font indicates which setting performs the best for a given model and dataset, while underlined results correspond to the second best setting.</caption><thead><tr><th rowspan="2"></th><th rowspan="2"></th><th colspan="6">CC</th><th colspan="6">CCB</th><th colspan="6">CAMAP</th></tr><tr><th>ARI</th><th>AMI</th><th>VM</th><th>FM</th><th>H</th><th>C</th><th>ARI</th><th>AMI</th><th>VM</th><th>FM</th><th>H</th><th>C</th><th>ARI</th><th>AMI</th><th>VM</th><th>FM</th><th>H</th><th>C</th></tr></thead><tbody><tr><td rowspan="3">TransE</td><td>V</td><td>0.179</td><td>0.140</td><td>0.141</td><td>0.596</td><td>0.143</td><td>0.139</td><td>0.183</td><td>0.147</td><td>0.147</td><td>0.649</td><td>0.164</td><td>0.133</td><td>0.564</td><td>0.783</td><td>0.785</td><td>0.736</td><td>0.951</td><td>0.668</td></tr><tr><td>P1</td><td><u>0.621</u></td><td><u>0.514</u></td><td><u>0.515</u></td><td><u>0.823</u></td><td><u>0.505</u></td><td><u>0.526</u></td><td><u>0.288</u></td><td><u>0.311</u></td><td><u>0.311</u></td><td><u>0.692</u></td><td><u>0.349</u></td><td><u>0.280</u></td><td><u>0.593</u></td><td><u>0.795</u></td><td><u>0.796</u></td><td><u>0.756</u></td><td><u>0.956</u></td><td><u>0.682</u></td></tr><tr><td>P2</td><td><u>0.680</u></td><td><u>0.570</u></td><td><u>0.571</u></td><td><u>0.849</u></td><td><u>0.563</u></td><td><u>0.579</u></td><td><u>0.299</u></td><td><u>0.314</u></td><td><u>0.314</u></td><td><u>0.698</u></td><td><u>0.353</u></td><td><u>0.284</u></td><td><u>0.646</u></td><td><u>0.815</u></td><td><u>0.816</u></td><td><u>0.791</u></td><td><u>0.964</u></td><td><u>0.708</u></td></tr><tr><td rowspan="3">DistMult</td><td>V</td><td>0.018</td><td>0.024</td><td>0.027</td><td>0.709</td><td>0.015</td><td>0.095</td><td>-0.024</td><td>0.009</td><td>0.010</td><td>0.770</td><td>0.006</td><td>0.028</td><td>-0.054</td><td>0.099</td><td>0.106</td><td>0.478</td><td>0.087</td><td>0.136</td></tr><tr><td>P1</td><td><u>0.891</u></td><td><u>0.817</u></td><td><u>0.817</u></td><td><u>0.948</u></td><td><u>0.813</u></td><td><u>0.821</u></td><td><u>0.762</u></td><td><u>0.673</u></td><td><u>0.673</u></td><td><u>0.907</u></td><td><u>0.708</u></td><td><u>0.642</u></td><td><u>0.655</u></td><td><u>0.705</u></td><td><u>0.707</u></td><td><u>0.798</u></td><td><u>0.782</u></td><td><u>0.646</u></td></tr><tr><td>P2</td><td><u>0.891</u></td><td><u>0.824</u></td><td><u>0.824</u></td><td><u>0.948</u></td><td><u>0.818</u></td><td><u>0.830</u></td><td><u>0.756</u></td><td><u>0.626</u></td><td><u>0.627</u></td><td><u>0.907</u></td><td><u>0.646</u></td><td><u>0.608</u></td><td><u>0.933</u></td><td><u>0.872</u></td><td><u>0.872</u></td><td><u>0.963</u></td><td><u>0.886</u></td><td><u>0.859</u></td></tr><tr><td rowspan="3">ComplEx</td><td>V</td><td>-0.011</td><td>0.009</td><td>0.011</td><td>0.694</td><td>0.007</td><td>0.033</td><td>-0.024</td><td>0.009</td><td>0.010</td><td>0.771</td><td>0.006</td><td>0.028</td><td>0.139</td><td>0.345</td><td>0.350</td><td>0.520</td><td>0.341</td><td>0.360</td></tr><tr><td>P1</td><td><u>0.897</u></td><td><u>0.827</u></td><td><u>0.828</u></td><td><u>0.951</u></td><td><u>0.823</u></td><td><u>0.832</u></td><td><u>0.619</u></td><td><u>0.560</u></td><td><u>0.560</u></td><td><u>0.845</u></td><td><u>0.606</u></td><td><u>0.521</u></td><td><u>0.969</u></td><td><u>0.916</u></td><td><u>0.917</u></td><td><u>0.983</u></td><td><u>0.939</u></td><td><u>0.896</u></td></tr><tr><td>P2</td><td><u>0.319</u></td><td><u>0.264</u></td><td><u>0.265</u></td><td><u>0.665</u></td><td><u>0.269</u></td><td><u>0.262</u></td><td><u>0.493</u></td><td><u>0.460</u></td><td><u>0.460</u></td><td><u>0.788</u></td><td><u>0.507</u></td><td><u>0.422</u></td><td><u>0.961</u></td><td><u>0.914</u></td><td><u>0.914</u></td><td><u>0.978</u></td><td><u>0.938</u></td><td><u>0.892</u></td></tr><tr><td rowspan="3">ConvE</td><td>V</td><td>0.311</td><td>0.234</td><td>0.235</td><td>0.691</td><td>0.223</td><td>0.249</td><td>0.271</td><td>0.242</td><td>0.243</td><td>0.687</td><td>0.271</td><td>0.220</td><td>0.591</td><td>0.608</td><td>0.611</td><td>0.76</td><td>0.672</td><td>0.559</td></tr><tr><td>P1</td><td><u>0.878</u></td><td><u>0.805</u></td><td><u>0.805</u></td><td><u>0.942</u></td><td><u>0.799</u></td><td><u>0.811</u></td><td><u>0.719</u></td><td><u>0.628</u></td><td><u>0.628</u></td><td><u>0.889</u></td><td><u>0.665</u></td><td><u>0.596</u></td><td><u>0.934</u></td><td><u>0.880</u></td><td><u>0.881</u></td><td><u>0.964</u></td><td><u>0.928</u></td><td><u>0.839</u></td></tr><tr><td>P2</td><td><u>0.916</u></td><td><u>0.848</u></td><td><u>0.848</u></td><td><u>0.960</u></td><td><u>0.847</u></td><td><u>0.849</u></td><td><u>0.790</u></td><td><u>0.697</u></td><td><u>0.697</u></td><td><u>0.919</u></td><td><u>0.727</u></td><td><u>0.669</u></td><td><u>0.954</u></td><td><u>0.915</u></td><td><u>0.915</u></td><td><u>0.975</u></td><td><u>0.968</u></td><td><u>0.868</u></td></tr><tr><td rowspan="3">TuckER</td><td>V</td><td>0.179</td><td>0.138</td><td>0.139</td><td>0.596</td><td>0.141</td><td>0.137</td><td>0.158</td><td>0.120</td><td>0.121</td><td>0.639</td><td>0.135</td><td>0.109</td><td>0.396</td><td>0.526</td><td>0.529</td><td>0.618</td><td>0.636</td><td>0.453</td></tr><tr><td>P1</td><td><u>0.069</u></td><td><u>0.051</u></td><td><u>0.052</u></td><td><u>0.543</u></td><td><u>0.053</u></td><td><u>0.051</u></td><td><u>0.137</u></td><td><u>0.094</u></td><td><u>0.094</u></td><td><u>0.631</u></td><td><u>0.105</u></td><td><u>0.086</u></td><td><u>0.376</u></td><td><u>0.635</u></td><td><u>0.637</u></td><td><u>0.599</u></td><td><u>0.795</u></td><td><u>0.531</u></td></tr><tr><td>P2</td><td>0.047</td><td>0.042</td><td>0.043</td><td>0.533</td><td>0.043</td><td>0.042</td><td>0.033</td><td>0.051</td><td>0.052</td><td>0.578</td><td>0.058</td><td>0.046</td><td>0.276</td><td>0.487</td><td>0.490</td><td>0.520</td><td>0.621</td><td>0.405</td></tr></tbody></table>

best performing clustering method are shown in Table 3. Lowest and highest scores are 0 (–1 for ARI) and 1, respectively.

**Experimental results.** First, it is worth noting that results on CAMAP are significantly higher than on the other two datasets. Recall that CAMAP includes entities belonging to five different classes, whereas CC and CCB only contain entities from **dbo:Cities** and **dbo:Countries**. In this experiment, the embeddings learnt by the KGEMs are most useful for performing EC with five ground-truth clusters than with two, which might seem counterintuitive. However, in [14], it is demonstrated that in a KG, only a small subset of its entities actually have an embedding with a meaningful semantic representation. This is the reason why for the clustering task, results are not consistent across subsets of entities [14]. This is in line with our experimental findings that lead to the same conclusion.

[25] is also concerned with the ability of KGEMs to embed entities from the same class to the same region of the embedding space. Their results suggest that entities belonging to rare classes

have embeddings that are less separable w.r.t. to entity embeddings of other classes compared with entities belonging to frequent classes. This assumption needs to be qualified in light of our experimental findings: lots of entities in DBpedia77k are **dbo:City** and **dbo:Country**. Following [25], we should expect better results for CC and CCB in Table 3. However, it should be noted that both **dbo:City** and **dbo:Country** are subclasses of **dbo:Place** (equivalent to **dbo:Location**), which is over-represented in DBpedia77k<fnref n="8" />. It is arguably harder to separate entity embeddings for CC and CCB, as all of them actually belong to the same parent class. In contrast, CAMAP contains albums, cities, companies, movies, and universities. In this case, **dbo:City** is a child class of **dbo:Place**, **dbo:Company** and **dbo:University** are child classes of **dbo:Agent** while **dbo:Album** and **dbo:Movie** are child classes of **dbo:Work**.

<footnote n="8">⁸In DBpedia77k, 11,586 entities belong to the **dbo:Place** class.</footnote>
<table><caption>**Table 4:** Node classification results on the 12 DLCC test cases. Listed are the results of the best classifier for each combination of model and test case. Bold fonts indicate which setting performs the best for a given model and test case. Underlined results show which combination of model and setting performs the best for each test case.</caption><thead><tr><th>Test Case</th><th>Setting</th><th>TransE</th><th>DistMult</th><th>ComplEx</th><th>ConvE</th><th>TuckER</th><th>Test Case</th><th>Setting</th><th>TransE</th><th>DistMult</th><th>ComplEx</th><th>ConvE</th><th>TuckER</th></tr></thead><tbody><tr><td rowspan="3">tc01</td><td>V</td><td><strong>0.702</strong></td><td><strong>0.810</strong></td><td><strong>0.820</strong></td><td><strong>0.885</strong></td><td>0.832</td><td rowspan="3">tc02</td><td>V</td><td><strong>0.654</strong></td><td>0.564</td><td>0.566</td><td>0.732</td><td>0.669</td></tr><tr><td>P1</td><td>0.647</td><td>0.754</td><td>0.784</td><td>0.875</td><td><strong>0.857</strong></td><td>P1</td><td>0.566</td><td>0.554</td><td>0.554</td><td><strong>0.764</strong></td><td>0.637</td></tr><tr><td>P2</td><td>0.637</td><td>0.794</td><td>0.769</td><td>0.852</td><td>0.792</td><td>P2</td><td>0.571</td><td><strong>0.627</strong></td><td><strong>0.627</strong></td><td>0.687</td><td><strong>0.679</strong></td></tr><tr><td rowspan="3">tc03</td><td>V</td><td>0.584</td><td><strong>0.571</strong></td><td><strong>0.554</strong></td><td>0.629</td><td><strong>0.669</strong></td><td rowspan="3">tc04</td><td>V</td><td>0.638</td><td>0.540</td><td>0.512</td><td>0.818</td><td>0.570</td></tr><tr><td>P1</td><td><strong>0.589</strong></td><td>0.541</td><td>0.519</td><td><strong>0.699</strong></td><td>0.664</td><td>P1</td><td>0.760</td><td><strong>0.812</strong></td><td><strong>0.832</strong></td><td><strong>0.870</strong></td><td>0.552</td></tr><tr><td>P2</td><td>0.521</td><td>0.544</td><td>0.526</td><td>0.622</td><td>0.556</td><td>P2</td><td><strong>0.805</strong></td><td>0.802</td><td>0.818</td><td>0.818</td><td><strong>0.680</strong></td></tr><tr><td rowspan="3">tc05</td><td>V</td><td>0.704</td><td>0.683</td><td>0.678</td><td>0.822</td><td>0.814</td><td rowspan="3">tc06</td><td>V</td><td>0.645</td><td><strong>0.940</strong></td><td><strong>0.938</strong></td><td><strong>0.940</strong></td><td><strong>0.950</strong></td></tr><tr><td>P1</td><td>0.887</td><td>0.937</td><td><strong>0.940</strong></td><td><strong>0.940</strong></td><td>0.814</td><td>P1</td><td>0.718</td><td>0.875</td><td>0.898</td><td>0.935</td><td><strong>0.950</strong></td></tr><tr><td>P2</td><td><strong>0.937</strong></td><td><strong>0.940</strong></td><td><strong>0.940</strong></td><td><strong>0.940</strong></td><td><strong>0.902</strong></td><td>P2</td><td><strong>0.740</strong></td><td>0.828</td><td>0.788</td><td>0.895</td><td>0.942</td></tr><tr><td rowspan="3">tc07</td><td>V</td><td>0.578</td><td>0.515</td><td>0.552</td><td>0.560</td><td>0.522</td><td rowspan="3">tc08</td><td>V</td><td>0.552</td><td>0.552</td><td>0.520</td><td><strong>0.598</strong></td><td>0.575</td></tr><tr><td>P1</td><td>0.505</td><td>0.540</td><td>0.560</td><td>0.560</td><td>0.552</td><td>P1</td><td>0.585</td><td>0.560</td><td>0.558</td><td>0.575</td><td>0.580</td></tr><tr><td>P2</td><td><strong>0.582</strong></td><td><strong>0.605</strong></td><td><strong>0.655</strong></td><td><strong>0.675</strong></td><td><strong>0.648</strong></td><td>P2</td><td><strong>0.595</strong></td><td><strong>0.588</strong></td><td><strong>0.588</strong></td><td>0.578</td><td><strong>0.672</strong></td></tr><tr><td rowspan="3">tc09</td><td>V</td><td>0.552</td><td>0.522</td><td>0.525</td><td>0.730</td><td>0.745</td><td rowspan="3">tc10</td><td>V</td><td>0.578</td><td><strong>0.572</strong></td><td><strong>0.562</strong></td><td><strong>0.640</strong></td><td>0.690</td></tr><tr><td>P1</td><td><strong>0.560</strong></td><td>0.538</td><td>0.552</td><td><strong>0.745</strong></td><td><strong>0.780</strong></td><td>P1</td><td><strong>0.598</strong></td><td>0.550</td><td>0.538</td><td>0.625</td><td><strong>0.692</strong></td></tr><tr><td>P2</td><td>0.550</td><td><strong>0.615</strong></td><td><strong>0.598</strong></td><td>0.672</td><td>0.742</td><td>P2</td><td>0.538</td><td>0.510</td><td>0.535</td><td>0.590</td><td>0.588</td></tr><tr><td rowspan="3">tc11</td><td>V</td><td><strong>0.632</strong></td><td>0.518</td><td>0.535</td><td><strong>0.688</strong></td><td><strong>0.700</strong></td><td rowspan="3">tc12</td><td>V</td><td><strong>0.650</strong></td><td>0.538</td><td>0.552</td><td>0.702</td><td>0.708</td></tr><tr><td>P1</td><td>0.628</td><td>0.542</td><td>0.522</td><td>0.618</td><td>0.685</td><td>P1</td><td>0.572</td><td>0.530</td><td>0.565</td><td>0.708</td><td>0.700</td></tr><tr><td>P2</td><td>0.630</td><td><strong>0.552</strong></td><td><strong>0.558</strong></td><td>0.638</td><td>0.678</td><td>P2</td><td>0.625</td><td><strong>0.802</strong></td><td><strong>0.738</strong></td><td><strong>0.802</strong></td><td><strong>0.752</strong></td></tr></tbody></table>

<figure><img src="image_10.png" /><figcaption>(a) TransE (V)</figcaption></figure>

<figure><img src="image_11.png" /><figcaption>(b) TransE (P2)</figcaption></figure>

<figure><img src="image_12.png" /><figcaption>(c) DistMult (V)</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>(d) DistMult (P2)</figcaption></figure>

<caption>**Figure 4:** PCA visualizations of YAGO14k entity embeddings using the first two principal components. Each point represents an entity, whose color denotes its ground-truth class.</caption>

With three different generic classes, the semantic similarities between entities in CAMAP is more easily reflected into the geometric similarities of their embeddings.

Interestingly, the discrepancies in results, i.e., better results on CAMAP compared with CC and CCB, are setting-dependent (V, P1, P2). Indeed, they are alleviated when using either P1 or P2. Under V, TransE, ComplEx, and ConvE provide substantially better results on CAMAP compared to CC. Once using P2, we cannot see any significant difference in results on CAMAP compared to CC for these models. This means that training with a protograph-assisted approach led the KGEMs to generate higher-quality embeddings in terms of class separability, especially for entities in CC and CCB that are harder to distinguish. With the sole exception of TuckER

whose performance may be due to non-optimal hyperparameter tuning, strong improvements are achieved w.r.t. the EC task on all datasets. Interestingly, not only V is outperformed by either P1 or P2 in each scenario, but actually by both of them at the same time. As a result, our schema-based, protograph-assisted learning approach MASCHInE proves effective for EC, regardless of the protograph design principle.

## 4.3 Node Classification

Following on from the previous clustering experiment that clearly showed the benefit of incorporating pre-trained protograph embeddings, in this section we are interested in performing node classification with such embeddings. In this experiment, entities are labelled on the basis of whether they comply with some description logic (DL) constructors.

**Datasets.** In this experiment, we use the synthetic DLCC node classification benchmark [19]. The DLCC benchmark helps analyze the quality of KGEMs by evaluating them w.r.t. which kinds of DL constructors they can represent. We use the synthetic gold standard datasets<fnref n="9" />. These are 12 synthetic datasets featuring labelled entities to classify and underpinned by a schema defining entity types, relation domains and ranges. The synthetic part of DLCC has been chosen because the authors argue that those pose classification problems which are not influenced by any side effects and correlations in the knowledge graph.

**Implementation details.** For these 12 test cases, their respective protographs are built. Then, the KGEMs are trained w.r.t. the LP task under the three different settings V, P1, and P2. In line with [19], multiple classifiers were trained for each test case: decision trees (DT), naive bayes (NB), k-nearest neighbors (k-NN), support vector machine (SVM), and multilayer perceptron (MLP). Based on the

<footnote n="9">⁹https://github.com/janothan/DL-TC-Generator</footnote>
best entity embeddings learnt during LP, these models are used to classify entities using the default scikit-learn parameters.

**Evaluation metrics.** NC performance is measured using F-score, which is the harmonic mean of precision and recall. Lowest and highest scores are 0 and 1, respectively. For each model and test case, results achieved with the best classifier are reported in Table 4.

**Experimental results.** First, not all of the gold standard test cases are equally hard for the task of NC. For instance, entities in tc01, tc05, and tc06 are obviously easier to label than in the other datasets. Particularly, the DLCC synthetic gold standard datasets are built on the basis of different DL constructors in order to analyze which KG constructs are more easily recognized by KGEMs.

For the less trivial cases, the gains that can be achieved by using protographs are more significant. Especially in cases which are fairly badly solved by vanilla embeddings (e.g., tc07, tc12), there are gains of more than 10 percentage points in F1. It is further remarkable that on this problem, compared to link prediction and entity clustering, TuckER can also benefit from the protographs.

When looking at which protograph construction strategy is more suitable for which kind of problem, we see that for those test cases which use a concrete class in their definition (particularly tc07, tc08, tc11, and tc12), P2 is clearly superior to P1. This may be due to the fact that P1 does not create class embeddings for all classes used in the respective constructors. Therefore, the resulting entity embeddings are less discriminating.

# 5 CONCLUSION AND FUTURE WORK

In this paper, we present MASCHInE – a novel model-agnostic and protograph-assisted approach for learning KGEs. MASCHInE first trains embeddings on protographs and then transfers these embeddings to learn the final KGEs. We have shown that this approach produces more versatile KGEs, which can yield significant improvements in entity clustering and node classification. This is particularly noticeable for entity clustering, as protographs encode information about classes, on which entity clustering heavily relies on. We also demonstrated the advantage of using MASCHInE for link prediction: the use of protographs in the training procedure leads KGEMs to make more semantically valid predictions, which clearly highlights that geometric distance in the embedding space can be influenced by the semantics of the underlying KG.

In future work, we will extend MASCHInE by adding new protograph design principles. While in the current form, the transfer from the RDFS-based protograph embeddings to the KG entity embeddings is done once, we will study more expressive ontological constructs (e.g., OWL) and alternatives for KG and protograph iterative co-training. Additionally, it would be worth exploring whether pre-training over protographs provides benefits compared to passing the relevant RDFS triples to the KGEM as part of the whole graph. Comparing our approach with KGEMs that take advantage of additional information in the KG – e.g. attributes – also deserves further exploration.

# REFERENCES

[1] Ivana Balazevic, Carl Allen, and Timothy M. Hospedales. 2019. TuckER: Tensor Factorization for Knowledge Graph Completion. In *Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing*, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019. Association for Computational Linguistics, 5184–5193.

[2] Peru Bhardwaj, John D. Kelleher, Luca Costabello, and Declan O'Sullivan. 2021. Poisoning Knowledge Graph Embeddings via Relation Inference Patterns. In *Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing*, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021. Association for Computational Linguistics, 1875–1888.

[3] Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In *Conf. on Neural Information Processing Systems (NeurIPS)*. 2787–2795.

[4] Sejla Cebiric, François Goasdoué, Haridimos Kondylakis, Dimitris Kotzinos, Ioana Manolescu, Georgia Troullinou, and Mussab Zneika. 2019. Summarizing semantic graphs: a survey. *VLDB J.* 28, 3 (2019), 295–327.

[5] Haochen Chen, Bryan Perozzi, Yifan Hu, and Steven Skiena. 2018. HARP: Hierarchical Representation Learning for Networks. In *Proc. of the 32nd AAAI Conf. on Artificial Intelligence*, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018. AAAI Press, 2127–2134.

[6] Zijun Cui, Pavan Kapanipathi, Kartik Talamadupula, Tian Gao, and Qiang Ji. 2021. Type-augmented Relation Prediction in Knowledge Graphs. In *35th Conf. on Artificial Intelligence*, AAAI 2021, 33rd Conf. on Innovative Applications of Artificial Intelligence, IAAI 2021, The 11th Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, February 2-9, 2021. AAAI Press, 7151–7159.

[7] Claudia d'Amato, Nicola Flavio Quatraro, and Nicola Fanizzi. 2021. Injecting Background Knowledge into Embedding Models for Predictive Tasks on Knowledge Graphs. In *The Semantic Web - 18th International Conf., ESWC 2021, June 6-10, Proc. (Lecture Notes in Computer Science)*, Vol. 12731). Springer, 441–457.

[8] Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2D Knowledge Graph Embeddings. In *Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence*, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018. AAAI Press, 1811–1818.

[9] François Goasdoué, Pawel Guzewicz, and Ioana Manolescu. 2020. RDF graph summarization for first-sight structure discovery. *VLDB J.* 29, 5 (2020), 1191–1218.

[10] Zengfeng Huang, Shengzhong Zhang, Chong Xi, Tang Liu, and Min Zhou. 2021. Scaling Up Graph Neural Networks Via Graph Coarsening. In *KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 2021*. ACM, 675–684.

[11] Nicolas Hubert, Pierre Monnin, Armelle Brun, and Davy Monticolo. 2022. Knowledge Graph Embeddings for Link Prediction: Beware of Semantics! In *Proceedings of the Workshop on Deep Learning for Knowledge Graphs (DL4KG 2022) co-located with the 21th International Semantic Web Conference (ISWC 2022)*.

[12] Nicolas Hubert, Pierre Monnin, Armelle Brun, and Davy Monticolo. 2023. Sem@K: Is my knowledge graph embedding model semantic-aware?

[13] Nicolas Hubert, Pierre Monnin, Armelle Brun, and Davy Monticolo. 2023. Treat Different Negatives Differently: Enriching Loss Functions with Domain and Range Constraints for Link Prediction. *CoRR abs/2303.00286* (2023).

[14] Nitisha Jain, Jan-Christoph Kalo, Wolf-Tilo Balke, and Ralf Krestel. 2021. Do Embeddings Actually Capture Knowledge Graph Semantics? In *The Semantic Web - 18th International Conference, ESWC 2021, Virtual Event, June 6-10, 2021, Proceedings (Lecture Notes in Computer Science)*, Vol. 12731). Springer, 143–159.

[15] Nitisha Jain, Trung-Kien Tran, Mohamed H. Gad-Elrab, and Daria Stepanova. 2021. Improving Knowledge Graph Embeddings with Ontological Reasoning. In *The Semantic Web - International Semantic Web Conf. ISWC*, Vol. 12922, 410–426.

[16] Denis Krompaß, Stephan Baier, and Volker Tresp. 2015. Type-Constrained Representation Learning in Knowledge Graphs. In *The Semantic Web - 14th International Semantic Web Conf. (ISWC)*, Vol. 9366. Springer, 640–655.

[17] Maria Angela Pellegrino, Abdulrahman Altabba, Martina Garofalo, Petar Ristoski, and Michael Cochez. 2020. GEval: A Modular and Extensible Evaluation Framework for Graph Embedding Techniques. In *The Semantic Web - 17th International Conference, ESWC 2020, Heraklion, Crete, Greece, May 31-June 4, 2020, Proceedings (Lecture Notes in Computer Science)*, Vol. 12123). Springer, 565–582.

[18] Marcin Pietrasik and Marek Z. Reformat. 2023. Probabilistic Coarsening for Knowledge Graph Embeddings. *Axioms* 12, 3 (2023), 275.

[19] Jan Portisch and Heiko Paulheim. 2022. The DLCC Node Classification Benchmark for Analyzing Knowledge Graph Embeddings. In *The Semantic Web - ISWC 2022 - 21st International Semantic Web Conference, Virtual Event, October 23-27, 2022 (Lecture Notes in Computer Science)*, Vol. 13489). Springer, 592–609.

[20] Théo Trouillon, Johannes Welbl, Sebastian Riedel, Eric Gaussier, and Guillaume Bouchard. 2016. Complex Embeddings for Simple Link Prediction. In *Proc. of the 33rd International Conf. on Machine Learning*, ICML, Vol. 48, 2071–2080.

[21] Meihong Wang, Linling Qiu, and Xiaoli Wang. 2021. A Survey on Knowledge Graph Embeddings for Link Prediction. *Symmetry* 13, 3 (2021), 485.
[22] Xueliang Wang, Jiajun Chen, Feng Wu, and Jie Wang. 2022. Exploiting Global Semantic Similarities in Knowledge Graphs by Relational Prototype Entities. CoRR abs/2206.08021 (2022). arXiv:2206.08021
[23] Ruobing Xie, Zhiyuan Liu, and Maosong Sun. 2016. Representation Learning of Knowledge Graphs with Hierarchical Types. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9-15 July 2016. IJCAI/AAAI Press, 2965-2971.

[24] Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. In 3rd International Conference on Learning Representations, ICLR.
[25] Amal Zouaq and Félix Martel. 2020. What is the schema of your knowledge graph?: leveraging knowledge graph embeddings and clustering for expressive taxonomy learning. In Proceedings of The International Workshop on Semantic Big Data, SBD@SIGMOD 2020, Portland, Oregon, USA, June 19, 2020. ACM, 6:1-6:6.