Multi-view Contrastive Learning for Entity Typing over Knowledge Graphs
Authors: Zhiwei Hu
Victor Gutierrez Basulto
Zhiliang Xiang
Ru Li
Jeff Z. Pan
Venue: EMNLP 2023 Main (2023)
Keywords: Knowledge Graphs
Entity Typing
Knowledge-based Typing

Abstract: Knowledge graph entity typing (KGET) aims at inferring plausible types of entities in knowledge graphs. Existing approaches to KGET focus on how to better encode the knowledge provided by the neighbors and types of an entity into its representation. However, they ignore the semantic knowledge provided by the way in which types can be clustered together. In this paper, we propose a novel method called Multi-view Contrastive Learning for knowledge graph Entity Typing MCLET,  which effectively encodes the  coarse-grained knowledge provided by clusters into  entity and type embeddings. MCLET is composed of three modules:  i)  Multi-view Generation and Encoder module, which encodes  structured information from entity-type, entity-cluster and cluster-type views; ii) Cross-view Contrastive Learning module, which encourages different views to collaboratively improve view-specific representations of entities and types; iii) Entity Typing Prediction module, which integrates multi-head attention and a Mixture-of-Experts strategy to infer missing entity types. Extensive experiments show the strong performance of MCLET compared to the state-of-the-art

Multi-view Contrastive Learning for Entity Typing over Knowledge Graphs
Zhiwei Hu*
Victor Gutiérrez-Basulto" Zhiliang Xiang
Ru Li Jeff Z. Pan
School of Computer and Information Technology, Shanxi University, China
School of Computer Science and Informatics, Cardiff University, UK
ILCC, School of Informatics, University of Edinburgh, UK
zhiweihu@whu. edu. .du.edd ca, lir.ssuu. edu. cn
ac. uk http:://koolllgg-rrrrressnntatai org/ z. pan/
[gutieerrezbasultov xiangz6 oiag:aasse
Abstract
Tanky
Knowledge graph entity typing (KGET) aims at inferring plausible types of entities in knowl- edge graphs. Existing approaches to KGET focus on how to better encode the knowledge provided by the neighbors and types of an entity into its representation. However, they ignore the semantic knowledge provided by the way which types can be clustered together. In this paper, we propose novel method called Multi-view Contrastive Learning for knowl- edge graph Entity Typing (MCLET), which effectively encodes the coarse-gre grrained knowl- edge provided by clusters into entity and type embeddings. MCLET composed of three modules: i) Multi-view Generation and En- coder module, which encodes structured in- formation from entiry-rype, entity-cluster and cluster-type views; ii) Cross-view Contrastive Learning module, which encourages different views to collaboratively improve view specifciiic representations of entities and types; iii) En- tity Typing Prediction module, which integrates multi-head attention and Mixture-of-FF ox-peersss strategy to infer missing entity types. Extensive experiments show the strong performance of MCLET compared to the state-of-the-art.
Pheguaet
leeteregeeeos Graph
Homegenenns Graph
Figure 1: Heterogeneous graph with entities, coarse- grained clusters, and fine-grained types, covering three homogeneous views entity-type, entity-cluster, and cluster-type.
answering (Hu et al., 2022b; Chen et al., 2019; Hu et al., 2023), and relation extraction (Li et al., 2019). However, one cannot always have access this kind of knowledge KGs are inevitably incom- plete (Zhu et al., 2015). For example, Figure the entity Joe Biden has types American_ mmoticunnpooittccan politician and American Jaaricen_lawyer,,, but should also have types male_polliiiae and male lawwyar.... This phe- nomenon is common in real-world datasets, for instance, 10% of entities in the FB15k KG have the type /music/artist, but missing the type /peo- ple/person (Moon al., 2017). Motivated by this, we concentrate on the Knowledge Graph Entity Typ- ing (KGET) task which aims at inferring missing entity types in a KG.
1 Introduction
Knowledge graphs (KGs) (Pan et al., 20)7a, store graph-like knowledge using triples of the form (s, r, o), indicating that entities s and are related to each other through relation type r. KGs also contain entity type knowledge described as (e, haa_haape, t), denoting that entity has type e.g., we can express that Joe Biden has type Amer- icun_politician, cf. Figure 1. Entity type knowl- edge plays a key role in various natural language processing related tasks, such as entity and rela- tion linking (Gupta et al., 2017; Pan et al., 2019), knowledge graph completion (Peng et al., 2022; Niu al., 2022; Wiharja et al., 2020), question
A wide variety of approaches to KGET have been already proposed, including embedding- (Moon et al., 2017: Zhao et al., 2020), transformer- (Wang et al., 2019; Hu et al., 2022a) and graph neural network (GNNs) based methods (Pan et al., 2021; Jin et al., 2022). Each of these approaches have their own disadvantages: embedding bbased methods ignore the existing neighbor information, transformer- are computationally costly due to the use of multiple transformers to encode dif- ferent neighbors and GNNs- basee ignore impor- tant higher-level semantic content beyond what is readily available in the graph-structure. Indeed,
"Contact Authors

---

in addition relational and type neighbors, entity types can often be clustered together to provide coarse-grained information. Importantly, this type of coarse- coarse-grain information is available in many KGs, such as YAGO (Suchanek et al., 2007), which provides an alignment between types and Word- Net (Miller, 1995) concepts. The example in Fig- ure 1 shows that between the entity Joe Biden and the type
Prediction module makes full use of the relational and known type neighbor information for entity type prediction. We also introduce multi-head attention with Mixture-of- nism to obtain the final prediction score. Our main contributions are the following:
(MoE) mecha-
We propose MCLET, method which effec- tively uses entity-type, entity-cluster and cluster- type structured information. We design a cross- view contrastive learning module to capture the interaction between different views.
there is layer with cluster-level information American and politician. On the one hand, compared with the type content, the cluster information has a coarser granularity, which can roughly give the possible attributes of an entity and reduce the decision-making space in the entity type prediction task. On the other hand, the introduction of cluster information can enhance the semantic richness of the input KG. For example, from the type assertion (Joe Biden, has st_ttyp,, American Amerricanppolttician politician) and the clusters American and politician corresponding to type American politician, we can obtain new semantic connection edges between entities and clusters, i.e,, (Joe Biden, has cluster, American) and (Joe Biden, hass_husse cls_utuuteerrrreee politicia). Note that as type might belong multiple clusters and a cluster may con- tain multiple types, similar phenomenon occurs the entity-type and entity-cluster levels, e.g, an en- tity might contain many clusters. Through the inter- connection among entities, coarse-grand grasned clusters and fine-grati aree-graidd types, a dense entity-cluster-e heterogencous graph with multi-level semantic re- lations can be formed.
We devise a multi-head attention with Mixture- of- af-Experts mechanism to distinguish the contri- bution from different entity neighbors.
We conduct empirical and ablation experiments two widely used datasets, showing the supe- riority of MCLET over the existing state-of-art models.
American_politician
Data, code, and an extended version with ap- pendix are available at https://gft /tttptb/ wtw.ictub.co..... zhiweihu1100/ET-MCLL
2 Related Work
Embedding- Methods. These methods have been introduced based on the observation that the KGET task can be seen as a sub-task of the comple- tion task (KGC). ETE (Moon et al., 2017) unifies the KGET and KGC tasks by treating the entity- type pair (entity, type) as a triple of the form (en- tity, has_type, type). ConnectE (Zhao et al., 2020) builds two distinct type inference mechanisms with local typing information and triple knowledge.
To effectively leverage the flow of knowledge between entities, clusters and types, we pro- pose a novel method MCLET, a Multi-view Contrastive Learning model for knowledge graph Entity Typing, including three modules: a Multi- view Generation and Encoder module, Cross- view Contrastive Learning module and Entity Typing Prediction module. The Multi-view Gener- ation and Encoder module aims to convert a het- erogeneous graph into three homogeneous graphs entity-type, entity-cluster and cluster-type to en- code structured knowledge at different levels of granularity (cf. right side of Figure 1). To collabo- ratively supervise the three graph views, the Cross- view Contrastive Learning module captures the in- teraction by cross-view contrastive learning mecha- nism and mutually enhances the view-specific rep- resentations. After obtaining the embedding repre- sentations of entities and types, the Entity Typing
Graph Neural Network-based Methods. Given that GNNs inherently capture structural knowl- edge from graphs (e.g. the neighborhood of an entity), they have been previously used for the KGET task (Jin et al., 2019; Vashishth et al., 2020; Pan et al., 2021; Zhuo et al., 2022: Zhao et al., 2022; Zou et al.. 2022a). For example, CET (Pan et al., 2021) introduces two mechanisms to fully utilize neighborhood information in an independent and aggregated manner. MiNer (Jin et al., 2022) proposes a neighborhood information aggregation module to aggregate both one-hop and multi-hop neighbors. However, these type of methods ignore other kind of semantic information, e.g., how types cluster together.
-ranslormer-basedd Methods. Many studies use transformer (Vaswani et al., 2017) for KGs related

---

tasks (Liu et al., 2022: Xie et al., 2022: Chen et 2022), including KGC. So, transformer-bssed methods for KGC, such as CoKE (Wang et al., 2019) and HittER (Chen et al., 2021) can be di- rectly applied to the KGET task. TET (Hu et al., 2022a) presents a dedicated method for KGET. However, the introduction of multiple transformer structures brings a large computational overhead, which limits its application for large datasets.
duction of cluster information into the type predic- tion process. Therefore, a natural question is how to determine the clusters to which a type belongs to. In fact, the Freebase (Bollacker et al., 2008) and the YAGO (Suchanek et al., 2007) datasets themselves provide cluster information. For the Freebase dataset, the types are annotated in hier- archical manner, so we can directly obtain cluster information using a rule-like approach based on their type annotations. For instance, the type /loca- tion/uk_owe _oreiluk__vrress____ _ belongs to the cluster lo- cation and the type /educcatio/edducational _egreee belongs to the cluster education. The YAGO dataset provides an alignment between types and WordNet concepts'.. Soo, we can directly obtain the words in WordNet (Miller, 1995) describing the cluster to which type belongs to. For example, for the type wikicategory People_from Dungannon. its cluster wordnet perrcdee_pprroo_.oo wtwppperrooooorrPP1 100007846, and for the type rikicategory_Mulle coule_actors__ frrrrsssssosaaaaeeee Arizona, its cluster is
3 Background
Task Definition. Let E, R and T respectively be finite sets of entities, relation types and entity types. A knowledge graph (KG) 9 is the union of Striples and Stypes, where Striptes denotes a set of triples of the form (s, r, o), with 8, o C & and r C R. and Gtypes denotes set of pairs of the form (e, t), with e E E and E T. To work with uniform rep- resentation, we convert the pair (e, t) to the triple (c, haas_aype,,), I), where has_type is special role type not occurring in R. Key to our approach is the information provided by relational and type neigh- bors. For an entity e, its relational neighbors is the set N₁, - {rr o) | (e, o) E Gtriples} and its type neighbors is the set Ne
4 Method
In this section, we introduce our proposed method MCLET, which consists of three components: (1)
t)
(c, has_type, t) E Gigpes). In this paper, we con- sider the knowledge graph entity typing (KGET) task which aims at inferring missing types from T in triples from Gtypes.
Multi-view Generation and Encoder ($4.1); (2)
Cross-view Contrastive Learning ($4.2); and (3)
Entity Typing Prediction ($4.3).
4.1 Multi-view Generation and Encoder
Type Knowledge Clustering. Before we intro- duce our approach KGET, we start by not- ing that it is challenging to infer types whose prediction requires integrating various pieces of information together. For example, to predict that the entity Barack Obama has type 20th- Lnntur_AAmeican_lane
For the KGET task, the two parts, Gtriples and Stypee: of the input KG can be used for inference. The main question is how to make better use of the type graph Grypess as this might affect the perfor- mance of the model to large extent. So, the main motivation behind this component of MCLET is to effectively integrate the existing structured knowl- edge into the type graph. After introducing coarse- grained cluster information into the type graph, a three-level structure is generated: entity, coarse- grained cluster, and fine-grained type, such that the corresponding graph will have three types of edges: entity-type, cluster-type, and entity-cluster. Note that different subgraphs focus on different perspec- tives of knowledge. For example, the entity-cluster subgraph pays more attention to more abstract con- tent than the entity-type subgraph. Therefore, fully utilize the knowledge at each level, we convert the heterogeneous type graph into homogeneous graphs and construct an entity-type graph Ge2t, a
we need to know his birth year (Barack Obuma, wra_born_jng 1961), place of birth (Barack Obama, place_of_binee birth, Hawaii), and occupation (Barack Obama, occu- pation, lawyer). Clearly, this problem is exac- erbated by the fact that the KG itself is incom- plete, which might more easily lead to predic- tion errors. However, in practice, type knowledge is often semantically clustered together, e.g., the types male_ lawyer, American srrriccanlawyer,,, and 19th- century_lawyer belong to the cluster lawyer. Natu- rally, this coarse-grained cluster information could help taming the decision-making process by pay- ing more attention to types within a relevant cluster, without considering 'irrelevant' types from other clusters. With this in mind, we explore the intro-

---

Type
Entity Cluster Graph
Postike Pairs
Prodiction
Inter vite-viio Negative Pairs
Tedatimal Veigfliier
Intrassion Nayative Pairs
Expert
LightGCN
Liieie O
O
Eatily Type Graph
Viee
Nulti krads
EXpOETS
Neighber Prodiction Mechanisn
LighTGN
Retational Neighbor
Type Neighbr
Cluster Type Graph
LightGCN CN
View
Tsse Neighber
Central Entily (d) Entity Typing Prediction
(a) Knowledge Graph
(b) Multi-cimw Generation and Encoder
(c) Cross-veew Contrastive L Learning
Figure 2: An overview of our MCLET model, containing three modules: Multi-view Generation and Encoder. Cross-view Contrastive Learning, and Entity Typing Prediction.
cluster-type graph Ge2e, and entity-cluster graph Ge20 separately.
abstraction. Therefore, its content has coarser gran- ularity and wider coverage. So, given an entity and a type f, for a triple (e, tha__tyee, t) from Ge2t and a triple is ggiis_custer___e of, t) from Gezt. we construct a triple (e,hasscluster c). where has clu_cclster is new relation type. Note that because type may belong to multiple clusters, entity with this type will also be closely lated multiple clusters. Consider for an exam- ple, the entity Barack Obama with type Ameri- can lawyer. Since American llaccerrnlwwerrrrrrrre belongs to the American and lawyer clusters, then there will be (Barack Obama, has mas_cluster, American) and (Barack Obama, has cluustr,e lawyer) in Geze-
Entity Type Graph. The entity type graph, de- noted Gezs. is the original type graph Gtypes from the input KG. Recall that different types of an en- tity describe knowledge from different perspec- tives, which might help inferring missing types. For example, given the type assertion (Barack Obama, has sas_, 20th-century American American_lawyer), we could deduce the missing type assertion (Barack Obama, has has_type, American llawyrr,,,, since the type 20th-century ican_lawyy
20th-century_Amrriannnnnno Larrrammmeemmaaaaaaaaaa
entails Amer-
Cluster Type Graph. The cluster type graph, de- noted G.21, is a newly generated graph based on how types are clustered. Type knowledge avail- able in existing KGs inherently contains seman- tic information about clusters of types. For in- stance, the type /people/appointer in FB FB1 1KETT, clearly entails the cluster people. A similar phe- nomenon occurs in the YAG043kET KG. Fol- lowing this insight, for type and its cluster c, we use a new relation type is ss_ccluter__of to connect and For instance, from the type (peaple/appointer and its cluster people we can obtain (people, is_ is_cluster_g _f,, /people/appointer). Note that type may belong to multiple clusters. For example, the type American mneye, belongs to the clusters American and lawyer.
Multi-view Encoder. We encode the different views provided by Ge2s, 9.25, and Geze into the representations of entities, types and clusters us- ing graph convolutional networks (GCN) (Kipf and Welling, 2017). More precisely, adopt Light- GCN's (He al., 2020) message propagation strat- egy encode the information propagation from entity-rype, cluster-type, and entity-cluster views. Our choice is supported by the following observa- tion. The three graphs, Ge2t, Ge2t, and Gete. are uni-relational, i.e., only one relational type is used, so there is no need a for a heavy relti-relational GCN model like RGCN (Schlichtkrull et al., 2018). Indeed, LightGCN is more efficient because it moves the self-connections from the graph and the nonlinear transformation from the information propagation function. To encode the three views, we use the same LightGCN structure, but no param- eter sharing is performed between the correspond-
Entity Cluster Graph. The entity cluster graph, denoted as 9e2os is generated based on 9cz and Ge2 Unlike the entity type graph, the entity clus- ter graph captures knowledge at a higher level of

---

ing structures. Taking the encoding of Gez as an example, to learn the representations of entities and types, the f-th layer's information propagation is defined as:
where {W1, W2] Rdxd and {b1,by R are the learnable parameters, f(-) is the ELU non- linear function. The embedding of the i-th entity in Ge2t² can be expressed as 22,4-c2t
(E D) & c2t
Step 2. Positive and Negative Samples. Let a node u be an anchor, the embeddings of the corre- sponding node u in two different views provide the positive samples, while the embeddings of other nodes in two different views are naturally regarded as negative samples. Negative samples come from two sources, intra-view nodes or inter-view nodes, Intra-view means that the negative samples are nodes different from t in the same view where node u is located, while inter-view means that the negative samples are nodes (different from u) in the other views where is not located.
ci c21 Σ
ViMallMe
tEMe
le c2-
Σ N11021
cEMi ViMillMa
where (x 221 12-211 E Rdd represent the embed-
dings of entity and type in the graph Ge2t, and d
is the dimension of embedding. (x x 22-
(0)
are randomly initialized embeddings at the begin- ning of training. MM. and Mr respectively denote the set of all types connected with entity e and the set of all entities connected with type t. By stack- ing multiple graph propagation layers, high-order signal content can be properly captured. We further sum up the embedding information of different lay- ers to get the final entity and type representation, defined as:
Step 3. Contrastive Learning Loss. We adopt cosine similarity 0(0, o) to measure the distance between two embeddings (Zhu et al., 2021). For example, take the nodes u, and Vj in different views, we define the contrastive learning loss of the posi- tive pair of embeddings (u;, Vÿ) as follows:
L-1 Σ
c2 Σ X c2
(2)
Clu;, vy) = log BluaNz)/T Hinta Hinter
where L indicates the number of layers of the Light- GCN. In the same way, we can get the type rep- resentation of cluster interaction x and the cluster representation of type interaction X-t-22 from Gezt, and the entity representation of cluster interaction c2c and the cluster representation of entity interaction x-re2. from Goze.
pasitiva
negnution paires
Hintra
Σ
keSm
Hinter
SE
(4) where: T is a temperature parameter, Hinra and Hinter correspond to the intra-view and inter-view negative objective function. If us is in Gozi and 2y is in Ge2c, then the positive pair embeddings (u,, Pÿ) represents (Z Ze)c) i.e., the i-th node embedding in Ge2r and the j-th node embedding in Geze represent the same entity; after the contrastive lcarning operation, the corresponding node pair em- bedding becomes (Ze,--22>> (o))- Considering that the two views are symmetrical, the loss of the other view can be defined as L(vy, u:). The final loss function to obtain the embeddings is the mean value of all positive pairs loss:
4.2 Cross-view Contrastive Learning
Different views can capture content at different lev- els of granularity. For example, the semantic con- tent of x x25-22 in Geze more coarse-grained than that of X--e2t Gezt- To capture multi-graie -aatiegga information, we use cross-vite view contrastive learn- ing (Zhu et al., 2021; Zou et al., 2022b; Ma et al., 2022) to obtain better discriminative embedding representations. For instance, taking the entity em- bedding as example, for the embeddings X- and x our cross-view contrastive learning module goes through the following three steps:
Le.. mean Σ E(C(u,))) + ((v),u))) (5)
Step 1. Unified Representation. We perform two layers of multilayer perceptron (MLP) operations to unify the dimension from different views as fol- lows:
4.3 Entity Typing Prediction
After performing the multi-view contrastive learn- ing operation, obtain two kinds of entity and
2 Wa(f(Wix' bb)) + b2
(3)
1/ c2c W2(f(WiX' clc b1)) + b2
We assume an arbitrary, but fixed order on nodes.

---

type representation. These two representations in- corporate entities, coarse-grained clusters and fine- grained type knowledge at the same time. In this way, the cluster information fully integrated into the representation of entities and types. For an en- tity e and type t, we obtain their final representation by respectively concatenating ze-e21 and Eite20 and 2,2 and 2iz
I/ respectively represent the number of experts in Mixture-of-Experts and the number of heads. ó and a represent the sofimax and sigmoid activation functions respectively. T; > 0 is the temperature controlling the sharpness of scores.
Prediction and Optimization. We jointly train the multi-view contrastive learning and entity type prediction tasks to obtain an end-to-ent model. For entity type prediction, we adopt the false-ne aware (FNA) loss function (Pan et al., 2021; Jin et al., 2022), denoted LET- We further combine the multi-view contrastive learning loss with the FNA loss, so we can obtain the joint loss function:
L. - 2e-e2il Reteeer 4 Li Li-e2ilkiiaa (6)
Neighbor Prediction Mechanism. The entity and type embeddings are concatenated to obtain ze [zz the embedding dictionary to be used for the entity type prediction task. We found out that there is strong relationship between the neigh- bors of an entity and its types. For unified repre- sentation, we collectively refer to the relational and type neighbors of an entity as neighbors. There- fore, our goal is to find way to effectively use the neighbors of an entity to predict its types. Since different neighbors have different effects tity, we propose neighbor prediction mechanism that each neighbor can perform type prediction independently. For an entity, its i-th neighbor can be expressed as (Z,, r2), where r, represents the relation embedding of the i-th relation. As previ- ously observed (Pan et al., 2021), the embedding of neighbor can be obtained using TransE (Bor- des et al., 2013), we can then perform nonlin- operation it, and further send it to the lin- layer to get its final embedding as follows: N/ (a))) W(z: rr) r.) + b, where W E Wxd b E R. are the learning parameters, and N rep- resents the number of types. We define the em- bedding of all neighbors of entity as follows: W. W (=i.rr): (z9.r2): (W),,)))) .... (.., (s....)), (.s.rr)))) where denotes the number of neighbors of e.
LET =
Σ
B(p.s Pog)log(1 -Pij)
(o)tt))
Σ
logPt.j
(EGez
C = + XLoL 1101//
(8)
where 8 is hyper-parameter used to control the overall weight of negative samples, À and y are hyper-parameters used to control the contrastive loss and L22 regularization, and θ the model parameter set.
Experiments
Datasets
FBISKET
YAGO43KET
# Entities
14,951
42,335
Relations
1,345
37
Types
3,584
45,182
# Clusters
1,081
1,124
Train Train.triples
483,142
331,686
Train.tuples
136,618
375,853
# Valid.tuples
15,848
43,111
# Test.tuples 15,847
43,119
Table I: Statistics of Datasets.
Expert Selection Mechanism. Different neigh- bors of an entity contribute differently the pre- diction of its types. Indeed, sometimes only few neighbors are helpful for the prediction. We in- troduce a Multi-Head Attention mechanism (Zhu and Wu, 2021; Jin et al., 2022) with Mixture-of- Experts (MHAM) to distinguish the information of each head. We compute the final score as:
Datasets. We evaluate our MCLET model on two knowledge graphs, each composed of Itriples and Giypes- For Striples we use the FB15k (Bordes et al., 2013) and YAG043k (Moon et al., 2017). For Stypes. we use the FB15KET and YAGO43kET datasets introduced by (Pan et al., 2021), which map entities from FB15k and YAGO43k to corre- sponding entity types. The statistics of the corre- sponding datasets are shown in Table 1.
Og di (W2(( ( V. bi)) bz)
(7)
p p- O W.)
We compare MCLET with three types of baselines. (i) embedding based models: ETE (Moon et al., 2017), ConnectE (Zhao et al.,
Baselines.
where W1 E Mxd W₂ E H bj E M and by E R are the learnable parameters. M and

---

Datasets
FBISKET
YAGO43ET
Metrics
MRR Hits@1
Hits@3
Hits@ 10
MRR
Hits@1
Hits@3
Hits@
Embedding based methods
ETE (Moon et al., 2017)
0,500
0.385
0,553
0.719
0.230
0.137
0.263
0.422
ConnectR (Zhao al., 2020)
0.590
0.496
0.643
0.799
0.280
0.160
0.309
0.479
CORE (Ge et al., 2021)
0.600
0.489
0.663
0.816
0.350
0.242
0.392
0.550
GNV-based methods
HMGCN (Jin et al., 2019)
0,510
0.390
0.548
0.724
0.250
0.142
0.273
0.437
AttEt (Zhuo al., 2022)
0.620
0.517
0.677
0.821
0,350
0.244
0.413
0.565
ConnectE-MRGAT (Zhao et al., 2022)
0.630
0.562
0.662
0.804
0.320
0.243
0.343
0.482
RACE2T (Zou al., 2022a)
0.640
0,561
0.689
0.817
0.340
0.248
0.376
0.523
CompGCN (Vashishth et al., 2020)
0.665
0.578
0.712
0.839
0.355
0.274
0.383
0.513
RGCN (Pan et ul., 2021)²
0.679
0.597
0.722
0.843
0.372
0.281
0.409
0.549
CET (Pan et al., 2021)
0.697
0.613
0.745
0.856
0.503
0.398
0.567
0.696
MiNer (Jin et al., 2022)
0.728
0.654
0.768
0.875
0.521
0.412
0,589
0.714
Transformer based methods
CoKE (Wang et al., 2019)*
0.465
0.3799
0.510
0.624
0.344
0.244
0.387
0.542
HiuER (Chen et al., 2021)*
0.422
0.333
0.466
0,588
0.240
0,163
0,259
0.390
TET (Hu al., 2022a)
0.717
0.638
0.762
0.872
0.510
0.408
0,571
0.695
Our methods
MCLET-Pool MCLET-MHA MCLET-MHAM
0.726
0.644
0.773
0.881
0.524
0.418
0.589
0.715
0.744
0.670
0.788
0.889
0.540
0.434
0.608
0.729
0.750
0.677
0.793
0.891
0.543
0.436
0.613
0.735
Table 2: Main evaluation results. O results are from the original papers. results are from our implementation of the corresponding models. Best scores highlighted in bold, the second best scores underlined.
2020) and CORE (Ge et al., 2021); (i)) GNN-
variants outperform existing SoTA baselines by a large margin across all metrics. In particular, compared to MiNer (the best performing baseline), our MCLET-MHAM respectively achieves 2.2% and 2.1% improvements on MRR in the FB15KET and YAGO43kET datasets. We lar improvement e.g. on the Hits @ metric, with an increase of 2.3% and 2.4% on FB15KET and YAG043kET, respectively. Our ablation studies below show the contribution of MCLET's compo- nents on the obtained improvements.
based models: HMGCN (Jin et al., 2019), At- tEt (Zhuo et al., 2022), ConAectE-MRGAT (Zhao
et al., 2022), RACE2T (Zou et al., 2022a),
CompGCN (Vashishth et al., 2020) RGCN (Pan al., 2021), CET al. 2021), and MiNer (Jin et al., 2022); (ii)) transformer-based models: CoKE (Wang al., 2019), HitER (Chen al., 2021) and TET (Hu et al., 2022a).
simi-
Evaluation Protocol. For every pair (e, in the test set, we obtain ranking list for the possi- ble types f. We choose five automatic evaluation metrics: mean rank (MR), mean reciprocal rank (MRR), and @its@k (KE {1, 3, )0)), MR measures the average positions of the first correct answer in a list of ranked results, MRR defines the inverse of the rank for the first correct answer, Hits @k calcu- lates the percentage of correct types ranked among the top-k, in addition the MR metric, the larger the value, the better the effect. Following the eval- uation protocol in most entity typing works (Pan et al., 2021; Hu et al., 2022a; Jin et al., 2022), all metrics are reported under the filtered setting (Bor- des et al., 2013).
We have evaluated three variants of MCLET to explore the effectiveness of the expert selec- tion mechanism: MCLET-Pool, MCLET-MHA. and MCLET-MHAM, MCLET-Pool and MCLET- MHA respectively replace our expert selection mechanism with the pooling approach introduced in CET and the type probability prediction module introduced in MiNer. We observe that the MHAM variant achieves the best results. For instance, on FB 1KET,, MHAM improves 2.4% and 0.6% over the Pool and MHA variants on the MRR metric. This can be intuitively explained by the fact that the neighbors of an entity have different contribu- tions to the prediction of its types. Indeed, by using the expert selection strategy, the information ob- tained by each head can be better distinguished. As a consequence, a more accurate final score can be
5.1 Main Results
The empirical results on entity type prediction are reported in Table We can see that all MCLET

---

Datasets
FBISKET
YAGO4RET
Setting
MRR
MR
Hits@1
Hits@3
Hits@ 10
MRR
MR
Hits@l 1
Hits@
Hits@ 10
w/o e2t
0.738
13
0.662
0.783
0.887
0.519
249
0.413
0.704
w/o c2t
0.745
12
0.670
0.788
0.891
0.540
170
0.433
0.609
0.731
w/o e2c
0.720
16
0.641
0.765
0.874
0.527
226
0.423
0.595
0.712
w/o
0.683
20
0.601
0.726
0.843
0,470
498
0.365
0.535
0.658
MCLET MCLET-MHAM
0.750
12
0.677
0.793
0.891
0,543
167
0.436
0.613
0.735
Table 3: Evaluation of ablation experiments with different views on 15KET and YAGO43KET. Best scores are highlighted in bold.
Datasets
FBI5KET
YAGO43KET
Setting
MRR
MR
Hits@
Hits@3
Hits 10
MRR
MR
Hits@1
Hits@3
Hits@ 10
1=1
0.744
11
0.670
0.786
0.888
0,543
167
0.436
0.613
0.735
1=2
0.747
12
0.671
0.792
0.894
0.524
219
0.411
0.595
0.726
(all)
3=3
0.747
12
0.670
0.793
0.894
0.340
405
0.245
0.377
0.527
[=4
0.750
12
0.677
0.793
0.891
0.220
1368
0.160
0.237
0.330
I=1
0.768
42
0.708
0.813
0.873
0.477
620
0.407
0.517
0,599
1=2
0.752
42
0,685
0.804
0.864
0.441
751
0.372
0,480
0.564
(1-4)
7=3
0.735
41
0.672
0.771
0.859
0.322
1125
0.267
0.350
0.419
E=4
0.674
48
0.603
0.712
0.809
0.297
1357
0.253
0.321
0.367
Table 4: Evaluation of ablation experiments with different LightGCN layers on FB 15KET and YAGO43kET, where [all] indicates the complete dataset, and {1 [1-4] indicates that the entities in the dataset only contain 1 to 4 type neighbors. Best scores are highlighted in bold.
obtained based on the prediction scores of each of the neighbors.
mainly because the cluster-type graph is smaller and denser, so the difference in the discriminative features of nodes is not significant.
5.2 Ablation Studies
Effect Different LightG Layers. We have observed that the number of layers on the FB FB15KTT and YAG043kET datasets has a direct effeet on the performance of LightGCN, cf. Table 4. For FB 15KT, the impact of the number of GCN layers the performance is relatively small. However, for YAGO43kET, the performance sharply declines as the number of layers increase. The main reason for this phenomenon is that in the YAGO43kET dataset most entities have a relatively small num- ber of types. As a consequence, the entity-type and entity-cluster views are sparse. So, when deeper graph convolution operations are applied, multi- hop information is integrated into the embeddings through sparse connections. As consequence, noise also introduced, which has a negative im- pact on the final results. We further constructed a dataset where each entity contains between to 4 type neighbors and performed experiments with LightGCN with layer numbers ranging from 1 to 4. In this case, we can observe that as the number of GCN layers increase, there is a significant decline in the performance FB 15KET as well. This is in line with the finding that the performance decreases in YAGO43kET as the number of layers increases.
To understand the effect of each of MCLET's com- ponents on the performance, we carry out ablation experiments under various conditions. These in- clude the following three aspects: a) the content of different views, see Table 3; b) different LightGCN layers, Table 4; c) different dropping rates, Table 5. Other ablation results and complexity analysis can be found in Appendix B and C.
Effect of Different Views. We observe that remov- ing any of the views most of the time will result in decrease in performance, of. Table 3. Further, if all three views removed there will be substantial performance drop in both datasets. For instance, the removal of all three views brings a decrease of 7.3% of the MRR metric on both datasets. This strongly indicates that the introduction of the three views is necessary. Intuitively, this is explained by the fact that each view focuses on a different level of granularity of information. Using the cross- then incorporate
view contrastive learning, different levels of knowledge into entity and type embeddings. We can also observe that the perfor- mance loss caused by removing the cluster-type view is much lower than that caused by remov- ing entity-type and entity-cluster views. This is
Effect of Dropping Rates of Relation Neighbors.

---

Dropping Rates
25%
50%
75%
90% %
Models
MRR
H@1 1 H@3
MRR
H@1
H@3
MRR
H@1
H@3
MRR
H@1
H@3
CompGCN (Vashishth et al., 2020)
0.661 0.573
0.705
0.655
0.565
0.702
0.648
0.559
0.697
0.633
0.544
0.679
RGCN (Pan al., 2021)
0.673 0.590
0.716
0.667
0.584
0.708
0.648
0.560
0.694
0.626
0.534
0.673
CET (Pan et ul., 2021)
0.697 0.613
0.744
0.687
0.601
0.733
0.670
0.580
0.721
0.646
0.553
0.698
TET (Hu al., 2022a)
0.712 0.631
0.758
0.705
0.624
0.753
0.689
0.606
0.733
0.658
0.574
0.701
MiNer (Jin et al., 2022)
0.714
0.634
0.760
0.703
0.620
0.749
0.683
0.596
0.731
0.652
0.556
0.706
MCAMT-MHAM
0.742 0.665
0.788
0.732
0.653
0.777
0.718
0.636
0.765
0.700
0.614 0.751
Table 5: Evaluation with different relational neighbors dropping rates on FB 15KET. H@N is an abbreviation for
Hits@N, N, N E (1,3). Best scores are highlighted in bold.
Dropping Rates
25% %
50%
75%
90 %
Models
MRR
H@l 1 H@3
MRR
H@1
H@3
MRR
H@1 I H@3
MRR
H@1
H@3
CompGCN (Vashishth al., 2020)
0.664
0.578
0.708
0.662
0.574
0.708
0.653
0.565
0.699
0.637
0.546 0.683
RGCN (Pan et al., 2021)
0.676
0.593 0.719
0.673
0.590
0.719
0.658
0.573 0.702
0.636
0.548 0.681
CET (Pan al., 2021)
0.699 0.617 0.743
0.694
0.610
0.742
0.675
0.588
0.721
0.653
0.564
0.700
TET (Hu et al., 2022a)
0.711 0.631
0.756
0,710
0.630
0.757
0.690
0.608
0.734
0.677
0.591
0.722
MiNer (Jin et al., 2022)
0.716
0.638
0.759
0.713
0.634
0.756
0.687
0.603
0.733
0.658
0.564 0.712
MCLET-MIIAM
0.747 0.674 0.790
0.747
0.675 0.787
0.729
0.652 0.773
0.703
0.619 0.750
Table Evaluation with different relation types dropping rates on FB15KET. H@N is an abbreviation for Hits@N, Ne E (1,3). Best scores are highlighted in bold.
Relational neighbors of an entity provide support- ing facts for its representation. To verify the ro- bustness of MCLET in scenarios where relational neighbors are relatively sparse, we conduct an ab- lation experiment on FB I 5kET by randomly re- moving 25%, 50%, 75%, and 90% of the relational neighbors of entities, as proposed in (Hu et al., 2022a). We note that even after removing different proportions of relational neighbors, MCLET still achieves optimal performance. This can be mainly explained by two reasons. On the one hand, our views are based solely on the entity type neighbors, without involving the entity relational neighbors. Thus, changes in relational neighbors do not signif- icantly affect the performance of MCLET. On the other hand, relational neighbors only serve auxil- iary knowledge for entity type inference, while the existing type neighbors of entities play a decisive role in predicting the missing types of entities.
neighbors. This is mainly due to the introduction of cluster information, which establishes a coarse- grained bridge between entities and types, this in- formation is not affected by drastic changes in the structure of the knowledge graph. Therefore, the incorporation of cluster information is necessary for entity type prediction tasks.
Conclusions
We propose MCLET, multi-view contrastive learning (CL) framework for KGET. We design three different views with different granularities, and use CL strategy achieve cross-view coop- eratively interaction. By introducing multi-head at- mechanism, we
tention with a Mixture-of-Exppe can combine different neighbor prediction scores. For future work, we plan to investigate inductive scenarios, dealing with unseen entities and types.
Acknowledgments
Effeet of Dropping Rates of Relation Types. Compared with YAGO43KET, FB1SKET has much more relations. To verify the robustness of MCLET when the number of relations is small, similar to (Hu et al., 2022a), we randomly remove 25%, 50%, 75%, and 90% of the relation types in FB15KET. From Table 6, we can observe that even with smaller number of relation types, MCLET still achieves the best performance. This demon- strates the robustness of our method in the presence of significant variations in the number of relational
This work has been supported by the Na- tional Natural Science Foundation of China (No.61936012, No.62076155), by the Key Re- search and Development Program of Shanxi Province (N0.2021020201010088) by the Sci- ence and Technology Cooperation and Ex- change Special Program of Shanxi Province NN..202240411 10120000 101016), by the Chang Jiang Schol- ars Program (J2019032) and by a Leverhulme Trust Research Project Grant RRPG-2021-140)

---

Association for 7 Limitations
the 2019 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HILT 2019, Minneapolis, MN. USA, June 2-7, 2019, pages 2913- 2923. Association for Computational Linguistics.
In this paper, we introduce coarse- grrsind clus- ter content for the knowledge graph entity typing task. Although we achieve good results, there are still limitations in the following aspects: 1) For the standard benchmark datasets we use the read- ily available cluster-level annotation information. However, for those datasets without cluster infor- mation, we would need to use clustering algorithms to construct implicit cluster semantic structures. 2) There is a related task named fine-grained entity prediction (FET), the difference lies in predicting the types of entities that are mentioned in given sentence, rather than entities present in knowl- edge graph. The corresponding benchmarks also have annotated coarse-grained cluster information. Therefore, it would be worthwhile exploring the transferability of MCLET to the FET task.
Xiou Ge. Yuncheng Wang, Bin Wang, and C.-C. Jay Kuo. 2021. CORE: A knowledge graph entity type prediction method via complex space regression and embedding. CoRR,
Nitish Gupta, Sameer Singh, and Dan Roth. 2017. En- tity linking via joint encoding of types. descriptions. and context. In Proceedings of the 2017 Conference Empirical Methods in Natural Language Process- ing, EMNLP 2017, Copenhagen. Denmark, Septem- ber 9-11, 2017, pages Computational Linguistics.
Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yong- Dong Zhang, and Meng Wang. 2020. Lightgen: Sim- plifying and powering graph convolution network for recommendation. In Proceedings of the 43rd In- ternational ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020, Virtual Event, China, July 25-30, 2020, pages 639- 648, ACM.
References
Kurt D. Bollacker, Colin Evans, Praveen K. Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the ACM SIG- MOD International Conference on Management of Data, SIGMOD 2008, Vancouver, BC, Canada, June 10-12, 2008 pages 1247-1250. ACM.
Nan Hu, Yike Wu, Guilin Qi, Dehai Min, Jiaoyan Chen, Jeff Z Pan, and Zafar Ali. 2023, An Empirical Study of Pre-trained Language Models in Simple Knowl- edge Graph Question Answering. In Journal of World Wide Web, pages 1-32.
Zhiwei Hu, Víctor Gutiérrez- Basulto,, Zhiliang Xiang, Ru Li, and Jeff Z. Pan. 2022a. Transformer-basee entity typing in knowledge graphs. In Proceedings of the 2022 Conference Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022. pages Linguistics.
Antoine Bordes, Nicolas Usunier, Alberto García- Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi- relational data. In Advances in Neural Informa- tion Processing Systems 26: 27th Annual Confer- ence on Neural Information Processing Systems, NeurIPS2013, Lake Tahoe, Nevada, United States, December 5-8, 2013, pages 2787-2795.
Empirical Methods in
Association for Computational
Zhiwei Hu, Victor GutiémezzBasulto, Zhiliang Xiang, Xiaoli Li. Ru Li, and Jeff Z. Pan. 2022b. Type-aware embeddings for multi- hop reasoning over knowledge graphs. In Proceedings of the Thirty-First Interna- tional Joint Conference Artificial Intelligence, IJ- CAI 2022, Vienna, Austria, 23-29 July 2022, pages 1078-3084. ijcai.org.
Sanxing Chen, Xiaodong Liu, Jianfeng Gao, Jian Jiao, Ruofei Zhang, and Yangfeng Ji. 2021. Hitter: Hierar- chical transformers for knowledge graph embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event/ Punta Cana, Dominican Re- public, 7-11 November, 2021, pages 10395-10407. Association for Computational Linguistics.
Hailong Jin, Lei Hou, Juanzi Li, and Tiansi Dong. 2019. Fine- grained entity typing via hierarchical multi graph convolutional networks. In Proceedings of the 2019 Conference on Empirical Methods in Nat- ural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, Novem- ber 3-7, 2019, pages 4968-4977 Association for Computational Linguistics.
Xiang Chen, Ningyu Zhang, Lei Li, Shumin Deng, Chuanqi Tan, Changliang Xu, Fei Huang, Luo Si, and Huajun Chen. 2022. Hybrid transformer with multi-level fusion for multimodal knowledge graph completion. In Proceedings of the 45th International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2022, Madrid, Spain, July 11 11-15, 2022, pages 904-915. ACM.
Zhuoran Jin, Pengfei Cao, Yubo Chen, Kang Liu, and Jun Zhao. 2022. A good neighbor, A found treasure: Mining treasured neighbors for knowledge graph en- tity typing. In Proceedings of the 2022 Conference
Yu Chen, Lingfei Wu, and Mohammed J. Zaki. 2019. Bidirectional attentive memory networks for question answering over knowledge bases. In Proceedings of

---

on Empirical Methods in Natural Language Process- ing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 480-490. Association for Computational Linguistics.
Jeff Z. Pan, Mei Zhang, Kuldeep Singh, Frank Van Harmelen, Jinguang Gu, and Zhi Zhang. 2019. Entity Fnabled Relation Linking. In Proc. of 18th Interna- rional Semantic Web Conference (ISWC 2019), pages 523-538.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In 3rd Inter- national Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. OpenReview.net.
J.Z. Pan, D. Calvanese, T. Eiter, I. Horrocks, M. Kifer, F. Lin, and Y. Zhao. 2017b, Reasoning Web: Logical Foundation of Knowledge Graph Construction and Querying Answering. Springer.
Weiran Pan, Wei Wei, and Xian-Ling Mao. 2021. Context-aware entity lyping in knowledge graphs. In Findings of the Association for Computational Lin- guistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021, pages 2240- 2250. Association for Computational Linguis- tics.
Thomas N. Kipf and Max Welling, 2017. Semi- supervised classification with graph convolutional networks. In Sth International Conference on Learn- ing Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
Xiaoya Li, Fan Yin, Zijun Sun, Xiayu Li, Arianna Yuan, Duo Chai, Mingxin Zhou, and Jiwei Li. 2019. Entity- relation extraction as multi-turn question answering. In Proceedings of the 57th Conference of the Asso- ciation for Computational Linguistics. ACL 2019, Florence, Italy, July 28- August 2, 2019, pages 1340- 1350. Association for Computational Linguisties.
Miao Peng, Ben Liu, Qianqian Xie, Wenjie Xu, Hua Wang, and Min Peng. 2022. Smile: Schema- augmented multi-level contrastive learning for knowl- edge graph link prediction. In Findings of the Associ- ation for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 0165-4177 Association for Computa- tional Linguistics.
Yang Liu, Zequn Sun, Guangyao Li, and Wei Hu. 2022. I know what you do not know; Knowledge graph em- bedding via co co-distillation learning. In Proceedings of the 31th ACM International Conference Infor- mation and Knowledge Management, CIKM 2022, Atlanta, GA, USA, October 17-21, 2022, pages 1329- 1338. ACM.
Michael Sejr Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titoy, and Max Welling. 2018. Modeling relational data with graph convolutional networks. In the Semantic W5h International Conference, ESWC 2018, Heraklion, Crete. Greece, June 3-7, 2018, pages 593-607 Springer.
Yunshan Ma. Yingzhi He, An Zhang, Xiang Wang, and Tat-Seng Chua. 2022. Crosscbr: Cross-view con-
Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: a core of semantic knowledge. In Proceedings of the 16th International Conference World Wide Web, WWW 2007, Banff, Alberta, Canada, May 8-12, 2007, pages 697-706. ACM.
trastive learning for bundle recommendation. In the 28th ACM SIGKDD Conference on Knowledge Dis- covery and Data Mining, Washington, KDD 2022,
DC, USA, August 14-18, 2022, pages 1233-1241. ACM.
Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha P. Talukdar. 2020, Composition-buns based multi- relational graph convolutional networks. In 8th Inter- national Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net.
George A. Miller. 1995. Wordnet: A lexical database for english. Commun. ACM, 38(11):39-44
Changsung Moon, Paul Jones, and Nagiza F. Samatova. 2017. Learning entity type embeddings for knowl- edge graph completion. In Proceedings of the 26th ACM International Conference on Information and Knowledge Management, CIKM 2017, Singapore, November 06-10, 2017, pages 2215 2215-2218. ACM.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez. Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems 30: Annual Conference on Neural In- formation Processing Systems, NeurlPS 2017, Long Beach, CA, USA, December 4-9, 2017, pages 5998- 6008.
Guanglin Niu, Bo Li. Yongfei Zhang, and Shiliang Pu. 2022. CAKB; A scalable commonsense-awan frame- work for multi-view knowledge graph completion. In Proceedings of the 60th Anmual Meeting of the As- sociation for Computational Linguisticx, ACI. 2022, Dublin, Ireland, May 22-27, 2022, pages 2867-2877. Association for Computational Linguistics.
Quan Wang, Pingping Huang, Haifeng Wang. Songtai Dai, Wenbin Jiang. Jing Liu, Yajuan Lyu, Yong Zhu, and Hua Wu. 2019. Coke: Contextualized knowledge graph embedding. CoRR, abs/191 I.. 1.02168.
J. Z. Pan, G. Vetere, J.M. Gomez-Perez, and H. Wu, ed- itors. 2017a. Exploiting Linked Data and Knowledge Graphs for Large Organisations. Springer.
Kemas Wiharja. Jeff Pan, Martin J. Kollingbaum, and Deng. 2020. Schema Aware Iterative Knowledge Graph Completion. Journal of Web Semantics.

---

Xin Xie, Ningyu Zhang, Zhoubo Li. Shumin Deng, Hui Chen, Feiyu Xiong, Mosha Chen, and Huajun Chen. 2022. From discrimination to generation: Knowl- edge graph completion with generative transformer. In Companion of The Web Conference, WWW 2022, Virtual Event/ / Lyon, France, April 25-29, 2022, pages 162-165. ACM.
Yu Zhao, Anxiang Zhang, Ruobing Xie, Kang Liu, and Xiaojie Wang. 2020. Connecting embeddings for knowledge graph entity typing. In Proceedings of the 58th Anmual Meeting of the Association for Com- putational Linguistics, ACI 2020, Online, July 5-10, 2020, pages 6419-6428. Association for Computa- tional Linguisties.
Yu Zhao, Han Zhou, Anxiang Zhang. Ruobing Xie, Qing Li, and Fuzhen Zhuang. 2022. Connecting embeddings based multiplex relational graph at- tention networks for knowledge graph entity typing. IEEF Transactions on Knowledge and Data Engi- neering.
Ke Zhu and Jianxin Wu. 2021. Residual attention: A simple but effective method multi-label recog- nition. In 2021 IEEE/CVF International Confer- ence on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 184-193. IEEE,
Man Zhu, Zhiqiang Gao, Jeff Pan, Yuting Zhao, Ying Xu, and Zhibin Quan. 2015. Thox learning from in- complete data by inference in belnet+. Knowl. Based Syst. 75:30-40.
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. 2021. Graph contrastive learning with adaptive augmentation. In Companion of The Web Conference, WWW 2021, Virtual Event Ljubl- jana, Slovenia, April 19-23, 2021, pages ACM.
Jianhuan Zhuo, Qiannan Zhu, Yinliang Yue, Yuhong Zhao, and Weisi Han. 2022. A neighborhood- attention fine-grained entity typing for knowledge graph completion. In the Fifteenth ACM Interna- tional Conference on Search Data Mining, WSDM 2022, Virtual Event / Tempe, AZ, USA, Febru- ary 21-25, 2022, pages 1525-1533. ACM.
Changlong Zou, Jingmin An, and Guanyu Li. 2022a. Knowledge graph entity type prediction with rela- tional aggregation graph attention network. In the Semantic Weh-19th International Conference, ESWC 2022, Hersonissos, Crete, Greece, May 29-Jume 2, 2022, pages 39-55. Springer.
Ding Zou, Wei Wei, Xian-Ling Mao, Ziyang Wang. Minghui Qiu, Feida Zhu, and Xin Cao. 2022b. Multi- level cross-view contrastive learning for knowledge- aware recommender system. In Proceedings of the 45th International ACM SIGIR conference re- search and development in Information Retrieval, SIGIR 2022, Mudrid, Spain, July 11-15, 2022, pages 1358-1368. ACM.

---

05
270
22, END
10.0
0000
N50
04
LU4
04
02
02
02
19019407
110/5601
Different head numbers
(b) Different loss weight
(c) Different temperature
Figure 3: The ablation studies results under different experimental conditions.
Appendix
We observe that the selection of different numbers of attention heads has slight impact the per- formance. should be noted that, shown in Equation using more heads will introduce more learning parameters, so when the performance the model equivalent, is better to use fewer heads.
A Details about Experiments
Parameter
(BBSSKET, YAG043KET)
Embedding dimensions
{100, 100)
Learning rate
(0.001, 0.001)
# Temperature
[0.6, 0.6)
# LightGCN layers
11
# Number of heads
(5, 5)
Effect of Different Loss Weight X. The parame- À in Equation determines the importance of the contrastive loss during the multi-loss training process. We set the weight values of the contrastive loss as E {0.1, 0.01, 0.001, 0.0001} to investigate its impact on the final entity typing prediction re- sults. From Figure 3 (b), one can see that different weight values have minimal effect on the over- all performance. This demonstrates the robustness of MCLET in terms of setting the weight for the contrastive loss, ensuring that the results do not undergo drastic changes due to improper weight values.
# Number of experts
(32, 32)
# B value
A value
(0.001, 0.001)
# 7 value
[le-S, le-5)
Table 7: The best hyperparameters of MCLET model in different datasets.
Hyperparameter Settings. All experiments were carried out on 32G Tesla V100 GPU, we use Adam (Kingma and Ba, 2015) as the optimizer, and determine the final value of each hyperparam eter based on the MRR value the validation set by using grid search. We fine-tune the hyper- parameters including the number of embedding dimensions from d E (50, 100, 150, 200), and the learning rate from E {0.001, 0.005, 0.01), }. the temperature parameter in contrastive loss I E {0.4, 0.5, 0.7). the number of Light- GCN layers L E 21,2,3, 3,2,, 4). the number of heads 1,4,5,,4 5, 6). the numbers of experts M {16, 32, 61), the weight of negative samples in FNA loss / E 2,,2,3, 4), the weight of contrastive loss À E (0.1, 0.001, 0.0001), the weight of L2 regularization 7 Elcc-5, 2c-5, 3c-5). Table 7 summarizes the best configurations in two datasets.
Effect of Different Temperature т. The temper- ature coefficient can be used to adjust the similarity measurement between samples. A higher tempera- ture coefficient will flatten the distribution of simi- larities, causing the model to pay more attention subtle differences between samples. On the other hand, lower temperature coefficient will concen- trate the distribution of similarities, emphasizing the overall differences between samples. From Figure (c), it can be observed that setting dif- ferent temperature coefficients has limited impact MCLET. Although the choice of temperature coefficient usually requires adjustment based on specific tasks and datasets, in our current model context, the selection of this coefficient does not have a decisive effect on the final performance the model.
H < (3,4,5,
B Additional Results
Effeet of Different Head Numbers. In Figure (a) we report the results testing the performance different numbers of attention heads MHAM.

---

Complexity Analysis
Model
FBISKET YAG043KET
CompGCN (Vashishth al., 2020)
98.395M
604.690M
RGCN (Pan ct al., 2021)
45.875M
578.330M
CET (Pan al., 2021)
504.627M
6.362G
TET (Hu al., 2022a)
2.016G
8,406G
MiNer (Jin et al., 2022)
1.127G
14,204G
MCLET
1.129G
13.620G
Table The amount calculations required by differ- ent models different datasets.
Computational Complexity. Considering that the prediction accuracy of embedding based meth- ods for KGET is relatively lower, we compared the computational complexity of MCLET with GNN- based models like CompGCN, RGCN, CET, TET, and MiNer, the corresponding results shown in Table 8. The computational cost refers to the num- ber of floating-pott point operations (FLOPs) required during training or inference. We incur greater computational overhead compared to CompGCN, RGCN, and CET, but the performance of these three baselines significantly lower than that of MCLET. In comparison to the SoTA models TET and MiNer, require less computational re- sources than TET the FB15KET dataset, and than MiNer the YAG043KET dataset. This in- dicates that we can achieve optimal performance without introducing a substantial computational overhead. Additionally, note that for the FB 15kET and YAGO43kET datasets, the MCLET model respectively converges in only 39 minutes and 7 hours 10 minutes, for Tesla V100 32G graphics card, which is reasonable time cost.


Reviews:
[{"review_id": "lYSCQPvldW", "reviewer": "Reviewer_Hx7N", "summary": "", "questions": "1. How are the clusters of types generated or obtained? To my knowledge, YAGO doesn't contain such information.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">  **Q1**: *It's unclear how the types are clustered or how are the clusters of types generated or obtained? To my knowledge, YAGO doesn't contain such information.*  \n\n**Answer:** \n1. For the FB15kET dataset, we can directly obtain cluster information using a rule-like approach based on their type annotations. For instance, the type *\"/location/uk_overseas_territory\"* belongs to the cluster *\"location\"* and the type *\"/education/educational_degree\"* belongs to the cluster *\"education\"*. \n2. For the YAGO43kET dataset, as mentioned in the submission, it provides an alignment between types and WordNet concepts. So, we can directly obtain the words in WordNet describing the cluster to which a type belongs to. For example, for the type *\"wikicategory_People_from_Dungannon\"*, its cluster is *\"wordnet_person_100007846\"*, and for the type *\"wikicategory_Male_actors_from_Arizona\"*, its cluster is *\"wordnet_actor_109765278\"*.\n3. Furthermore, one can find the following description on the official website of the YAGO dataset: *\"YAGO classifies each entity into a taxonomy of classes. Every entity is an instance of one or multiple classes. Every class (except the root class) is a subclass of one or multiple classes. This yields a hierarchy of classes — the taxonomy. The taxonomy consists of 4 layers.\"* The introduction to the third layer is as follows: *\"The middle layer of the taxonomy consists of classes that have been derived from Wikipedia categories. For example, one class is <wikicategory_American_rock_singers>, derived from the Wikipedia category American rock singers. Each of these classes is connected to one class of the WordNet layer by a rdfs:subclassOf relationship. In the example, <wikicategory_American_rock_singers> rdfs:subclassOf <wordnet_singer_110599806>.\"* So, at this layer every type from Wikipedia can be aligned with a category from WordNet. This is the data source for us to obtain the type cluster information in the YAGO dataset.\n\n>  **Q2**: *It doesn't make sense to me how the entity-entity relationships are not considered and missing in the model.*  \n\n**Answer:** \n1. Note that we did not directly utilize the entity-entity graph, but we indirectly incorporated information from the entity-entity graph into the *\"Neighbor Prediction Mechanism\"* of the *\"Entity Typing Prediction\"* module. In this mechanism, we utilize the relational neighbors and type neighbors of entities to predict missing entity types. The relational neighbors of entities are derived from the entity-entity graph. \n2. There are two main reasons to not directly employ the entity-entity graph. \n    - On the one hand, as discussed in the *\"Task Definition\"*, a knowledge graph comprises two subgraphs: $\\mathcal G_{\\textit{triples}}$ and $\\mathcal G_{\\textit{types}}$. The former corresponds to the entity-entity graph you referred to, while the latter represents the entity-type graph. Given the goal of the KGET task, our focus is on finding ways to better utilize the entity-type graph. We introduced type cluster information into the entity-type graph, constructing three views: entity-type, cluster-type, and entity-cluster, and employed cross-view contrastive learning to obtain better embeddings. \n    - On the other hand, the entity-entity graph is a multi-relational graph, encoding it would require a computation-intensive encoder such as RGCN. In contrast, the three views we acquired (entity-type, cluster-type, and entity-cluster) are single-relational graphs. For them, a lightweight encoder like LightGCN suffices. This choice ensures performance enhancement without introducing excessive computational burden. \n3. In addition, we have conducted a computational cost analysis of MCLET compared to CompGCN, RGCN, CET, TET, and MiNer models on the FB15kET and YAGO43kET datasets (as shown in the table below). The computational cost refers to the number of floating-point operations (FLOPs) required during training or inference. We can observe that, when compared to the SoTA models TET and MiNer, MCLET exhibits lower computational costs on the FB15kET dataset than TET and on the YAGO43kET dataset than MiNer. \n\n\nModel | FB15kET | YAGO43kET\n| :--: | :--: | :--: |\nCompGCN | 98.395M | 604.690M\nRGCN | 45.875M | 578.330M\nCET | 504.627M | 6.362G\nTET | 2.016G | 8.406G\nMiNer | 1.127G | 14.204G\nMCLET | 1.129G | 13.620G", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. It's novel to consider coarse-to-fine entity types through type clusters.\n2. The content is solid, and the experiments are thorough and complete.", "reasons_to_reject": "1. It's unclear how the types are clustered (see questions below).\n2. It doesn't make sense to me how the entity-entity relationships are not considered and missing in the model.\n\n===== After Rebuttal =====\n\nThanks to the author for the detailed response. It addressed most of my questions and concerns. However, I decided not to change the current scores.", "questions_to_authors": "1. How are the clusters of types generated or obtained? To my knowledge, YAGO doesn't contain such information.", "paper_topic_and_main_contributions": "This paper proposed a new approach, named MCLET, to predict missing entity types in knowledge graphs (KGs). Relationships between entities, coarse-grained, and fine-grained types are considered. Three modules are included in MCLET to process multi-view KGs, cross-view KGs, and entity type prediction, respectively.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "5I8EBhmjVb", "reviewer": "Reviewer_sXvZ", "summary": "", "questions": "1. Can your model handle unseen entities?\n\n2. The proposed method consists of multiple modules and I have seen similar modules from different applications. Which newly proposed modules do you think are the most important to the overall framework?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">**Q1:** *I am not sure familiar with all previous models on entity tying. It would be a bit shock to me if the previous methods have never considered multi-view information aggregation of KGs. If a similar concept exists, how the methods and motivations are different from the current manuscript?*  \n\n**Answer:** \n1. To our knowledge, existing models have not adopted a multi-view approach for the KGET task. The primary reason is that the existing baseline models have not incorporated cluster knowledge. So, different perspectives of knowledge are not formed. By introducing cluster information, we established three levels of information: entity, coarse-grained cluster, and fine-grained type. We can construct a view by considering only two levels, and then the three different views can be formed by combining the three in pairs. However, current methods only consider information at the entity and fine-grained type level. \n2. In a nutshell, the main innovation points of our submission are: \n    - Firstly, we introduced coarse-grained type cluster knowledge into the KGET task for the first time, creating three distinct views: entity-type, cluster-type, and entity-cluster, and different views can capture content at different levels of granularity.\n    - Then, we use a cross-view contrastive learning mechanism to capture multi-grained information to obtain better embedding representations. \n    - Finally, we devise a multi-head attention with a Mixture-of-Experts strategy to distinguish the contribution from different neighbors. \n\n>**Q2:** *The proposed method includes a couple of modules and builds up a quite complex model. How efficient is the proposed method compared to the previous method? Especially compared with GNN-based and embedding-based methods. A complexity analysis would be very helpful.*  \n\n**Answer:** \n1. Considering that the prediction accuracy of embedding-based methods for KGET is low, we compared the computational complexity of MCLET with the GNN-based models like CompGCN, RGCN, CET, TET, and MiNer, see the table below. The computational cost refers to the number of floating-point operations (FLOPs) required during training or inference. We incur greater computational overhead compared to CompGCN, RGCN, and CET, but the performance of these three baselines is significantly lower than that of MCLET. In comparison to the SoTA models TET and MiNer, we require less computational resources than TET on the FB15kET dataset, and than MiNer on the YAGO43kET dataset. This indicates that we can achieve optimal performance without introducing a substantial computational overhead. \nModel | FB15kET | YAGO43kET\n:--: | :--: | :--:\nCompGCN | 98.395M | 604.690M\nRGCN | 45.875M | 578.330M\nCET | 504.627M | 6.362G\nTET | 2.016G | 8.406G\nMiNer | 1.127G | 14.204G\nMCLET | 1.129G | 13.620G\n2. Additionally, note that for the FB15kET and YAGO43kET datasets, the MCLET model respectively converges in only 39 minutes and 7 hours 10 minutes, for a Tesla V100 32G graphics card, which is a reasonable time cost.\n\n>**Q3:** *Can your model handle unseen entities?*  \n\n**Answer:** \n- No, MCLET cannot handle unseen entities. Not only MCLET but also all baselines, like CET, TET, and MiNer, are unable to handle unseen entities (in other words, the inductive scenario). Indeed, there are not KGET task datasets that cater to this inductive condition. This is a promising direction for exploration, and we intend to further develop this aspect in our future work.\n\n>**Q4:** *The proposed method consists of multiple modules and I have seen similar modules from different applications. Which newly proposed modules do you think are the most important to the overall framework?*  \n\n**Answer:** \n- In terms of the impact on model performance, the *\"Multi-view Generation and Encoder\"* and *\"Cross-view Contrastive Learning\"* are the most important. This follows from the ablation experiments shown in Table 2. The experiments demonstrated that removing all views led to a significant performance drop, both on the FB15kET and YAGO43kET datasets. This aligns with the initial motivation of introducing type cluster knowledge to the KGET task. Indeed, these results confirm the importance of constructing multiple views and encoding them after incorporating cluster knowledge. Furthermore, the embeddings obtained from different views capture  specific information of each view, without considering interactions between embeddings across views. We addressed this by introducing a cross-view contrastive learning mechanism, which facilitated mutual awareness between different view contents. This enhancement led to more discriminative embedding representations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The idea of incorporating multi-view information is intuitive. It divides KG into several aspects and handles them separately. Finally, all information is fused to make the final decision for entity types.\n\n2. Good performance is shown compared to previous models.", "reasons_to_reject": "1. I am not sure familiar with all previous models on entity tying. It would be a bit shock to me if the previous methods have never considered multi-view information aggregation of KGs. If a similar concept exists, how the methods and motivations are different from the current manuscript?\n\n2. The proposed method includes a couple of modules and builds up a quite complex model. How efficient is the proposed method compared to the previous method? Especially compared with GNN-based and embedding-based methods. A complexity analysis would be very helpful.", "questions_to_authors": "1. Can your model handle unseen entities?\n\n2. The proposed method consists of multiple modules and I have seen similar modules from different applications. Which newly proposed modules do you think are the most important to the overall framework?", "paper_topic_and_main_contributions": "The paper targets entity type prediction in KGs. It proposes multi-view modeling of entity neighbouring information and a mixture of multiple aspects to make the final prediction and achieve better results.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "UPVgrampHB", "reviewer": "Reviewer_sir8", "summary": "", "questions": "1. Can you please compare the computational complexity of different approaches for entity typing?\n2. Need more introduction and explanation of LightGCN\n3. At the bottom of page 4, please give a better explanation of reason behind adding up graph convolutional layers.\n4. Please explain more clearly how to find positive samples and negative samples, perhaps with a figure illustration.\n", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ">**Q1:** *My only concern is the model might be overly complex and how applicable is it to type prediction for large knowledge graphs, and whether this increase in complexity brings enough benefit.*   \n\n**Answer:** \n1. Regarding the analysis of computational complexity, we compared our model with the five baselines, see details in Q2 below. \n    - On the one hand, compared to the SoTA models TET and MiNer, MCLET did not introduce any additional computational overhead. It even required less overhead on the YAGO43kET dataset than the MiNer model, and less overhead on the FB15kET dataset than TET model. \n\n    - On the other hand, with respect to the MRR metric, on both the FB15kET and YAGO43kET datasets, MCLET outperformed MiNer, achieving improvements of 2.2% in each.  From the computational complexity perspective, these results show the feasibility of migrating MCLET to larger scale KGs. Of course, during the migration process, obtaining information about clusters of types is essential. \n2. In addition, we investigate the convergence time of MCLET on the FB15kET and YAGO43kET datasets. For a Tesla V100 32G graphics card, MCLET respectively takes 39 minutes and 7 hours 10 minutes for model convergence on the FB15kET and YAGO43kET datasets. So, the cost of running the model is acceptable.\n\n>**Q2:** *Can you please compare the computational complexity of different approaches for entity typing?*  \n\n**Answer:** \n- The computational cost refers to the number of floating-point operations (FLOPs) required during training or inference. We compared the computational complexity of CompGCN, RGCN, CET, MiNer, TET and MCLET on the FB15kET and YAGO43kET datasets, as shown in the table below. \n    - For the FB15kET dataset, compared to the SoTA  models TET and  MiNer models, MCLET has a lower number of FLOPs than TET and on par with the MiNer model. Although CompGCN, RGCN and CET models have smaller FLOPs values on the FB15kET dataset, the accuracy of these models is not of the same order as that of TET, MiNer and MCLET. \n    - For the YAGO43kET dataset, MCLET has lower number of FLOPs than MiNer. However, compared with TET, MCLET requires more computational overhead, but considering that the MRR metric is 3.3% higher than TET, it is a tolerable increase. \n    - To sum up, MCLET can achieve a relatively large performance improvement with a computational cost that is not much different from SoTA, so we believe the design of the model is reasonable.\n\n \nModel | FB15kET | YAGO43kET\n| :--: | :--: | :--:\nCompGCN | 98.395M | 604.690M\nRGCN | 45.875M | 578.330M\nCET | 504.627M | 6.362G\nTET | 2.016G | 8.406G\nMiNer | 1.127G | 14.204G\nMCLET | 1.129G | 13.620G\n\n>**Q3:** *Need more introduction and explanation of LightGCN.*  \n\n**Answer:** \n- We start by noticing that we considered LightGCN solely as an encoder for different views. Since it is not the focus of our work and due to space restrictions, we did not provide a very detailed introduction of LightGCN. We plan to include a detailed introduction to LightGCN in the final version, as there will be an additional page available.\n\n>**Q4:** *At the bottom of page 4, please give a better explanation of reason behind adding up graph convolutional layers.*  \n\n**Answer:** \n1. We first note that the practice of summing and averaging the stacked layers encoded by LightGCN is a popular paradigm. We provide a few references here as examples: [1-4]. We will include them in the final version. \n\n    - [1] LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. SIGIR, 2020. \n    - [2] Improving Graph Collaborative Filtering with Neighborhood-enriched Contrastive Learning, WWW, 2022. \n    - [3] Knowledge Graph Self-Supervised Rationalization for Recommendation, KDD, 2023. \n    - [4] Multi-Behavior Recommendation with Cascading Graph Convolution Networks, WWW, 2023.\n\n2. Furthermore, based on the FB15kET dataset, we conducted ablation experiments involving the encoding results of different layers of LightGCN for subsequent operations, see table below. It's worth mentioning that the main reason we did not perform experiments on the YAGO43kET dataset is that, for this dataset, the number of layers chosen for LightGCN was 1 (cf. Tables 6 and 3in the appendix of the original manuscript). Therefore, conducting ablation experiments on different layers was not necessary for YAGO43kET. \n\nSetting | MRR | Hits@1 | Hits@3 | Hits@10\n| :--: | :--: | :--: | :--: | :--: |\nmean-all-layers (1-4 layers) | 0.750 | 0.677 | 0.793 | 0.891\nmean-middle-layers (2-3 layers) | 0.744 | 0.676 | 0.778 | 0.881\nmax-all-layers (1-4 layers) | 0.748 | 0.677 | 0.788 | 0.887\nmax-middle-layers (2-3 layers) | 0.745 | 0.675 | 0.783 | 0.881\nlast-layer | 0.746 | 0.675 | 0.785 | 0.888\n\n3. For the FB15kET dataset, we set the number of layers for LightGCN to 4 and considered two fusion methods for different layers: taking the mean and taking the maximum value. We investigated the experimental results under the conditions of all-layers, middle-layers, and  last-layer. As there is only one layer in the last layer condition, no mean or max operations are needed. It was evident that the best experimental results were achieved under the all-layers condition, and the mean operation approach exhibited a slight advantage over the max operation. However, generally speaking, the performance difference between mean and max is not very large, so the chosen strategy has no decisive impact on the final model results. The results reported in our original manuscript were based on the mean operation applied to all-layers. \n\n>**Q5:** *Please explain more clearly how to find positive samples and negative samples, perhaps with a figure illustration.*  \n\n**Answer:** \n- Unfortunately, we can't show how the positive and negative samples are obtained through a figure, which would be the most adequate.  We next provide an explanation. We adopted a method for obtaining positive and negative samples consistent with the CET, TET and MiNer baselines. \n    - For positive samples, the known (entity, type) pairs in the training set can be used as positive samples. For example, if (Joe Biden, American_politician), (Barack Obama, male_lawyer), and (Joe Biden, male_politician) exist in the training set, then these three tuple pairs can be used as positive samples. \n    - To gather the negative samples, a simple choice is to treat all (entity, type) pairs that do not occur in the training set as negative samples. However, some (entity, type) pairs are valid but happen to be missing in current knowledge graphs due to their incomplete nature. So, considering them as negative samples would introduce a serious false-negative phenomenon during training. For example, suppose the training set contains the tuples (Joe Biden, American_politician), (Barack Obama, male_lawyer), and (Joe Biden, male_politician). When considering negative samples for (Joe Biden, American_politician), we might obtain (Joe Biden, male_lawyer) as a negative example, because it doesn't exist in the knowledge graph. However, in reality, (Joe Biden, male_lawyer) should be a positive example but is incorrectly considered negative due to the incompleteness of the knowledge graph, causing what is known as the *\"false negative”* phenomenon. We mentioned in  the *\"Prediction and Optimization\"* part of the submision that FNA can be used to solve the false negative problem. We are consistent with CET, TET and MiNer on this point. \n\n>**Q6:** *The subscript notation is not very clearly explained in the paper which affects understanding. All subscript notations need better explanation. The part for how to select positive and negative samples need to be reworded.*  \n\n**Answer:** \n- For the ambiguity of the subscript, we will make corrections in the final version. For the problem that the positive and negative sample selection process is not described, we will add some examples in the final version to explain in detail.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Clearly discuss type knowledge clustering, entity type graph, cluster type graph, entity cluster graph and offer sensible examples for each of the definition. Introducing LightGCN instead of RGCN as multi-view encoder and clearly explain the choice of encoder. This paper also introduce contrastive learning loss that considers both intra-view and inter-view negative samples, and making the contrastive learning loss working together with false-negative aware loss function. Predicting type by propagating neighbor type information is also very logical. The robustness of Multi-head attention with a mixture of experts also backed by results comparison of dropping relation neighbors. This paper conducts ample experiments and shows results of different variants and plenty of ablation studies. Presentation of results in tables are also very clear. Overall this paper is well written and shows clear effort and thoughts into presentation and organization.", "reasons_to_reject": "My only concern is the model might be overly complex and how applicable is it to type prediction for large knowledge graphs, and whether this increase in complexity brings enough benefit.", "questions_to_authors": "1. Can you please compare the computational complexity of different approaches for entity typing?\n2. Need more introduction and explanation of LightGCN\n3. At the bottom of page 4, please give a better explanation of reason behind adding up graph convolutional layers.\n4. Please explain more clearly how to find positive samples and negative samples, perhaps with a figure illustration.\n", "paper_topic_and_main_contributions": "This paper proposes a knowledge graph entity typing approach by leveraging entity-type, entity-cluster and cluster-type structured information. A cross-view contrastive learning module is designed to capture the interaction between different views. A multi-head attention with Mixture-of-Experts mechanism is introduced to distinguish the contribution from different entity neighbors.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "The subscript notation is not very clearly explained in the paper which affects understanding. All subscript notations need better explanation. The part for how to select positive and negative samples need to be reworded."}}]

Author rebuttal:

Meta-review: This paper approaches the problem of entity typing and proposes a method that leverages entity-type, entity-cluster, and cluster-type information. Reviewers praised the papers clarity, thoroughness of experiments as well as the design and performance of the method. The main concerns for the work were whether the additional complexity of the method was justified by the amount of performance increase tradeoffs and if the methods were sufficiently different from prior approaches.
Decision: Accept-Main