LBMKGC: Large Model-Driven Balanced Multimodal Knowledge Graph Completion
Authors: Yuan Guo
Qian Ma
Hui Li
Qiao Ning
Furui Zhan
Yu Gu
Ge Yu
Shikai Guo
Venue: NeurIPS 2025 poster ()
Keywords: Multi-modal Knowledge Graphs
Knowledge Graph Completion
Cross-modal Interaction
Large Vision-Language Model
Primary area: deep_learning
TL;DR: To tackle the challenges of imbalance and heterogeneity in multimodal knowledge graph completion (MKGC), this paper proposes a novel MMKGC framework that uses a large vision-language model, cross-modal alignment, and adaptive multimodal fusion.

Abstract: Multi-modal Knowledge Graph Completion (MMKGC) aims to predict missing entities, relations, or attributes in knowledge graphs by collaboratively modeling the triple structure and multimodal information (e.g., text, images, videos) associated with entities.
This approach facilitates the automatic discovery of previously unobserved factual knowledge.
However, existing MMKGC methods encounter several critical challenges: (i) the imbalance of inter-entity information across different modalities; (ii) the heterogeneity of intra-entity multimodal information; and (iii) for a given entity, the informational contributions of different modalities are inconsistent across contexts.
In this paper, we propose a novel **L**arge model-driven **B**alanced **M**ultimodal **K**nowledge **G**raph **C**ompletion framework, termed LBMKGC.
Subsequently, to bridge the semantic gap between heterogeneous modalities, LBMKGC aligns the multimodal embeddings of entities semantically by using the CLIP (Contrastive Language-Image Pre-Training) model.
Furthermore, LBMKGC adaptively fuses multimodal embeddings with relational guidance by distinguishing between the perceptual and conceptual attributes of triples.
Finally, extensive experiments conducted against 21 state-of-the-art baselines demonstrate that LBMKGC achieves superior performance across diverse datasets and scenarios while maintaining efficiency and generalizability.
Our code and data are publicly available at: https://github.com/guoynow/LBMKGC.

LBMKGC: Large Model-Driven Balanced Multimodal Knowledge Graph Completion
Yuan Guo Dalian Maritime University guoynow@gmail.cmm com
Qian Ma" Dalian Maritime University maqian@dlmu. edu. on
Hui
Dalian Maritime University hui@dlmu. edu. cn
Qiao Ning
Furui Zhan
Yu Gu
Jiangnan University ningg6690jiangann edu. on
Dalian Maritime University izfree@dlmu, edu. on
Northeastern University, China guyu@mail neu. edu. cn
Ge Yu Northeastern University, China yuge@mail- neu. edu. cn
Shikai Guo
Dalian Maritime University shikai guoêdlmu. edu. cn
Abstract
Multi-modal Knowledge Graph Completion (MMKGC) aims to predict missing entities, relations, attributes in knowledge graphs by collaboratively modeling the triple structure and multimodal information (e.g., text, images, videos) associ- ated with entities. This approach facilitates the automatic discovery of previously unobserved factual knowledge. However, existing MMKGC methods encounter several critical challenges: (i) the imbalance of inter-entity information across different modalities; (ii) the heterogeneity of intra-entity multimodal information; and (iii) for given entity, the informational contributions of different modali- ties are inconsistent across contexts. In this paper, we propose a novel Large model-driven Balanced Multimodal Knowledge Graph Completion framework, termed LBMKGC. Initially, LBMKGC employs the Stable Diffusion XL (a Large generative vision model) augment the imbalanced information across modali- ties. Subsequently. to bridge the semantic gap between heterogeneous modalities, LBMKGC aligns the multimodal embeddings entities semantically by using the CLIP (Contrastive Language-Iman
Pre-Trainn Training) model. Furthermore, LBMKGG adaptively fuses multimodal embeddings with relational guidance by distinguishing between the perceptual and conceptual attributes of triples. Finally, extensive exper- iments conducted against 21 state-of-the-ar baselines demonstrate that LBMKGC achieves superior performance across diverse datasets and scenarios while main- taining efficiency and generalizability. Our code and data are publicly available at: http://gihbb.com//guuuo//////..
Introduction
Multimodal Knowledge Graphs (MMKGs) represent advanced semantic framework that extends traditional knowledge graphs. Unlike conventional knowledge graphs, which primarily consist of structured iiiity-relaion-aatti associated with entities, including textual descriptions, images, audio, and videos. This comprehensive approach allows MMKGs to capture and describe complex real-world phenomena across multiple
triples, MMKGs integrate variety of multimodal data
"Corresponding authors
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

dimensions. By fusing structural and multimodal information, MMKGs facilitate finer- amer-grained reasoning in applications such as recommender systems [28]. natural language processing [6].
methods can be categorized into two primary approaches. The first focuses on multimodal fusion mechanisms, aiming to achieve completion by systematically modeling the cross- moosl correlations among structural topologies, visual representations, and textual semantics [23 [3]. The second approach emphasizes negative sampling optimization algorithms, which enhance completion performance by improving the quality of negative samples through vision-text joint discrimination
and computer vision g Current mainstream MMKGC
30
32]. However, existing methods
often overlook following critical challenges.
(1) The imbalance of inter-entity information across different modalities. real-world MMKGs, modality distributions are diverse and complex. Some entities may possess rich multimodal informa- tion (e-g., images and text), while others suffer from modality scareity (e.g., text-only entities with missing visual data). This imbalanced distribution complicates knowledge graph completion tasks, as missing modalities may lead to critical knowledge omissions, thereby compromising accuracy. Although existing MMKGC methods [36] employ frameworks like adversarial training to mitigate imbalance issues, they fail to explicitly model latent semantics in modality-missing scenarios.
The heterogeneity of intra-entity multimodal information. Significant disparities exist in data structures and semantic comprehension across modalities. For instance, visual information is encoded as 2D pixel matrices in images, while textual modalities convey symbolic logic through word sequences. This dual gap both data representation and semantic parsing mechanisms poses substantial challenges for cross-modal alignment. Existing MMKGC methods predominantly rely on single-modality pretrained models (e-g.. VGG (20) for vision, BERT (5) for text) to independently extract visual or textual features, thereby neglecting the construction of a cohesive cross-modal representation space. Consequently, downstream tasks often struggle to fully capture the deep semantic interdependencies amongst modalities.
Yale University
blue
Steven C. Hebert
Yale University
color
employer
88L
CB
YALE
YALE
The picture of Yole University is very helpful aiding my understonding of this triple!
Maybs I need more information about the knowledge grophs to understond this triple
perceptual triple
conceptual triple
Figure 1: The salience of information carried by different modalities varies across different types of triples.
(3) The inconsistency in the volume of information represented by various modalities for a given entity. In knowledge graphs, triples can be categorized into perceptual (concrete) and conceptual (abstract) types. For the same entity, the salience of information carried by different modalities varies across these two types of triples. Taking the entity "Yale University" as an example, in the MKG-W dataset, the triple (Yale University, color, blue) is perceptual triple. In this triple, the visual modality of Yale University provides discriminative evidence, and thus should be assigned higher weight during the feature fusion stage. However, the triple (Steven C. Hebert, employer, Yale University) is a conceptual triple, as it involves an employer relationship, which is social construct and cannot be directly perceived. Therefore, Yale University in this context relies more on the structural information of the knowledge graph, and higher weight should assigned structural information during feature fusion. Similarly, for entities, "blue" is perceptual, where the visual modality provides discriminative evidence, while "law" is conceptual, relying more on symbolic reasoning from textual modalities. Existing MMKGC methods struggle to dynamically adjust inter-modal weights for entities based on

---

the perceptual or conceptual attributes of triples, leading to inefficient and imprecise utilization of multimodal information.
To address the aforementioned challenges, we propose a novel I Large model-driven Balanced MMKGC framework, named LBMKGC. LBMKGC comprises three key modules called Large Generative Visual Model-basee -base Modality Completion (LvMC) module, Cross-Modality Alignment (CMoA) module, and Context-Guidee Adaptive Fusion (CGuAF) module, respectively. To address the imbalance issue, the LvMC module utilizes the large generative visual model SDXL (Stable Diffusion XL) [6] augment the missing modality information. To address the heterogeneity issue, the CMoA module focuses on aligning the entities between various modalities by employing the pre-trained CLIP model. To address the inconsistency issue, the CGuAF module adaptively moderates the weight of each modality with relational guidance to learn joint cross cross-modality representations. Our major contributions are summarized as follows.
We propose a novel MMKGC framework, named LBMKGC, designed to address issues of modality imbalance, modality heterogeneity, and the inconsistency of information quantity for entities across different modalities.
We innovatively tackle the modality imbalance issue in MMKGC by leveraging the capabilities of the SDXL, a state-of-the-art large generative visual model, instead of relying on conventional adversarial training methods.
We propose a CMoA module that dynamically adjusts the alignment of multimodal embeddings, which effectively mitigates biases arising from data heterogeneity, ensuring sufficient utilization of multimodal information.
We first consider the impact of the perceptual and conceptual attributes of triples during joint cross-modality representation learning, and accordingly propose a novel context-guided adaptive fusion module.
To evaluate the performance of LBMKGC, conduct comprehensive experiments and fur- ther explorations on three public benchmarks. Empirical results demonstrate that LBMKGC outperforms 21 recent baselines, achieving new state-of-the-art MMKGC results.
2 Preliminaries
2.1 Problem Statement
provide a robust foundation for knowledge representation and reasoning by integrating structured triples with rich multimodal entity attributes (e.g., images, text, audio, video). In recent years, significant progress has been made in MMKGC [30, which aims to automatically discover new knowledge from incomplete MMKGs
MMKGs 14]
We define MMKGs as G = (E, R, where & is the set of entities, R is the set of relations, 1)),, E, R) is the set of triples, and M is the set of modalities,
T= ((,, r, different modalities in MMKGs (such as images, text, videos, audio, etc.). For an entity e E & in modality m E M, its modality information denoted as ems such as text modality denoted as et. and visual modality denoted as Ep.
aecapsulating
The MMKGC model embeds entities and relations into continuous vector space. For each entity e E E, we define its structure, text, and visual embeddings as camb camb, eemb, For each relation R, define its structure embedding as rm Additionally, the MMKGC model can use scoring function F to measure the plausibility of each triple (h, r, l). F calculates the score through the embedding representation of triples. In the inference phase of the MMKGC task, for given query (h, r, ?) or (?, r, 1), the model scores each candidate entity corresponding to the triple and ranks them make predictions.
2.2 Related Work
In recent years, numerous KGC methods have been developed. Traditional unimodal approaches, e.g., TransE M.. TransD [10), RotatE (23). PairRE [4) DistMult (33] and ComplEx [25]. learn knowledge graph embeddings (KGE) by projecting entities and relations into continuous vector space that preserves the observed triple structure. To capture heterogeneous graph topology, DGS 9 first separately embeds the cyclic instance structure and hierarchical ontology structure of two-view knowledge graphs into spherical and hyperbolic spaces, unifying them via bridge space.
3

---

On the multi-modal front, IKRL 31 first augments TransE with visual features; TBKGC [9 further incorporates textual cues. Building on KGE, MMKGC methods such as OTKGE D. VISTA [.] RSME 277.. TransAR RQ IMF 13 and QBE (29 leverage rich multi-modal signals to refine
entity representations. MMRNS [32 and MANS (3 instead focus on exploiting modalities for higher-quality negative sampling. Modality-imbalance has also been tackled by generative adversarial schemes like MMKRL [15] and AdaMF-MAT [36), which synthesize missing-moda features and use RotatE-style scores discriminators. Yet these models (i) capture only local semantics and ignore global cross-modal correlations, and (ii) require separate adversarial networks per modality, aggra- vating heterogeneity. eerryee Consequently, modality imbalance, structural heterogeneity and inconsistent saliency across modalities remain largely unresolved.
In this work, we propose LBMKGC to tackle these issues and achieve state-of-the-as performance.
3 Method
In this section, we will introduce our MMKGC framework, LBMKGC. An overview of LBMKGC is shown in the Figure 2 We will introduce the main components in the following paragraphs.
These components the Large Generative Visual Model-basee Modality Completion module, Cross-Modal Alignment module, Context-Guidee Adaptive Fusion module, and MMKGC Module.
(o) LYMC
(b) CMaA
(c) OGUAF
(d) MAKGC
straced Deforsatoon
Missing Image
Contrcct
Vision Transformer
Vicul
Emoodding
Gettee
Gate
SDXL
Emsaddiin
Bale
ReTeeer
Decsder
Loarnig
Alyyed Erbbediige
Loyer
120
Snowten modal of per pf of
siw,, especialy
by dhikd.....
Ewinkkon
Embedding
Tockal Information
Modalty Wogh
Figure 2: Architectural Overview of the LBMKGC Model.
3.1 Large Generative Visual Model-based Modality Completion
To address the issue of imbalanced multimodal information, we propose a modality completion module based on SDXL (Stable Diffusion XL. [16] as shown in the Figure 2 (a). In the field of knowledge graph, obtaining visual information is usually more diffieult than obtaining text information. Therefore, this paper focuses on the problem of multimodal information missing in visual modality, that is, given an entity E, LvMC populates the prompt template with e; to generate e,.
The core component of LvMC is the generative model SDXL. SDXL employs two-stage generative process facilitated by two specialized models: a Base model and Refiner model. The Base model is primarily responsible for generating coherent initial image based on the text prompt, ensuring overall semantic accuracy and composition. Subsequently, the Refiner model operates on the output (typically in the latent space) of the Base model to enhance fine-grained details and visual fidelity. For clarity, a simplified representation of the overall generation process is provided as follows:
First, LvMC generates a text embedding vector by combining two CLIP encoders:
C₁ = Concat( W, CLIPPiga(e)) W2 CL.PvtrI.e:)),,
(1)
where W, and We are projection matrices, CLIP and CLIPwTA are respectively OpenCLIP ViT-bigG 18 and OpenAl CLIP ViT-L 17 encoders.
The image resolution (w, h) is embedded as Fourier features:
faize [sin (2 ) ( c2s (2*mn)))()) Mi=1
(2)
where The = R and nt & ... with R denoting the base resolution used during training (typically 1024). I. represents the number of frequency bands. The final combined embedding vector is then formed by concatenating the text embedding and the size embedding: Concat(C;, faiz).

---

The Base model generates the initial resolution potential through DDIM sampling 21
V
Z
vI 0i E0 base
.,,c)
/1 āc-1 cp(2) base t,, c)2
(3)
vā,
where the initial noise sampled from a standard Gaussian distribution: zT (0, I). Then, each reverse step t, the previous state zb=ę is computed by first estimating the predicted noise (zzbee t, c) using its UNet, and then applying the DDIM update rule based on the cumulative noise schedule oe and the conditioning embedding c, which combines text and size information.
The Refiner model adds noise to base.
2 felice
relined
Or 20
NO I),
(4)
where refine denotes the running timestep variable in the Refiner's 's reverse diffusion process.
The Refiner model refines the image step by step:
refined =Vo+-1
Vit
Pefined
VI
Ciy to (zfehindd 4.0)) + v1 0x-1 - c0(2) (z.efined ,,,),
(5)
vās
where Eg is the noise predictor of the Refiner model.
Finally, the high-resolution image in the variable zefined is decoded through the VAE decoder the real image space:
(2 (effiiee)
(6)
3.2 Cross-Modal Alignment
To address the issue of heterogeneity in multimodal information within a single entity, we proposed Cross-Modal Alignment (CMoA) module, shown in the Figure (b). This module leverages the pre-trained CLIP [17] model to achieve semantic alignment between an entity's image and text representations.
We first extract multimodal feature encodings from the original image and text description of the entity using a pre-trained model, which is necessary step for all MMKGC methods. The text modality encoded into text features ft (Ti, T2, TN} using BERT [5). and the visual modality is encoded into visual features f. {Vi, 12,,.., .., VNj J using VIT (7]. The cosine similarity matrix S
between the two is calculated, where S represents the similarity between the i-th visual feature Vi and the j-th text feature Ty:
V T
S. = [Val
(7)
|T
The symmetric contrastive loss function with learnable temperature parameter is defined as:
CCLPP
1 log Σ SughT
SaalT
1 N
Σ log
StalT
(8)
N
N
ΣN Sgult
Based on the cross-modality feature alignment, we obtain the aligned visual features fa and text features ft. Subsequently, through a projection layer, each entity e E in modality m E M is embedded into representation em PPm(mm) Here Pm.. is projection layer for modality m, aimed at projecting different modality embeddings into the same vector space. Each Pn, consists of two MLP layers with ReLU the activation function.
3.3 Context-Guided Adaptive Fusion
To address the issue that existing MMKGC methods struggle to adaptively adjust the weights of different modalities based on the specific characteristics the three-element entities, leading to the inability to effectively utilize different modalities, we propose Context-Guided Adaptive Fusion (CGuAF) module, as shown in the Figure 2 (c). This mechanism divided into two stages. The first stage performs adaptive fusion of multiple modalities, and the second stage considers the upper
5

---

and lower context information to generate joint embedding representation for each entity in the three-element set.
The mechanism adjusts the weight win of each modality based on the entity's (perception or concept) characteristics. For perceptual entities (such as color), if the visual modality can provide strong evidence, it is assigned a higher weight, and the text modality is assigned a lower weight. Conversely, for conceptual entities (such as force), it is difficult to express them with images, so a higher weight is assigned to the text modality, and lower weight is assigned to the visual modality. Specifically, the weight of the modality m for entity e, wm (c), is calculated as follows:
exp (V tanh (em))
Wm (m( - EnEm exp (V tanh (en))*
(9)
where V a learnable vector.
Based on these weights, we obtain the fused multimodal information mm
Way m
(10)
MEM
For the same entity, the salience information carried by different modalities varies across different types of triples (e.g.. perceptual and conceptual). Accordingly, we design relational temperature G- that allows an entity to adapt its joint emhedding based on the type of triple, rather than using a static one, thus providing more effective relational context.
eemb = h +1 (1) ammb
(11)
Similarly, the joint embedding of the tail entity is derived using the same fusion method.
Traditional Multimodal Knowledge Graph Completion (MMKGC) approaches typically rely on static, entity-level modality fusion strategies, which overlook the specific context of the input triple. In comparison, our proposed context-guided adaptive fusion module dynamically recalibrates entity embeddings conditioned on the triple type. This fine-graing fine-grained adaptive adjustment mechanism significantly enhances the model's ability interact with multimodal features in different relationship contexts.
3.4 Multimodal Knowledge Graph Completion
As shown in the Figure (d), based on the multimodal joint embedding representation of entities, we introduce the RotatE [23]] score function based on complex space rotation operations to evaluate the rationality of triples (h, r, f), which has universal modeling capability for complex relationship patterns in knowledge graphs. Specifically, for a triple (h, r, f), its plausibility score is defined as the negative distance between the head entity embedding after rotation and the tail entity embedding, that is:
(12) where is the rotation operation in complex space, and higher score indicates greater rationality of the triple In the training process, we use a loss function based on negative sampling to optimize the parameters, which can be expressed as:
F(h, r, -Ih o - E|].
K logoo F(h,r,t)) Σ p(k, logo (2)
Cage
Σ
- -)),
(13)
(har.))T
where is the sigmoid function; 7 is a fixed boundary; (4rr,44 E T', (i 1.2, 22....,K) K) K negative samples of the triple rh, Additionally, p(h,, To (.t))) is the self-adversarial weight proposed in RotatE (23]
Experiments
In this section, we evaluate our method using mainstream task in KGC link prediction task. We first introduce the experimental setup and then present the results, We mainly explore the following four research questions (RQs) regarding LBMKGC:

---

RQ1: Can our model LBMKGC surpass the existing baselines and make substantial progress the MMKGC task link prediction?
RQ2: How much does the design of each module in LBMKGC contribute to performance?
RQ3: When the multimodal information is imbalanced, can LBMKGC maintain stable performance in the MMKGC task?
RQ4:
Can LBMKGC adaptively adjust multimodal weights based on different entities?
4.1 Experimental Settings
Datasets. To better explore the MMKGC task in diverse and complex environment, we conducted comprehensive experiments and further explorations three public benchmarks. The DB15K [14] dataset was built by crawling search engine images and aligning them with DBpedia [2]. while the MKG-W and MKG-Y Y [2) datasets were created by extracting subsets from Wikidata 26 and We used the pre-trainn trained model of CLIP to extract multimodal features.
YAGO 12
Baselines. To demonstrate the effectiveness of our method, we conducted a comprehensive compar- ison and analysis with 21 different state-of-the-ar KGC and MMKGC models as our baselines. They can be categorized into three types:
(i) Uni-modal KGC methods, this paper employs 6 of the most advanced uni-modal KGC methods, including TransE [T. DistMult [33], ComplEx (25). RotatE [23). PairRE [4] GC-OTE [24], which
designed elegant scoring functions and considered structural information in model design.
(ii) Multi-modal KGC models, this paper utilizes 12 MMKGC methods that consider multi-modal information and triple structural information, including IKRL (31), TBKGC [19]. TransAE (30),
MMKRL (15), RSME [27]. OTKGE [3) IMF [13. AdaMF (3), QEB [29], VISTA (11], AdaMF.
MAT 36
and MyGO
35
(iii) Negative sampling methods, this paper compares our method with different negative sampling methods, including KBGAN (2). MANS (4, MMRNS (32]. Among them, KBGAN is an adversarial sampling method designed for traditional KGC, applying reinforcement learning to optimize the model. MMRNS utilizes multi-modal information to enhance the negative sampling process.
Metrics. To evaluate our method, we conducted link prediction tasks on three datasets. Link prediction an important task in knowledge graph completion, aiming to predict the missing entity for given query (h, ?) (?,r,t) t). The link prediction task consists of two parts: head prediction and tail prediction.
Following previous work, we use ranking- based metrics such as Mean Reciprocal Rank (MRR) and @it@ (K 1, 3, 10) to evaluate the results. MRR and @it@ be calculated as:
Taaa
MRR
(14)
[Tees
Thi
Tt.i
Hit@K
(1(7h,s S K) + 1(re,s $ K)),
(15)
Trsst
where Th, and Ft, are the results of head prediction and tail prediction, respectively.
To ensure fair comparisons, the filter setting is applied to all results to remove candidate triples that already exist in the training set.
Implementation details. We implemented our LBMKGC model based on the famous open-source KGC library OpenKE. We conducted experiments Linux server with Ubuntu 24.04.01 operating system and a single NVIDIA GeForce 4090 GPU. We reproduced some advanced models, and some baseline results refer to MyGO (35].
In the LBMKGC, we fix the batch size to 512 and set the training epoch from (1000, 1250, 1500). The embedding dimensions are tuned from (300, 400, 500) and the negative sampling number K is tuned from (32, 64, 128). The margin 7 is tuned from (8, 12, 16, 20, 24) and the temperature set to 4. We optimize the model with Adam and the learning rate is tuned from ( [e=5, 2c-5, 5e-5).
7

---

For baselines, we reproduce the results following the methodology and parameter setting described in the original papers and their open-source official code.
4.2 Main Results (RQ1)
We conducted link prediction experiments and presented the experimental results in Table Our method, LBMKGC, achieved superior performance on all evaluation metrics across three public benchmarks. The experimental results of this method show that our approach enables the model to capture a richer set of candidate relation patterns, enhancing semantic coverage, and more stably retaining correct answers in the top region of the candidate set (Hit @3 and @it@ 10), while still maintaining advanced performance the exact first match metric @Hit@ 1). has stronger practicality in real- woalwwrldd scenarios that require multi-cane multi-candidate selection (such as recommendation systems, open-domain question answering).
4.3 Ablation Study (RQ2)
To provide detailed demonstration of the effectiveness of each module design and answer RQ3, we conducted ablation studies, as shown in T We replaced the adaptive fusion guided by the context with mean fusion to verify the effectiveness of CGuAF. Furthermore, verify the contribution of different information in MMKGC, we covered visual information (textual informa- tion/structural information) for link prediction tasks. Additionally, models (VGG16 [20]. BERT (5)) to independently extract visual and textual features to verify the effectiveness of cross-modal feature alignment. Finally, we randomly initialized the missing modal information to verify the importance of the real missing modal information in explicit modeling.
used uni-modal pre-training
The results of the ablation study show that when each module each modal information of method is removed, the link prediction results will decrease, which demonstrates their effectiveness.
Table 1: Knowledge Graph Completion Models Comparison
DB1SK
MKG-W
MKG-Y
MRR
Hit@1
Hit
Hit@
MRR
Hit@
Hilee
Hit@:0
MRR
Hit@1
Hit@3
Hit@ 10
TransE
24.86
12.78
31.48
47.07
29.19
21.06
33.20
44,23
30.73
23.45
35.18
43,37
DistMalt
23.03
14.78
39.59
20.99
15.93
30.86
25.04
19.33
35.95
ComplEx
27.48
18.37
31.57
45.37
24.93
19.06
26.69
36.73
28.71
22.26
32.12
40,93
38.35
PairRE
31.13
21.62
35.91
49.30
34.40
24
36.71
46.04
32,01
25.53
35.84
43,89
GC-OTE
31,85
22.11
36.52
51.18
33.92
26.55
39.65
46.90
32,95
26.77
38.41
44.08
IKRL
26.82
14.09
34.93
49.09
32.36
26.11
34.75
44.07
33.22
30.37
34.28
38.26
TBKGC
28.40
15.61
37.03
49.86
31.48
25.31
33.98
43.24
33.99
30.47
35.27
40.07
TransAB
21.25
41.17
30.00
28,
25.31
MMKRL
26.81
13.85
35.07
49.39
30.10
22.16
34.09
69
36.81
31.66
39.79
45.31
RSME
32.12
40.29
29.23
40.43
34.44
31.78
36.07
39.09
OTKGE
23.86
18.45
25.89
34,23
34.36
28.85
36.25
44.88
35.51
31.97
37.18
41.38
IMF
32.25
24.20
36.00
48.19
34.50
28.77
36.62
45.44
35.79
32,95
37.14
40.63
QEB
28.18
14.82
36.67
$1.55
32.38
25.47
35.06
45.32
34.37
29.49
36.95
42.32
VISTA
30.42
22.49
33.56
45.94
32.91
26.12
35.38
45.61
30.45
24.87
32.39
41.53
AdaMF
KBGAN
25.73
9.91
36.95
$1.93
29.47
22.21
34.87
40.64
29.71
22.81
34.88
40.21
MANS
28.82
16.87
36.58
49.26
30.88
24.89
33.63
41.78
29.03
25.25
31.35
34.49
MMRNS
32.68
23,01
37.86
51.01
35.03
28.59
37.49
47.47
35.93
30.53
39.07
45.47
AdaMP-MAT
35.14
25.30
41.11
$2.92
35.77
28.87
38.85
48.71
38.57
34.34
40.59
45,76
MyGO
51.87
30.02
46.89
39,96
Ours
37.23
27.78
42.75
54.71
38.46
31.20
41.78
51.46
40.03
33.89
43.11
$0.81
+0.16%
+2.6%
+3.38
+6.6%
+3.9%
+7.5%
+3.5%
+6.2%
+11.0%
Table 2: Ablation Study
Settings
DB15K
MKG-W
MKG-Y
MRR 37.23
Hit@1 1 27.78
Hit@3 42.75
Hit@ 10 54.71
MRR 38.46
@it@1 31.20
Hit@3
Hit@10 10 51.46
MRR 40.03
Hit@1 33.89
Hit@3
Hit@ 10
LBMKGC
41.78
50.81
w/o LVMC
36.73
27.19
42.01
53.76
37.12
29.43
40.28
49.16
39.40
33.42
42.47
49.59
CMoA
36.81
27.69
41.56
53.37
38.03
29.88
40.86
49.34
39.34
33.58
42.28
49.37
oola CGGAF
36.36
26.55
42.39
54.48
37.53
30.51
41.76
50.77
39.29
33.01
42.92
50.86
who cemá
31.21
16.96
41.12
$4.60
34.76
23.84
41.86
51.97
37.79
29.33
42.43
$1.02
w/o egmb
36.51
27.09
41.45
53.19
36.72
30.01
40.02
48.79
38.33
32.72
40.95
47.67
eomá
37.89
50.89
8

---

4.4 Modality-missingg Results (RQ3)
We conducted link prediction experiments with modality dropout on the DB15K dataset. The visual modality was discarded at rates of 17 (0.25, 0.5, 0.75, 1.0). Specifically, for LBMKGC, used the LvMC module to fully compensate for the missing visual modality information. For other models, we used the commonly used random initialization method 199 compensate for the missing information. The experimental results are shown in the Figure as the missing modality rate increases, the baseline models (TBKGC [19] and AdaMF-MAT (36)) exhibit significant performance degradation, whereas our proposed LBMKGC demonstrates relatively stable performance. LBMKGC's explicit modeling of missing modalities based on large-scale pre- traitrd models (i.e., generating real modality information) boosts MMKGC's performance and robustness more effectively than strategies using adversarial training or random initialization.
Perfortunce
HIT@ Performance
LBMKCG
AdMMF-MAT
TBKGC
LBMKCG
AMMM-MMT
TBKGC
53-
MRR
HTT
52-
32-
51-
30
50-
28
49-
25
75
100
25
75
100
Miss Rate
Miss Rate
Figure 3: Link prediction results on modality DB15K dataset.
Text Firholdings
Vieual Finhaldings
300
Iropesty
Narked Tiskonky
8:3
0.30
0.35
0.40
0.45
0.50
Weight Value
0.6
0.7
Histogram of Visual Modality Weights Distribution
(b) Comparison Chart of Text and Visual Modality Weights
Figure 4: Fusion weights of different modalities.
4.5
Case Study (RQ4)
To intuitively demonstrate the effectiveness CGuAF, we present the histogram of visual modality weight distribution for all entities in the MKG-W dataset in Figure 4 (a). selected some entities" textual and visual modality weights for comparison in Figure F for the conceptual entity
(a). Based on this, we randomly
film)", the weight of textual information is 71%, while that of visual modality is 29%, whereas for the perceptual entity "Australian National University". the weight National__niversiitee of textual information is 47%, and that of visual modality is 53%. This reflects that CGuAF can dynamically adjust the weight of each modality according to the specific (perceptual) or abstract (conceptual) characteristics of different triad groups of entities, enhancing and retaining the truly important parts of multi-modal information, thereby significantly enhancing the model's 's ability to interact with modai features under scenarios of complex relationships.

---

5 Conclusion and Future Work
In this paper, mainly discussed the imbalance of multi-modal information among entities, the heterogeneity of multi-modal information within entities, and the issue of inconsistent information quantities across different modalities. We proposed novel MMKGC framework, LBMKGC, to address the limitations of existing methods. LBMKGC consists of three core modules: LvMC, CMoA, and CGuAF, which sequentially solve the issues of imbalance, heterogeneity, and inconsistent information quantities. We conducted in-depth theoretical analysis to demonstrate the rationality of our design and comprchensive experiments on 21 benchmarks to show the effectiveness and robustness of our framework.
However, our study currently focuses text and image modalities, without integrating semantically rich information such as audio and video into a unified framework. Additionally, our experiments general domain datasets, leaving the adaptability and robustness
have been conducted exclusively in specialized fields like biomedical data and social networks unverified. In the future, MMKGC tasks should more comprehensively consider these modalities in the real world and design generalizable and scalable model architectures to adapt to various tasks and modality combinations, thereby enhancing the ability model complex relationships.
6 Acknowledgements
This work was supported by the National Natural Science Foundation of China (No. NN.66200039, 62472062, U23B2019).
References
[1] Antoine Bordes, Nicolas Usunier, Alberto Garcfa-Durán, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States, pages 2787-2795 2013.
[2] Liwei Cai and William Yang Wang. KBGAN: adversarial learning for knowledge graph embeddings. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1-6, 2018, Volume (Long Papers), pages 1470-1480. Association for Computational Linguistics, 2018
[3] Zongsheng Cao, Qianqian Xu, Zhiyong Yang, Yuan He, Xiaochun Cao, and Qingming Huang. OTKGE: multi-modal knowledge graph embeddings via optimal transport. In Advances in Neu- ral Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
[4] Linlin Chao, Jianshan He, Taifeng Wang, and Wei Chu. Pairre: Knowledge graph embeddings via paired relation vectors. In Proceedingy of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1h International Joint Conference on Natural Lanuuage Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 4360-4369. Association for Computational Linguistics, 2021.
[5] Jacob Devlin, Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume / (Long and Short Pupers), pages 417 4171-410 -4186.1 Association for Computational Linguistics, 2019.
[6] Junnan Dong, Qinggang Zhang, Huachi Zhou, Daochen Zha, Pai Zheng, and Xiao Huang. Modality-aware integration with large language models for knowledge-based visual question answering. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume Long Papers), ACI. 2024, Bangkok, Thailand, August 11,,, 2024, pages 2417-2429. Association for Computational Linguistics, 2024.
10

---

[7] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[8] Xin Gu, Guang Chen, Yufei Wang, Libo Zhang, Tiejian Luo, and Longyin Wen. Text with knowledge graph augmented transformer for video captioning. In IEEE/CVE Conference on Computer Vision and Puttern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pages 18941-18951. IEEE, 2023.
[9] Roshni G. Iyer, Yunsheng Bai, Wei Wang, and Yizhou Sun. Dual-geomen space embedding model for two-view knowledge graphs. In KDD '22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 18, 2022, pages 676-686. ACM, 2022.
[10] Guoliang Ji, Shizhu He, Liheng Xu, Kang Liu, and Jun Zhao. Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume I: Long papers), pages 687-696, 2015.
[11] Jaejun Lee, Chanyoung Chung, Hochang Lee, Sungho Jo, and Joyce Jiyoung Whang. VISTA: visual-textual knowledge graph representation learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 7314-7328. Association for Computational Linguistics. 2023.
[12] Jens Lehmann, Robert Isele, Max Jakoh, Anja Jentzsch, Dimitris Kontokostas, Pablo N. Mendes, Sebastian Hellmann, Mohamed Morsey, Patrick van Kleef, Sören Auer, and Christian Bizer. Dbpedia- A large-scale, multilingual knowledge base extracted from wikipedia. Semantic Web, 612):167-195, 2015.
[13] Xinhang Li, Xiangyu Zhao, Jiaxing Xu, Yong Zhang, and Chunxiao Xing. IMF: interactive multimodal fusion model for link prediction. In Proceedings of the ACM Web Conference 2023, WWW 2023, Austin, TX, USA, 30 April 2023 4 May 2023, pages 2572 25800. ACM, 2023.
[14] Ye Liu, Hui Li. Alberto Durefa, Mathias Niepert, Daniel Rubio,,ubio, and David S. Rosenblum. MMKG: multi-modal knowledge graphs In The Semantic Web - 16th International Conference, ESWC 2019, Portorož, Slovenia, June 2-6, 2019, Proceedings, volume 1 11503, pages 474, 2019.
[15] Xinyu Lu, Lifang Wang, Zejun Jiang, Shichang He, and Shizhong Liu. MMKRL: A robust embedding approach for 52(7):7480-7497, 2022.
knowledge graph representation learning. Appl. Intell.,
[16] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: improving latent diffusion models for nigh-resolution image synthesis. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024.
[17] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceed- ings of the 38th International Conference on Machine Learning. ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 8748-88 8763. PMLR, 2021.
[18]
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaezmarezyk, and Jenia Jitsev. LAION-5B: an open large-scale dataset for training next generation image-text models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.

---

[19] Hatem Mousselly Sergieh, Teresa Botschen, Iryna Gurevych, and Stefan Roth. A multimodal translation-bast approach for knowledge graph representation learning. Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, *SEM@ "SEM@NAACI NAACI-HLTT 2018, New Orleans, Louisiana, USA, June 5-6, 2018, pages 225-234. Association for Computational Linguistics, 2018.
[20] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large- image recognition. In Yoshua Bengio and Yann LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.
[21] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[22] Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. Yago: core of semantic knowledge. In Carey L. Williamson, Mary Ellen Zurko, Peter F. Patel-Schneider and Prashant J. Shenoy, editors, Proceedings of the 16th International Conference on World Wide Web, WWW 2007, Banff Alberta, Canada, May 8-12, 2007, pages 697-706. ACM, 2007.
[23] Zhiqing Sun, Zhi-Hong Deng, Jian- Nie, and Jian Tang. Rotate: Knowledge graph em- bedding by relational rotation in complex space. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, IA, USA, May 6-9, 2019. OpenReview.net, 2019.
[24] Yun Tang, Jing Huang, Guangtao Wang. Xiaodong He, and Bowen Zhou. Orthogonal relation transforms with graph context modeling for knowledge graph embedding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACI 2020, Online, July 5- 5-10, 2020, pages 2713-2722. Association for Computational Linguisties, 2020.
[25] Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. Complex embeddings for simple link prediction. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and Conference Proceedings, pages 2071 1-2000. JMLR.org. 2016.
[26) Denny Vrandecic and Markus Krötzsch. Wikidata: a free collaborative knowledgebase. Com- mun. ACM, 57(10):78-85 2014.
[27] Meng Wang. Sen Wang. Han Yang, Zheng Zhang. Xi Chen, and Guilin Qi. Is visual context really helpful for knowledge graph? A representation learning perspective. In MM '21: ACM Multimedia Conference, Virtual Event, China, October 20 24, 2021, pages 7743. ACM, 2021.
[28] Xiang Wang, Xiangnan Hc, Yixin Cao, Meng Liu, and Tat-Seng Chua. KGAT: knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019, pages 358.958.444 ACM, 2019.
[29] Xin Wang, Benyuan Meng, Hong Chen, Yuan Meng, Ke Lv, and Wenwu Zhu. TIVA-KG: A multimodal knowledge graph with text, image, video and audio. In Proceedings of the 31st ACM International Conference on Multimedia, MM 2023, Ottawa, ON, Canada, 29 October 2023- November 2023, pages 2391-2399, ACM, 2023.
[30] Zikang Wang, Linjing Li, Qiudan Li, and Daniel Zeng. Multimodal data enhanced representation learning for knowledge graphs. In International Joint Conference on Neural Networks, IJCNN 2019 Budapest, Hungary, July 14-19, 2019, pages 1-8, IEEE, 2019.
[31] Ruobing Xie, Zhiyuan Liu, Huanbo Luan, and Maosong Sun. Image-embodied knowledge representation learning. In Carles Sierra, editor, Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017, pages 3140-3146 orcai.org, 2017.
12

---

[32] Derong Xu, Tong Xu, Shiwei Wu, Jingbo Zhou, and Enhong Chen. Relation-enhanee negative sampling for multimodal knowledge graph completion. MM 22: The 30th ACM International Conference Multimedia, Lisboa, Portugal, October 10 - 14, 2022, pages 3857-3866. ACM, 2022.
[33] Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. In 3rd International Conference Learning Representations. ICLR 2015, San Diego, CA, USA, May 7-9, 2015. Conference Track Proceedings, 2015.
[34] Yichi Zhang, Mingyang Chen, and Wen Zhang. Modality-aware negative sampling for multi- modal knowledge graph embedding. In International Joint Conference on Neural Networks, IJCNN 2023, Gold Coast, Australia, June 18-23, 2023, pages 1-8. IEEE, 2023.
[35] Yichi Zhang, Zhuo Chen, Lingbing Guo, Yajing Xu, Binbin Hu, Ziqi Liu, Wen Zhang, and Huajun Chen. Tokenization, fusion, and augmentation: Towards fine- tne-grained multi-moda entity representation. AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, February 25 March 4, 2025, Philadelphia, PA, USA, pages 13322-13330. AAAI Press, 2025.
[36] Yichi Zhang. Zhuo Chen, Lei Liang, Huajun Chen, and Wen Zhang. Unleashing the power of imbalanced modality information for multi-modal knowledge graph completion. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/COLING 2024, 20-25 May, 2024, Torino, Italy, pages 1710-17130. ELRA and ICCL., 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We stated our contributions and the problems addressed in the Abstract and Introduction, and compared our results with more than twenty baselines, achieving state-of-the-ard (SOTA) performance.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer:
[Yes]
Justification:
We discussed the limitations of our work in Section Conclusion.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-specfiiaatinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
14

---

Answer: [Yes]
Justification: We have provided the corresponding formulas in the Section as well as complete set of assumptions and correct proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: We have provided the code and data, and introduced the experimental settings in the Section 41
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
15

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We provided link to access the code and data the Abstract and detailed the experimental settings.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ btiaps..c/puulic/guue
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We detailed the experimental settings in the Section 4.1 and provided link to the code and data.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We utilized three open-source benchmark datasets and conducted train- ing/testing splits and other initialization tasks in manner consistent with other papers (such as MyGO (35), AdaMF-MAT (36), MMRNS (32) to ensure the validity of our experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
16

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We described the server model and operating system used, along with other experimental settings, in the Section 4.1
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't iudn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https::/neuripss httpp//rrrppppcpppppppccccchhhta cwwciideeeennniieeeeeneenee
Answer: [Yes]
Justification: The data we used are all from open-source datasets, and we ensure that our research complies with the NeurlPS Code of Ethics in all respects.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: Our research focuses on knowledge graph completion, which can be applied to tasks such recommendation systems, and does not involve social impacts.
Guidelines:
17

---

The answer NA means that there is no societal impact of the work performed.
If the authors answer NA No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
Justification: Our paper does not involve such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The resources used are all open-source, and we have cited them in references.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
18

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperssi thood cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We provided a link to the code in the Abstract and all the resources we used are open-source.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [No]
Justification: Our paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [No]
Justification:
Our paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
19

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification: We descrihed the details and principles of the LLMs and explained how to use them in the provided link
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy (https://neurl)) httoree//eerrpp.cc//ooffeeeeeeee / for what should or should not be described.
20

---

A Technical Appendices and Supplementary Material
A.1 Dataset Analysis
In real-world multimodal knowledge graphs, the modal infor- mation diverse and complex. Some entities may have rich multimodal information such as images and text, while others may have scarce modalities (e.g., containing only text modality with the absence of image modality). This uneven distribution makes the task of knowledge graph completion exceptionally challenging. The missing modal information can lead to the omission of key knowledge, thereby affecting the accuracy of the completion process. As shown in Figure 5) we present the phenomenon of imbalance across different datasets. Specifi- cally, we focus on the imbalance of the image modality and calculate the proportion of entities containing image modality within the entire set of entities. Regarding the text modality, even if an entity lacks corresponding textual description, the entity itself contains textual information (for example, the triple <snowman, state, stillness>, which consists three words and is also considered form of textual information), so we consider the text modality to be balanced.
100
Percentage (%)
25
MKG-W
MKG-Y
DB15K
Datascts
Figure 5: Imbalance Situations in Different Datasets
A.2 Generalization Experiments
We conducted generalization experiments on four different MMKGC models (namely TBKGC, IKRL, AdaMF and QEB) using the multimodal features obtained from the LvMC and CMoA modules. We employed the same parameters as LBMKGC on the DB15K dataset. In fact, with more meticulous parameter tuning tailored different MMKGC models, these models could potentially exhibit even better performance. We report the Mean Reciprocal Rank (MRR) and Hit@ 10 results on the DB15K dataset, as shown in Figure 6 Based on the experimental results, we can conclude that by addressing the imbalance of multimodal information between entities and the heterogeneity of multimodal information within entities, MMKGC models can achieve significant performance gains. This indicates that the LvMC and CMoA modules can serve universal components applicable to various models in downstream tasks.
win CCMDA
36.21
winn CMCA
36
$3.69
53.31
31.51
$1.95
32
$1.55
MRR
Hit
30.16
30
29.33
29.65
28.4
49.80
49.00
26
TBKGC
IKRL
AdaMI
QEB
TBKGC
IKRL
AdaMI
QEB
Figure 6: Imbalance Situations in Different Datasets
A.3
Statistical Testing
To rigorously evaluate the stability and reproducibility of our proposed LBMKGC method, we have conducted five independent experiments on each of the three benchmark datasets (MKG-W, MKG-Y, and DB15K) without fixing the random seed during training.
21

---

The comprehensive results across all datasets, as depicted in Figure 7 demonstrate highly consistent performance patterns throughout the evaluation metries. On the MKG-W dataset, MRR values remain confined to narrow interval of 0.3840-0.3866 while Hit@1 scores exhibit stability, ranging from 0.3119 to 0.3152. similar degree of consistency is observed on the MKG-Y Y dataset, where MRR varies between 0.3992 and 0.4031, accompanied by Hit@) 1 scores spanning 0.3387-0.3499. The DB15K dataset also reflects minimal performance deviation, with MRR consistently maintained between 0.3707 and 0.3734. The standard deviations across all evaluation metrics MKG-W, MKG-Y, and DB15K datasets remain below 0.0026, demonstrating the exceptional algorithmic stability of our proposed method.
Algorithmic Stuhility on the Dataset
Algorichmic Stability on MKG-Y
Alporithmic Stability on
0.50
0.5
0.55
Notr Valuz
0.30
Valne
Moticc Valuec
0.15
0.3
0.30
Rma
Times
Figure 7: Algorithmic Stability on Different Datasets
A.4 Hyperparameter Sensitivity
MBR
MBR IKS1
MBS
1931
0.55
19
Netne Vahez
Matric Valax
Metric Vahee
0.55
01-
0:45
0:40
833
830
80
350
630
-10
2.6
150
400
900
0.30
18
120
222
150
100
6.30
Enibeddnng Dincesion
Namber of Negativve Samples
Margia
Figure 8: Hyperparameter Sensitivity MKG-W Dataset
We have conducted experiments on the core hyperparameters (including embedding dimension, number of negative samples, and margin) using the MKG-Y dataset. To ensure the faimesss of these experiments, we fixed the random seed at 42 and employed controlled variable approach for multiple independent experiments. The experimental results are shown as Figure
The experimental results indicate that while there is some variability in the outcomes under different hyperparameter settings, this variability is minor and does not significantly impact the model's performance.
A.5 Time Efficiency
We have conducted time efficiency experiments on the DB15K dataset using an NVIDIA GeForce 4090 GPU, employing the LBMKGC, TBKGC, QEB, RSME, and IKRL methods. The results are as Figure D this deceleration is within an acceptable range and leads to substantial performance improvements. The increased computational time is primarily due to the additional processing required for assigning weights to the text and image modalities, as well as the graph structural information regulated by relational context.
22

---

475 mic
Training Time (minutes)
400
301 ziin
300
231 mis
100
LBMK3C
TBRGC
QFB
RSME
IKRI
Methods
Figure 9: Training Time Efficiency DB15K Dataset
Overall, the LBMKGC model achieves commendable balance between efficiency, performance, and stability. The additional computational investment yields disproportionately valuable returns in terms of prediction accuracy and model robustness, making particularly suitable for applications where performance is prioritized over minimal training time.
23


Reviews:
[{"review_id": "ZHlX2rbSqK", "reviewer": "Reviewer_AYg4", "summary": "In this paper, the authors propose a novel Large model-driven Balanced  Multimodal Knowledge Graph Completion framework, termed LBMKGC. Initially, LBMKGC employs the Stable Diffusion XL (a large language model) to augment the imbalanced information across modalities. Subsequently, to bridge the semantic gap between heterogeneous modalities, LBMKGC aligns the multimodal embeddings of entities semantically by using the CLIP (Contrastive Language-Image Pre-Training) model. Furthermore, by distinguishing between the perceptual and conceptual attributes of entities, LBMKGC learns entity representations through the adaptive fusion of embeddings from various modalities, guided by structural information. The authors show comprehensive experiments and provide source code of their work for reproducibility.", "questions": "It is unclear to me about the novelty of the proposed architecture. It seems that it is a combination of already proposed techniques e.g., Bert, Vision Transformer, MLP etc. The authors need to more clearly highlight this in their work. \n\nThe Related Work is missing a recent work on KGC, which the authors should reference for completeness and recency.", "limitations": "The authors have provided some discussion on this as part of their Conclusions section, though this can be elaborated upon in its own section.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strength: The problem is well motivated with the sections of (1) The imbalance of inter-entity information across different modalities and (2)The heterogeneity of intra-entity multimodal information, with the contributions clearly highlighted. The problem formulation along with knowledge graph completion is clearly identified. Figure 2 is highly informative. Experiments are comprehensive and show solid performance.\n\nWeakness: It is unclear to me about the novelty of the proposed architecture. It seems that it is a combination of already proposed techniques e.g., Bert, Vision Transformer, MLP etc. The authors need to more clearly highlight this in their work. \n\nThe Related Work is missing a recent work on KGC, which the authors should reference for completeness and recency. \n\n[KDD 2022] Dual-Geometric Space Embedding Model for Two-View Knowledge Graphs. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '22). Association for Computing Machinery, New York, NY, USA, 676–686. https://doi.org/10.1145/3534678.3539350", "was_revised": true, "final_justification": "Thank you for the insightful comments by the authors. Upon reviewing the new results and explanations on novelty, I have decided to raise my score.", "rebuttal": "Dear reviewer, \n\nWe sincerely appreciate your thorough review and insightful comments on our paper. Your feedback is highly valued, and we have carefully considered your suggestions. As a result, we have made detailed responses and revisions accordingly. Below, we provide our specific responses to the issues and recommendations you have raised: \n\n(1) Further elaboration on the novelty of the proposed architecture: \n\nWhile components such as BERT, ViT, and MLP are widely utilized across various domains, their integration and synergy within the LBMKGC framework are unique. Specifically: (i) In LLmMC (LLM-Based Modality Completion) module: We are the first to introduce an advanced generative vision model, i.e., Stable Diffusion XL (SDXL), into the task of multimodal knowledge graph completion (MMKGC). The core innovation of this module lies in its ability to explicitly address the imbalance of cross-entity modal information, as opposed to the conventional reliance on random initialization. (ii) In CMoA (Cross-Modality Alignment) module: To ensure a fair evaluation of model performance and to facilitate meaningful comparisons with baseline approaches, we utilized mainstream pre-trained models commonly used in existing MMKGC methods for extracting visual and textual features. However, current methods tend to overlook the modality gap issue during multimodal feature extraction. This limitation hinders downstream tasks from fully capturing the deep semantic associations between modalities. Within the LBMKGC framework, we innovatively apply these techniques. By adopting a contrastive-learning-based approach, BERT and ViT are jointly leveraged to extract multimodal features, ensuring that representations from different modalities are aligned within a unified semantic space. (iii) In CGuAF (Context-Guided Adaptive Fusion) module: Traditional MMKGC methods typically rely on simple concatenation or averaging operations to fuse the multimodal information of entities. Unlike traditional methods, our approach distinguishes between the perceptual and conceptual attributes of entities and innovatively incorporates relational context information during the fusion process. Consider the triplet <Snowman, State, Still> as an example. Here, \"Snowman\" is a perceptible and concrete entity, making the visual modality particularly discriminative. As a result, we assign a higher weight to the image modality during feature fusion for entity \"Snowman\". Conversely, \"Still\" is a conceptual entity relying more on symbolic logical reasoning from textual data. Therefore, we assign a higher weight to the text modality during feature fusion for it. On the other hand, we incorporate relational context information to dynamically adjust the weight distribution between structural and multimodal information during the adaptive fusion process. Taking the triplet 〈Snowman, State, Still〉 as an example: since “State” is an abstract relation whose semantics hinge on logical associations between entities, we assign higher weight to the structural information of “Snowman.” This allows the model to more precisely interpret the relation “State” and thereby capture the abstract concept “Still.” Conversely, in the triplet 〈Snowman, Color, White〉, “Color” is a concrete relation. Here, textual, visual, and other multimodal signals provide the model with more discriminative cues than structural information alone. Therefore, we increase the weight given to these multimodal features of “Snowman,” enabling the model to accurately grasp the relation “Color” and the specific concept “White.”\n\nExperimental results demonstrate that our proposed LBMKGC framework achieves state-of-the-art performance across various datasets and scenarios when compared with 21 state-of-the-art baseline algorithms.\n\nMeanwhile, LBMKGC exhibits outstanding algorithmic stability. Specifically, we have conducted five independent experiments on each dataset (MKG-W, MKG-Y, DB15K) without fixing the random seed. The results for each dataset are as follows:\n\n(i) MKG-W (MRR: 0.3846 Hit@1: 0.312 Hit@3: 0.4178 Hit@10: 0.5146)\n\n1）MRR: 0.386091 Hit@1: 0.314343 Hit@3: 0.417525 Hit@10: 0.516729\n\n2）MRR: 0.384785 Hit@1: 0.312939 Hit@3: 0.417642 Hit@10: 0.516144\n\n3）MRR: 0.385227 Hit@1: 0.313407 Hit@3: 0.417057 Hit@10: 0.519303\n\n4）MRR: 0.386556 Hit@1: 0.315161 Hit@3: 0.419279 Hit@10: 0.516378\n\n5）MRR: 0.384019 Hit@1: 0.311886 Hit@3: 0.417291 Hit@10: 0.515910 \n\nSTD: 0.001013 STD: 0.001263\tSTD: 0.000879\t STD: 0.001380\n\n(ii) MKG-Y (MRR: 0.4003 Hit@1: 0.3389 Hit@3: 0.4311 Hit@10: 0.5081)\n\n1）MRR: 0.403128\tHit@1: 0.344912 Hit@3: 0.431656\tHit@10: 0.503380\n\n2）MRR: 0.399168\tHit@1: 0.338716 Hit@3: 0.427901\tHit@10: 0.506947\n\n3）MRR: 0.400182\tHit@1: 0.338903 Hit@3: 0.431093\tHit@10: 0.504882\n\n4）MRR: 0.401752\tHit@1: 0.341344 Hit@3: 0.432219\tHit@10: 0.506008\n\n5）MRR: 0.401017\tHit@1: 0.340030 Hit@3: 0.432970 Hit@10: 0.506196\n\nSTD: 0.001508 STD: 0.002536 STD: 0.001953 STD: 0.001388\n\n(iii) DB15K (MRR: 0.3723 Hit@1: 0.2778 Hit@3: 0.4275 Hit@10: 0.5471)\n\n1）MRR: 0.373437\tHit@1: 0.278732 Hit@3: 0.429105 Hit@10: 0.547566\n\n2）MRR: 0.372852\tHit@1: 0.277419 Hit@3: 0.430014 Hit@10: 0.547667\n\n3）MRR: 0.371596\tHit@1: 0.277166 Hit@3: 0.425823 Hit@10: 0.545698\n\n4）MRR: 0.371958\tHit@1: 0.275449 Hit@3: 0.429004 Hit@10: 0.547566\n\n5）MRR: 0.370715\tHit@1: 0.275651 Hit@3: 0.425823 Hit@10: 0.546051\n\nSTD: 0.001065 STD: 0.001356 STD: 0.001984  STD: 0.000954\n\n(2) The addition of related work: \n\nThank you very much for pointing out the omission in our related work section. The \"Dual-Geometric Space Embedding Model for Two-View Knowledge Graphs\" is indeed a significant contribution in this field, introducing an innovative dual-geometric space embedding model (DGS) specifically designed to handle knowledge graphs with two views. We deeply appreciate this contribution, and to ensure the comprehensiveness and currency of our research, we have incorporated a citation to this work in the \"Related Work\" section along with a detailed discussion in the text. We believe that this comprehensive literature review and discourse will allow our paper to more effectively demonstrate its advancements and contributions relative to existing research. Thank you again for your invaluable feedback, and we remain committed to enhancing the quality and depth of our research.\n\n(3) Further Discussion on Limitations: \n\nWe sincerely appreciate the reviewer’s insightful feedback. We have recognized the importance of discussing research limitations and future directions. Therefore, we have added a specific \"Limitations and Future Work\" paragraph in the Conclusion section, where we detail the current limitations of our study and potential avenues for future research. We begin by outlining the primary limitations: our study currently focuses on text and image modalities, without integrating semantically rich information such as audio and video into a unified framework. Additionally, our experiments have been conducted exclusively on general domain datasets, leaving the adaptability and robustness in specialized fields like biomedical data and social networks unverified. Following this, we propose directions for future research, including the formal integration of audio and video modalities into the multimodal knowledge graph completion framework. For specialized scenarios, such as biomedical knowledge graphs and social networks, it is crucial to establish tailored multimodal benchmarks and evaluation protocols to advance MMKGC from generic modeling to domain-specific applications. By including this dedicated section, we believe our paper more comprehensively demonstrates the research's depth and breadth, offering readers a clearer trajectory and directions for future development. We once again thank the reviewer for their valuable suggestions and remain committed to continually refining the paper to achieve higher academic standards."}, {"review_id": "gwmbIaqgJM", "reviewer": "Reviewer_dZV7", "summary": "This paper introduces LBMKGC, a benchmark and evaluation suite aimed at assessing large language models (LLMs) for knowledge graph completion (KGC) tasks. The framework covers various KGC paradigms—link prediction, triple classification, and query answering—across multiple datasets. It further dissects performance by task format (e.g., prompt-based vs. scoring-based) and by evaluation granularity (e.g., head/tail/generalized split). The authors compare LLMs such as GPT and GLM with traditional structure-based KGC models.", "questions": "a) The paper uses GLM and ChatGPT, but there is no comparison with strong open-source instruction-tuned LLMs (e.g., LLaMA-2-chat, Mistral, Claude) or modern prompting strategies (e.g., CoT, self-consistency).\nb) Current tasks focus heavily on triple completion/classification. However, real-world KGs involve complex reasoning.\nc) Prompt design, sampling strategy, and evaluation conditions (e.g., few-shot template consistency) can strongly affect LLM performance.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\na) Evaluating LLMs for KGC is important as LLMs increasingly replace specialized symbolic systems.\nb) The benchmark breaks down KGC evaluation across multiple axes, which offers useful diagnostic insights.\nc) The authors run consistent experiments comparing LLMs (in zero-shot or few-shot) with structure-based methods.\n\nWeaknesses\na) The paper uses GLM and ChatGPT, but there is no comparison with strong open-source instruction-tuned LLMs (e.g., LLaMA-2-chat, Mistral, Claude) or modern prompting strategies (e.g., CoT, self-consistency).\nb) Current tasks focus heavily on triple completion/classification. However, real-world KGs involve complex reasoning.\nc) Prompt design, sampling strategy, and evaluation conditions (e.g., few-shot template consistency) can strongly affect LLM performance.", "was_revised": true, "final_justification": "I have read other reviewers comments, so I would like to change the score.", "rebuttal": "Dear reviewer, \n\nWe sincerely appreciate the time and effort you have dedicated to reviewing our manuscript. However, upon thorough examination, it seems that the comments you provided do not align with the content of our submission. We suspect these comments might have been inadvertently linked to another submission. \n\nThe detailed comments are as follows: (i) The framework covers various KGC paradigms—link prediction, triple classification, and query answering—across multiple datasets. It further dissects performance by task format (e.g., prompt-based vs. scoring-based) and by evaluation granularity (e.g., head/tail/generalized split); (ii) The authors compare LLMs such as GPT and GLM with traditional structure-based KGC models. (iii) The authors run consistent experiments comparing LLMs (in zero-shot or few-shot) with structure-based methods. Since these comments do not pertain to our manuscript, we would appreciate your confirmation regarding this matter.\n\nTo save your precious time, please allow us to reiterate our key contributions:\n\nSpecifically: (i) In LLmMC (LLM-Based Modality Completion) module: For the first time, we introduce the advanced generative vision model Stable Diffusion XL into multimodal knowledge-graph completion. The core innovation lies in explicitly rectifying cross-entity modal imbalance instead of relying on conventional random initialization, thereby delivering semantically richer and more accurate completions for subsequent alignment and fusion. (ii) In CMoA (Cross-Modality Alignment) module: Guided by contrastive learning, we jointly leverage BERT and ViT to extract multimodal features, ensuring that representations from different modalities are aligned in a unified semantic space. This effectively alleviates intra-entity heterogeneity and markedly enhances the capture of cross-modal associations. (iii) In CGuAF (Context-Guided Adaptive Fusion) module: Whereas prior methods often resort to simple concatenation or averaging, we innovatively incorporate relational context and distinguish between perceptual and conceptual entity attributes to dynamically re-weight modalities. Experimental results demonstrate that our proposed LBMKGC framework achieves state-of-the-art performance across various datasets and scenarios when compared with 21 state-of-the-art baseline algorithms."}, {"review_id": "sUKllNm4M8", "reviewer": "Reviewer_UBFY", "summary": "This paper presents LBMKGC, a multimodal knowledge graph completion framework targeting key challenges such as modality imbalance, heterogeneity, and semantic inconsistency. LBMKGC employs the Stable Diffusion XL to augment the imbalanced information across modalities. LBMKGC aligns the multimodal embeddings of entities semantically by using the CLIP. Comprehensive experiments on 21 benchmarks show the effectiveness.", "questions": "See Strengths And Weaknesses.", "limitations": "See Strengths And Weaknesses.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. This paper is well writing and easy to read.\n2. The perceptual-conceptual dual-branch fusion module is innovative and contributes to improved semantic coherence.\n\nWeaknesses:\n1. In the LLM-Based Modality Completion section, it appears that the main contributions largely rely on the existing work of SDXL (Stable Diffusion XL). The authors should more clearly articulate the innovations and improvements this paper makes upon prior work to help readers understand its original contributions.\n\n2. This paper lacks an analysis or empirical evaluation of time complexity, which should be included to better assess the method's efficiency.\n\n3. While the paper emphasizes modality modeling and fusion, it pays relatively little attention to structural modeling of the KG. The influence of graph structure on the completion task is underexplored.\n\n4. The authors have provided a non-anonymous GitHub link in the abstract, which appears to violate the anonymity guidelines. While I do not wish for this paper to be desk rejected because of it, it should be addressed.", "was_revised": true, "final_justification": "The authors addressed my concerns. I will maintain my score of 4.", "rebuttal": "Dear reviewer, \n\nThank you for reviewing our paper and for your valuable feedback. We greatly appreciate your insights and have made detailed responses and revisions based on your comments. Below are our specific responses to the issues and suggestions you raised:\n \n(1) Response regarding the concern that the main contributions seem to heavily rely on existing SDXL work: \n\nWe appreciate the reviewer's attention to this aspect. It is important to clarify that the SDXL model itself, as a powerful open-source generative foundation, has not had its core architecture modified in our work. We fully agree with the reviewer’s point that our original contributions need to be more clearly defined. The core innovation of our work lies not in improving the internal mechanisms of SDXL, but in the systematic application and integration of it for the specific task of multimodal knowledge graph completion. Specifically: (i) In LLmMC (LLM-Based Modality Completion) module: We are the first to introduce an advanced generative vision model, i.e., Stable Diffusion XL (SDXL), into the task of multimodal knowledge graph completion (MMKGC). The core innovation of this module lies in its ability to explicitly address the imbalance of cross-entity modal information, as opposed to the conventional reliance on random initialization. (ii) In CMoA (Cross-Modality Alignment) module: To ensure a fair evaluation of model performance and to facilitate meaningful comparisons with baseline approaches, we utilized the pre-trained models commonly used in existing MMKGC methods for extracting visual and textual features. However, current methods tend to overlook the modality gap issue during multimodal feature extraction, failing to construct a coupled representation space that enables effective cross-modal interaction. This limitation hinders downstream tasks from fully capturing the deep semantic associations between modalities. Within the LBMKGC framework, we innovatively apply these techniques. By adopting a contrastive-learning-based approach, BERT and ViT are jointly leveraged to extract multimodal features, ensuring that representations from different modalities are aligned within a unified semantic space. (iii) In CGuAF (Context-Guided Adaptive Fusion) module: Traditional MMKGC methods typically rely on simple concatenation or averaging operations to fuse the multimodal information of entities. Unlike traditional methods, our approach distinguishes between the perceptual and conceptual attributes of entities. On the other hand, we incorporate relational context information to dynamically adjust the weight distribution between structural and multimodal information during the adaptive fusion process. Finally, experimental results demonstrate that our proposed LBMKGC framework achieves state-of-the-art performance across various datasets and scenarios when compared with 21 state-of-the-art baseline algorithms.\n\n (2) Response concerning the analysis or empirical evaluation of time complexity:\n\nWe sincerely thank the reviewer for pointing out this oversight, as efficiency evaluation is indeed crucial for assessing the practicality of the method. We have conducted time efficiency experiments on the DB15K dataset using an NVIDIA GeForce 4090 GPU, employing the LBMKGC, TBKGC, QEB, RSME, and IKRL methods. The results are as follows: LBMKGC: 7h53min; TBKGC: 5h1min; QEB: 8h34min; RSME: 5h38min; IKRL: 3h51min. The experimental results indicate that although LBMKGC reduces training speed slightly, this deceleration is within an acceptable range and leads to substantial performance improvements. The increased computational time is primarily due to the additional processing required for assigning weights to the text and image modalities, as well as to the graph structural information regulated by relational context. Overall, the LBMKGC model achieves a commendable balance between efficiency, performance, and stability. Additionally, we will carry out further experiments on runtime efficiency on other datasets and include the results in the revised manuscript. We are grateful for the reviewer's insightful comments, as we are confident that these enhancements will enable readers to thoroughly evaluate the efficiency of our method.\n\n(3) Response concerning the exploration of the impact of graph structure on completion tasks:\n\nThank you for appointing this. In this paper, we employ RotatE to learn graph structural information. The core idea of RotatE is to represent relations as rotational operations on the entity embeddings in the complex vector space. It captures implicit topological structure information through the intricate interactions between entities and relations. Moreover, graph structural information is crucial for our model. During the adaptive fusion process, we innovatively incorporate relational contextual information to dynamically adjust the weight distribution between structural and multimodal information. Furthermore, the efficacy of graph structure utilization is detailed in our ablation studies. When graph structural information is omitted in the LBMKGC, the hit@1 metric significantly drops: from 27.78 to 16.96 on the DB15K dataset, from 31.20 to 23.84 on the MKG-W dataset, and from 33.89 to 29.33 on the MKG-Y dataset. This clearly demonstrates the essential role of graph structural information in our LBMKGC framework.\n\n(4) Response concerning the non-anonymous GitHub link in the abstract: \n\nThank you very much for pointing out this oversight. The link was originally intended to share our research's code and resources openly. We have promptly removed the link in the revised manuscript to fully comply with the double-blind review requirements. We appreciate the reviewer’s understanding and patience regarding this matter."}, {"review_id": "7GlGGJxqR5", "reviewer": "Reviewer_zfjV", "summary": "This paper introduces LBMKGC, a novel framework for Multimodal Knowledge Graph Completion designed to address three key challenges: modality imbalance, intra-entity modality heterogeneity, and inter-modality information inconsistency. Through evaluations and comparisons with state-of-the-art methods, the effectiveness of LBMKGC is demonstrated.", "questions": "-  Is it possible to provide more detailed statistics, such as error bars?\n- Are the methods sensitive to the hyperparameters? Lack of sensitivity analysis.", "limitations": "The limitations are mentioned in the paper  in the conclusion section.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\nThe paper is well-written, clearly presenting the motivation and structure of the proposed method. The key contributions of the paper include: introducing the   Large model-driven Balanced MMKGC (LBMKGC) for multimodal knowledge graph completion, evaluation of LBMKGC against state-of-the-art MMKGC methods; and a good level of detail is provided on the methodology with the strong performance of the proposed approach.\n\nWeaknesses:\n\n- The paper restricts itself to text and images. Though this is reasonable given scope, discussions around audio/video modalities and potential future extensions are only briefly mentioned.\n\n- Despite strong performance margins, the lack of statistical testing (e.g., standard deviations) weakens claims of consistency. Including these is crucial to allow the reader to draw meaningful conclusions from the results.", "was_revised": true, "final_justification": "This work is complete, well-written, clear and useful.\nI think it is to be acceptable.", "rebuttal": "Dear reviewer, \n\nThank you for reviewing our paper and providing valuable feedback. We greatly appreciate your comments and have made detailed responses and revisions based on your suggestions. Below are our specific replies to the issues and recommendations you raised:\n\n(1) Response concerning the further discussion on limitations:\n\nWe sincerely appreciate the reviewer’s insightful comments, which have prompted us to provide a more detailed discussion of the audio and video modalities and potential future directions. First, we summarize the primary limitations of our research: our framework currently focuses only on two modalities, i.e., text and images, without incorporating other semantically rich information such as audio and video. For instance, in cases where the entity is an animal, the audio features of its sounds or its dynamic behaviors captured in video are essential for completing related triples. Additionally, existing experiments are conducted on general domain datasets, leaving the adaptability and robustness in specialized domains such as biomedical fields and social networks unvalidated. Next, we explore directions for future research: future efforts can formally integrate audio and video modalities into the multimodal knowledge graph completion system. This includes developing scalable architectures that effectively balance high capacity with low complexity. Moreover, for vertical scenarios like biomedical knowledge graphs and social networks, constructing specialized multimodal benchmarks and evaluation protocols will be essential to advance MMKGC from general modeling to domain-specific applications. We believe that with these additions, our paper will offer a more comprehensive portrayal of the research's depth and breadth, providing readers with a clearer understanding of the study's trajectory and future development avenues. Once again, we thank the reviewer for their suggestions. We remain committed to enhancing the paper's content to meet higher academic standards.\n\n(2) Response concerning the statistical tests (e.g., standard deviation):\n\nWe appreciate your insightful feedback. Previously, to ensure stable reproducibility, our experiments were conducted using a fixed random seed. However, based on your suggestion, we recognize that omitting quantitative indicators of result variability, such as standard deviation, can indeed impact the assessment of the method's robustness and the reliability of our conclusions. This represents a shortcoming in the experimental section of the original manuscript, and we apologize for this oversight. To address this issue comprehensively and provide readers with more statistically significant insights, we have conducted five independent experiments on each dataset (MKG-W, MKG-Y, DB15K) without fixing the random seed. The results for each dataset are as follows:\n\n(i) MKG-W (MRR: 0.3846 Hit@1: 0.3120 Hit@3: 0.4178 Hit@10: 0.5146)\n\n1）MRR: 0.386091 Hit@1: 0.314343 Hit@3: 0.417525 Hit@10: 0.516729\n\n2）MRR: 0.384785 Hit@1: 0.312939 Hit@3: 0.417642 Hit@10: 0.516144\n\n3）MRR: 0.385227 Hit@1: 0.313407 Hit@3: 0.417057 Hit@10: 0.519303\n\n4）MRR: 0.386556 Hit@1: 0.315161 Hit@3: 0.419279 Hit@10: 0.516378\n\n5）MRR: 0.384019 Hit@1: 0.311886 Hit@3: 0.417291 Hit@10: 0.515910 \n\nSTD: 0.001013 STD: 0.001263 STD: 0.000879 STD: 0.001380\n\n(ii) MKG-Y (MRR: 0.4003 Hit@1: 0.3389 Hit@3: 0.4311 Hit@10: 0.5081)\n\n1）MRR: 0.403128\tHit@1: 0.344912 Hit@3: 0.431656\tHit@10: 0.503380\n\n2）MRR: 0.399168\tHit@1: 0.338716 Hit@3: 0.427901\tHit@10: 0.506947\n\n3）MRR: 0.400182\tHit@1: 0.338903 Hit@3: 0.431093\tHit@10: 0.504882\n\n4）MRR: 0.401752\tHit@1: 0.341344 Hit@3: 0.432219\tHit@10: 0.506008\n\n5）MRR: 0.401017\tHit@1: 0.340030 Hit@3: 0.432970 \tHit@10: 0.506196\n\nSTD: 0.001508 STD: 0.002536 STD: 0.001953 STD: 0.001388\n\n(iii) DB15K (MRR: 0.3723 Hit@1:0.2778 Hit@3: 0.4275 Hit@10: 0.5471)\n\n1）MRR: 0.373437 Hit@1: 0.278732 Hit@3: 0.429105 Hit@10: 0.547566\n\n2）MRR: 0.372852\tHit@1: 0.277419 Hit@3: 0.430014 Hit@10: 0.547667\n\n3）MRR: 0.371596\tHit@1: 0.277166 Hit@3: 0.425823 Hit@10: 0.545698\n\n4）MRR: 0.371958\tHit@1: 0.275449 Hit@3: 0.429004 Hit@10: 0.547566\n\n5）MRR: 0.370715\tHit@1: 0.275651 Hit@3: 0.425823 Hit@10: 0.546051\n\nSTD: 0.001065 STD: 0.001356 STD: 0.001984 STD: 0.000954\n\nThe experimental data demonstrate that our results exhibit excellent stability. We will highlight quantitative indicators of result variability in the results section. We sincerely appreciate the reviewer’s suggestions and will continue to strive for improvements to achieve higher academic standards in our paper.\n\n(3) Response concerning the hyperparameter sensitivity:\n\nThank you for your suggestions regarding the analysis of hyperparameter sensitivity. We have conducted experiments on the core hyperparameters (including embedding dimension, number of negative samples, and margin) using the MKG-Y dataset. To ensure the fairness of these experiments, we fixed the random seed at 42 and employed a controlled variable approach for multiple independent experiments. The experimental results are shown below.\n\n(i) Dim (embedding dimension)\n\nWhen Dim=300:\n\nMRR: 0.3887 Hit@1: 0.3191 Hit@3: 0.4271 Hit@10: 0.5084   \n\nWhen Dim=400:\n\nMRR: 0.3980 Hit@1: 0.3349 Hit@3: 0.4307 Hit@10: 0.5071   \n\nWhen Dim=500:\n\nMRR: 0.4003 Hit@1: 0.3389 Hit@3: 0.4311 Hit@10: 0.5081  \n\nWhen Dim=600:\n\nMRR: 0.3980 Hit@1: 0.3385 Hit@3: 0.4266 Hit@10: 0.5024   \n\nWhen Dim=700:\n\nMRR: 0.3961 Hit@1: 0.3368 Hit@3: 0.4253 Hit@10: 0.5023\n\n(ii) neg_num (number of negative samples)\n\nWhen neg_num = 32:\n\nMRR: 0.3965 Hit@1: 0.3237 Hit@3: 0.4358 Hit@10: 0.5212\n\nWhen neg_num = 64:\n\nMRR: 0.3980 Hit@1: 0.3295 Hit@3: 0.4348 Hit@10: 0.5152  \n   \nWhen neg_num = 128:\n\nMRR: 0.4003 Hit@1: 0.3389 Hit@3: 0.4311 Hit@10: 0.5081  \n\nWhen neg_num=192:\n\nMRR: 0.3960 Hit@1: 0.3342 Hit@3: 0.4296 Hit@10: 0.5008\n\n(iii) Margin \n\nWhen Margin=8:\n\nMRR: 0.3856 Hit@1: 0.3286 Hit@3: 0.4163 Hit@10: 0.4818   \n\nWhen Margin=12:\n\nMRR: 0.3912 Hit@1: 0.3321 Hit@3: 0.4223 Hit@10: 0.4869  \n\nWhen Margin=16:\n\nMRR: 0.3954 Hit@1: 0.3355 Hit@3: 0.4275 Hit@10: 0.4936 \n  \nWhen Margin=20:\n\nMRR: 0.3981 Hit@1: 0.3376 Hit@3: 0.4294 Hit@10: 0.5032    \n\nWhen Margin=24:\n\nMRR: 0.4003 Hit@1: 0.3389 Hit@3: 0.4311 Hit@10: 0.5081 \n\n\nThe experimental results indicate that while there is some variability in the outcomes under different hyperparameter settings, this variability is minor and does not significantly impact the model's performance. Additionally, we plan to carry out further experiments about hyperparameter sensitivity over the remaining datasets and include these findings in the revised manuscript. We sincerely appreciate the reviewer’s valuable comments, which help ensure that readers can thoroughly assess the stability of our method."}]

Author rebuttal:

Meta-review: This paper proposes LBMKGC, a novel large-model-driven framework for multimodal knowledge graph completion (MMKGC). Extensive experiments across 21 baselines and three datasets demonstrate state-of-the-art performance with good efficiency and generalizability. The paper presents a clearly defined problem and proposes the LBMKGC framework, which innovatively integrates existing techniques to systematically address modality imbalance and heterogeneity in multimodal knowledge graph completion. In their rebuttal, the authors thoroughly responded to all reviewer concerns by providing extensive additional experiments—including five independent runs, runtime analysis, hyperparameter sensitivity studies, and ablation experiments— and expanded discussions. These revisions led to consistently improved or maintained scores from three reviewers. Overall, the work demonstrates significant performance gains, reproducibility, and practical value, warranting acceptance.
Decision: Accept (poster)