Uncertain Knowledge Graph Completion via Semi-Supervised Confidence Distribution Learning
Authors: Tianxing Wu
Shutong Zhu
Jingting Wang
Ning Xu
Guilin Qi
Haofen Wang
Venue: NeurIPS 2025 spotlight ()
Keywords: Uncertain Knowledge Graph Completion
Confidence Distribution Learning
Meta Self-Training
Primary area: applications

Abstract: Uncertain knowledge graphs (UKGs) associate each triple with a confidence score to provide more precise knowledge representations. Recently, since real-world UKGs suffer from the incompleteness, uncertain knowledge graph (UKG) completion attracts more attention, aiming to complete missing triples and confidences. Current studies attempt to learn UKG embeddings to solve this problem, but they neglect the extremely imbalanced distributions of triple confidences. This causes that the learnt embeddings are insufficient to high-quality UKG completion. Thus, in this paper, to address the above issue, we propose a new semi-supervised Confidence Distribution Learning (ssCDL) method for UKG completion, where each triple confidence is transformed into a confidence distribution to introduce more supervision information of different confidences to reinforce the embedding learning process. ssCDL iteratively learns UKG embedding by relational learning on labeled data (i.e., existing triples with confidences) and unlabeled data with pseudo labels (i.e., unseen triples with the generated confidences), which are predicted by meta-learning to augment the training data and rebalance the distribution of triple confidences. Experiments on two UKG datasets demonstrate that ssCDL consistently outperforms the state-of-the-art baselines in different evaluation metrics.

Uncertain Knowledge Graph Completion via Semi-Supervised Confidence Distribution Learning
Tianxing Wu Shutong Zhu'
Jingting Wang' Guilin Qi Haofen Wang*
Ning Xu
School of Computer Science and Engineering, Southeast University, China Key Laboratory of New Generation Artificial Intelligence Technology and its Interdisciplinary Applications (Southeast University), Ministry of Education, China 3 College of Design and Innovation, Tongji University, China {t (tianxingwu, shutong_zhu, xning)@seu. edua.eeduuee. cn
Abstract
Uncertain knowledge graphs (UKGs) associate each triple with a confidence score to provide more precise knowledge representations. Recently, since real-world UKGs suffer from the incompleteness, uncertain knowledge graph (UKG) comple- tion attracts more attention, aiming to complete missing triples and confidences. Current studies attempt to learn UKG embeddings to solve this problem, but they neglect the extremely imbalanced distributions of triple confidences. This causes that the learnt embeddings Thus, in this paper, to address the above issue, we propose a new semi- cumi-supervised Confidence Distribution Learning (ssCDL) method for UKG completion, where each triple confidence is transformed into a confidence distribution to introduce supervision information of different confidences reinforce the embedding learning process. ssCDL. iteratively learns UKG embedding by relational learning labeled data (i.e., existing triples with confidences) and unlabeled data with pscudo labels (i.e., unseen triples with the generated confidences), which are pre- dicted by meta-learning to augment the training data and rebalance the distribution of triple confidences. Experiments on two UKG datasets demonstrate that ss ssCDL consistently outperforms state-of-the -fethrrrrt baselines in different evaluation metrics.
insufficient to high-quality UKG completion.
Introduction
Knowledge Graphs (KGs) are usually defined as multi-relational graphs describing knowledge with deterministic triples, each of which is the form of (subject, predicate, object), c.g., (Michael Jordan, Nationality, U.S.). Such kind of structured knowledge has supported many applications, including question answering 14, semantie search 77, decision-male makinn systems [27], and etc. Recently,
uncertain KGs (UKGs), such as NELL [1]. ConceptNet [17], and Probase 224 10 have received
much more attention. UKGs measure the uncertainty knowledge by associating each triple with confidence score, which also denotes the likelihood of that triple to be true. Such setting benefits to precise knowledge representation and reasoning in the real world.
Most KGs suffer from incompleteness 21 since new knowledge is always emerging over time, and so are UKGs. Thus, various uncertain knowledge graph (UKG) embedding methods [ 12 2
26 20 22 are proposed to perform link prediction and confidence prediction for UKG completion. UKG embedding learns the representations of entities and relations in low-dimensional space where graph structures and triple confidences are preserved. During the learning process, the above methods neglect the fact that the distributions of triple confidences are extremely imbalanced in
"Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

1.4
Description degree
Description degioe
1.2
Couns
1,0
0.81
0.4
02
0.0
0.2
0.4
0.8
0.78
0.76 0.77 0,78 0.79 0.30
Triple Confidence
Triple Confidence
Triple Confidence
(a)
(b)
Figure 1: (a) The histogram of triple confidence in NELL; (b) An example of a triple confidence equal to 0.78, which is transformed to confidence distribution.
most UKGs, i... only high-come ceghiconfieence triples are reserved. For example, as shown in Figure Ta). NELL only contains the triples with confidences larger than 0.9, and this is because there is need to store low-confidence triples which are probably erroneous. Learning on such imbalanced data will cause that the embedding-basse which lowers the quality of the generated embeddings for UKG completion. In this paper, we study how to reinforce the learning process under the imbalanced confidence distribution to obtain high-qualit embeddings for UKG completion. This problem is non-trivial, and we try to solve by the reinforcement strategies with labeled data (i.e., existing triples with confidences) and unlabeled data (i.e., sampled unseen triples without confidences), which poses two challenges as follows:
models cannot fit relatively lower-confidence samples,
Challenge 1: Reinforcement with Labeled Data. In UKG embedding learning, labeled data triples with confidences in the training data, and such confidences are imbalanced, so the challenge is how effectively capture the supervision signals of unseen confidences or the confidences with a small number from labeled data to reinforce the learning process.
Challenge Reinforcement with Unlabeled Data. Since the training data is short of low- confidence triples, we aim to apply negative sampling to add unseen triples which are often false and probably low-confidence to the training data. Thus, the challenge is how to generate reliable confidences for the unlabeled data (i.e., unseen triples) to reinforce the learning process.
To solve both challenges, propose new supi-supereised Confidence Distribution Learning (ssCDL) method for UKG completion. In ssCDL, each triple confidence transformed into a confidence distribution. The triple confidence in UKG is relatively fuzzy concept, e.g., there is no obvious distinction that the confidence of triple is 0.77 or 0.78 or 0.79. This inspires us that the triples with neighboring confidences can be utilized while learning features for particular confidence, which is similar to the usage of label distribution in facial age estimation [ As shown in Figure [bb), after transforming the confidence 0.78 into confidence distribution for a triple the labeled data, the supervision signals of more confidences (e.g., 0.76, 0.77, 0.79, and etc.) can be introduced into the learning process, even if such confidences are few or unseen, which can help solve challenge 1.
ssCDL has two components: Confidence Distribution Learning based Relational Learner (CDL-RL) and Pseudo Confidence Distribution Generator (PCDG). CDL-RL iteratively learns UKG embeddings with labeled data and pseudo labeled data (i.e., negative sampled unseen triples with pseudo confidence labels) generated by PCDG for UKG completion. PCDG selects high-quality pseudo confidence labels for unlabeled data, and it is meta optimized by CDL-RI. with labeled data. The whole process is actually meta self-traine celf-training, which can alleviate the problem of gradual drifts (15) and introduce more reliable confidence labels for unlabeled data, thereby overcoming the challenge 2. Experiments on real-world datasets show the effectiveness and superiority of ssCDL compared with the -ateeoffthe-aan baselines in both UKG completion tasks of confidence prediction and link prediction.
Contributions. The main contributions of this paper are summarized as follows:
We propose a new semi-supervised method SsCDL, which applies meta self-training generate reliable confidences for unlabeled data in UKG embedding learning. This fully exploits unlabeled data augment the training data so as to resolve the problem of imbalanced confidence distribution.
We design new confidence distribution learning strategy in UKG embedding learning, which trans- forms triple confidences into confidence distributions and this benefits to capture the supervision information of few unseen confidences in the labeled data.

---

We conduct comprehensive experiments on UKG datasets, which not only shows that ssCDL outperforms baselines in different evaluation metrics for different tasks, but also verifies the effectiveness of confidence distribution learning and meta self-training for UKG completion.
2
Related Work
In this section, we review the existing studies on UKG completion, which refers to confidence prediction and link prediction. Relational learning is widely used to acquire UKG embeddings for UKG completion, and the core idea to embed entities and relations with the structure and confidence information in the UKG, which is called normal relational learning [22]. Besides, few-shot relational learning further consider the long-tail distribution relations in modeling real-world UKGs.
Normal Relational Learning for UKG. UKGE 3 is a classic method in this field, which uses the
scoring function of DistMult (25) to model triple confidences and solves the false negative problem with probabilistic soft logic. PASSLEAF [5] extends UKGE for other types of scoring functions, and also uses semi-supervised learning to alleviate the false negative problem. BEUIRE [2] applies box embedding for UKG embedding learning, in which entities represented as boxes, relationships are modeled as affine transformations head and tail entity boxes, and triple confidences are modeled by the intersection between transformed boxes. UPGAT [20] incorporates subgraph features and generalizes graph attention network for UKG completion. UKGsE (26) treats each triple as a short sentence and learns the confidence using LSTM. GTransE [22 and FocusE [16] associate triple confidences with margin operations the loss functions, which only solve the task of link prediction for UKGs. UKRM 4 tries to mine rules using transformer to link prediction and leverage pre-trained language model to compute triple confidences.
Few-Shot Relational I Learning for UKG. Recently, few-shot UKG completion has attracted much attention, e.g., GMUC (28) and GMUC+ (23) apply metric learning to learn UKG embeddings and achieve good performance, but the few-shot problem is not the focus of this paper. unKR (22) is UKG embedding learning and completion tool which re-implements both works of few-shot relational learning and normal relational learning for UKG.
There also exist some works (e.g.. regarding reasoning for -neeerinn on UKG, but the focus is not UKG completion, so we will not compare our method with such works. Existing relational learning methods for UKG completion neglect the fact that the distribution of triple confidences is extremely imbalanced in most UKGs, which causes the performance of UKG completion is still unsatisfactory. Our proposed method ssCDL aims to solve this problem by reinforeing the UKG embedding learning process using both labeled data and unlabeled data.
3 Preliminaries
3.1 Problem Definition
Uncertain Knowledge Graph. An uncertain knowledge graph is repository of factual knowledge denoted as a set of quadruples G [(h,r,, (1,h,,,,,,)) s),,, &,r E RR E [0, 1]}. where E and R respectively the sets of entities and relations, and is the confidence score measuring the triple uncertainty, which represents the likelihood of the triple (h, r, being true.
Definition 1.
Definition 2. Uncertain Knowledge Graph Completion. Uncertain knowledge graph completion has two sub-tasks, which are confidence prediction and link prediction. Given a query (h, ?) where (h, r, t) is a factual triple, confidence prediction to estimate the missing triple confidence. Given another query (h' .)) where h is a head entity and r is a relation, link prediction is to predict the missing tail entity.
3.2 Confidence Distribution Learning
Confidence distribution learning (CDL) aims to learn a model which can accurately estimate the confidence distribution of each given triple. CDL is variant of label distribution learning [ (LDL) applied in UKG completion. L.L. is machine learning paradigm that not only predicts the labels relevant to instances, but also quantifies the degree of relevance of each label. Before CDL, triple
3

---

confidences are transformed into confidence distributions to capture the supervision information of few or unseen confidences the labeled data.
Confidence distribution is defined as a discrete distribution in this paper, and this is also the setting of LDL. [8 In this way, since the confidence interval is [0, 1]. we directly the confidence labels at granularity of n. and the ordered confidence label set is {0, 2 n-1 11.. For given quadruple (h, r, 1, in UKG, we define the confidence distribution of (h, where > 2i=0 = 1 and s¹ E [0, 1] is a description degree that the confidence label describe the triple rh,r,t). In this paper, the confidence distribution $ is generated by a Gaussian distribution (ss o²), where is the standard deviation, and the confidence is the mean, which causes that has the highest description degree. Thus, in CDL, each piece of knowledge can be represented as a quadruple (h, t, s), where ~ (s,, o²) is the confidence distribution. Here, empirically set as 100, i.e., we have 101 confidence labels in total.
(s') RR+1,
Methodology
4.1 Overview
Figure 2 a) provides the overview of our semi-supervised confidence distribution learning method ssCDL, which consists of two key components: CDL-based relational learner (CDL-RL) and pseudo confidence distribution generator (PCDG). At first, we apply the strategy mentioned in Section 3.2 to transform all triple confidences in the labeled data into confidence distributions. We utilize CDL-RL to learn UKG embeddings with labeled data and pseudo labeled data generated by PCDG. PCDG generates high-guality pscudo confidences labels for unlabeled data, and CDL-RL iteratively exploits these pscudo labeled data and further improves its performance. CDL-RL and PCDG have the same structure (Figure (())). but their training processes the losses on confidence prediction and link prediction with labeled data and pseudo labeled data, while PCDG evaluates the performance of CDL-RL after exploiting pseudo labeled data generated by PCDG and takes it as the meta learning objective. PCDG meta optimized by CDL-RL with labeled data. SsCDL learned by meta self-training with iteratively training CDL-RL and PCDG.
different. CDL-RL is optimized by minimizing
(o) The overview SCCDL
(b) The fromework of COL-RL and PCDG.
The process of nete self-trarit
Updote
Lobalied deve
CDL-based Relational Learner (0)
(hel Ael
de (btor (pa)))
L (..
Netwerk
Uptlate
Presdoo
Enbeckingg Layer
Pseudo Confidence Distribution Generator (1))
fei (Ba (el
Aagatin
Figure 2: (a) The overview of ssCDL; (b) The framework of CDL-RL and PCDG.
4.2 CDL-based Relational Learner
The main purpose of CDL-RL is to learn the embeddings of entities and relations. Given the i-th quadruple (he, rsy d,, 8i) in the labeled data, it will be transformed into 4 = (h4, Fúy li, 8;) as the input of CDL-RL after mapping the confidence 8; to the confidence distribution using a Gaussian distribution. As shown in Figure CDL-RL is trained by minimizing the losses on confidence prediction and link prediction, i.c., LCP and LLP, and explain the details as follows.
To compute the loss Lcp on confidence prediction, we first concatenate the embeddings of h4 rrs and t; and feed it into two-layer fully connected network (FCN), which produces an (n + -ddimensional vector. We then apply the Softmax function as the activation function, and the predicted confidence distribution & of the triple (h;, ri, t4) can be computed as:
a = Softmar( FCN, (hal|ra]tt
where represents the concatenation between embeddings, and FCN, a function that the FCN transforms the concatenated embedding into an (n + -)dimennsional vector. Thus, the Kullback-

---

Leibler (KL) divergence can be utilized to measure the similarity between the predicted confidence distribution and the ground confidence distribution. Besides, we apply the loss function of the mean squared error (MSE) between the expectation [E[&;] of predicted confidence distribution and the ground confidence 8t. Based on these, we formulate the learning objective of CDL.RLL on confidence prediction as minimizing the KI. divergence and MSE together, and define LCP as follows:
Lop= =
ΣΣ a=0
S In
8 8
+
Σ
[E|
8t)2
KL divergence
MSE
where 8, and & are the a-th elements in 8; and & respectively, D denotes the training set, and B E [0, is hyper-parameter that controls the influence of the MSE.
To compute the loss LL on link prediction, we first feed the concatenated embedding of hy, ru and into another two-layer FCN to compute the rank score of the triple (h4, Tú, t.). This FCN outputs single scalar normalized by the Sigmoid function as the rank score, which is computed as:
g Sigmoid (C (h:||r:|((4))
Here, FCN2 is a function that the FCN transforms the concatenated embedding into a scalar. We specifically design margin-based -asse ranking loss function optimize CDL-RL on link prediction, and define LL as follows:
LLP ΣΣ D D
8:(7 + 3 gi]+
where jf is the rank score of a negative sample which is generated by replacing either the head tail entity of the given positive sample with a randomly chosen entity, the number of negative samples for each positive sample is empirically set as 50, D' is the set of all negative samples, [x]+ max[0, æ] denotes the positive part of x, and 7 > 0 a margin hyper-parameter.
To balance LCP and LL in the training process, we use the idea of uncertainty weights M to dynamically adjust the proportion of loss for each task (we only have two tasks: confidence prediction and link prediction) during training. The final loss function of optimizing CDL-RL is defined as:
1 CoP
C(D,0)
LLP + log()Cp ALP)
2X2
2X2
where Ner and AL are observation noise parameters of confidence prediction and link prediction respectively, Φ E [0, 1] (empirically set as 0.1) is a weight limiting the influence of relatively larger number of negative samples, which may cause LLP becoming too large, and 0 represents the parameters of CDL-RL.
Besides the labeled data used for training CDL-RL, we apply PCDG (details will be given in Section 4.3 to generate pseudo labeled data, which are also utilized to reinforce the training of CDL-RL. Since most pseudo labeled data the triples with low confidences, which has almost no impact on minimizing LLP, we only use such pseudo labeled data to minimize LCp to avoid ineffective computations. Thus, we re-define Ler follows:
Lcr =
ΣΣ D
8
8; In
+ 3
Σ
[E[$.]
xt) + ppl
EE [[(S)] D
-z))2) (6)
80
a=
D
6=0
where Dp denotes the set of pseudo labeled data for training CDL-RL, 8, and 8, respectively represent the pseudo confidence distribution and the predicted confidence distribution the j-th pscudo labeled quadruple/ l, generated by PCDG, and wp is the weight of pscudo labeled data.
4.3 Pseudo Confidence Distribution Generator
Since we aim to make full use of unlabeled data to improve the quality of UKG embedding learning in CDL-RL, we propose PCDG, which is used for generating high-quality pseudo confidence distributions for unlabeled data. PCDG based on the idea meta-learning, which solves the gradual shift [15) of traditional self-training. At first, perform negative sampling to get unlabeled
5

---

data (usually low-confidence triples) and feed them to PCDG. Given a positive sample in the labeled data, we generate one negative sample by replacing either the head or tail entity with a randomly chosen entity. Then, PCDG uses the most updated CDL-RRI as the meta learning objective to evaluate the quality of pseudo confidence distributions by checking whether such data generated by PCDG can effectively improve CDL-RL.
In Figure ()), PCDG has the same structure as the CDL-RL, and we denote the parameters of the PCDG as In the training process, PCDG first generates pseudo confidence distributions for unlabeled data, and such pseudo labeled data contains gradient information of PCDG, which enables that it can be optimized. We denote these pseudo labeled data as Dimp. Note that although D Dimp and Dp are both generated by PCDG, they are different and so do their roles. Dimp generated during the phase of optimizing PCDG, while Dp is generated in the phase of training CDL-RL. Then, we minimize the loss on the labeled data (i.e., D) after CDL-RL updates once (i.e., CDL-RRL performs one gradient descent step on both DD and Dmmp). The loss function of PCDG can be expressed as:
((n = (+)
where L(D, ++) (computed by Equation 5 is the loss of CDL.RL on DD with 0+. Here, o+ represents the parameters of the CDL-RL after one gradient update as:
0+ = - a Vo U Demp: 0))
(8)
where a is the learning rate, and L(D Demp: 0) is the loss of CDL-RL on D and Demp with 0.
4.4 Meta Self-training
In this subsection, we introduce the overall meta self- ceff-training process of ssCDL, which iteratively trains CDL-RL and PCDG. In the process of selecting pseudo labeled data from PCDG and inputting them into CDL-RL, we apply a simple yet effective strategy. In the labeled data, the original confidence of each triple should take the lead in the transformed confidence distribution. Thus, the pseudo labeled data, if the highest description degree of pseudo confidence label is larger than fixed threshold, the corresponding pseudo confidence distribution is treated as high-quality and the pseudo labeled data will be selected for training CDL-RL, while others will be removed.
Algorithm gives the details on meta self- self-training of ssCDL. We define two important time points, i.e., the epoch of starting training PCDG T PCDD and the epoch of starting using pseudo labeled data for CDL-RL TCDLRI. We first initialize the parameters and 1) of CDL-RL and PCDG respectively, and take the labeled data D and sampled unlabeled data D,, as the input of ssCDL. When the number of the current epoch Neur is less than TrODG, we only need to optimize CDL-RL with D (line 2-4). This setting tries to make UKG embeddings become stable in this period, which will help us train PCDG soon. When New. is larger than or equal to TPCDG but less than TEDERI it indicates that UKG embeddings have stabilized, and we start to train PCDG. PCDG first generates Demp for unlabeled data, and feeds Drmp and D together into the most updated CDL-RL, so we get the parameters 8+ of CDL-RL updated for one more time (line 6-7). PCDG will utilize A to update itself with D (line 8). In this period, we do not directly use PCDG to generate Dp and feed it to CDL-RL because in the carly stages of PCDG training. the quality of the generated labels cannot be guaranteed. Therefore, the training process of CDL-RL is the same as before (line 9). If Newr is larger than or equal TEDLRL, we will exploit pseudo labeled data D, to help train CDL-RL (line 11-18). We use PCDG to generate Dp and apply the pseudo label selection strategy, and the selected D, will be used together with D to optimize CDL-RL, while the optimization of PCDG is the same as before (line 12-14). Afterwards, PCDG and CDL-RL will continuously repeat the above training process and optimize themselves until the maximum epoch is reached, i.e., PCDG generates pseudo labeled data for CDL-RL, and CDL-RL provides the meta learning objective for PCDG. This iterative training process our complete meta self-training. During training, all parameters including embeddings, are updated using stochastic gradient descent (SGD) in each minibatch.
5 Experiments
In this section, we present experiments to show the effectiveness and superiority of SSCDL on the UKG completion tasks of confidence prediction and link prediction. We also analyze the effects of

---

Algorithm 1: Meta Self-Training of ssCDL
Input:
Labeled data D, unlabeled data Du., the parameters of CDL-RRL 0, the parameters of PCDG 1/, the number of current epoch Neur = 1, the number of maximum epoch T man- the epoch of starting training PCDG TPDDG, the epoch of starting using pseudo labeled data for CDL-RL IEDLRL. the learning rate a.
while Nour S max do Newr < TCDG then + 0 aV (VL((D))
Update CDL-RL with labeled dota.
end
if (TPCDG & Neur < TEDLRL then Demp fn(Du): @t 0 a -VV U Dimps 0);
1 11 a,, ,,,C(D, 0+);
D Meta update PCDG labeled dota.
0 + 0 a (D)
Update CDL-RL with labeled dota.
10
end
if Newr 2 TCDLRI then
12
Dmmp = (D((a):
13
8+ + 8 aVo((D U Dimp. 0);
14
17 - 7, - -V ,V,C(DDe 0t));;
Io Meta update PCDG with labeled data.
Dj (D(D.):
16
Dj Select(Dp).
b Pscudo labeled data selection.
0 0 a (DL(DD
Update with labeled and prendo
18
end
19
N Neur
+ +;
end
CDL and meta self-training in ssCDL with ablation experiments, and further explore the performance of ssCDL on predicting the confidences of low-confidence triples. We also investigate the sensitivity of three key hyper-parametent of ssCDL, which is shown in Appendix C The source code ssCDL is publicly available at: httts:///itthbb wwwp://www.....cccttatoto wttp:i/inwwnnnteteeeeeeeeeeeeeee BatpaiinnnnRRRRSSDDDDS
Datasets. We conducted experiments on two widely used UKG datasets, i.e., NI.27k extracted from NELL and CN15k extracted from ConceptNet (more details are in [T. We followed the setting of UKGE partition the dataset into 85% for training. for validation, and 8% for testing.
Baselines. We compared ssCDL with the state-of-art embedding-based UKG completion methods, including UKGE (3), PASSLEAF (5), UKGsE (26). BEUrRE (2). and UPGAT (20), in confidence prediction and link prediction. More details of such baselines are given in Appendix
5.1 Experimental Setup
Implementation Details. SsCDL was implemented by Pytorch-lightning, and all the experiments were conducted on an RTX3090 GPU card. The optimal hyper-parameters of ssCDL on NL.27k are as follows: the standard deviation of Gaussian distribution a 0.6, the weight of pseudo labeled data
"p = 0.7, the MSE weight 3 = 1, the learning rate a = 0.001, the margin 7 = 0.1, the threshold for pseudo labeled data selection: 0.03, the batch size: 4096, and the embedding size: 128. The optimal hyper-parameters of ssCDL on CN15k are as follows: the standard deviation of Gaussian distribution a 0.6, the weight of pseudo labeled data Wp = 0.3, the MSE weight 3 1, the learning rate a = 0.001, 0.1, the threshold for pseudo labeled data selection: 0.015, the batch size: 4096, and the embedding size: 512. We conducted sensitivity analysis on a,, Wpp and the threshold for
Table 1: The statistics of UKG Datasets.
Dataset
#Entities
#Relations
#Quadruples
NL27k CN15k
27.221 15,000
404
175,412
36
241,158
the margin
7

---

pseudo labeled data selection, and more details are given in Appendix C The experimental results of all baselines refer to the implementation of unKR [22], and more details are given in Appendix [
Evaluation Protocol. We evaluated ssCDL and baselines on the tasks of confidence prediction and link prediction with the following evaluation metrics. For confidence prediction, we selected Mean Squared Error (MSE): the mean squared error of predicted confidences and ground confidences, and Mean Absolute Error (MAE): the mean absolute error of predicted confidences and ground confidences. For link prediction, we chose Hits @ 1: the proportion of ranks equal to one for all tail entities, and Weighted Mean Reciprocal Rank (WMRR): the weighted average multiplicative inverse of the ranks for all tail entities.
5.2 Confidence Prediction
We compared ssCDL with all baselines on CN15K and NL27k in the task of confidence prediction. As shown in Table 2 ssCDL outperforms all baselines in both MSE and MAE. Compared with the best baseline on NL.27,, ssCDL, reduces MSE and MAE by 52.6% and 17.6%, respectively. Compared with the best baseline CN15k, SsCDL reduces MSE and MAE by 63.8% and 53. 33.2%, respectively. It demonstrates that ssCDL can effectively capture the semantics, structure, and confidence information in UKGs, enabling more accurate prediction of triple confidences.
We noticed that all methods' performance on NL27k are better than that on CN15k. This can be attributed to the triple confidences in ConceptNet are determined solely by its data sources and the frequency with which they are mentioned. While different data sources may of different quality, most triples in ConceptNet are generally correct. As result, there is no significant distinction in reliability between high-confidence and low-confidence triples ConceptNet. This situation may cause that the performance of all methods on CN15K is not that good, so it is necessary to build better UKG completion benchmark datasets.
Table 2: The comparison results between ssCDL and baselines NL27k and CN15k for confidence prediction and link prediction. The best results are indicated by bold numbers, while the runner-up results are indicated by underlined numbers (CP: confidence prediction, LP: link prediction).
Dataset
NL27k
CN15k
Task
CP
LP
CP
LP
Metric
MSE
MAE
WMRR
Hits@1 1
MSE
MAE
WMRR
Hits @
UKGEtogi
0.029
0.060
0.593
0.462
0.246
0.409
0.118
0.072
UKGE
0.033
0.071
0.580
0.452
0.202
0.364
0.127
0.060
BEUIRE
0.089
0.222
0.272
0.117
0.117
0.283
0.138
0.039
PASSLEAF Dist
0.023
0.051
0.676
0.553
0.216
0.379
0.170
0.078
PASSLEAF COOSLEACCComLE.. Ex
0.024
0.052
0.708
0.586
0.231
0.400
0.196
0.086
PASSLEAF RotatE
0.019
0.063
0.715
0.580
0.094
0.248
0.137
0.037
UKGSE
0.122
0.271
0.064
0.031
0.103
0.256
0.012
0.002
UPGAT
0.029
0.101
0.658
0.530
0.149
0.308
0.165
0.078
SSCDL
0.009
0.042
0.727
0.636
0.034
0.116
0.207
0.133
5.3 Link Prediction
Table also presents the comparison results on link prediction, and ssCDL achieves the best Hits@) I and WMRR, which demonstrates that ssCDL is capable of predicting more accurate tail entities for different queries. Compared with confidence prediction, SsCDL does not have quite significant improvement on link prediction compared with baselines. This is because our designed CDL and meta self- self-training trainini are used to optimize confidence prediction to get better UKG embedding, and link prediction only benefits from such embeddings besides minimizing the margin-based ranking loss. This relatively implicit optimization may cause the improvement in link prediction is not as significant as that in confidence prediction. However, SsCDI. still outperforms all baselines on link prediction, indicating that the training of ssCDL is effective multi-task learning process.

---

5.4 Ablation Study
To investigate the contributions of CDL and meta self- self-training in ssCDL, we conducted ablation study on both datasets, and the results are given in Table 3
Ablation on Confidence Distribution Learning. To evaluate the effectiveness of our CDL strategy. we did not use confidence distributions of triples, but only utilized their ground confidences for training (denoted as w/o edl). This variant shows significant performance decline on both datasets, which confirms the effectiveness of CDL. This also indicates that confidence distributions enable ssCDL to better utilize the supervision information of different confidences in the training data and reinforce the embedding learning process.
Ablation on Meta Self-Training. To verify the effectiveness of meta self-training, we removed PCDG, and only applied labeled data to train CDL-RL (denoted as w/o mst). This variant also exhibits performance decline, demonstrating that the pseudo labeled data generated by PCDG do assist ssCDL in learning better UKG embeddings. Besides, w/o mst outperforms w/o cdl, illustrating that the supervision information of the labeled data has more important influence on the entire learning process compared to the potential information of the unlabeled data.
Table The ablation study of ssCDL on confidence prediction and link prediction. The best results are highlighted in bold. w/o edl refers to removing confidence distribution learning, and w/o mst refers to removing meta self-training (CP: confidence prediction, L.P: link prediction).
Dataset
NL27k
CN15k
Task
CP
LP
CP
LP
Metric
MSE
MAE
WMRR
Hits
MSE
MAE
WMRR
Hits@ I
w/o edl
0.015
0.057
0.586
0.482
0.044
0.141
0.149
0.090
w/o mst
0,010
0.045
0.718
0.619
0.035
0.118
0.200
0.128
SsCDL
0.009
0.042
0.727
0.636
0.034
0.116
0.207
0.133
5.5
Low-Confidence Triples Analysis
As mentioned before, ssCDL designed to reinforce the UKG embedding learning process under the imbalanced confidence distribution. Since nearly 80% of the triples in CN15k and NL27k have
the confidence higher than 0.5, we con- predicting
ducted the experiments confidences actual low-confidence triples demonstrate that ssCDL can alle- viate the problem of imbalanced confidence distribution in confidence prediction. Since low-confidence triples are less reliable, it is unnecessary perform link prediction. We randomly selected the triples with the confidences lower than 0.5 from the test set of each dataset, and used MAE to evaluate the performance on confidence prediction. The comparison results, as illustrated in Figure 3 reveal that most baselines are not good at handling confidence prediction on low-confidence triples, while ssCDL still demonstrates the robustness and outperforms all baselines on both datasets. This fully reflects the effectiveness of ssCDL in predicting the confidences of low-confidence triples.
0.45
0.45
0.40
URM
0.00
0.35
0.25
HCB
0.25
20.20
0.20
0.19
0.5
0.5
(a)
(b)
Figure The comparison results of confidence predic- tion on low-confidence triples in NL27k and CN15k.
6 Conclusion
In this paper, we propose new semi-supervised confidence distribution learning method ssCDI. for UKG completion. ssCDL is composed of CDL-based relational learner and pscudo confidence

---

distribution generator, which are iteratively trained by meta self- seff-training. Such semi- learning framework and the introduction of triple confidence distributions benefit to solve the problem of extremely imbalanced distributions of triple confidences in UKG embedding learning for UKG completion. Experimental results demonstrate that ssCDL has the best performance on real-world UKG datasets compared with the state-of -f-he-f-tte-rrtt baselines. The effectiveness of different strategies used ssCDL has been verified in the ablation study.
As for the future work, we plan to study rule learning and reasoning on UKGs based on confidence distribution learning. We will also explore apply large language model UKG completion, and use UKGs for reliable retrieval- augmented generation.
7 Acknowledgements
This work is supported by the NSFC (Grant No. 62376058, 52378009, 62576093, U23B2057, 62176185, 62476058), the Southeast University Interdisciplinary Research Program for Young Scholars, and the Big Data Computing Center of Southeast University.
References
[1] Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam Hruschka, and Tom Mitchell. Toward an Architecture for Never-Ending Language Learning. In Proc. of AAAI, volume 24, pages 1306-1313, 2010.
[2] Xuelu Chen, Michael Boratko, Muhao Chen, Shib Sankar Dasgupta, Xiang Lorraine Li, and Andrew McCallum. Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning. In Proc. of NAACL, pages
2021.
[3] Xuelu Chen, Muhao Chen, Weijia Shi, Yizhou Sun, and Carlo Zaniolo. Embedding Uncertain Knowledge Graphs. Proc. of AAAI, volume 33, pages 3363-3370, 2019.
[4] Yilin Chen, Tianxing Wu, Yunchang Liu, Yuxiang Wang, and Guilin Qi. Uncertain Knowledge Graph Completion with Rule Mining. In Proc. of WISA, pages 100-112, 2024.
[5] Zhu-Mu Chen, Mi- Yen Yeh, and Tei-Wei Kuo. PASSLEAF: A Pool-bAsed Semi- Supi-Super LEArning Framework for Uncertain Knowledge Graph Embedding. In Proc. AAAI, volume 35, pages 4019-4026, 2021.
[6] Weizhi Fei, Zihao Wang, Hang Yin, Yang Duan, Hanghang Tong, and Yangqiu Song. Soft reasoning on uncertain knowledge graphs. arXiv preprint (xXi:22403001000 2024.
[7] Sainyam Galhotra and Udayan Khurana. Semantic Search over Structured Data. In Proc. of CIKM, pages 3381 1381-3384, 2020.
[8] Xin Geng. Label Distribution Learning. IEEE Transactions on Knowledge and Data Engineer- ing, 28:1734-1748, 2016.
[9] Xin Geng, Chao Yin, and Zhi-Hua Zhou. Facial Age Estimation by Learning from Label Distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35:2401-2412, 2013.
[10] Lei Ji, Yujing Wang, Botian Shi, Dawei Zhang, Zhongyuan Wang, and Jun Yan. Microsoft concept graph: Mining semantic concepts for short text understanding. Data Intelligence, ((3):238-200 2019.
[11] Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-tas learning using uncertainty weigh losses for scene geometry and semantics. In Proc. of CVPR, pages 7482-7491, 2018.
[12] Natthawut Kertkeidkachom, Xin Liu, and Ryutaro Ichise. GTransE: Generalizing Translation- Based Model on Uncertain Knowledge Graph Embedding. In Proc. of JSAI, pages 170-178, 2020.
[13] Diederik P. Kingma and Jimmy Ba. Adam: method for stochastic optimization. In Proc. of ICLR, 2015.
10

---

[14] Yunshi Lan, Gaole He. Jinhao Jiang, Jing Jiang, Wayne Xin Zhao, and Ji- Ji-Rong Wen. A Survey on Complex Knowledge Base Question Answering: Methods, Challenges and Solutions. In Proc. of IJCAI, pages 4483-4491, 2021.
[15] Shikun Liu, Andrew Davison, and Edward Johns. Self-Supervised Generalisation with Meta Auxiliary Learning. In Proc. of Neurips, 2019.
[16] Sumit Pai and Luca Costabello. Learning embeddings from knowledge graphs with numeric edge attributes. In Proc. of IJCAI, pages 2869- 2875, 2021.
[17] Robyn Speer, Joshua Chin, and Catherine Havasi. ConceptNet 5.5: An Open Multilingual Graph of General Knowledge. In Proc. of AAAI, volume 31, 2017.
[18] Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. RotatE: Knowledge Graph Emhedding by Relational Rotation in Complex Space. In Proc. ICI. 2019.
[19] Théo Trouillon, Johannes Welbi, Sehastian Riedel, éric Gaussier, and Guillaume Bouchard. Complex Embeddings for Simple Link Prediction. In Proc. of ICMI, pages 2071-2080, 2016.
[20] Yen-Ching Tseng, Zu-Mu Chen, Mi-Yen Yeh, and Shou-De Lin. UPGAT: Uncertainty-Awar Pseudo-neigg nheudo-neighbor Augmented Knowledge Graph Attention Network. In Proc. of PAKDD, pages 63-65,22 2023.
[21] Jing Wang. Shuo Zhang, and Runzhi Li. Gate feature interaction network for relation prediction in knowledge graph. Data Intell., 6(3):799-77,, 2024.
[22] Jingting Wang. Tianxing Wu, Shilin Chen, Yunchang Liu, Shutong Zhu, Wei Li, Jingyi Xu, and Guilin Qi. unKR: A Python Library for Uncertain Knowledge Graph Reasoning by Representation Learning. Proc. of SIGIR, pages 2822-2826, 2024.
[23] Jingting Wang, Tianxing Wu, and Jiatao Zhang. Incorporating Uncertainty of Entities and Relations into Few-Shot Uncertain Knowledge Graph Embedding. In Proc. of CCKS, pages 16-28, 2022.
[24] Wentao Wu, Hongsong Li, Haixun Wang, and Kenny Q Zhu. Probase: A Probabilistic Taxonomy for Text Understanding. In Proc. of SIGMOD, pages 481-492, 2012.
[25] Bishan Yang, Scott Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. Embedding entities and relations for learning and inference in knowledge bases. In Proc. of ICLR, 2015.
[26] Shihan Yang. Weiya Zhang, Rui Tang, Mingkai Zhang, and Zhensheng Huang. Approximate in- ferring with confidence predicting based on uncertain knowledge graph embedding. Information Sciences, 609:679-666 6600, 2022.
[27] Mingshan You, Jiao Yin, Hua Wang, Jinli Cao, Kate Wang, Yuan Miao, and Elisa Bertino, A knowledge graph empowered online learning framework for access control decision-making. World Wide Web, 26(2):827-848, 2023.
(28) Jiatao Zhang, Tianxing Wu, and Guilin Qi. Gaussian Metric Learning for Few-Shot Uncertain Knowledge Graph Completion. In Proc. of DASFAA, pages 256-271, 2021.

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The contributions and scope of the paper accurately reflected by the main claims presented the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitation of our work in Appendix D
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
12

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide experimental setup Section 5.1 and Appendix code is accessible.
B and our source
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
13

---

reproducible, they Answer: [Yes]
Justification:
We have uploaded the code on Github.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
/ //nips httpcc/cpplllgggiiieee httgg/dddee//euubbiiiimmmm
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide comprehensive and detailed experimental setup in Section 5.1 and Appendix B
Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Our experiments have demonstrated significant performance improvements that strongly validate our findings. Following other papers in this field, we did not report error bar save the computational cost.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
14

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have provided sufficient information on the computer resources in Section
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported paper (e.g., preliminary or failed experiments that didn't It make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The research conducted in our paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
15

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have cited the original papers of the datasets. We have checked the licenses of used datasets and followed them.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
16

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide code and instructions on how run the code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
17

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
18

---

A Baselines
We compared ssCDL with the state-of-art embedding-basse UKG completion methods published in recent years, including:
UKGE [3] is a classic UKG completion method, which is the first work on relational
learning on UKG and it has two variants: UKGElogs using the logistic function as the mapping function, and UKGErect which takes a bounded rectifier the mapping function.
PASSLEAF [3] improves the generalization ability of UKGE and applies semi cemi-supervised learning for the first time in UKG completion to solve the false negative problem. Now it is the best UKG completion method. PASSLEAF DAsSLEAFDissMtttt
Mist PASSLEAFCommplErr and three variants of PASSLEAF, which use the scoring functions of DistMult [25]. ComplEx 19 and RotatE [8] respectively.
PASSLEAF Rotat Rotatt
UKGsE [66 treats each knowledge fact as short sentence, and is typical model of UKG
completion leveraging pre-trained language model.
BEUrRE 2 representative UKG completion model based on box embedding. The geometry of boxes endows the model with calibrated probabilistic semantics and facilitates the incorporation of relational constraints.
UPGAT [20] generalizes the graph attention network and uses it to capture the local
structural information in UKG completion. It is a typical UKG completion method modeling structural contexts in UKG.
B Implementation Details.
We used Adam optimizer [13] for SGD training. For hyper-parameter tuning, we searched the best hyper- parameter from the following settings: the batch size E [512. 1024, 2048, 4096), the embedding size { (128, 256, 512), the margin 7 E (0.01, 0.05, 0.1, 0.5, 1), and the MSE weight [0.6, 0.8, 1). We conducted detailed sensitivity analysis on the following parameters, including the standard deviation of Gaussian distribution a E (0.02, 0.2, 0.6, 1, 2). the weight of pseudo labeled data wp E (0.1, 0.3, 0.5, 0.7, 0.9), and the threshold for pseudo labeled data selection (0.01, 0.015, 0.02, 0.025, 0.03, 0.035, 0.04).
The search results indicate that the optimal hyper-parameters of ssCDL on N127k are as follows: the standard deviation of Gaussian distribution a 0.6, the weight of pseudo labeled data wp 0.7, the MSE weight 3 = 1, the learning rate = 0.001, the margin = 0.1, the threshold for pseudo labeled data selection: 0.03, the batch size: 4096, and the embedding size: 128. The optimal hyper-parameters of ssCDL on CN15k are as follows: the standard deviation of Gaussian distribution 0.6, the weight of pseudo labeled data upp 0.3, the MSE weight / = 1, the learning rate 0.001, the margin y = 0.1. the threshold for pseudo labeled data selection: 0.015, the batch size: 4096, and the embedding size: 512. We evaluated ssCDL on the validation set every ten epochs. The maximum epochs on NL27k and CN15k were empirically set to 500 and 300, respectively.
C Parameter Sensitivity
We conducted sensitivity analysis three byper-paramettrs, including the standard deviation of Gaussian distribution a,, the weight of pseudo labeled data Wp, and the threshold for pseudo labeled data selection. We utilized MAE and WMRR to evaluate the performance on confidence prediction and link prediction, respectively. Figure shows the impacts of different hyper-parameters of ssCDL for confidence prediction and link prediction on both datasets.
Impact of a. As shown in Figure 4a) and 4dd), d), SsCDL achieves the optimal performance, then the effect deteriorates, as the value of a increases on both datasets. When the value of a is relatively small, the high description degree is mainly distributed in the confidence near the ground confidence, preventing the method from capturing the global features of UKGs, which leads to an inaccurate estimation of the confidence distribution. However, as a increases, the differences of confidence distributions generated by different ground confidences become small, which gradually weakens the ability of distinguishing between different confidences. ssCDL. achieves the best results on both datasets when a 0.6.
19

---

(h354
Ch154
CNI5A N.2
NL2%
4127%
M/55 on MLZ7N
NAL un DriSS
0.350
1004
0.04
6150
0.20
0024
0.331
1004
6130
0004
0130
0.3
Mg
Threshole
(a)
(b)
(c)
The impacts of blyee-parametten on confidence prediction. (a) (b) (c) show the impacts of ar wp. and the threshold for pscudo labeled data selection NL27k and CN15k respectively.
0.240
0.200
70003
0.00
Au00
UMAR 01
0215
0.735
0215
10773
ME on MLZTC
0.15
0.355
0.340
0.5
0.540
0.00
01
11.
05
0.7
0.0
0,500
0.00
DatS
DIZ DizzS DuI Ddas D01
0.543
Thredate
(d)
(c)
(f)
The impacts of hhper-parametee on link prediction. (d) (e) (f) show the impacts of a, Wpp and the threshold for pseudo labeled data selection on NI.27k and CN15k respectively.
Figure 4: Sensitivity experiments on NL27k and CN15k.
Impact of Wip. For the hyper-parameter Wps Figure bb) and 4e) show that the performance of ssCDL improves first and then decreases with the growing of Wps which indicates that reasonable balance between labeled data and pseudo labeled data necessary to achieve the optimal performance of ssCDL. Our experimental findings indicate that 0.7 is the optimal Wp NL27k and 0.3 is the optimal wp on CN15k.
Impact of the threshold for pseudo labeled data selection. As shown in Figure c) and (().. with the increase of the threshold, the performance of ssCDL also exhibits a trend of first increasing and then decreasing. A quite low threshold will cause that the pseudo label selection strategy to select noisy data, while a quite high threshold will limit the use of high-quality pseudo labeled data. ssCDL achieves the best results when the threshold is set as 0.03 on NL27k and 0.015 on CN15k, respectively.
D Limitation
Our current study has conducted experiments CN15k. Although CN15k is classic benchmark for UKG completion, since the most triples in ConceptNet are generally correct, there is no significant distinction in reliability between nigh-confidence and low-confidence triples. We will build better UKG completion benchmark datasets in the future.
20


Reviews:
[{"review_id": "QCPWbgGpkK", "reviewer": "Reviewer_SkHP", "summary": "Uncertain knowledge graph(UKG) represents the uncertainty of knowledge, and uncertain knowledge graph completion is the task of reasoning over UKG. This paper notices the fact of extreme imbalance of triple-confidence scores(most stored facts have very high confidence, while low-confidence facts are rarely kept) in KGs like Nell. Then this paper propose ssCDL (Semi-supervised Confidence-Distribution Learning), a two-component framework: 1.CDL-RL converts confidence into a confidence distribution; 2.PCDG that assigns pseudo confidence distributions to negative-sampled triples. Empirical results the effectiveness of this method.", "questions": "See weekness.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1. The finding of the imbalance of triple-confidence scores and the design of ssCDL with its two component is novel. The motivation is reasonable and the methodology makes sense.\n\n2. The semi-supervised meta self-training is described clearly and succeeded in using unlabeled data to rebalance the data. This part of paper is presented clearly and easy to follow.\n\n3. The experiment shows SOTA result, with considerable improvement against the baseline. \n\n\nWeakness:\n\n1. Only two medium-sized datasets are included in the experiment, I wonder whether it can include especially web-size UKGs.\n\n2. The computation of $\\hat{s}_i$ and $\\hat{g}_i$ should use equation environment.", "was_revised": true, "final_justification": "I am basically content with the author response. Though the paper can be further improved in aspects like complexity discussion and writing clarity, it's generally ok and I will maintain my positive rating.", "rebuttal": "We thank the reviewer for the valuable comments, and will take them into account to improve our paper. Our responses are as follows:\n\n**W1:** \nWe chose NL27k and CN15k because they are widely used benchmarks in previous studies on UKG completion, such as UKGE [1], PASSLEAF [2], and BEUrRE [3]. We agree that testing on web-scale UKGs would further validate the robustness of our method. However, to the best of our knowledge, no publicly available web-scale UKG benchmarks currently exists. A benchmark should support fair, repeatable, and meaningful comparison, which requires careful construction, annotation, and validation. Creating such a benchmark is a time-consuming process, typically involving manual or semi-automated labeling. Thus, it is hard for us to build a web-scale UKG benchmark and conduct comparison experiments in a short period of time, but we recognize its importance for better evaluation in the research of UKG completion. In our future work, we plan to build such a web-size UKG benchmark to support more comprehensive evaluations for UKG completion.\n\n\n**W2:**\nWe thank the reviewer for pointing out this issue, and will revise those equations to use proper equation environments in the final version.\n\n\n[1] Xuelu Chen, Muhao Chen, Weijia Shi, Yizhou Sun, and Carlo Zaniolo. Embedding Uncertain Knowledge Graphs. In Proc. of AAAI, volume 33, pages 3363–3370, 2019.\n\n[2] Zhu-Mu Chen, Mi-Yen Yeh, and Tei-Wei Kuo. PASSLEAF: A Pool-bAsed Semi-Supervised LEArning Framework for Uncertain Knowledge Graph Embedding. In Proc. of AAAI, volume 35, pages 4019–4026, 2021.\n\n[3] Xuelu Chen, Michael Boratko, Muhao Chen, Shib Sankar Dasgupta, Xiang Lorraine Li, and  Andrew McCallum. Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning.  In Proc. of NAACL, pages 882–893, 2021."}, {"review_id": "nGMFy8HAqd", "reviewer": "Reviewer_kGBE", "summary": "ssCDL addresses confidence imbalance in Uncertain Knowledge Graphs by transforming triple confidences into distributions, introducing semi-supervised learning with pseudo-labels to augment training data. It combines relational embedding learning with a meta-learning-based pseudo-confidence generator to iteratively refine embeddings using both labeled and unlabeled triples. Experiments claim consistent SOTA performance on UKG benchmarks, though technical novelty and cost-benefit trade-offs require scrutiny.", "questions": "Please refer to the Weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The paper identifies confidence distribution skew as a legitimate challenge in UKG embedding learning, an underexplored issue in prior work. \n\n2. The core idea of representing confidences as continuous distributions rather than discrete values introduces useful regularization that may enhance model robustness. \n\n3. The semi-supervised framework leverages unlabeled data resourcefully through pseudo-labeling, offering a promising approach to data augmentation.\n\nDrawbacks:\n\n1. The claim that neighboring confidences share transferable features lacks empirical validation. Confidence scores typically reflect extraction certainty or source reliability—not semantic relatedness—making spatial correlation between confidence values conceptually questionable.\n\n2. Performance gains over baselines are minimal yet incur substantial overhead. Iterative pseudo-label refinement requires multiple training passes. And meta-learning introduces additional optimization loops.\n\n3. The strong performance of ssCDL w/o CDL suggests distributional confidence encoding—not the full architecture—drives improvements. The value of meta-learned pseudo-labels remains unproven, while a deeper analysis of distributional confidence encoding would be more insightful.", "was_revised": true, "final_justification": "The paper addresses the novel and underexplored problem of confidence distribution skew through a promising approach using continuous distributions. The author's response has addressed my concerns. While performance gains are marginal given the additional overhead, the cost remains acceptable in practical offline applications.", "rebuttal": "We thank the reviewer for the valuable comments, and will take them into account to improve our paper. Our responses are as follows:\n\n**W1:** \nAs the reviewer pointed out, confidence can reflect extraction certainty or source reliability, but according to the definition [1] used in UKGs, confidence actually represents the likelihood of the relation fact to be true. Thus, similar confidences suggest that the feature representations on modeling the likelihood for triples to be true should also be similar. This is why we argue that neighboring confidences share transferable features. This idea is inspired by the use of label distribution learning in facial age estimation. Geng et al. [2] transformed each face image into an example associated with a label distribution consisting of age labels, with the assumption that the facial appearances of a person at close ages (e.g., 35, 36, and 37) tend to be similar. Similarly, in UKGs, triples with close confidences should exhibit similar features that encode this “truthfulness”. Conversely, if the confidences of two triples differ significantly, it suggests a difference in the features that support their likelihood of being true. This is why we transform confidence into confidence distribution and introduce confidence distribution learning, and its effectiveness is demonstrated by our ablation study in Sec. 5.4. \n\n\n**W2:** \nActually, according to the previous studies on UKG completion, our method achieves notable improvements over the state-of-the-art baselines. For instance, on NL27k, PASSLEAF$\\_{DistMult}$ [3] reduces the MSE and MAE of UKGE$\\_{logi}$ [1] by 20.7% and 15.0%, respectively, while our method achieves reductions of 52.6% in MSE and 17.6% in MAE compared to the best-performing baseline. These results demonstrate that the performance improvement is not marginal in the field of UKG completion. The method we propose indeed introduces additional computational cost due to its complexity. However, since it is designed for offline rather than online UKG completion, we think this additional cost remains acceptable in practical applications. In this work, efficiency is not the primary focus, but we acknowledge its significance and we plan to explore optimizations on efficiency in real-world large-scale UKG completion for our future work.\n\n\n**W3:** \nWe have conducted ablation study by removing meta self-training (denoted as w/o mst), as shown in Sec 5.4. The results indicate that the pseudo labeled data generated by pseudo confidence distribution generator (PCDG) assist ssCDL for better UKG completion. However, the influence is relatively smaller compared to that of removing confidence distribution learning (CDL). This is because our PCDG is primarily designed to generate pseudo confidence distributions for unlabeled data obtained through negative sampling. Since such data are often associated with low confidences, our PCDG focuses on enhancing the performance of our method on low-confidence triples in the tasks of confidence prediction and link prediction. However, the test sets contain relatively few low-confidence triples, which limits the potential impact of PCDG in this setting. In contrast, CDL enhances the method’s ability to handle both high-confidence and low-confidence triples, thereby providing more effective performance improvements. \n\n\n[1] Xuelu Chen, Muhao Chen, Weijia Shi, Yizhou Sun, and Carlo Zaniolo. Embedding Uncertain Knowledge Graphs. In Proc. of AAAI, volume 33, pages 3363–3370, 2019.\n\n[2] Xin Geng, Chao Yin, and Zhi-Hua Zhou. Facial Age Estimation by Learning from Label Distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35:2401–2412, 2013.\n\n[3] Zhu-Mu Chen, Mi-Yen Yeh, and Tei-Wei Kuo. PASSLEAF: A Pool-bAsed Semi-Supervised LEArning Framework for Uncertain Knowledge Graph Embedding. In Proc. of AAAI, volume 35, pages 4019–4026, 2021."}, {"review_id": "j5XZCyrfjs", "reviewer": "Reviewer_hnMb", "summary": "The paper addresses the challenge of imbalanced confidence distributions in Uncertain Knowledge Graphs (UKGs), where high-confidence triples dominate, leading to suboptimal embeddings for UKG completion. The authors propose ssCDL, a semi-supervised method combining Confidence Distribution Learning (CDL) and Meta Self-Training. Experiments on NL27k and CN15k show ssCDL outperforms SOTA baselines in both confidence prediction and link prediction tasks. Ablation studies further confirm the effectiveness of the proposed model.", "questions": "Please refer to \"weakness\".", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n• Quality: The model design is technically sound. CDL effectively captures uncertainty via confidence distributions, and meta self-training robustly handles pseudo-labels. Also, experiments conducted on confidence prediction and link prediction prove the effectiveness of the model.\n• Clarity: The paper is well written. The structure is clear and easy to follow.\n• Significance: This paper addresses a meaningful problem, confidence imbalance, in UKG completion, improving robustness for low-confidence triples.\n• Originality: The paper based on novel integration of confidence distributions and meta self-training for UKGs.\n\nWeakness:\n• Quality: The evaluation metrics are relatively limited. For example, link prediction only uses Hits@1 and WMRR. It is recommended to include commonly used metrics such as Hits@3 and Hits@10.\n• Significance: The proposed model still relies on initial training with existing high-confidence samples. It would be valuable to further explore its performance under zero-shot or few-shot supervision settings.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable comments, and will take them into account to improve our paper. Our responses are as follows:\n\n**W1:** \nIn this paper, we only use Hits@1 and WMRR as our evaluation metrics for link prediction. Hits@k focuses on whether the correct entity is among the top k predicted entities, and Hits@1 is the most strict evaluation metric among all Hits@k metrics, as it measures whether the predicted entity reaches the highest rank. Therefore, in real-world applications, Hits@1 is a more practical metric compared to Hits@k with larger k values. WMRR means the weighted average multiplicative inverse of the ranks for all correct tail entities, and it is a more comprehensive metric that focuses more on the impact of entities ranked higher on link prediction. These two metrics measure different aspects, and are complementary and representative. In fact, we have also evaluated our method and baselines using other Hits@k metrics, and we present the results on Hits@3 and Hits@5 in Table 1, which also demonstrates that our method outperforms all baselines. This fully validates the effectiveness of our method, and we will include additional Hits@k results in the final version if space allows.\n\nTable 1. The comparison results between ssCDL and baselines on NL27k and CN15k for link prediction, evaluated by Hits@3 and Hits@5.\n| Dataset           | NL27k     |              | CN15k          |              |\n|-------------------|-----------|--------------|----------------|--------------|\n| Metric                  | Hits@3    | Hits@5       | Hits@3         | Hits@5       |\n| UKGE$_{logi}$     | 0.597     | 0.651        | 0.146          | 0.182        |\n| UKGE$_{rect}$     | 0.595     | 0.655        | 0.132          | 0.172        |\n| BEUrRE            | 0.307     | 0.366        | 0.175          | 0.223        |\n| PASSLEAF$_{DistMult}$ | 0.676     | 0.724        | 0.178          | 0.228        |\n| PASSLEAF$_{ComplEx}$  | 0.703     | 0.746        | 0.223          | 0.268        |\n| PASSLEAF$_{RotatE}$   | 0.706     | 0.737        | 0.167          | 0.223        |\n| UKGsE             | 0.062     | 0.080        | 0.006          | 0.010        |\n| UPGAT             | 0.654     | 0.702        | 0.168          | 0.215        |\n| **ssCDL**         | **0.760** | **0.809**    | **0.246**      | **0.275**    |\n\n**W2:** \nOur work focuses on UKG completion, a scenario where exists a large amount of high-confidence labeled data, i.e., existing triples with high confidences. We believe that leveraging these labeled data will lead to better performance. It is true that there exists few-shot or zero-shot scenarios at certain confidences. In fact, both the introduced confidence distribution learning and pseudo confidence distribution generator can address the few-shot and zero-shot issues to some extent. Moreover, few-shot or zero-shot settings are more common in another relevant task, i.e., computing triple confidences for deterministic KGs, and we also plan to explore few-shot or zero-shot strategies in this task for our future work."}, {"review_id": "BFe9gwuC76", "reviewer": "Reviewer_FH1k", "summary": "This paper proposes to use semi-supervised confidence distribution learning for uncertain knowledge graph completion, which tries to alleviate the problem of extremely imbalanced distributions of triple confidences. The proposed method ssCDL has a very interesting idea that transforms a triple confidence to a confidence distribution to introduce more supervision information of diverse confidences. Besides, meta self-training is applied to generate reliable confidences for unlabeled data so that reinforcement with unlabeled data could be achieved for UKG completion. Experiments show that ssCDL outperforms the state-of-the-art baselines in both tasks of link prediction and confidence prediction. The ablation study and low-confidence triple analysis also provide evidences to demonstrate the effectiveness of specialized design of ssCDL.", "questions": "See weaknesses.", "limitations": "yes", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n1.\tThe paper is the first work to propose the problem of imbalanced distributions of triple confidences in UKG completion. It presents a novel semi-supervised method, ssCDL, which integrates meta self-training and confidence distribution learning to obtain high-quality embeddings for UKG completion under the imbalanced confidence distribution in UKGs.\n\n2.\tThis paper extends the idea of label distribution learning to confidence distribution learning for UKG embedding learning. Such an idea skillfully leverages confidence distribution to reinforce the learning process with existing labeled data.\n\n3.\tThis paper presents a comprehensive set of experiments, including comparison with baselines, ablation studies, analysis of low-confidence triples, and parameter sensitivity analysis, which collectively provide strong evidences for the effectiveness and superiority of the proposed ssCDL.\n\n4.\tThis paper is clearly structured and well-written, with a logical flow that makes it easy for readers to follow the motivation, methodology, and experimental results.\n\n5.\tThis paper has released the source code and provided a corresponding README file for guidance, ensuring the reusability of the proposed method.\n\nWeaknesses:\n1. The differences of the datasets used in experiments are unclear. I understand that they are widely used benchmarks, but we need more details to show the experiments on such datasets are rational, and the results do make sense.\n\n2. In ablation study, with the designed meta self-training strategy, the improvement of the performance in both tasks on both datasets is not that significant. Could you please explain more on this point?", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the valuable comments, and will take them into account to improve our paper. Our responses are as follows:\n\n**W1:**\nThe benchmarks CN15k and NL27k used in our experiments exhibit distinct characteristics in terms of relation diversity, graph density, and confidence distribution. Specifically, CN15k contains only 36 relations, while NL27k includes 404 relations, presenting a much broader relational space. This difference enables us to evaluate whether the proposed method can generalize across knowledge graphs with both low and high relational diversity. In terms of graph density, CN15k includes 241,158 quadruples over 15,000 entities, whereas NL27k has 175,412 quadruples over 27,221 entities. These differences indicate that CN15k is relatively denser for each entity, while NL27k is much sparser. Such structural diversity allows us to examine the performance of our method under both dense and sparse conditions. For confidence distribution, both datasets are dominated by high-confidence triples, but they differ in terms of the average confidence and the concentration range of confidences. Specifically, the average confidence of CN15k is 0.629, with most confidences concentrated in the range of [0.7, 0.8]. NL27k has a higher average confidence of 0.797, with most confidences falling within [0.9, 1.0]. Both datasets thus provide varied confidence characteristics for evaluation. Therefore, using both datasets enables a more comprehensive evaluation of the proposed method.\n\n**W2:** \nAs the reviewer pointed out, the impact of meta self-training is not that significant, as shown in Sec. 5.4. In meta self-training, we design pseudo confidence distribution generator to generate pseudo confidence labels for unlabeled data and feed the pseudo labeled data to confidence distribution learning (CDL) based relational learner. Since such unlabeled data (generated by negative sampling) are often associated with low confidences, meta self-training mainly improves the performance of our method on low-confidence triples. However, due to the test sets contain relatively few low-confidence triples, the improvement of meta self-training is not that significant compared with CDL under this dataset setting."}]

Author rebuttal:

Meta-review: The paper first points out the extremely imbalanced distributions of triple confidences in the context of uncertain knowledge graph learning. Then, it proposes a semi-supervised confidence distribution learning method for UKG completion.
Reviewers agree that the authors' viewpoint itself is novel and the proposed method successfully obtain high-quality embeddings for UKG completion under the imbalanced confidence distribution in UKGs.
One concern is the effectiveness of meta-learned pseudo-labels. However, it would be a minor concern.
The area chair nominates the paper for a spotlight since the paper's viewpoint (imbalanced distributions of triple confidences) itself is novel so it has the potential to lead many followers.
Decision: Accept (spotlight)