Title: Uncertain Knowledge Graph Completion via Semi-Supervised Confidence Distribution Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Uncertain Knowledge Graph Completion

Confidence Distribution Learning

Meta Self-Training
Abstract: Uncertain knowledge graphs (UKGs) associate each triple with a confidence score to provide more precise knowledge representations. Recently, since real-world UKGs suffer from the incompleteness, uncertain knowledge graph (UKG) completion attracts more attention, aiming to complete missing triples and confidences. Current studies attempt to learn UKG embeddings to solve this problem, but they neglect the extremely imbalanced distributions of triple confidences. This causes that the learnt embeddings are insufficient to high-quality UKG completion. Thus, in this paper, to address the above issue, we propose a new semi-supervised Confidence Distribution Learning (ssCDL) method for UKG completion, where each triple confidence is transformed into a confidence distribution to introduce more supervision information of different confidences to reinforce the embedding learning process. ssCDL iteratively learns UKG embedding by relational learning on labeled data (i.e., existing triples with confidences) and unlabeled data with pseudo labels (i.e., unseen triples with the generated confidences), which are predicted by meta-learning to augment the training data and rebalance the distribution of triple confidences. Experiments on two UKG datasets demonstrate that ssCDL consistently outperforms the state-of-the-art baselines in different evaluation metrics.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes to use semi-supervised confidence distribution learning for uncertain knowledge graph completion, which tries to alleviate the problem of extremely imbalanced distributions of triple confidences. The proposed method ssCDL has a very interesting idea that transforms a triple confidence to a confidence distribution to introduce more supervision information of diverse confidences. Besides, meta self-training is applied to generate reliable confidences for unlabeled data so that reinforcement with unlabeled data could be achieved for UKG completion. Experiments show that ssCDL outperforms the state-of-the-art baselines in both tasks of link prediction and confidence prediction. The ablation study and low-confidence triple analysis also provide evidences to demonstrate the effectiveness of specialized design of ssCDL.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1.\tThe paper is the first work to propose the problem of imbalanced distributions of triple confidences in UKG completion. It presents a novel semi-supervised method, ssCDL, which integrates meta self-training and confidence distribution learning to obtain high-quality embeddings for UKG completion under the imbalanced confidence distribution in UKGs.\n\n2.\tThis paper extends the idea of label distribution learning to confidence distribution learning for UKG embedding learning. Such an idea skillfully leverages confidence distribution to reinforce the learning process with existing labeled data.\n\n3.\tThis paper presents a comprehensive set of experiments, including comparison with baselines, ablation studies, analysis of low-confidence triples, and parameter sensitivity analysis, which collectively provide strong evidences for the effectiveness and superiority of the proposed ssCDL.\n\n4.\tThis paper is clearly structured and well-written, with a logical flow that makes it easy for readers to follow the motivation, methodology, and experimental results.\n\n5.\tThis paper has released the source code and provided a corresponding README file for guidance, ensuring the reusability of the proposed method.\n\nWeaknesses:\n1. The differences of the datasets used in experiments are unclear. I understand that they are widely used benchmarks, but we need more details to show the experiments on such datasets are rational, and the results do make sense.\n\n2. In ablation study, with the designed meta self-training strategy, the improvement of the performance in both tasks on both datasets is not that significant. Could you please explain more on this point?\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-01"}

comments: SUMMARY:
The paper addresses the challenge of imbalanced confidence distributions in Uncertain Knowledge Graphs (UKGs), where high-confidence triples dominate, leading to suboptimal embeddings for UKG completion. The authors propose ssCDL, a semi-supervised method combining Confidence Distribution Learning (CDL) and Meta Self-Training. Experiments on NL27k and CN15k show ssCDL outperforms SOTA baselines in both confidence prediction and link prediction tasks. Ablation studies further confirm the effectiveness of the proposed model.

STRENGTHS AND WEAKNESSES:
Strengths:
• Quality: The model design is technically sound. CDL effectively captures uncertainty via confidence distributions, and meta self-training robustly handles pseudo-labels. Also, experiments conducted on confidence prediction and link prediction prove the effectiveness of the model.
• Clarity: The paper is well written. The structure is clear and easy to follow.
• Significance: This paper addresses a meaningful problem, confidence imbalance, in UKG completion, improving robustness for low-confidence triples.
• Originality: The paper based on novel integration of confidence distributions and meta self-training for UKGs.

Weakness:
• Quality: The evaluation metrics are relatively limited. For example, link prediction only uses Hits@1 and WMRR. It is recommended to include commonly used metrics such as Hits@3 and Hits@10.
• Significance: The proposed model still relies on initial training with existing high-confidence samples. It would be valuable to further explore its performance under zero-shot or few-shot supervision settings.

QUESTIONS:
Please refer to "weakness".

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-07-01

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nssCDL addresses confidence imbalance in Uncertain Knowledge Graphs by transforming triple confidences into distributions, introducing semi-supervised learning with pseudo-labels to augment training data. It combines relational embedding learning with a meta-learning-based pseudo-confidence generator to iteratively refine embeddings using both labeled and unlabeled triples. Experiments claim consistent SOTA performance on UKG benchmarks, though technical novelty and cost-benefit trade-offs require scrutiny.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper identifies confidence distribution skew as a legitimate challenge in UKG embedding learning, an underexplored issue in prior work. \n\n2. The core idea of representing confidences as continuous distributions rather than discrete values introduces useful regularization that may enhance model robustness. \n\n3. The semi-supervised framework leverages unlabeled data resourcefully through pseudo-labeling, offering a promising approach to data augmentation.\n\nDrawbacks:\n\n1. The claim that neighboring confidences share transferable features lacks empirical validation. Confidence scores typically reflect extraction certainty or source reliability—not semantic relatedness—making spatial correlation between confidence values conceptually questionable.\n\n2. Performance gains over baselines are minimal yet incur substantial overhead. Iterative pseudo-label refinement requires multiple training passes. And meta-learning introduces additional optimization loops.\n\n3. The strong performance of ssCDL w/o CDL suggests distributional confidence encoding—not the full architecture—drives improvements. The value of meta-learned pseudo-labels remains unproven, while a deeper analysis of distributional confidence encoding would be more insightful.\n\nQUESTIONS:\nPlease refer to the Weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
Uncertain knowledge graph(UKG) represents the uncertainty of knowledge, and uncertain knowledge graph completion is the task of reasoning over UKG. This paper notices the fact of extreme imbalance of triple-confidence scores(most stored facts have very high confidence, while low-confidence facts are rarely kept) in KGs like Nell. Then this paper propose ssCDL (Semi-supervised Confidence-Distribution Learning), a two-component framework: 1.CDL-RL converts confidence into a confidence distribution; 2.PCDG that assigns pseudo confidence distributions to negative-sampled triples. Empirical results the effectiveness of this method.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The finding of the imbalance of triple-confidence scores and the design of ssCDL with its two component is novel. The motivation is reasonable and the methodology makes sense.

2. The semi-supervised meta self-training is described clearly and succeeded in using unlabeled data to rebalance the data. This part of paper is presented clearly and easy to follow.

3. The experiment shows SOTA result, with considerable improvement against the baseline. 


Weakness:

1. Only two medium-sized datasets are included in the experiment, I wonder whether it can include especially web-size UKGs.

2. The computation of $\hat{s}_i$ and $\hat{g}_i$ should use equation environment.

QUESTIONS:
See weekness.

LIMITATIONS:
Yes.
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-07-06