Title: Personalized Federated Conformal Prediction with Localization
Venue: NeurIPS 2025
Accepted: yes
Keywords: Personalized federated learning; Conformal prediction; Localization; Conditional coverage
Abstract: Personalized federated learning addresses data heterogeneity across distributed agents but lacks uncertainty quantification that is both agent-specific and instance-specific, which is a critical requirement for risk-sensitive applications. We propose personalized federated conformal prediction (PFCP), a novel framework that combines personalized federated learning with conformal prediction to provide statistically valid agent-personalized prediction sets with instance-localization. By leveraging privacy-preserving knowledge transfer from other source agents, PFCP ensures marginal coverage guarantees for target agents while significantly improving conditional coverage performance on individual test instances, which has been validated by extensive experiments.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper considers the setup of federated learning, where the data comes from multiple agents, and the goal is to quantify the uncertainty of a test agent. The challenge is that the source and target agents have different distributions. This work proposes “personalized federated conformal prediction”, a method that constructs prediction sets that contain the responses of the test agent at a user-specified probability. This method builds on conformal prediction and localized conformal prediction. The theoretical analysis indicates that the proposed aggregation of conditional distribution estimators should attain better conditional coverage compared to the baseline Generalized Localized Conformal Prediction. Experiments on synthetic and real datasets reveal that the proposed PFCP performs better than prior work.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n1.\tThe proposed PFCP attains better performance compared to the baseline GLCP and prior work.\n2.\tThis paper provides a theoretical analysis for the worst-case conditional coverage based on the accuracy of the density estimation. While the assumptions are complex, the authors provide an interpretation for each assumption.\n3.\tThe PFCP and GLCP methods have a valid marginal coverage guarantee, by building on conformal prediction.\n4.\tThe paper is written clearly, presented well, and easy to follow.\n\n\n## Weaknesses\n1.\tThe proposed GLCP is very similar to the localized conformal prediction [1] and to distributional conformal prediction [2]. In contrast to the statement in line, 67, in my opinion, this extension is marginal and not novel.\n2.\tThe experiments do not include the test-conditional miscoverage error of FedCP and FedCP-QQ.\n3.\tThe experiments in [3] show that FedCP-QQ achieves a marginal coverage rate that is very close to $1-\\alpha=90\\%$ on the same real datasets. In contrast, the experiments in this paper show that the coverage rate attained by this method is far from the nominal level. This discrepancy between the two results is not addressed by the authors.\n4.\tThe real-dataset experiments do not include a dataset that is suitable for federated learning setup, such as MedMNIST used in [4].\n\n\n\n## References\n[1] Leying Guan. Localized conformal prediction: A generalized inference framework for conformal prediction. Biometrika, 110(1):33–50, 2023\n\n[2] Victor Chernozhukov, Kaspar Wüthrich, and Yinchu Zhu. Distributional conformal prediction. In Proceedings of the National Academy of Sciences, volume 118. National Acad Sciences, 2021.\n\n[3] Pierre Humbert, Batiste Le Bars, Aurélien Bellet, and Sylvain Arlot. One-shot federated conformal prediction. In International Conference on Machine Learning, pages 14153–14177. PMLR, 2023\n\n[4] Charles Lu and Jayasheree Kalpathy-Cramer. Distribution-Free Federated Learning with Conformal Predictions, 2022.\n\nQUESTIONS:\n## Questions\n1.\tIn line 200, what does “valid” estimator mean?\n2.\tThe informativeness of the bounds in Lemma 2.6 and Theorem 2.8 is not clear. Is it possible that the bounds might get trivial in practice?\n3.\tIn line 249, $\\tilde{\\delta}$ is defined as an integral over $x$ or $s$?\n4.\tIn line 266, over what the randomness is taken? Random data splits?\n5.\tHow is the test-conditional miscoverage computed in the experiments?\n\n## Minor comments\n1.\tThe proof for Lemma 2.1 follows directly from the theory of conformal prediction, as in [1], but it is not stated in the paper. Since there is no proof for this Lemma, it should be at least noted that it follows from general results on conformal inference.  \n2.\tThe weight $\\pi$ is defined in line 111, but it is only used in line 147. It should be defined close to its first use for clarity.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-15"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a conformal prediction method to quantify the uncertainty of algorithms in a personalized Federated Learning (FL) setting. An important distinction from prior works is that each agent has its own prediction function, a characteristic of the personalized FL setting. The authors claim to propose a method that 1) constructs conformal prediction sets for a target agent using data from all agents in a privacy-preserving manner 2) ensures marginal coverage and asymptotic conditional coverage. Finally, they show the efficiency of their method through various experiments.\n\nSTRENGTHS AND WEAKNESSES:\n------- Strengths -------\n\n1. The paper is well-written, and the subject is of significant interest to the conformal prediction community. \n\n2. The method is clearly explained. \n\n3. The related works section is thoroughly detailed.\n\n\n------- Weaknesses -------\n\n1. The article is really just an implementation of a new score function adapted to the federated learning framework.\n\n2. The bound in Theorem 2.4 only depends on $n$, the number of points for the $K+1$-th agent. In contrast, in Theorem 2.2 of [1] for instance, the bound depends on $N$ the total number of points (i.e. the summation of the number of points of each agent).\n\n3. Overall, it seems to me that the main contribution of this article is more about how to build CDF estimators in FL than how to perform conformal prediction in a personalized federated learning context (in a sens that the conformal part used only standard results).\n\n4. (minor) The size of returned sets must always be indicated. For instance, this is not the case in Figure 1.\n\n[1] Conformal Prediction for Federated Uncertainty Quantification Under Label Shift, Plassier et. al\n\nQUESTIONS:\n1\\ As already said, Theorem 2.4 only depends on $n$ and not on all the data points. Hence, how do you theoretically know that your method uses all the points for the coverage? \n\n2\\ Since the article's contribution is mainly a new score function, what are the particularities of the techniques used for the proofs compared to other CP articles that don't deal with federated learning?\n\n3\\ It is say that the method is private but the privacy is never quantify. How do you know that your method is private? And are you able to give theoretical results about that?\n\n4\\ In line 186, it is said that « we obtain superior estimates that simultaneously improve conditional coverage accuracy and yield tighter prediction sets. » How do you know that since no theoretical results are given about the size of the sets ?\n\n5\\ In experiments, which score function is used for FedCP and FedCPQQ ?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 1, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose PFCP (Personalized Federated Conformal Prediction), which aims to combine personalized federated learning with conformal prediction. This can provide uncertainty sets that are both agent-specific and instance-specific. The authors also introduce GLCP (Generalized Localized Conformal Prediction) an extension of personalized federated learning that can accommodate any conditional distribution estimator. The proposed method PFCP shows how to leverage data from source agents to improve prediction sets --compared to GLCP which can be excessively large-- for a target agent while maintaining privacy and marginal coverage guarantees.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1.  Combining both personalized federated learning and conformal prediction addresses an important field.\n2.  The paper provides formal marginal coverage guarantees for their method.\n3. The paper is well written and easy to follow.\n\nWeaknesses:\n1. The authors mention that the works of Plassier et al. [27, 28] are the most closely related works. However, they do not compare against these important baslines. Could the authors highlight the key differences of why they are not in the comparison table? Otherwise, could they add the results of these methods in Table 1?\n2. Limited discussion of when the method might fail. Could the authors expand on this?\n3. I am not sure if \"personalized\" federated learning is the proper term to be used in this context. This method only improves the prediction sets for the target agent satisfying Assumption 2. Could the authors reflect on this?\n\n[27] Vincent Plassier, Alexander Fishkov, Maxim Panov, and Eric Moulines. Conditionally valid probabilistic conformal prediction. stat, 1050:1, 2024.\n[28] Vincent Plassier, Nikita Kotelevskii, Aleksandr Rubashevskii, Fedor Noskov, Maksim Velikanov, Alexander Fishkov, Samuel Horvath, Martin Takac, Eric Moulines, and Maxim Panov. Efficient conformal prediction under data heterogeneity. In International Conference on Artificial Intelligence and Statistics, pages 4879–4887. PMLR, 2024.\n\nQUESTIONS:\nPlease address Weaknesses W1, W2, and W3.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nAuthors propose a personalization framework for  (conditional) conformal prediction in federated settings using engression and federation of a learnable conditional density estimator.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n* The personalization problem of federated conformal prediction is novel and interesting. \n* Leveraging a federated conditional distribution estimator to leverage PFCP is novel. \n\n**Weaknesses**\n* Evaluation with unknown datasets compared to the baselines[1].\n* Some missing baselines [2,3]\n* GLCP is not clear. Is it also federated, or is there no communication between the clients in this case?\n* As long as coverages are greater then 1 - $\\alpha$, the coverage results are fine. However, the effect of personalization over the set size as opposed to [1] is not clear to me.\n* The personalization part is not clear throughout the steps in lines 172-180. Is personalization done over the aggregation level , or not sharing some specific model?\n* Notation is overall hard to follow as opposed to previous works in federated CP. \n\n[1]  Charles Lu, Yaodong Yu, Sai Praneeth Karimireddy, Michael Jordan, and Ramesh Raskar. Federated conformal predictors for distributed uncertainty quantification. In International Conference on Machine Learning, pages 22942–22964. PMLR, 2023.\n\n[2]Vincent Plassier, Alexander Fishkov, Maxim Panov, and Eric Moulines. Conditionally valid probabilistic conformal prediction. stat, 1050:1, 2024.\n\n[3] Vincent Plassier, Nikita Kotelevskii, Aleksandr Rubashevskii, Fedor Noskov, Maksim Velikanov, Alexander Fishkov, Samuel Horvath, Martin Takac, Eric Moulines, and Maxim Panov. Efficient conformal prediction under data heterogeneity. In International Conference on Artificial Intelligence and Statistics, pages 4879–4887. PMLR, 2024\n\nQUESTIONS:\n* Why the proposed algorithm do not operate on classification datasets used in [1]. This limits the credibility of the algortihm as the FedCP is not tested on these datasets as well. \n* Some very closest works mentioned [1,2] , but not included in the baselines. What is the exact reason? Some of the datasets in this study has been used \n* What distribution shift do you consider in the proposed algorithm?\n\n\n[1]  Charles Lu, Yaodong Yu, Sai Praneeth Karimireddy, Michael Jordan, and Ramesh Raskar. Federated conformal predictors for distributed uncertainty quantification. In International Conference on Machine Learning, pages 22942–22964. PMLR, 2023.\n\n[2]Vincent Plassier, Alexander Fishkov, Maxim Panov, and Eric Moulines. Conditionally valid probabilistic conformal prediction. stat, 1050:1, 2024.\n\n[3] Vincent Plassier, Nikita Kotelevskii, Aleksandr Rubashevskii, Fedor Noskov, Maksim Velikanov, Alexander Fishkov, Samuel Horvath, Martin Takac, Eric Moulines, and Maxim Panov. Efficient conformal prediction under data heterogeneity. In International Conference on Artificial Intelligence and Statistics, pages 4879–4887. PMLR, 2024\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}