Title: DIsoN: Decentralized Isolation Networks for Out-of-Distribution Detection in Medical Imaging
Venue: NeurIPS 2025
Accepted: yes
Keywords: Out-of-Distribution Detection

Decentralized Learning

Medical Imaging

Reliability
Abstract: Safe deployment of machine learning (ML) models in safety-critical domains such as medical imaging requires detecting inputs with characteristics not seen during training, known as out-of-distribution (OOD) detection, to prevent unreliable predictions. Effective OOD detection after deployment could benefit from access to the training data, enabling direct comparison between test samples and the training data distribution to identify differences. State-of-the-art OOD detection methods, however, either discard the training data after deployment or assume that test samples and training data are centrally stored together, an assumption that rarely holds in real-world settings. This is because shipping the training data with the deployed model is usually impossible due to the size of training databases, as well as proprietary or privacy constraints. We introduce the Isolation Network, an OOD detection framework that quantifies the difficulty of separating a target test sample from the training data by solving a binary classification task. We then propose Decentralized Isolation Networks (DIsoN), which enables the comparison of training and test data when data-sharing is impossible, by exchanging only model parameters between the remote computational nodes of training and deployment. We further extend DIsoN with class-conditioning, comparing a target sample solely with training data of its predicted class. We evaluate DIsoN on four medical imaging datasets (dermatology, chest X-ray, breast ultrasound, histopathology) across 12 OOD detection tasks. DIsoN performs favorably against existing methods while respecting data-privacy. This decentralized OOD detection framework opens the way for a new type of service that ML developers could provide along with their models: providing remote, secure utilization of their training data for OOD detection services. Code available at: https://github.com/FelixWag/DIsoN


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a non-parametric out-of-distribution (OOD) detection method that scores test samples based on the convergence time of a binary classifier neural network trained to distinguish the test example from the training dataset. The framework is built under decentralized assumptions and can be deployed in private environment, extinguishing data sharing and only allowing for parameter sharing between server and client.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\nThe paper addresses an important limitation of non-parametric methods for OOD detection: data privacy concerns. It also introduces a novel sample-wise optimization-based approach for OOD detection. The proposed method demonstrates strong performance on a relevant medical image classification benchmark.\n\n### Weaknesses:\n- Limited mathematical formulation. In Section 3, the preliminary formulation does not include labels or pseudo-labels, even though the method relies on class-conditional information. Providing a more comprehensive formulation would improve clarity.\n- Insufficient ablation study on class-conditional importance. There is a limited discussion of how class-conditional information contributes to the method's effectiveness.\n- Limited discussion of algorithm runtime. The paper only briefly mentions runtime in the conclusion. A more thorough evaluation of runtime would help practitioners determine the method's applicability and would strengthen the paper's contributions.\n- Minor: writing. The writing could be improved. Excessive use of passive voice makes some passages difficult to read.\n\nQUESTIONS:\n1. What if the predicted class is wrong? Could the authors elaborate how much it impacts CC-DIsoN? Showcasing robustness under misclassification could improve the method's trustworthiness.\n2. Which other methods listed in Table 1 employ a class-conditional criterion? Clarifying this point would help readers better understand how DIsoN compares to existing approaches.\n3. Why were $E_{stab} = 5$ and $\\tau = 0.85$ chosen? Providing more detail on the rationale behind these parameter choices would strengthen the methodology section.\n4. Could the authors please elaborate on why the isolation forest exhibits such low performance, with AUROC values even below 0.5 in Table 2? Although you cite isolation forests as inspiration for isolation networks, the results for iForest are poor. Some discussion or analysis regarding this discrepancy would be helpful.\n5. In Figure 4, only the average number of rounds to convergence is reported. Could you provide the full distribution of this metric? Reporting the median, as well as other quantile statistics, could be more informative, as there may be significant variability and some samples might be much harder to isolate than others. Presenting this additional information would help practitioners better assess whether the method is suitable for their specific applications.\n\nI'm willing to increase my score if these concerns are satisfactorily addressed.\n\nLIMITATIONS:\nAuthors briefly mention limitations of their work. The proposed method is costly and lacks adequate runtime analysis.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents DIsoN, a decentralized solution for out-of-distribution (OOD) sample detection in medical imaging. It leverages an Isolation Network to assess OOD samples based on convergence speed and enables collaborative training between source and target nodes without sharing raw data. Experimental results demonstrate that DIsoN achieves significant improvements in OOD detection performance. However, the article has some shortcomings in terms of expression clarity, selection of experimental datasets, and comparison methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. The overall structure of the article is well-organized, and the images in this article effectively illustrate the method's flow.\n2. The proposed method utilizes convergence rate as an indicator for OOD detection and incorporates collaborative learning to achieve online inference while ensuring privacy protection.\n\nWeaknesses:\n1.\tThere are some typos, including a misspelling of \"odd\" on line 45. Additionally, the sentences on lines 155-157 lack clarity and ambiguity, particularly regarding the data composition of SN and TN.\n2.\tWhile the paper highlights its distinctions from federated learning, the proposed setup resembles a two-client federated learning scenario, with aggregation occurring on a few-shot client. The derivation of formulas also represents a special case of FedAvg. The authors should discuss and compare their work with few-shot federated learning.\n3.\tThe datasets (Dermatology, Breast Ultrasound, and Chest X-Ray) used artificial artifacts as OOD data, resulting in significant differences. While Table 1 demonstrates the method's effectiveness, Table 2 shows that the advantages of the proposed method are not as pronounced with real medical images. The authors should analyze the reasons for this discrepancy and consider using real medical image data from different domains for OOD tasks to construct a more robust test dataset.\n4.\tOOD images often exhibit strong similarities to source domain images, making fine-tuning the backbone prone to overfitting. An interesting practical approach is to fine-tune the classification head instead. The authors could enhance the discussion around this practice to underscore the practical significance of the method.\n\nQUESTIONS:\nSee Weaknesses\n1.\tImproving the clarity of the article's expression will make it easier for the audience to understand.\n2. The authors should include a discussion on few-shot federated learning. It is strongly recommended to incorporate a comparison with recent few-shot federated learning methods， such as [1]. This will provide a more comprehensive assessment of the effectiveness of your method.\n[1] FedAffect: Few-Shot Federated Learning for Facial Expression Recognition, ICCV Workshops, 2021.\n3. While the authors discuss the impact of /alpha on performance and speed in the ablation study, the values of |Bs| and N differ significantly across datasets. Could the authors provide more details? Clarifying the amount of data required for fine-tuning is crucial for the practical application of the method.\n4. Why did the authors choose generated data for OOD detection? Is it possible to use similar real-world datasets as OOD data to demonstrate the method's effectiveness on actual images, including natural images?\n\nLIMITATIONS:\nSee Questions.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces an original OOD detection method, Isolation Network. The core proposition is that it would be beneficial to train a binary classifier to provide an estimation of how difficult, for example, it would be to isolate an OOD target test sample from an easily accessible training set, taking convergence speed as an OOD measure. The paper also introduces its decentralized analogue, DIsoN, which counters real-world complications which are encountered in application domains like medical imaging where data privacy and communication constraints rule out accessing OOD detection training sets.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. The paper is written in a relatively fluent manner. Its structure is logically organized, and the language used throughout is clear and accessible, making it easy for readers to follow \n\n2. The paper is on OOD detection's practical challenges for the application of medical image analysis, where data transmission and privacy make it hard to maintain direct access to training data at deployment's side.\n\nWeaknesses\n\n[W1] DIsoN only processes one target sample at any given time. While its authors envision future development for handling multiple samples at once, that limits its effectiveness for most batch processing environments.\n\n[W2] DIsoN creates considerable computational overhead at inference time and can range from 40 sec to 4 min on each sample. Such latency can be a significant limitation on applications where fast or real-time inference is needed.\n\n[W3] The manuscript says that α=0.8 performs well on all experiments and validates its effectiveness via ablation experiments. It lacks, though, an indepth theoretical study to guide optimal α selection or an automatic selection mechanism.\n\nQUESTIONS:\n[Q1] In practical application, communication latency and volatility across both ends may have an impact on the efficiency of DIsoN. Has the paper discussed how such practical network conditions would have an impact on convergence speed and OOD detection efficacy?\n\n[Q2] The statistical significances of OOD detection results are significant for establishing method robustness. Although there is some warrant for computational cost for running multiple seeds on all configurations, would it be possible to report mean and standard deviation for significant experimental results such as comparisons between CC-DIsoN and best baseline between Tables 1 and 2? Reporting them would greatly enhance confidence of findings.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}