DIsoN: Decentralized Isolation Networks for Out-of-Distribution Detection in Medical Imaging
Authors: Felix Wagner
Pramit Saha
Harry Anthony
Alison Noble
Konstantinos Kamnitsas
Venue: NeurIPS 2025 poster ()
Keywords: Out-of-Distribution Detection
Decentralized Learning
Medical Imaging
Reliability
Primary area: machine_learning_for_sciences
TL;DR: We propose DIsoN, a decentralized method for out-of-distribution detection that enables a deployed model to directly compare incoming samples to the training data without data sharing.

Abstract: Safe deployment of machine learning (ML) models in safety-critical domains such as medical imaging requires detecting inputs with characteristics not seen during training, known as out-of-distribution (OOD) detection, to prevent unreliable predictions. Effective OOD detection after deployment could benefit from access to the training data, enabling direct comparison between test samples and the training data distribution to identify differences. State-of-the-art OOD detection methods, however, either discard the training data after deployment or assume that test samples and training data are centrally stored together, an assumption that rarely holds in real-world settings. This is because shipping the training data with the deployed model is usually impossible due to the size of training databases, as well as proprietary or privacy constraints. We introduce the Isolation Network, an OOD detection framework that quantifies the difficulty of separating a target test sample from the training data by solving a binary classification task. We then propose Decentralized Isolation Networks (DIsoN), which enables the comparison of training and test data when data-sharing is impossible, by exchanging only model parameters between the remote computational nodes of training and deployment. We further extend DIsoN with class-conditioning, comparing a target sample solely with training data of its predicted class. We evaluate DIsoN on four medical imaging datasets (dermatology, chest X-ray, breast ultrasound, histopathology) across 12 OOD detection tasks. DIsoN performs favorably against existing methods while respecting data-privacy. This decentralized OOD detection framework opens the way for a new type of service that ML developers could provide along with their models: providing remote, secure utilization of their training data for OOD detection services. Code available at: https://github.com/FelixWag/DIsoN

DIsoN: Decentralized Isolation Networks for Out-of-Distribution Detection in Medical Imaging
Felix Wagner' Pramit Saha"! Harry Anthony"
J. Alison Noble'
Konstantinos Kamnitsas'
Department of Engineering Science, University of Oxford felix. waw.xxwwgnereengggo ax.ac.uk
Abstract
Safe deployment of machine learning (ML) models in safety-critical domains such as medical imaging requires detecting inputs with characteristics not seen during training, known out-of-distribution (OOD) detection, prevent unreliable predictions. Effective OOD detection after deployment could benefit from access to the training data, enabling direct comparison between test samples and the training data distribution to identify differences. State-of-the--e OOD detection methods, however, either discard the training data after deployment or assume that test samples and training data are centrally stored together, an assumption that rarely holds in deployed model is usually impossible due to the size of training databases, as well as proprietary or privacy constraints. We introduce the Isolation Network, an OOD detection framework that quantifies the difficulty of separating a target test sample from the training data by solving a binary classification task. We then propose Decentralized Isolation Networks (DIsoN), which enables the comparison of training and test data when shati-hharing is impossible, by exchanging only model parameters between the remote computational nodes of training and deployment. We further extend DIsoN with candiioondi,,,,ng, comparing a target sample solely with training data of its predicted class. We evaluate DIsoN four medical imaging datasets (dermatology. chest X-ray, breast ultrasound, histopathology) across 12 OOD detection tasks. DIsoN performs favorably against existing methods while respecting data-privacy. This decentralized OOD detection framework opens the way for new type of service that MI developers could provide along with their models: providing remote, secure utilization of their training data for OOD detection services. Code available at: htttps:/ tet:::/gitttn //gtttttt....... www. Coom/FellaagDDIolll
settings. This is because shipping the training data with the
1 Introduction
Consider the standard setting where an organization, such as a software company, develops and trains Machine Learning (ML) model to perform a task of interest, for example disease classification in medical images, using a training database. The organization then provides the model to a user (e.g., client) and deploys it, for example in hospital, to make predictions Because of heterogeneity in real-world data or user-error, a deployed model may receive inputs unlike anything seen in the training data. Such inputs are called out-of-distribution (OOD) samples, in contrast to the in-distribution (ID) samples that follow the distribution of the training data. For example, OOD patterns medical imaging can be artifacts from suboptimal acquisition unknown diseases. Performance of ML models often degrades unexpectedly on OOD data 43). Thus, to ensure safe deployment in safety-critical applications such healthcare, deployment frameworks should
test samples.
"Equal second-author contribution.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

include mechanisms to detect OOD inputs, so that they can be flagged to human users and avoid adverse effects of using potentially wrong model predictions in the downstream workflow.
Multiple OOD detection methods have been previously developed, which can be broadly catego- rized into post-hoc and training-time time regularization methods [4]] Post-hoc methods assume that a "primary" model has been trained and deployed to perform a task of interest, such as disease classification, and they aim to perform OOD detection without alterations to this primary model [5]. On the other hand, training-time regularization methods alter the embedding space of the primary model during its training. to allow improved OOD detection during inference 30
13
22
23
27
29
5
14
36 28].
Few OOD detection methods leverage direct comparison of test-samples against the training data to identify differences between them for OOD detection, such as KNN-based (3 or density-based methods [6]. These require, however, the training data to be available at the site of deployment, for
direct comparison with test-samples. Therefore they are impractical in many applications where sharing and storing the training samples or their embeddings at each site of deployment is impossible due to privacy, legal, or proprietary constraints. Because of this, most other methods do not compare the test-samples against the training data [K 21 or make indirect comparisons using auxiliary
representations of training data, such as via summary statistics (22), prototypes (29] 33) or synthetic
samples 77. Such derived representations, however, do not faithfully capture all intricacies of the
original training data. Hypothesizing that direct comparison with the original training data would be
useful infer whether test sample is OOD, this paper addresses the following question:
Can design OOD detection algorithu that compares test samples against the original training data, without requiring transfer of training data to the point of deployment?
This paper describes a novel OOD detection framework (Fig. [T) with the following key aspects: (1) We introduce Isolation Networks for OOD detection. To infer whether new test sample is OOD, a neural network is trained to learn a binary classification boundary that separates (iso- lates) test sample from the original ID training samples. The network's 's convergence rate is then used as a measure of whether the target sample is OOD similar to the training data. The intuition is that test samples with OOD patterns will be easier separate from the training samples than ID test samples without OOD patterns. Isolation Networks draw inspiration from Isolation Forests (25). which train decision trees to isolate each training sample. To infer whether a test- sample is OOD, the number split nodes needed to isolate it is used (c.f. Sec. [p. Isolation Networks instead train a different network for each test sample, separate it from the training data, and measure convergence as OOD score. (2) We then introduce Decentralized Isolation Networks (DIsoN), a decentralized training framework that enables training Isolation Networks the practical setting when the ID training data are held a different computational node than the node of deployment, where the test samples are processed. (3) We extend DIsoN to class-conditional training. where the class of a test sample is first predicted and then used to compare it only against samples in the ID training database of the same class. This reduces variability within the distribution of training samples that the model uses for comparison, making it harder to isolate ID sam- ples that closely resemble their class than OOD samples, improving OOD detection.
Model Provider
Deployment Site
Current OOD Detection Appreach
Deployed
Model Training
Mone
Training Data
New Peberr
Dul-gh- Callautan
Our Decentralized DisoN Approach
New Peter: Scan
Training Data ID Datal
Figure 1: (Top) Most OOD detection methods do not use training data af- ter deployment because it cannot be shipped with the model. (Bottom) DI- soN enables decentralised comparison of test samples with the training data via model parameter exchange.
We evaluate the capabilities of DIsoN and state-of-the-art OOD detection methods in identifying OOD patterns that oceur real-world applications. For this, we experiment on 4 medical imaging datasets, including dermatology, chest X-rays, breast ultrasound and histopathology, where we

---

evaluate 12 OOD detection tasks. Results show DIsoN outperforms current state-of-the-an methods, demonstrating the effectiveness of remotely obtaining information from the training data during inference to identify unexpected patterns in test samples, while adhering to data-sharing constraints.
Related Work
OOD Detection methods. Post-hoc methods for OOD detection assume a "primary" model pre- trained for task of interest, such as disease classification, and aim to perform OOD detection without modifying it. A common baseline is using Maximum Softmax Probability (MSP) [13] as a measure of whether a test sample is OOD. Another common approach is to represent the training data via summary statistics, such as using class conditionl Gaussians in the model's ' latent space, and Mahalanobis distance detect OOD samples [ In a similar fashion, other post-hoc approaches use the logit space (energy score (27). gradient space, (GradNorm [15] and feature space (fDBD (26), combine multiple representations (ViM
37) to derive measure for OOD
detection. Another group of methods use training- regularization. Regularization modifies the embedding space of the primary model during training to bolster OOD detection performance at inference 14 5 39 9 A representative example of the state-of-the-arr is CIDER [30], which uses loss function optimize the model's feature space for intra-class compactness and inter-class dispersion. Recently, PALM [29] this further by modeling each class with multiple prototypes.
Most OOD detection methods, such as the above, do not directly utilize the training data after the model's deployment. Thus they do not directly compare them with the test samples processed after deployment, which could facilitate detecting patterns that differentiate them. There few notable exceptions, such methods based on KNN 34) and Local Outlier Factor 6 These compare an input's s embedding the training data embeddings. They require, however, shipping and storing the training data or their feature vectors to each deployment site, which in many practical applications can be infeasible due privacy constraints or the size of training databases.
Inspiration for this work was drawn from the seminal work on Isolation Forests (iForest 25 which has received multiple extensions such Deep Isolation Forests [40]. They train decision trees using the original ID training data or their deep embeddings respectively, to learn partitions that isolate each sample. To infer whether test-sample is OOD, they count the number of split nodes applied by the trained trees until it is isolated. OOD samples should need less splits than ID nodes. The algorithm introduced herein trains neural network classifier isolate a single test-sample from the ID training data, focusing on extracting patterns that distinguish the specific sample. We demonstrate the use of convergence rate as scoring function infer whether test-sample is OOD. We also demonstrate the use of decentralised training to enable such an isolation algorithm without data-sharing.
Federated Learning and OOD Detection. Training of DIsoN uses decentralised optimization similar to Federated Learning (FL). Using FL for OOD detection is a recent area. The few related works, such as 44 24), studied settings significantly different from ours: they assume the training data is decentralized across multiple computational nodes, where each node's data follow a different distribution. Using FL. they train a model that measures distribution-shift, between nodes. After training, the model is applied on test-samples for OOD detection. Our algorithm does not require training data from multiple distributions data and a second node holds a single test-sample, which we infer whether it is OOD via decentralised training of a binary classifier. Thus the motivation, use-case and technical challenges are distinct. Moreover, DIsoN may resemble FL with a few-shot client 33 38 Few-shot methods are designed regularize against overfitting the limited data of few-shot client(s), train a model for predictive task (e.g. disease diagnosis) that generalizes to new samples. DIsoN does not train a model for generalization. It optimizes for distinguishing data of the source node from target sample, using convergence speed to infer if the latter is OOD. Hence, few-shot methods are not directly applicable.
multiple nodes. It assumes node holds all ID training
3 Method
Detection. Let X' be the input data space and y (1, .,, C} the label space for a classification task with C' classes. We denote the source training dataset as D,- = {(x;, yr)) ) assumed i.i.d. from the joint distribution Pxy. The marginal distribu- tion over the input data space X' is denoted as paR The goal of OOD detection is to determine, during
Preliminary:
Out-of-Distribute
3

---

DisoN Training: Dotection Dustra--Distrbbutionn Scans
In Distribution Bcan 4tter 50 Connurication frsunds tegersingie fir
Provider
Deployment
Out of Distribution Bean
Fustert Scan
Alet
thie Targe Sample
ving Data
Terget Semple
Decidir Jounder
scan is obtained at the deployment site, DIsoN is trained using parameter updates from both the deployment site and the model provider, who holds the ID training data, to isolate the target sample from the training data. The deployment site trains on the single target sean, while the model provider trains on the source data (optionally ..ndicenniiioned)..............) Only model parameters are exchanged. (Right) After convergence, the scan is classified as OOD if it is isolated in few rounds, and as ID otherwise.
Figure 2: Overview of DIsoN:
When
inference, whether target sample x, E X' originates from the source distribution Px, or an unknown OOD distribution Toout This can be achieved by defining an OOD scoring function S X' - R, that assigns high scores to ID samples and low scores to OOD samples. We label a sample x as OOD when the scoring function S(x) s", and ID if S(x) 2 s*, with chosen threshold s*.
3.1 Isolation Networks
The core idea of our approach is that the difficulty of training a binary classifier to separate a single target sample x from the source data D, gives an indication whether x, is ID or OOD. Intuitively, target sample that ID will share characteristics with samples in D.. As a result, it is harder for classifier to isolate (separate) the target sample and will require more update steps. On the other hand, an OOD target sample has patterns that differ from D., making it easier to isolate and requiring fewer update steps during training. Fig. 2 gives a visual intuition of the idea.
Formally, consider neural network consisting of feature extractor X Z, parameterized by 05, and a binary classification head h : Z + [0, 1], with parameters 0hh The full network is parameterized by 8 = (0f, 0²). We use m E {0, 1} for isolation labels and reserve for class labels. In idealized centralized setting with full access to D3 and the target sample xx, network h for the following binary classification problem: Assign label m = 0 to all source to the target sample x,. Typically, D, has a high number of
train the
samples x, D. and label samples compared to the single target sample x,, [D&] > 1. Therefore, directly training on this highly imbalanced setup is suboptimal, as it can lead to trivial classifier that always predicts the majority class (m = 0), ignoring the minority class (m 1). To emsure that the target sample has an impact during training, we over-sample x, within each mini-batch. Specifically, we construct
x; (N) where B, is set of source samples drawn from D., and (x(" ..... x (N) are N replicated copies of the same target sample x,. The empirical loss for this optimization objective is:
bati-e as B = B, U fx(") (1) "
1 Lo (,, Ba + N
Σ L(0:x,, m = 0) + N L(0; mm -(0 =1) Ax.B.
(I)
where L(0; x. m) denotes the binary cross-entropy loss. In Sec. 3.2 we compare the gradient of the centralized version (0; Ba, x,, N) = VeL.(0; B., Xx, N) to our proposed decentralized version.
Convergence Time OOD Score. To quantify the "isolation difficulty"" we track the number of training steps required by the binary classifier to separate X, from D.. Let ((k) be the model parameters after à optimization steps with a stochastic gradient-based optimizer (e.g. SGD or Adam (20) using the gradients from the loss of Eq I Formally, we define our convergence time K as:
4

---

Definition 3.1 (Convergence Time K). Let (()) h(f(;;; )( be the sigmoid after l updates. Fix a window Estab N and accuracy threshold T E (0, 1). The convergence time K the smallest & 2 Estab such that
min
( 0.5 and
1 [D.] E,ED.
(xs) 5 2
Jelk-Batab++
Intuitively, the conditions ensure that the classifier correctly classifies x; for the last Estab consecutive updates (first condition), while achieving an accuracy of at least T on the source data (second condition). We set Estab = 5 and r = 0.85. Further details on these parameter choices are provided in the Appendix D We define our OOD scoring function as S(x,) = K, following the convention where ID samples have higher scores. We emphasize that this setup corresponds to the centralized version of the algorithm, where both D. and Ta are accessible on the same node. In Sec. 3.2] we extend to the decentralized case, where source data and the target sample reside on separate nodes.
3.2 Decentralized Isolation Networks (DIsoN)
Problem Setting: Decentralized OOD Detection. The Isolation Network described above assumes direct access both the source data D, and the target sample x₃. However, as described Sec this assumption does not hold in many real-world settings. We therefore formalize a decentralized setting involving two sites: the Source Node (SN) and the Target Node (TN), SN represents a model provider that holds the source training dataset D., on which it has pre-trained primary model Mare for the main task of interest, such as a C-class classifier of disease. Myre consists of a feature extractor (parameterized by (PFr) and C-class classification head. TN holds target samples x, and represents the site where More is deployed to process X,, Therefore, the aim of the OOD detection algorithm is to infer whether given x; on TN is ID or OOD, to support safe operation of Myre- To this end, TN can exchange model parameters with SN, but not raw data or their embeddings due to privacy and regulatory constraints.
DIsoN Method Overview. To approximate the training dynamics of an Isolation Network (Sec. 3.1 without data sharing and without requiring centralized access to both the source data and target samples, we propose DIsoN. Fig Fig@ssoows an overview of DIsoN. Inspired by the Federated Learning framework, our algorithm trains parameters of an Isolation Network over multiple communications rounds between SN and TN. We keep track of global set of Isolation Network parameters. At start of each round r. they are transmitted to SN and TN. In each round both SN and TN update them locally by performing E 2 local optimization steps. At the end of a round, the global set of parameters is updated via weighted aggregation of the local updates. In more detail:
I) Initialization. Let ((+) be the global model parameters of the Isolation Network at the start of round r. The feature extractor of of the initial global model ( = (0f, 8h) is initialized with the traitra parameters gare of primary model Mpre. while the binary classification head is initialized
randomly. This initialization is done SN, after which ((0) is transmitted TN.
2) Local Updates. Each site performs local training for E steps using its own data: The Source Node initializes its local model 0f--0) = g(r) and performs E optimization steps on mini-batches B, ~ D., minimizing L(0; x 0). resulting in e) Similarly, the Target Node initializes its local model of--0) g(r) and performs E optimization steps the single target sample xt, minimizing L(0; xx, 1), resulting in O(T.E)
3) Model Aggregation. After local updates, TN sends Bf: back SN. The models are then aggregated into an updated global model using weighted averaging:
gir+1) a- (lr 3. g(r.
(2)
where the aggregation weights are 8 = - a. The updated global model ((++1) is then sent to TN to start the next local training round.
Before each local training round starts, we evaluate the current global model ((r) using the con- vergence criteria in Def. 3.1 (TN evaluates on x,, SN on D.). converged, we record R = r and
terminate. The OOD score for the target sample x; in the decentralized setting is based on the
5

---

number of communication rounds R required to converge, analogous to the number of steps K in the centralized case: SDION(X;) R. We can draw a connection between DIsoN and our centralized version. We show that under specific conditions, the decentralized and the centralized versions result in equivalent model parameters:
Proposition 3.1 (DIsoN and Centralized Isolation Network Equivalence for 1). Let deent be the model parameters from our centralized algorithu and Osec be the parameters from DIsoN. Let each site perform one local SGD step (E 1) with learning rate тT, and aggregate with |B,l a B.I+N 8
N
Then the decentralized update equals the centralized one:
[B,1+N.
0(++1) ((r) - (n(ags(0()) + (gg(()))) =0(r) (790((())) -((tt))
where gs (0) = [5.1 Ex,eB, VeL(O; xx, 0), gr) = VeL(0; 1) and N is the number of times Xt is oversampled in the centralized version.
Proof sketch. Insert r(r = ((r) (0()))))), of = g(r) - ngr into the weighted average; factor out g(r), use 3 1- 1a--a a. Full derivation can be found the Appendix.
Prop. 3.1 shows that our decentralized training exactly replicates the centralized version when = and aggregation weights are chosen accordingly. This theoretical equivalence motivated our design of DIsoN: it preserves the core idea of the Isolation Network, while meeting our decentralized data sharing constraints. In practice, we allow E > to reduce communication overhead, which causes the decentralized updates to deviate from the centralized version. However, as we show in Sec. 4 DIsoN achieves promising results even with the approximation. We found DIsoN to be robust across a broad range of a values, as shown in an ablation study in Sec. 4.2
Practical Techniques: Augmentation and Normalization. One challenge in DIsoN is to avoid rapid memorization and overfitting of superficial features the target sample x,, making the isolation task trivial regardless whether the sample is OOD. To prevent this, we apply stochastic data augmentations (e.g- random crops, horizontal flips), which regularize the model learn invariant features, so that the separation is based on semantic characteristics. Furthermore, in DIsoN, Normalization (IN) [35] instead of the widely used Batch Normalization (BN) [16) layers for feature normalization. BN relies on batch-level statistics, which are not suitable for our single-sample TN. IN instead normalizes each feature map per sample and therefore fits better for our decentralized, single target sample scenario.
use Instance
3.3 Class-Conditional Decentralized Isolation Networks (CC-DIsoN)
To further improve DIsoN, we introduce its class-conditional variant, CC-DIsoN. The intuition is that ID samples should be especially hard to isolate from source samples of the same class, they likely to share similar visual features compared to samples from other classes. To use this idea, we modify the source data sampling strategy during local training at SN. After the initialization phase, TN uses the pre-trained model Mpre to predict the class of the target sample: i = arg maxe[M pPree(xx (x+)]e Afterwards, TN sends the predicted label ý to SN. During local training SN, mini-batches sampled only from source data from class ÿ: B, C ((x., E D. ys = ÿ}. The other steps of our method remain the same. Our empirical results in Sec. 4.2 confirm that this improves OOD detection.
now
4 Experiments & Results
Datasets. We evaluate DIsoN four publicly available medical imaging benchmark datasets covering dermatology, breast ultrasound, chest X-ray, and histopathology. All datasets consist of real, clinically acquired images and no synthetic data is used. The first three benchmarks use images with naturally occurring non-diagnostic artifacts as OOD samples (e.g.. rulers, pacemakers, annotations), while histopathology focuses semantic and covariate shifts across domains. Example images shown in Fig
Dermatology & Breast Ultrasound; We adopt the benchmark setup from 3 using images without artifacts as the training and ID test data, and images with artifacts (rulers and annotations) as OOD samples. For breast ultrasound (BreastMNIST [1l),, the artefacts embedded text annotations, and

---

for dermatology (D7P [19 the artefacts are black overlaid rulers. For breast ultrasound, the primary model Mrr is trained for 3-class classification hoomma//enignmmaiigaant)) The 228 annotated seans with artifacts are used as OOD test samples, while the remaining artifact-free scans are split 90/10 into training and ID test sets. For dermatology. the primary model Mar trained for binary classification (nevus/non-nevus). The annotated 251 images with rulers are used as OOD samples and the remaining 1403 are split 90/10. Images are resized to 224 x 224.
Chest X-Ray: Following the benchmark from (2]. we use frontal-view X-ray scans (from CheXpert [7] containing no-support devices as the training and ID test data, and scans containing pacemakers as OOD samples. The primary model Mpre was trained for the binary classification of cardiomegaly. We use the 23,345 annotated scans without any support devices as our training data, and randomly hold out 1000 ID samples for testing. The OOD test set includes 1000 randomly sampled scans with pacemakers. All images are resized to 224 x 224.
Histopathology: We use the MIDOG benchmark from OpenMIBOOD [.. The MIDOG dataset 4 consists of 50 x 50 image patches extracted from Hematoxylin & Eosin-stained histological whole-slide images, grouped into different "domain" data sets corresponding to differ- ent imaging hardware, staining protocols, or cancer types. The primary model is trained for 3-class classification amitotic/impooserrotheer on domain 1a. Following 4 we evaluate on two settings: (i) Near-OOD: Domains 2-7 are treated as separate OOD detection task with only moder- ate domain shifts. (ii) Far-OOD: Using CCAgT [ and FNAC 2019 [32] dataset as the OOD task, which differ significantly due to being completely different medical applications. We use the 251 test ID samples from domain la and randomly sample 500 OOD samples from each of the near- and far-OOD domains.
X-Ray
Dermatology
OOD, Pacemaker
00S: Ruter
Ultrasound
Histopathalogy
00D
Training Details. We use ResNet18 [12 with Instance Normalization (as per Sec(3.2) pre-trained on the dataset- specific task as initialization for DIsoN. DIsoN is trained with Adam (Ir=0.001 for dermatology and ultrasound; 0.003 for X-ray). For histopathology SGD with momen- tum (Ir=0.01, momentum=0.9) is used (since [[] suggests pretraining with SGD). Local iterations per communica- tion round are chosen to approximately match one epoch on the training data. We use standard augmentations (e.g. random cropping, rotation, color-jitter) and the aggrega- tion weight is fixed to a 0.8, since it performs consis- tently well across all our experiments (see Sec. 4.2 for effect of a:). Experiments were run on NVIDIA RTX A5000. More training details and hyperparameters are provided in the Appendix. OOD detection performance is evaluated with two metrics: (i) area under the receiver operating characteristics curve (AUROC, higher
Figure 3: Examples of data. X-Ray:
ID X-ray vs. OOD scan with pacemaker. Dermatology: ID lesion vs. OOD im- age with ruler. Ultrasound: ID artifact- free ultrasound vs. OOD scan contain- ing annotations. Histopathology: ID mitotic-cell patch vs. near-OOD patch with different cancer type and far-OOD patch with different staining.
is better) (ii) the false positive rate at 95% true positive rate (FPR95, lower is better). All results in Tables D and 2 are averaged over three runs with different seeds, and we report mean and standard deviation. Standard deviations for Tab are in Appendix duu to space limitations.
Baselines. We compare method against state-of-the-an OOD detection methods from the two main categories: post-hoc and training-ting time regularization methods. The post-hoe methods include:
MSP [3). MDS [22). fDBD 26). ViM 37]. Deep iForest [40]. For training-tinn regularization methods, we evaluate the recent methods CIDER 30 and PALM 29 (both using contrastive
learning). We use MDS for OOD scoring on their learned feature representations. Several baselines already use a form of class-conditioning based on the model's 's predicted class: MDS, PALM, and CIDER via Mahalanobis distances to class clusters; ViM uses the maximum logit of the predicted class; MSP via softmax probability; and fDBD via distance to the decision boundary (all implicitly or explicitly conditioned on the predicted class). Only iForest does not. CC-DIsoN similarly relies solely on the predicted class from Mpre for class-cos caas--oondiiooall sampling and uses no ground-trum labels, providing no additional information beyond these methods.
7

---

Table 1: OOD detection performance evaluated three OOD datasets: Chest X-Ray, Dermatology, and Breast Ultrasound and reported as mean + standard deviation over three random seeds. + means smaller is better and means larger is better. Bold numbers highlight the best results, second best results are underlined.
Chest X-Ray
Dermatology
Breast Ultrasound
Average
Method
AUROCT
FPR95,
AUROCT
FPR95,
AUROCT
FPR95
AUROCT
FPR95,
MSP
60.44:42
100,00=00
65.39=39
100,00=00
58.85=45
100,00=ao
61 56.20
100,00=00
MDS
53.82g15
90.47.10
69.36_51
76.06
61.02a11
74.40
61.40g21
20.31
IBB
68.26.n4
78.07
63.59.2a
88.26
60.73121
87.50147
64.1911a
84.61
62.60s4a
80=7.3
68.39=1a
75.35-61
59.44=24
63.48a1.
12=41
iForest
$6.35=56
88. 27+5.1
56.68-68
87 87.31=37
47.02=44
95. 82-22+11
53.35=33
90.47=15
CIDER
70.47a61
79.00 79.00.99
$1.98-11
56.57-52
58.03
82.73a53
70.16s34
72.77=21
PALM
65.4116
85.7319d
77.25_
62.44_nx
59.35
75.601*1
67.34114
74.59
CC-DIsoN
84.94:03
61.85=13
89.54=1.
42.49=44
65.62=12
73.21:447
80.00
59.20=31
Table 2: OOD detection performance across nine different OOD detection task of MIDOG, split into near-OOl and far-OOD setting. Each cell shows AUROC reported as mean over three random seeds. Bold numbers highlight the best results, second best results are underlined.
near-OOD
far-OOD
Methods
2
4
6a
6b
7
Avg.
CCAsT
FNAC
Avg.
MSP
14,1994.8
47,4/93.4
55.7/
39.0993.5
54.8/4477
45.6/5.5
16,192244
53.3/93.4
78.4/52.2
82. 62.7/63.6
80.6/57.9
MDS
67.4/80.0
67.2/79.8
63.7/81.9
2/.2/
60.6/87.4
55.5/89.5
51,0916
61.2/85.4
87.1/43.6
86.6/39.6
86.8/416
IBBD
57.9/83.0
52.5/81
57.5/88.7
60.28990
57.9/85.9
51.2/86.2
58,29222
1/.1/
74.6/58.3
1/637
78. 78.3/61.0
68.077988
66.2/77.7
63.586699
11.977.
55.7010
$4,993.1
62,4/84.8
92.6/27.0
92.622 92.6/28.0
Foorst
37.9/98.3
38.6/96.9
39.7/97.7
41.6/97.5
39.5/98.0
40.497.7
46.495.77
40.6/97.4
27.7/99.2
32. 32.3/96.8
30.098.0
CIDER
71.4/84.
65.6/89.0
55.5/91.5
64 84.2889.8
57.4/92.2
47.796.5
64.1/88.2
60.9/90.2
82.5/77.2
95.4/18.3
89.0/47.8
PALM
73.5/78.1
59.2/90.3
66.3/77.7
64.3/93.6
62.1//08
8.7.6
61.68336
61.2/88.8
97.121.8
99.6/1.5
98_3116
CC-DioNN
75.4/28.8
79.5/61.7
72.6/79.7
63 63.0/89.4
64.0/85.3
70.7/79.9
6188/2.0
69.6/81.0
98.3/4.8
98.4/40
98.3/4.4
4.1 Evaluation on Medical OOD Benchmarks
Dermatology, Chest X-Ray and Breast Ultrasound. Tab. compares CC-DIsoN with the base- lines on our three medical OOD benchmarks where the OOD task is to detect artifacts. We can see that the post-hoc methods struggle detect these domain-specific artifacts: fDBD and ViM average only between 63- - 64% AUROC while having high FPR95 (84.6% and 78.1% respectively). Training-time regularization methods like CIDER do better (70.2% average AUROC), but still have high FPR95 of 72.8%. CC-DIsoN performs strongly compared to the baselines and shows consistent improvement across all datasets. Compared fDBD, improves AUROC by 15.8% and reduces FPR95 by 25.4%. Against the best baseline (CIDER), it improves AUROC by 9.8% and lowers FPR95 by 13.6%. This demonstrates that directly comparing test samples against the training data during inference has positive impact on OOD detection.
Effects of Caasi-Condiiinnnng Tab. 3a shows the benefits of class-conditioning (Sec. 3.3 On average, CC-DIsoN improves AUROC by 1.7%. More notably, CC-DIsoN reduces FPR95 by 8.2 % across all three datasets. Class-conditioning consistently lowers FPR95, demonstrating that focusing the isolation task on the predicted same-class samples improves OOD detection.
Table 3: (a) Comparison of DIsoN and CC-DIsoN on three medical datasets using AUROC (higher is better) and FPR95 (lower is better). (b) Effect of incorrect predicted classes for condiiioni sam- pling in CC-DIsoN (AUROC). "Best Baseline" is the strongest baseline from Tab. (for reference); "ID & OOD wrong": all targets assigned incorrect classes; "ID wrong": only ID targets assigned incorrect classes; "OOD wrong": only OOD targets assigned incorrect classes.
(a)
(b)
Dataset
DisoN
AUROCT
CC DisoN
DiseN CC DiaNN
FPR95
Dalreet
Best Rostine
Standand OC-MbaN
ID & 000
wnong
wmin
ID
wnoin
000
Ultrasound
67.0
65.6
78.6
73.2
Ultassund
Damatology
56.4
89.5
50.0
42,5
Dermutology
89.5
83.1
RaR
$1.4
84.9
73.6
61,9
Ray
81.9
77.9
74.0
Arcrage
67.
59,2
Impact of Predicted Class Accuracy. Since CC-DIsoN conditions on the predicted class from the pre-trained model Mpre, analyse how misclassification affect its performance. We simulate three
8

---

controlled scenarios by asssigning wrong predicted classes to: (i) all target samples (ID & OOD), (ii) only ID targets while OOD targets use their predicted classes, and (iii) only OOD targets while ID targets use their predicted classes. Results in 3a show that performance drops mainly when ID targets are incorrect, as cansiionnii compares them to unrelated classes, making isolation easier and reducing the separation in convergence rounds between ID) and OOD samples. In contrast, performance slightly increases when OOD targets are mislabeled. as comparing them to unrelated classes makes convergence faster. Even under such extreme settings, CC-DIsoN remains competitive. Further details are provided in the Appendix
vs Aggiegatinn Neught
Comergencc Speed nue tCamrt ned)
cheil dass
smrngy
030 Adh
Raci
(a)
(b)
Figure 4: (a) Effeec of aggregation weight Left: AUROC vs. Higher a improves OOD detection by emphasizing the source updates. Right: Mean number of communication rounds until convergence (ID and OOD targets combined). Trade-off: Lower values speed up convergence but reduce OOD performance. (b) Network Size. Global AUROC vs. FPR95 plot for three backbones (Slim Re ResNet18, ResNet34) across the same datasets. R gives the balance between AUROC and FPR95. Grey dashed lines link backbones per dataset.
Histopathology: MIDOG. While earlier benchmarks focused on detecting non-diagnostic artifacts as OOD task, MIDOG evaluates semantic and covariate shift as OOD task. Tab. shows that CC-DIsoN also performs strongly in this setting. In the most challenging near- OOD setting, CC-DIsoN achieves an average AUROC of 69.6%, which is 8.4% improvement compared the best regularization- based method PALM and a 7.2% % gain compared to the best post-hoc method ViM. It also achieves the lowest FPR95 in this setting, albeit with smaller margin. In the less challenging far-OOD setting, CC-DIsoN achieves an AUROC of 98.3% and a FPR95 of 4.4%, indicating almost perfect separation. Out of all compared methods, only PALM achieves same AUROC performance in the far-OOD task, although CC-DIsoN performs better in the near-OOD task. IForests were originally developed for tabular data, where they perform well. However, prior works [34 [0] shows poor performance on nigh-diimessioal imaging data, which explains the low AUROC in our experiments. Overall, these results demonstrate that CC-DIsoN does not only perform well for OOD tasks with localised artifacts but also effective in identifying semantic and covariate shifts across medical imaging tasks.
4.2 Further Analysis of DIsoN
Sensitivity Analysis of Hyperparameter a. We analyze the effect of the aggregation weight a (Eq. 2 on both the OOD detection performance and convergence rate across Dermatoloyy. Ultrasound, and X-Ray in Fig. 4a We can see that lower a values, which give relatively more weight to the target updates, reduce OOD detection performance, since the isolation task becomes dominated by the target sample, and loses the comparison signal to the source data. Increasing a improves AUROC consistently across datasets, with performance plateauing at = 0.8. This shows that a stronger OOD performance. However, the number of
emphasis on training data improves the isolation-basee communication rounds required for convergence increases with a. This aligns with Proposition 3.1 where a controls the implicit oversampling ratio N: smaller a increases the target signal (larger N) (faster isolation), while larger slows convergence (more rounds needed to incorporate the target's signal). In practice, this creates a trade-off between convergence speed and detection performance. The goal of this analysis is not to identify one "optimal" value of a, but show that method remains robust across a wide range of values. Performance stays quiet stable for a between 0.5 and 0.95 (Fig. 4)), suggesting good generalization across tasks without extensive hyperparameter tuning.
Effect of Network Size. To quantify how capacity of the Isolation Network affects the OOD scores, compare three network sizes: a "Slim" ResNet18 (0.5x channel widths), the standard

---

Demmatilggy
Dermatalogy
g
Z24e U19904M Derwitinyy
62 romabbe
Figure 5: Effect of image augmentation. Left: Bar plots show that using random augmentation ("Yes Aug") during training improves AUROC and FPR95 for three datasets. Right: Density plots for dermatology: without augmentation, ID and OOD curves overlap heavily, whereas with augmentation OOD samples isolate fast and ID samples require more updates.
ResNet18, and a deeper ResNet34. Fig. 4b shows each backhone's AUROC vs. FPR95 trade-off on Breast Ultrasound (circles), Dermatology (squares), and Chest X-Ray (triangles). Neither the Slim- ResNet18 nor the ResNet34 outperforms the standard ResNet18: the slim model might lack sufficient representational capacity detect the subtle visual difference, while the deeper network does not yield consistent OOD gains and roughly doubling the required compute. This is likely because a binary isolation task does require this excessive parameter capacity. Therefore, ResNet18 8 gives the best balance between good ID/OOD separation and computational efficiency.
Effect of Image Augmentations. Fig. 5 shows the effects of applying image augmentations during DIsoN training on Ultrasound, Dermatology, and X-Ray. Augmentations consistently improves AUROC: +4.43 % Ultrasound, +13.69% Dermatology, and +2.76% on X-ray. FPR95 also decreases, most notably on Dermatology -30.99 %. The density plot of dermatology dataset shows that without augmentation, ID and OOD samples converge in similar number of few rounds, resulting in poor separation. With augmentations, OOD target samples isolate quickly, but ID target samples require many more rounds. This demonstrates the regularization effect of augmentations: they prevent the model from quickly memorizing a single target sample, regardless of whether it is ID or OOD.
Runtime Analysis and Practical Considerations. We extend the runtime analysis in Fig. 4 by reporting detailed statistics, including quantiles and results across different a values, in Appendix G Since wall-clock time depends hardware and network conditions, we primarily measure runtime communication rounds, offering a hardware-indes inndeereinddeeedeett estimate. Although DIsoN introduces extra computation, each target sample requires isolation task, this design trades speed for improved OOD detection performance. In practice, real-time inference rarely required in healthcare, where scans are often reviewed hours days later (31 (except in emergencies), and diagnostic accuracy is typically prioritised over speed. Further runtime comparisons with baselines, as well as notes on parallelisation and more detailed practical considerations, are provided in Appendix H
Conclusion
In this paper, we propose Decentralized Isolation Networks (DIsoN), novel OOD detection frame- work that, unlike most existing methods, actively leverages training data at inference, without requiring data sharing. DIsoN trains binary classification task measures the difficulty of isolating a test sample by comparing it to the training data through model parameter exchange between the source and deployment site. Our cansiconniiional variant, CC-DIsoN, further improves performance and achieves consistent gains in AUROC and FPR95 across four medical imaging datasets and 12 OOD detection tasks, compared state-of-the-art methods. One limitation of DIsoN is, that it requires additional compute during inference for target samples. In practice, DIsoN requires roughly 40s to 4 min per sample (depending on the dataset), thus it practical for applications where inference delay of this magnitude is not an issue. We show that this overhead can be controlled via the aggregation weight a (convergence speed) and backbone size, enabling a trade-off between efficiency and detection performance. In future work, we aim extend DIsoN to handle multiple target samples simultaneously to improve efficiency. Overall, our results demonstrate that leveraging training data during inference can improve OOD detection in privacy-sensitii deployment scenarios.
10

---

Acknowledgements
FW is supported by the EPSRC Centre for Doctoral Training Health Data Science (EP/S02428X/1) the Anglo-Austrian Society, and an Oxford-Reuben scholarship. PS is supported by the EPSRC Programme Grant [EP/T028572/1], UKRI grant [EP/X0401 86/1], and EPSRC Doctoral Training Partnership award. HA is supported by scholarship via the EPSRC Doctoral Training Partnerships programme [EP/W52431 1/1, EP/T51781 1/1]. The authors acknowledge the use of the University of Oxford Advanced Research Computing (ARC) facility http://dx.doo.or///11/ htwg://0000121222oo22225555555
References
[1] Amorim, J.G.A., Macarini, L.A.B., Matias, A.V., Cerentini, A., Onofre, F.B.D.M., Onofre, A.S.C., Von Wangenheim, A.: A novel approach on segmentation of agnor-stained cytology images using deep learning. In: 2020 IEEE 33rd International Symposium Computer-Base
Medical Systems (CBMS). pp. 552-557. IEEE (2020)
[2] Anthony, H., Kamnitsas, K.: On the use of mahalanobis distance for out-of of-distribution detection with neural networks for medical imaging. In: International Workshop on Uncertainty for Safe Utilization of Machine I Learning in Medical Imaging. pp. 136-146. Springer (2023)
[3] Anthony, H., Kamnitsas, K.:
Evaluating reliability in medical dnns: critical analysis of feature and confidence-based ood detection. In: International Workshop on Uncertainty for Safe Utilization of Machine Learning in Medical Imaging- pp. 160-170. Springer (2024)
[4] Aubreville, M., Wilm, F., Stathonikos, N., Breininger, K., Donovan, T.A., Jabari, S., Veta, M., Ganz, J., Ammeling, J., van Diest, P.J., et al.: A comprehensive multi-domain dataset for mitotic figure detection. Scientific data 10(1). 484 (2023)
[5] Blundell, C., Cornebise, J., Kavukcuoglu, K., Wierstra, D.: Weight uncertainty in neural network. In: International conference on machine learning. pp. 1612.1622. PMLR (2015)
[6] Breunig, M.M., Kriegel, H.P., Ng. R.T., Sander, J.: Lof: identifying density- based local outliers. In: Proceedings of the 2000 ACM SIGMOD international conference Management of data. pp. 93-104 (2000)
[7] Chen, J., Qu, X., Li, J., Wang. J., Wan, J., Xiao, J.: Detecting via class-conditional impressions reappearing. In: ICASSP 2023-2023 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE (2023)
examples
[8] Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. pp. 248-255. leee (2009)
[9] DeVries, T., Taylor, G.W.; Learning confidence for out-of-distribution detection in neural networks. arXiv preprint
(2018)
[10] Fan, K., Liu, Qiu, X., Wang, Y., Huai, L., Shangguan, Z., Gou, S., Liu, F., Fu, Fu, Y., et al.: Test-time linear out-of-distribution detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 23752-23722 (2024)
[11] Gutbrod, M., Rauber, D., Nunes, D.W., Palm, C.: Openmibood: Open medical imaging benchmarks for out-of-distribution detection. arXiv preprint 1rXiv:2503.16247 (2025)
[12] He, K., Zhang, X., Ren, S., Sun, Deep residual learning for image recognition. In: Pro- ceedings of the IEEE conference on computer vision and pattern recognition. pp. 770-778 (2016)
[13] Hendrycks, D., Gimpel, K.: A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint 111.v:1610.02136 (2016)
[14] Hendryeks, D., Mazeika, M., Dietterich, T.: Deep anomaly detection with outlier exposure. In: International Conference on Learning Representations (2019)

---

[15] Huang, R., Geng, A., Li, Y: On the importance of gradients for detecting distributional shifts in the wild. Advances in Neural Information Processing Systems 34, 677-689 (2021)
[16] loffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International conference machine learning. pp. 448 448-456. pmlr (2015)
[17] Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ileus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., et al.: Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In: Proceedings of the AAAI conference artificial intelligence. vol. 33, pp. 590-597 (2019)
[18) Kamnitsas, K., Winzeck, S., Kornaropoulos, E.N., Whitehouse, D., Englman, C., Phyu, P., Pao, N., Menon, D.K., Rueckert, D., Das, T., et al.: Transductive image segmentation: Self- training and effect of uncertainty estimation. In: MICCAI Workshop on Domain Adaptation and Representation Transfer. pp. 79-89. Springer (2021)
[19] Kawahara, J., Daneshvar, S., Argenziano, G., Hamarneh, G.: Seven-point checklist and skin lesion classification using multitask multimodal neural nets. IEEE journal of biomedical and health informatics 23(2), 538-546 (2018)
[20] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
[21] Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems 30 (2017)
[22] Lee, K., Lee, K., Lee, H., Shin, J.: A simple unified framework for detecting outtof-distribution samples and adversarial attacks. Advances in neural information processing systems (2018)
[23] Liang, S., Li, Y., Srikant, R.: Enhancing the reliability of out-of-distribution image detection in neural networks. arXiv preprint arXiv: 1eX.v:1106.000 776.02690 (2017)
[24] Liao, X., Liu, W., Zhou, P., Yu, F., Xu, J., Wang, J., Wang, W., Chen, C., Zheng, X.: Foogd: Federated collaboration for both -iitt--diitiitttionn generalization and detection. arXiv preprint arXiv:2410.1 1397 (2024)
[25] Liu, FT., Ting, K.M., Zhou, Z.H.: Isolation forest. In: 2008 eighth ieee international conference on data mining, pp. 413-422. IEBE (2008)
[26] Liu, L., Qin, Y.: Fast decision boundary based out-of out-of-distribution detector. In: International Conference on Machine Learning. pp- 31728 31746. PMLR (2024)
[27] Liu, W., Wang, X., Owens, J., Li, Y.: Energy-based of detection. Advances in neural information processing systems 33, 21464-214755 (2020)
[28] Liu, W., Wang. X., Owens, J., Li, Y.: Energy-based ofttof-distribution detection. Advances in neural information processing systems 33, 21464-21475 (2020)
[29] Lu, H., Gong, D., Wang. S., Xue, J., Yao, L., Moore, K.: Learning with mixture of proto-
types for out-of-distribution detection. In: The Twelfth International Conference on Learning Representations (2024)
[30] Ming, Y., Sun, Y., Dia, O., Li, Y.: How to exploit hyperspherical embeddings for out-of- distribution detection? In: The Eleventh International Conference on Learning Representations (2023)
[31] NHS-Engie England: Diagnostic imaging reporting turnaround times. https: h/wp:::///ww... england nhs. wollooongrrde seaadddiagnostii 2025-08-20
wur.grgguurrruundd-- times/ accessed:
[32] Saikia, A.R., Bora, K., Mahanta, L.B., Das, A.K.: Comparative assessment of enn architectures for classification of breast fnac images. Tissue and Cell 57, 8-14 (2019)
12

---

[33] Shome, D., Kar, T.: Fedaffect: Few-shot federated learning for facial expression recognition. In: Proceedings of the IEEE/CVF international conference on computer vision workshops. pp.
4168 4175 (2021)
[34] Sun, Y., Ming, Y. Zhu, X., Li, Y.: Out-of-distribution detection with deep nearest neighbors, In: International Conference on Machine Learning. pp. 20827-20840. PMLR (2022)
[35] Ulyanov, D., Vedaldi, A., Lempitsky, V:: Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1 (tt::1607.088000 (2016)
[36] Van Amersfoort, J.. Smith, L., Teh, Y.W., Gal, Y:: Uncertainty estimation using a single deep
deterministic neural network. In: International conference on machine learning. pp. 9690-9700. PMLR (2020)
[37] Wang, H., Li, Z., Feng, I.,, Zhang, W.: Vim: Out-of-distribution with virtual-logit matching. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp.
4921 4930 (2022)
[38] Wang. S., Fu, X., Ding, K., Chen, C., Chen, H., Li, J.: Federated few-shot learning. In: Proceedings of the 29th ACM SIGKDD Conference Knowledge Discovery and Data Mining. pp. 2374-2385 (2023)
[39] Wen, Y., Vicol, P.. Ba, J., Tran, D., Grosse, R.: Flipout: Efficient pseudo-independent weight perturbations mini-batches. arXiv preprint arXiv: 1803.04366 (2018)
[40] Xu, H., Pang, G., Wang, Y., Wang. Y.: Deep isolation forest for anomaly detection. IEEE Transactions on Knowledge and Data Engineering 35(12), 12591-12604 (2023)
[41] Yang, J., Shi, R., Wei, D., Liu, Z., Zhao, L.,, Ke, B., Pfister, H., Ni, B.: Medmnist v2-a large-scale lightweight benchmark for and 3d biomedical image classification. Scientific Data 10(1), 41 (2023)
[42] Yang, J.. Wang, P., Zou, D., Zhou, Z., Ding, K., Peng, W., Wang, H., Chen, G., Li. B., Sun, Y., et al.: Openood: Benchmarking generalized iitoo--distribution detection. Advances Neural Information Processing Systems 35,
(2022)
[43] Yang, J., Zhou, K., Li, Y., Liu, 7.: Generalized duiti--dist detection: A survey, Interna- tional Journal of Computer Vision 132(12), 5635-5662 (2024)
[44] Yu, S., Hong, J.. Wang. H., Wang, Z., Zhou, J.: Turning the curse of heterogeneity in federated learning into blessing for distribbbttoonn detection. In: 2023 International Conference on Learning Representations (2023)
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes. Our main contributions detailed in Sec. 1 See Sec 3 for our method description and See for experimental evidence.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Yes. Please see See 5 where we discuss limitations and future research plans.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification: Yes. Sec includes Proposition with its proof sketch. The full proof can be found the Appendix. Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Yes. Secß includes detailed description of our method. See includes description of the hyperparameters, we also use publicly available datasets. A more detailed hyperparameter description is included in the Appendix. We also upload the code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
15

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We use publicly available datasets as described in See We also upload the code and once the review process finished, we will open source the code and instructions.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to reproduce the results, See the NeurIPS code and data submission guidelines (https //nip.cc. wttpcccccppcccccceececc /eodedeee/ooeebbiieeeeee wwteiissisonnniiieeoooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes. Sec describes the training details, including hyperparameters, type of optimizer and dataset splits. A more detailed description about the training and test details is provided in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
We now report mean and standard deviation over three random seeds for our main experiments (added after rebuttal). Table I we show mean and standard deviation. For the larger Histopathology table (Table 2 we show means in the main text and include the corresponding standard deviations in Appendix (eanndix[[[aabblll (Tables] [3 and 14). Additionally, our method is evaluated across multiple diverse medical datasets and tasks (Sec.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
16

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (c.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Sec. describes the type of GPU that was used for our experiments. With this GPU type, all of our experiments can be executed.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental rums as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
Yes. Our research conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
17

---

Our research is intended to have potential positive societal impact in medical imaging as discussed in the paper. The paper mentions possible negative impacts OOD methods do not work.
Justification:
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out, For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do expect high risk of misuse of this work.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes, we credited them properly.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
18

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperssi thoode com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our work does not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the datasct/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and sereenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing and human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our paper does not involve crowdsourcing and human subjects
Guidelines:
19

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: LLMs were not and important, original, or non-standard component of our research.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM for what should or should not be described.
20

---

A DIsoN and CC-DIsoN:
Algorithm
In this section, we present pseudocode for our DIsoN method and its class-onnitional variant, CC-DIsoN. Algorithm| describes the code that is executed the Source Node, while Algorithm| runs on the Target Node. Lines specific to CC-DIsoN are highlighted in blue. The initialization step occurs in lines 4-5 of Algorithm and line 5 of Algorithm 2 The local updates step performed lines 7-12 (Source) and 7-8 (Target), followed by the aggregation step on line 14 on the Source Node. If the convergence criteria (Def. 3.1) are not met, another communication round starts.
The algorithms demonstrate that implementing CC-DIsoN requires only minor changes compared to DIsoN: before initialization, the Target Node predicts the target sample's 's class using the rre-trained model and sends it to the Source Node (lines 2-4 of Algorithm 2 The Source Node then filters its training data to sample batches from the predicted class only (line of Algorithm Tp.
Algorithm Source Node: DIsoN CC-DIsoN
1: function SOURCENODE(D,, (Pre)
2: if CC-DIsoN:
3: receive ý from target
4: 05 <- (pre; 0h rand; ((0) 0h)
initialize global model
5: send ((0) to Target; As 0(0)
6:
for 1 to R do
b communication rounds local updates on Source
7:
for e = to do
8:
if CC-DIsoN:
9:
B₂ [(xx ED, M/ 0
filter B. on predicted x; class
10:
else
11:
sample B₃ ~ D,
12:
0s + 05 - 1/ V [T. Σx,en, L(0s; X., 0)]
13:
receive 0r from Target
14:
aggregation: g(r) + ass + (1 - (1-a) Ar
> Eq. 2 Test criteria 2 (Def. 3.1)
15:
SourceConverged + ()nverged(Otn), D.)
16:
send ((r) and SourceConverged to Target
17:
Os (())
update Source model for next comm. round
Algorithm 2 Target Node: DIsoN CC-DIsoN
1: function TARGETNODE(TA, Mpre; R)
2:
if CC-DisoN:
3:
+ arg maxe (Mpre(Xx)))
4: send source
5: receive ((0) from Source; Or f- 0(0) init. Target model with global model b communication rounds local updates on Target
6:
for = to R do
7:
for e = to do
8:
0T 0T Ve, ((0; xx,
9:
send 0r to Source
10:
receive updated ((r) and Sour ceCovveree from Source
11:
if converged (0()) ((0r) xx) and SourceConverged: Test crit. & 2 (Def. 3.1) break
13:
Or + ((r)
update Target model for next comm. round
14:
return SpIeoN(Xi) =
B Connection between DIsoN and Isolation Network: Proof Proposition 3.1
This section provides the full proof Proposition 3.1, which states that if we set and the aggregation weights in Eq. 2 accordingly, DIsoN becomes equivalent the centralized Isolation
21

---

Network. This result also demonstrates how the aggregation weight a implicitly controls the number of times the target sample oversampled (N) in Eq. 1 for the Isolation Network.
We begin by restating Proposition 3.1:
Proposition 3.1 (DIsoN and Centralized Isolation Network Equivalence for E 1). Let Ocent be the model parameters from our centralized algorithm and Adec be the parameters from DIsoN. Let each site perform one local SGD step (E 1) with learning rate π,, and aggregate with
|B,l R. I
3 |B,++N' Then the decentralized update equals the centralized one:
N
g(++1) ((r) - n((=gs(0())) + gr (( = = dec
-7 (7(60())) = =0(xt1)
where Is (0) [8.1 Ex,en, VoL(O; 0), gT( VoL(0; and N is the number of times X2 is oversampled in the centralized version.
Proof. Recall that B, is mini-batch of [&l source samples and let the target sample x be oversam- pled N times in the Isolation Network. The local gradient the source node Is and on the target node 9r are defined as:
Is ( = B₂
(oL(0;x,,0) 0),
(0) Vo x1; 1);
X,EB,
and the centralized mini-batch gradient of the Isolation Network is defined as:
(e)
1
Σ
VeL(0:x,.0) + VoL(0;xx,
(A.1)
B. + x,EB,
Step 1. Local parameter updates the Source Node and Target Node. After SGD step
(E = 1) with learning rate 7) we have,
g g(r) (gs()())),
0 = -(()) -1gr((()))
(A.2)
|
Step 2.
Weighted aggregation. Using the aggregation Eq. 2 with
=
B. + N
B =
N
B. + N B=1- a gives
g g(r+1)
= a( (gss) + (((((r) - -99T) (by (A.2))
(a + 3)0(r) n gT)
g(0) - n + 3 gr).
(A.3)
Step 3. Centralized gradient of the Isolation Network.
|B.
Σ
VoL(0;x..0)
Bal + N Σ (oL(O:X,. x,EB,
a = [.
+
B3
x.EB,
N
gT
BB + N VoL(O:Xx,1) 1).
Adding the two terms reproduces (A.1):
Σ VoL(O;x,,0) 0) NNNN VoL((;; 1)) 9c(()).
a gs + 3 gT B, +
(A.4)
X,EB,
Step 4. Equality of parameter updates. Substituting (A.4) into (A.3) results in g(r+1) = g(r) - ?)()) = (r++1)
(A.5)
which is exactly the centralized Isolation Network SGD update. This shows that the decentralized
and centralized updates are equivalent when E = and the aggregation weights are chosen as
BBll |B,+N 3 N
22

---

Table 4: Class-wise dataset splits used in our experiments, showing the total number of ID images per class, splits into pre-training and ID test sets, OOD detection task, number of OOD test samples, and image resolution. For Histopathology, we report near-OOD (domains 2-7) and far-OOD (CCAgT, FNAC 2019) tasks separately.
Dataset
Class
Total: ID Images
Pre-train: Images
# ID Test
OOD Task
OOD Test Img. Size
Desmatology
Not Nevas
Nevus
571
832
745
516
87
55
black ruler
251
224x224
Nermal
Ulrrasunnd
Benign
text
224x224
Malignant
157
20
Chest X- X-Ray
No Cardiomegaly
Cardiomegaly
21557
1788
20634
1711
923
77
pacemaker
1000
22
Matotic
Histoputhology
Neither
Impester
1063
663
940
123
far OOD: CCApT FNAC
mean-0DD: domains 2-7
per domain
50x50
C Dataset Details.
We evaluate DIsoN on four medical imaging datasets with two main OOD settings: (i) artifact detection (Dermatology, Breast Ultrasound, Chest X-ray) and (ii) semantic/covariate shift detection (Histopathology). In this section, we provide more detailed dataset information with class-wise splits used in our experiments. Table 1 reports reports the total number of images per class, splits into pre- pre-traiming and ID test samples, the number of OOD test samples, the OOD task and the image size.
For Dermatology (1403 artifact-free ID images) and Breast Ultrasound (552 artifact-free ID images), we follow the benchmark setup from [ using manually annotated artifact-free images for pre- training and ID testing, and ruler/text annotation artifacts OOD. The Chest X-ray dataset 23,345 frontal-view seans without support devices as ID data, following the setup from [2], with scans with pacemakers OOD artifacts. In all three datasets, the ID data is split 90/10 into pre-training for the main task of interest and ID test sets.
For Histopathology, similar setup to the well-known OpenOOD benchmark [42), but is specialized on medical images. OPenMIBOOD demonstrates that many state-of-thee OOD detection methods fail to generalize to medical data. The dataset is split into multiple domains. The dataset consists of Hematoxylin & Gosin-staineds histology patches grouped into multiple domains. Following the protocol in [11], use the training split of domain 1a (1,896 ID images) for pre-training and its test split (251 images) the ID evaluation set.
follow the recently published OpeNMIBOOD [1I) benchmark, which uses
Domains 2-7 are treated as near-OOD and include seven distinct cancer types (breast carcinoma, lung carcinoma, lymphosarcoma, cutaneous mast cell tumor, neuroendocrine tumor, soft tissue sarcoma, and melanoma), from both human and canine. These domains introduce semantic shifts in cell types as well covariate shifts due to differentt staining protocols and imaging hardware. For OOD, two external datasets are used, that introduce a strong semantic shift: CCAgT [[]. which uses AgNOR staining, and FNAC 2019 (32), which uses Pap staining, both differ significantly from the Hematoxylin & Eosin staining used for ID data and are also used for different medical applications.
D Training Details & Hyperparameters
In addition to the training details provided in Section 4, this section presents further training details and hyperparameters.
Table 5: Pre-training hyperparameters. These settings are used to train the main classification task before initializing DIsoN. LR: learning rate; BS: batch size.
Dataset
Main Task
Arch.
Optim.
Epochs
LR
BS
Dermatology
Nevus Non-Nevus
ResNet18
Adam
750
1 10-3
Breast Ultrasound
Normal / Benign/ / Malignant
ResNet18
Adam
1000
1 xx 10-3
32
Chest X- X-Ray
Cardionegaly vs. No Cardiomegaly
ResNer18
Adam
500
1 10-3
32
Histopathology
Mitotic/ Imposter / Neither
ResNet18
SGD (0.9 momentum)
300
5 10-4
128
23

---

Table 6: Training byperparameters used for DIsoN experiments. Overview of architecture, optimizer, learning rate (LR), and batch size (BS) used to train DIsoN for each dataset.
Dataset
Arch.
Optim.
LR
BS
Dermatology
Re
Adam
1 x 10-3
16
Breast Ultrasound
ResNet18 8
Adam
1 x 10-3
8
Chest X-Ray
ResNet18
Adam
3 10-3
16
Histopathology
ResNet18
Adam
1 10-
16
D.1 Pre-Training for the Main Task of Interest
We pre-train a ResNet18 on each dataset's main classification task. DIsoN models use Instance Normalization, as described in Section 3. For baseline methods, we use Batch Normalization (BN) to ensure a fair comparison, since they assume BN in their original setups. training settings, using the dataset split in the "Pre-train" column of Table 4 For the Histopathology dataset, we follow the protocol from [TI], initializing with ImageNet- Ik [8] pre-trained weights to reduce training time, and use SGD with momentum. All other models are trained from scratch using Adam.
J the
D.2 Training DIsoN
This subsection provides a more detailed description of the DIsoN training setup introduced in Section 4. Table summariiss the training hyperparameterss. The Source Node uses the pre-training split from Table 4 as its training data. As described earlier, we set the number of local iterations per communication round such that it approximately matches one epoch over the training data. To limit runtime, we also use a maximum number communication rounds for each dataset. If convergence is not reached within this limit, assign the maximum round Rmax as the OOD score. We use Rmax = 300 for Dermatology and Ultrasound, and Rmax 100 for the longer-running Chest X-ray and Histopathology datasets.
Image Augmentations. As described in Section 3, we apply standard stochastic image augmenta- tions across all datasets. The augmentations for each dataset are mostly identical, with only minor datasct-specific adjustments. For Histopathology, due to the smaller image size, reduce the rotation range and leave out random cropping. For Breast Ultrasound, we replace random cropping with color jitter. The full set of augmentations listed below:
Random rotation: 115 (use + for Histopathology)
Random crop: 224 224 with padding=25 (applied Chest X-Ray Dermatology)
Color jitter: brightness=O.1, contrast=0.1 (applied to Breast Ultrasound)
Convergence Parameter Choices for Estab and 7. The parameters Estab and 7 were chosen to capture the convergence behavior of the isolation process. Specifically, Estab acts patience parameter, conceptually similar to early-stopping criteria in learning-rate schedulers, requiring correct classification of X; for several consecutive rounds to ensure stable convergence. We set Estab 5, commonly used value schedulers that effectively captures convergence. The confidence threshold 0.85 ensures that the model learned to separate the target sample with sufficient confidence. This value follows standard practice uncertainty-be values consistently produced stable and reliable results, so they were kept fixed across all datasets to avoid dataset-specific tuning or overfitting.
literature [18]. In preliminary experiments, these
Effect of partial Fine-Tuning We also examined whether freezing parts of the network could be beneficial for the isolation task by simplifying optimization. Freezing the backbone, however, assumes that pre-trained feature extractor already provides sufficiently expressive embeddings to distinguish ID from OOD samples. We found this assumption too restrictive: in early experiments, partial fine-tuning of the network (e.g., only the head last block) led to worse OOD detection performance than fine-tuning the entire model.
24

---

E Accuracies for Caas-Conditional Sampling
Additional Details: Ablation Study for Impact of Predicted Class
CC-DIsoN uses predictions of a primary model Mpre- deployed for a task of interest (e.g., car- diomegaly classification in X-rays; see Sec. 4. for class-conditional sampling. This enables compar- ing a target sample to the most relevant subset of training data (its predicted class) rather than to all training samples. Our ablations on the effects of class-conditioning (Tab, 33) demonstrated that class- conditional sampling improves OOD detection performance, while DIsoN without class conditioon still performs well (see comparison between Tabs. B and DT
To further analyze the influence of class-condiiioning and how misclassifications affect the results, we provide here a more detailed analysis of the ablation study on predicted class accuracy introduced in Section 4.1 We first measure the classification accuracy of Myre on ID, OOD, and combined target samples, along with the corresponding CC-DIsoN AUROC scores (Tab. 7 Typically, especially in safety-critical domains such as healthcare, high ID accuracy is requirement for deployment. is also expected that classification accuracy decreases on OOD samples due to domain shift. Comparing these accuracies with OOD detection performance shows no clear correlation, for example, the Dermatology dataset achieves the highest AUROC (89.5) despite the lowest classification accuracy (63.0%). This indicates that CC-DIsoN's OOD detection performance does not depend on perfectly accurate predicted classes but rather on the data characteristics.
Table Classification accuracy of the primary models on ID. OOD, and combined target samples, together with CC-DIsoN OOD detection performance (AUROC from Tab. [T.
Dataset
ID Acc. (%) OOD Acc. (%)
ID&OOD Acc. (%)
CC-DIsoN AUROC
Dermatology
77.6
54.0
63.0
89.5
Chest X-Ray
93.0
70.8
82.2
84.9
Ultrasound
78.6
61.4
64.8
65.6
MIDOG (near+far)
77.3
68.6
69.2
76.0
In Section 4.1 and Tab. 3p, we further investigate the effect of incorrect predicted classes on class- conditioning in controlled settings. Here, we describe these settings in more detail. We designed three experiment variants for Dermatology, Breast Ultrasound, and Chest X-ray by manually assigning wrong predicted classes from the primary model to specific target samples:
All targets mislabeled (ID + OOD wrong): All target samples, regardless of whether they are ID or OOD, are assigned random incorrect class. This represents the extreme case of a completely inaccurate primary model (0% Accuracy), which would not be realistic in deployment but interesting setting to obtain insights.
2. Only ID targets mislabeled: All target ID samples are assigned incorrect classes (0% Accuracy on ID), while OOD targets use their predicted classes. Again, this represents an impractical scenario for deployment, since model with 0% Il) accuracy would not be used in real-world, but useful to study misclassification behavior. Central motivation for class- conditioned sampling was to make the isolation task more difficult for ID target samples, as they should be harder to isolate from source (training ID) samples of the same class, as they share common visual features. This should lead to slower convergence and increase the difference than when separating OOD samples from source (training ID) samples, even of the same class, which is faster due domain-shift. If we instead compare ID target samples to source (training) samples from the wrong class, the isolation will be casier, resulting in worse OOD detection.
3. Only OOD targets mislabeled: All OOD target samples are assigned incorrect classes (0% Accuracy on OOD), while ID targets use their predicted classes. This mimics situation where domain shift causes the primary model to misclassify OOD samples, and gives us insights what would happen in the most extreme cases of domain shift. Since OOD data already differ visually from the source distribution, assigning and comparing them with unrelated classes can make their isolation even easier, and we expect an improvment in detection.
25

---

The results in Tab. confirm these expectations. Performance decreases when ID targets are misla- beled, where class-conditioning most important, while slightly increasing when OOD targets are mislabeled. Even in the extreme "all wrong" case, CC-DIsoN remains competitive and outperforms the strongest baseline (Tab. 3D) ) in all but one dataset. This shows that CC-DIsoN's ' improvement does not rely perfectly accurate class predictions and that the method remains robust to moderate number of misclassification errors. Overall, comparison with visually similar source samples is beneficial for DIsoN and provides stable OOD detection performance even when predicted classes are not perfect.
F Ablation Study: Further Investigation of Image Augmentations
In Section 4.2, we showed that applying image augmentations during DIsoN training improves OOD detection across Dermatology, Chest X-ray, and Breast Ultrasound. Here, extend this ablation study by comparing four augmentation settings: no augmentation, augmentations only on the Source Node data, only the target sample, and both nodes. Fig. 6 reports AUROC and FPR95 for all settings. Applying augmentations on both nodes consistently performs best. Interestingly. for Dermatology, applying augmentations only on the target sample gives best AUROC overall. We can also see that target-only augmentation outperforms source-only augmentations in most cases. One exception is Ultrasound FPR95, where source-only yields better results. These findings further strengthen our finding of the importance of regularization on especially the target sample to prevent fast memorization of superficial image-specific features, and encourage learning more meaningful features for differentiation of ID and OOD data, as discussed in Section 4.2.
AUROC by Aug Setting
FPR95 by Aug Setting
80
Augmentation
80
Source Only
60
Target Only
60
Yes
AUROC (%)
FPRO5
40
20
20
0-
Chest Chest X-Ray
Dermatology
Ultrasound
Chest X-Ray
Dermatology
Ultrasound
Figure 6: Effect of applying augmentations on different nodes for DIsoN. AUROC (left) and FPR95 (right) across three medical datasets under four augmentation settings: no augmentation, aug- mentations only on source node, only on the target sample, and on both. Applying augmentations on both nodes performs best overall. Target-only augmentation outperforms source-only in most settings, highlighting the importance of regularizing the target sample during isolation.
G Runtime Statistics
Figure (a) in the main paper reports the average number of rounds required for the isolation task to converge. To provide more detailed view of runtime variability, report additional statistics in Table 8 including the 25th percentile, median, and 75th percentile of convergence rounds for both ID and OOD target samples using default a = 0.8. Reporting these quantiles helps practitioners assess expected runtime under different ID/OOD ratios. Since wall-clock runtime depends hardware, we report the number of rounds hardware-inde sendwarr-indeeeddent measure. OOD samples generally converge in fewer rounds than ID samples, consistent with their easier separability from the source distribution.
We further analyze how the runtime distribution changes with a. Tables 9 consistently reduces the number of rounds required for convergence. Importantly, OOD samples
show that decreasing
26

---

Table Quantiles (25th, median, 75th percentile) of convergence rounds for ID and OOD samples across datasets. OOD samples converge in fewer rounds and show lower variability, while ID samples display higher variability due to within-distribution complexity.
Dataset
Sample Type
25th Perc.
Median
75th Perc.
Dermatology
ID
68.0
126.5
184,0
OOD
22.0
28.0
40.0
Ultrasound
20.0
29.0
100.0
OOD
16.0
20.5
32.0
Chest X-Ray
ID
28.0
41.0
92.0
OOD
16.0
20.0
26.0
Histonathology (near OOD)
ID
44.3
59.5
74.0
OOD
34.0
44.0
57.0
converge faster across all a, while ID samples exhibit broader variability due to wuthin-distribution complexity. Note that, as described above, score when convergence was not reached. For these additional runtime experiments, we set a lower maximum number of rounds for the Dermatology dataset from (Rtma 300) (Rmax 150) in order to reduce computational cost. For a 0.90 and a = 0.95, the majority of ID) samples did not converge before reaching this limit. Therefore, the 25th percentile equals Rmax those cases.
the maximum number of rounds Rmax the OOD
Table Distribution of convergence rounds (25th, median, 75th percentile) for Breast Ultrasound across values. Lower accelerates convergence, while OOD samples consistently require fewer rounds than ID samples.
Sample Type
25th Pere.
Median
75th Perc.
0.95
ID
39.8
50.0
300.0
OOD
33.0
42.0
59.0
0.90
ID
25.0
39.5
297.0
OOD
22.0
27.0
42.0
0.50
ID
13.8
17.0
26.3
OOD
13.0
15.0
19.0
0.40
ID
11.0
13.0
17.0
OOD
11.0
14.0
17.0
Table 10: Distribution of convergence rounds (25th, median, 75th percentile) for Chest X-Ray across values. Lower a reduces the number rounds required for convergence, with OOD samples converging faster and more consistently than ID samples.
Sample Type
25th Perc.
Median
75th Perc.
0.95
ID
51.0
58.0
75.0
OOD
44.0
48.0
0.90
ID
26.0
35.0
100.0
OOD
19.0
23.0
27.0
0.50
ID
20.0
27.0
51.5
OOD
14.0
20.0
0.40
ID
15.0
17.0
23.0
OOD
11.0
13.0
15.0
H Practical Deployment Considerations And Runtime Comparisons
In many medical imaging workflows, real-time inference not required. Diagnostic scans are often reviewed hours days after acquisition due to limited clinical staff availability, and in some cases, turnaround times can extend to several weeks (3I). Within such workflows, Al models can operate asynchronously, processing scans during this waiting period. Therefore, inference times of several minutes per sample are acceptable when they provide more reliable detection of OOD samples, and similar latency is acceptaale in many other non-real-time workflows outside of healthcare. DIsoN was
27

---

Table 11: Distribution of convergence rounds (25th, median, 75th percentile) for Dermatology across values. Convergence becomes substantially faster as a decreases, and OOD samples consistently require fewer rounds than ID samples.
Sample Type
25th Perc.
Median
75th Perc.
0.95
ID
150.0
150.0
150.0
OOD
72.0
87.0
122.0
0.90
ID
150.0
150.0
150.0
OOD
37.0
44.0
66.0
0.50
ID
16.0
17.0
21.0
OOD
13.0
15.0
16.0
0.40
ID
5.0
11.0
15.0
OOD
5.0
11.0
specifically developed for such workflows, where reliability and privacy are prioritised over speed, and these latencies remain practically acceptable. To provide complete view of computational cost, we report both communication rounds (Appendix G) and wall-clock runtimes for comparison with other methods. Together, these results help practitioners assess whether DIsoN is suitable for their deployment setting.
DIsoN introduces additional computation at inference time (approximately 40 s-4 min per sample in our experiments on an NVIDIA RTX A5000), as each target sample requires iterative isolation training until convergence. In contrast, most compared baselines complete inference within milliseconss, since they rely on single forward pass of CNN backbone followed by lightweight post-processing to compute an OOD score (e.g., distance computation, linear projection, or tree traversal), Note that iForest, conceptually the closest to DIsoN, cannot be trained jointly source and target data due to privacy constraints, unlike DIsoN. Following the original iForest formulation 25 we pre-train it only on the source (ID) data and then apply it with single forward pass per target sample to determine whether it is or OOD, which explains its much lower runtime.
While this makes the baselines faster, DIsoN's s iterative optimisation results in more reliable separation between ID and OOD samples, as demonstrated in our results. Average per-sample runtimes for representative baselines on the Ultrasound dataset are reported in Table
Table 12: Average inference time per target sample (seconds, NVIDIA RTX A5000). Baseline methods compute OOD scores after a single forward pass of the pre-trained network followed by lightweight post-processing, whereas CC-DIsoN performs iterative optimisation until convergence, which increases latency but yields more reliable OOD detection performance.
Method
Avg. Runtime
MDS
0.006
ViM
0.005
iForest
0,015
CIDER
0,007
PALM
0.007
CC-DIsoN (ID samples)
155.2
CC-DIsoN (OOD samples)
90.9
Single-Sample vs. Batch Processing. DIsoN is designed to process one target sample at time, as it learns a decision boundary that separates a single target sample from the training distribution using convergence behaviour as the OOD score. This design aligns with many real-world scenarios, especially in healthcare, where each patient scan is acquired and analysed individually rather than in large batches. If batch processing is feasible, multiple DIsoN instances can be executed in parallel to use available hardware efficiently. In our experiments, up to eight DIsoN runs could be executed concurrently single NVIDIA RTX A5000 (24 GB VRAM). Although not traditional deep learning batch-tensor setup, this parallel execution allows scaling for pre-collected datasets.
Network Conditions and Communication Latency. DIsoN involves single target client communi- cating with a source node, making it simpler and more robust than large-scale multi-client federated systems. Network delays volatility only affect wall-clock time, not the number of communication
28

---

rounds or the quality of convergence. In practice, network disconnections can be handled through standard retry or timeout mechanisms used in distributed systems.
Overall, these experiments and considerations demonstrate that while DIsoN trades inference speed for reliability, its runtime remains predictable and manageable across deployment conditions, making it suitable for safety-critical settings where performance is prioritised and real-time inference is not required.
I Additional Multi-Seed Results for Histopathology
Due to space limitations, Table 2 reports means over three seeds only, the standard deviations are omitted. In this section, we provide the complete tables including standard deviations. Tables [3] (AUROC 1) and (FPR95 4) list the mean + std over three random seeds for the Histopathology dataset. For the Avg. columns, the + values are computed as the sample standard deviation across seeds of the per-seed macro averages.
Table 13: Complete AUROC (f) results for the Histopathology dataset (MIDOG). Each cell shows mean + standard deviation over three random sceds. This extends the means reported in Table 2 with standard deviation.
near-OOD
far-OOD
Methods
2
4
5
6a
6b
7
Avg.
CCAgT
FNAC
Avg.
MSP
54.1s58
47.4=13.
55.7=15
59.1=12
54.8*06
45.6+9.
56.1#53
53.3435
78.4+14
82.7=43
80.6+14
MDS
67.4a30
67.2
65.7
61.2a45
60.6116
55.6459
51 11001:11
61.2218
87.
86.62118
86.8=105
57.9183
52.6188
57.5144
60.2,24
57.9
51.218.
55.3144
16.
74.6
82.1.ne
78.3171
VIM
68.0236
66. 66.7+26
66.2#12
63.5a14
61.9209
55.7470
54.9
62.4215
92.5a34
92.6=52
92.64440
37.9158
38.7148
39.7.18
41.6265
39.5
40.4143
46.4194
40.6219
27.7..64
32.3
30.01302
CIDER
71.4a33
65,62411
55 5:55
64,2s25
$7.4
47.7=100
64.1
60.9=22
82.5a44
95.4u34
89.0s2.
PALM
73.5225
59.2_111
66.3115
64.3211
41.8145
61.6233
61.3154
97.0.44
99.6.nm
98.3
CC DisoN
75.4115
79.5111
72.6106
63.0g
64.0123
70.7128
61.81:8
69.6107
98.3200
98.4.m
98.31a.
Table 14: Complete FPR95 (4) results for the Histopathology dataset (MIDOG). Each cell shows mean + standard deviation over three random seeds. This extends the means reported in Table 2 with standard deviation.
Methods
near-OOD)
far-OOD
Avg.
CCAgT
FNAC
Avg.
MSP
94,8142
93.4
89.4
93.5:00
94,7114
95.51:4
92.4:35
93.4119
52.21100
63.6188
57.9154
MDS
80.0:30
79.8+1.
81.9:50
87.4=34
87.4=28
89.5+24
91.6+29
85.4+27
43.6+220
39.6+286
41.6=2311
IDBD
83.0:72
81.4188
88.7:26
89.0.22
85.9:99
86.2:98
92.2113
86.6148
58.31103
63.7:29
61.0i53
VIM
79.8
77.8144
77.7
86.9111
87.3u07
91.0u11
1221
84.8176
29.
27.0
28.0
iForest
98.3120
96.9124
97.711
97.5a12
98.0a17
97.7
95.7112
97.4114
99,22n4
96,8141
98.0:21
CIDER
84.6144
89.0118
91.5 M
89.8
92.21m
96.5109
88, 88.2
90.2174
77.2
18.3
47.8162
PALM
78.1
90.3+18
77.7
93.6204
90.8=12
97.6214
93.643.
88.8443
21.8
1.5525
11.6:11
CC-DIsoN
78.8+44
61.7+36
79.7=27
89.4=24
85.3=52
79.9=50
92.0=a0
81.0+12
4.8:18
4.0=14
4.4a16
Qualitative Visualization of DIsoN Isolation Process
In this section, we provide qualitative visualizations of the DIsoN isolation process over commu- nication rounds. Figure shows PCA projections of a target sample and the source data after communication rounds O, T5, and 25. Each row corresponds to a target image from the Dermatology dataset. the OOD example (top row), the target sample (orange star) rapidly drifts away from the source distribution (blue points), achieving clear separation already after communication round 15. In contrast, the ID target (bottom row) remains closely clustered with the source data, showing that is harder to isolate. This side-by-side comparison shows DIsoN's core idea/motivation: OOD samples isolate quickly, while ID samples require more rounds.
29

---

After Communication Round 0
After Communication Round 15
After Communication Round 25
Target Sample
Target Sample
Figure 7: Isolation Process over Communication Rounds. PCA projections of source samples (blue) and larget sample (orange star) after communication rounds 0, 15. 25 of DIsoN training. The top row shows a target sample that is OOD, the bottom row shows target sample that is II) (from the Dermatology dataset). The OOD sample becomes separated already after round and clearly isolated by round 25. In contrast, the ID sample remains entangled with the source distribution throughout, demonstrating that it harder to isolate.
30


Reviews:
[{"review_id": "Fsox1vZEzz", "reviewer": "Reviewer_SHNi", "summary": "The paper introduces an original OOD detection method, Isolation Network. The core proposition is that it would be beneficial to train a binary classifier to provide an estimation of how difficult, for example, it would be to isolate an OOD target test sample from an easily accessible training set, taking convergence speed as an OOD measure. The paper also introduces its decentralized analogue, DIsoN, which counters real-world complications which are encountered in application domains like medical imaging where data privacy and communication constraints rule out accessing OOD detection training sets.", "questions": "[Q1] In practical application, communication latency and volatility across both ends may have an impact on the efficiency of DIsoN. Has the paper discussed how such practical network conditions would have an impact on convergence speed and OOD detection efficacy?\n\n[Q2] The statistical significances of OOD detection results are significant for establishing method robustness. Although there is some warrant for computational cost for running multiple seeds on all configurations, would it be possible to report mean and standard deviation for significant experimental results such as comparisons between CC-DIsoN and best baseline between Tables 1 and 2? Reporting them would greatly enhance confidence of findings.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. The paper is written in a relatively fluent manner. Its structure is logically organized, and the language used throughout is clear and accessible, making it easy for readers to follow \n\n2. The paper is on OOD detection's practical challenges for the application of medical image analysis, where data transmission and privacy make it hard to maintain direct access to training data at deployment's side.\n\nWeaknesses\n\n[W1] DIsoN only processes one target sample at any given time. While its authors envision future development for handling multiple samples at once, that limits its effectiveness for most batch processing environments.\n\n[W2] DIsoN creates considerable computational overhead at inference time and can range from 40 sec to 4 min on each sample. Such latency can be a significant limitation on applications where fast or real-time inference is needed.\n\n[W3] The manuscript says that α=0.8 performs well on all experiments and validates its effectiveness via ablation experiments. It lacks, though, an indepth theoretical study to guide optimal α selection or an automatic selection mechanism.", "was_revised": true, "final_justification": "The paper presents practical value, and therefore we have given it a positive score.", "rebuttal": "Dear reviewer, thank you for the constructive and valuable feedback to improve our paper. We are glad you acknowledge the **novelty of the method as original** (\"an\noriginal OOD detection method\"), introducing the Isolation Network\n(ISoN), a novel algorithm that trains a model to separate a\ntest sample from training data, using convergence to determine\nif the test sample is OOD. We appreciate recognizing the\nvalue of our introduced decentralised algorithm (DISoN) for\n**addressing real-world practical challenges** (\"counters\nreal-world complications\") of OOD detection in privacy-sensitive settings like healthcare. We are happy you found our paper \"**logically structured**\" and\n\"**easy to follow**\".\nThe reviewer raised few points about computational\nefficiency, hyperparameter configuration, and confirmation of performance stability across seeds. We address your concerns\nbelow and have revised our paper accordingly.\n\n## \\[W1\\]: Processing one target sample at a time limits batch-processing.\n\nWe understand the reviewer's consideration that processing one target\nsample at a time may seem computationally suboptimal by not allowing\nbatch-processing. This is not entirely true, however, and computational resources can be optimized by parallelism, if\nrequired. We want to emphasize that many (if not\nmost) applications require inference on 1 sample at a time. For\nexample, 1 patient is scanned and the image needs AI processing\nto detect pathologies as soon as possible, without waiting for a batch\nof images from other patients. Similarly for applications like a user requests classification of a bird photo they just took. Thus, we designed our novel OOD algorithm with the requirement to process 1 target sample at\na time, by learning a decision boundary that\nseparates 1 target sample from the training data, using convergence as OOD score. \n\nThat said, if a deployment setting allows processing multiple datapoints simultaneously\n(e.g. a pre-collected batch of scans from multiple patients), multiple samples can simply be processed via separate instances of our algorithm in parallel. This isn't a traditional \"batch\" tensor in Deep Learning libraries, but enables parallel execution of independent processes, making full use of available hardware. In our experiments, the moderate model size allowed up to 8 DIsoN runs in parallel on a single NVIDIA RTX\nA5000 GPU (24 GB VRAM). Our\ncode includes scripts to run multiple samples in parallel. We aim to develop more batch-effective variants in Future Work. However, the current work is an original and already effective\nalgorithm that makes a novel contribution to knowledge, warranting\npublication to enable further exploration and extensions by the\ncommunity of this type of decentralised OOD detection framework.\n\nNeurIPS allows 1 extra page for post-rebuttal\nadditions to the camera-ready, we've added the above discussion to clarify. Thank you for the great point.\n\n## \\[W2\\]: Computational overhead at test time, suboptimal for fast or real-time inference\n\nAs the reviewer correctly notes, DIsoN introduces some latency per\ntarget sample at inference (40 sec - 4 mins /sample in our settings).\nIndeed, this novel algorithm is not intended for real-time inference.\nHowever, a lot of applications don't require\nreal-time inference, whilst a few seconds (or even minutes/hours) are\nacceptable to gain enhanced reliability by detecting unexpected OOD \nsamples. In many safety critical medical workflows,\ndiagnostic accuracy is prioritized over speed.\n\nCurrent medical workflows involve patient scans waiting hours/days before radiologists review them, due to limited personnel.\nSome scans can wait for a month! If AI is used to\nautomate the process or create diagnostic predictions to inform\nhumans, integration in the workflow is possible even if\nit needs minutes, or hours/days, as long as it is reliable. The same\napplies to many applications outside healthcare. Thus, we argue the\nmethod is applicable to a variety of applications, and this should not\nbe considered as a reason for rejection of the work.\n\nTo help practitioners assess DIsoN's suitability for their use case, we have **added a detailed runtime analysis to the\nAppendix**. The table below summarizes the min/max/median runtime in seconds (s) for ID and OOD samples, plus statistics of communication rounds (R). For brevity, we show only the Ultrasound dataset here. Full results for all datasets are in the updated Appendix.\n\n|Sample Type|Minimum (s)|Median (s)|Max (s)|Mean (R)|25th Percentile (R)|Median (R)| 75th Percentile (R)|\n|-|-|-|-|-|-|-|-|\n|ID|28.2|106.4|274.3|48.2|20.0|29.0|100.0|\n|OOD|18.7|72.8|212.1|28.1|16.0|20.5|32.0|\n\nWe also **added a \"practical considerations\" sub-section** to\nthe paper, incorporating these points. Thank you for raising\nthis point, we believe the added runtime and\nexplanation improves clarity.\n\n## \\[W3\\] Theoretical study to find optimal alpha hyperparameter.\n\n$\\alpha$ (Eq.2) is indeed a hyperparameter of the algorithm,\nconfiguring how much the global model should be updated according to the\nupdates from the source training database (Server) versus the\ntarget sample (target node). Like many ML methods hyperparameters, its value needs to be chosen empirically. We don't believe that deriving a theoretical framework for finding an optimal value is possible. In these cases, we believe ideally an algorithm should be relatively robust to choice of values, with similar values performing reasonably well across tasks (i.e., hyperparameter values \"generalizing\" in some sense). This is why we provided an empirical study for $\\alpha$ values (Fig.4.a), showing that across studied tasks, performance remains quite robust\nfor a wide range of $\\alpha$ values. We see little performance change for\n$\\alpha$ values $[0.6-1.0]$, which is a wide range considering\n$\\alpha$ is $[0,1]$. Purpose of this sensitivity-analysis\nwas not to show that value of $\\alpha=0.8$ is optimal, although we\nunderstand it may be misread like this. Instead, we wanted to\nprovide empirical evidence (in absence of theoretical)\nthat values in such a range are expected to be reasonable for new,\nunseen tasks, given that they are reasonable for all studied tasks.\n\nWe added explanation about the above in the interpretation of Fig\n4.(a) results, to inform readers about possible reasonable $\\alpha$ value choices. Given that many ML methods' hyperparameters are empirically chosen\n(without theoretical framework), and\nour algorithm shows reasonably robustness to the $\\alpha$\nvalue, we believe this does not constitute a strong weakness of the\nalgorithm.\n\n## Questions\n\n## \\[Q1\\] How (internet) network volatility may affect the method: This\nis a good practical question. Since the algorithm is designed to\n\"serve\" only 1 client per process, it's much simpler to manage\nthan multi-client decentralised systems (e.g. federated consortia with\n10s/100s of clients). If the network is slow/unstable, the server\nwould simply \"wait\" for their single client to respond/reconnect. As DIsoN's OOD score depends on successful communication\nrounds for convergence, physical network delays would\nonly affect **wall-clock time**,\nwhich cannot be avoided for services over the internet.\nFortunately, it would **not compromise convergence quality or\noptimization,** or the number of rounds needed for it. Thus our OOD\nscore is robust against this. If the client disconnects, the\nprocess can continue or restart when it's available\nagain (each training session is short, up to 4 mins in our experiments,\nso minimal time lost). Production systems would require timeouts/retry mechanisms, but this is standard practice in\ndecentralised systems engineering and beyond the paper's scope, which focuses on\nalgorithmic development. We added discussions about these practical considerations to the paper.\n\n## \\[Q2\\] Re-run multiple seeds to enhance confidence on performance:\nWe absolutely agree that statistical significance is important and\nshould be reported whenever possible. We want to sincerely thank the\nreviewer for acknowledging the computational cost (we wish\nevery reviewer did this!), which not all labs can afford, thus this\nconsideration helps with inclusivity in science. During the limited rebuttal time, we managed to run our CC-DIsoN and the strong SOTA method (PALM) with three seeds each, reporting the mean and standard deviations, as requested. Due to the large\nMIDOG table size, we only provide here the average scores for\nnear OOD and far OOD over all tasks (averages of means and stds). We\nupdated the full table in the paper:\n\n|Method|Avg. Near AUROC ↑|Avg. Near FPR ↓|Avg. Far AUROC ↑|Avg. Far FPR ↓|\n|-|-|-|-|-|\n|PALM|$61.3\\pm4.4$|$88.8\\pm2.9$|$98.3\\pm2.3$|$11.6\\pm19.1$|\n|CC‑DIsoN|$69.6\\pm1.7$|$81.0\\pm3.3$|$98.3\\pm0.1$|$4.4\\pm1.6$|\n\nMultiple seed runs for X-Ray, Dermatology, and Ultrasound:\n\n|Method|X-Ray FPR ↓|X-Ray AUROC ↑|Derma. FPR ↓|Derma. AUROC ↑|Ultra. FPR ↓|Ultra. AUROC ↑|\n|-|-|-|-|-|-|-|\n|PALM|$85.7\\pm9.2$|$65.4\\pm6.5$|$62.4\\pm0.8$|$77.3\\pm1.0$|$75.6\\pm5.5$|$59.4\\pm3.1$|\n|CC-DISoN|$61.9\\pm1.2$|$84.9\\pm0.9$|$42.5\\pm4.4$|$89.5\\pm1.5$|$73.2\\pm4.7$|$65.6\\pm1.2$|\n\nWe will run multiple seeds for all methods until camera-ready and update Tab2 and 3. Our method's average performance is higher than\nthe best baseline by a considerable margin and a small standard deviation over 3 seeds, providing\nfurther confidence in our findings. Though we note it should be interpreted with\ncaution given it is calculated with only 3 seeds.\n\nWe thank the reviewer again for the constructive feedback! It\nhelped us improve the paper and clarify these considerations. We\nhope this allows you and readers to better appreciate the strengths\nof our novel framework for decentralized OOD detection services, deviating from current approaches, that we believe should\nbe communicated to the community to inspire further developments. Please\nlet us know if any concerns remain. We are happy to engage in\nfurther discussion!"}, {"review_id": "WTBhLT7wqs", "reviewer": "Reviewer_vx6Z", "summary": "This paper presents DIsoN, a decentralized solution for out-of-distribution (OOD) sample detection in medical imaging. It leverages an Isolation Network to assess OOD samples based on convergence speed and enables collaborative training between source and target nodes without sharing raw data. Experimental results demonstrate that DIsoN achieves significant improvements in OOD detection performance. However, the article has some shortcomings in terms of expression clarity, selection of experimental datasets, and comparison methods.", "questions": "See Weaknesses\n1.\tImproving the clarity of the article's expression will make it easier for the audience to understand.\n2. The authors should include a discussion on few-shot federated learning. It is strongly recommended to incorporate a comparison with recent few-shot federated learning methods， such as [1]. This will provide a more comprehensive assessment of the effectiveness of your method.\n[1] FedAffect: Few-Shot Federated Learning for Facial Expression Recognition, ICCV Workshops, 2021.\n3. While the authors discuss the impact of /alpha on performance and speed in the ablation study, the values of |Bs| and N differ significantly across datasets. Could the authors provide more details? Clarifying the amount of data required for fine-tuning is crucial for the practical application of the method.\n4. Why did the authors choose generated data for OOD detection? Is it possible to use similar real-world datasets as OOD data to demonstrate the method's effectiveness on actual images, including natural images?", "limitations": "See Questions.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The overall structure of the article is well-organized, and the images in this article effectively illustrate the method's flow.\n2. The proposed method utilizes convergence rate as an indicator for OOD detection and incorporates collaborative learning to achieve online inference while ensuring privacy protection.\n\nWeaknesses:\n1.\tThere are some typos, including a misspelling of \"odd\" on line 45. Additionally, the sentences on lines 155-157 lack clarity and ambiguity, particularly regarding the data composition of SN and TN.\n2.\tWhile the paper highlights its distinctions from federated learning, the proposed setup resembles a two-client federated learning scenario, with aggregation occurring on a few-shot client. The derivation of formulas also represents a special case of FedAvg. The authors should discuss and compare their work with few-shot federated learning.\n3.\tThe datasets (Dermatology, Breast Ultrasound, and Chest X-Ray) used artificial artifacts as OOD data, resulting in significant differences. While Table 1 demonstrates the method's effectiveness, Table 2 shows that the advantages of the proposed method are not as pronounced with real medical images. The authors should analyze the reasons for this discrepancy and consider using real medical image data from different domains for OOD tasks to construct a more robust test dataset.\n4.\tOOD images often exhibit strong similarities to source domain images, making fine-tuning the backbone prone to overfitting. An interesting practical approach is to fine-tune the classification head instead. The authors could enhance the discussion around this practice to underscore the practical significance of the method.", "was_revised": true, "final_justification": "Most of the concerns have been addressed. I will thus increase my score to weak accept.", "rebuttal": "Dear reviewer, thank you for your review and feedback. We are glad that\nyou acknowledge our work is \"well organized\" and it achieves\n\"**significant improvements**\" in OOD detection. With all due respect,\nwe believe the reviewer has misunderstood important aspects of the work,\nespecially regarding the data for evaluation (misunderstood as\nartificial) and method performance (characterized \"not as pronounced\" in\nTable 2), which did not allow the reviewer to appreciate the work's\nsignificance. We address these points below, and sincerely urge the\nreviewer to reconsider the novelty of the OOD framework, strength of\nevaluation and overall significance of the work and assigned reviewer\nscore.\n\n## [W1] and [Q1]: Typos & Clarity of L155-157 regarding data composition in SN & TN\n\nThank you for raising this. We will carefully proofread and fix typos and unclear phrasing.\n\nWe appreciate you pointed out that L155-157 confused you regarding data\ncomposition on Source Node (SN) and Target Node (TN). We want to clarify, SN is defined in Sec.3.2, L177 and TN is defined in Sec.3.2, L180. Sec 3.1 and L155-157 do not relate to SN and TN, which are introduced only later for the decentralized setting in Sec.3.2. Instead, because the Isolation Network algorithm is entirely novel, we first derive its\ncentralized version in Sec.3.1, before describing the more complex decentralized version (Sec.3.2).\n\nThus L155–157 describe the composition of a training batch $B$ in a centralized setting (stated in L150). To disambiguate L155–157 as requested, we introduce math notation for the batch to complement the natural-language description. Specifically, a batch $B= B_s \\cup ( x_t^{(1)}, ..., x_t^{(N)} )$, where $B_s$ is a set of $|B_s|$ source samples from $D_s$, and all $x_t^{(i)}$ are $N$ copies of the same target sample $x_t$ (oversampling it to counter ID/OOD imbalance in training batches and avoid trivial optimization solutions, cf. L154). We will also clarify in Sec. 3.1 that this refers to a centralized setting and make its connection to the decentralized SN/TN setup clearer.\n\n## [W3] & [Q3]: Misunderstanding that Datasets used are synthetic and inappropriate\n\nThe reviewer states as main weakness (W2) \"The datasets (Dermatology,\nBreast Ultrasound, and Chest X-Ray) used artificial artifacts as OOD\ndata\" and \"consider using real medical image data from different domains\nfor OOD tasks to construct a more robust test dataset.\". They ask why we\nused \"[generated data]\" (Q3). Apologies but we believe this is a\nmajor misunderstanding.\n\nAll data we used, **including** X-Rays, Breast Ultrasound, Dermatology,\nare real-world clinical data, collected from real clinics, and widely\nused (see references to each clinical dataset in Sec. 4). All these OOD\nartifacts are naturally present in the original data. No external\nmodifications or synthetic patterns were added/used. Although the\nreviewer may be more accustomed to different types of domain shifts\n(e.g. whole image quality/appearance change due to different\ncamera/scanner), data with localised OOD artifacts as used herein (e.g.\npacemakers in Xrays) are often employed as OOD sets in literature, eg.\n[3]).\n\nWe believe this misunderstanding is from misinterpreting [2] and [3] that we cite as sources of the datasets. [2,3] separated which images in real clinical databases (e.g. CheXPert) have abnormal artifacts and which do not, to create OOD and ID subsets, which we use in this study. They [2,3] also created some *synthetic* data to perform their OOD analysis but we do not use them. We only use real ID and OOD images (see Sec.4, real skin rulers, ultrasound annotations, Xray pacemakers). We updated Sec.4 to state that we only use the real data.\n\nWe do not include Natural images that reviewer suggests, as it is outside our research agenda. We find that clinical data, with their unique challenges, are fantastic benchmarks and inspire innovation (novelty and performance acknowledged by other reviewers).\n\n## [W3 & Q3] Unconvincing performance on real medical data\n\nAs per the above, the reviewer misunderstood that Table 1 is artificial data, hence not convincing.\nThis is coupled by what we think is a second misunderstanding, when the reviewer states (W3) that Table 2 “shows that the advantages of the proposed method are not as pronounced”.\n\nWe point out that in Table 2, on average (Column “Avg”) we achieve improvement of 5.5% AUROC compared to the best SOTA method and lowest FPR95 for near-OOD artifacts.\nIn far-OOD we surpass 6 out of 7 methods, and only PALM surpasses our method by 1% AUROC.\n\nPerhaps the reviewer got confused when reading performance on Table 2,\nbecause FPR is best when it is low, but AUROC is best when high? We\nnoted this as up/down arrows in captions, but will state it explicitly\nin updated manuscript.\n\nWe hope this clarification shows our method's\nvery strong performance on real medical data, and we urge the reviewer to\nreconsider their overall opinion about our method, which may have been\naffected/disappointed due to the above misunderstandings.\n\n## [W2, Q2] Discuss/compare with Few-Shot Federated Learning\n\nThe reviewer raises a similarity with Few-Shot Federated Learning. While we agree DIsoN’s decentralized training shares similarities with 2-client Federated Learning (FL) (acknowledged in L123, “DIsoN uses decentralised optimization similar to Federated Learning”), there are fundamental differences. Objective, data composition, and output fundamentally differ from FL and Few Shot FL:\n\n**Optimization objective:** \n\na. **Few-Shot FL:** Trains a model for a specific task (e.g. disease classification) with the aim of **generalizing (predicting)** new unseen data. \n\nb. **DIsoN for OOD:** Generalization is not a goal. We optimize a model to separate 1 target sample from the source data to determine if it is OOD.\n\n**Data Composition:**\n\na. Few-Shot FL assumes clients have limited labels for different classes (and possibly unlabeled data, eg. FedAffect [REF1]).\n\nb. In DIsoN the Source Node has a big dataset, with all samples assigned label 0, indicating it is the *source*. The Target Node has only 1 sample, labelled 1 to indicate the *target*. Client has no labels of different classes.\n\n**Output:**\n\na. **Few-Shot FL:** Produces a generalizable model to make predictions on new data.\n\nb. **DIsoN:** Trained model is discarded. Only rounds to convergence is useful as OOD score.\n\nFew-Shot FL methods (including FedAffect [REF1] provided by reviewer) are not designed for this setting and OOD detection. Thus, comparison is not possible. We will add explanation about the relation with Few-Shot FL to Sec.2 to clarify for readers.\n\n## [W4] Fine-Tuning only Classification Head to avoid over-fitting.\n\nFreezing the backbone assumes that the pre-trained feature extractor\nhas sufficiently expressive embeddings to distinguish ID from OOD\nsamples. This may not be the case as the isolation task of a single\ntarget sample is distinct from common tasks where the model is trained\nto fit large data. Thus allowing optimization of all embeddings may be\nbeneficial. In fact, in early (not reported) experiments in Dermatology\nwe explored freezing parts of the network:\n\n- Head-only finetuning: Unstable, not appropriate convergence.\n\n- Head-Only + Final ResNet Block finetuning: AUROC: 84.23% / FPR95: 65.49%\n\n- Full Network finetuning (our method): AUROC: 87.96% / FPR95: 40.14% (both metrics better)\n\nFinetuning only the head did not work, while finetuning head and final\nResNet block was worse than fine-tuning whole model, confirming the\nabove intuition. So we did not pursue those further.\n\nWe **added a discussion about fine-tuning** the classification head to\nour paper to clarify this.\n\nFinally, we note that our work has introduced a novel\n(decentralised) OOD framework that trains an isolation network for\neach target sample separately, and uses convergence speed as OOD\nscore, not to make generalizable predictions on new data. Overfitting and regularization play a different role in this setting. We already explored one\nregularization method, stochastic feature augmentations (Sec.3.2, and experiments\nin Sec.4.2). While further exploration is interesting, the possible methods are\nendless, and there is no strong evidence that partial finetuning is the\nnext best step. We urge the reviewer to consider that the work has introduced a novel algorithm with promising performance and already considered analysis of a\nregularization method. Thus this should not be seen as a weakness, but\navenues for future work inspired by the intriguing nature of this novel\nalgorithm, which warrants its publication.\n\n## [Q3] Details why values of \\|Bs\\| and N differ across datasets?\n\nAllow us to clarify, as this may also be a small misunderstanding. $|B_s|$ refers to the number of source samples per mini-batch used at SN and is fixed to 16 for all datasets except Breast Ultrasound, where it is 8 due to smaller dataset size (see Appendix, Sec. D – Table 6). \nN denotes how often a target sample is repeated (L157) in an idealized centralized setting (see response to your [W1, Q1] as the misunderstandings may be related). N is not explicitly used in our experiments, because we only use the final decentralized DIsoN algorithm (not centralized), where N is not used. \nInstead, in the decentralised algorithm, N is implicit as a function of $\\alpha$. This is shown by Proposition 3.1 with $\\alpha = \\frac{|Bs|}{(|Bs|+N)}$, where defining value of $\\alpha$ implicitly defines N. Given that we use a fixed $\\alpha =0.8$ across all datasets (see Fig 4.a for choice of $\\alpha$), and $|B_s|=16$ except for Ultrasound with $|B_s|=8$, we can derive N=4 for all datasets except Ultrasound N=2. Therefore, N also does not vary except for the smaller ultrasound dataset.\n\n**References**\n\n**[REF1]** FedAffect: Few-Shot Federated Learning for Facial\nExpression Recognition, ICCV Workshops, 2021"}, {"review_id": "Zh6sLccoy1", "reviewer": "Reviewer_8S2S", "summary": "The paper proposes a non-parametric out-of-distribution (OOD) detection method that scores test samples based on the convergence time of a binary classifier neural network trained to distinguish the test example from the training dataset. The framework is built under decentralized assumptions and can be deployed in private environment, extinguishing data sharing and only allowing for parameter sharing between server and client.", "questions": "1. What if the predicted class is wrong? Could the authors elaborate how much it impacts CC-DIsoN? Showcasing robustness under misclassification could improve the method's trustworthiness.\n2. Which other methods listed in Table 1 employ a class-conditional criterion? Clarifying this point would help readers better understand how DIsoN compares to existing approaches.\n3. Why were $E_{stab} = 5$ and $\\tau = 0.85$ chosen? Providing more detail on the rationale behind these parameter choices would strengthen the methodology section.\n4. Could the authors please elaborate on why the isolation forest exhibits such low performance, with AUROC values even below 0.5 in Table 2? Although you cite isolation forests as inspiration for isolation networks, the results for iForest are poor. Some discussion or analysis regarding this discrepancy would be helpful.\n5. In Figure 4, only the average number of rounds to convergence is reported. Could you provide the full distribution of this metric? Reporting the median, as well as other quantile statistics, could be more informative, as there may be significant variability and some samples might be much harder to isolate than others. Presenting this additional information would help practitioners better assess whether the method is suitable for their specific applications.\n\nI'm willing to increase my score if these concerns are satisfactorily addressed.", "limitations": "Authors briefly mention limitations of their work. The proposed method is costly and lacks adequate runtime analysis.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths:\nThe paper addresses an important limitation of non-parametric methods for OOD detection: data privacy concerns. It also introduces a novel sample-wise optimization-based approach for OOD detection. The proposed method demonstrates strong performance on a relevant medical image classification benchmark.\n\n### Weaknesses:\n- Limited mathematical formulation. In Section 3, the preliminary formulation does not include labels or pseudo-labels, even though the method relies on class-conditional information. Providing a more comprehensive formulation would improve clarity.\n- Insufficient ablation study on class-conditional importance. There is a limited discussion of how class-conditional information contributes to the method's effectiveness.\n- Limited discussion of algorithm runtime. The paper only briefly mentions runtime in the conclusion. A more thorough evaluation of runtime would help practitioners determine the method's applicability and would strengthen the paper's contributions.\n- Minor: writing. The writing could be improved. Excessive use of passive voice makes some passages difficult to read.", "was_revised": true, "final_justification": "The authors have answered all points I raised during the rebuttal. I've read the other reviews and I don't see strong reasons to reject. Thus, I increase my score towards acceptance.", "rebuttal": "Dear reviewer, thank you for your effort reviewing this work and the\nconstructive feedback! We are glad that you found our work\naddresses an \"**important limitation of non-parametric methods for OOD\ndetection: data privacy concerns**\", acknowledging its originality as it\n\"introduces a **novel**\" method for OOD detection with \"**strong\nperformance**\". \nWe address your points below, and added these clarifications and results to the revised paper (using the extra camera-ready page).\n\n## [W1] Mathematical preliminary formulation does not include labels?\n\nApologies, we’re not 100% sure what caused this confusion. The\npreliminary math notation defines labels space $\\mathcal{Y}$ and\n(true) labels $y_i$ early on (Sec.3 L133-5), with predicted class (pseudo-label) defined later in Sec. 3.3 L242. We do not define pseudo-labels earlier since they don't affect the derivation of intermediate theory. We agree clarity can be improved though and added notation for pseudo-labels at the start of Sec.3, disambiguated notation of true and pseudo-labels and ID/OOD prediction, and added explanations in Sec.3.3.\n\n## [W2] Limited ablation study / discussion about class-conditional sampling\n## And\n## [Q1] What if the predicted class is wrong?\n\nCC-DIsoN uses predictions of a primary model $M_{pre}$ deployed\nfor a task of interest (e.g. classify cardiomegaly in Xrays, Sec.4) as\npseudo-labels for class-conditional sampling. This allows comparing the target sample to the closest relevant cluster (predicted class) rather than all training data. The original ablation (Tab. 3) showed class-conditional sampling helps, and DIsoN without CC-sampling\nstill works well (compare Tab. 3 with Tab. 1 baselines).\n\nYour suggestion to explore misclassification effects is insightful and led to the following experiments.\n\nTo assess misclassification rates, we report the pre-trained primary models $M_{pre}$'s accuracy on target ID, OOD, and combined samples (col. 1-3). To deploy a model in real-world deployment a high ID accuracy is required. Generalisation is impeded by domain-shift, however, thus predictions for OOD samples are expected less accurate than on ID. We compare for trends against performance of CC-DIsoN for OOD detection (col.4).\n\n|Dataset|Classif Acc. on ID target samples|Classif Acc. on OOD target samples|Classif Acc. on ID+OOD target samples|Performance on OOD detection by CC-DIsoN (AUROC, Tab.1)|\n|-|-|-|-|-|\n|Dermatology|77.6%|54.0%|63.0%|87.96|\n|X-Ray|93.0%|70.8%|82.2%|83.94|\n|Ultrasound|78.6%|61.4%|64.8%|66.58|\n|MIDOG (near+far)|77.3%|68.6%|69.2%|75.50|\n\nCC-DIsoN's performance does not correlate with classification acc.. E.g. Dermatology shows highest AUROC (87.96) despite lowest classifier acc. (63), suggesting\nthat performance depends more on data type than classifier acc. .\n\nTo further investigate misclassification effects, we designed\n3 experiments manually assigning wrong class pseudo-labels for certain samples:\n\n- **Predict wrong class for all data**: We assign wrong class as pseudo-labels for \\*all\\* target samples, regardless if ID or OOD. This represents the extreme case where the primary classifier has 0% acc. . Such model would never be deployed in real-world.\n\n- **Predict wrong class for ID data**: All target ID samples are assigned wrong class, while OOD samples use predicted classes. Central motivation for class-conditioned sampling was to make isolation more difficult for ID target samples, as they should be harder to isolate from source (training ID) samples of the same class, due to common visual features. This should lead to slower convergence and increase the difference than when separating OOD samples from source (training ID) samples, even of the same class, which is faster due to domain-shift. If we instead compare ID target samples to source (training) samples from the wrong class, the isolation will be easier, resulting in worse OOD detection.\n\n- **Predict wrong class for OOD data**: All target OOD samples are assigned a wrong class, while ID samples use the predicted class. Domain-shift degrades acc. of ML models on OOD data. Hence this experiment simulates the most extreme cases of domain shift. Since OOD samples already differ visually from the source distribution, comparing them to unrelated classes will make their convergence even faster, thus we expect easier OOD detection.\n\n|Dataset|Model|AUROC (%) ↑|\n|-|-|-|\n|**Dermatology**|Best baseline|83.4|\n||DIsoN|86.4|\n||CC-DIsoN|88.0|\n||ID+OOD wrong|83.1|\n||ID wrong|84.7|\n||OOD wrong|86.7|\n|**Breast Ultrasound**|Best baseline|61.0|\n||DIsoN|67.0|\n||CC-DIsoN|66.6|\n||ID+OOD wrong|61.5|\n||ID wrong|34.6|\n||OOD wrong|89.6|\n|**Chest X-Ray**|Best baseline|77.7|\n||DIsoN|81.4|\n||CC-DIsoN|83.9|\n||ID+OOD wrong|77.9|\n||ID wrong|74.0|\n||OOD wrong|88.3|\n\nThe table confirms expected results: CC-DIsoN's OOD detection AUROC degrades when ID samples are mislabeled, and tends to increase when OOD samples are. Even under extreme and impractical cases of misclassification, CC-DIsoN outperforms the best baseline (Tab. 1 in paper) in all but one case, demonstrating robustness. In practice, especially in safety critical domains like healthcare, we would only deploy methods with high ID acc. But domain-shift of OOD samples is always an issue for performance. Hence CC-DIsoN's design and behavior is appropriate. Notably, DIsoN (without CC sampling) also outperforms the best\nbaseline.\n\nWe added this ablation to Sec.4.1,\nextended discussion on CC sampling, and added classification accuracies\n(top table) to Appendix. Many thanks, this strengthens the analysis of CC sampling a lot.\n\n## [W3] Limited discussion of algorithm runtime\n\n## and\n\n## [Q5] More info about number of rounds to convergence (medians/quantiles)\n\nWe agree that more details could help practitioners. As requested, we provide below medians and quantiles of the distribution of convergence rounds (for default $\\alpha=0.8$).\nSince wall-clock runtime is hardware-dependent, we report number of rounds. Reporting ID and OOD separately helps practitioners estimate runtime based on their expected ID/OOD ratio.\n\nFor the camera-ready, we will extend this to other $\\alpha$ values and visualise them in Fig.4, with the full table in the Appendix.\n\n|Dataset|Sample Type|25th Perc.|Median|75th Perc.|\n|-|-|-|-|-|\n|Dermatology|ID|68.0|126.5|184.0|\n||OOD|22.0|28.0|40.0|\n|Ultrasound|ID|20.0|29.0|100.0|\n||OOD|16.0|20.5|32.0|\n|X-Ray|ID|28.0|41.0|92.0|\n||OOD|16.0|20.0|26.0|\n|Histop. (near)|ID|44.3|59.5|74.0|\n||OOD|34.0|44.0|57.0|\n\n## \\[W4\\] (Minor) Improve writing / passive voice\n\nThank you for the advice, we revised phrasings and reduce passive\nvoice and improve flow where helpful.\n\n## Questions:\n\n## Q1 and Q5 were answered above.\n\\\n\n## Q2: Which other methods in Table 1 employ a class-conditional criterion?\n\nTo avoid any possible misunderstanding, we clarify that our method \\*never\\* uses any real labels of the \\*true\\*\nclass for the target samples to perform Class-Conditional sampling (or otherwise). It only uses the predictions of the primary model $M_{pre}$ as pseudo-labels. Hence, it has no advantage over other methods in terms of input information for the target samples.\n\nUsing its class predictions as part of OOD detection method is common in\nliterature. Methods differ in how explicitly or implicitly they do it.\n\nAmong the compared methods, MDS measures Mahalanobis distance of a\ntarget sample from the cluster of the closest class (=model's predicted class), thus implicitly using a \"class-conditioning\ncriterion\". PALM and CIDER also use Mahalanobis distance. MSP uses the predicted class's softmax probability. fDBD measures distance to the decision boundary\nthus also implicitly the model's class prediction. ViM explicitly uses class-information, via the max logit of the predicted class. Only iForest does not use class predictions, which is a limitation. Since pseudo-labels are model-derived (no ground-truth), our method has no advantage over the compared methods. We agree this is worth clarifying and added this clarification in Sec.4.\n\n## Q3: How are values $E_{stab}=5$ and $tau=0.85$ chosen?\n\nIntuitively, these parameters capture the convergence of the isolation\nprocess. $E_{stab}$ acts as a \"patience\" parameter,\nconceptually similar to patience in learning-rate schedulers, requiring\ncorrect classification for 5 consecutive rounds. Value 5 is common in\nschedulers to capture convergence. With $\\tau$ we require the model to have learned to separate the target sample confidently enough, giving softmax probability \\>0.85. The\nvalue 0.85 was chosen because it's common in uncertainty-based\nliterature (e.g.\\[3\\]). In preliminary experiments these values worked well enough, thus\nwe kept them constant and did not calibrate them further to avoid overfitting to our data. We added explanations for the above to Sec. 3.1.\n\n## Q4: Isolation Forest does not perform well?\n\nIsolation Forests were originally developed for tabular data, were they\nwere found to perform well and built their reputation. For imaging data,\nhowever, they are often found to not perform well (e.g. \\[1,2\\]),\ntherefore this is no surprise in our study. We are inspired by ideas,\nhowever, not performance numbers. Hence one of the motivations of our\nwork was to develop an approach that exploits the great idea of\n'isolation' in iForests and makes it work well for imaging with\nnetworks. We added a clarification about iForest performance in Sec.\n4.1.\n\nThank you again for the detailed points to improve the paper! We believe\nthe additional experiments and clarifications address your concerns.\nPlease let us know if any questions remain and we are happy to engage in\nfurther discussion.\n\n## References\n\n[1] Sun et al, Out-of-distribution detection with deep nearest\nneighbors, 2022\n\n[2] Fan et al, Test-Time Linear Out-of-Distribution Detection, 2024\n\n[3] Kamnitsas et al, Transductive image segmentation: Self-training\nand effect of uncertainty estimation, 2021"}]

Author rebuttal:

Meta-review: All the reviewers have actively engaged with the discussion and unanimously voted for acceptance. After checking the review, the rebuttal, the manuscript, and the discussion, the AC agrees with this assessment.
Decision: Accept (poster)