Finally Outshining the Random Baseline: A Simple and Effective Solution for Active Learning in 3D Biomedical Imaging
Authors: Carsten T. Lüth
Jeremias Traub
Kim-Celine Kahl
Till J. Bungert
Lukas Klein
Lars Krämer
Paul F Jaeger
Klaus Maier-Hein
Fabian Isensee
Venue: Accepted by TMLR ()

Abstract: Active learning (AL) has the potential to drastically reduce annotation costs in 3D biomedical image segmentation, where expert labeling of volumetric data is both time-consuming and expensive. Yet, existing AL methods are unable to consistently outperform improved random sampling baselines adapted to 3D data, leaving the field without a reliable solution.
We introduce Class-stratified Scheduled Power Predictive Entropy (ClaSP PE), a simple and effective query strategy that addresses two key limitations of standard uncertainty-based AL methods: class imbalance and redundancy in early selections. ClaSP PE combines class-stratified querying to ensure coverage of underrepresented structures and log-scale power noising with a decaying schedule to enforce query diversity in early-stage AL and encourage exploitation later.
Our implementation within the nnActive framework queries 3D patches and uses nnU-Net as segmentation backbone.
In our evaluation on 24 experimental settings using four 3D biomedical datasets within the comprehensive nnActive benchmark, ClaSP PE is the only method that generally outperforms improved random baselines in terms of both segmentation quality with statistically significant gains, whilst remaining annotation efficient.
Furthermore, we explicitly simulate the real-world application by testing our method on four previously unseen datasets without manual adaptation, where all experiment parameters are set according to predefined guidelines. The results confirm that ClaSP PE robustly generalizes to novel tasks without requiring dataset‑specific tuning.
Within the nnActive framework, we present compelling evidence that an AL method can consistently outperform random baselines adapted to 3D segmentation, in terms of both performance and annotation efficiency in a realistic, close-to-production scenario.
Our open-source implementation and clear deployment guidelines make it readily applicable in practice.
Code is at https://github.com/MIC-DKFZ/nnActive.

Published in Transactions on Machine Learning Research (01/2026)
Finally Outshining the Random Baseline: A Simple and Effective Solution for Active Learning in 3D Biomedical Imaging
Carsten T. L,th Jeremias Traub Kim-Celine Kahl".2,5, Till Bungert Lukas Klein 1.255 Lars Kraemer 1,2,3 Paul F. Jacger2.0, Klaus Maier-Hein 12,887,.8 Fabian Isensee
German Cancer Research Center (DKFZ) Heidelberg, Division Medical Image Compating, Germany Helmholtz Imaging, German Concer Research Center (DKF7), Heidelberg, Germany
Faculty of Mathematics and Computer Science, University of Heidelberg, Germany
German Cancer Research Center (DKFZ) Heidelberg, Division of Intelligent Medical Systems, Germany Institute for Machine Learning, ETH Zürich,, Switzerland
German Cancer Research Center (DKFZ) Heidelbery, Interactive Machine Learning Croup, Germany 7 Pottern Analysis and Learning Group, Department of Radiation Oncology, Heidelberg University Hospital, Germany National Center for Tumor Diseases (NCT) Heidelberg, Germany
{ (carsten.luet) jeremias. traub) trrebb @dkfs-heidebbery.dd de
*/t: These authors contributed equalty to this work.
Reviewed on OpenReview: https: // openreview. net/ forum? id= VamKueEa YW
Abstract
Active learning (AL) has the potential to drastically reduce annotation costs in 3D biomed- ical image segmentation, where expert labeling of volumetric data is both timo-consuming and expensive. Yet, existing AL methods are unable to consistently outperform improved random sampling baselines adapted 3D data, leaving the field without reliable solution. We introduce Class-stratified Scheduled Power Predictive Entropy (ClaSP PE), a simple and effective query strategy that addresses two key limitations of standard uncertainty-basee AL methods: class imbalance and redundancy in early selections. ClaSP PE combines class- stratified querying to ensure coverage of underrepresented structures and log-scale power noising with a decaying schedule to enforce query diversity in early-stage AL and encourage exploitation later. Our implementation within the nnActive framework queries 3D patches and uses nnU-Net as segmentation backbone. In our evaluation on 24 experimental settings using four 3D biomedical datasets within the comprehensive nnActive benchmark, ClaSP PE is the only method that generally outperforms improved random baselines in terms of both segmentation quality with statistically significant gains, whilst remaining annota- tion efficient. Furthermore, we explicitly simulate the real-world application by testing our method on four previously unseen datasets without manual adaplation, where all experi- ment parameters are set according to predefined guidelines. The results confirm that ClaSP PE robustly generalizes to novel tasks without requiring dataset-specific tuning. Within the nnActive framework, we present compelling evidence that an AL method can consistently outperform random baselines adapted to 3D segmentation, in terms of both performance and annotation efficiency in realistic, close-to poose-toppe ton-ttuuooooiioonn scenario. Our open-source im- plementation and clear deployment guidelines make it readily applicable in practice. Code is at

---

Published in Transactions on Machine Learning Research (01/2026)
1 Introduction
Annotation in 3D biomedical imaging is particularly expensive due to the requirement for highly specialized expertise and the inherently time-consuming nature of creating detailed segmentation masks for volumetric data (Litjens et al. 2017]. Any approach that reliably reduces annotation effort in 3D biomedical imaging has the potential to unlock new tasks and applications for deep learning models in clinical and research settings where annotation cost represents the main bottleneck. Consequently, reducing the need for fully annotated datasets has become a major research focus. Various strategies are being explored, including enhanced annotation tools with interactive segmentation (Diaz-Pinto al.) (2024), improvements of the model training via self-supervisod learning (Zhou et al. 2021 Wald et al.) 2024), semi-supervised learning (Li et al., 2020 and learning from partial annotations (Can et al.) 2018 pretrained foundation models (Ma et al. 2024a).. These approaches share the common goal of minimizing manual labor for annotation while maintaining or improving model performance.
Active Learning (AL) offers a promising strategy which is orthogonal to all the aforementioned approaches and aims to reduce annotation costs by selectively querying only the most informative data points for annotation, thereby maximizing model performance with minimal labeling effort. As the annotation cost reduction of AL upon application can not be validated (validation paradox) (Lüth et al. 2023 which hinders both method selection and optimization, it is of critical importance that an AL method demonstrates strong empirical evidence to yield reductions in annotation cost in a realistic scenario (Settles 2022).
2011 Munjal et al.
However, despite its transformative potential, the effectiveness of AL in reducing annotation costs remains largely unproven for 3D biomedical image segmentation.
Several studies emphasize that random sampling remains a surprisingly strong baseline (Nath et al., 2021 Burmeister et al. 2022), and show that commonly used AL methods do not consistently outperform it GGallochet et al. 2023a b Vepa et al. 2024). Föllmer et al. (2024) state that 'Further research is necessary to prove the elfectiveness of active learning for medical image segmentation'. Most notably, the only two works that rigorously evaluate random strategies specifically adapted to the 3D biomedical context (improved random strategies) report that, under current methodological standards, there is insufficient evidence to generally recommend AL over improved random baselines (Lüth et al. 2025 Burmeister et al. 2022), despite the naive random baselines being commonly outperformed.
Our proposed query method, Class-stratified Scheduled Power Predictive Entropy (ClaSP PE), is designed to be a generalizing solution to reduce annotation cost. It combines two simple yet effective extensions to standard uncertainty-basee AL method that directly addresses their empirically observed shortcomings in the context of 3D biomedical segmentation:
1. A stratification of standard uncertainty and class-specific uncertainties, which directly addresses the voxel-wise imbalance of classes while still retaining the ability prioritize hard-to-predict cases.
2. An exponential scheduler for Power-Noising of scores (Kirsch et al. 2023) which addresses the low diversity of queries especially in early stage AL by perturbing the scores stronger in early AL stages and gradually reducing the noise towards later stages.
ClaSP PE is the first AL method for 3D biomedical image segmentation with compelling evidence to achieve general annotation cost. reductions during application scenarios as it outperforms both standard and improved random basclines in terms of segmentation quality whilst not sacrificing annotation efficiency. We base this strong claim on the most comprehensive evaluation of AL methods for 3D biomedical segmentation to date which captures wide range of realistic evaluation scenarios. We clarify our claim of realism for our evaluation based on the nnActive framework in section P alongside the challenges of applying AL to 3D biomedical segmentation.
The empirical evidence from our evaluation is delivered in two steps: As a first step, in section 4 we demonstrate that ClaSP PE consistently outperforms all other AL methods and random sampling strategics on the nnActive benchmark (Lüth et al.) (205)), the most comprehensive benchmark to date for AL in 3D biomedical imaging. This encompasses four 3D biomedical datasets, each with three annotation budgets
2

---

Published in Transactions on Machine Learning Research (01/2026)
(Label Regimes) that are evaluated with two distinct query designs (query patch sizes), resulting in 24 distinct experimental setups for AL, experimenis. In the second step, in section 5 we validate the generalization capabilities of ClaSP PE on four additional datasets by explicitly simulating real-world use-case scenarios (Roll-Out), demonstrating its practical applicability and robustness beyond the benchmark setting. We make sure to set up all parameters for the AL pipeline during Roll-Out. according to our Guidelines for Real- World Deployment without manual adaptions which can serve as a recipe for practitioners when applying ClaSP PE to novel datasets and tasks.
In summary, our main contributions are:
We propose ClaSP PE, a simple and effective query method that systematically addresses key limitations of current uncertainny-basse AL methods.
We conduct a large-scale evaluation, demonstrating that ClaSP PE brings reliable performance improve- ments over standard and improved random sampling baselines for 3D biomedical image segmentation on the unActive benchmark spanning four datasets and six annotation budgets each.
We provide evidence for the generalization capability of ClaSP PE by means of a Roll-Out study four additional datasets to explicitly simulate real-world use-case with all parameters being set based our Guidelines for Real- World Deployment.
We wish to emphasize that the focal point of our work does not lie in methodological novelty but in providing a simple solution obtained by intuitive adaptations of existing methods for the challenging and long-standing problem of general effectiveness in 3D biomedical AL, which is backed up by empirical rigorous evaluation [Lipton & Steinhardt, 2019).
2 Challenges of Active Learning for 3D Biomedical Image Segmentation
The design and evaluation of AL pipelines must account for the characteristies 3D biomedical segmentation, or it risks not delivering on its promise of reducing annotation effort. We will now start by giving a short recap on segmentation for 3D biomedical images and then introduce our approach for evaluation followed by highlighting the key differentiating factor to previous works in AL. for 3D biomedical segmentation which is the query design as a 3D patch.
Segmentation on 3D biomedical images. 3D volumetric images are very large, often exceeding 500 x 500 500 voxels for a single volume (e.g., an upper body CT scan). These images oftentimes feature many homogencous structures, such as organs, which are located in specific characteristic areas of the images. Further, these datasets commonly contain dominant background class that occupies most of the volume but is not target of interest, and there frequently exist strong volumetric differences between different structures or classes of interest, such as most tumors being much smaller than organs. The community for 3D biomedical images has adapted to these challenges by designing specific training techniques where less frequent classes are oversampled and models are either trained on smaller 3D patches of the data or 2D slices [sensee et al. 2021 2024) with 3D U-Net-like models (Ronneberger et al. 2015) generally performing best.
Evaluation of Active Learning Methods. Our evaluation directly builds upon Lüth et al. 2025 who propose the nnActive framework and benchmark, which dircctly address four pitfalls commonly occurring in the evaluation of AL in 3D biomedical imaging Concretely, these pitfalls are (1) Evaluation is restricted to too few settings; (2) Model Training does not incorporate partial annotations; (3) Random Baseline is not adapted to 3D setting; (4) Annotation cost is measured in voxels. The occurrence of these pitfalls directly hinders the ability to draw conclusions regarding the reduction of annotation effort in practically relevant settings. The framework and benchmark address these by: (1) ensuring diverse set of datasets and multiple annotation budgets (Label Regimes); (2) using nnU-Net with partial loss, ensuring well-configured models that make efficient use of annotations during training; (3) comparing our AL method against improved random baselines (Foreground Aware Random strategies) which oversample foreground regions in class- balanced fashion to handle the inherent class imbalance between foreground and background as well as between different foreground classes; (4) proposing the Foreground Efficiency (FG-Eff) measure which relates
For detailed information, we refer to this paper.
3

---

Published in Transactions on Machine Learning Research (01/2026)
the number of queried foreground voxels to the model performance by means of an exponential fit, we can identify whether an AL method selects foreground more effectively rather than just selecting more of it. The exact details of our evaluation are given in section
Query Design. The nnActive Framework combines multiple improvements over the evaluation schemes of related works and most notably uses 3D nnU-Net with partial loss (Isensee et al. 2021 Gotkowski et al. 2024) which enables arbitrary design of a query (e.g. 3D patches, 2D slices or single voxels). The general design of the query is a crucial factor in AT. for 3D segmentation, requiring a careful trade-off between allowing the human to annotate queries efficiently whilst allowing the Query Method (QM) to focally query structures of interest. When annotating entire 3D images, a lot of effort is spent annotating regions with redundant information which is why it is typically better to use partial annotations in form of 2D slices or 3D patches, especially when the used AL method can find the most informative regions.
We utilize 3D query patches of fixed size in combination with partial loss integrated into nn U-Net (Gotkowski et al.) 2024; Isensee et al., 2021). allowing us to train 3D models following Lüth et al. (0025). This design strikes a balance between annotation efficiency and informativeness while maintaining flexibility in query selection, as the query patch size can be selected based on the structures of interest instead of model constraints. The combination of 3D query design and 3D models represents major differentiating factor
of our work from most related works, which either rely on querying entire 3D images (Nath et al. 2021 or
restrict queries to 2D slices with 2D models [Burmeister et al. 2022 Gaillochet et al. 2023a b 2024b
Ma et al.
Föllmer et al. 2024 Vepa et al. 2024 Shi et al. 2024a
While the ability of a QM to directly select 3D patches corresponding to regions of interest is elegant and potentially powerful, it also introduces significant complexity to the general query algorithm with multiple overlapping candidate patches. This complexity largely hinders the implementation of representation- QMs, such as Core-Set (Sener & Savarese 2018), or more sophisticated unccrtainty-basee QMs like USIM [Fölmee al.) 2024), , due to both runtime and memory constraints arising from the transition from 2D slices to 3D patche As our input shape is not necessarily the query patch shape, it is an open research question what representation of our query patch is. Gencrally, obtaining representations for 3D volumes is a major challenge for AL as noted by Liu et al. (2023) in their evaluation for starting budget selection. Further, there is a general consensus that even for 2D slices and 2D models, representation-Daase methods
like Core-Set are performing worse than uncertainty-brat based AL methods (Burmeister et al., 2022 Föllmer
ct al. 2024). We hypothesize that this stems from the skip connections of the utilized U-Nets (Ronneberger
eet al. 2015); ; which may lead to the representations, typically taken from the bottleneck layers, not capturing the fine details necessary to allow optimal data selection.
3 Method
Our proposed query strategy, Class-stratified Scheduled-Power Predictive Entropy (ClaSP PE), is designed to improve AL for 3D biomedical segmentation by effectively balancing informativeness, class reprosentation, and diversity of the queried patches and thereby solves prominent issues of top-k sampling uncertainty methods (as illustrated in fig. D Starting from a slandard Uncertainty-Basee commonly employed in top-k sampling which returns an uncertainty map u(z) for each image x, we introduce two key modifications: Class Stratified Sampling and an Exponential Scheduler for Score Perturbation. Importantly, these extensions are agnostic to the specific uncertainty scoring function used and can be applied on top of any existing uncertainty-base method.
scoring
Class Stratified Sampling. To encourage class-balanced selection of queries, we implement stratified sampling procedure. Specifically, we select equal number of patches per predicted class based the model's predictions. For each image x, compute class-specific uncertainty scores
(c(x) = (p(x)) u(x),
(1)
For example, on the KITS dataset, one median 3D volume has 188 x 10a potential queries using patches compared to 500 queries using slices using the setup described appendix
4

---

Published in Transactions on Machine Learning Research (01/2026)
Top-K Uncertainty Methods
Uncertainty (e.g. PE)
Top-k queried Patches
Aggregate
3D Volume Data
MRI/CT Image
Segmented Organ Class
Rank best patches over entire datasse
Organ
Class Imbelaacce
Our Solution
Uncertainty (e.g. PE)
Predicted Class Probabilities
Top-k queried Patches
Added Noise
Top- k Class
X P
Top-k
Top-k ik non stratifliad
Balancing
Diversification earker Stage Querin
through Stratificttion
Figure 1: Overview of the ClaSP PE query strategy. We overcome two key limitations of standard uncertainty-based Active Learning methods (e.g. Predictive Entropy), class imbalance and low diversity of the queries, by adding two simple modifications: (1) class-stratified sampling for 66% of the query budget based on predicted class probabilities, and (2) a scheduler decreasing the noise for score perturbation via log-scale power noising enhance diversity during query selection.
where p(Y = c|zz) denotes the predicted probability for class Patches are then ranked per class according to ue(x), and the top Ne patches from each class are selected, where N₂ is chosen such that all classes contribute equally to the stratified subset. This ensures that underrepresented classes are not neglected, which naturally supports metrics that average performance across classes (c.g., mean Dice). Im- portantly, by leveraging the model predictions our approach does not require any additional label information. To our knowledge, balancing queries in this way has not been used in the AL literature before. Crucially, only fraction of the samples is selected using this stratified approach, with the remaining' - a samples being selected based the standard uncertainty map u(x) to retain sensitivity to highly uncertain examples regardless of class distribution.
An Exponential Scheduler for Score Perturbation via Log-scale Power Noising. To enforce diversity among selected queries, especially in earlier AL cycles, we apply power noising to the scores (on patch-level) before selecting the top-k samples (Kirsch al. 2023 Specifically, we perturb the scores on logarithmic scale by adding Gumbel noise Gumbel F-.). Additionally, we use an exponential schedule for the perturbation strength g-1 such that it decreases towards later AL cycles from initial value 90 to final value B in order to gradually shift the focus from exploration to exploitation:
3(1) = exp 41 In(3)) + In(@maxx): 1= 1=0...,,
(2)
where t indexes the current AL cycle and T is the total number of AL cycles.
For our final ClaSP PE method we utilize Predictive Entropy to obtain uncertainty-based scores as it was highlighted as the overall best performing AL method on the nnActive benchmark (Lüth et al.) 2025). We
then apply the stratified selection to a = 66% of the budget based on our analysis in section 4.2 For the
We also experimented with linear and sigmaid schedules but found that exponential schedules generally performs par or better.

---

Published in Transactions on Machine Learning Research (01/2026)
AUBC
Final Dice
FG-Eff
8
6
Method
4
Nemany: Signficance Groups
Nomeny Signficancee Groups
Nomenyi Signif Groupe
BALD
SoftrankBALD Predictive Entropy
PowerPE
Random 33% FG
ClaSP
Random
Figure 2: ClaSP PE delivers substantial performance improvements without sacrificing annota- tion efficiency. The plots show average method rankings (lower is botter) with standard error for AUBC, Final Dice, and FG-EIT across the nnActive benchmark. Results Regimes, and 2 query patch sizes, each evaluated with 4 random seeds, providing robust estimates of method performance. The brackets indicate groups of methods that do not differ significantly based on a post-hoc Nemenyi Lest at significance level 0.05.
aggregated over 4 dalasets, 3 Label
exponential scheduler, we fixed 3o and 9max 100 for all evaluation settings and no additional tuning was performed.
This method is simple to implement and flexible, yet effective, as our empirical studies sections 4 and 5
demonstrate. We provide an implementation of ClaSP PE in the nnActive framework (Lüth et 2025 and detailed pseudo-code of the method in appendix B
4 Experimental Results on the nnActive Benchmark
We evaluate the effectiveness of our proposed query strategy ClaSP PE on the nnActive benchmark (Lüth et al.] )022). which is, to our knowledge, the most comprehensive AL suite currently available for 3D biomedical segmentation. To this end, we perform over 1000 nnU-Net training runs across 24 distinet settings (4 datasets X 3 Label Regimes 2 query patch sizes) including dedicated ablations. This comprehensive setup captures a wide range of segmentation challenges and enables statistically mcaningful conclusions about the robustness, efficiency, and generalizability of our method.
Datasets, Label Regimes & query patch sizes. The nnActive benchmark spans four prominent med-
ical imaging datasets: AMOS2022 (challenge task 2) (Ji et al. 2022), Medical Segmentation Decathlon Hippocampus (Antonelli et al. 2022), KiTS2021 (Heller et al. 2023), and ACDC (Bernard et al. 2018 Each of these datasets evaluated under three distinct Label Regimes (Low-, Medium- and High-Label) correspouding to a specific annotation budget defined a number of total patches. Further, the entire benchmark entails two distinct query patch sizes (referred to Main and Patchx }),, with the latter being half the size along each dimension. For more information regarding datasets and Label Regimes, we refer to appendix
Basclines. We compare ClaSP PE against the standard random bascline and two improved random basc- lines (Random 33% and 66% FG) (Lüth et al.)
2025 as well as the following five uncertainly-besse QMs: Predictive Entropy [Settles] 2009). Bayesian Active Learning by Disagreement (BALD) (Houlsby et al. 2011 Gal al. 2017 PowerBALD (Kirsch al. 2023), SoftrankBALD (Kirsch et al. 2023 and PowerPE
[Krschh al.] 2023). Random 33% and 66% FG simulate the process of selecting a patch around random
foreground region for X% of their budget. See appendix D for more details.

---

Published in Transactions on Machine Learning Research (01/2026)
Experimental Sctup. Our experimental setup is identical to the nnActive benchmark using four seeds with fixed test split, and using a custom nnU-Net trainer with 200 Epochs in the 3D full resolution configuration with each AL experiment consisting of 5 cycles. We evaluate AL performance with the following metrics operating on the mcan Dice score (Dice 1945): The Final Dice score achioved after the final AL cycle; the Area Under Budget Curve (AUBC) (Zhan et al., 2021: 2022) which aggregates the mean Dice scores across one AL trajectory over all cycles to measure the overall performance; the Foreground Efficiency (FG-EIf) (Lüth et al., 2025),, which acts as proxy for annotation efficiency by setting the performance in relation to the queried foreground voxels by means of an exponential fit; the Pairwise Penalty Matrix (PPM) (Ash et al.) 2020), which quantifies along the entire AL trajectory how often one method significantly outperforms another based on paired t-tests and can thus simply be aggregated over e.g. datasets. The exact implementation and more details with regard to the evaluation metrics are provided in appendix D
Results. As our bascline models are well adapted to medical datascts by means of proper Data Augmen- talion, Model Architecture and loss formulation, we observe as expected that absolute performance gains for single datasets can be small in absolute value (Mittal et al. 2019 Lüth et al.) [0023] Beck et al.) 2021). There- fore, our evaluation is performed on the highest aggregation level the goal of AL is to bring generalizing performance improvements for specific annotation budget. Figure 2 shows the method rankings averaged across the nnActive benchmark. Exact numerical results are provided in appendix F We find that ClaSP PE achieves the best overall performance in terms of both AUBC and Final Dice, generally outperforming both improved random baselines and established AL methods. Importantly, our approach delivers these segmentation quality gains while maintaining high annotation efficiency, as indicated by FG-Efff: although ClaSP PE docs not always achieve top FG-Eff, it consistently ranks among the most efficient methods. This reflects an inherent interplay between segmentation performance and annotation efficiency, where met that strongly focus on highly informative regions can improve Dice scores but may risk inefficient use of annotated foreground (e.g., Predictive Entropy). Our ablations (sce section 4.2 further show that score per- turbation is crucial for preventing such inefficiencies, and that gradually reducing the noising strength boosts segmentation performance at the cost. of only a slight reduction in FG-EI. Overall, ClaSP PE achieves a favorable balance across this trade-off, providing efficient, informative, and diverse query selection through our proposed modifications.
In addition to the average rankings, fig. 2 includes statistical significance groups derived from the conser- vative Nemenyi post-hoc test (Nemenyi 1963) with significance level of p 0.05. These groups provide exploratory evidence for the robustness of ClaSP PE: it forms a distinct top-performing group for segmen- tation performance measured by AUBC and Final Dice, while also remaining competitive in FG-EII. In contrast, the naive random baseline is consistently ranked lowest and is significantly outperformed by all other methods. Overall, Cl&SP PE shows the most consistent separation from random and uncertainty- baselines across all three metrics. Importantly, although SoftrankBALD also appears in the top Nemenyi group; ClaSP PE shows a clearer overall advantage when considering both the average rankings (fig. 2 and absolute performance (table D. Detailed results of the Nemenyi tests are provided in appendix E.
Additionally, when comparing the average Final Dice and AUBC over all settings, ClaSP PE is the only AL method that improves over improved random strategies, as shown in table [] Both PowerBALD and PowerPE outperform their top-k counterparts BALD and Predictive Entropy for the Final Dice performance metric contrary to the rankings in fig. 2 which provides further evidence for the more stable performance of these methods across annotation budgets, as already noted in Lüth et al. (2025).
ClaSP PE performs well overall and generally delivers substantial performance improvements on the KiTS dataset, can be seen in table 5 and table 7 However, especially on the AMOS dataset for smaller amno- tation budgets, ClaSP PE underperforms improved random strategies, but shows smaller underperformance compared to the other AL methods (shown in table 7 This behavior is further discussed in section 4.1
For ACDC and Hippocampus, the absolute performance differences are generally small (table G and often fall within the respective error bars. This highlights two important points: (1) broad evaluation across many datasets and label regimes is essential reveal overall trends, and (2) even when such trends clearly
These are performed without famely-wise error rate correction following Ash et al. 2020 Beck al.
2021
Föllmer al.
2024
7

---

Published in Transactions on Machine Learning Research (01/2026)
Table 1: ClaSP PE achieves better average performance than both random and AL baselines. Average Performance aggregated over all 24 distinct AL settings of the nnActive benchmark for AUBC and Final Dice alongside the 95% Confidence Interval (higher is better as indicated by green colorization). Details for the computation are given in appendix E.
Query Method
AUBC
Final Dice
BALD
62.39
0.30
65.43 1 0.41
PowerBALD
61.81
0.35
67.93
0.29
SoftrankBALD
63.74
0.32
67.32
+ 0.28
Predictive Entropy
63.27
0.40
67.35
+ 0.68
PowerPE
64.85
0.35
68.01
0.38
Random
60.57
0.39
61,65 + 0.43
Random 33% FG
66.00
0.27
69.74
0.32
Random 66% FG
67.14
0.22
71.14
0:22
ClaSP PE
67.6 07.62
0.33
72.81
0.30
BALD
13
2.5
2.5
11
36
22
21
0.8
SottoniBALD
12
5
6.7
5.8
-47
22
18
Pred Entropy
13
13
Medrod styltaatt w/
PowerFF
Bandom
8.3
0
1.7
4.2
0
0.8
3.3
0.8
Random 60% Pc
26
23
22
26
45
2.
3.9
C3:PP
39
43
Maon-
15
14
14
15
21
16
BALD
PomerBALD
SokmesibALG
medictive Enampy
Flwenng
Randoon
Reoden 338
Randen 46%
ClAsP ye
Method siyrificntt losses 1977
Figure 3: ClaSP PE consistently outperforms both random and AL baselines across the nnActive benchmark. The Pairwise Penalty Matrix summarizes statistically significant wins and losses from pairwise t-tests (p=0.05) between methods. Results are aggregated over 24 distinct AL settings the nnActive benchmark, including 4 datasets x 3 Label Regimes query patch sizes. Remaining lose scenarios against Random 66% FG stem from challenging Low-Label settings on the AMOS dataset (discussed in section
favor given method, this does not imply that it will yield significant gains over all other methods in every individual scenario.
To complement the aggregate metric rankings and average segmentation performance, fig, 3 presents the PPM, assessing pairwise performance differences on the nnActive benchmark. ClaSP PE clearly emerges as the strongest. method overall, outperforming all random and AT. baselines more frequently than is out- performed. This underscores the mothod's robustness and generalizability across diverse scttings. Further, we show that the overall trends of the PPM are persistent across different p-values and when using the Bonferroni- Holm method (Holm 1979) to account for the family-wise error rate appendix E.6 Nonetheless, in roughly 20% of the comparisons, Random 66% FG surpasses ClaSP PE. These cases are concentrated almost exclusively on the AMOS dataset under Low-Label Regimes, a particularly challenging scenario due to the high number of classes and the raistr annotation budget. We investigate this dataset-specific behavior in more detail in section 4.1
Finally, we note that the combination of score perturbation and stratified sampling substantially boosts the performance of standard Predictive Entropy across all evaluation metrics. Our large-scale evaluation provides
8

---

Published in Transactions on Machine Learning Research (01/2026)
200 Epochs
500 Epochs
60
50
40
30
Fractioon of
20
10
Win
Loss
Win
Loss
Win
Tie Random
Win
Loss
vs. Random
vs. Random
PE vs. Random
Label Regime:
Medium
High
Outcome:
Win
Loss
Figure 4: Longer training amplifies the advantage of ClaSP PE over random selection. Shown are fractions of significant wins, losses, and resulting ties of ClaSP PE against improved random baselines on the AMOS dataset, as computed via the PPM. We compare models trained for 200 (left) and 500 (right) epochs, well as different. Label Regimes (color-coded). Each Label Regime carries 33% of the entire fraction of experiments which is then divided into wins, losses and ties. While at 200 epochs ClaSP PE loses on 60% of the experiments to FG66 and tics in the rest, it outperforms Random FG 66% in 20%, tics in 48% and loses in only 32% when trained for 500 epochs.
clear empirical evidence for the effectiveness and robustness of these simple yet impactful modifications. Additional qualitative analyses can be found in appendix !
4.1 Investigating Loss Scenarios on AMOS
To better understand the limited performance gains of ClaSP PE compared to improved random basclines on the AMOS dataset, we conducted an ablation study that evaluates the influence of longer training on AL performance.
Specifically, compare the performance of ClaSP PE against the improved random baselines (Random 33% FG and Random 66% FG) on the Low- Medium-, and High-Label Regimes (with a total budget of 200, 1000, and 2500 patches, respectively). All mothods are trained for 200 and 500 epochs, and we conduct the comparison on the Main nnActive Benchmark, which results distinct evaluation settings.
We observe that increasing the training duration from 200 500 epochs substantially improves the win- to-lose ratio of ClaSP PE relative to the random basclincs. Figure E shows that in the 500-epoch setting, the number of lose-cases is reduced and primarily confined to the lower Label Regimes. In particular, ClaSP PE now consistently outperforms Random 66% FG in the High-Label Regime, whereas the Low- Label Regime is still dominated by lose-cases. Compared to the Random 33% FG baseline, ClaSP PE shows clear and consistent gains in both the Medium- and High-Label Regimes, underscoring the benefits of extended training. Detailed results are shown appendix F.4
These findings suggest that longer training amplifies the advantage of ClaSP PE over random selection. We hypothesize that the large number of 15 classes on AMOS makes the Low-Label especially challenging the 200 patches annotation budget, when evenly spaced across all classes, could capture less than 14 examples per class (compared to 67 on KiTS, for 3 classes). This highlights the scnsitivity of AL performance not only to the training dynamics but also to lask-specific factors such as the number of classes. Further, we observe in an analysis for AMOS with class-level dice that the loss scenarios on the low-label regime mainly stem from the segmentation performance on the right and left adrenal gland which is also less frequently queried compared to Random 66%FG. We show the detail in appendix E.5 We therefore emphasize the importance of adapting the annotation budget to the number of classes for practitioners.

---

Published in Transactions on Machine Learning Research (01/2026)
AUBC
Final Dice
FG-Eff
10
9
Avg.
5
BALD
Predictive Entropy
Random 33% FG
Cla PE 33%
ClaP PE
SoftrankBALD
Random
Figure 5: ClaSP PE achieves the best trade-off between segmentation quality and annotation efficiency. Average method rankings on the nnActive Main benchmark (4 datasets x 3 Label Regimes x 1 query patch size), with additional method variants, Cla PE 66%, Cla PE 33% and ClaP PE.
4.2 Ablating the Influence of ClaSP PE Components
Our proposed method, ClaSP PE, combines two simple yet effective components: (1) class-balanee sampling applied to a certain fraction of queries, and (2) log-scale power noising applied to the scores prior to top-k patch selection. In this ablation, we analyze the contribution of each component and justify our final design choice. To this end, evaluate additional method variants, Cla PE with 33% and = 66% to isolate the effect of class-balances sampling without power noising and further ablate the fraction of queries for which it is applied, as well as ClaP PE which is identical to ClaSP PE using a = 66% but uses a constant noise value 3 = 1 instead of a scheduler. We report their performance across the nnActive Main benchmark.
From the aggregated results, displayed in fig. 5 we observe the following: (1) Class-balanced qucrying improves performance across the board: Both Cla PE 66% and Cla PE 33% outperform standard PE on all evaluation metrics. Morcover, higher stratification rates lead to better segmentation quality: We find that increasing the fraction of stratified queries from 33% to 66% yields improvements in AUBC and Final Dice, with only minor decrease in FG-Eff. (2) The addition of power-noising substantially improves the FG-Eff, indicating improved annotation cost-efficiency through enhanced diversity, but leads to a reduction in absolute performance measured by AUBC and Final DICE, as can be observed when comparing ClaP PE and Cla PE 66%. (3) Gradually decayed power noising leads to the overall best. tradeoff with regard annotation efficiency and absolute performance
is across all three metrics among the best. This supports the notion that the decaying schedule leads to more diverse set of queries in carly iterations of AL, which gradually become more focused on harder cases when the model has adapted to the dala distribution. Detailed results are shown in appendix F.2
Overall, the combination of 66% stratified querying and gradually docayed power noising provides the best trade-off between segmentation quality and annotation efficiency, justifying the choice ClaSP PE as our final method.
5 Simulating Real-World Active Learning in Roll-Out Study
To evaluate the generalization and practical utility of ClaSP PE, we conduct a roll-out study across a diverse set real-world biomedical segmentation datascts. Importantly, we do not perform any dataset- specific finetuning, treating this as a plug-and-play scenario that mirrors how one might apply ClaSP PE in practical, previously unseen tasks.
10

---

Published in Transactions on Machine Learning Research (01/2026)
The methods compare include proposed ClaSP PE, standard Predictive Entropy, which ranked just behind ClaSP PE on the nnActive benchmark, uniform random sampling, and Random 66% FG, stronger baseline incorporating foreground-awar sampling.
We follow all design decisions of the nnActive experiment setup, such as the starting budget and dataset preprocessing, but introduce two new components tailored for real-world deployment: (1) systematic selection of query patch size based on the median connected component sizes of the target structures, and (2) normalized query budgets, set to 50 100 patches per class depending on task complexity (e.g. the expected homogeneity). These additions ensure that queries remain representative and task-appropriate. Our full Guidclines for Real-World Deployment are provided in appendix F
We evaluate performance on four datasets that vary widely in task complexity, number of foreground classes, and annotation difficulty: LiTS (Bilic et al. 2023 a two-class foreground segmentation task for liver and tumor; WORD (Luo al. 2022), 16-class organ segmentation task; Tooth Fairy 2 (Bolelli al., 2025; Lumetti al. 2024), which requires dense labeling of 42 dental structures; and MAMA MIA [Garrucho) 2025), lesion segmentation task with single target class. A fixed data split is used for all experiments (75% train & pool, 25% test), which is identical across four random seeds. Detailed dataset characteristics are provided in appendix
2024
al.
As summarized in Table 2 ClaSP PE overall performs on par or better than all baseline methods across datasets and metrics. It delivers reliable segmentation quality improvements while maintaining or exceeding annotation efficiency, without any task-specific method tuning. While Random shows high FG-Eff on LiTS and WORD, this results from querying only a very small amount of foreground, which artificially inflates FG- Eff without translating into segmentation performance gains. Predictive Entropy partially shows competitive performance with ClaSP PE in terms of segmentation performance, while ClasP PE demonstrates improved FG-Eif over PE across all roll-out datasets. On the large scale MAMA MIA breast cancer dataset, featuring many redundant structured for a highly complex task, ClaSP PE performs substantially better. Further, the results on the unActive benchmark (fig. [ reveal that PE fails to reliably outperform random baselines, whereas ClaSP PE shows consistent improvements. Together, these results underscore the robust out-of- the-box performance of the ClaSP PE method and establish it as practical and effective solution for active learning in real-world 3D biomedical segmentation tasks.
Similarly, the PPM shown in fig. reveals that ClaSP PE showcases the overall best performance being never significantly outperformed by Random and Random 66% FG while winning in over 50% of all cases and also outperforming Predictive Entropy significantly in 25% all cases while being significantly outperformod in 5%. We provide detailed results in appendix G
Predictive Citspy
Kandom 55% no-
cuve
25
50
Pestubee Enteer
Lessio rD
AgoRFvE sypattmmed
Figure ClaSP PE shows overall strongest performance on the roll-out study, PPM for the roll-out study aggregated over all settings. In all settings, ClaSP PE wins against tics with the random baselines.
11

---

Published in Transactions on Machine Learning Research (01/2026)
Table 2: ClaSP PE provides robust performance gains on out-of-the-bon deployment. Perfor-
mance on the Roll-Out datasets, measured by AUBC, Final Dice, and FG-Eff (higher is better, indicated by green colorization).
Dataset (Msumples) Metric
LITS (n=09)
WORD (n=90)
Tooth Pairy 2 (n=360)
MAMA MIA (1:3))0)
AUBC
Finul Dice
FG-Eit
AUBC
Final Dice
PG-E0
AUBC
Final Dice
FG-Eff
AUBC
Final Dice
FG-Ei
Random
51.23
52.38
16 25
77.35
61.83
61.32
55.23
Random 60% PG
48.63
50.06
1.27
78.19
78.25
1.34
65.
68.01
10.85
44.38
45.10
-4.67
Predictive Entropy
57.81
66.35
38. 9-1
78.43 43
78.90
0.91
06.60
71:97
16:25
59.07
64.74
9.43
ClaSP PE
60.30
65.80
39.60
78.27
78.
1.33
67.32
71.49
20.07
63.85
68.62
57.36
100% Data Dice
80.7
72.6
6 Limitations
While ClaSP PE demonstrates strong performance across both benchmark and roll-out evaluations, several limitations remain. First, like all AL methods, it faces the risk of benchmark-speciffe overfitting, due to the necessity of empirically validating design decisions (Shi et al. 2024a Föllmer et al. 2024 Gaillochet et al, 2023b; Vepa. et al. 2024). Our dual evaluation mitigates this concern but cannot fully eliminate it. Further, as the entire evaluation is based on the average Dice which is the default overlap-based metric for semantic segmentation (Maier- Hein et al. 2024), our results do not necessarily extend to houndary-baseen evaluation metrics or when only specific classes are of interest.Second, the method depends on the predictive capacity of the underlying model: when initial segmentation quality insufficient, stratified qucrying bocomes less effective, though our guidelines for employing ClaSP PE mitigate this risk, and the use of pre-trained models may further improve early-stage segmentation quality (Gupte et al. (024).. Third, AL is inherently economic tradc-off: reduced annotation cost must be weighed against additional computational overhcad, and the optimal balance is context dependent (Settles 2011). Fourth, while we compared against established strong baselines, more complex AL strategies (s.a. Hübotter et al.] (2024); Föllmer et al. (2024)) could potentially offer further gains, though their adaptability for querying 3D patches remains uncertain. Fifth, ClaSP PE relies on small set of hyperparameters governing stratification and power-noising Although validated across diverse datasets, these may benefit from adaptive tuning to better match dataset-specific characteristics. Finally, since our empirical evidence is obtained using the nnActive framework with 3D patches as query design, conclusions may differ under meaningful deviations from it, such as alternative segmentation backbones (Munjal et al., 2022 or 2D slice queries. A detailed discussion of these limitations is provided in Appendix
On the Importance of Query Design and Annotation Technique. The design of the query, whether it is a whole 3D image, 3D volumetric patch, a 2D slice, or even single voxel, substantially impacts the annotation process and tooling efficiency. However, no consensus exists on which query design and annotation process, such sparse annotation, super-pixels perr-pixxss/veeeheeee voxels, scribbles, is the most economical, as each one has its own ad advantages and drawbacks depending on the specific task and currently available tooling (Tajbakhshi et al. 2020 Shi et al. 2024b). We consider annotation technique selection critical for maximizing economic effectiveness.
Our evaluation uses 3D patches, which support various annotation processes including sparse 2D slice-wise
schemes ÇCçek et al. 2016 Burmeister et al. 2022) and scribble annotations (Li al. 2024 Gotkowski et al. 2024). With promptable foundation models like SAM (Kirillov et al.. 2023 MedSAM Ma et al. 2024a and inInteractive (Isensee et al.) 2025 3D patches as annotation tools 3D patches enable targeted annotation, verification, and correction of specific structures within localized image regions. We focused on selecting informative patches rather than explicitly evaluating these annolation processes; examining how different techniques interact with patch- patch-based querying remains future work.
12

---

Published in Transactions on Machine Learning Research (01/2026)
7 Conclusion
We propose ClaSP PE, the first AL query method with substantial evidence of reducing annotation effort over random strategics for 3D biomedical segmentation in a close-to-production environment. ClaSP PE offers consistent performance gains across a wide range of datasets and AL. scenarios. In addition to its strong performance, ClaSP PE is conceptually lightweight and easy implement, enabling seamless integration into existing AL frameworks. Its computational cost remains comparable to standard top-k selection methods, making it well-suited for practical deployment.
For developers and rescarchers, ClaSP PE can serve as strong and casy-to-implement bascline for future AL research. Our open-source code and results reduce the experimental overhead for developers and enable fair and reproducible comparisons in methodological studies.
For practitioners, our implementation ClaSP PE offers a solution that can be integrated into real-world annotation workflows. It comes embedded in AL pipeline that includes guidelines for setting all relevant parameters. This allows it to be implemented efficiently for use in the 3D biomedical segmentation domain when used inside the nnActive framework. For real-world deployment, the results of our evaluation lead to the following recommendations:
Use ClaSP PE within the mnActive framework querying 3D patches, using the auto-configuration of nnU-Net.
Train models for 1000 epochs, as AL performance generally improves for longer training durations.
Follow our Guidelines for Real-World Deployment for patch size and query size (see appendíx F
Acknowledgments
This work was funded by Helmholtz Imaging (HT); platform of the Helmholtz Incubator on Information and Data Science. This work is supported by the Helmholtz Association Initiative and Networking Fund under the Helmholtz AI platform grant (ALEGRA (ZT-L-PF-5-121))
The authors gratefully acknowledge the computing time provided on the high-performance computer HoreKa by the National Herfo-Perooraacce Computing Center KIT (NHR@KIT). This center is jointly supported by the Federal Ministry of Education and Research and the Ministry of Science, Research and the Arts of Baden- Wirtteeert,,,,rrr, part of the National High- Performarre Computing (NHR) joint funding program /tt/ooo/prtnooss///// HoreKa is partly funded by the German Rescarch Foundation
(DFG).
References
Michela Antonelli, Annika Reinke, Spyridon Bakas, Keyvan Farahani, Annette Kopp-Schneider, Bennett A Landman, Geert Litjens, Bjoern Menze, Olaf Ronneberger, Ronald M Summers, et al. The medical segmentation decathlon. Nature communications, 13(1):4128, 2022.
Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds. orXiv:1906.00 03671 [es, statf, February 2020.
Nathan Beck, Durga Sivasubramaniaan, Apurva Dani, Ganesh Ramakrishnan, and Rishabh Tyer. Effective evaluation of deep active learning on image classification tasks. arXiv preprint ar arXix:2166. 150.11534444 2021.
Olivier Bernard, Alain Lalande, Clement Zotti, Frederick Cervenansky, Xin Yang, Pheng- Ann Heng, Irem Cetin, Karim Lekadir, Oscar Camara, Miguel Angel Gonzalez Ballester, et al. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: Is the problem solved? IEEE transactions on medical imaging, 37(11):2514 2525, 2018.
Patrick Bilic, Patrick Christ, Hongwei Bran Li, Eugene Vorontsov, Avi Ben-Cohen, Georgios Kaissis, Adi Szeskin, Colin Jacobs, Gabriel Efrain Humpire Mamani, Gabriel Chartrand, et al. The liver tumor seg- mentation benchmark (lits). Medical image analysis, 84:102680, 2023.
13

---

Mélanie Gaillochet, Christian Desrosiers, and Hervé Lombaert. Active learning for medical image segmen- tation with stochastic batches. Medical Image Analysis, 90:102958, December 2023a. ISSN 13618415. doi: (tt.:..mmmmia.2222102222... Published in Transactions on Machine Learning Research (01/2026)
Federico Bolelli, Luca Lumetti, Shankeeth Vinayahalingam, Mattia Di Bartolomeo, Arrigo Pellacani, Kevin Marchesini, Niels van Nistelrooij, Pieter van Lierop, Tong Xi, Yusheng Liu, Rui Xin, Tao Yang, Lisheng Wang, Haoshen Wang, Chenfan Xu, Zhiming Cui, Marek Wodzinski, Henning Müller, Yannick Kirchhoff, Maximilian R. Rokuss, Klaus Maier-Hein, Jachwan Han, Wan Kim, Hong-Gi Ahn, Tomasz Szezepański, Michal K. Grzeszezyk, Przemyslaw Korzeniowski, Xavier Caselles Ballester, Vicent amd Paolo Burgos- Artizzu, Ferran Prados Carrasco, Stefaan Berge', Bram van Ginneken, Alexandre Anesi, and Costantino Grana. Segmenting the Inferior Alveolar Canal in CBCTs Volumes: the ToothFairy Challenge. IEEE Transactions Medical Imaging, pp. 1-17, Dec 2024. ISSN 1558-254X. doi: https:/ wt///oiiiog/.... 202.33334..
Federico Bolelli, Kevin Marchesini, Niels van Nistelrooij, Luca Lumetti, Vittorio Pipoli, Elisa Ficarra, Shankeeth Vinayahalingam, and Costantino Grana. Segmenting Maxillofacial Structures in CBCT Vol- ume. In IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 1 10. IEEE,
Mar 2025.
Josafat-Mattias Burmeister, Marcel Fernandez Rosas, Johannes Hagemann, Jonas Kordt, Jasper Blum, Simon Shabo, Benjamin Bergner, and Christoph Lippert. Less Is More: A Comparison of Active Learning Strategics for 3D Medical Image Segmentation, July 2022.
Yigit B Can, Krishna Chaitanya, Basil Mustafa, Lisa M Koch, Ender Konukoglu, and Christian F Baum- gartner. Learning to segment medical imagos with scribble-supervision alone. In Deep Learning in Medi- cal Image Analysis and Multimodal Learning for Clinical Decision Support: 4th International Workshop, DLMTA 2018, and 8th International Workshop, MD-CDS 2018, Held in Conjunction with MICCAT 2018, Granada, Spoin, September 20, 2018, Proceedings 4, pp. 236 244. Springer, 2018.
Özgün Çişek, Ahmed Abdulkadir, Soeren S Lienkamp, Thomas Brox, and Olaf Ronneberger. 3d u-net: learning dense volumetric segmentation from sparse annotation. In International conference on medical image computing and computer-assisn assisted intervention, pp. 424-432. Springer, 2016.
J. Cohen. Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates, 1988.
Janez Demsar. Statistical comparisons of classifiers over multiple data sets. Journal of Machine learning research, 7(Jan):1-30, 2006.
Andres Diaz-Pinto, Sachidanand Alle, Vishwesh Nath, Yucheng Tang, Alvin Ihsani, Muhammad Asad, Fernando Pérez-García, Pritesh Mehta, Wenqi Li, Mona Flores, et al. Monai label: A framework for ai-assisted interactive labeling of 3d medical images. Medical Image Analysis, 95:103207, 2024.
Lee R Dice. Measures of the amount of ecologic association between species. Ecology, 26(3):297-302, 1945.
Bernhard Föllmer, Kenrick Schulze, Christian Wald, Sebastian Stober, Wojeiech Samek, and Marc Dewey, Active learning with the nnUNet and sample selection with uncertaint--awaan submodular mutual infor- mation measure. In Medical Imaging with Deep Learning, 2024.
Mélanie Gaillochet, Christian Desrosiers, and Hervé Lombaert. TAAL: Test-time Augmentation for Active Learning in Medical Image Segmentation, January 2023b.
Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. International Conference on Machine Learning, pp. 1183-1192. PMLR, 2017.
Lidia Garrucho, Kaisar Kushibar, Claire-Anne Reidel, Smriti Joshi, Richard Osuala, Apostolia Tsirikoglou, Maciej Bobowicz, Javier del Riego, Alessandro Catanese, Katarzyna Gwoždziewicz, Maria-Laura Cosaka, Pasant M Abo-Elhoda, Sara. W Tantawy, Shorouq Sakrana, Norhan O Shawky-AAddlfattal Amr Muham- mad Abdo Salem, Androniki Kozana, Eugen Divjak, Gordana Ivanac, Katerina Nikiforaki, Michail E Klontzas, Rosa García-Dosda, Meltem Gulsun-Akpinar, Oğuz Lafei, Ritse Mann, Carlos Martín-Isla, Fred
14

---

Published in Transactions on Machine Learning Research (01/2026)
Prior, Kostas Marias, Martijn P A Starmans, Fredrik Strand, Oliver Diaz, Laura Igual, and Karim Lekadir. A large-scale multicenter breast cancer dee-mri benchmark dataset with expert segmentations. Scienlific Data, 12(1):453, 2025. doi: 10.108//441997-05000007777.
Karol Gotkowski, Carsten Lüth, Paul F Jäger, Sebastian Ziegler, Lars Krämer, Stefan Denner, Shuhan Xiao, Nico Disch, Klaus H Maier-HHiin, and Fabian Isensce. Embarrassingy simple scribble supervision for 3D medical segmentation. Xix preprinl arXiv:2403. 12834, 2024.
Sanket Rajan Gupte, Josiah Aklilu, Jeffrey J. Nirschl, and Serena Yeung-Levy. Revisiting Active Learning in the Era of Vision Foundation Models, January 2024.
Nicholas Heller, Fabian Isensee, Dasha Trofimova, Resha Tejpaul, Zhongchen Zhao, Huai Chen, Lisheng Wang, Alex Golts, Daniel Khapun, Daniel Shats, et al. The kits21 challenge: Automatic segmentation of kidneys, renal lumors, and renal cysts in corticomeduullrr--phass ct. arXiv preprinl 0xi:2307700188, 2023.
Sture Holm. A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics, pp. 65 70, 1979.
Neil Houlsby, Ferene Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian Active Learning for Classi- fication and Preference Learning. drXiv:1112.5745 [es, stat), December 2011.
Jonas Hübotter, Bhavya Sukhija, Lenart Treven, Yarden As, and Andreas Krause. Information-basse Trans- ductive Active Learning, March 2024.
Fabian Isensee, Paul F. Jaeger, Simon A. A. Kohl, Jens Petersen, and Klaus H. Maier-Hein. nnU-Net: A self-configuring method for deep learning- based biomedical image segmentation. Nat Methods, 18(2):
203 118-211, February 2021. ISSN 1548-7091, 1548-7105. doi: 10. 10.18///141922-00 000--020--1111 2
Fabian Isensee, Tassilo Wald, Constantin Ulrich, Michael Baumgartner, Saikat Roy, Klaus Maier-Hein, and Paul F Jaeger. Nnu-net revisited: A call for rigorous validation in 3d medical image segmentation. In International Conference on Medical Image Computing and Computer-Assio Assisted Intervention, pp. 488 498. Springer, 2024.
Fabian Isensee, Maximilian Rokuss, Lars Krämer, Stefan Dinkelacker, Ashis Ravindran, Florian Stritzke, Benjamin Hamm, Tassilo Wald, Moritz Langenberg, Constantin Ulrich, et al. nninteractive: Redefining 3d promptable segmentation. arXiv preprint ar arXiv:2503.0887 2025.
Yuanfeng Ji, Haotian Bai, Chongjian Ge, Jie Yang, Ye Zhu, Ruimao Zhang, Zhen Li, Lingyan Zhanng, Wanling Ma, Xiang Wan, et al. Amos: A large-scale abdominal multi-organ benchmark for versatile medical image segmentation. Advances in neural information processing systems, 35:36722-36732, 2022.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on compuler vision, pp. 4015 4026, 2023.
Andreas Kirsch, Sebastian Farquhar, Parmida Atighehchian, Andrew Jesson, Frederic Branchaud-Charron, and Yarin Gal. Stochastic Batch Acquisition: Simple Baseline for Deep Active Learning, September 2023.
Shuailin Li, Chuyu Zhang, and Xuming Hc. Shape-aware semi-supervised 3d semantic segmentation for medical images. In Medical Image Computing and Compuler Assisted Intervention MICCAI 2020: 23rd International Conference, Lima, Peru, October 4-8, 2020, Proceedings, Part I 23, pp. 552-561. Springer, 2020.
Zihan Li, Yuan Zheng, Dandan Shan, Shuzhou Yang, Qingde Li, Beizhan Wang, Yuanting Zhang, Qingqi Hong, and Dinggang Shen. Scribformer: Transformer makes enn work better for scribble-based medical image segmentation. IEFE Transactions on Medical Imaging, 23():222 2265, 2024.
15

---

Published in Transactions on Machine Learning Research (01/2026)
Zachary C Lipton and Jacob Steinhardt. Troubling trends in machine learning scholarship: Some ml papers suffer from flaws that could mislead the public and stymie future research. Queue, 17(1):45 77,5--77 2019.
Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen Awm Van Der Laak, Bram Van Ginneken, and Clara I Sánchez. A survey deep learning in medical image analysis. Medical image analysis, 42:60 88, 2017.
Han Liu, Hao Li, Xing Yao, Yubo Fan, Dewei Hu, Benoit Dawant, Vishwesh Nath, Zhoubing Xu, and Ipek Oguz. COLosSAL: A Benchmark for Cold-start Active Learning for 3D Medical Image Segmentation, July 2023.
Luca Lumetti, Vittorio Pipoli, Federico Bolelli, Elisa Ficarra, and Costantino Grana. Enhancing Patch- Learning for the Segmentation of the Mandibular Canal. IEEE Access, pp. 1-12, 2024. ISSN 2169-3536. doi: https://doi.org/// 10. /CC/A/CCCSSS2222..44000
Xiangde Luo, Wenjun Liao, Jianghong Xiao, Jieneng Chen, Tao Song, Xiaofan Zhang, Kang Li, Dimitris N Metaxas, Guotai Wang, and Shaoting Zhang. Word: A large scale dataset, benchmark and clinical applicable study for abdominal organ segmentation from et image. Medical Image Analysis, 82:102642, 2022.
Carsten Lüth, Till Bungert, Lukas Klein, and Paul Jaeger. Navigating the pitsalls of active learning evalu- ation: A systematic framework for meaningful performance assessment. Advances in Neural Information Processing Systems, 36:9789 9836, 2023.
Carsten T. Lüth, Jeremias Traub, Kim-Celine Kahl, Till Bungert, Lukas Klein, Lars Krämer, Paul F Jaeger, Fabian Isensee, and Klaus Maier-Hein. nnactive: A framework for evaluation of active learning in 3d biomedical segmentation. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL
https://openreeieer net//foruurrtessJJaall........
Jun Ma, Yuting He, Feifei Li, Lin Han, Chenyu You, and Bo Wang. Segment anything in medical images. Nature Communicaatins,, 15(1):654, 2024a.
Siteng Ma, Haochang Wu, Aonghus Lawlor, and Ruihai Dong. Breaking the Barrier: Selective Uncertainty- based Active Learning for Modical Image Sogmentation, Jamuary 2024b.
Lena Maier-Hein, Annika Reinke, Patrick Godau, Minu D Tizabi, Florian Buettner, Evangelia Christodoulon, Ben Glocker, Fabian Isensee, Jens Kleesick, Michal Kozubek, et al. Metries reloaded: recommendations for image analysis validation. Nature methods, (2): 212, 2024.
Sudhanshu Mittal, Maxim Tatarchenko, Özgün Çigek, and Thomas Brox. Parting with Illusions about Deep Active Learning, December 2019.
Prateek Munjal, Nasir Hayat, Munawar Hayat, Jamshid Sourati, and Shadab Khan. Towards robust and reproducible active learning using neural networks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 223 232, 2022.
Vishwesh Nath, Dong Yang, Bennett A. Landman, Daguang Xu, and Holger R. Roth. Diminishing Uncer- tainty within the Training Pool: Active Learning for Medical Image Segmentation. IEEE Trans. Med. Imaging, 20(10):2534-25)) 25477, October 2021. ISSN 0278-0062, 1548-254X. doi: N0.1109TMM.2020.0 3048055..
P. Nemenyi. Distribution-free multiple comparisons. PhD Thesis, Princeton University, 1963.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. ar xrx:::1505 15550044000 [cs), May 2015.
Ozan Sener and Silvio Savaresc. Active Learning for Convolutional Neural Networks: A Corc-Set Approach. arXiv: 70..70..000 00489 (cs, statf, June 2018.
Burr Settles. Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison 2009.
16

---

Published in Transactions on Machine Learning Research (01/2026)
Burr Settles. From theories to queries: Active learning in practice. In Isabelle Guyon, Gavin Cawley, Gideon Dror, Vincent. Temaire, and Alexander Statnikov (eds.), Active Learning and Experimental Design Workshop in Conjunction with AISTATS 2010, volume 16 of Proceedings of Machine Learning Research, pp. 1-18, Sardinia, Italy, May 2011. PMLR.
Jun Shi, Shulan Ruan, Ziqi Zhu, Minfan Zhao, Hong An, Xudong Xue, and Bing Yan. Predictive accuracy- based active learning for medical image segmentation. In Proceedings of the Thirly Third International Joint Conference on Artificial Intelligence, IJCA1-24, pp. 4885 4893. International Joint Conferences on Artificial Intelligence Organization, August 2024a. doi: 10.24963 2/0/3//jaai2224//553
Yuyan Shi, Jialu Ma, Jin Yang, Shasha Wang, and Yichi Zhang. Beyond pixel-wisc supervision for medical image segmentation: From traditional models to foundation models. arXiv preprint arXiv:2404.1323, 2024b.
Nima Tajbakhsh, Laura Jeyaseelan, Qian Li, Jeffrey N Chiang, Zhihao Wu, and Xiaowei Ding. Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation. Medicol image analysis, 63:101693, 2020.
Arvind Murari Vepa, Zukang Yang. Andrew Choi, Jungseock Joo, Fabien Scalzo, and Yizhou Sun. Integrating deep metric learning with coreset for active learning in 3D segmentation. In The Thirty-Eighth Annual Conference Neural Information Processing Systems, 2024.
Tassilo Wald, Constantin Ulrich, Stanislav Lukyanenko, Andrei Goncharov, Alberto Paderno, Maximilian Miller, Leander Maerkisch, Paul F Jäger, and Klaus Maier-Hein, Revisiting mae pre-training for 3d medical image segmentation. Xixiv preprint arXiv:2410.238 13..3332, 2024.
Xueying Zhan, Huan Liu, Qing Li, and Antoni B Chan. A comparative survey: Benchmarking for pool-based active learning. In IJCAI, pp. 4679-4686, 2021.
Xueying Zhan, Qingzhong Wang, Kuan-hao Huang, Haoyi Xiong, Dejing Dou, and Antoni B. Chan. A Comparative Survey of Deep Active Learning, May 2022.
Zongwei Zhou, Vatsal Sodha, Jiaxuan Pang, Michael Gotway, and Jianming Liang. Models genesis. Medical image analysis, 67:101840, 2021.
Author Contributions
This work was carried out over 6 months and the core idea for the algorithm was developed by Fabian Isensee, Carsten Lüth, and Jeremias Traub. The exact implementation of the algorithm done by Carsten Lüth, and then Jeremias Traub and Carsten Lüth designed the experiments and ablated the design decisions. All experiments were performed by Jeremias Traub. The writing was done by Jeremias Traub and Carsten Lüth in equal parts with reviews by all other authors.
17

---

Published in Transactions on Machine Learning Research (01/2026)
Appendix
Table of Contents
A Task Description
19
ClaSP PE Algorithm
19
Dataset Details
22
D Active Learning Framework
22
D.I Evaluation Metrics
24
D.2
Experiment Details
25
E
nn Antivee Benchmark Results
26
F.1
Results aggregated over Main Benchmark and Patchx, Setting
26
E.2
Main Benchmark Results
................................
29
E.3
Patchx Setting results
................................
33
E.4
500 Epochs Setting results
36
E.5
Analyzing ClaSP PE performance on AMOS on a class level
37
E.6
Comparing Pairwise Penally Matrix with different p-values
40
E.7
Mean Performance estimate
42
Guidelines for Real-World Deployment of ClaSP PE
43
G
Roll-Out Results
Limitations
44
Qualitative Results
45
Query Visualization
45
Stratification Visualization
19
18

---

Published in Transactions on Machine Learning Research (01/2026)
A Task Description
As we use the AL framework proposed by Lüth et al. (025 we refer to their work for detailed task (Appendix B). Here, we only provide a high-level overview.
In the context of Active Learning (AT.) for 3D biomedical image segmentation, acquiring complete annota- tions for entire volumetric scans is often prohibitively expensive and time-consuming, due to the need for expert annotators and the high dimensionality of the data. To address this, recent approaches advocate for the use of partial annotations, where only selected subregions of 3D image such spatial patches- labeled. This strategy enables models to learn effectively while significantly reducing annotation effort. The AL process is thus centered around a query method that strategically selects informative regions to annotate, allowing training to proceed using only a subset of the full data.
This framework can formalized by considoring the training data as 3D volumetric images E RMMxXXXD with dense labels Y E fI...., C) #xWxP, Rather than providing the full Y, query function Q(X) identifies subsets Y C Y for annotation. Specifically, this work focuses querying 3D patches within each image, defined by locations and patch sizes. During training, only the labeled regions Y are used to compute the loss, with the unannotated portions ignored or treated with weak supervision. This partial supervision spt allows the AL framework to scale efficiently to large 3D datasets without the prohibitive cost of full annotation.
ClaSP PE Algorithm
H[D]
Hpl-Pp
10
0.5
08
0.4
022
T
00
1
Figure 7: Ternary plot visualizing the difference of the entropy u H [p) and our proposed class-specific measure U1 = Hpp Pı for E (1,2,3).
We start by giving a short recap of our proposed query method (QM) to introduce the notation. Followed by additional implementation details to support reproducibility by means of two complementary representations of the algorithm for ClaSP PE.
Class Stratified Sampling Given image uncertainty map u(x), and predicted class probabilities (r(x)) p(Y = cla)) we oblain the class-specific scores
ue(x) (x) uxx)
A direct example of how these class specific scores bchave in a class scenarios is visualized in fig. We then select samples in a stratified fashion for each class e based on ua, respectively. To our knowledge, this approach of balancing the queries using stratification has not been used in the AL literature before. Crucially, we do not select all samples with the stratified approach but only a fraction a with the remaining a samples being selected based on the standard uncertainty map u(x) to retain sensitivity highly uncertain examples regardless of class distribution.
19

---

Published in Transactions on Machine Learning Research (01/2026)
An Exponential Scheduler for Score Perturbation via Log-scale Power Noising Our exponential
scheduled power-noising is straight extension of the work by Kirsch et al. (0233 works as follows:
$ClaSP PE(() log sCla PE + c(t)
(4)
where
(1) ~ (0,
(5)
with & E (O,.., which represents the current AL cycle where T is the maximum number AL cycles counting only those with a Query step. fg is the initial value, while Smax is the final value for the last cycle.
8(t) exp([1 $1 (no0) + T (a(((aaa))
(6)
First, provide Python-style pscudocode in algorithm that abstracts away specific implementation details, focusing instead on the core structure and logic of the method. Second, we present fully detailed algorithmic version that outlines our exact implementation inside the nnActive framework shown in algorithm [ This combination provides a high-level overview while also being transparent about our implementation.
Implementations
As the high-level Python-style pscudocode abstracts away the patches, it therefore can serve as foundation for implementations where overlap checks not necessary.
Algorithm Abstracted ClaSP PE in a Python-style pseudocode with patches abstracted away
Input: unlabeled ppol:: unlabeled dataset, model: python model, t: current loop, max loop with query, beta. starting beta, beta, max: final beta, alpha: fraction stratified, num classes: number of classes, query size
PseudoCode
images =
for in unlabeled pool: # Computing ClaSP PE for sample (orwar)))
u entropy(p)
11 C cat (pfwhhoot class] * unsqueczo(u, 0), unsquecze(u, 0))
u + gumbel noise(u noise(u_c.shape, (xp(-(1-t/))lln(betta 0) t/T *In(beta (abeeta_max))
images.append(u
# Selecting Query over entire samples __dugeets- = floor(n' Caannaaa query = for e in Bange(C[withunn bg class)):
best = argsort(u trgsort(u_imagcss:..o c))
best.pop(i) for in query
query.append [best[::-]]ss [ulgees_)
best argsort(u f_aaaasmmmsss, c))
best.pop(i) for in query (uery [:=1][1- (s budgels}*C))
return query
20

---

Published in Transactions on Machine Learning Research (01/2026)
Algorithm 2 Exact ClaSP PE algorithu implemented in the nnActive Framework
Input:
Timl: query size n, labeled set L, Uncertainty function U, number of classes C, fraction class specific a, aggregation method with scheduled powernoiseing (A)
Set of images (X
Output: Final query set Q
ê {0]==1 Initialize stratified query set for each image X {X do
+ M(X) # compute probability for image # compute uncertainty for image 5: UAgg + A(U) # aggregate uncertainties to patch-level C1 # initialize best patches for current image do
U+ U(X(),
6:
Qluage + [0]
7:
for CE Shuffle(f1....
U.
UGARK + A(U) # aggregate uncertainties patch-level
10:
for q in Aogg( do # sort in descending order according to uncertainty
11:
if overlap(g, Oimage U S then # ensure that Qm.Im &- Qc, U {4)
12:
13:
end if if len (c.Image) >= a * n/C then Break end if end for
14:
15:
16:
17:
18:
end for
19:
for in sort(UAgg)[:: (::1]] do # sort in descending order according to uncertainty
20:
if overlap(y, Qimago UC) < then # ensure that
21:
2c+1,Image + 2c+1. U {g}
22:
end if
23:
if len (e0c++1,mmae)) >= a nnnnCCC then
24:
Break end if 26: end for
25:
27:
Q + QU Oimage
28: end for
29: for CE (1, ..., C} # Build final query with stratified samples do
30: Q Q U sort(Q.)[:: -1]: a ***** n/C1
31: end for
32: + uort(Q...) 1][: (a n/C)] # Add unstratisfied samples
33: Return Q
21

---

Published in Transactions on Machine Learning Research (01/2026)
C Dataset Details
Key characteristies of the datasets used in the nnActive benchmark (soction dircctly match with Lüth et al. (2025) and are shown in table 3 For the roll-out study (section dataset characteristics ics are shown in table All images are resampled to the median dataset spacing. Further details on the different segmentation tasks are given in table
The MAMA MIA dataset is additionally preprocessed using only the subtraction image where the pre- contrast image is subtracted from the first available post-contrast image.
Table 3: Dataset details and configurations for the nnActive study.
Dataset
ACDC
AMOS
KITS
Hippocampus
# Classes w.o. Background
3
15
2
Median Shape
16.5x237x206
237.5x582x582
526x512x612
36x50x35
Used Spacing
2x0.6875x0.6877
5x1 1.505xx1.662
0.78125x0.78 75150007815x0...00 78125
1x1x1
# Pool & Training
150
150
225
195
Validation
50
75
65
Query Patch Size
4x40x40
32x74x74
60x64x64
20x20x20
Budget: Low # (% Voxels)
150 (0.75%)
200 (0.20%)
200 (0.16%)
100 (6,51%)
Budget: Medium
Patches] (% Vozels)
300(1.50%)
1000 (1.30%)
1000 (0.80%)
200 (13,02%)
Budget: ligh [# Patches] (% Voxels)
(50(2.)5%)
2500 (3.25%)
2500 (2.00%)
300 (19,54%%
Query Patch Size (Patchx
2x20x20
16x37x37
30x32x<22
10x10x10
Budget: Low Patches] (% Voxels)
150 (0.09%)
200 (0.03%)
200 (0.02%)
100 (0.77%)
Budget: Medium # Patches]] (% Voxels)
300(9.19%)
1000 (0.16%)
1000 (0.10%)
200 (1.63%)
Budget: High [# Patches] (% Voxels)
450(0.28%()
2500 (0.41%)
2500 (0.25%)
300 (2.44%)
Test set Mean Dice (1000 Epochs)
0.912
0.893
0.773
0.895
Test set Mean Dice (500 Epochs)
0.912
883
0.751
0.895
Test set Mean Dice (200 Epochs)
0.910
0.860
0.705
0.895
Table 4: Dataset details and configurations for the roll-out study.
Dataset
LiTS
WORD
Tooth Fairy 2
MAMA MIA
# Classes w.o. Background
2
16
42
1
Median Shape
495x512x512
200x512x512
169x344x371
80x256 256
Used Spacing
1x0.7676x0.7676
3x0.9766x0.9766
0.3x0.3x0.3
2x0.7031 x0.7031
# Pool & Training
99
90
360
1130
# Validation
32
120
376
Budget # Patches] (% Voxels)
750 (0.19%)
4,000 (15.8%)
10,500 (4.5%
500 (0.09%)
Query Patch Size
28x44x39
29x74x87
33x34x35
16x48x57
Test set Mean Dice (1000 Epochs)
0.799
0.845
0.752
0.765
Test set Mean Dice (500 Epochs)
0.797
0.829
0.745
0.746
Test set Mean Dice (200 Epochs)
0.773
0.807
0.726
0.710
D Active Learning Framework
Our work builds directly on the existing nnActive framework (Ltth et al. 2025 preserving its design choices to ensure seamless applicability in both benchmarking and real-world annotation workftows. To maintain compatibility with the nnU-Net training and data management pipcline, all annotation updates are performed within the nnU-Net dataset structure. In particular, we store all queried patch meladata in loop_ boo_XXX.jssom files within the unUNet_ raw folder, where each file corresponds to a particular AL loop and contains information about the queriod regions. These modifications in the nnUNet_ now dircc- tory are automatically reflected in the preprocessod datasets used for training by running the standard an UNe. preprocessing step. For the query stage, follow the patch-wise inference strategy of nnU-Net.
22

---

Published in Transactions on Machine Learning Research (01/2026)
Table 5: Foreground class names for all datasets.
Dataset
Class names in order of labels (ascending)
ACDC
right ventricle, myocardium, left ventricular cavity spleen, right kidney, left kidney, gall bladder, esophagus, liver, stomach, aorta, postccav,, pancreas, right adrenal gland, left adrenal gland, duodenum, bladder, prostate/uterus
AMOS
Hippocampus
anterior hippocampus, posterior hippocampus kidney, kidney-tumor, kidney-cyst
KiTS
LITS
liver, cancer
WORD
liver, spleen, left kidney, right kidney, stomach, gallblader, esopha- gus, pancreas, duodenum, colon, intestine, adrenal, rectum, bladder, Head of femur L, Head of femur R
Tooth Fairy 2 Lower Jawbone, Upper Jawboue, Left Inferior Alveolar Canal, Right Inferior Alveolar Canal, Left Maxillary Sinus, Right Maxillary Sinus, Pharynx, Bridge, Crown, Implant, Upper Right Central Incisor, Upper Right Lateral Incisor, Upper Right Canine, Upper Right First Premolar, Upper Right Second Pre- molar, Upper Right First Molar, Upper Right Second Molar, Upper Right Third Molar (Wisdom Tooth), Upper Left Central Incisor, Upper Left Lateral Incisor, Upper Left Canine, Upper Left First Premolar, Upper Left Second Pre- molar, Upper Left First Molar, Upper Left Second Molar, Upper Left Third Molar (Wisdom Tooth), Lower Left Central Incisor, Lower Left Tateral Incisor, Lower Left Canine, Lower Left First Premolar, Lower Left Second Premolar, Lower Left First Molar, Lower Left Second Molar, Lower Left Third Molar (Wisdom Tooth), Lower Right. Central Incisor, Lower Right Lateral Incisor, Lower Right Canine, Lower Right First Premolar, Lower Right Second Premo- lar, Lower Right First Molar, Lower Right Second Molar, Lower Right Third Molar (Wisdom Tooth)
MAMA MIA
lesion
23

---

Published in Transactions on Machine Learning Research (01/2026)
After all ensemble members have predicted each image, the AL method is applied in final step to compute uncertainty maps and select patches to be labeled. Our implementation of standard top-k uncertainty-based methods, such as PE or BALD, follows the algorithm described in algorithm
Algorithm 3 Active Learning Patch Selection
Input:
Set of images (X) 1=11 query size labeled set L, Uncertainty function U, Aggregation function A, allowed overlap Output: Final query set Q
Initialize final query set Q ++0 0
for each image X (X()) do
3: u U+-U(X U(X(), # compute uncertainty for image
4: MAgs + A(U) # aggregate uncertainties to patch-level
5: Oimage + # initialize best patches for current image
6:
for in sort (org((//gg]|:--] do # sort descending order according to uncertainty if overlap(2 Qimago U C) S then # ensure that
Oimage t Omage {g} end if 10: end for
11: Q t QU Oimage
12: end for
13: Q + sort(Q)[:-11 # sort in descending according to uncertainty
14: Return Q
D.1 Evaluation Metrics
We adopt the comprehensive set of evaluation metrics used in the mnActive benehmark Lüth et al. 2025 to assess the performance of different QMs.
Final Dice The Final Dice score reflects the segmentation performance after the full annotation budget has been spent. It particularly emphasizes the effectiveness of a QM in the later stages of AL and allows for straightforward interpretation.
Area Under the Budget Curve (AUBC) The AUBC measures overall performance across the entire AL trajectory. It is computed as the area under the Mcan Dice curve using the trapezoid method. Higher values indicate better performance. We normalize AUBC such that it lies in the range [0, 1]. We refer to 2022 for further details.
Zhan et al. 2021
Pairwise Penalty Matrix (PPM) The PPM compares methods pairwise using a two-sided t-lest with significance level = 0.05 (see (Ash et al. 2020) for further details). It quantifies how often one method significantly outperforms another across datasets and Label Regimes. Each row shows the fraction of wins, and each column shows the fraction of losses, expressed in percentages.
Foreground Efficiency (FG-Eff) We use FG-Eff as metric for annotation efficiency, quantifying how quickly a method reaches full-data performance as function of the annotated foreground voxels (a proxy for annotation effort.. FG-Eff is based on fitting an exponential decay curve:
(t) = (o(fo) - Iran) exp((>7tt6x) + Øtall
(7)
Here, / E [0, 1] is the fraction of annotated foreground voxels, Itul is the model's Dice score on the full dataset, and ((()) is its performance on the starting budget. higher (FG-EIT) indicates faster convergence to full performance with less annotation.
FG-Eff complements performance metrics by quantifying annotation efficiency. good QM performs well in terms of FG-Eff and traditional metrics (Final Dice, AUBC, PPM). High FG-Eff with low overall performance
24

---

Published in Transactions on Machine Learning Research (01/2026)
should be viewed skeptically, the metric can be hacked by querying very small amount of foreground. Importantly, FG-EIf is only meaningful when QMs are compared under the same model, training regime, and annotation budgets, since Utall and }(fo) are experiment-dependenn We refer to Lüth et al. (2025) for further details.
D.2 Experiment Details
For the AL experimental setup, we follow Lüth et al. (0025): We use a starting budget and query size equal to 20% of the full annotation budget of each Label Regime. To ensure a representative starting budget, it is allocated to sample random foreground regions of each class, so that all classes are present in at least two patches. The rest of the starting budget is selected using the Random 33% FG strategy. Details on the annotation budget and query design for each nnActive benchmark dataset are provided in table For the roll-out datasets (table 4), we employ the guidelines detailed in appendix F
We use unU-Net (Isensee et al. 2021), self-configuring deep learning framework, our segmentation model. If not explicitly stated otherwise, all models are trained for 200 epochs using the 3D full resolution configuration of nnU-Net. To increase model robustness, we use an ensemble of five models trained via 5-fold cross-validation. We perform complete retraining of the models for each AL loop. The training of the models themselves is not seeded, but all fataset-related parameters are. All results are averaged across four seeds.
Hyperparameters We directly took the Random FG configurations from nnActive (Lüth et al. 2025 As standard 3 values for PowerBALD, PowerPE and SoftrankBALD we used as detailed in Kirsch et al. (2023) and following the evaluation in Lüth et al. (2025). For a, the fraction of samples that is selected using the stratified approach in ClaSP PE, we compared 33% and 66% as shown in our ablations, following the same values as the FG percentage of the Random FG methods. The initial and final noising strength, 30 and Smaz 100 were chosen following the evaluation of Lüth et al. (2025) (Appendix G.3), which parsed a similar range showing that the most crucial factor is a general reduction of ß for larger annotation budgets, and further tuning of the method parameters
done.
For the Tooth Fairy 2 dataset, we train without mirroring. For runtime savings, we omit Test-Time Aug- mentation during validation for MAMA MIA and Tooth Fairy 2 and we set pred tile__isteeee_ size 0.75 for inference on MAMA MIA.
Compute Resources All experiments are performed as single-GPU trainings on A100 GPUs. In total, the large-scale evaluation of the ClaSP PE method on the nnActive benchmark and the roll-out datasets required around 20,000 GPU hours, each with around 180 GB of RAM.
25

---

Published in Transactions on Machine Learning Research (01/2026)
E nnActive Benchmark Results
In this section, we provide detailed results on the nnActive benchmark. We refer to the nnActive main benchmark as the experiment configuration described in Section 5.1 in Lüth et al. (2025), which encompasses 12 distinct settings across 4 datasets and Label Regimes. Further exlending the method evaluation, Lüth et al. (2025) define Patchx & setting, which uses query patch size that is halved along each dimension compared to that of the main benchmark. The specific settings are provided in table 3
E.1 Results aggregated Main Benchmark and Patchx Setting
The results presented in this section are aggregated over both the main benchmark and the Patch setting, resulting in 24 distinct experiment configurations across 4 datasets, Label Regimes, and 2 query patch sizes. Specifically, fig. [ shows the results of Nemenyi post-hoc tests, based on Friedman tests (Demšar) 2006), to analyze the signilicance of performance differences, and fig. shows the PPMs for each dalaset.
The Friedman tests are conducted across all k = methods under comparison using N = 24 configurations (i.c., 24 paired performance outcomes per method) and show significant results (at 0.05). The Nemonyi post-hoc analysis evaluates all pairwise differences in average ranks. Using the standardized z-score of a Nemenyi test with ranking difference A (Demšar) 2006)
k(k 1)
A
6N
we can compute the effect size as r- 2/VN. In our setup, following Cohen's guidelines (Cohen 1988), the effect sizes of small (r 0.1), medium (r 0.3), and large (r = 0.5) correspond to the average ranking differences of A = 0.39, A 1.16, and A 1.94, respectively. As an cxample, an average ranking difference of 0.5 would correspond to small effect size of around 0.129, which highlights the conservative nature of the Nemenyi test, particularly with many methods and relatively small sample size (Nemenyi 1963). meaning that some practically meaningful differences may not reach significance.
We report the exact p-values in fig. S and use a significance threshold of 0.05 to form the groups shown in fig. [ The resulting significance groups should be interpreted as exploratory evidence rather than definitive proof of method superiority; indeed, the Nemenyi test conservative, which means that the significant separations observed in our results likely understate, rather than overstate, the true differences between methods [emenyy 1963
26

---

Published in Transactions on Machine Learning Research (01/2026)
Nemonyl with
(p=0.00%
Post Hoc Nemenyl tast for Rinal Dice with Friedman tost (p- 0.00%
DAUD
1:20
3.50
1.03
3.03
SALD
1.00
1:00
1.02
5.50
0.22
1.02
PowerDALD
1.20
3.00
1.02
0:32
1.20
1.00
1.00
1.00
1.50
2.50
0.9
1.03
3.08
Nemood
Proditive Entropy
1.20
1.00
1.00
1.00
1.50
2.00
1.00
1.03
3.03
Prud Entropy
1.00
0.92
0.93
2.00
0.72
FrwerFFE
1:00
n0
1.00
00
1:00
3.00
1.0
1.03
3.05
POWCRE
1.00
1:30
2.50
0.99
1.09
6:00
1.00
1.05
fandom
0.00
0.00
0.00
0.00
3:00
1:00
0.01
0.03
3.00
Random
0.00
0.90
0.00
0.00
0.00
2.00
9.00
0.00
0.00
Random 73% Rq
1:00
0.46
0.94
1.00
1:00
9.21
1.00
0.46
3.00
31%
0.09
1.30
0.71
1.00
5.00
1.00
1:02
Random 56% PG
1.00
1.00
1.00
1.00
2.00
2.20
0.96
1.00
2.25
ClusP re
0.03
0.12
0.08
0.00
2.25
2.50
0.00
0.15
2.23
BALD
fowe
Predrru thorpy
PowoTe
Rundon
Sandon 1744
Clss
SebuntsAL
Bunden
fardsm 32%
Kendon
Poetetbv
Query
Query Nerhod
(a) AUBC
(b) Final Dice
Post-Hoc Nemenyl best for FG-E!! with Friedman test (p-0.00%)
Query Method
0.00
0.
0.01
Random 33
0.99
1.00
0.00
1:00
Randum 69% FG
1.00
0.01
0.53
3.00
0.00
0.01
0.30
1.00
0.04
CluSP
0.02
0.69
0.00
0.99
1.02
1.00
0.04
1.00
Rundom 698
Rundom
Quary Method
(c) FG-Eff
Figure 8: p-values for the Nemenyi post-hoc tests, based on Friedman tests, on the nnActive benchmark for all evaluation metrics. Results are aggregated across datasets x Label Regimes x 2 query patch sizes. The corresponding significance groups for p = 0.1 are indicated in fig. 2
27

---

Published in Transactions on Machine Learning Research (01/2020)
Parnioo Nomalty Marre (Pai
PIECHASSE ACBO PHL
Palrerie Se Peraily Matra: pWai PabChw AMDS PHE
BALD.
BALD
PowerooD
1/
17
60
PHAEWRALD
11
31
Agerttn akgutftrra
Aggettin arpettone
Nwart
5
60
Toworte
10
Rendhen
Randees
67
13
Wansom 30%-PG
5.7
3.5
6.7
5F
Tandon 23% PC
87
50
73
77
73
3.3
Kavism 6696 F0
13
17
100
6.7
20
63
23
3.3
fandon 66% FO
80
80
77
77
80
80
77
TE
1.3
6.7
Cudd 72
BrtresibAL
Prerppe
Ranisom
dasmn re
Peodl Eneopy
Pleaong
Randiom
Raudim 5WV
Namstm 688
class 1
Rendaam
Agerbrm cutseffered by
Alaar thes outpe formes
ACDC
AMOS
3.3
PowerRLLD
5.3
10
5.7
PowERBALD
10
10
5.7
SeTroFBBALD
10
13
30
2G
10
34T/OHBALD
3.3
20
0
5.7
P2
47
47
Proiictie Evmmy
6.7
21
1.
17
33
27
27
6.7
Prederive Eet Entray
13
21
37
23
42
40
Iavort
10
13
Poworfe
6.7
15
Rondrem
Randoee
3.3
10
17
AB
Cudf
57
33
Moar
12
0.8
1.7
8.3
24
19
10
1.7
Moan
1.8
24
15
8.5
23
35
BALD
New Antivve Fierary
Ramisom
Ramdonn K3Pe
Pimani doe
Gasrre
BALL
Wrededive Knerapy
Rardive
Randim 33N
Remtee Pd
Hippocampus
(d) KITS
Figure 9: Pairwise Penalty Matrices aggregated over all Label Regimes and both query patch sizes for each dataset.
28

---

Published in Transactions on Machine Learning Research (01/2026)
E.2
Main Benchmark Results
The results shown this section are obtained on the nnActive main study settings. Detailed results for AUBC, Final Dice, and FG-Efff including standard deviations based on four seods, are provided in table 0
The table includes results for the methods Cla PE 66% and 33%, as assessed in section 4.2 The overall PPM is shown in fig. 10 the respective dataset-specifie PPMs are in fig. 11
29

---

Published in Transactions on Machine Learning Research (01/2026)
Table 6: Fine- sene-grained Results for the nnActive Main Study for each dataset. Higher values are better, and colorization goes from dark green (best) to white (worst) with linear interpolation. AUBC and Final Dice are multiplied 100 for improved readability. AUBC, Final, and FG-Eff can only be directly compared within each Label Regime on each dataset. The respective dataset characteristics are detailed in table
(a) ACDC
Datasct
ACDC Medium
Label Regine
Low Fizel Dice
High Finnd Dike
Melric
AUBC
FG-EIl EIl
AUBC
Fiand Dice
PG-EI
AUBC
FG-BA
Query Methorl
BALD
86.44
0.39
26.98
86.85 0.45
82
0.15
21.91
87.
0.33
0ML-47
0.18
15.00
PowerBALD
81.18
0.58
46
0.55
46 10
+ 15.10
85.63
0:37
80.07
0.21
21.75
4.00
87.50
0.44
80.80
0.17
1V.04 1.3
SoAtranBBALD
50.71
0.95
0.27
0.14
11.53
Preticlive Entrogy
80,02
54
86.54
0.95
26,09 1 4.40
85
53
0.50
89 82
1 0.07
21.16
1: 3.11
87 67.05
0.27
90.52 1: 0.06
13.58 1: 1.22
Power PI
86.56
86
0.22
Random
76.63 = 0.81
80.34
1.64
30:25
+ 33.50
82:24
1,25
83,46
0.57
35.22
8.43
84.89
0.96
86,28
1.08
21.02 + 3.29
Random 33% FG
91:25
0.56
85,00
1:15
10.58
9.71
0.65
$7.51
0.50
21.20
1.49
96
$0,000
0.04
15.81
1:41
Random 66% FG
62.32 1: 0.33
86,70
0.28
31.20
1: 4.32
86 16
0.44
02
0.52
18.95
2.13
87. 56
0.33
01.9111
0.00
13:41 1: 0.79
Cla PE 33%
81.00
0.74
86.38
0.84
28:70
2.71
85 67
0.55
57
0.08
194.93
2.22
87 83
0:37
900.00
0.30
14.04 +: 0.92
0076
82.12
0.71
87.45
0.87
28.30
81
0.23
69.
0.15
18.04
1,39
88.05
0.15
00.55
0.06
13.86
ClaP PE
80.40 0.55
80.11
0.50
39.92
8.07
$6.33
0.67
S0.27
0.47
55.61
6.300
87 01
0.35
89.97
0.12
19.27
2.01
ClaSP
91.31
0.78
0.31
31.97
0.00
(b) AMOS
Dataset Regine
AMOS
Medium Pinal Dice
High Final Dice
Metric
AUBC
Final Dice
PG-EXI
AUBC
PG-BIT
AUBC
PG-BIT
Query Method
BALD
38
1. 2.34
31.05
1.58
22 65
52.56
2.71
59. 26 1. 2.73
1.49
0.22
09.38
0.70
71.95
2.38
0:45 0.20
PowerBAL BALD
50.34 1: 3.00
56:18
1.24
3.67 + 14.54
11.11
1:47
75.02 1 2.01
18.10
0.44
77.88
0.14
80.
18
0.18
8.76 0.08
BAADD
44.451
45.75
0.95
-11.37
4.19
00.01
0.60
66.72 + 0.65
0.10
76.22
1.46
23
1.18
3.51 0.30
Predictive Entrropo
38.02
+ 3.35
39.19
6.79
-17.91
48
$6,10 # 1,78
62.07 + 1.30
2.62 + 0.17
11.27
1.32
$0.79
2.07
1.01 + 0.41
PowerPE
19.78
2.50
0.02
07.92
50.52
Random
42:26 +: 2.56
36.96
1: 2.02
-134.74
4: 88.92
54.66 1: 2.82
56.22
4: 4.61
10.09
+: 3.26
73.88
0.50
75.48 4: 0.33
7/33 e: 0.62
Random 33% PG
1.03
11.41
71.T8
1.16
75.600
0.37
96:44
2.54
T9.53
0.19
14.42 0.47
Random 66% FG
62.81
1,88
71.11
1.42
03.64
1: 9.81
74.87
1. 0.64
80.72
1. 0.54
32,50
6.08
800
0.19
881 81 0.52
12:32 0.43
Cl PE 33%
45.01
+ 2:14
40.855
1:01
-6.04
3.50
64:200
2.09
71 11.6441
+ 3.62
6.62 + 0.21
79.32
0.49
83.57
0:30
6.06
0.04
66%
1.49
53.995
1.75
1.10
60.00
1.71
78.50 + 0.9/2
90.84
0.18
0.07
V:47
ClaP PE
53.60 +: 2.03
39.60
3.02
15,56
13.73
20.61
1.45
78.51
4 0.52
25.17 0.97
79.83
0:24
60.22
2: 0:26
11.34
0:27
ClsEP
4.15
71:28
1.23
79.51
0.29
20,01
0:12
54. 10.10
0:18
(c) Hippocampus
Dataset Regime
Hippocatmpus Medicm Final Dice
High Final Dice
Mesric
AUBC
Final Dior
FC-RF
AUBC
PC-BI
AUBC
RC-FIF
Query Method
BALD
58 + D:03
88
+ 0.06
9.68
0.
S8
79
0:02
89.18
0.07
4.52 + 0.06
S0.03 + 0:05
SD
+ 0.00
3-18 :: 0.12
Powe
56 1 0.08
87.77
2 0.11
9.21
0.19
88.70 1 0.01
86 1 0.06
5.56
0.07
$8.98 1 0.07
$9.29 1 0.10
3. 3.90
0.15
SoftrankBALD
58.44
0.11
88.93
0.18
9.61
0.98
88.72
0.08
82.12
0.02
3.90
0.05
$0.03
0.06
59.42
0.07
3.60 0.12
Predecive Eatropy
$8.50 + 0.06
88.90
0.10
9.75
1.01
85.81
0,04
89.18 0.07
4.23
+: 0.06
88.07
+ 0.07
50.54
+ 0.03
3.73
0.15
PowerPE
56.16 1 0.08
58.70
0:11
9:25
0.52
88.63
0.09
80.07
0.21
4.41
0.10
$8.97
1 0.07
59.93
1 0.18
4.38
0.24
Random
88.07 + 0.10
88.58 + 0.08
0.47 8.76
0.11 $8.65
89.07
0.04
5.10
0.08
$8.96 + 0.09
99.29
+ 0.20
4.41
0.25
Rondon 33% FG
88
3:8G 0.16
Random 68% FG
98.24 + 0.10
88. 76
0.14
9.88
0.73
S8.63
0002
8000
0.04
4.21
0.03
$8.92 + 0.08
40.26 + 0.06
3:11
0.11
Cla PE 33%
0.84
0:06
3-48
0.21
Cla PE 60%
88.23 1 10
88. 90 - 0.14
8.99
0.64
77 1 0:08
89.08 1 0.12
4.02 + 0.08
89 03
1 0.06
50. 26 4 0.08
3.51
1: 0.14
PE
0.06
0.08
5.25
$8.91
0.11
ClaSP PE
88.
1: 0.12
85. 89
- 0.13
9.59 0.71
88, 70
+ 0.11
89.15 4 0.14
1.79 + 0.11
88.97 1: 0:11
50.01 4: 0.00
3,56 0.22
(d) KiTS
Dataset
Label Reginee
Low Final Dice
Medium
Hlghh Final
Metric
AUBC
FG-ER
AUBC
Fina Dice
AUBC
Query Method
BALD
10.58
41003
3.18
7,96
55.06
20
1919
9.37
BAWL
45.10 2.91
47.07
3.63
25.24
54.53
1.40
30.51
1.15
10.16 0.41
61.24 + 0.57
65.04
D:81
11.92 + B.64
SchraakBALD
42,87 2.91
47.12
3.34
12:41
2:03
54.83
1.79
61.44
2.02
6.90 = 0.27
62.49
0.74
87.00
0.97
9.84
0.66
Precictive Entropy
40.62 2.74
45.53
3.57
7.05
A 0.64
57.42
0.54
65.39
1 0.51
6.19 +: 0.10
64.00
11 0.15
68.74
0.65
7.84 1: 0:21
PowerP
15.50
2.05
19:02
1:15
28,70
34
51.76
1110
58.07
1: 1.55
9:68
0.28
60006
0.66
63
9.62
0151
Random
38.15 11 3.36
19.19
1: 4.13
28.47 1 19.48
47.80
1.84
48.11
4: 1.99
4.03 4: 2.75
53.80
1: 0.88
55.12
1.27
8.03
1.22
Random 33% PG
43.70
0.87
07.35
:: 2.10
10.19
1.53
51.50
1.97
51.08
1: 2.76
3.27 1: 0.15
56.30
1.26
56.79
1.88
0.01
Random 6636 PG
44.97
2.01
46.83
2.53
11.28
1.30
50.78
0.97
51.67
+ 2.31
1.24 + 0.02
53.73
1.78
55.50
0.84
0.68
0.07
Cla. PE 33%
45.62
2.32
53.07
1.36
12.70
0.60
10.03
09:41
0.59
0.07
644.22
01:4
0144
8.14
0.00
PE 68%
2:00
54.30
13.97
61.27
0.63
08.12
8.08
0.35
0.23
ClaP PE
16.80
L,96
52:72
1.05
20.0S
3.47 18.49
50.22
1:20
G3.91
1.21
12.82 1 0.75
63. 1: 0.28
07.68
0.85
11. 0G
1. 1.71
ClsSP PE
1.63
0.70
0.68
30

---

Published in Transactions on Machine Learning Research (01/2026)
Pairwise
BALD
17
3.3
1.7
18
22
27
1.7
PowerBALD
12
17
6.7
18
17
SoRtankSALD
8.3
10
8.3
10
15
Predictive
22
13
23
27
27
3.3
Algottin supperssms
PowerPE
10
17
15
12
3.3
Random
Random
Random
ClaSh
16
BALD
PowerBALD
SottankkBLD
Prodiativ Entropay
Pawerr PP
Random
Raraton Jatle PG
Randerr 48% FG
ClSSP PE
Algoritmm oupprrooreed
Figure 10: PPM aggregated over the Antivive main study experiments.
31

---

Published in Transactions on Machine Learning Research (01/2020)
Pairwite Penalty ACDC
Psirwise Marric AMOS
POAnrBALD
6.7
60
33
0
PowerBALD
B7
60
67
G.7
Prodict ve Entrpy
13
1
27
60
40
20
Predictive Entrppy
11
0
6.7
0
Argarimm SupHrerm
Random
0
0
6.7
Fandom
40
0
27
Random 65% rc
20
27
13
13
60
73
Bandom 60% PG
80
80
73
80
80
80
60
73
BAL
TowerBALD
Sotrensbobp
Prelictove Entrapy
Pewrery
Randler
Rendon Abhi
Cast
BAID
PowesDaDD
SetrassiAUD
Medirtbo Entrap?
Randon
Randor Kpk
Guo
Remiin
Agarishr sutperformed
Algor sdperfomedd
ACDC
(b) AMOS
Paitstoe Peralry Verris Cla Hilppcampic USA
Fairsdioe Peralty Martis (Mea KTS 26D
PoweRRALD.
6.7
0
13
PowerBAL
33
0
80
47
SofranHBALD-
Softrankee MBALD
67
40
ve Entropy
13
27
6.7
20
33
40
Prodictive Ertropy
20
47
53
47
30
Alpo uppafo
PowerPE
Power FFE
13
80
47
Random
0
0
Random
0
0
.00
Mig Biprthm
6.7
6.7
6.7
33
6.7
Random 68%
6.7
6.7
0
6.7
0
6.7
Sandom 60%
20
0
6.7
27
13
ClsSP PE
60
33
80
67
Mewn
Mean
BALD
Predittive Ensrage
Randees
Random 3335
Ranatim Pd
Clasp Pb
BALD
PowerBAb
Predictivr Ensrapo
Fowerng
Raanbaa
Renstun 33%
Cadd
Raniom
Aignnite suspertermed
Alpor thms oupprfomed by
(c) Hippocampus
(d) KiTS
Figure 11: Pairwise Penalty Matrix aggregated over all Label Regines for each dataset of the main study.
32

---

Published in Transactions on Machine Learning Research (01/2026)
E.3 Patchx Setting results
Analogous to appendix E.2 this section provides results for the Patchx settings, including detailed results table (table 7. and the overall (fig. 12 and datset-specifie (fig. 13 PPMs. The specific query patch sizes as well as further dataset characteristics are detailed in table 8
Table 7: Fino-grained Results for the patch ablation with sctting Patchx 1 for each dataset. Higher values 2 are better, and colorization goes from dark green (best) to white (worst) with linear interpolation. AUBC and Final Dice are multiplied x100 for improved readability. AUBC, Final, and FG-Eff can only be directly compared for each Label Regime each dataset. The respective dataset characteristics are detailed in table
(a) ACDC
Datases
ACDC Medium
Labal Regime
Low
High Firal
Metric
AUBC
Tiice
FG--RI
AUBC
Finsll Dies
FG-RI
Query Method
BALD
68
23
1
2:31
77.72
1.46
200.38
$00.80
25.80
1.10
$2.50
39
1:21
169 S1
438
79.50
1.05
S3. $3.99
1 0.85
75.
60:33
Powe
1.25
121
SoftrankBAt
66.81
3.68
75.08
0.13
245.20
76.84
1.31
16
0.47
78.00
84.00
181.08
Predictive Entropy
65 27
2:45
75.79
2:45
186.91
201 36
70.07
16
81
1.32
119. 52
162.65
79.25
0.95
83, 58
1.39
84.96 1 102,80
PowerP
3.900
74.45
492.56
1.50
75
200.80
Random
M550
13:09
VO.99
70.50
75009
Random 33% FC
6y
+ 1:51
75:43
0.27
215.114 + 046.27
70000
1.28
81 65
+ 1.00
127
# 65.30
29:06
1.12
841 =: 0.012
87 20.
+ 27.50
Random 66% FC
13
1:1
75.07
0.55
101 + 46.52
74.60
0.28
18
72.00 + 06
80.33
0.50
85
0.60
$6.00
8.42
4.45
78:43
75,90
70.82
136.97
(b) AMOS
Dataset Label Regime
AMOS Mediun
Low
High
AUBC
AUBC
FG-EE
AUBC
Query Method
RALD
13.98 + 1.24
10:96
+ 2.19
-1 14
374.12
16.03+ + 2.33
17:85 + 4.50
110.08
20.38
30.15 + 1.72
27.72
0.83
-19.23 + 3.35
11.74
2.50
-248.75
772.7
21.71
1.28
25
1.86
42.40
8.53
SoftsanRRALD
13.63 + 2.60
11.30
1.68
-127 75
330.40
19.05 + 1.55
23-48
5.19
-0.60
8.12
35.13 + 2.40
30.37
1.87
-3.05
5.38
Precictive Entropy
13.83
2.12
12:28
1.98
83.91
260.52
21.81
2.34
27:57
5.21
7,21
2.16
36.63
6.19
03.88
6.88
1.71
4.83
PowerPE
12.00
4.65
23 28
+
27.00
2.03
8.08
1.7%
47.34
18:72
3.42
Rankom
12.78
+ 2.00
8.80 + 1.91
-942.06 + 5820000
14.14 + 1.62
16.00
3.92
-89-28 - 166.24
37.56
1.57
$7.28
3.44
-3.38 + 23.06
Random 35% PG
20
24.10
1:37
11.00
105
2.08
1.21
103.81
21.46
56.
1.86
615.5
0.69
03.00
11.01
20.20
0.34
131.71
+:
00
0.45
02.07
0.77
710.71
8.53
ClaSP PE
20:32
2.58
25.83
2:45
6.94
1 59.73
31 $1.54
2.77
43.55
2.88
36
06.95.1
5.04
63.15
3.14
67. 03
1.35
322 86
3.25
(c) Hippocampus
Hipperammpss Medium Final Dieo
Label Reginee
Low Finel Dice
High Finol Dice
Metric
AUBC
EG-EIT
AUBC
G-
AUBC
FG-BIT
Query
BALD
N:.
0.47
87.85
0:15
72.18 + 173.33
87
0.11
88.43
0.19
15.02 2.49
97.90 0.16
88.76
0:88
12.46 1.33
Powe
97.84
$7.32
87
88.47
17.28
SoftrankBAALD
N4.44
0.33
0.32 87.06
73.07 + 163.84
87.14
+ 0.17
88.27
0.10
16.63 + 2.19
87.92
0.07
88.68 + 0.14
15.13
1.68
Predictive Eatropy
86
87.00
63.64 + 128.00
87.43
0.14
88.41
0.09
13.37
67.99
88.74
0.09
12:30
1.71
PowePE
20
87.56
0.51
51.35
14115
$7.43
0.11
58 29
0.11
19:81 2.65
$7.94
$8.43
0.15
18.11 2.41
Ramdon
85.
65
86:74
0.31
118:43
272.37
8F $7.06-10
1 0.21
67.76
0.10
25.64 2 4.49
58 0.15
88.13 0.15
26.07
3.15
Randoo 33% PG
85
87.02
78.70
124.77
87.26
0.17
17.19
87.
0.06
85:31
16:41
Random 60% FC
15
87.04
0:15
57.16
56.78
87.49
+: 0.21
58. 27
0.10
14.1 = 1.15
87 N5
2:3-20000 0.21
84.04
0:17
12.43 = 0.64
21
0.06
0.10
0.08
(d) KITS
Datasct Label Regine
KITS Medlum
Low
High
Dioe
FC-EI
Dioe
FC-EF
Query Mechod
BALDD
25.
76
38.56
PoweBALD
27.91
29.39
1.30
L84.2
578. 47
11.70
1.09
45.59
78.84
11.77
49.80
0.95
51.00
1.25
02.02
17.06
SoftoonkBBALD
25.67
2.68
31.47
1.54
70
00
41.08
1.00
46.14
1.77
45:46
16.84
10.08
1.12
54 30
1.53
31 +: 10:200
Predictive Entropy
24.08
29.07
14.50
PowerPE
06
9.63
30 38
4.84
207.47
651 68
12.26
0.77
46.55
0.95
$3.36
61.36
19:48
1.57
53.50
1.09
49.14
21.87
Random
22.00
1.62
22.85
2.15
1299.90
35.14
37.95
1.74
98.75
138.00
42.73
44.35
1.60
73.85
Random 33% PG
23.88
9:43
83
1.36
19.00 1.00011.330 $1.48
0.74
61.24
0.88
17.55
1.47
42.28
1.19
44.19
1.31
3.48
0.15
Razdoma 60% PG
24.43
2: 1.56
26. 30
2,00
20.62 4: 7.36
24.12 2b: 1.40
30.52
1.24
4.34 d 0.27
40.24
1.31
42.58
0.88
02.68
4: 0.04
ClaSP PE
08:74
31.13
19:77
5:400
33

---

Published in Transactions on Machine Learning Research (01/2026)
BALD
14
1.7
3.3
3.3
23
15
-70
PowerBALD
12
13
45
20
20
1.7
SoRtankBAAD
1.7
3.3
3.3
25
20
Predictive
3.3
3.3
17
20
Algonthn suppersees
PowerPE
13
8.3
23
17
1.7
-00
Random
6.7
Random
30
Random
ClaSh
BALD
PowerBALD
SotrankBBLD
Prodiativ Entropay
Pawerr PP
Random
Raraton J0tle PG
Randerr 46% FG
ClSSP Pe
Algoritim oulperforeed
Figure 12: PPM aggregated over the Patchx settings. Mean row results change compared to the nn nntivve main study (fig. 10
34

---

Published in Transactions on Machine Learning Research (01/2020)
Paitwise Penalty Hartic (NJ
Palrwise Hatric pharchecuz
POARrBALD
6.7
6.7
27
60
6.7
13
6.7
PowerDALD
27
20
6.7
20
13
Prodict ve Entrpyy
6.7
40
1.5
Predictive Entrppy
20
0
333
1.
6.7
33
50
Random
0
Fandom
27
0
0
6.7
Random 65% TO
6.7
6.7
6.7
5:3
6.7
6.7
Bandom 60% PG
80
80
80
80
80
78
8D
BALD
PowerBALD
Sotrensbobp
Prelictove Entrapy
Pewrery
Randler
Rendon Abhi
Rardion FG
BAUD
PowedDaLD
SetrassiAULD
Medietbo Entrapt
Rundoon
Randor KpN
Guo
Remiiam
Agorishr sutporformed
Algor ddpertfmmed
(a) ACDC
(b) AMOS
Painstee Penally Mentis Darrtst Hppersmpii (S3)
Painstte Persity Natri (turchal KITS (S6
27
PoweRRALD
20
0
PowerBAL
33
20
80
60
67
SofranHBALD-
53
ve Entropy
20
13
33
13
Prodictive Ertropy
6.7
0
40
27
47
Alpo uppafo
PowerPE
Migetth tprthm supittmm
Power FF
80
60
Random
0
6.7
0
Random
0
6.7
20
400
20
20
Random 68%
6.7
40
20
Random 60% PG
0
0
6.7
6.7
67
ClsSP PE
33
80
87
Mewn
Mean
BALD
Preditiive Ensrage
Pomarig
Randeen
Randoon 3335
Ranatim Pd
Clasp Pb
BALD
PowerBALD
Predictivr Essropo
Fowerng
Raanbar
Renstun 33%
Cadd
Ranioo
Aignntte suspertormed
Alpor thm oupprfomed by
(c) Hippocampus
(d) KiTS
Figure 13: Pairwise Penalty Matrix aggregated over all Label Regimes for each dataset for the Patchx ablation.
35

---

Published in Transactions on Machine Learning Research (01/2026)
E.4 500 Epochs Setting results
For the ablation on the loss scenarios on AMOS in section 4.1 provide detailed results in table
Table 8: Fine-grained Results for the AMOS experiments when training for 500 epochs. Higher values are better, and colorization goes from dark green (best) to white (worst) with linear interpolation. AUBC and Final Dice multiplied x 1000 for improved readability. AUBC, Final, and FG-Eff only be directly compared for each Label Regime each dataset.
Datnoct
AMOS Medium Final Dice
Label Regime
Low Final Dice
High Final Dice
Metric
AUBC
FG-EIt
AUBC
FG-EI EI
AUBC
PG EI
Query Method
BALD
76.76
1:20
81.67
2.40
7.90 + 0.54
84.37 + 0.10
17.18
0
7:71 0.12
0.33
56.35
SoltrokBALD
70.04
0.20
83.05
0.08
14.59 0.33
84.80 +: 0.26
86.83 0.16
9.27 0.11
Entropy
0.53
7.09
PowePE
0.39 79.27
0.21 $3,05
22 N0 0.00
84.32 +: 0.12
56.23 0.19
11.50 4: 0.27
Random 35% PG
65.08 4: 1.59
71:22 +: 2.39
64.00 23 13
70.41
0:48
83,00
0.35
33.90
3.19
84 16 0.14
86.04 +: 0.13
14:43 1: 0:49
Randam 66% PC
2,02
Cla PE 60%
65.18 2: 0.80
71.87 4 0.86
24.29
2.84
80.37
0.56
85.23
0.30
14:08 0.21
85 35
-: 0.02
87.66
0.11
8.00 + 0.07
ClaSP PE
73.44
1.30
36.022
6.94
3,092
87.63
1.3.73
36

---

Published in Transactions on Machine Learning Research (01/2026)
E.5 Analyzing ClaSP PE performance AMOS class level
We analyze the performance of ClaSP PE relative to Random 66%FG on the Final Dice with regard to each class and the percentage of voxels queried on the AMOS dataset for 200 and 500 epochs on the main setting across all label regimes in fig. 14 We observe that the longer training Icads to ClaSP PE gaining more performance than Random 66% FG, but also leads to a general increase in queried foreground. Overall, the esophagus, postcava, pancreas, right adrenal gland, left. adrenal gland and duodenum (classes 5, 9, 10, 11, 12 and 13) are the most challenging classes. Importantly, the performance differences for right and left adrenal gland reduce with larger annotation budgets and longer training. While the largest performance gains stem from the bladder and prostate (classes 14 and 15).
In the low-label regime, especially the classes 11 and 12 (right and left adrenal gland), lead to relative performance losses ClaSP PE relative to Random 66% FG which are also less frequently queried. In the medium and low-label regime with more queries for these classes this performance difference diminishes for 200 epochs and vanishes for 500 epochs below random noise.
37

---

Published in Transactions on Machine Learning Research (01/2026)
GUaP
(a) Lowelabel 200 Epochs
(b) Low-Label 500 Epochs
0.20
(c) Medium-Labbe 200 Epochs
(d) Medium-Lane Label 500 Epochs
Nitrrase twar Putt Reshor oudter Seager
Nilooman bos Cur Kuster Rec.n outterc
0.24
(e) High-Label- 200 Epochs
High-Label 500 Epochs
Figure 14: Visualization of the difference of the percentage of voxels for all classes alongside Final Dice
performance AMOS in the Main setting from ClaSP PE to Random 66% FG trained with 200 & 500 epochs. Error bars denote the Standard Deviation.
38

---

Published in Transactions on Machine Learning Research (01/2026)
(a) Low-Label-- 200 Epochs
(b) Low-Label 500 Epochs
12
(c) Medium-Label 200 Epochs
(d) Medium- Label 500 Epochs
Dldt me audare
(e) High-Label- 200 Epochs
High-Label 500 Epochs
Figure 15: Visualization of the percentage of voxels for all classes alongside Final Dice performance on AMOS in the Main setting from ClaSP PE trained with 200 & 500 epochs. Error bars denote the Standard Deviation.
39

---

Published in Transactions on Machine Learning Research (01/2026)
E.6 Comparing Pairwise Penalty Matrix with different p-values
Here we will compare the results of the pairwise penalty matrix aggregated over the main and patch & experiments when using p-values set to 0.05, 0.02 and corrected with the Holm-Bonferroni method (Holm. 1979h to reduce the probability of false rejections due to multiple tests for a p-value of 0.05.
The correction performed for each singular experiment (one dataset and one label-regime) with the value m used divide the p-value:
#QMs (#QMs -1
1) #Loops 180,
when using #QMs and #Loops = 5 based our experiment setup. This computation of the m-value factors in each pairwise comparison.
In common practice, the pairwise penalty matrix is used without any correct ions for multiple tests and with fixed p-value of 0.05 (Ash et al.
2020 Föllmer et al. 2024 Lüth et al. 2023 2025 Beck et al. 2021
The results are shown in fig. 16 Overall, the trend for the mean row (lower is better) is similar for p-values 0.05 and 0.02, where ClaSP PE is the best performing method, followed by group of QMs (Predictive Entropy, SoftrankBALD, PowerPE, and PowerBALD), Random 66% and 33% FG, with Random the least performant method.
Meanwhile, for the Holm-Bonferroni method, ClaSP PE remains the best performing method, but now Predictive Entropy is the only other AL method that outperforms the random basclincs.
40

---

Published in Transactions on Machine Learning Research (01/2026)
Faleite Eimally Marra MMee
Pareie Peadly Mara (Man Peckeg IRC
15
GT
17
3
6.T
5-8
47
20
75
17
23
17
35
24
23
22
33
38
F3
12
12
54
17
75
13
Tendon
0.2
3
1.7
4.2
9.0
133
0.8
Terdor
5.8
88
3.5
2-7
Nance
26
kandonn 66%
22
20
20
63
25
OXPYE
20
22
23
22
DAU
Eaenge
Westom Alvve
Aaedem ra
MeD
Brnnche Eabead
Asstim rieer
p-value=0.05
(b) p p-value=0.02
Pimmaty Matro Maan Fatewai De1
1-2
0.4
1/3
Paneft
US
0.3
Randen
0.6
0.5
CLPP
1.
Mer
3.3
2.5
-onferonii
Figure 16: As expected, for smaller p-values, the mumber of significant results decreases, especially for the Holm-Bonferroni method due to the number of comparisons (180). When discussing the overall trend based on the Mean row (lower is better), we observe that ClaSP PE exhibits the lowest fraction of scenarios where it is outperformed by another method. This trend is similar for p-values 0.05 and 0.02, where ClaSP PE is the best performing method, followed by a group QMs (Predictive Entropy, SoftrankBALD. PowerPE, and PowerBALD), Random 66% and 33% FG, with Random as the least performant method. Mcanwhile, for method, ClaSP PE remains the best performing method, but now Predictive Entropy is the only other AL method that outperforms the random baselines.
the
41

---

Published in Transactions on Machine Learning Research (01/2026)
E.7 Mean Performance estimate
The computation for the values in table 1 are obtained for the mean
S
where is the mean performance for each setting. The standard error is obtained using the gaussian error propagation for the mean errors:
ATT
Ap.)
$
==1
where Apt,- VN xx N represents the number of sceds per experiment and is the standard deviation for single experiment.
We 95% confidence interval 1.96 Ap.
42

---

Published in Transactions on Machine Learning Research (01/2026)
F Guidelines for Real-World Deployment of ClaSP PE
In the following, we provide details on the systematic selection of query patch size and query budget param- eters for applying ClaSP PE to unseen datasets. The parameters that we obtain for the roll-out datasets are provided in table 4 Despite our extensive validation, demonstrating the generalization capabilities, we can not guarantee good model performance beyond the tested settings, especially for lower fractions of annotated data.
Query Size Selection We recommend normalize the query size based on the number of foreground classes in the dataset to optimally leverage the performance gains through class-stratified sampling. In our roll-out experiments, we calculate the total query budget (per AL cycle) by counting contribution of 50 100 patches per class, depending on the task complexity of segmenting certain target structures. Specifically, we use a query budget contribution of 100 for classes where we expect higher variance, such as the tumor class compared to the liver class in LiTS.
Number of AL Loops In our experiments we performed five AL loops. Since AL performance typically improves or remains stable relative to random strategies in later stages or with larger annotation budgets, extending the number of AL loops is generally safe and may further improve performance. We hypothesize that leaving the value for / (inverse power noising strength) after the 5th loop at 100 should be sufficient for ClaSP PE, as the segmentation model should by then be able to effectively exploit its understanding of the task. Crucially, we do not advise to reduce the number of AL loops.
Starting Budget Selection The starting budgets should selected with a mix completely random patches and random class balanced selection of patches. We used factor of 33% of patches which surely feature foreground. In practice, the patches which surcly fcature foreground can be simply obtained by going into random images and just selecting some patches featuring larget structures of interest while counting the amount of patches to ensure that they somewhat class balanced.
Query Patch Size Selection For the size of the query patches, recommend choosing the median size of the target structures, in order to obtain representative samples. This is realizable in practice, as an ostimation of the object sizes can typically be donc efficiently, without necessitating the availablity of finc- grained annotations. In the benchmarking scenario, we proceed as follows: First, we compute the median bounding box size per class based on the largest. connected component per image. Then, we take the overall median bounding box size across classes. An exception is the LITS dataset, where we only consider the tumor class (omitting the liver class), the liver is significantly larger while the tumor structures are of particular interest.
43

---

Published in Transactions on Machine Learning Research (01/2026)
G Roll-Out Results
Detailed results for the roll-out study in section 5 are provided in table 9 The corresponding PPM is shown in fig. 6
Table 9: Fine-grained Results for the Roll-Out Scenario. Higher values are better, and colorization goes from dark green (best) to white (worst) with linear interpolation. AUBC and Final Dice are reported with a factor (x (x100) for improved readability. AUBC, Final, and FG-Fff can only be directly compared for each Label Regime on each dataset. The respective dataset characteristics are detailed in table
LTS Rall-aat Fmal Dion
Tooth
MAXA MEA
Loled Bogine
Rall-aat Final Dioe
Rodl Frall Dicn
RailOot Final Dice
Motre
AUBC
FG-E8
AUBC
FG-EB
AUBC
FG-EN
AUBC
PG B2
Query Morlad
Rarbon
51.58 + 1:21 0.13 1:23
62,85 2:91
17.85 J04
78,08
8888
61 58 + 0:25
64.22 = 0.58
11.88 0.15
Bandoon
90m 1.3
1:2 635
18.1 0031
28255 1115
14.38 3,88
0M G1M
45.0
Estroge
0.0
ClsSP
78.55
H Limitations
Benchmark overfitting. ClaSP PE was developed on the nnActive benchmark and therefore carries the risk of over-optimization. However, the general strong performance on the Roll-Out Study against the Predictive Entropy and Random FG 66% FG, which were the other best performing methods on the nnActive benchmark, shows its generalization capabilitics to novel scenarios. Wc also wish to highlight that virtually all AL methods face the danger of being overdesigned for specific benchmark
they necessitate design decisions that need to be evaluated empirically (Shi et al. 2024a Föllmer et al. 2024 Gaillochet et al. 2023b; Vepa et al. 2024). The combined evaluation on the nnActive Benchmark and Roll-Out study, which to our knowledge is the most comprehensive to date, mitigates the risk of benchmark-specifee overfitting relative to earlier approaches (Lüth et al. 2025). Further, the performance of ClaSP PF on the nnActive benchmark suggests generalization capabilities beyond our conservative Guidelines for World Deployment. We encourage future benchmarking efforts of AL methods for 3D biomedical segmentation, demonstrating their generalization capabilities on novel datasets separate from method development, thereby redueing potential conflicts of interest.
Our empirical findings are specific to the nnActive framework using mnU-Net as the segmentation network and 3D patches as queries. Results may differ with alternative architectures, regularization techniques, self-supervised or semi-supervised learning approaches, or different query designs (2D slices or whole 3D images).
Dependency on model predictive capacity. ClaSP PE relies on the model producing sufficiently accurate multi-class segmentations, since these predictions underpin the stratified query selection. In the low-label regime of the AMOS dataset (see section 4.1 Query Design), we observed that limited initial labels can result in inadequate segmentation quality, reducing the offectiveness of stratification. Our final recommended guidelines for using ClaSP PE mitigate this risk by using a query size based on the number of classes that is most likely to lead to sufficient initial segmentation quality, exemplified by the results in the Roll-Out study. Moreover, the use of pre-trained foundation models may further improve early-stage segmentation quality (Gupte et al. 2024).
Economic trade-offs of AL. We wish to emphasize that AL inherently represents a wager with the aim of reducing the overall cost of building an ML pipeline where compute cost is better against expected reduction in annotation effort (Settles 2011p. The decision whether to employ AL or not is an economic question dependent on multiple factors, such as annotation cost and computational resources. In this work, we demonstrate that ClaSP PE within the nnActive Framework shows strong evidence to reduce the anno- tation cost when employing AL in wide variety of settings. However, the cost of employing AL needs be evaluated in comparison to the expected gains, which is outside the scope of this work and represents a fruitful direction for future research.
44

---

Published in Transactions on Machine Learning Research (01/2026)
Comparison to more complex AL baselines. While more sophisticated AL strategies (s.a. Hübotter et al. 2024); Föllmer et al. (2024) could in principle yield similar gains, there is currently little evidence that they can be made practical in our setting. In particular, extending such methods to 3D biomedical image segmentation remains an unsolved challenge: qucrying 3D patches instead of 2D slices while intograting the full complexity of stale-of-Che-art segmentation pipelines poses substantial computational and algorithric hurdles (see section 2 Our focus here was therefore on designing an approach that is robust and easily deployable across new datasets, leaving the open problem of adapting more advanced AL techniques to the 3D domain for future work.
Hyperparameters of ClaSP PE. We addressed the need for class-balanced dataset and hard-to- predict cases, and early-stage diversification and later exploitation through careful balancing of the stratified sampling ratio a and a pre-defined scheduling for power-noising of B. We empirically rigorously validate these modifications to ensure overall benefits on wide variety of datasets based on a large-scale benchmark for development and an evaluation on held-out roll-out datasets. However, our results also show that this exact setup not always the optimal solution, and it might be even more favorable have heuristics adapt both and the schoduling of based dataset characteristies and other confounding information, such model performance.
Metric Dependence and Generalization Our evaluation metrics are based on the mean Dice measure of segmentation performance, which assigns equal importance to all classes. We do not demoonstraee that our method generalizes to scenarios where evaluation metrics weigh classes differently. Although mean Dice widely used in medical image segmentation, there are tasks where not all classes are equally impor- tant. For example, class importance may be proportional the frequency (micro-averaged performance) or dictated by clinical relevance (weighted macro-average). While the class-balanced sampling in ClaSP PF is naturally aligned with mean Dice, alternative sampling weights may be desirable when evaluation metrics assign unequal importance to classes. Finally, we emphasize that class balance sampling serves broader purpose maintaining semantic diversity across AL cycles, which remains beneficial independent of the chosen metric.
Qualitative Results
In this section, we provide additional qualitative results to demonstrate the effects of the class stratification used in ClaSP PE, as compared to standard Predictive Entropy.
1.1 Query Visualization
In figs. 17 to 20
we provide exemplary visualizations of the queried patches of PE and ClaSP PE after the first AL loop on all nnActive benchmark datasets on the low-label regime using the main settings. For ACDC (fig. 17), the stratification of ClaSP PE leads to a more diverse query selection and more foroground being queried. Further, ClaSP PE mitigates the risk of an overly focus on prominent classes, such as the posterior hippocampus (fig. 18), the tumor class KITS (fig. 19), the liver class AMOS (fig. 20
45

---

Published in Transactions on Machine Learning Research (01/2026)
(a) PE
(b) ClaSP PE
Querry
cevtt
myocardium
right
Figure 17: Exemplary visualization of the queried patches using PE (a) and ClaSP PE (b) after the first AL loop the ACDC dataset (same seed to ensure comparability). For 2D visualization, selected the center slice of the 3D patches. Best viewed zoomed in.
46

---

Published in Transactions on Machine Learning Research (01/2026)
(a) PE
(b) ClaSP PE
Query
postarior
artarior
Figure 18: Exemplary visualization the queried patches using PE (a) and ClaSP PE (b) after the first AL loop on the Hippocampus dataset (same seed ensure comparability). For 2D visualization, we selected the center slice of the 3D patches. Best viewed zoomed in.
47

---

Published in Transactions on Machine Learning Research (01/2026)
(a) PE
(b) ClaSP PE
Quory
cyst
tumer
Money
Figure 19: Exemplary visualization of the queried patches using PE (a) and ClaSP PE (b) after the first. AL loop on the KITS dataset (same seed to ensure comparability). For 2D visualization, we selected the center slice of the 3D patches. Best viewed zoomed in.
48

---

Published in Transactions on Machine Learning Research (01/2026)
Wcthloghh
lethhoogh
(a) PE
(b) ClaSP PE
Quary
right odney
exsphagua
eot glard
act glond
Boodemue
aradee
protate
Figure 20: Exemplary visualization of the queried patches using PE (a) and ClaSP PE (b) after the first AL loop the AMOS dataset (same seed to ensure comparability). For 2D visualization, selected the center slice of the 3D patches. Best viewed zoomed in.
1.2 Stratification Visualization
Figure 21 illustrates the class-wise stratification defined eq. 33 based on predictive entropy. For this example, we use models from the first loop the Low-label regime in the main nnActive benchmark setting,
The figure shows that this stratification shifts the regions of high uncertainty for each class toward the areas where the model's predictions indicate these classes present..
49

---

Published in Transactions on Machine Learning Research (01/2026)
GT
H[x]
H[x x (1())
H[] x (2(x)
GT
H[1
H[x] p1(x)
H[x] Px()
Prediction
Ps: kidney
Pvi tumor
Prediction
Ps: Anterior
Pai Posterior
(a)
KiTS (case 67)-
(b) Hippocampus (case 8).
GT
H[x]
H[x] x px(x)
H[x] x p2(x)
H[x] x p3(x)
Prediction
Pr: LVC
(c) ACDC (patient 3, frame 1).
GT
H[x]
H[x] x pz(x)
H[x] pa(x)
H[x] x pa(x)
Hix] xxr(x)
H[x] pa(x)
H[x] palx)
H[x] p1a((
Prediction
Psi
(d) AMOS (case 4).
Figure 21: Visualizations of the stratification mechanism of ClaSP PE for exemplary cases of each nnActive benchmark dataset. For each predicted class in the displayed slice, the class probabilities pc(x) as well as the weighted entropy maps [x] (x) are shown, which lay the basis of the stratified querying. The colormaps rescaled for each individual image. To avoid outliers distorting the color mapping, we clip high values at the 98% quantile of the data.
50


Reviews:
[{"review_id": "csJA8hVGUw", "reviewer": "Reviewer_5sG6", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a new method for AL for 3D image segmentation for biomedical applications. Their method is based on a simple modification (which is not a flaw) of predictive-entropy sampling. The main motivation is to address two modes of failure of the  uncertainty-based AL in this setting.\n1. Imbalance between background and foreground classes and between foreground classes\n2. Redundant samples early in the AL cycles that are concentrated close to each other spatially \nAuthors have used nnunet as a backbone, with the nnactive benchmark. Four seeds per configuration provides some validity to the work. \n\nAlso the main claim here is that there method provides the first compelling evidence of an AL method that beats strong baselines in terms of segmentation quality.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Not fully, but mostly yes, based on the results they presented for the central empirical claim. Also partially, for the bigger general claim.\n\nThe main claims/hypotheses they present are:\n1. Class‑stratified uncertainty and their scheduling approach to power‑noising can fix the two issues as I wrote above.\n2. These improvements will generalize to new datasets when using their approach for patch size/ budgets/schedule.\n3. At least for the biomedical segmentation tasks there method provides the strong evidence of consistently beating a strong random baseline.\n\nFor 1) the nnactive benchmark is an acceptable choice and the range of ROIs is good (ACDC, AMOS, KiTS, Hippocampus), each with three labels  and 2 query patch sizes. I do wonder how such an algorithm will do with scenarios where you have plenty of very small objects (e.g., lung nodules <5mm). The variants of BALD and random baselines are also acceptable given their focus is on uncertainty-based sampling methods. A lot of tricks used are good practices in the current 3d seg AL literature (patch-based query, partial annotations,  foreground-aware random baselines).\nFor 2, the generalization study they have used four more datasets with different anatomy and characteristics, and they have configured AL in a somewhat principled manner (e.g., median target size for patch size and budgets proportional to number of classes etc) and achieved without tuning for individual new datasets. This is good practice for the generalization experiment.\nFor 3, conceptually the main experiments are okay, where we have a nnactive evaluation and a generalization experiment.\n\nSome of my reservations and subdued enthusiasm is due to the fact that by author's own admission in the appendix, the hyperparameter tuning wasn't done rigorously, and it is unclear if random baselines had a fair \"compute budget\" to be competitive with random baselines. In my own work this is the main problem I have seen before.\n\nIn terms of the actual results themselves, fig2 and fig3 support their claim of Clasp PE's superiority compared to random, BALD variants and other PE methods, although it is worth noting that the gains are typically modest (~1 to 2%). On the positive side, these appear to be consistent in direction. With the nnunet backbone which is strong and small budgets, from my experience in non-3d and non-biomedical datasets, I have seen that a few percentage points of improvement can easily lie within the error interval around point estimates, but I have no evidence to say this is what's happening here.\n\nOne related interested experiment that the authors do talk about is amos experiment with longer training (200 vs 500 epochs), where the PPM on AMOS shows that clasp-pe’s win/loss ratio vs rrandom 66% fg improves as the models are left to train for longer duration. They note that the \"loose\" cases are mostly in the low‑label regimes. This supports there argument that some of the residual random‑baseline wins are due to under‑trained models rather than inherent AL limitations, but on the other hand, I would have liked to see a greater emphasis on hyper-parameter tuning at the expense of number of epochs.\n\nThe gains are relatively bigger for their generalization study (eg fig6). I liked that they presented these results, even though I think that they could have tried with datasets like LIDC IDRI or similar where you can end up with many small ROIs to segment rather than 1 or few large in a given image.\n\nSomewhat problematic for me are claims like “only method with statistically supported improvements”. This is a statement about average ranks over 24 settings with a fairly loose threshold of statistical significance (p=0.1).\n\nThe PPM does show that Random 66% FG still wins in about 20% of settings and the paper acknowledges this but then moves back to making the bigger claims (“generally outperforms random baselines”).\n\nFrom a statistical viewpoint what the results show is that within nnActive + nnU‑Net + patch‑based AL setup, ClaSP PE is a strong approach, which does appear to be consistently competitive method, with good generalization experimental results to support it.\n\nThis part I do believe.\n\nThe jump from here to the “first compelling evidence” of general AL superiority in 3D segmentation is an over-reach and should be toned down. I do not blame the authors for using such language which unfortunately I see increasingly as authors struggle to be taken seriously in a review process - I have been there, but let's restore some faith in the review system :)\n\nPaper would also benefit from tighter presentation of language used to describe statistics but I am moving that to the requested changes section.", "audience_yn": "Yes", "audience_explanation": "Yes, work is relevant.", "requested_changes": "Some main points are below\n\n1. Tone down statements like “first compelling evidence” and “first plug‑and‑play AL solution”. Be specific and direct, e.g., “within nnActive with nnU‑Net” or “within patch‑based AL for 3D segmentation”.\n2. Be clear in the abstract and conclusion that the evidence is strong for this specific framework and set of datasets. But to be fair and good shepherds of science, let's also be clear that extension to other architectures / training pipelines is a plausible but untested hypothesis.\n3. Explicitly describe how many configurations enter each Friedman/Nemenyi test (24) and what a difference in average rank of, say, 0.5 means in terms of effect size.\n4. Explain more clearly why a loose threshold of p=0.1 was chosen. On a related point, please do frame the resulting groups as exploratory evidence rather than definitive proof.\n5. For PPM, make it explicit that multiple pairwise t‑tests are used without FWER adjustment and that PPM is intended as a descriptive summary. Please do add a short discussion of robustness (e.g., whether main qualitative conclusions persist under a stronger threshold or a non‑parametric test).\n6. Figure 2 / 3 and Table 5 can include summary statistics that show the mean difference in Final Dice and AUBC between ClaSP PE and the best improved random baseline, with a 95% CI.\n7. In section 6 / appendix H please state that the evidence applies to the nnU‑Net as the backbone, patch‑based querying, and the specific foreground‑aware random baselines; and that it does not directly address other architectures, slice‑based training, or non‑nnActive pipelines. Other studies (including some cited in this manuscript) have shown that changes in architecture can render such results non-applicable. Please be direct in relating this to previous findings that AL conclusions can change with architecture and regularization, to help readers understand the significance of the results within the right context.", "broader_impact_concerns": "Nothing in particular to note.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "lKQPkgSB95", "reviewer": "Reviewer_Cfj7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents a simple active learning method for the annotation-efficient segmentation of 3D medical images. The method, called Class-stratified Scheduled Power Predictive Entropy (ClaSP PE), combines two ideas: 1) stratified class sampling based on prediction pseudo-labels, a power-noising of scores based on (Kirsch et al., 2023) which induces a more uniform random sampling of patches in the early stages of active learning (AL). A comprehensive set of experiments is used to evaluate the proposed method, including a various datasets and baselines. Results show the method's competitive advantage over other AL approaches.", "claims_and_evidence_yn": "Yes", "claims_explanation": "* Authors acknowledge that their work does not focus methodological novelty but instead on providing\na simple solution built on existing strategies. I agree with this assessment: the power noising approach is adopted from Kirsch et al. (2023), and the class-wise stratified sampling method, while interesting, is a relatively straightforward idea. \n\n* I also concur with authors that random selection remains a hard to beat baseline, in particular when at early stages of AL training, and that having a stratified class sampling can avoid over-selecting larger or less certain classes.\n\n* Regarding the results, although the proposed method performs well in ranking-based analyses, the actual gains in AUBC and final Dice are less evident. While the authors acknowledge this limitation, it raises questions about the method’s practical usefulness.", "audience_yn": "Yes", "audience_explanation": "* Active learning is a crucial challenge in medical imaging, where obtaining annotations is and often requires expert knowledge. The proposed method is simple and effective, and could easily be implemented in current pipelines.\n\n* Beyond introducing the proposed method, the paper provides a comprehensive set of experiments that can serve as a valuable reference for future research in the field.", "requested_changes": "* The claim that the proposed method is the first to outperform (improved) random baselines for 3D image segmentation may be overstated. Previous approaches, in particular (Gaillochet et al., 2023), also achieved similar results. While these approaches performed segmentation in 2D, they could be easily adapted to the 3D patch setting of the paper. Adding this approach as baseline would strengthen the claim.\n\n* Some statistical analyses are conducted at a low level of significance (p<0.1). I suggest the authors to try a more common p-value threshold such as p<0.05.", "broader_impact_concerns": "N/A", "additional_comments": "* The paper uses 3D patches as samples to annotate. While this choice makes sense for 3D segmentation, I am not sure it is optimal for the expert performing the annotation as labelling a 3D patch is harder than labelling a 2D image. It would interesting to discuss this point in the paper.\n\n* Figure 4 is a bit hard to follow, in particular, how the label regimes are considered in the analysis. \n\n* p. 10: \"while while winning\"", "extra_scores": {}, "extra_text": {}}, {"review_id": "kjXNWjZFni", "reviewer": "Reviewer_6yXK", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a patch-based active learning method for 3D medical segmentation which leverages uncertainty-based scoring, class stratified sampling, and an exponential scheduler with robust improvement over random baselines for several different evaluation settings.", "claims_and_evidence_yn": "No", "claims_explanation": "The first main contribution of the paper is that \"we propose ClaSP PE, a simple and effective query method that systematically addresses key limitations of current uncertainty-based AL methods.\"\n\nIn general, the method seems straightforward and reasonable. The paper argues that class-stratified sampling has not been done before and addresses key limitations of entropy-based methods. However, class-stratified sampling has not been clearly defined—how is the sampling done precisely? Would the goal be that the sum of the probabilities of the different classes is roughly equal after obtaining all patches? If the goal is class balancing, I would argue that this approach is beneficial because the metric used is the mean DICE (the average DICE score across all classes). The method may not generalize to other evaluation metrics. This should be clearly emphasized in the paper. While taking the average performance over all classes is common, there are situations in which this may not be the ideal solution.\n\nAdditionally, while the paper argues that their method is hyperparameter-free and robust to overfitting, it provides few details about the hyperparameters used for the exponential scheduler. Please provide more information about this and indicate whether any of it was changed across different evaluation settings. \n\nThe second main claim of the paper is that \"we conduct a large-scale evaluation, demonstrating that ClaSP PE is the first AL method to bring reliable performance improvements over standard and improved random sampling baselines for 3D biomedical image segmentation in a close-to-production environment.\"\n\nOverall, I appreciate the choice of metrics: final DICE score, AUBC, FG-Eff, and PPM all seem reasonable choices that generally reflect the main goals of active learning. Additionally, I generally appreciate the choice of entropy methods. While the inclusion of diversity-based methods would be nice, as you mentioned, applying them with pairwise comparisons on the patches is computationally intensive. \n\nWhile the method generally improves on the baselines, I believe the relatively lower performance on FG-efficiency warrants further discussion. You state, \"it is also among the best-performing methods measured by FG-Eff,\" which dismisses the issue instead.\n\nThe discussion on the failure case on the AMOS dataset was appreciated. The discussion that more complex cases require longer training is appreciated for real-world application. Additionally, the ablation was good, but ideally, there could be some ablation results related to the scheduler.\n\nThe rollout performance is good. The colors in Table 1 are hard to read - it would be better if the differences were more clearly visible. It should also be easier to compare performance between the first and second settings to see how much the performance suffered in the real-world setting, if at all (to me, the rollout performance may have been comparatively worse, but it's hard to tell).\n\nI would be wary of the tone of statements like this: \"Importantly, the partially competitive performance of PE does not undermine our conclusions\" because it is suggestive of bias, as it assumes there is a competition between your method and this one, in which case, you would obviously be biased towards your method. You then make a statement \"the results on the nnActive benchmark (fig. 2)...\"; however, as mentioned, the hyperparameters for your particular method may be fine-tuned to that evaluation task. In general, this paragraph could be written in a more even-handed manner. The following paragraph as well could be written in a more even-handed manner and use less bias (\"also outperforming Predictive Entropy significantly in 25% of all cases while only being outperformed in 5%\").\n\nThe third main contribution of the paper is that \"we provide practical Guidelines for Real-World Deployment to configure AL pipelines on new datasets, facilitating straightforward adoption of ClaSP PE.\" However, this is not discussed in detail in the main paper and would arguably not be a main contribution of this work.", "audience_yn": "Yes", "audience_explanation": "Yes, the robust evaluation of this method, as well as others, would be very interesting to the medical imaging community.", "requested_changes": "Please address the concerns above. Additional context, as discussed above, would be helpful to the research community.  Additionally, while the paper discussed slice-based and patch-based active learning, there are other options, including scribbles, points, superpixels, text, etc. Several of these annotation methods are less expensive than full annotations and may be less expensive than patches. Also, the annotation complexity can vary across patches. While a comprehensive experimental comparison is obviously not feasible, a good discussion on the pros/cons of the different choices would be beneficial to researchers and help others contextualize the contributions of your work.\n\nSome minor comments. Please consider reorganizing the method section. Ideally, provide the whole method overview first, and then give each of the sub-components, rather than each of the components, and then the full method. It wasn't very clear on the first read.\n\nAnother minor comment: on page 8, you forgot to include a percent sign with the alpha value", "broader_impact_concerns": "N/A", "additional_comments": "N/A", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)