Diffusion-Driven Two-Stage Active Learning for Low-Budget Semantic Segmentation
Authors: Jeongin Kim
Wonho Bae
YouLee Han
Giyeong Oh
Youngjae Yu
Danica J. Sutherland
Junhyug Noh
Venue: NeurIPS 2025 poster ()
Keywords: Low-Budget Active Learning
Semantic Segmentation
Diffusion Models
Primary area: general_machine_learning
TL;DR: A two‐stage active learning pipeline that uses diffusion‐based feature sampling and entropy‐augmented disagreement to pick the most informative pixels under extreme labeling constraints.

Abstract: Semantic segmentation demands dense pixel-level annotations, which can be prohibitively expensive -- especially under extremely constrained labeling budgets. In this paper, we address the problem of low-budget active learning for semantic segmentation by proposing a novel two-stage selection pipeline. Our approach leverages a pre-trained diffusion model to extract rich multi-scale features that capture both global structure and fine details. In the first stage, we perform a hierarchical, representation-based candidate selection by first choosing a small subset of representative pixels per image using MaxHerding, and then refining these into a diverse global pool. In the second stage, we compute an entropy‐augmented disagreement score (eDALD) over noisy multi‐scale diffusion features to capture both epistemic uncertainty and prediction confidence, selecting the most informative pixels for annotation. This decoupling of diversity and uncertainty lets us achieve high segmentation accuracy with only a tiny fraction of labeled pixels. Extensive experiments on four benchmarks (CamVid, ADE-Bed, Cityscapes, and Pascal-Context) demonstrate that our method significantly outperforms existing baselines under extreme pixel‐budget regimes. Our code is available at https://github.com/jn-kim/two-stage-edald.

Diffusion-Driven Two-Stage Active Learning for Low-Budget Semantic Segmentation
Jeongin Kim
Wonho Bae²
YouLee Han' Danica J. Sutherland
Giyeong Oh
Youngjac Yu?
Junhyug Noh
Ewha Womans University UUniversity of British Columbia Yonsei University "Scoul National University Amii
(jn.kim, oneglass, Oewha. ac. {whbae, dsuth]@cs. wbc.
hard2251 @02221i..neei.. kc. youngjaeyußsnu. act.snnuuaa.......... ac.kr
Abstract
Semantic segmentation demands dense pixel-level annotations, which can be pro- hibitively expensive especially under extremely constrained labeling budgets. In this paper, we address the problem of low-budget active learning for semantic segmentation by proposing a novel two-stage selection pipeline. Our approach leverages a pre-trained diffusion model extract rich multi-scale features that capture both global structure and fine details. In the first stage, we perform hierar- chical, representation-basee candidate selection by first choosing a small subset of representative pixels per image using MaxHerding, and then refining these into a diverse global pool. the second stage, we compute an entropy- -aggmeed disagreement score (eDALD) over noisy multi-scale diffusion features to capture both epistemic uncertainty and prediction confidence, selecting the most infor- mative pixels for annotation. This decoupling of diversity and uncertainty lets us achieve high segmentation accuracy with only a tiny fraction of labeled pix- els. Extensive experiments on four benchmarks (CamVid, ADE-Bed, Cityscapes, and Pascal-Context) demonstrate that our method significantly outperforms ex- isting baselines under extreme pixel-budget regimes. Our code is available at btttps//giis //tpthgttttt...// com/ /a//jjnn-e wat--iii/wwwo stage-edald
Introduction
Semantic segmentation a core task computer vision: assign a class label to each pixel in an image.
Applications abound in areas such as autonomous driving, robotics, and medical image analysis.
Despite the impressive performance of convolutional [T, and Transformer-basee models [5.).. an overarching challenge remains: how do we obtain the pixel-level annotations required to train
these models effectively? Large-scale datasets like Cityscapes (6] and Pascal-Context 7 require
meticulous per-pixel labeling, rendering data collection extremely costly.
Active learning (AL) aims mitigate labeling costs by strategically selecting a subset of unlabeled data points, for example, pixels, for annotation [8-10]. However, traditional sneertiinty-bsse AL methods for segmentation often select redundant pixels, since the & most uncertain pixels are often near one another [I] 12]. Representation-basee based methods do not select redundant pixels, but they tend to miss informative pixels near the boundary of objects. Furthermore, as representation- baaeed methods usually require computation of the pairwise similarity of candidates, it is prohibitive to consider all the available pixels at once.
*Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

In real-world scenarios, the total number of pixels is massive; for example, N images each of resolution H W yield a total of N. H W pixels. However, because of the high annotation cost, the labeling budget can be orders of magnitude smaller. We therefore formalize a low-budger active learning setting for semantic segmentation: each AI. round, one may annotate only b pixels from the entire unlabeled pool, with b < N W.H.W. In our standard (extreme) regime, we allocate one pixel per image for 10 rounds- - i.e., b = 0.1N pixels per round and a total of x N pixel labels across the whole process.
promising direction to address this data-scarce labeling scenario is to harness the representation power of diffusion models [33 [4]. Diffusion models are able to iteratively denoise over multiple time steps, generating highly detailed images (13). The initial steps of this reverse diffusion process capture the overall structure of objects, while the later steps focus on generating fine- ane-ggrined details. Baranchuk et al. [15] demonstrate that multi-timestamp features from diffusion models are beneficial for semi- eemi-supervised segmentation tasks. We will show that they can be used for uncertainty estimation, due to their ensemble-like nature.
We propose a two-stage strategy that efficiently pinpoints both diverse and epistemically uncertain pixels. In Stage 1, employ representation- AL method, in particular MaxHerding (16), to extract a set of candidate pixels which is manageable in size yet representative; we first narrow down candidates in each image, then refine across all images ensure global diversity. In Stage we exploit the diffusion backbone's 's stochastic multi-scale features to compute an entropy-ant augreted disagreement score (eDALD): we measure mutual information between noisy feature samples and labels, then add single-sample entropy term to capture prediction confidence. This combined criterion prioritizes pixels that both lie in under-explored regions of feature space and carry high model uncertainty, maximizing the benefit of each annotation under an extremely limited pixel budget.
Our contributions are summarized follows:
We formalize and address the challenging low-budget active learning for semantic segmentation, where mere fraction of pixels can be annotated per round.
We introduce a scalable two-stage pipeline (coverage - uncertainty) for pixel-level AL: a local- then- glebal MaxHerding stage yields representative candidate pool. followed by uncertainty- -bsed refinement.
We develop a diffusion-nativy uncertainty criterion (eDALD) that combines disagreement from stochastic multi-timestep features with a single-past entropy term; used after coverage, it comple- ments MaxHerding and yields substantial gains over one-stage uncer or coverage-only variants under tiny budgets.
We present comprehensive experiments on benchmark datasets Vid, ADE-Bed, Cityscapes, and Pascal-Context demonstrating consistent gains over multiple baselines when budgets are severely constrained.
2
Related Work
2.1 Active Learning for Classification
Active learning (AL) is a learning framework to improve the data efficiency of training machine learning models by strategically selecting the most informative data points for annotation. AL methods can be broadly categorized into two main approaches: uncertainty-based methods and representation--asee methods.
Uncertainty-basee Methods. Uncertainty-bart -bsed methods [17-20 generally select data points near the decision boundary. These simple methods are generally effective; however, they can be prone to selecting pixels with erratic or extreme predicted class probabilities. These pixels, such as isolated noise artifacts or very rare patterns, often exhibit high uncertainty yet lie far from the true decision boundary, contributing little to model improvement. To avoid this problem, some work has attempted to measure the change of model for given candidate data point and its corresponding pseudo- label 23] or the change of model outputs (44:7]]; in practice, these methods typically do not perform much better than the simple ones when using predictors defined by deep networks.
From a Bayesian perspective, on the other hand, the goal of AL is essentially to select data points that maximize mutual information (or equivalently, information gain) between model parameters 0

---

and the true label Y to be observed, given an input x and a model f.
X
E argmax I(0; Y [x) where I(0; Y xx) H(Y xx) E H(Y 8 0,xx
(1)
xEu
This objective is known as Bayesian Active Learning by Disagreement (BALD), since it selects a data point where model parameters under the posterior distribution disagree the most for the output 29 avoids redundant top-k selection by sampling without replacement from "powered" BALD distribution p(t) x s. Balanced Entropy (BalEnt) and its acquisition variant BalEntAcq [30] fit each softmax marginal to a Beta distribution, producing a bounded score that balances epistemic and aleatoric uncertainty.
label (28). PowerBALD
Representation-basee Methods. These methods instead select samples that are representative of the entire unlabeled data pool, often by analyzing learned data embeddings or features [IT] 31). Core-set methods, for example, aim to find small subset of the unlabeled data that best covers the diversity of the entire dataset in a feature space ( 31 32). Another notable approach is Variational Adversarial Active Learning (VAAL), which employs a variational autoencoder and discriminator to identify
samples that are most distinct from the already labeled data in learned latent space [32 33).
Hacohen et al. 34 recently demonstrated that representation--aaee methods perform significantly methods in low-budget regimes. A sequence of works has proposed
better than uncertainty-baase improving notions of coverage to select better representing samples: probability coverage (35]
generalized coverage
[6] and uncertainty coverage (3)..
2.2 Active Learning for Semantic Segmentation
Active learning for semantic segmentation extends classification - style AL to the pixel-wise labeling domain, where annotation costs are dramatically higher. Unlike classification, every pixel's label matters, so AL methods must balance selection granularity, spatial coherence, and computational efficiency. Broadly, existing approaches fall into three categories image-level, region-level, and pixel-level - each trading off annotation cost against precision and implementation complexity.
Image-level Methods. The most straightforward adaptation of classification-style active learning to segmentation is to select entire images for annotation. Many image-level selection strategies for semantic segmentation directly adapt techniques from classification. For instance, uncertainty-baat based methods often calculate a per-pixel uncertainty score and then aggregate these scores across the entire image, such as by averaging, to decide which images to query (IT). Xie et al. [31] also employ image-level selection, though they enhance it by considering semantic difficulty. Similarly, representation-baan -baed approaches like Core-set VAAL can be applied by selecting images whose overall feature representations contribute most to dataset diversity or are most distinct from already labeled data 33 33]. While simple implement, image-level querying quickly exhausts annotation budgets because every pixel even in well-understood regions- must be annotated.
Region-level Methods. To reduce per-query cost, region-level approaches annotate clusters of pixels such as superpixels 37, (38) rectangular patches (39], or bounding boxes [40). These methods save clicks by labeling chunks at once and often combine uncertainty with coverage learned difficulty. However, they assume a minimally competent model to estimate region scores: under extreme low budgets ("cold-start"), unreliable uncertainty difficulty estimates can misrank regions and degrade performance. Moreover, multi-class regions can introduce label ambiguity when boundaries cross cluster edges.
Pixel-level Methods. Pixel-level querying - selecting individual pixels - incurs the lowest per-unit cost and avoids eggion-boundary ambiguity but has received less attention, cach query yields limited information. PixelPick [4T] applies margin sampling to individual pixels, and Didari et al. use BalEntAAq [30 as Bayesian pixcl-uncertainty measure. Purely incertainty-drive pixel selection, however, demands many queries and often fails in low-budget regimes.
42
In contrast to these prior works, we introduce the first practical framework for low-budger active learning in semantic segmentation. Our two-stage pipeline first uses representation-bon based sampling to build compact, diverse candidate pool, then applies uncertainty-driven selection to pick the final pixels achieving high segmentation accuracy with only a handful of annotated points in low-budget scenarios.
3

---

Resresentation based (ty)
Diflusion Slep
Upcompling Concatemation
Diflusion Step Kg
every
F.
6,00
(cDALDOG)
Add
X2
MLL
0D o
Snyy?
9,00
Image
Unpentointy eDAL.D())
(a) Per-pixel inference
(h) Two-stage selection strategy
Figure I: Overview the proposed pipeline (illustrative hyperparameters)) (a) For each pixel x, the pre-trained diffusion model yields feature representation Z., uncertainty estimate eDALD(z). and the class prediction ((x) (here shown with T diffusion steps and L = 3 layers purely for illustration). (b) These outputs drive two-stage selection: Stage picks K 5 representative
pixels per image to form a global pool of size M 10, then Stage applies the uncertainty criterion to select the final b pixels (all parameter values are example settings, not the exact experimental hyperparameters)
2.3 Diffusion Models in Vision
Generative Diffusion. Denoising Diffusion Probabilistic Models (DDPM) [3]
score SDE [43
45
introduce novel paradigm of generative models: progressively destroy image by adding noise, then learn to gradually reverse this process. Follow-up works such as Latent Diffusion Models (LDM)
14
improve scalability for high-resolution images.
Diffusion for Downstream Tasks. Some recent research explore leveraging the representations from diffusion models for tasks beyond generation (46). In particular, medical image segmentation [7 and few-shot segmentation [15] benefit from multi-scale features spanning from broad global context (carlier denoising steps) to detailed object boundaries (later steps). Nevertheless, the integration of diffusion representations into active learning remains unexplored.
Proposed Method
In this work, we present novel two-stage active learning pipeline tailored for semantic segmentation under strictly low-budget setting. Our approach integrates the strengths of pre-trained diffusion models for feature extraction with two-pronged sample selection strategy that first ensures represen- tation diversity and then refines the selection using uncertainty estimation. The overall pipeline illustrated in Figure and detailed in Algorithm
3.1 Preliminaries
Problem Setup. Let = ( >==1 denote of N images, where each image I, contains H x W pixels. At each AI. round E [ with a labeled pixel set L., and an unlabeled set Ur, we select subset S. 14, such that |8,| for annotation, where is a pre-defined labeling budget per round. More specifically, active learning involves the following steps in each r-th round:
1. Measure how informative each pixel in (or subset of) U, is
2. Select subset S, C 24, that consists of "most informative" pixels
3. Update both L,+1 + L, U S, and U4 U, S,
4. Retrain the segmentation model on Cy+1
In this work, we focus on extremely low-budget setting i.e., b & (N x H x W), the total number of pixels. The challenge then to select highly informative pixels for robust generalization in semantic segmentation tasks.

---

Algorithm Two-Stage Low-Budget Active Learning at Round r
Require:
Image set Z, labeled pixel set Lr. unlabeled pixel set Ur diffusion feature extractor S. current segmentation head sø, the number of candidates per image K, global candidate pool size M, per-round budget b.
Ensure: Update labeled set C, and unlabeled set set with new annotated pixels.
1: Mo 0 // Initialize the global candidate pool Candidate Selection
2: // Stage
Representation- 3: for each image I E I do
4: Extract multi-scale features Z xxxIxeXxu using f
5: Select a set of K representative pixels R by applying MaxHerding to Z
6: MM - Mo U.
7: end for
8: Obtain the global candidate pool M of size M by applying MaxHerding to Mo
9: // Stage 2: incertainty-driver Selection
10: for each x E .M do
11:
Compute eDALD(x)) in Eq. using the segmentation head sa and f 12: end for
13: B, < -oop- pixels from M sorted by descending eDALD(x)
14: C,+1 C, U B,, Ur+1 6 U, \ B,
15: return Up7+
Diffusion-based Semantic Segmentation. Our segmentation framework builds on the LEDM architecture (15) and exploits pre-trained diffusion model denoted as - DDPM [13] to extract robust multi-scale features. Note that it is crucial to utilize good representations, particularly low-budget regimes, as a segmentation model trained on small labeled set itself would not be sufficient to learn good representations.
For image I, the diffusion model produces feature maps multiple denoising timesteps (ti, t2....,,ty brrr and layers E (4, (2. For a given pixel x in an image I (at coordinates (w,h), h), where S s W and < h S H), let fx)(x) E P denote feature vector from layer I at denoising timestep l. We obtain a multi-scale representation by concatenating these features:
Zx (x),,()); (x): ft. (x); (x): fer (x): f (x) RR.
where D= ΣT =j=1 Dr..d- This comprehensive feature vector 2x encapsulates rich semantic cues learned during the diffusion process as demonstrated in [5]. We omit the subscript when is clear from context.
The predicted probability of segmentation for C' objecc classes produced by a lightweight seg- mentation head 89 RDD C It consists of 2-layer MIP with ReLU activation and batch normalization, followed by a softmax layer. Note that we deliberately add parameter notation only for the segmentation head not for the diffusion model to describe that we only update the segmentation head not the diffusion model.
3.2 Representation- based Candidate Selection
Representation--bsse AL methods rely the pairwise similarity measure. Hence, it is often infeasible to select pixels from all possible (N x H x W) pixels, as it requires H22 computation for pairwise comparison. To tackle this problem,
adopt two-step strategy that leverages the MaxHerding algorithm (16), a representation-base based AL method, in each step to ensure diversity while reducing computational complexity; please refer to Line 1-8 in Algorithm
We first identify K representative pixels within each image I. More specifically, for each pixel we obtain multi-scale feature vector z extracted from the diffusion model ff using Eq. (2). We then greedily select optimal pixel x* at time using generalized coverage Č₂ [16] follows:
X* E argmax C+(C {x)) where (CU(U( = 2 &EL
1
Σ XEW
max 1(xxx)) (ECC
(3)
5

---

Here, k is a function that measures pairwise similarity between x and x'. We simply use RBF kernel We repeat this until we obtain K pixels yielding a compact for & as k(x, x') = exp ||z, /z=-z//
o²
yet diverse subset R. for each image T.
By restricting the search space for selecting pixels to an individual image, we reduce the size of candidates to (H W) compared to (N x H x W). We obtain the initial global pool Mo by merging the selected subset R for each image I. This pool has size (N x K), which is much smaller than apply MaxHerding again across the entire
(N x H x W). To further refine these candidates, merged set to select M representative pixels, which form a candidate pool for the final b pixels. The final global pool M of size M thus achieves good coverage over all images collectively. Typically, M & H W.
Through this stage, we ensure that our final candidate set M is both diverse (capturing the data manifold effectively) and manageable in size, laying the foundation for more targeted selection via uncertainty in the next step.
3.3 Uncertainty-driven Selection
Diffusion-based Active Learning by Disagreement (DALD). From the diverse candidate pool M, we further refine our selection by identifying the most informative b pixels using an uncertainty-basee measure; refer to Line 9-13 in Algorithm Our method uniquely exploits the structure of the diffusion model: each pixel x is associated with multiple feature vectors.
Inspired by the BALD objective in Eq. ( we propose Diffusiion based Active Learning by Disagree- ment (DALD) that selects a new data point follows:
x" E argmax 1(Ý; Z x, s8.
(4)
XEU
where y denotes a random variable for the predicted label of a pixel x and Z denotes a random variable for the concatenated multi-scale features from computed using Eq. (2). Note that the stochasticity of Z comes from noise added to at f1....,ty br timesteps. We define this conditional distribution of Z given x as q(- x; f). Furthermore, suppose X denotes a random variable for pixel x, then the computational probabilistic graphical model defined as X + Z Y, assuming that X does not provide additional information about knowing Z.
The motivation of DALD is to select a data point where information gain for a noised multi-scale feature maximized given label y. Intuitively, if the information gain of a multi-scale feature is close to 0, knowing the label y does not help removing the ambiguity of the feature while if the information gain of multi-scale high, knowing the label y significantly removes the uncertainty of the noised feature.
The mutual information in Eq. (4) can be decomposed into the following two terms:
I(Ý: Z x, Sg. f) H(Y S0
E
H Z 2,
(5)
Unconditional entrupy
Conditional entropy
We compute the uncertainty of the predicted label ý instead of multi-scale feature Z to make computation easy.
The computation of conditional entropy is relatively more straightforward. As y is conditionally independent to a clean pixel x given a multi-scale feature the expected conditional entropy be approximated using M samples z, as follows:
M
E
H(
Z
x,x)
R=
Do (e(l log pe (m)
(6)
2--g(-1x)
M
m= yey
It can happen either when aleatoric uncertainty is very high epistemic uncertainty is very low.

---

where (b(l) = so (z)y.. Therefore, conditional entropy is approximated as the mean of entropy. In contrast, we compute the unconditional entropy as the entropy of the mean predictions as follows:
H( x, 80,
Σ po po(y)x) log (M)lx, where po (o(xx)) BEY
1 M
po (yly)l(1))
Entropy-Augmenttr DALD (eDALD). Although disagreement-basee selection has the advantage of being less sensitive to aleatoric (or irreducible) uncertainty, it also has a clear limitation: it does not account for the confidence of model predictions. For example, two different segmentation models may output identical mutual information for given pixel, although one may produce low entropy (high confidence) output, while the other yields high entropy (low confidence) output. This discrepancy illustrates that disagreement alone may fail to capture the full picture of predictive uncertainty.
As simple and computationally inexpensive remedy to this limitation, we introduce an additional entropy term based on separate sample. Specifically, our selection objective becomes the following:
argmax cDALD(x), where cDALD(x) I(;; Z x, 88. f) H( z .). xEll
(8)
where z (0) denotes an independently drawn sample, separate from sample m = 1, 2....... M used to estimate the mutual information. This extra entropy term highlights pixels where the model is less confident, further enhancing the sensitivity of acquisition to both disagreement and absolute uncertainty.
3.4 Overall Selection and Training Procedure
Our complete active learning pipeline is summarized in Algorithm I each of the R rounds, we (1) extract multi-scale diffusion features Zx for all unlabeled pixels x E 2, (2) select b pixels through our two-stage sampler (MaxHerding - eDALD), (3) annotate their labels and update the labeled/unlabeled sets, and (4) train on the expanded labeled set.
During training, the segmentation head 8a is optimized via cross-entropy loss computed over the up-to-date labeled set L:
fr E argmin
Σ
Do x), f).
[C]
0
(X.ylEL
This cycle is repeated over R rounds, allowing the model to progressively learn from a carefully curated and highly informative set of pixels while keeping the annotation costs minimal.
4 Experiments
4.1 Datasets and Setup
We evaluate our low-budget active learning pipeline on four standard semantic segmentation bench- marks. All images are processed at 256 x 256 resolution for compatibility with the diffusion backbone:
Cam Vid [48]: An urban driving dataset with 367 train and 233 test images, each labeled into 11 classes. All images are center-cropped to 256 256.
ADE-Bed: subset of ADE20K (49) consisting of bedroom images with 964 train and 650 test images annotated with the 30 most common object classes. We resize the shorter side to 256 pixels,
preserve aspect ratio, then center-crop to 256 x 256.
Cityscapes [6]: A street-scene dataset comprising 2,975 train and 500 validation images over 19 classes. We resize and center-crop to 256 x 256.
Pascal-Context [7 A scene parsing dataset providing dense semantic labels for more than 400 categories. Following convention [7, (7,50 51). we use 33 most frequent categories. It contains 4,998
train and 5.105 validation images. Images resized to 256 256 using bilinear interpolation.
fix a total annotation budget B to be the average of labeled pixels per image, i.., B = N. For Pascal-Context with N= 5,000, this implies B = 5,000 pixels practical scale for large datasets. This budget is evenly split across pixels per round. As intended, we only annotate 0.0015% of the total pixels after 10 AL rounds.
Budget Setting.
AL rounds, yielding B/R R = O.

---

4.2 Implementation Details
Diffusion Model. We adopt an ImageNet pre-trained diffusion model [3] for feature extraction from publicly available guided diffusion repo] To capture multi-scale cues, we sample T = 3 timesteps ( = 50, l2 150, l3 = 250). At each timestep, we extract the features from I, = layers (1 = 5.12 8, 1a 12, 14 17), resulting in rich, concatenated representation for each pixel. For uncertainty estimation (DAI.DeeDALD)), we draw M 5 noisy feature samples per pixel.
apply MaxHerding [16] to the pixels of each image to obtain K = 50 representative samples, which are then merged across images into an initial global pool MM. We further apply MaxHerding to Mo obtain the final candidate set M, which contains half of the samples in Mog
Candidate Selection. Following Section 3.2
Training Procedure. We Adam with an initial learning rate of 1 10-3 for training the pixel classifier, with a batch size of We apply early stopping if the segmentation loss does not improve for 50 consecutive iterations and the target pixel accuracy exceeds 95%. This training procedure is repeated for 10 active learning rounds, each time adding newly labeled pixels to the training set.
4.3 Quantitative Results
Representation-Fiss Uncertainty-Only: All results in Table use our iuffuusionbbccbare on CamVid dataset. We compare "UC Only" (single-stage: uncertainty) against "Herding - UC" (two-stage: representation
uncertainty), reporting absolute and relative gains. Overall, enforcing diversity first consistently boosts simple criteria entropy gains +5.51 mloU (+21.81% %), and margin sampling gains +1.5 mloU (+4.8%) confirming that representation filtering improves even basic uncertainty sampling.
However, pure DALD (random-noise and BALD (MC-Dropout) actually worsen after MaxHerding ( (-2.66 and 1.8 mloU, respectively). suggesting that disagreement-only measures can overempha- size noisy redundant regions once diversity is already enforced. In contrast, their intropy-augmente versions show dramatic improvements: eBALD gains +6.16 mloU (+23.73%) and eDALD gains +10.98 mloU (+43.68%%). This striking boost reflects the complementary roles of the two signals: disagreement identifies perturbation-sensinn recovers consistently low-confidence areas that disagreement misses. Their combination reshapes the acquisition ranking rather than merely scaling one criterion, leading to more balanced and informative selections. Eventually, "Herding eDALD" achieves the highest mloU of 36.12, demonstrating that combining disagreement with confidence-awaan uncertainty, along with representation diversity, is crucial achieve substantial gains under extreme low-budget settings.
pixels that entropy alone may overlook, while entropy
Table I: Effect of representation--fist filtering on uncertainty sampling on Cam Vid, measured in mloU (%). "UC Only" shows single-stage performance; "Herding UC" adds MaxHerding before uncertainty. Gains are shown in absolute points and percentages, with positive values in blue and negative in red. All results are based on the mean std from three independent runs.
Uncertainty
UC Only
Herding + UC
Gain (pp)
Gain (%)
Entropy
25.26 + 0,36
30.77 1 0.44
+5.51
+21.81
Margin
31.27 + 1.10
32.77 = 0.75
+1.50
+4,80
BALD
24.59 0.97
22.79 0.89
-1.80
-7.32
DALD
23.81 3.60
21.05 1.05
-2.76
-11.59
PowerBALD
30.03 0.76
31.57 0.79
+1.54
+5.13
PowerDALD
31.30 + 1.22
32.00 + 0.66
+0.70
+2,24
eBALD (Entropy BALD)
25.96 + 1.92
32.12 + 0.40
+6.16
+23.73
eDALD (Entropy DALD)
25.14 + 0.57
36.12 0.24
+10.98
+43.68
Performance Comparison with Baselines. cobpeeecompares mloU after 10 rounds under a severe pixel-budget across four datasets. Pixel-level AL research is still nascent, so we benchmark against two representative methods PixelPick (41) and Didari et al. [42 both originally designed for settings where a fixed number of pixels per image can be labeled each round. In our extreme low- budget regime (on average only 0.1 pixels per image per round are labeled), we select the top-b

---

most uncertain pixels from the unified candidate pool across the dataset according to the acquisition criterion.
Given PixelPick is equivalent to Margin selection with Deepl V3 backbone, the benefit of DDPM- backhone over V3 can be estimated by comparing PixelPick (1st row) Margin (5th row):
DDPM-backbone improves the performance by 11.81 mloU on average (22.85 85 vs. 34.66), and by up 21.68 mloU on the ADE-Bed dataset (8.35 vs. 30.03). Similarly, Didari et al. [42] (BalEntAcq + DeepLabV3, 2nd row) generally performs worse than BalEntAcq with DDPM-backbone (6th row). Putting all together, our proposed method, 2-Stage eDALD, achieves the best overall results, consistently outperforming all baselines on average by non-trivial margins (from 2. 2.48 to 17. 17.36 mloU). diffusion- mutual information within diverse candidate pool is notably effective under extreme low-budget regimes.
This demonstrates that the
Table 2: mloU (%) of active learning methods under a low-budget regime (10 rounds). Our two-stage method is highlighted gray, All results are based on the mean std from three independent runs.
Backhone
Method
CamVid
ADE-Bed
Cityscapes
Pascal-C
Avg
PixelPick
41
29.93 = 0.12
8.35 + 0,41
26.82 + 0.14
26.28 + 0,09
22.85
De
52
Didari et al.
42
22.47 = 0.10
8.66 + 0.53
19.85 + 0.07
28.15 + 0.11
19.78
Random Entropy
25.91 = 1.23
17.83 + 0.62
27.13 + 1.38
41.70 + 2.08
28.14
25.26 0.36
23.02 + 1.64
28.62 + 1.05
42.09 = 1.99
29.74
Margin
31.27 = 1.10
30.03
0.37
32.23 + 1.21
45.11 2.45
34.66
DDPM 13
BalEntAcq
19.37 = 1.10
17.48 + 1.36
24.04 - 2.07
33.06 - 4.18
23.49
eDALD
25.14 + 0.57
23.06 + 1.29
29.44
1 1.38
43.05 1 0.12
30.17
2-Stage eDALD
36.12 = 0.24
31.12 1 0.20
33.34 1 0.78
47.98
0.41
37.14
Round-Wise Learning Curves. Figure 2 plots mloU over 10 AL rounds for various methods across four datasets. Similar to Table PixelPick [41] and Didari al. [42] use the DeepLabV3 backbone, whereas the others are based on DDPM. Across all benchmarks, our two-stage scheme (Herding > eDALD) consistently achieves the highest performance by the final rounds. In contrast, the single- stage uncertainty- improvements throughout the rounds, leading comparatively smaller overall gains. Together, these results highlight the effectiveness of the proposed two-Stage eDALD strategy.
methods (Entropy, Margin, BalEntAcq, and eDALD) show only gradual
PixcPaik
Didarietal d
Randum
Entnpy
Mangin
BalEsAcy
DALD
2GSge cDALD
35
30
30
mnooi (%)
mitu (%)
minli (%)
20-
mlol]
2,5
300
20-
20
10
15
9
10
6
9 10
3
6 Round (6) Cityscapes
9
10
6
9
Round
(a) Camvid
(b) ADE-Bed
(d) Pascal-Contex:
Figure 2: Active learning progression: mloU (%) vs. Al. rounds on (a) Cam CamVid, (b) ADE-Bed, (c) Cityscapes, and (d) Pascal-Conters Curves compare Random sampling, single-stage uncertainty methods, and two-stage variants.
Convergence to Fully-Supervised Performance. We evaluate how quickly each method ap- proaches fully supervised performance under tiny per-round budget 6 0.1N. The fully supervised mloUs are: ADE-Bed: 45.58, Cam 52.22, Cityscapes: 43.04, Pascal-Context: 60.68. Motivated by our low-budget findings and the adaptive-queryiee perspective, we use a simple two-phase sched- ule: rounds 1-10 (the extreme low-budget phase) use our two-stage eDALD pipeline (coverage - uncertainty) to establish broad, non-redundant coverage; for rounds > 10 we drop the coverage stage and continue with uncertainty-only (Margin). This switch reflects two observations: (i) once coverage is established, diversity offers diminishing returns while residual errors concentrate near boundaries, where Margin is effective; and (ii) removing Stage slightly reduces overhead while accelerating late-phase convergence.

---

Figure shows mloU vs. AL rounds at b = 0.1N. Across all four datasets, the proposed schedule consistently reaches 90% of the fully supervised mloU far fewer rounds than PixelPick (41). Concretely, two-stage eDALD (early) - Margin (late) attains the 90% target within 21-47 rounds specifically, VamVid: rounds, ADE-Bed: 47, Cityscapes: 28, Pascal-Context: 21 while labeling only 0.003% 0.007% of all pixels in total.
Fully Supervised
PixlPPic
2-Singe
60
40
mloU (%)
IntoU (%)
mnool (%)
mnool (36)
-50
40
-00
20
20
30
10
201
A0 0
+2
+9
9
19
+2
5
49
(8) Camvid
(b) ADE-Bed
(c) Cityscapes
Puscal-Context
Figure 3: mIoU AL rounds (budget b 0.1N). Horizontal gray lines denote fully supervised mloUs, and green lines denote the 90% thresholds. We use two-stage eDALD for rounds 1 and switch to Margin thereafter.
4.4 Qualitative Analysis
Figure 4 compares pixel selections four examples from different benchmarks. Margin (Baseline) tends to select overlapping pixels mostly near object boundaries whereas our two-stage method evenly covers object boundaries, thin structures, and small classes even under extremely low pixel budgets.
Cityscapes
CanVid
Pascal -Context
ADE-Bed
(a) Image
(b) Baseline
(c) Ours
(a) Image
(b) Baseline
(d) Ours
Figure 4: Qualitative pixel-selection comparison. (a) Input image; (b) baseline selections; (c) our
selections. Light blue dots indicate selected pixels. Our method covers boundaries and fine details more broadly.
Conclusion
We addressed the problem of low-budget active learning for semantic segmentation by introducing novel, two-stage pipeline that first narrows down candidate pixels through clustering with diffusion- based representations, then refines the selection via disagreement based uncertainty sampling. Our approach effectively balances diversity and informativeness under extreme labeling constraints, as validated four standard benchmarks. This work would open up a new direction for integrating advanced generative backbones such as diffusion models, to budget-constrained annotation pipelines, and paves the way for more practical, large-scale semantic segmentation systems.
The proposed diffusion- based active learning by disagreement (DALD) is tailored for a diffusion-backbone it requires sampling based on different random noises. Thus, DALD is applicable only with the diffusion-backbone However, antropy-augmentatie and two-stage sampling based MaxHerding is agnostic the choice of backbones.
Limitations.
10

---

Acknowledgments and Disclosure of Funding
This work was supported by Institute of Information & communications Technology Planning Evaluation (ITPP) grant (No. 55-2002--00155966 Artificial Intelligence Convergence Innovation Human Resources Development (Ewha Womans University)) and the National Research Foundation funded by the Korea government (MSIT), as well as by the Natural Sciences and Engineering Research Council of Canada, the Canada CIFAR Al Chairs program, Calcul Québec, the BCI DRI Group, and the Digital Research Alliance of Canada.
of Korea (NRF) grant (No. RS-2025-16070555
References
[1] Karen Simonyan and Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition". ICLR. 2015.
[2] Kaiming He, Xiangyu Zhang. Shaoqing Ren, and Jian Sun. "Deep residual learning for image recognition". CVPR. 2016, pp. 770-778.
[3] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. "An Image is Worth 16x16 Words: Transformers for Image
Recognition at Scale". ICLR. 2021.
[4] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. "Swin transformer: Hierarchical vision transformer using shifted windows". International Conference on Computer Vision. 2021, pp. 10012-10022.
[5] Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. "Training data-efficient image transformers & distillation through attention". International Conference on Machine Learning. PMLR. 2021, pp. 10347-10357.
[6] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. "The cityscapes dataset for semantic urban scene understanding". Proceedings of the IEEE conference on computer vision and pattern recognition. 2016, pp. 3213-3223.
[7] Roozbeh Mottaghi, Xianjie Chen, Xiaobai Liu, Nam-Gyu Cho, Scong- Whan Lee, Sanja Fidler, Raquel Urtasun, and Alan Yuille. "The Role of Context for Object Detection and Semantic Segmentation in the Wild". Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, 2014, pp. 891-898.
[8] Alex Kendall and Yarin Gal. "What uncertainties do we need in bayesian deep learning for computer vision?" Advances in neural information processing systems 30 (2017).
[9] Ozan Sener and Silvio Savarese. "Active Learning for Convolutional Neural Networks: A Core-Set Approach" ICLR. 2018.
[10] Donggeun Yoo and In So Kweon. "Leearning loss for active learning". Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019, pp. 93-102.
[11] Sudhanshu Mittal, Joshua Niemeijer, Jörg P Schäfer, and Thomas Brox. "Best practices in active learning for semantic segmentation". DAGM German Conference on Pattern Recognition. Springer. 2023, pp. 427-442.
[12] Siteng Ma, Haochang Wu, Aonghus Lawlor, and Ruihai Dong. "Breaking the barrier: Selective uncertainty-bast based active learning for medical image segmentation". ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEBE, 2024, pp. 1531-1535.
[13] Jonathan Ho, Ajay Jain, and Pieter Abbeel. "Denoising diffusion probabilistic models". vances in neural information processing systems 33 (2020), pp. 6840-6851.
[14] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. "High-resoluse image synthesis with latent diffusion models". Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022, pp. 10684-10695.
[15] Dmitry Baranchuk, Andrey Voynov, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Lffel-Efficient Semantic Segmentation with Diffusion Models" ICLR. 2022.
[16] Wonho Bae, Junhyug Noh, and Danica J Sutherland. "Generalized Coverage for More Robust Low-Budget Active Learning". European Conference on Computer Vision. Springer. 2024,
pp. 318-334.

---

[17] David D Lewis and Jason Catlet.. "Heterogeneous Uncertainty Sampling for Supervised Learning.. Machine learning proceedings 1994. Elsevier, 1994, pp. 156-156.
[18] David D. Lewis and William A. Gale. "A sequential algorithm for training text classifiers" Proceedings of the 17th Annual International ACM SIGIR Conference Research and Development in Information Retrieval. Springer- Opringer-Verlag. 1994, Pp. 3-12.
[19] Tobias Scheffer, Christian Decomain, and Stefan Wrobel. "Active hidden markov models for information extraction". International symposium on intelligent data analysis. Springer. 2001, pp. 309-318.
[20] Dan Wang and Yi Shang. "A new active labeling method for deep learning".. 2014 International joint conference on neural networks (IJCNN) IEEE. 2014, pp. 112-119.
[21] Burr Settles, Mark Craven, and Soumya Ray. "Multiple-instance active learning". Vol. 20.
2007.
[22] Burr Settles. "Active Learning Literature Survey". University of Wisconsin-Madison Depart- ment of Computer Sciences, 2009.
[23] Jordan T. Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. "Deep batch active learning by diverse, uncertain gradient lower bounds". ICLR (2020).
[24] Alexander Freytag. Erik Rodner, and Joachim Denzler. "Selecting influential examples: Active learning with expected model output changes". European conference on computer vision. Springer. 2014, pp. 562-577.
[25] Christoph Käding, Erik Rodner, Alexander Freytag, and Joachim Denzler. "Active and Contin- uous Exploration with Deep Neural Networks and Expected Model Output Changes". NIPSW. 2016.
[26] Christoph Käding, Erik Rodner, Alexander Freytag, Oliver Mothes, Björn Barz, Joachim Denzler, and Carl Zeiss AG. "Active Learning for Regression Tasks with Expected Model Output Changes." " BMVC. 2018, p. 103.
[27] Mohamad Amin Mohamadi, Wonho Bae, and Danica Sutherland. "Making look-ahead active learning strategies feasible with neural tangent kernels". NeurlPS. Vol. 35. 2022, pp. 12542- 12553.
[28] Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel. "Bayesian active learning for classification and preference learning". arXiv preprint arXiv: /2..5745 (2011).
[29] Andreas Kirsch, Sebastian Farquhar, Parmida Atighehchian, Andrew Jesson, Frederic Branchaud-Charron and Yarin Gal. "Stochastic batch acquisition: A simple baseline for deep active learning". arviv preprint arXiv:2106. 12059 (2021).
[30] Jae Oh Woo. "Active learning in bayesian neural networks with balanced entropy learning principle". arXiv preprint arXiv:2105. 14559 (2021).
[31] Shuai Xie, Zunlei Feng, Ying Chen, Songtao Sun, Chao Ma, and Mingli Song. "Deal: Difficulty- aware active learning for semantic segmentation". Proceedings of the Asian conference on computer vision. 2020.
[32] Zongyao Lyu and William Beksi. "Semi-Supervised Variational Adversarial Active Learning via Learning to Rank and Agreement-Basee Pseudo Labeling". International Conference on Pattern Recognition. Springer. 2024, pp. 1-16.
[33] Samarth Sinha, Sayna Ebrahimi, and Trevor Darrell. "Variational adversarial active learning". Computer Vision (ICCV). 2019, pp. 5971-5980
2019 IEEE/CVF International Conference
[34] Guy Hacohen, Avihu Dekel, and Daphna Weinshall. "Active Learning on Budget: Opposite Strategies Suit High and Low Budgets". Proceedings of the 39th International Conference on Machine Learning (ICML). PMLR, 2022, pp. 8175-8195
[35] Ofer Yehuda, Avihu Dekel, Guy Hacohen, and Daphna Weinshall. "Active learning through a covering lens". NeurlPS. 2022.
[36] Wonho Bac, Gabriel L Oliveira, and Danica J Sutherland. "Uncertainty Herding: One Active Learning Method for All Label Budgets". ICLR. 2025.
[37] Lile Cai, Xun Xu, Jun Hao Liew, and Chuan Sheng Foo. "Revisiting Superpixels for Active Learning in Semantic Segmentation With Realistic Annotation Costs". Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021, pp. 10988- 10997.
12

---

[38] Hoyoung Kim, Minhyeon Oh, Schyun Hwang, Suha Kwak, and Jungseul Ok. "Adaptive super- pixel for active learning in semantic segmentation", International Conference on Computer Vision. 2023, pp. 943-953.
[39] Arantxa Casanova, Pedro O. Pinheiro, Negar Rostamzadeh, and Christopher J. Pal. "Reinforeed active learning for image segmentation" (2020).
[40] Tejaswi Kasarla, Gattigorla Nagendar, Guruprasad M Hegde, Vineeth Balasubramanian, and CV Jawahar. "Region-bade based active learning for efficient labeling in semantic segmentation"
2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE. 2019,
pp. 1109-1177.
[41] Gyungin Shin, Weidi Xie, and Samuel Albanic. "All You Need Are a Few Pixels: Semantic Segmentation With PixelPick". ICCVW. 2021.
[42] Sima Didari, Wenjun Hu, Jae Oh Woo, Heng Hao, Hankyu Moon, and Seungjai Min. "Bayesian Active Learning for Semantic Segmentation". arXiv preprint arXiv:2408. 01694 (2024).
[43] Yang Song and Stefano Ermon. "Generative modeling by estimating gradients of the data distribution" Advances in neural information processing systems 32 (2019).
[44] Yang Song and Stefano Ermon. "Improved techniques for training score-based generative models"... Advances in neural information processing systems 33 (2020), pp. 12438- 12448.
[45] Yang Song. Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. "Score-Based Generative Modeling through Stochastic Differential Equations" (2021).
[46] Yuanfeng Ji, Zhe Chen, Enze Xie, Lanqing Hong, Xihui Liu, Zhaoqiang Liu, Tong Lu, Zhenguo Li, and Ping Luo. "Ddp: Diffusion model for dense visual prediction". International Conference on Computer Vision. 2023, pp. 21741-21752.
[47] Junde Wu, Rao Fu, Huihui Fang, Yu Zhang, Yehui Yang, Haoyi Xiong, Huiying Liu, and Yanwu Xu. "Medsegdiff:: Medical image segmentation with diffusion probabilistic model". Medical Imaging with Deep Learning. PMLR. 2024, pp. 1623-1639.
[48] Gabriel J Brostow, Julien Fauqueur, and Roberto Cipolla. "Semantic object classes in video: A high-definition ground truth database". Puttern recognition letters 30.2 (2009), pp. 88-97.
[49] Bolei Zhou, Hang Zhao, Xavier Puig, Sanja Fidler, Adela Barriuso, and Antonio Torralba. "Scene parsing through ade20k dataset". Proceedings of the IEEE conference computer vision and pattern recognition. 2017, pp. 633-641.
[50] Aayush Bansal, Xinlei Chen, Bryan Russell, Abhinav Gupta, and Deva Ramanan. "Pixelnet: Towards general pixel-level architecture". arXiv preprint arXiv: 000990 (2017).
[51] Zhangxuan Gu, Hanwang Lin, Piotr Koniusz, Hong Li, Ming Yang. Yu Qiao, and Yanwei Fu. "CaGNet: Context-aware Feature Generation for Zero-shot Semantic Segmentation". Proceedings of the 28th ACM International Conference on Multimedia. 2020, pp. 1921-1929.
[52] Liang- CCing Chen, George Papandreou, Florian Schroff, and Hartwig Adam. "Rethinking atrous convolution for semantic image segmentation". arXiv preprint arXiv:1706.. 05587 (2017).
[53] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan- Yen Lo, et al. "Segment anything". International Conference on Computer Vision 2023, pp. 4015-4026.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes]
Justification: Yes, the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope. Guidelines:
The answer NA means that the abstract and introduction do not include the claims
made in the paper.
The abstract and/or introduction should clearly state the claims made, including the
contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
Justification: We discuss the "cold-start" limitation - dependence on a minimally competent initial model leading to unreliable uncertainty estimates and potential biases in the Broader Impacts and Related Work sections. Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms
and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledgee in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: Our work contains no formal theoretical results or proofs, focusing entirely on methodological and empirical contributions.
Guidelines:
The answer NA means that the paper does not include theoretical results.
14

---

All the theorems, formulas, and proofs in the paper should be numbered and cross
referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)? Answer: [Yes]
Justification: We provide complete details for all experiments including dataset splits and preprocessing, exact budgets per round, diffusion model timesteps and layers, segmentation head architecture, optimizer settings (learning rate, weight decay, batch size), number of epochs, and the two-stage AL algorithm parameters - to enable full reproducibility of our claims.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken
to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c)
If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g.. with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: All benchmarks used (Cam ADE-Bed, Cityscapes, Pascal-Context) publicly available. We release our code, experiment configs, and pretrained checkpoints
15

---

with step-by-step instructions (environment, data prep, and scripts to reproduce the main tables/fggures).
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
SeriideesCCoeeSumissiiioooo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to rum to reproduce the results. See the NeurlPS code and data submission guidelines (https wwwdddSSumisssooooo...o
//nips.co muublic/guudess
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: The paper has included all the training and test details including data splits, hyperparameters, how they were chosen, and type of optimizer.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Main results are reported as mean + standard deviation over three random seeds.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
16

---

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are of range (e.g. negative
error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text. 8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: Detailed compute specifications including GPU types, memory, and runtime estimates are provided the supplementary material, ensuring full reproducibility.
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics /tttps:// http://ppp.....pppcccccccccceeae Answer:
[Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. there a special consid-
eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Ye]]
Justification: The paper has included Broader Impacts section at the supplementary mate- rial.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal
impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks,
17

---

mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA]
Question:
Justification: The paper poses no such risks. Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite and link all external datasets and pretrained backbones, and use them under their respective research/non-commercd licenses. We do not redistribute third- party data or weights; the README and appendix list sources and license notes, and our released code is accompanied by explicit open-source license. Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. cww hom/datasetss has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
18

---

Question: For crowdsoureing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects. Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required. Answer: [NA]
Justification: The core method development this research does involve LLMs as any important, original, or non-standard components. LLMs are only used for language clarity. Guidelines:
The answer NA means that the core method development in this research does not involve LLMMs as any important, original, non standaadd components. Please refer to our LLM policy httt::///eerrips..c//oooeeeeeee for what should or should not be described.
19

---

Appendix
A
Broader Impacts
Our two-stage low-budget active learning pipeline can reduce annotation requirements by orders of magnitude, making high-quality semantic segmentation accessible in domains with scarce labeling resources such as medical imaging (e.g., histopathology. radiology), environmental monitoring (e.g., land-cover mapping, wildlife surveys), and infrastructure inspection in developing regions. By building on frozen, pre-trained diffusion backbone, we also cut down on repeated large-scale scale training, thereby lowering computational demands and carbon emissions. This democratizes cutting- edge segmentation tools for academic labs, start-ups, and NGOs that lack extensive annotation budgets compute clusters.
On the other hand, any automated selection mechanism risks perpetuating biases present in the pre-trained diffusion model or in the small initial labeled set potentially under-representing rare or sensitive classes (e.g., skin lesions medical scans, minority populations in urban scenes). Moreover, ease of pixel-level segmentation could be misused for large-scale surveillance or privacy-invaase monitoring if deployed without strict governance. We therefore recommend rigorous bias audits, transparent reporting of model behavior on under-represented groups, and adherence to data-privacy regulations when applying our approach in sensitive settings.
B Implementation Details
This section provides all settings needed to reproduce our experiments, All code, configuration files, and training/evaluation scripts publicly available at https://github. com/ jan//nnnimmm txo-stage-edall
B.1 Hyperparameters
Table 3 summarizes the key hyperparameters and implementation details used across the datasets in our experiments.
Table 3: Hyperparameters and implementation details across datasets.
Details
ADE-Bed
Cam Vid
Cityscapes
Pascal VOC
Diffusion model
1sun lsun_bedit bedroom Center crop
imagenet_ _56 Bilinear
imagenet_22 256
imagenet_ Bilinear
Resize method
Center crop
Image resolution
256 256
Diffusion Steps
(50, 150, 250}
Feature blocks
(5, 8, 12, 17] 5
Batch size
Learning rate
1 x 10-3
Weight decay Optimizer
1 10-5
Adam
Scheduler
cosineAnnealingg..t
Classifier
MI.P
Notes:
imagenet 256 refers unconditional ImageNet diffusion model trained at 256 resolution. bedroom refers to a diffusion model trained on LSUN bedroom data (three classes) at 256 resolution.
Diffusion steps and feature blocks were chosen based on preliminary analysis to capture both low- and high-level semantics; see Section C.7 for further discussion.
20

---

B.2 Feature Extraction from the Diffusion Model
To build comprehensive multi-scale representations, we extract intermediate features from a pre- trained diffusion model at denoising steps t E (50, 150, 250). These timesteps were selected from the later steps of the reverse diffusion process, as prior studies [15] have shown that activations at these stages yield more discriminative semantic features, thereby enhancing pixel-level predictions. For each timestep, we collect feature maps from selected decoder blocks (specifically, blocks 5, 8, 12, and 17). The exact shapes of the outputs of these blocks (excluding skip connections) during forward pass are summarized in Table These feature maps are first resized (using bilinear interpolation) to the input resolution (256 x 256), and then concatenated channel-wise to form the final pixel-level representation.
Table 4: Output shapes of feature blocks extracted from the diffusion model. Highlighted rows indicate the selected blocks.
Block Index
Output Shape
Block Index
Output Shape
Block 0
[1024, 8, 8]
Block
[S12, 64, 64]
Block
[1024, 8, 8]
Block 10
[S12, 64, 64]
Block
[1024, 16, 16]
Block 11
[512, 128, 128]
Block
[1024, 16, 16]
Block
[256, 128, 128]
Block
[1024, 16, 16]
Block 13
[256, 128, 128]
Block 5
[1024, 32, 32]
Block 14
[256, 256, 256]
Block 6
[512, 32, 32]
Block 15
[256, 256, 256]
Block
[512, 32, 32]
Block 16
[256, 256, 256]
Block
[512, 64, 64]
Block
[256, 256, 256]
Concatenation Strategy. For each pixel, given T timesteps and L 4 selected blocks per timestep, we resize and concatenate the corresponding feature maps to obtain feature of shape:
TT x [channels from Block Block Block 12 + Block 17]. 256, 256)
For instance, if Block outputs 1024 channels, Block outputs 512 channels, and both Blocks 12 and 17 output 256 channels each, the final representation will have
3 (1024 + 512 + 256 + 256) = 3 x 2048 = 6144 channels.
B.3 Structure and Training of the Segmentation Head
multilayer perceptron (MLP) with hidden layers, incorporating ReLU activations and batch normalization layers, following the architecture used in [15). The classifier takes feature vector for each pixel input and outputs softmax probability distribution over the classes. Its parameters are updated using Adam optimizer with an initial learning rate of 1 x 10-3 and weight decay of 1 x 10-5, We apply cosine annealing learning rate scheduler with T and Minn 1 x 10-6
For pixel classification,
C Ablation Studies
C.1 Hyperparameter Sensitivity
We study how our two-stage pipeline responds to the key hyperparameters K (per-image representa- tives) and (global pool size) on CamVid. We vary
K E {20, 50, 1003, M {0.25MM. 0.4Mo, 0.5Mo),
where Mo N K is the size of the merged local pool (or initial global pool).
Figure shows that our default hyperparameter (K=100, M=0.5.M) yields 36. mloU, while the best configuration (K=20, M=0.4MM) achieves 38.70 mloU. This indicates that reducing K may degrade and can even improve performance when paired with suitably scaled global pool. Section C.8 discusses the resulting acmputeyaccuracy trade-off's in more detail.
21

---

mnlo (56
20
Local:
Local
Loodl
Local
10
Lood:
Lace:
10
Round
Round
Round
(a) Global: 0.25.M.
(b) Global: 0.4M,
Global: 0.5.M.
Figure 5: Hyperparameter sensitivity Vid mloU achieved after 10 AL rounds for varying per- image representative count K {20, 50, 100} and global pool size M E {0.25M., 0.4Mo, 0.5.M)-- Each subplot shows one M setting.
50
2-Stage eDALD
40-
mioU (%)
(%
miou 195
mioU (%)
30
Pooo
20
2
4
5
8
10
3
4
6
9
10
4 5 6 7
8
10
2 3 5
6
8
9 10
Round
Round
Round
Round
(a) Budget: O.IN
(b) Budget: 0.2N
(c) Budget: 0.5N
(d) Budget: IN
Figure 6: Results on Cam Vid with various budgets: O.IN, 0.2N, 0.5N, and 1N pixels per round.
C.2 Budget Sensitivity
plost the mloU of our two-stage eDALD e over AL rounds on CamVid under four annotation budgets: O, 0.2,N, 0.5.N, and 1N pixels per round (with 0.1.N z 0.0015% of all pixels). Under the smallest budget (O), two-stage cDALD converges to ~~ 35.57% mloU but improves more rapidly in early rounds. As the budget increases, the final (round 10) mloU of cDALD rises accordingly: 40.3% at 0.2,, 1% at 0.5N,, and 48.9% at L.V.
C.3
Convergence to Fully Supervised: Additional Details
This section expands the convergence analysis discussed in Section 4.3 Table reports, for each dataset, the number of rounds and labeled-pixel fractions required to reach 80% and 90% of the fully supervised mloU under budget b 0.1.N (iee,, average 0.1 pixel/image/round) We use two-stage eDALD for rounds 1 10 and switch to Margin thereafter.
Across datasets, two-stage eDALD (rounds 1-10) followed by Margin converges to strong perfor- mance with vanishingly few labeled pixels. Reaching 80% of fully supervised mloU takes only 11-27 rounds, corresponding to 0.0018 0.0043% of all pixels. For 90%, convergence occurs within 21-47 rounds, using merely 0.0034 0.0073% of pixels (+= one out 30k to 14k pixels). Dataset-
Table 5: Rounds and labeled-pixt pixel fraction (percentage all pixels) needed to reach 80% and 90% of the fully supervised mloU (FSI.) under the 0.1.N budget. We two-stage eDALD for rounds 1 10 and switch to Margin thereafter.
Target
Metric
CamVid
ADE-Bed
Cityscapes
Pascal-C
Rounds &Pixels
19
27 0.0043%
12
11 0.0018%
80% FSL
0.00319 %
0.0020%
Rounds %Pixels
32
47
28
21 0.0034%%
90% FSL
0.0050%
0.0073%
0.0044%
22

---

wise, Lascal-Contexx is fastest to 90% (21 rounds, 0.0034%), followed by Cityscapes (28, 0.0044%) and CanVit Vid (32, 0.0050%). while ADE-Bed is most demanding (47, 0.0073%), likely reflecting indoor-scene variability and pretraining mismatch. These results substantiate our switching policy: diversity -> uncertainty (two-stage) most sample-efficien under extreme budgets, and late switch to pure uncertainty (Margin) accelerates the final approach to the supervised ceiling. Overall, near- supervised quality is attainable at labeling rates that are orders of magnitude below 1%, reinforcing the practicality of our low-budget AL setting.
C.4 Additional Backbone Results
To compare DDPM-based two-stage eDALD with other architectures, we extend the two-stage framework to two widely used backhones, DeepLabV: V3 [52] and ViT (3). Since eDALD leverages the intrinsic stochasticity of diffusion models, employ its counterpart, eBALD, which estimates entropy-augmente mutual information via Monte Carlo dropout for non-diffusion backhones. The results in Table demonstrate that the DDPM-based approach consistently outperforms the others across all datasets, highlighting the strong synergy between diffusion-driven uncertainty estimation and MaxHerding under the low-budget regime.
We further present a ViT-specific comparison in Table comparing the two-stage eBALD with single-stage Margin and BalEntAcq. The two-stage approach consistently achieves the highest final-round mloU, demonstrating that entropy-augmenttd disagreement remains beneficial even for wansformer-based backbones.
Table 6: mloU (%) of two-stage methods under budget regime (10 rounds) with three backbones: DeepLabV3, VIT. and DDPM. DDPM uses 2-Stage eDALD, while DeepLabV3 and ViT use 2-Stage eBALD.
Backbone
Cam Vid
ADE-Bed
Cityscapes
Pascal-C
V3 52
29.48 .8.48
9.75
31.64
28.52
VIT 3
31.48
10.80
32.27
31.98
DDPM 13
36.12
31.12
33.34
47.98
Table 7: ViT-specific comparison of active learning methods in mloU (%) under the low-budget
regime.
Method
Cam Vid
ADE-Bed
Pascal-C Cityscapes
Margin
29.52
9.51
27.36
29.43
BalEntAcq
24.67
9.81
21.05
30.53
2-Stage eBALD
31.48
10.80
32.27
31.98
C.5 Comparison with Diversity-based Methods
To assess the effectiveness of different diversity-drive selection strategies, we compare pipelines using either MaxHerding [16] or Core-set (9] as the first-stage selector, followed by eDALD for uncertainty-baat based refinement. Core-set corresponds to the k-center greedy algorithm, which iteratively selects the farthest sample in feature space to ensure coverage. Table summarizes the final-round mloU across four datasets, comparing both one-stage and two-stage variants with two backbones: Deepl.ab .bb33 V3 and DDPM.
The results reveal two consistent trends. First, MaxHerding provides stronger coverage than Core-set across all configurations, whether used alone or within a two-stage pipeline. Second, this advantage consistent across both backbones but particularly pronounced for DDPM, where the combination MaxHerding - eDALD achieves the best overall performance. An exception is observed on Pascal- Context, where MaxHerding alone achieves higher performance than its two-stage variant. We attribute this to the dataset's ' large class count (33) and diverse scene distribution, where maximizing pure feature-space coverage can be more effective for capturing rare or spatially scattered classes. Overall, these findings demonstrate that while MaxHerding alone is strong representation-bane baseline, its combination with eDALD uncertainty consistently yields superior performance in most practical scenarios.
23

---

Table 8: Final-round mloU comparison of diversity methods (Core-set vs. MaxHerding) under the low-budget regime. For DeepLab we report two-stage variants with eBALD, while for DDPM we report both one-stage and two-stage with eDALD.
Backbone
Method
ADE-Bed
Cam Vid
Cityscapes
Pascal-C
Core-set
9.21
26,49
30.42
25.83
MaxHerding
9.64
28.78
31.05
28.12
DeepLabV3
52
Core-set eBALD
9.53
26.50
30.48
26.70
MaxHerding - eBALD
9.70
29.24
31.45
28.50
Core-set
18.49
16.19
22.74
23.27
MaxHerding
24.06
31,83
25.80
52.04
DDPM
13
Core-set - cDALD
26.70
32,95
32.20
44.58
MaxHerding - eDALD
31.12
36.12
33.34
47.98
C.6
Pixel-to-Region Expansion with SAM
We extend our two-stage eDALD pipeline to practical region-level supervision by integrating SAM (53) under striet click-parity budget (b=0.1N, ie.. average 0.1 pixel/imagg//ound)) Each selected pixel is used as one positive point prompt for SAM (no extra clicks and no iterative refine- ments). SAM returns multiple mask proposals with predicted IoU scores; we keep the highest-scoring proposal if it exceeds a confidence threshold 7 and otherwise fall back to the single-pixel label. Thus, each click can expand to a dense region label when reliable, yielding many more supervised pixels without increasing annotation cost.
Table reports 10-round results for the baseline two-stage eDALD (w/o SAM) and the SAM- augmented variant (w/ SAM). Across datasets, SAM consistently accelerates early rounds and often improves final mIoU under the same budget: CamVid the final gain is +5.24 pp (41.34 vs. 30), while ADE-Bed, Cityscapes, and Pascal-Context show modest but steady improvements (+0.48 pp. +-6.8 pp, and +0.27 pp. respectively). Overall, single-click, region-level supervision via SAM provides faster convergence and better cost-efficiency in the extreme low- low-budget setting.
Table 9; Active learning performance (mloU, %) over 10 rounds for two-stage eDALD with and without SAM under the b=0.1N budget (strict click parity: one positive point per selected pixel).
Dataset
Method
R1
R2
R3
R4
R5
R6
R7
R8
R9
R10
w/o SAM
17.90
18.63
24.52
27.00
28.40
27.40
29.50
32.90
34.60
36.10
CamVid
w/ SAM
20.51
28.68
32.03
28.74
31.93
34.78
37.75
36.74
39.56
41.34
w/o SAM
12.70
15.40
18.70
22.60
25.30
27.10
28.20
28.50
30.20
31.20
ADE-Bed
w/ SAM
14.00
16.37
19.94
24.13
29.11
29.58
30.34
31.49
32.43
31.68
w/o SAM
20.40
24.20
26.57
29.06
30.01
30.40
31.00
31.80
31.90
33.34
Cityscapes
w/ SAM
20.31
25.85
27.20
26.37
26.55
27.20
28.05
29.83
33.89
34.02
w/o SAM
25,80
29.75
36.61
39.89
41.42
43.43
44.33
45.98
46.24
47.44
Pascal-Context
w/ SAM
30.42
38.98
40.45
41.90
42.33
43.22
43.48
44.53
46.87
47.71
C.7 Decoder-Block Selection Ablation
We compare four decoder-block configurations for multi-scale feature extraction on ADE-Bed:
{2,3): deep blocks (high-dim features, low spatial resolution)
[1-117: : shallow blocks (low-dim features, high spatial resolution)
[5, 6, 7, 8, 12): original LEDM setting [15
85,8, 12, 17): our compact selection
Table reports mloU for each. The compact set (5, 8, 12, 17) (45.5%% matches the original five-
block configuration (46.41% 41%), while deep-only [2,3) (4.. 19%) and shallow-only (11-17} (22.68%) both degrade significantly. This confirms that retaining both spatial detail and high-level semantics is crucial. We therefore adopt (5, 8, 12, 17) in all experiments reduces feature dimensionality by 28% and speeds up without sacrificing accuracy.

---

Table 10: Ablation on decoder-block choice (ADE-Bed). Channel dim. is the sum over selected blocks.
Blocks
Channel Dim.
mloU (%)
12, 3) (deep-only)
6,144
42.19
111 17] (shallow-only)
6,144
22.68
(5, 6, 7,8, 12) (original)
8,448
46.41
{5, 8, 12, 17] (compact)
6,144
45.58
C.8 Computational Complexity
The dominant cost in our pipeline is candidate selection via MaxHerding. Given a global pool of size M with d-dimensional features and per-round budget B, forming the full RBF kernel costs d( time and ((M2) memory. The subsequent greedy selection over B points adds ((M²B) time, giving an overall complexity of O (M2 max((, (d, B}). Since M = K grows with the number of images N and the per-image candidate count K, this step can become the bottleneck on large datasets.
Runtime/memory in practice. On ADE-Bed, our two-stage eDALD with N K=50 (M=48, 200) required: feature extraction (8m 22s, 25.4 GB), local herding (4m 24s, 3.7 GB), and global herding (14m 27s, 38.1 GB), compared to training (19m 14s, 22.7 GB). Overall. MaxHerding accounts for 41% of the per-round wall-clock time, and global herding dominates peak memory GB).
Reducing peak memory. To further lower the memory footprint, adopt split-and-herd variant: partition the global pool into memory-safe splits, allocate a proportional sub-budget to each split, and run greedy herding sequentially while conditioning selections from earlier splits preserve cross-split coverage. This relaxes full optimality but prevents out-of- of-mmrrry under higher K or nigh-resolution settings, effectively reducing peak memory from (M²) roughly ((M/S)²) per split when using splits, with negligible impact on accuracy in our experiments.
D Qualitative Results
qualitatively compare some active learning selection results from the proposed two-stage eDALD against the single-stage Margin baseline each of four datasets.
D.1 Final Pixel Selections
Figure overlays the selected pixels on the ground-trutt -trtt maps for both methods. Margin tends to concentrate selections on boundary regions and often chooses redundant pixels within the same class region, resulting in less diverse annotations. contrast, our two-stage pipeline disperses pixels evenly across object edges, small classes, and scene context, ensuring that the most informative locations are annotated.
D.2 Selection Progression
Figure g visualizes the selected pixels of two-stage eDALD across rounds. Early rounds focus on broad structural cues; later rounds refine boundary regions and rare classes. This progression highlights how representation filtering first ensures coverage, then eDALD uncertainty hones in on the remaining ambiguous pixels.
D.3 Segmentation Outputs
For representative images, Figure 9 shows the predicted segmentation maps. Margin often misses thin structures and small objects (e.g., distant pedestrians, traffic signs), producing fragmented or smoothed regions. In contrast, our eDALD-trane --trineed model yields cleaner boundaries and recovers fine details more faithfully.
25

---

Image
Baseline
(c) Ours
Image
Bascline
(c) Ours
CamVid
Cityscapss
Pascal- Context
ADE-Bed
Figure 7: Final pixel selections overlaid on ground truth: (a) input image; (b) Margin selections; (c) two-stage eDALD selections. Light blue dots mark chosen pixels.
Round
Round 5
Round 10
Round
Round 5
Round 10
Vid
Cityscapes
Pascal- Context
ADE-
Figure Two-stage eDALD pixel selections at rounds 1, 5, and 10 on multiple example images.
26

---

Cam Vid
Cityscapes
Pascal Context
(a) Image
(b) Groundtruth
(c) Baseline
(d) Ours
Figure 9: Example segmentation results. Columns show (a) input image, (b) ground truth, (c) Margin prediction, and (d) 2-stage eDALD prediction.
27


Reviews:
[{"review_id": "fmEjyTyADz", "reviewer": "Reviewer_gmyd", "summary": "This paper addresses the challenge of low-budget active learning for semantic segmentation, where only a tiny fraction of pixels can be labeled per round. It introduces a two-stage selection pipeline that leverages a pre-trained diffusion model to obtain rich multi-scale pixel representations.\n\nStage 1 (Representation Filtering) uses MaxHerding to pick a small set of diverse candidate pixels per image, then refines these across the dataset to form a global pool.\n\nStage 2 (Uncertainty Scoring) computes an entropy-augmented disagreement score (eDALD) over noisy diffusion features-combining mutual information (BALD-style) with a single-sample entropy term-to select the most informative pixels.\n\nExtensive experiments on CamVid, ADE-Bed, Cityscapes, and Pascal-Context under extreme pixel-budget regimes demonstrate substantial mIoU gains (e.g., +10.94 pp over DALD alone) and outperform prior pixel-level baselines by margins up to 21 mIoU points.", "questions": "1. Could you report standard-error bars (over multiple random seeds) for mIoU to confirm that the observed gains are statistically significant?\n\n2. What are the wall-clock times and memory costs for diffusion-feature extraction and MaxHerding on large datasets?\n\n3. How sensitive is performance to choices of K (candidates per image), M (global pool size), and the number of diffusion timesteps T? Some crucial ablation studies are missing.\n\n4. Can the two-stage eDALD approach be applied with non-diffusion backbones (e.g., ViT, CNN) by replacing diffusion features?\n\n5. Since segmentation quality often hinges on boundary accuracy, have you analyzed whether selected pixels concentrate on object edges or uniform regions, and how this affects final segmentation?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Quality\n\n**Strength**: solid methodological design combining diversity and uncertainty; comprehensive experimental evaluation across four benchmarks.\n\n**Weakness**: no statistical significance (e.g., error bars or multiple seeds) reported for mIoU improvements; computational cost of extracting multi-scale diffusion features and MaxHerding at scale is not analyzed.\n\n# Clarity\n\n**Strength**: clear exposition of the two-stage algorithm with pseudocode (Alg. 1) and illustrative figures; well-structured related work and motivation.\n\n**Weakness**: some implementation details (e.g., choice of kernel bandwidth $\\sigma$ in RBF, hyperparameters for MaxHerding) are only briefly mentioned; description of how batch formation for adapted baselines is carried out could be clearer.\n\n# Significance\n\n**Strength**: practical framework for extremely low-budget pixel-level active learning in segmentation; demonstrates consistent, large gains.\n\n**Weakness**: strict reliance on a diffusion backbone limits applicability to settings where such models are unavailable.\n\n# Originality\n\n**Strength**: novel integration of diffusion-model features into active learning; introduction of entropy-augmented DALD (eDALD).\n\n**Weakness**: the two-stage idea echoes previous coverage-then-uncertainty approaches; the main novelty is the use of diffusion features rather than core AL strategy.", "was_revised": true, "final_justification": "The detailed experiment results shown in the rebuttal have resolved most of my concerns. I decide to change the rating to 4: borderline accept.", "rebuttal": "Thank you for your thorough review and insightful feedback. We appreciate the time and effort you invested in evaluating our work and your constructive suggestions. If our paper is accepted, we will gladly incorporate all of your recommendations to improve clarity, rigor, and impact.\n\n---\n## 1. Statiscal Significance\nAs suggested, we report mean $\\pm$ standard deviation over three seeds to reflect the statistical significance of our mIoU improvements. The results show MaxHerding yields consistent gains across uncertainty measures.\n\nThe table below shows two key effects on CamVid: (1) how our DALD objective improves performance over BALD, and (2) how MaxHerding further boosts performance over uncertainty-based (UC) acquisition alone. Gains are reported in absolute points and percentages. Similar improvements are observed from DALD and MaxHerding as in single-seed results.\n\n|Uncertainty|UC Only|Herding → UC|Gain (pp)|Gain (%)|\n|-|-:|-:|-:|-:|\n|Entropy|25.3±0.4|30.8±0.4|5.5|21.8|\n|Margin Sampling|30.7±0.9|32.8±0.8|2.1|6.6|\n|BALD|24.6±1.0|22.8±0.9|−1.8|−7.3|\n|DALD|23.8±3.6|21.1±1.0|−2.7|−11.6|\n|PowerBALD|24.3±5.8|31.6±0.8|7.3|29.9|\n|PowerDALD|31.3±1.2|32.0±0.7|0.7|2.2|\n|Entropy + BALD|26.0±1.3|32.1±0.4|6.1|23.7|\n|Entropy + DALD|25.1±0.6|35.7±0.2|10.6|42.1|\n\nThe next table reports mIoU (%) under the same low-budget regime (10 rounds). It shows similar trends to those reported in Table 2 of the manuscript: the two-stage eDALD consistently outperforms all baselines across multiple datasets.\n\n|Backbone|Method|CamVid|ADE‐Bed|Cityscapes|Pascal‐C|\n|-|-|-:|-:|-:|-:|\n|DeepLabV3|PixelPick|29.9±0.1|8.4±0.4|26.8±0.1|26.3±0.1|\n|DeepLabV3|Didari et al.|22.5±0.1|8.7±0.5|19.6±0.1|28.2±0.1|\n|DDPM|Random|25.9±1.2|17.8±0.6|27.1±1.4|41.7±2.1|\n|DDPM|Entropy|25.3±0.4|23.0±1.6|28.6±1.1|42.1±2.0|\n|DDPM|Margin|30.7±0.9|30.3±0.3|32.2±1.2|43.1±0.4|\n|DDPM|BalEntAcq|19.3±1.1|17.4±1.3|24.0±2.0|33.1±4.1|\n|DDPM|eDALD|25.1±0.5|23.0±1.2|29.4±1.3|43.1±0.1|\n|DDPM|2-stage eDALD|35.7±0.2|30.1±0.7|33.3±2.4|47.4±1.1|\n\n---\n\n## 2. Computational Cost\nBelow is a detailed breakdown of per-round wall-clock time and peak GPU memory for each method on Cityscapes (N=2,975, K=15 ⇒ M=44,625), measured on a single NVIDIA A100 40 GB.\n\n|Method|Local Herding Time|Global Herding Time|Feature Extraction Time|Training Time|Total Time|\n|-|:-:|:-:|:-:|:-:|:-:|\n|2-stage eDALD|10m 17s|1m 49s|36m 43s|30m 14s|79m 3s|\n|2-stage Margin|12m 20s|1m 44s|7m 28s|26m 54s|48m 26s|\n|Didari et al. (diff.)|–|–|39m 01s|33m 28s|72m 29s|\n|Margin (diff.)|–|–|8m 02s|24m 59s|33m 01s|\n|Didari et al. (DeepLab)|–|–|1m 14s|9m 10s|10m 24s|\n|Margin (DeepLab)|–|–|1m 07s|8m 45s|9m 52s|\n\n|Method|Local Herding Mem|Global Herding Mem|Feature Extraction Mem|Training Mem|Peak Mem|\n|-|:-:|:-:|:-:|:-:|:-:|\n|2-stage eDALD|6.6 GB|33.5 GB|26.6 GB|21.0 GB|33.5 GB|\n|2-stage Margin|6.6 GB|33.5 GB|25.9 GB|22.4 GB|33.5 GB|\n|Didari (diff.)|–|–|24.9 GB|21.4 GB|24.9 GB|\n|Margin (diff.)|–|–|25.8 GB|22.6 GB|25.8 GB|\n|Didari (DeepLab)|–|–|1.65 GB|1.80 GB|1.80 GB|\n|Margin (DeepLab)|–|–|1.55 GB|1.76 GB|1.76 GB|\n\nOn diffusion backbones, the two-stage pipeline takes ~80 minutes per round, with feature extraction and training each taking about half that time, and herding ~25%. MaxHerding adds 12–14 minutes (≈20–25%), which remains manageable within the overall cycle.\n\nMemory usage is dominated by global herding over 6,144-dim diffusion features, peaking at 33.5 GB; feature extraction also demands over 25 GB. In contrast, DeepLab-based methods finish in under 10 minutes per round and stay below 2 GB, illustrating a clear efficiency–accuracy trade-off.\n\nOverall, while diffusion backbones are more demanding, the herding overhead is modest, making the method practical for large-scale data given adequate GPU resources.\n\n---\n## 3. Implementation Details and Hyperparameter Sensitivity\n\n**1. RBF bandwidth $\\sigma$**\\\nWe set the Gaussian‐kernel bandwidth $\\sigma$ via the **median heuristic**: subsample 1% of candidate features, compute all pairwise distances, and take their median. To test robustness, we also evaluated the 30th, 40th, 60th, and 70th percentiles of the same distance distribution. Across all five settings, **final‐round mIoU varied by $\\le0.7$ pp**, with the median choice giving the best performance. This confirms that our method is **insensitive** to the exact $\\sigma$ and that the median heuristic is a **reliable, data‐driven** default.\n\n|σ|R1|R2|R3|R4|R5|R6|R7|R8|R9|R10|\n|-|-|-|-|-|-|-|-|-|-|-|\n|median|13.1|14.3|19.8|25.7|29.3|30.2|31.8|34.5|35.5|35.9|\n|30%|14.5|16.1|22.1|23.7|26.3|30.9|31.1|32.6|32.8|35.2|\n|40%|11.6|19.5|18.0|23.0|26.1|25.9|27.7|29.3|34.1|34.7|\n|60%|11.3|15.3|17.2|23.0|26.3|32.0|33.1|32.4|31.6|35.0|\n|70%|11.0|12.6|18.6|26.0|27.6|27.8|29.6|31.5|32.8|32.5|\n\n**2. MaxHerding hyperparameters ($K$, $M$)**\\\nIn Section C.2 of the supplementary, we report an ablation on **per‐image candidates** $K \\in \\lbrace 20,50,100 \\rbrace$ and **global pool size** $M \\in \\lbrace 0.25M_0, 0.4M_0, 0.5M_0 \\rbrace$, where $M_0 = N \\times K$. On CamVid, final mIoU fluctuates by at most **2.8 pp** across all nine configurations, and our default ($K=100$, $M=0.5M_0$) lies near the top. This demonstrates that our two‐stage pipeline is **not overly sensitive** to these choices, and that smaller $K$ or $M$ can even **improve** performance in some cases — helping trade off compute versus accuracy.\n\n**3. Number of diffusion timesteps $T$**\\\nFollowing LEDM [15], we use $T=${50,150,250} by default. To isolate the effect of $T$, we ran eDALD on CamVid with $T=${50}, {50,150}, and {50,150,250}.\n\n|$T$|R1|R2|R3|R4|R5|R6|R7|R8|R9|R10|\n|-|-|-|-|-|-|-|-|-|-|-|\n|{50}|12.7|8.8|23.2|23.1|24.7|27.0|27.3|31.2|30.6|30.5|\n|{50,150}|12.3|17.3|21.3|28.2|29.8|31.6|33.8|33.8|35.3|36.7|\n|{50,150,250}|15.6|14.8|22.5|24.0|25.8|27.3|32.3|34.9|35.7|37.1|\n\nThe final‐round mIoUs were:\n- **{50}**: 30.5\n- **{50,150}**: 36.7\n- **{50,150,250}**: 37.1\n\nAdding each extra timestep **increases mIoU** by 6—7 pp, confirming that our three‐step combination best balances **feature richness** versus **dimensionality cost**. While each additional step linearly raises feature dimension, the resulting gains in query diversity and classifier learning **justify** the modest extra memory and compute.\n\n---\n## 4. Applicability Beyond Diffusion Backbones\nWe acknowledge that parts of our framework — particularly **eDALD** uncertainty measure — are specific to diffusion models. The DALD term estimates mutual information from the stochastic denoising process unique to diffusion backbones. We introduced it to better exploit their inherent stochasticity and capture epistemic uncertainty. However, the **two‐stage sampling pipeline** (representation‐first $\\to$ uncertainty‐second) is entirely backbone‐agnostic.\n\nTo validate this, we applied the same two‐stage flow to **DeepLabV3** by replacing eDALD with **eBALD**, where mutual information is estimated via MC‐dropout.  We then compared 1‐stage vs. 2‐stage eBALD on both DeepLabV3 and our diffusion model:\n\n|Backbone|Method|CamVid|Cityscapes|\n|-|-|-|-|\n|DeepLabV3|1-stage eBALD|27.8|29.6|\n|DeepLabV3|2-stage eBALD|29.5|31.6|\n|Diffusion|1-stage eBALD|26.0|28.6|\n|Diffusion|2-stage eBALD|32.1|33.3|\n\n- **Representation stage (MaxHerding)** improves performance on both backbones (gains of 1.7–6.1 pp), confirming its generality.\n- While diffusion yields the best results due to richer self-supervised features, the DeepLabV3 + 2-stage pipeline still outperforms its 1-stage variant by ~1.8–2.1 pp, especially on Cityscapes.\n\nIn conclusion, any feature extractor (ViT, CNN, etc.) can slot into our two‐stage framework: swap in an uncertainty estimator suited to that backbone (e.g., MC‐dropout BALD for CNNs) for Stage 2, and retain MaxHerding for Stage 1.\n\n---\n## 5. Novelty of Our Method\nOur focus has been on building an effective yet efficient framework that incorporates both coverage and uncertainty through diffusion-based features, under realistic resource limits.\n\nThank you for the thoughtful comment. We are aware of hybrid active learning methods such as Weighted $k$-means [1], ALFA-Mix [2], and BADGE [3], which combine coverage and uncertainty. However, to our knowledge, none adopt a two-stage coverage-then-uncertainty strategy tailored to neural network-based active learning. If the reviewer knows of such a method, we would greatly appreciate the reference and would be happy to compare.\n\nWe excluded the above methods due to computational constraints. Specifically, [1] and [3] perform $k$-means clustering over all dataset pixels, which is impractical even with multiple GPUs. ALFA-Mix [2] requires pixel-wise similarity computations, which are also costly at scale. In general, many hybrid methods are hard to scale to pixel-level tasks like semantic segmentation.\n\nOur focus, therefore, has been on building an effective yet efficient framework that incorporates both coverage and uncertainty through diffusion-based features, under realistic resource limits.\n\n[1] Zhdanov et al., \"Diverse mini-batch active learning\", arXiv:1901.05954, 2019.\\\n[2] Parvaneh et al., \"Active Learning by Feature Mixing\", CVPR 2022.\\\n[3] Ash et al., \"Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds\", ICLR 2020.\n\n---\n## 6. Pixel Selection\nThank you for pointing this out. We qualitatively analyzed the spatial distribution of selected pixels across multiple benchmarks; see Fig. 3 and 8.\n\nThe baseline method (Margin) often focuses on object boundaries and selects redundant pixels within the same class region, limiting structural and semantic diversity.\n\nIn contrast, our two-stage method first uses MaxHerding to ensure diverse coverage in feature space, then selects epistemically uncertain pixels via eDALD.\nThis results in broader spatial and semantic coverage, providing richer supervision signals.\nThis improved diversity aligns with our quantitative results, where our method consistently achieves more stable and higher mIoU across datasets."}, {"review_id": "eXc3tUFiVY", "reviewer": "Reviewer_xSso", "summary": "This article studies active learning for semantic segmentation using features from a diffusion model to select pixels of an especially high content of information. In a first step, the authors let a diffusion model noise the images in the data set. From this, the inner features of the score vector field  defining the drift in the diffusion process are read out at the respective position of the noised image. As this is realized with a fully convolutional architecture, this is possible. Using these features, a classifier network is trained on the data set of pixels that have been uncovered so far. This classifier is then used to select the point that improves the coverage of the unlabeled features (in the sense of maximum similarity in feature space) the most. (representation based selection) or the point that improves the mutual information between the label of the new point and its feature vector the most. This expression can be approximately evaluated using the classifier, see eq. (5)--(7). This is first done via  a preselection of pixels within one image followed by second round of competition between all selected pixels. From the 2nd competition, the number of pixels is chosen that corresponds to the round budget. Thereafter the model is retrained and a second, third... round is started until the procedure finally is terminated.\nThe authors provide numerical tests that operate on an extremely low budget of pixels, in average 1px per image in total over 10 rounds. They compare their two methods with simple and more elaborate baselines and find significant improvement over naive baselines and limited but consistent improvements over more elaborate baselines (mesured in mIoU). The tests are repeated fora number of data sets.", "questions": "Q1: AL is often evaluated at budget for 95% performance of the model trained on the full data set. Would this AL procedure be able to go there?\nQ2: Did you test the assumption that Y is conditionally independent of X given Z, e.g. by a classification experiment with and without X and a likelihood ratio test? (see lines 201->202) What consequences do you see for (5)-(7) if the effect of X is significant?\nQ3: What is the outcome of a click-cost based comparison of your methods with region based methods?\nQ4: in l. 152 - on which data sets is the frozen diffusion model trained - are these disjoint from your AL set?\nQ5: l. 224 Why there is no weighting factor between I and H?", "limitations": "None", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 1, "originality": 3, "strengths_and_weaknesses": "Strength\n* The paper impresses with a method that operates on an extremely low pixel budget of in average 1px/image\n* To invoke diffusion features for AL is an original idea. \n* The authors consistently gain performance over baselines\n* The paper is well written and carefully prepared\n\nWeaknesses\n* The technical relevance of this paper is somewhat limited, as segmentation click-work  labels are produced by polytopes, not by labeling single, where each edge of a polytope affords one click plus one click for the class label. These clicks nowadays are significantly reduced using models like SAM, where the polytope contours are suggested and only need to be slightly improved. Hence a practical comparison would be on click basis, here one pixel would only require one class click. It would have been of interest to see click based comparisons with non pixel based AL for SemSeg methods as well.\n* The mIoUs achieved <= 40% seem to be too low for most practical purposes.        \n\nMinor remarks:\n*l. 68: which outliers,describe precisely.\n* l. 158 the set notation at the end of the line is inprecise\n* l. 185 is M the same as b?\n* Table 1: mention the interpretation of numbers (mIoU?!) in the caption.", "was_revised": true, "final_justification": "While the paper contains some interesting insight on training semantic segmentation with extremely low amounts of information, the practical relevance of pixel wise labeling is limited.", "rebuttal": "Thank you for your thoughtful review and insightful comments. We appreciate the time you took to evaluate our work and the constructive suggestions you provided. We have addressed each of your concerns in detail below and will incorporate all accepted changes into the final version.\n\n---\n## 1. Click‐Cost and Compatibility with Region-Based Labeling\nThank you for the insightful suggestion on click-cost comparisons. While region-level AL can annotate a large area with a single click, it introduces two key challenges:\n\n1. **Annotation ambiguity.**  Automatically-generated regions (e.g., superpixels or SAM masks) may span multiple objects or miss thin structures, making class labels ambiguous and reducing annotation quality. \n2. **Inference overhead.**  Generating accurate regions — whether via SAM prompts or superpixel algorithms — requires additional model inference or preprocessing steps, which can outweigh the savings from fewer clicks.\n\nOur two-stage sampler (MaxHerding → eDALD) selects $B$ pixel seeds per round that are both diverse and epistemically uncertain.  When these seeds are fed into any region-proposal tool, they act as high-value “anchor clicks,” guiding the tool to produce masks that align more precisely to object boundaries and small classes.  In practice, this hybrid workflow can:\n\n- **Reduce ambiguity** by seeding proposals across the entire scene manifold — avoiding over-concentration on a few high-uncertainty patches. \n- **Improve mask fidelity** for thin structures and adjacent objects, since seed diversity and uncertainty refinement correct common region-proposal failures. \n- **Leverage future advances** in region-proposal quality — our pixel-level acquisition serves as a drop-in enhancement for any click-based model, yielding greater synergy as region-segmentation backbones improve.\n\nWe view this integration as a promising direction: it maintains the low click-cost of region-based AL while delivering more precise, unambiguous annotations under extreme budgets.\n\n---\n## 2. Low Absolute mIoU\nThank you for raising this important point. To the best of our knowledge, **we are the first** to study active learning for semantic segmentation in an **extreme low-budget** regime — labeling only $0.1\\times N$ pixels per round for 10 rounds (total budget $=1\\times N$) — a setup that brings annotation cost down to truly practical levels. Under these constraints, even basic uncertainty sampling (e.g., BALD, Entropy) yields unusable models, whereas our two-stage pipeline (MaxHerding → eDALD) recovers a substantial fraction of full-data performance.\n\nFor example, on ADE-Bed the fully supervised mIoU is $45.58\\%$ (Appendix C.4), while our two-stage eDALD achieves $35.68\\%$, closing nearly a $10\\%$ gap with only **one labeled pixel per image**. This demonstrates that — in contrast to prior AL work using per-round budgets of $5\\times N$ or more — we can achieve high-quality segmentation with an order-of-magnitude fewer annotations.\n\nMoreover, our framework naturally supports an **adaptive querying** transition:\n- **Early rounds:** enforce diversity + uncertainty to build a broad, informative seed set. \n- **Later rounds:** shift focus to rare classes or boundary refinement to push toward higher mIoU.\n\nFollowing the spirit of SelectAL [1], one could define a budget threshold at which to switch from low- to high-budget acquisition strategies, ultimately approaching the 95\\% recovery mark as labeling resources grow. We will clarify this pathway in the revision.\n\n[1] O. Hacohen and D. Weinshall, “How to select which active learning strategy is best suited for your specific problem and budget,” *NeurIPS* 36, 2023. \n\n---\n## 3. Theoretical Assumption on $Y \\perp X \\mid Z$\nThank you for the constructive feedback. As noted in Shah and Peters [1], however, conditional independence testing is known to be extremely challenging to perform reliably. More importantly, our assumption of $X \\rightarrow Z \\rightarrow Y$ aligns with a common modeling choice in probabilistic graphical models, where $X$ is an input variable, $Z$ represents latent or feature variables, and $Y$ is a label variable. Here, the stochasticity in $Z$ arises either from noise in the input or from the model itself e.g. dropout. A wide range of neural networks with inherent stochastic behavior is based on this assumption. \n\nWhile it is possible to test for the additional direct effect of $X$ on $𝑌$ given $Z$, for instance, by incorporating a parallel deterministic model to represent $X \\rightarrow Y$, such an ensemble structure tends to yield only marginal improvements in practice. This is especially true when the base model (in our case, UNet) is already large, as the benefits of ensembling tend to diminish with increasing model capacity.\n\nTherefore, given the capacity of modern large models including the one used in our work, it is reasonable to assume the causal relation $X \\rightarrow Z \\rightarrow Y$. However, we acknowledge that rigorously validating this assumption is extremely challenging in practice. If the reviewer is aware of a principled method for doing so, we would greatly appreciate the suggestion and are willing to incorporate it into our analysis.\n\n[1] Rajen D Shah, Jonas Peters, The Hardness of Conditional Independence Testing and the Generalised Covariance Measure.\\\n[2] Taiga Abe, E. Kelly Buchanan, Geoff Pleiss, Richard Zemel, Deep Ensembles Work, But Are They Necessary?, NeurIPS 2022.\n\n---\n## 4. Pretraining Data for Diffusion Models\nThese pretraining details are provided in Appendix B.1. In summary, the diffusion backbone for ADE-Bed was trained on the LSUN-Bedroom dataset, and for CamVid, Cityscapes, and Pascal-Context we used a class-unconditional ImageNet-256 diffusion model. Crucially, none of these pretraining datasets overlap with our active-learning benchmarks, so all reported gains come from true out-of-domain feature transfer.\n\n---\n## 5. Weighting Between $I$ and $H$\nWe chose equal weights ($\\alpha=1$, $\\beta=1$) to avoid introducing extra hyperparameters and to keep the balance between the mutual‐information term $I(Y;Z\\mid x)$ and the confidence term $H(Y\\mid z^{(0)},x)$. To validate this choice, we ran an ablation on CamVid (all other settings fixed) with several $(\\alpha,\\beta)$ pairs:\n\n|$\\alpha$|$\\beta$|R1|R2|R3|R4|R5|R6|R7|R8|R9|R10|\n|-:|-:|-:|-:|-:|-:|-:|-:|-:|-:|-:|-:|\n|1.0|1.0|12.81|23.04|18.99|23.00|25.11|32.95|32.23|35.30|33.95|36.14|\n|0.2|1.0|12.81|14.04|18.07|21.18|22.20|27.83|28.36|28.86|33.76|32.65|\n|1.0|0.2|9.69|13.38|19.27|23.27|25.23|27.02|26.89|31.22|32.72|31.94|\n|0.05|4.0|13.24|14.91|18.84|20.34|21.86|19.70|19.44|23.04|19.73|18.66|\n|0.05|2.0|13.24|15.20|20.18|18.98|21.08|23.11|23.70|23.54|24.12|25.71|\n|0.02|2.0|13.24|16.49|16.19|16.83|19.38|19.27|18.22|20.86|18.88|20.37|\n\nThe equal‐weights setting ($1,1$) achieves the highest final‐round mIoU (36.14), confirming that our default choice is both simple and effective without requiring dataset‐specific tuning. \n\n---\n## 6. Minor Remarks\nThank you for these helpful suggestions. We will incorporate the following clarifications in the revision:\n\n- **Line 68 (“outliers”)** \n  We will replace “outliers” with a precise description: pixels whose predicted class probabilities are extreme or erratic — e.g., isolated noise artifacts or very rare patterns — resulting in high uncertainty yet lying far from the true decision boundary and contributing little to model improvement.\n\n- **Line 158 (set notation)** \n  We agree that the notation can be tightened. We will clarify that \n  > “Let $I_{w,h}$ denote the pixel at coordinates $(w,h)$ in image $I$. Then $\\lbrace I_{w,h} \\mid 1\\le w\\le W, 1\\le h\\le H\\rbrace$ enumerates all pixels of $I$, and writing $x \\in \\lbrace I_{w,h}\\mid\\cdots\\rbrace$ simply means that $x$ is one such pixel.”\n\n- **Line 185 ($M$ vs. $b$)** \n  We will explicitly note that \n  > “$M$ is the size of the global candidate pool produced by MaxHerding in Stage 1, whereas $b$ is the per-round labeling budget in Stage 2; they serve different roles and generally take different values.”\n\n- **Table 1 (interpretation of numbers)** \n  We will amend the caption to read: \n  > “Table 1: Mean Intersection-over-Union (mIoU, %) of single-stage vs.\\ two-stage selection on CamVid. ‘UC Only’ is one-stage uncertainty; ‘Herding → UC’ adds representation filtering.”"}, {"review_id": "pIz5y90Yuz", "reviewer": "Reviewer_V27y", "summary": "The paper proposes a novel two-stage diffusion-based pixel-level active learning method for semantic segmentation.", "questions": "- What is the overall performance of MaxHerding for the ablation study?\n- Can you explain why DALD performs so poorly, but eDALD (just adding an entropy term) performs so much better?\n- Is the diffusion-based approach better than other approaches? Please provide comparisons with different backbones for entropy-based methods and results on diversity-based methods.\n- How does the novelty of your approach compare against other diffusion-based AL approaches?\n\nAddressing these concerns would move my score to borderline accept.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- Overall, a well-motivated diffusion-based active learning method\n- Overall, good metrics, ablation results, and curves \n- The paper is well written and easy to follow\n\nWeaknesses:\n- What is the practical benefit of pixel-level active learning? The cost of having a human annotate particular pixels is potentially higher than region-level methods.\n- The diffusion-based approach seems to be dependent on having a diffusion model trained on imaging similar to that which you would want to segment. Thus, if your dataset lacks a relevant diffusion model, your approach would not be feasible. \n- It would be nice to see what the overall performance of MaxHerding is in the ablation study as a baseline\n- It is very weird to me that DALD performs so poorly but just adding Entropy to it produces the best result\n- It seems like all the entropy-based methods are utilizing the diffusion model backbone? This assumes that the diffusion model-based approach is better than other approaches. It would be good to see the baseline algorithms with other backbones, at least compared to the novel diffusion model approach. Also it seems like there are no diversity-based algorithms compared against, like Coreset.\n- There is no comparison against other diffusion model-based active learning approaches.", "was_revised": true, "final_justification": "Most of my previous concerns have been addressed. The larger impact of the work remains unclear, which is why I can't raise the score any higher.", "rebuttal": "We thank the reviewer for thoughtful feedback and constructive comments. We will incorporate these suggestions in the revised manuscript to strengthen our presentation and clarify the practical benefits of our approach.\n\n---\n## 1. Practical Benefit of Pixel‐Level AL\nThank you for raising this important point. Region-level annotation can indeed offer click-efficiency — one click can label an entire superpixel or prompt‐generated region. However, it also introduces two key challenges:\n\n1. **Semantic Ambiguity:** Precomputed regions (e.g., superpixels) can span multiple object classes or mix foreground and background. Fixing such errors often requires extra clicks, diminishing the “one-click” advantage.\n2. **Segmentation Overhead:** Generating high-quality regions (e.g., via SAM or oversegmentation) adds compute and latency costs that may be nontrivial in practice.\n\nImportantly, our two‐stage pixel‐level strategy is fully compatible with region‐based interfaces. Once informative pixels are selected:\n\n- **Superpixel Aggregation:** Selected pixels can be grouped into regions and labeled with a single click.\n- **Prompt‐Based Propagation:** Selected pixels can serve as prompts for models like SAM, automatically expanding to a full mask after one or two clicks.\n\nThis hybrid setup combines our method’s *selection benefit* (via diverse and uncertain pixels) with the *labeling efficiency* of region-based tools, enabling a highly click-efficient pipeline. We see this integration as a promising direction and will highlight it in the revised manuscript.\n\n---\n## 2. Dependence on Diffusion Pretraining Data\nThank you for this observation. We agree that closer domain alignment can further improve feature relevance, but it is not strictly required for our method to be effective:\n\n- **General‐Purpose Backbone:** For CamVid, Cityscapes and Pascal-C, we employed a single DDPM model pretrained on ImageNet-256 (see Appendix B.1). Despite being street-scene datasets, the ImageNet-leveraged features still delivered consistent gains over all uncertainty and two‐stage baselines.\n- **Domain‐Specialized Backbone (ADE-Bed):** ADE-Bed consists of indoor bedroom scenes, which differ substantially from ImageNet. Here, we used a DDPM pretrained on LSUN-Bedroom, achieving the best performance — suggesting that in highly specialized domains, a closer-aligned model can help.\n\nOverall, our results show that **loosely aligned** diffusion features (ImageNet-256) suffice to outperform prior pixel‐level AL methods across benchmarks. In cases of major domain shift, switching to a domain-specific model (e.g., LSUN-Bedroom) is a simple and effective plug-in with no changes to our two-stage pipeline.\n\n---\n## 3. MaxHerding‐Only Performance\nWe agree that showing MaxHerding as a standalone 1-stage baseline is important. Below we present two tables:\n\n**1. Round-by-Round Progression**\\\nMean mIoU over 10 AL rounds using **MaxHerding only** (no uncertainty):\n|Dataset|R1|R2|R3|R4|R5|R6|R7|R8|R9|R10|\n|-|-|-|-|-|-|-|-|-|-|-|\n|CamVid|12.82|21.78|23.67|23.77|28.07|27.13|29.97|26.77| 30.41|31.83|\n|ADE-Bed|8.40|12.10|17.60|17.77|17.51|18.99|21.21|20.98| 21.78|24.06|\n|Cityscapes|18.39|20.78|22.25|22.60|22.42|23.49|24.30| 23.64|23.34|25.80|\n|Pascal-C|14.63|30.67|29.87|37.67|39.69|44.56|48.53|45.32|51.99|52.04|\n\nEven without any uncertainty measure, MaxHerding’s coverage yields steady mIoU gains as more pixels are labeled.\n\n**2. Final mIoU Comparison**\\\nFinal (round 10) mIoU of three approaches:\n|Dataset|MaxHerding only|eDALD only|MaxHerding $\\to$ eDALD|\n|-|:-:|:-:|:-:|\n|ADE-Bed|24.06|23.06|30.18|\n|CamVid|31.83|25.14|35.65|\n|Cityscapes|25.80|29.44|33.34|\n|Pascal-C|52.04|43.05|47.44|\n\nOn ADE-Bed, CamVid and Cityscapes, our 2-stage eDALD substantially outperforms both 1-stage baselines, confirming the benefit of combining diversity and uncertainty. On Pascal-Context — where scenes are extremely diverse and there are 33 classes — MaxHerding alone slightly surpasses our full pipeline. We believe that in such settings, pure feature-space coverage can better capture rare or widely distributed classes without the noise introduced by uncertainty estimates.\n\nThese tables demonstrate that while MaxHerding alone is a strong representation-based baseline, adding eDALD uncertainty consistently boosts performance in most practical scenarios. \n\n---\n## 4. Method Comparison: Diversity-Based Strategies and Backbones\n\n\n**1 Diversity-based Algorithm Comparison**\\\nTo validate that MaxHerding is not merely “reinventing” CoreSet in diffusion feature space, we plugged both diversity modules into our two‐stage pipeline (diversity $\\to$ eDALD) under a fixed low‐budget ($b=0.1N$ pixels per round) and measured round‐by‐round mIoU on four benchmarks (we provided results on only two datasets due to space limit).\n\n**CamVid (2‐stage, $b=0.1N$)** \n|Method|R1|R2|R3|R4|R5|R6|R7|R8|R9|R10|\n|-|-|-|-|-|-|-|-|-|-|-|\n|CoreSet|12.81|15.95|17.85|17.39|16.92|17.98|18.17|17.43| 19.66|16.19|\n|MaxHerding|12.81|20.33|19.64|25.48|26.95|30.18|29.84|32.24|34.91|35.37|\n\n**ADE-Bed (2‐stage, $b=0.1N$)** \n|Method|R1|R2|R3|R4|R5|R6|R7|R8|R9|R10|\n|-|-|-|-|-|-|-|-|-|-|-|\n|CoreSet|7.54|9.62|10.68|12.36|14.34|15.64|16.64|16.41|17.66|18.49|\n|MaxHerding|10.37|16.20|18.71|22.41|25.81|26.12|28.30|29.77|29.83|30.49|\n\nAcross all four datasets, **MaxHerding** yields substantially higher mIoU — especially in later rounds (R5–R10) — while **CoreSet** often saturates or even degrades. This confirms that MaxHerding provides stronger structural coverage in the diffusion‐feature space and better primes the second‐stage uncertainty selection. These results demonstrate that our diversity module (MaxHerding) is a competitive and effective alternative to existing diversity-based methods like CoreSet, particularly in the context of diffusion feature space and low-budget active learning.\n\n**2 Backbone‐Agnostic Two‐Stage Sampling**\\\nOur two-stage sampling strategy — first covering the data distribution, then selecting uncertain pixels — is backbone-agnostic and applicable to various architectures. However, the eDALD uncertainty measure is specific to diffusion models. Its DALD component estimates mutual information via the stochastic denoising process, leveraging diffusion’s unique properties to better capture epistemic uncertainty.\n\nTo test generality, we replaced the DDPM backbone with DeepLabV3 and substituted eDALD with eBALD (MC-Dropout BALD + entropy). Results on CamVid under a 10% pixel budget are shown below:\n\n| Backbone|Method|CamVid (mIoU \\%)|\n|-|-|:-:|\n|DeepLabV3|Entropy|26.21|\n||eBALD|27.78|\n||2-stage Entropy|31.54|\n||2‐stage eBALD|29.54|\n|**Diffusion**|Entropy|25.26|\n||eDALD|25.14|\n||2-stage Entropy|30.77|\n||**2‐stage eDALD**|**35.73**|\n\n- On both backbones, **2‐stage** variants improve by $\\sim4$–$5$ pp over 1‐stage baselines, confirming the general benefit of separating diversity and uncertainty. \n- **Diffusion + eDALD** (35.73 \\%) remains the best, showcasing diffusion’s added representational power. \n- **DeepLabV3 + 2‐stage Entropy** (31.54 \\%) outperforms its 1‐stage counterpart by over 5 pp and approaches the diffusion model’s 2‐stage Entropy result — demonstrating broad applicability of our pipeline. \n\nThis demonstrates that our two‑stage approach also works effectively on a standard CNN backbone, while diffusion representations combined with eDALD yield the largest gains.\n\n---\n## 5. DALD Performance\nWe agree that the substantial performance gain from adding an entropy term to DALD is somewhat surprising. However, as discussed at the end of Section 3.3 (lines 215–228), this difference stems from the distinct roles of DALD and entropy.\n\nDALD captures disagreement between predictions on noised versions of the same input. This disagreement serves as a proxy for informativeness, as high disagreement typically indicates areas where the model is less confident or more sensitive to input perturbations.\n\nHowever, some informative pixels may not exhibit high disagreement, especially when the model remains consistently uncertain across all noise levels. In such cases, the entropy term captures uncertainty that DALD may miss. By combining both signals, eDALD better identifies a broader range of informative pixels, leading to improved performance.\n\nWe appreciate the reviewer bringing this up, as it highlights the complementary nature of disagreement and uncertainty in our framework.\n\n---\n## 6. Novelty of Our Method\nIn response to the reviewer’s question, we identified two recent diffusion-based active learning (AL) works:\n\n- **Barba et al. [1]** apply a diffusion model for AL in CT reconstruction, selecting measurements via posterior variance on image reconstructions.\n- **Du et al. [2]** use latent diffusion for active domain adaptation, sampling unlabeled domains via diffusion‐driven uncertainty tests.\n\nBoth focus on *image-level* selection in specialized settings (inverse problems or domain shift), whereas our method is the **first** to tackle **pixel-level** AL for **semantic segmentation** under *extremely low budgets*, combining:\n\n1. **Diffusion‐based multi-scale feature herding** to form a diverse candidate pool,\n2. **DALD/eDALD**, a diffusion-specific mutual‐information plus entropy criterion,\n3. **Per-pixel** annotation with only **0.1 click per image per round**.\n\nThese aspects — pixel granularity, two-stage design, and practical segmentation gains — distinguish our work from prior diffusion-driven AL methods. If the reviewer is aware of any diffusion-based AL methods directly addressing supervised segmentation, we would greatly appreciate the reference and will include a comparison in a future revision. \n\n[1] Barba et al., \"Diffusion Active Learning: Towards Data-Driven Experimental Design in Computed Tomography,\" arXiv:2504.03491.\\\n[2] Du et al., \"Diffusion-Based Probabilistic Uncertainty Estimation for Active Domain Adaptation,\" NeurIPS 2023."}, {"review_id": "us1W06Bw0q", "reviewer": "Reviewer_1nUB", "summary": "This paper presents an active learning algorithm for semantic segmentation for pixel selection. Specifically, it proposes using a diffusion model to estimate pixel-level uncertainty, augmented with entropy. It also leverages a hybrid pixel acquisition function that combines representation diversity and uncertainty for subset selection.", "questions": "Please refer to the items listed in the weakness section. It would be helpful if the authors could provide an explanation to improve the clarity of the paper.", "limitations": "Limitations are discussed at the end of the paper. This paper does not introduce potential negative socialtal impact.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "-- Strengths\n\nUsing diffusion models and mutual information for uncertainty estimation is interesting and intuitively motivated.\n\nCombining representation-based and uncertainty-based strategies is an effective choice for selecting pixels, overcoming the limitations of each method individually.\n\nThe experimental results are promising, showing a consistent advantage over baselines, especially in low-labeling-budget scenarios.\n\n-- Weaknesses\n\nThis paper may lack of deeper analysis of the uncertainty mechanism. It would be helpful if the paper could provide an empirical analysis of the mutual information distribution across pixels to validate its effectiveness as an uncertainty metric.\n\nIn addition, it is recommended to provide at least some discussion on how to determine the weighting factor between the diffusion-based term and entropy term in Equation 8 (currently set to 1).\n\nOverfitting could pose a significant challenge when training the model with limited labeled data. It would be helpful if the paper included a discussion on mechanisms employed to mitigate overfitting.\n\nThe representation-based sample selection strategy with RBF kernel primarily builds upon established kernel herding methods, with additional top-K selection at both the image and global levels. Therefore, the methodological novelty of the approach appears limited from this standpoint.", "was_revised": true, "final_justification": "My initial rating assumed the listed issues could be resolved during the discussion phase. The authors' rebuttal has addressed most of my concerns. Considering other reviewers' comments, I intend to maintain my original (positive) rating.", "rebuttal": "Thank you for your thoughtful feedback and for highlighting important aspects of our work. We appreciate the opportunity to address your concerns and refine our presentation accordingly.\n\n---\n## 1. Uncertainty Mechanism Analysis\nWe carried out an in-depth empirical study of our mutual information term (DALD) to understand how it reshapes pixel selection beyond simple entropy. Working on the ADE-Bed dataset, we computed three uncertainty maps for each image at AL rounds 1, 5, and 10: (i) Entropy ($E$), (ii) DALD ($D$), and (iii) eDALD ($H = E + D$, computed as a simple unweighted sum without normalization).\n\nFor each round, we compared the top-$b$ pixels chosen by $E$ versus those chosen by $H$.  The following table reports the Jaccard overlap and Spearman’s rank correlation between the two rankings:\n\n|Round|Jaccard$(E\\cap H)$|Jaccard$(D\\cap H)$|Spearman $\\rho(E,H)$|\n|:-:|:-:|:-:|:-:|\n|1|0.52|0.040|−0.07|\n|5|0.51|0.005|0.15|\n|10|0.34|0.011|−0.12|\n|Avg|0.56|0.010|0.26|\n\nBy round 10, only 34 % of the entropy-selected pixels remain among the eDALD picks, and the low Spearman $\\rho$ (mean 0.26, sometimes negative) shows that DALD substantially reshuffles the uncertainty ranking rather than merely amplifying entropy. DALD alone contributes very few of the same pixels (Jaccard ≈ 0.01), indicating that it brings new, highly uncertain areas into focus.\n\nNext, we examined the DALD magnitudes of the “eDALD-exclusive” pixels (those selected by $H$ but not by $E$).  The table below shows how extreme their DALD values are relative to the global distribution:\n\n|Round|Median multiplier|10% multiplier|1% multiplier|\n|:-:|:-:|:-:|:-:|\n|1|25.75×|7.29×|3.14×|\n|5|5.35×|2.28×|1.63×|\n|10|4.31×|1.90×|1.43×|\n|Avg|7.87×|2.87×|1.77×|\n\nOn average, eDALD-exclusive pixels have a median DALD nearly 8× the dataset median, and even their 1 % percentile is 1.8× the global 99 %-tile.  This confirms that DALD targets the heavy tail of the disagreement distribution, surfacing pixels of true epistemic uncertainty that entropy alone misses.\n\nFinally, we evaluated how these shifts impact segmentation performance.  The table below compares one-stage (pure uncertainty) versus two-stage (MaxHerding → uncertainty) mIoU on ADE-Bed:\n\n|Method|1-stage mIoU|2-stage mIoU|Absolute gain (pp)|Relative gain (%)|\n|:-:|:-:|:-:|:-:|:-:|\n|Entropy|23.02|28.27|+5.25|+22.8%|\n|**eDALD**|23.06|30.18|**+7.12**|**+30.9%**|\n\nWhile DALD alone matches entropy in a single stage (23.06 vs. 23.02 mIoU), integrating disagreement into our two-stage pipeline yields a **+7.12 pp** boost — much higher than the +5.25 pp from entropy. This demonstrates that the combination of coverage (MaxHerding) and disagreement (DALD) in two stages drives the largest gains and validates our uncertainty mechanism as both theoretically sound and practically impactful.\n\nIn summary, our empirical analysis shows that:\n- DALD identifies high-uncertainty “needle-in-the-haystack” pixels that entropy misses.\n- eDALD reshapes the selection ranking, not merely scaling entropy.\n- The combination of coverage (MaxHerding) and disagreement (DALD) in two stages drives the largest gains, validating our uncertainty mechanism as both theoretically grounded and practically impactful.\n\n---\n## 2. Weighting Factor in eDALD\nWe chose equal weights ($\\alpha=1$, $\\beta=1$) to avoid introducing extra hyperparameters and to keep the balance between the mutual‐information term $I(Y;Z\\mid x)$ and the confidence term $H(Y\\mid z^{(0)},x)$. To validate this choice, we ran an ablation on CamVid (all other settings fixed) with several $(\\alpha,\\beta)$ pairs:\n\n\n|$\\alpha$|$\\beta$|R1|R2|R3|R4|R5|R6|R7|R8|R9|R10|\n|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|\n|1.0|1.0|12.81|23.04|18.99|23.00|25.11|32.95|32.23|35.30|33.95|36.14|\n|0.2|1.0|12.81|14.04|18.07|21.18|22.20|27.83|28.36|28.86|33.76|32.65|\n|1.0|0.2|9.69|13.38|19.27|23.27|25.23|27.02|26.89|31.22|32.72|31.94|\n|0.05|4.0|13.24|14.91|18.84|20.34|21.86|19.70|19.44|23.04|19.73|18.66|\n|0.05|2.0|13.24|15.20|20.18|18.98|21.08|23.11|23.70|23.54|24.12|25.71|\n|0.02|2.0|13.24|16.49|16.19|16.83|19.38|19.27|18.22|20.86|18.88|20.37|\n\nThe equal‐weights setting ($1,1$) achieves the highest final‐round mIoU (36.14), confirming that our default choice is both simple and effective without requiring dataset‐specific tuning. \n\n---\n## 3. Overfitting Mitigation\nWe thank the reviewer for raising the important point of overfitting under extreme label scarcity. Several factors in our design help mitigate this risk:\n\n1. **Frozen backbone.** We only train the lightweight segmentation head (0.79 M parameters for CamVid/Cityscapes; 1.61 M for ADE-Bed/Pascal-Context), while the diffusion model that produces the $6{,}144$-dim features remains fully frozen. This dramatically reduces the number of trainable weights and prevents the model from “memorizing” individual labeled pixels.\n2. **Fixed, high-level representations.** Because each pixel’s input is a semantically rich feature vector extracted once by the backbone, the classifier sees only stable, high-level patterns rather than raw pixel noise. This further constrains the head’s capacity to overfit.\n3. **Regularization.** We apply standard weight decay ($1\\times10^{-5}$) and early stopping based on validation-set mIoU to halt training once performance plateaus.\n\nCollectively, these measures keep our segmentation head's effective capacity in check, allowing it to generalize from as few as $N$ total labeled pixels (one per image) up to only $0.1\\times N$ per round, without evidence of overfitting in our empirical learning curves.\n\n---\n## 4. Novelty of Our Method\nWe thank the reviewer for this observation. While kernel herding itself is a well‐studied coverage technique, our work advances it in three key ways that together constitute a novel contribution:\n\n1. **Scalable Local $\\to$ Global Herding for Pixels.**\n   Classical MaxHerding over all $N \\times H \\times W$ pixels is intractable. We introduce a two‐stage pipeline — first selecting $K$ representatives *within* each image, then re‐herding those $N K$ candidates *across* images — to reduce complexity to minutes on tens of thousands of features. This adaptation makes herding practical for dense, per-pixel active learning in semantic segmentation.\n2. **Extreme Low‐Budget, Pixel‐Granular Setting.**\n   We tackle a novel regime — only *one pixel per image per round* (total budget $=1\\times N$ pixels) — where neither whole-image AL nor conventional region-based methods suffice. Designing an AL strategy that yields strong mIoU under these constraints required rethinking both coverage and uncertainty at the pixel level.\n3. **Diffusion‐Native Acquisition (eDALD).**\n   Beyond herding, we propose a new uncertainty measure — **Entropy‐Augmented DALD** — that combines mutual information from stochastic multi‐scale diffusion features with a Shannon entropy term. Unlike BALD or MC‐Dropout, eDALD exploits the diffusion model’s intrinsic noise process, capturing epistemic uncertainty without Bayesian networks or multiple forward passes. To our knowledge, no prior work has introduced such an acquisition function for semantic segmentation.\n\nTaken together, our **two‐stage pipeline** — scalable herding + diffusion‐based uncertainty — enables efficient, per‐pixel selection under extreme labeling constraints. Although each component draws inspiration from existing ideas, their integration in this context (pixel‐level AL, diffusion features, 1‐pixel/image budgets) and the demonstration of substantial segmentation gains represent a genuine methodological advance in active learning for dense prediction tasks."}]

Author rebuttal:

Meta-review: The paper received three borderline accept ratings, and one borderline reject rating. The major reason for reject recommendation seems to be practical relevance of pixel wise labeling due to region based contemporary styles, and low absolute mIoU. This has been defended by the authors by claiming that they focus on extremely low budget, and have shown experiments with SAM. AC is satisfied by the rebuttal and of the opinion that reasons for acceptance outweigh reasons for rejection and recommend acceptance.
Decision: Accept (poster)