Active Diffusion Subsampling
Authors: Oisín Nolan
Tristan Stevens
Wessel L. van Nierop
Ruud Van Sloun
Venue: Accepted by TMLR ()

Abstract: Subsampling is commonly used to mitigate costs associated with data acquisition, such as
time or energy requirements, motivating the development of algorithms for estimating the
fully-sampled signal of interest $x$ from partially observed measurements $y$. In maximum-
entropy sampling, one selects measurement locations that are expected to have the highest
entropy, so as to minimize uncertainty about $x$. This approach relies on an accurate model
of the posterior distribution over future measurements, given the measurements observed so
far. Recently, diffusion models have been shown to produce high-quality posterior samples
of high-dimensional signals using guided diffusion. In this work, we propose Active Diffusion
Subsampling (ADS), a method for designing intelligent subsampling masks using guided dif-
fusion in which the model tracks a distribution of beliefs over the true state of $x$ throughout
the reverse diffusion process, progressively decreasing its uncertainty by actively choosing
to acquire measurements with maximum expected entropy, ultimately producing the pos-
terior distribution $p(x | y)$. ADS can be applied using pre-trained diffusion models for any
subsampling rate, and does not require task-specific retraining – just the specification of
a measurement model. Furthermore, the maximum entropy sampling policy employed by
ADS is interpretable, enhancing transparency relative to existing methods using black-box
policies. Experimentally, we show that through designing informative subsampling masks,
ADS significantly improves reconstruction quality compared to fixed sampling strategies on
the MNIST and CelebA datasets, as measured by standard image quality metrics, includ-
ing PSNR, SSIM, and LPIPS. Furthermore, on the task of Magnetic Resonance Imaging
acceleration, we find that ADS performs competitively with existing supervised methods in
reconstruction quality while using a more interpretable acquisition scheme design procedure.
Code is available at https://active-diffusion-subsampling.github.io/.

Published in Transactions on Machine Learning Research (02/2025)
Active Diffusion Subsampling
Oisín Nolan Tristan S. W. Stevens
o.i. n.s.nolan@lue.n t.s. w..steeens@tve. w.d. w.w.erop@tue.....
Wessel L. Nierop
Ruud J.G. van Sloun
r-j-g. n. w.sloun @tue.nl
Eindhoven Universily of Technology
Reviewed on OpenReview: https: openreview. net/ forun? id= fotofftooon
Abstract
Subsampling commonly used mitigate costs associated with data acquisition, such as time or energy requirements, motivating the development of algorithms for estimating the fully-sampled signal of interest from partially observed measurements y. In maximum- entropy sampling, one selects measurement, locations that are expected to have the highest entropy, so as to minimize uncertainty about x. This approach relies on an accurate model of the posterior distribution over future measurements, given the measurements observed so far. Recently, diffusion models have been shown to produce high-quality posterior samples of high-dimenssonal signals using guided diffusion. In this work, we propose Active Diffusson Subsampling (ADS), method for designing intelligent subsampling masks using guided dif- fusion in which the model tracks distribution of beliefs over the true state of throughout the reverse diffusion process, progressively decreasing its uncertainty by actively choosing to acquire measurements with maximum expected entropy. ultimately producing the pos- terior distribution p(x y). ADS can be applied using pre-trained diffusion models for any subsampling rate, and does not require task-specific retraining just the specification of a measurement model. Furthermore, the maximum entropy sampling policy employed by ADS is interpretable, enhancing transparency relative to existing methods using black-bo box policies. Experimentally,
show that through designing informative subsampling masks, ADS significantly improves reconstruction quality compared to fixed sampling strategies the MNIST and CelebA datasets, as measured by standard image quality metrics, includ- ing PSNR, SSIM, and LPIPS. Furthermore, on the task of Magnetic Resonance Imaging acceleration, we find that ADS performs competitively with existing supervised methods in reconstruction quality while using a more interpretable acquisition scheme design procedure. Code is available https://active diffusion subsampling hwt.ssammiinggghhuu...... github.io///
Target
Intermediate Belief Distribution
Generated Mask
Rewnstruction
O
Reverse Diffusion
Figure 1: Active Diffusion Subsampling jointly designs subsampling mask and reconstructs the target signal in single reverse diffusion process.

---

Published in Transactions on Machine Learning Research (02/2025)
1 Introduction
In recent years, diffusion models have defined the state of the art in inverse problem solving, particularly in the image domain, through novel posterior sampling methods such as Diffusion Posterior Sampling (DPS) (Chung al. 2022) and Posterior Sumpling with Lalend. Diffusion (PSLD) (Rout et al. 2024). These meth- ods are often evaluated on inverse imaging problems, such impainting, which is akin image subsampling, Typical benchmarks evaluate inpainting ability using naive subsampling masks such as randomly masked pixels, or box mask in the center of the image (Rout et al. 2024). In real-world applications, however, more sophisticated subsampling strategies are typically employed, for example, in Magnetic Resonance Imaging (MRI) acceleration (Lustig & Pauly 2010: Bridson 2007). These subsampling strategies are usually designed by domain experts, and are therefore not generalizable across tasks. Some recent literature has explored learning subsampling masks for various tasks (Bahadir et al. 2020) Baldassarre et al. 2016 Huijben et al, 2020 Van Gorp et al., (021), but these methods typically depend on black-box policy functions, and require lask-specific training. In this work, introduce Aclive Diffusion Subsampling (ADS), an algorithm for automatically designing task- and sample-adaptive subsampling masks using diffusion models, without the need for further training or fine-tuning (Figure 2). ADS uses white-box policy function based on maxi- mum entropy sampling (Caticha 2021 in which the model chooses sampling locations that are expected to maximize the information gained about the reconstruction target. In order to implement this policy, ADS leverages quantities that are already computed during the reverse diffusion process, leading to minimal additional inference time. We anticipate that ADS can be employed by practitioners various domains that diffusion models for subsampling lasks but currently rely on non-adaplive subsampling strategies. Such
domains include medical imaging, such MRI, CT, and ultrasound (Shaul et al. 2020; Pérez et al.
2020
Faltin et al. 2011), geophysics (Campman et al. 2017 Cao et al. 2011 and astronomy [Fng et al.
2023
Our main contributions are thus as follows:
novel approach to active subsampling which can be employed with existing diffusion models using popular posterior sampling methods;
A white-box policy function for sample selection, grounded in theory from Bayesian experimental design;
Significant improvement in reconstruction quality relative to baseline sampling strategies.
Application to MRI acceleration and ultrasound scan-line subsampling.
Diffusion model
Posterior samples
Unobserved
reconstruction
A
Measurement
y
Choose
a4t
Mask m
GMM
Figure 2: Schematic overview of the proposed Active Diffusion Sampling (ADS) method.
2
Related Work
Methods aiming select maximally informative measurements appear in many domains, spanning statis- ties, signal processing, and machine learning, but sharing foundations in information theory and Bayesian
2

---

Published in Transactions on Machine Learning Research (02/2025)
inference. Optimal Bayesian Experimental Design (Lindley 1956) aims to determine which experiment will be most informative about some quantity of interest 0 RRainforth et al., 2024). typically parameters of a statistical model. Active learning (Houlsby et al.) 2011) performs an analogous task in machine learning, aiming to identify which samples, if included in the training set, would lead to the greatest performance gain on the Irue data distribution. While our method focuses on subsampling high-dimensional signals, it could also be interpreted as a Bayesian regression solver for the forward problem y = =((x) + n, in which x is now seen as parameters for a model f, and the active sampler seeks measurements y which minimize uncertainty about the parameters x, relating it to the task of Bayesian experimental design.
From a signal processing perspective, ADS can be characterized as a novel approach to adaptíve compressive sensing, in which sparse signals are reconstructed from measurements with sub-Nyquist sampling rates (Rani et al. 2018), and typically applied in imaging and communication. Measurement matrices relating observed measurements to the signal of interest x are then designed so as to minimize reconstruction error on x. In Bayesian compressive sensing, measurement matrices are designed so as to minimize a measure of uncertainty about the value of x. Adaptive approaches, such as that of Braun el al. (2015), aim to decrease this uncertainty iteratively, by greedily choosing measurements that maximize the mutual information between x and y Ux + n. These methods typically assume a Gaussian prior on x, however, limiting the degree to which more complex prior structure can be used. More recently, a number of methods using deep learning to design subsampling strategies have emerged. These approaches typically learn subsampling strategies from data that minimize reconstruction error between x and y. Methods by Huijben et al. (2020) and Bahadir et al. (2020) learn fixed sampling strategies, in which a single mask is designed a priori for given domain, and applied to all samples for inference. These methods can be effective, but suffer in cases where optimal masks differ across samples. Samplc-adaptive methods (Van Gorp et al. 2021 Bakker et al.) 2020 Yin ct al.
2021 Stevens et al.] (022) move past this limitation by designing sampling strategies at inference time. A popular application of such methods is MRI acceleration, spurred by the fastMRI benchmark (Zbontar et al. 2018), in which a full MRI image must be reconstructed from sub-sampled R-space measurements. In A-DPS (Van Gorp et al. 2021), for example, a neural network is trained to build an acquisition mask consisting of M k-space lines iteratively, adaptively adding new lines based on the current reconstruction and prior context. Bakker et al. (2020) implements the same procedure using reinforcement learning agent. One drawback of these methods is their reliance on black-box policics, making it difficult to detect and interpret failure cases. Generative approaches with transparent sampling policies circunvent this issue. For example, methods by Sanchez et al. (2020) and/van van de Camp et al. (2023) take a generative approach to adaptive MRI acquisition design, using variants of generative adversarial networks (GANs) to generate posterior samples over MRI images, and maximum- maximum-variance sampling in the k-space as the measurement selection policy. The performance of such generative approaches depends on how well they can model the true posterior distribution over x given observations. This motivates our choice of diffusion models for generative modeling, as they have shown excellent performance in diverse domains, such computer vision (Dhariwal & Nichol 2021
medical imaging (Chung & Ye, 2022, Stevens et al. 2024), and natural language processing (Yu et al 2022
The existing works discussed above have in common that they run the reconstruction model N times for N measurements during inference. Due to the slower iterative inference procedure used by diffusion models, running inference N times may not be feasible in real-world applications, which are typically time- sane-bound by the acquisition process. ADS alleviates this issue by using the Tweedie estimate of the posterior distribution as an approximate posterior, allowing N measurements to be acquired in a single reverse diffusion process. We also highlight a related concurrent work by Elata et. al. (2025), which uses denoising diffusion restoration models (DDRMs) to generate sensing matrices for adaptive compressed sensing, which differs from ADS in that it requires full DDRM inference in order to acquire each sample.
3 Background
3.1 Bayesian Optimal Experimental Design
In Bayesian Optimal Experimental Design (Rainforth et al. 2024h and Bayesian Active Learning (Houlsby lt al. 2011),, the objective is to choose the optimal design, or set of actions A = A", leading to new observations of measurement variable y that will minímize uncertainty about a related quantity of interest
3

---

Published in Transactions on Machine Learning Research (02/2025)
X, measured by entropy H. It was shown by Lindley (1956) that this objective is equivalent to finding the actions A that maximize the mutual information 7 between and y, L.e. selecting actions leading to observations of y that will be most informative about x:
A arg min[H AA,) = arg max[[(y;3))) A)]
(1)
This objective is commonly optimizod actively, wherein the design is created itcratively by choosing actions that maximize mutual information, considering past observations when selecting new actions (Rainforth let al. 2024). This active paradigm invites an agent-based framing, in which the agent's goal is to minimize its own uncertainty, and belicfs over are updated as measurements of y are taken. The active design objective can be formulated as follows, where o, is possible action at time A A-1 a, is the set of actions (oo...., )r) taken so far at time i, and Vr-1 is the set of partial observations of the measurement variable y until time t - 1:
of = arg max[ As, de
arg max (x (. Ar, y.-) - log | x, Ar-1))
(2)
d,
arg ma A Ac,yc-1) Hy, Ac,
42
We can interpret the agent's behavior from Equation as trying to maximize marginal uncertainty about Yt, while minimizing model uncertainly about what value y: should take on given a particular x (Houlsby et al. 2011). Active designs are typically preferred over fized designs, in which a set of actions chosen up-front, as opposed being chosen progressively as measurements are acquired (Rainforth et al.] 2024). While fixed designs may be more computationally efficient, they are less sample-specific, which can lead to lower information gain about x. Finally, it is worth noting that this active optimization scheme, while greedy, has been shown to be near-optimal due to the submodularity of conditional entropy (Golovin & Krause 2011).
3.2 Subsampling
Generally image reconstruction tasks can be formulated as inverse problems, given by:
== Ux + n,
(3)
where y E PI is a measurement, x E X the signal of interest and E A some noise source, typically Gaussian. For the subsampling problem, the measurement matrix U E MMN can be expressed in terms of a binary subsampling matrix using one-hot encoded rows such that we have an element-wise mask m = diag(U UU,, where only the diagonal entries of U are retained, representing the subsampling pattern. We can relate the subsampling mask m through the zero-filled measurement which can be obtained through Yat = U m x + n.
Since are interested in the adaptive design of these masks, we express their generation as m U(A.) U(A)), where the measurement matrix is now function of the actions A {oo, ... az) taken by the agent up to time t. The th clement of that mask is defined as follows:
m [U(A)) = =(0 if A otherwise.
(4)
The measurement model equation can now be extended to an active setting via y: U(A))x n, Note too that some applications we have an additional forward model f, mapping from the data domain to the measurement domain, yielding y: U(A))((x) nn.
3.3 Posterior Sampling with Diffusion Models
Denoising diffusion models learn reverse stochastic differential equation (SDE) that progressively noises samples towards standard Normal distribution (Song et al.] 2020). The (variance preserving) SDE
4

---

Published in Transactions on Machine Learning Research (02/2025)
defining the noising process is follows:
dx = B(T)] 2
xdr + BB(T)dw
(5)
where x(0) E IRd is an initial clean sample, 7 E [0, ((T) is the noise schedule, and w is a standard Wiener process, and x(T") (0, 1). According to the following equation from Anderson (1982), this SDE can be reversed once the score function Vx lo (x) is known, where w is standard Wiener process running backwards:
((() 2
dx
V(T) V p, (x:) dr + V ((T)
(6)
Following the notation by Ho et al. (2000) and Chung et al. 2022 the discrete setting of the SDE is represented using I, x(TT/N), B, = ( /N), a, = Br at,
II. Ogs, where N is the numbers of discretized segments. The diffusion model achieves the SDE reversal by learning the score function using a neural network parameterized by 0, sq (x., T) = xx, logpr(x,).
The reverse diffusion process can be conditioned on measurement, y to produce samples from the posterior p(xly). This can be done with substitution of the conditional score function Xx, log P- (x+) in equation 6 The likelihood Xx, (y|p-(y(xx)) which follows from refactoring the posterior using Bayes' rule has led to various approximate guidance schemes to compute these gradients with respect to partially-noised sample X, (Chung et al., 2022 Song et al. 2023 Rout et al. 2024). Most of these rely on Tweedie's ' formula (Efron 2011 Robbins 1992), which serves as a onc-step denoising process from T - 0,
intractability of the noise-perturie
denoted D+((); yielding the Minimum Mean-Squared Error denoising of x, (Milanfar & Delbracio 2024
Using our trained score function T) we can approximate Xo as follows:
N0 E[xo|x,] D,(x+)=
(x+ + - ((r)))ss((x;;)) Vā,
(7)
Diffusion Posterior Sampling (DPS) uses equation to approximate Vx, log px))))))))))))))))))))>>)))))p R Vx, (ogp(y/8o). In the case of active subsampling, this leads to guidance term Vx, lly: U(A:)f(Xo)I) indicating the direction in which x, should step in order to be more consistent with y, = U(A,)((x) + n. The conditional reverse diffusion process then alternates between slandard reverse diffusiin steps and guidance steps in order to generate samples from the posterior p(x yt). Finally, we note that number of posterior sampling methods for diffuuson models have recently emerged, and refer the interested reader to the survey by Daras et al.)
2024h.
4 Method
4.1
Active Diffusion Subsampling
ADS (Algorithm operates by running a reverse diffusson process generating a batch {x! (x. O,..., possible reconstructions of the larget signal guided by an evolving set of measurements {yr) which are revealed to it through subsampling actions taken at reverse diffusion steps satisfying T E S, where is a subsampling schedule, or set of diffusion time steps at which to acquire new measurements. For example, in application in which one would like to acquire 10 measurements, {yr) across diffusion process 90). consisting of T 100 steps, one might choose línear subsampling schedule S = (0, 10, 20)...90)). Then,
.....,,, Np of
during the diffusion process, if the diffusion step T is an element of S, a new measurement will be acquired, revealing it to the diffusion model. These measurements could be, for example, individual pixels or groups of pixels. We refer to the elements of the batch of reconstructions (xf") } as particles in the data space, they implicitly track belief distribution over the true, fully-denoised = throughout reverse diffusion. These particles are used lo compute estimates of uncertainty about x, which ADS aims to minimize by choosing actions a, that maximize the mutual information between x and y: given az Figure 3 illustrates this action selection procedure. The remainder of the scction describes how this is achieved through (i) employing running estimates of xu given by D, (ii) modeling assumptions on the measurement entropy, and (iii) computational advantages afforded by the subsampling operator.
5

---

Published in Transactions on Machine Learning Research (02/2025)
Algorithm 1: Active Diffusion Subsampling
Require: T, Np, S, ( } {a, 1r--0 Ainit
0: Ao = Amtti
yo = (xA))((x)) no: (x/ ~ ((0,
for = T to 0 do
// Batch process in parallel for efficient inference
3
for = to N₂ - do
4
$ - so (x .)
8 + D,(x7) (
Vu, (x, + (1 (8) // Calculate Tweedie estimate
5
$10) - [(x)0) // Simulate full measurement
7
z ~~ I),(1
8
x--1 +- Va, (1-ū, (1)
(1)
1-6,
X
Vār-18-
1-a,
X0 + // Reverse diffusion step
( lolly, ((A,)5111 // Data fidelity step x--1= f 7-1 - cV
(1)
if E S then
t + ++1
// Find the group of pixels with maximum entropy across simulated neasurements
Σ 19
Np
N₂ log >
at arg max
exp
Icel
20
13
4. = AL-1Uai
Yr ((A))f(x) + n, Acquire new measurements
end
(2(0) N--1 return:
Beturn posterior samples
ADS follows information-maxiiiiiin policy, selecting measurements af arg max,, [ Ac: Assuming measurement model with additive noise Ye U(A)f(()) + n., the posterior entropy term H(y x, At, in the mutual information is unaffected by the choice of action d;: it is solely determined by the noise component H(y. Yt-) to maximize. This entropy term (Equation can be computed as the expectation of loyp At-y;,,,, or the expected distribution measurements y: given the observations so far, and a subsampling matrix Ac. We approximate the true measurement posterior in Equation as a mixture of N₂ isotropic Gaussians, with means set estimates of future measurements under possible actions, and variance I. The means y (x (()) are computed by applying the forward model to posterior samples 8n p(x Yt-1), which are estimated using the batch of partially denoised particles x via
This simplifies the policy function, leaving only the marginal entropy term
x (x.")
(0) yielding:
p(y. At, = Σ ws oz
(8)
Note the parameter o which controls the width of the Gaussians.. The entropy of this isotropic Gaussian Mixture Model can then be approximated as follows, as given by Hershey & Olsen (007):
N
N wy exp
8 11
H(yel A,,1-11 Rs constant + Σ wy log Σ
20
Intuitively, this approximation computes the entropy of the mixture Gaussians by summing the Gaussian error between each pair of means, which grows with distance according the hyperparameter oz. This hyperparameter oz can therefore be tuned to control the sensitivity of the entropy to outliers.

---

Published in Transactions on Machine Learning Research (02/202))
2
3
Entropy per K-Space line
X
D
5
A t
T
Figure 3: Illustaation of single action selection using ADS. ! shows the current batch of partially-denoised images (x at diffusion step r'. In @ this batch of particles is mapped using Tweedie's formula to batch of fully-denoised particles (x(") ), constituting the belief distribution at time r'. The forward model f is then applied in ① to simulate the set of measurements that result from the belief distribution, used to approximate the measurement posterior GMM. Given this GMM, the measurement entropy is computed in ( using Equation 10 Finally, in ① the maximum entropy line is selected as the next measurement location.
Finally, we leverage the fact that U(A:) is a subsampling matrix to derive an efficient final formulation of our policy. Because U(A,) is a subsampling matrix, the optimal choice for the next action a will be at the region of the measurement space with the largest disagreement among the particles, as measured by Gaussian error in Equation Therefore, rather than computing separate set of subsampled particles for each possible next subsampling mask, instead compute single set of fully-sampled measurement particles g(9) = (x(()), and simply choose the optimal action as the region with largest error. For example, when pixel-sbssampling an image, the particles ((()) become predicted estimates of the full image, given the pixels observed far, and the next sample is chosen whichever pixel has the largest total error across the particles. Similarly, in accelerated MRI, the next k-space line selected is the one in which there is the largest error across estimates of the full k-space. Denoting as I E the set of indices sampled by cach possible action and assuming equal weights for all particles, wy = wy, Vi,j, the final form of the policy function is given as follows (see Appendix A.1 for derivation):
Σ (5 lEa,
( - 860122
N
N
af = arg mnx
log
exp
(10)
20 y
5 Experiments
We evaluate our method with three sets of experiments, covering a varicty of data distributions and applica- tion domains. The first two experiments, in Sections 5.1 and 5.2 evaluate the proposed Macimumm-Bee EEntroy subsampling strategy employed by ADS against baseline strategies, keeping the generative model fixed to avoid confounding. Next, in Scction 5.3 we evaluate the model end-to-end on both sampling and recon- struction by applying in the real-world task of MRI acceleration with the fastMRI dataset, and comparing it to existing supervised approaches.

---

Published in Transactions on Machine Learning Research (02/2025)
5.1 MNIST
In order to evaluate the effectiveness of the Maximum Entropy subsampling strategy employed by ADS, we compare it to two baseline subsampling strategies on the task of reconstructing images of digits from the MNIST dataset (LeCun et al.) 1998). To this end, a diffusion model was trained on the MNIST training dataset, resized to 32 x 32 pixels. See Appendix AA.2. for further details on training and architecture. Using this trained diffusion model, each subsampling strategy was used to reconstruct 500 unseen samples from the MNIST test set for various subsampling rates. Both pixel-based and line-based subsampling were evaluated, where line-based subsampling selects single-pixel-wic columns. The mcasurement model is thus y, = U(,,)x, as there is no measurement noise or measurement transformation, i.e. f(x) =x x. The baseline subsampling strategics used for comparison were as follows: (i) Random subsampling selects measurement locations from a uniform categorical distribution without replacement, and (ii) Data Variance subsampling
selects measurement locations without replacement from a categorical distribution in which the probability of a given location proportional to the variance across that location the training set. This can therefore be seen as a data-driven but fixed design strategy.
Inference was performed using Diffusion Posterior Sampling for measurement guidance, with guidance weight (=1 and T = 1000 reverse diffusion steps. For ADS, measurements were taken at regular intervals in the window [0, 800]. with NN 16 particles and By = 10. For the fixed sampling strategies, the subsampling masks were set priori, such that all diffusion steps are guided by the measurements, as is typical in inverse problem solving with diffusion models. The results to this comparison are illustrated by Figure | with numbers provided in Appendix [A.3] We use Mean Absolute Error as an evaluation metric since MNIST consists of single channel brightness values. It is clear from these results that ADS out performm fixed mask baselines, most notably in comparison with data-variance sampling: for pixel-based sampling, we find that maximum entropy sampling with budget of 100 pixels outperforms data variance sampling with budget of 250 pixels, i.e. actively sampling the measurements is as good as having 2.5x the number of measurements with the data-variance strategy. We also find that the standard deviation of the reconstruction errors over the test set is significantly lower for 25% and 50% subsampling rates (typically ~ 2-3x than basclincs, leading to more reliable reconstructions). Further experiments on MNIST are carried out in Appendix A..3 where ADS compared to Active Deep Probabilistic Subsampling Van Gorp et al. (2021). an end-to-end supervised method for active subsampling.
Random
Random
Data Varlance
Data Varfance
ADS
ADS
10-1
10-1
MAE
MAE
10-2
10-2
10
25
50
100
250
500
2
4
B 16
24
Number pixels sampled
Number of lines sampled
(a) Pixel- based subsampling
(b) Line-based subsampling
Figure& 4: Comparison of ADS (ours) with two non-adaptive baselines. Evaluated based on reconstruction Mean Absolute Error (MAE) on N 500 unseen samples from the MNIST test set. Note that MAE is plotted log scale.
5.2 CelebA
In order to evaluate ADS on natural image datasot, a diffusion model was trained on the CelebA [Liu et al. (015)) training dataset 128 x 128 resolution (see Appendix(A.22 A.222 for training details). ADS was then benchmarked N=100 samples from the CelebA test set against DPS with baseline sampling strategies.
8

---

Published in Transactions on Machine Learning Research (02/2025)
This evaluation was carried out for number sampling rates |S| {50, 100, 200, 300, 500). The measurement scheme employed here samples 'boxes' of size 4x pixels from the image. The number of diffusion steps taken during inference was chosen based on sampling rate, with 400 steps for |S| < 300, 600 steps for |S| = 500. In each case, for sampling rate [S] the sampling window [10. + 10) was used. These sampling windows were chosen empirically, linding that some initial unguided reverse diffusion steps help create belter initial particles with dy = were used to model the posterior distribution. The
estimate of the posterior. N2 peak signal-to-noise ratio (PSNR) values between the posterior mean and test samples are provided in Figure 5a (a) and Table 1 with further metrics, including metrics across individual posterior samples, are provided in Appendix A.5 It is clear in these results that ADS outperforms baseline sampling strategios with DPS across & number of sampling rates. Note, for example, that ADS with 200 measurements achieves a higher PSNR than random-mask DPS with 300 measurements. In Figure 56 & selection of samples are provided lo exemplify the mask designs produced by ADS. Note here that the masks focus on important features such as facial features in order to minimize the posterior entropy, leading to higher information gain about the larget, and ultimately better recovery.
ADS
DPS Random Mask
Random RPS
40
Data Varianus DPS
Target
Mask
Meserenents Pesterior Mon
Mask
Mexeuroments
Posterior Mean
ADS
35
30
25
20
15
jen
en
50
100 200
300
500
Number boxes sampled
(a)
(b)
Figure 5: In (a), PSNR (t) scores for ADS vs DPS with random and data variance sampling on N=100 unseen samples from the CelebA dataset are plotted, for increasing nunbers of measurements. A measurement here is a x 4 box of pixels. (b) shows some examples of ADS inference versus DPS with random measurements on the CelobA dataset from the evaluation with 200 boxes sampled.
# Samples (Max %)
Random DPS
Data Variance DPS
ADS
50 (4.88%)
18.099 (0.019)
17.747 (0.018)
18.932 (0.024)
100 (9.76%)
20.580 (0.019)
19.892 (0.017)
22.725 (0.024)
200 (19.53%)
23.555 (0.020)
22.895 (0.018)
27.954 (0.026)
300 (29.29%)
25.919 (0.019)
24.779 (0.019)
31.483 (0.027)
500 (48.82%)
29.123 (0.018)
27.512 (0.018)
35.630 (0.027)
Table 1: PSNR (f) on test samples from the CelebA dataset. Next to each # Samples |S| indicating the maximum % a 128 x 128 image that |S] x 4 boxes could cover.
is a 'Max %'
5.3 MRI Acceleration
To assess the real-world practicability of ADS, it was evaluated on the popular fastMRI (boonar et al. 2018) 4x acceleration benchmark for knee MRIs. In this lask, one must reconstruct. a fully-sampled knee MRI image given a budget of only 25% of the x-space measurements, where each K-space measurement is a vertical line of width pixel. We compare with existing MRI acceleration methods focusod specifically on learning sampling strategies, namely PG-MRI (Bakker al. 2020). LOUPE (Bahadir al., 2020), and SeqMRI A.9. We use the same data train / validation
(Yin et al. 2021), each of which are detailed in Appendix

---

Published in Transactions on Machine Learning Research (02/2025)
test split and data preprocessing as Yin et al. 2021 for comparability. In particular, the data samples are k-space slices cropped and centered at 128 128, with 34, 732 train samples, 1, 785 ion samples. and 1, 881 test samples. We train a diffusion model on complex-valuer image space samples x E C128x128 obtained by computing the inverse Fourier transform of the k-space training samples (sec Appendix A.2.3 for further training details). The data space is therefore the complex image space, with x-space acting as the measurement space. This yields the measurement model y: = ((A))((x) + nt, where F is the Discrete Fourier Transfor,, n, ~ NC, on) is complex Gaussian measurement noise, and U(A.) is the subsampling matrix selecting samples at indices A.. ADS proceeds by running Diffusion Posterior Sampling in the complex image domain with guidance from s-Space measurements through the measurement model, selecting maximum entropy lines in the k-space. We observed on data from the validation set that ADS reconstruction performance increases with the number of reverse diffusion steps, although with diminishing returns as steps increased. This indicates that in applying ADS, one can choose to increase sample quality at the cost of inference time and compute. To showcase the potential for ADS, we chose a large number of stops, T = 10k. Further, we choose guidance weight Ç = 0.85, and sampling schedule S evenly partitioning [50, 2500], and an initial action set Asnit = {63), starting with single central k-space line. N, 16 and øy 50 were used to model the posterior distribution. Reconstructions were evaluated using the structural similarity index measure (SSIM) (Wang et al., 2004) to compare the absolute values of the fully-sampled target image and reconstructed image. The SSIM uses a window size of 4 x 4 with kn = 0.01 and k2 = 0.03 as set be the fastMRI challenge. Table 2 shows the SSIM results on the test set, comparing ADS to recent supervised methods, along with a fixed-mask Diffusion Posterior Sampling using the same inference parameters to serve as a strong unsupervised baseline. The fixed-mask used with DPS measures the 8% of lines at the center of the и-space, and random lines elsewhere, as used by Zbootarr et al. (2018). It is clear from the results that ADS performs competitively with supervised baselines, and outperforms the fixed-mask diffusion-based approach. Figure G shows two reconstructions created by ADS. See Appendix [A.1I] for a histogram of all SSIMs over the test set for the biffusion-based approaches. Finally, we note the inference time for this model, which is an important factor in making active sampling worthwhile. Our model for fastMRI (Table 2 uses 40ms////// / step with 76 steps per acquisition, leading to 3040ms per acquisition on our NVIDIA GeForce RTX 2080 Ti GPU. A typical acquisition time for a k-space line in MRI is 200ms-2500mss or higher, depending the desired quality (Jung & Weigel, 2013). Given modern hardware with incrcased FLOPs, believe that this method is already near real-time, even without employing additional tricks to accelerate inference,
such as quantization (Shang et al. 2023) or distillation (Slimans & Ho 2022
Unsupervised
Method
SSIM (t)
x
PG-MRI (Bakker et al.
2020
87.97
x
LOUPE [Bahadir et al.
2020
89.52
Fixed-mask DPS
90.13
ScqMRI (Yin et al.
2021)
91.08
ADS (Ours)
91.26
Table 2: SSTM scores for fast knee test set with 4x acceleration. See Appendix A.11 for SSIM histograms for diffusion- based methods.
5.4 Focused Ultrasound Scan-Line Subsampling
As a final experiment, we explore focused ultrasound scan-line subsampling (Huijben et al. 2020). In focused ultrasound, set of focused scan-lines spanning region of tissue is typically acquired, with the lines being displayed sidc-by-side to create an image. By acquiring only small subset of these scan-lines, the power consumplion and data transfer rate requirements of the probe can be reduced. Successfully applying scan- line subsampling could find applications in (i) extending battery life in wireless ultrasound probes (Baribeau ct al., 2020), (ii) reducing bandwidth requirements for cloud-based ultrasound (Federici et al.) 2023) and (iii) increasing frame-rates in 3D ultrasound (Giangrossi et al., 2002). In this experiment, we demonstrate using ADS to identify informative subsets of scan-lines, from which the full image can be reconstructed.
10

---

Published in Transactions on Machine Learning Research (02/2025)
Target
Reconstruction
K: k-space mask
Target
Reconstruction
K: K-space mask
Figure 6: Sample fastMRI reconstructions produced by ADS, including the generated sispace masks. The SSIMs are 95.8 for the left, and 94.4 for the right.
To this end, we trained diffusion model on the samples from the CAMUS (Leclerc et al. 2019 dataset, with the Irain set consisting of 7448 frames from cardiac ultrasound scans across 500 patients, resized lo 128 x 128 in the polar domain. Each column of pixels in the polar domain taken to represent a single beamformed scan-line, yielding the model y: U(A))x, where x the fully-sampled target image, U(A,) is mask selecting & of scan-lines, and yı the set of scan-line measurements selected by A,- Inference takes place in the polar domain, and results are scan-converted for visualization. For ADS inference, the parameters chosen
T' 400 steps, S evenly partitioning the interval [0. 320], N, 16, σy 1, and Ç = 10. We benchmark ADS against DPS using random and data-variance fixed-mask strategics on a test set consisting of frames from 50 unseen patients, finding that ADS significantly outperforms both in terms of reconstruction quality metrics. Figure 7 presents the PSNR results along with a sample reconstruction produced by ADS versus random fixed-mask DPS, with further quantitative results and sample outputs provided in Appendix A.3
Target
Measurements
Reconstruction
32
Random DPS
Data Verianoe DPS ADS
30
ADS
PSNR
Random DPS
20-
18
32
Number of lines sampled
(a)
Figure In (a), a comparison of selected measurements and reconstructions generated by ADS versus DPS with random mask on the CAMUS (Leclerc al. 2019) dataset. The measurement budget is 16/128 = 12.5% of the scan-lincs. (b) shows the PSNR scores across held-out test set consisting of 875 frames from 50 unseen patients, with measurement budgets of 16 and 32 scan lines, representing subsampling rates of 12.5% and 25%, respectively.
11

---

Published in Transactions on Machine Learning Research (02/2025)
6 Discussion
While ADS appears to outperform fixed-mask baselines on MNIST, CelebA, and fastMRI, it is interesting to note that the relative improvement offered by ADS is less pronounced in the case of fastMRI. For line-based image subsampling on MNIST with 25% samples, ADS achieves a 50% reduction in reconstruction error versus a fixed-mask approach (MAE = 0.019 vs 0.037), whereas with line-based k-space subsampling for fastMRI with 25% samples, ADS achieves only a 12% relative improvement (SSIM 91.26 vs 90.13). We find that the size of this performance gap between fixed and active mask design strategies can be explained by examining the distribution of masks designed by ADS on each task (See Appendix AA.1).. Indeed, the masks designed for fastMRI are very similar, whereas those designed for MNIST typically differ depending on the sample. When mask designs are similar, then fixed masks will perform similarly to actively designed masks. This is in part a feature of the data distribution - for example, most information in k-space is contained in the center, at lower frequencies. Tasks in which one might expect significantly better performance from ADS are therefore those in which optimal masks will be highly sample-dependent.
Another interesting trend appears in the results when observing the relative improvement of ADS over fixed- mask strategies as a function of the number of samples taken. One might expect that the relative improvement is highest when the subsampling rate is lowest, monotonically decreasing as more samples are taken. In fact, however, we observe that the largest relative improvement appears not at the lowest subsampling rates, but rather at medium rates, e.g. 10% - 50%. One possible explanation for this is the diffusion posterior sampling in scenarios with very few measurements, leading to inaccurate estimates of the posterior distribution and therefore sub-optimal sampling strategies. Further study of the performance of posterior sampling methods in cases with very few measurements could provide an interesting direction for future work.
of
We also observe that data variance sampling outperforms random sampling on MNIST, but not on CelebA. This somewhat surprising result can be explained by noting that the variance in MNIST appears in a highly informative region, namely around the center of the images where the digils are contained. In contrast, the variance in CelebA appears mostly in the background, where backgrounds may exhibit large deviations in brightness and color. Despite the large variance in the backgrounds, they are typically homogeneous blocks of color that can be inpainted well from sparse samples. Hence, taking most samples in this high-variance background region leading to suboptimal sampling strategies and reconstructions.
A final point of discussion is on choosing the number of particles Np for a particular task. We find in AopendixAA.3rrr that 8-16 particles are sufficient for modelling the posterior in CelebA, with diminishing returns for increasing Np- Tn general, we advise choosing Np such that the modes of the posterior distribution can be sufficiently represented, so that regions of disagroement between modes can be identified by the entropy compulation. For example, an MNIST dalaset containing only 2 digits will require fewer particles than an MNIST dataset containing 10.
In conclusion, we have proposed method for using diffusion models as active subsampling agents without the need for additional training, using a simple, interpretable action selection policy. We show that this hod significantly outperforms -aakk baselines on MNIST, CelebA, and ultrasound scan-line subsampling, and competes with existing supervised approaches in MRI acceleration without tasks-spocific training. This method therefore takes a step towards transparent active sensing with automatically generated adaptive strategies, decreasing cost factors such exposure time, and energy usage.
7 Limitations & Future Work
While experiments in Section 5 eviiencee some strengths of ADS against baseline sampling strategies, it is not without limitations. For example, the duration of inference in ADS is dependent on that of the diffusion posterior sampling method. Since low latency is essential in active subsampling, future work could aim to accelerate posterior sampling with diffusion models, leading to accelerated ADS. Another limitation is that the number of mcasurements taken is upper-bounded by the number of reverse diffusion steps T; this limitation could be overcome by extending ADS to generate batch designs (Azimi al.) 2010), containing multiple measurements, from single posterior estimate. Future work applying ADS in diverse domains
12

---

Published in Transactions on Machine Learning Research (02/2025)
would also help to further assess the robustness of the method. Finally, mentioned in Section 6 note that ADS does not show significant improvements over baseline sampling strategies under very low sampling rates, e.g. 5%. We hypothesize that this may be due to inaccurate posterior sampling with very few measurements, and believe that this presents an interesting direction for future study.
References
Brian DO Anderson. reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):313-326, 1982.
Javad Azimi, Alan Fern, and Xiaoli Fern. Batch Bayesian optimization via simulation matching. Advances in neural information processing systems, 23, 2010.
Cagla D Bahadir, Alan Q Wang, Adrian V Dalca, and Mert R Sabuncu. Deep-learning- based optimization the under-sampling pattern in MRI. IEEE Transactions on Computational Imaging, 6:1139-11533 2020.
Tim Bakker, Herke van Hoof, and Max Welling. Experimental design for MRT by greedy policy search. Advances in Newral Information Processing Systems, 33:18954 18966, 2020.
Luca Baldassarre, Yen-Huan Li, Jonathan Scarlett, Baran Gözcü, Ilija Bogunovic, and Volkan Cevher. Learning-basee compressive subsampling. IEFE Journal of Selected Topics in Signal Processing, 10(4): 809 822, 2016.
Yanick Baribeau, Aidan Sharkey, Omar Chaudhary, Santiago Krumm, Huma Fatima, Feroze Mahmood, and Robina Matyal. Handheld point-of-care ultrasound probes: the new generation of POCUS. Journal of cardiothoracic and vascular amesthesia, 34(11):3139 3145, 2020.
James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: compos- able transformations of Python-Numly programs, 2018. URL http://github com com/google/jax
Gábor Braun, Sebastian Pokutta, and Yao Xie. Info greo-greedy sequential adaptive compressed sensing. IEEE Journal of selected lopics in signal processing, 9(4):601-611, 66660116,,,111661111111111111111 2015.
Robert Bridson. Fast Poisson disk sampling in arbitrary dimensions. SIGGRAPH sketches, 10(1):1, 2007.
Xander Campman, Zijian Tang, Hadi Jamali-Rad, Boris Kuvshinov, Mike Danilouchkine, Ying Ji, Wim
Walk, and Dirk Smit. Sparse seismic wavefield sampling. The Leading Edge, 36(8):654 660, 2017.
Jingjie Cao, Yanfei Wang, Jingtao Zhao, and Changchun Yang. A review on restoration of scismic wavefields based on regularization and compressive sensing. Inverse Problems in Science and Engineering, 19(5): 679 704,704, 2011.
Ariel Caticha. Entropy, information, and the updating of probabilities. Entropy, 23(7):895, 2021.
François Chollet et al. Keras. https://koras.1o 2015.
Hyungjin Chung and Jong Chul Ye. Score-based diffusion models for accelerated MRI. analysis, 80:102479, 2022.
Medical image
Hyungji Chung, Jeongsol Kim, Michael T Mccann, Mare L Klasky, and Jong Chul Ye. Diffusion posterior sampling for general noisy inverse problems. arXio preprint arXiv:2209. 14687, 2022.
Giannis Daras, Hyungjin Chung, Chich-Hsin Lai, Yuki Mitsufuji, Jong Chul Ye, Peyman Milanfar, Alcxan- dros G Dimakis, and Mauricio Delbracio. A survey on diffusion models for inverse problems. arXiv preprint arXiv:2410.0000 2024.
13

---

Published in Transactions on Machine Learning Research (02/2025)
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs image synthe- sis. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan
(eds.), Advances in Neural Information Processing Systems, volume 34, pp. 8780 8794. Curran Associates, Inc., 2021. URL
ftlle//paper/002//E1I
Saaeaadleccsabbeeroo2sssssaaaaas pdf
Bradley Efron. Tweedie's formula and selection bias. Journal of the American Statistical Association, 106 (496):1602-161) 1614, 2011.
Noam Elata, Tomer Michaeli, and Michael Elad. Adaptive compressed sensing with diffusion-based posterior sampling. In European Conference on Computer Vision, pp. 290-308. Springer, 2025.
Peter Faltin, Kraisorn Chaisaowong, Thomas Kraus, and Til Aach. Markov- Markov-Gibbs model based registration of CTlung images using subsampling for the follow-up assessment of pleural thickenings. In 2011 18th IEEE International Conference on Image Processing, pp. 2181 2184. IEEE, 2011.
Beatrice Federici, Ben Luijten, Andre Immink, Ruud JG van Sloun, and Massimo Mischi. Cloud-basod ultrasound platform for advanced real-time image formation. In Dulch Biomedical Engineering Conference, 2023.
Berthy T Feng, Jamie Smith, Michael Rubinstein, Huiwen Chang, Katherine L. Bouman, and William T Freeman. Score-based diffusson models as principled priors for inverse imaging. In Proceedings of the IEEE/CVP International Conference on Computer Vision, pp. 10520 10531, 2023.
Claudio Giangrossi, Alessandro Ramalli, Alessandro Dallai, Daniele Mazierli, Valentino Meacci, Enrico Boni, and Piero Tortoli. Requirements and hardware limitations of high-frame-rate 3-d ultrasound imaging systems. Applied Sciences, :2(13):6562, 2022.
Daniel Golovin and Audreas Krause. Adaptive submodularity: Theory and applications in active learning and stochastic optimization. Journal of Artificial Intelligence Research, 42:427-488 186,746,6667 2011.
John R Hershey and Peder A Olsen. Approximating the Kullback Leibler divergence between gaussian mixture models. In 2007 IEEE International Conference on Acoustics, Speech and Signal Processing- ICASSP'07, volume 4, pp. IV-317. IEEE, 2007.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Neil Houlsby, Ferenc Huszár, Zoubin Ghahramani, and Máté Lengyel. Bayesian active learning for classifi- cation and preference learning. arXie preprint arXt::1112.5744 2011.
Iris Hujjben, Bastiaan Veeling, and Ruud JG van Sloun. Deep probabilistic subsampling for task-adaptive compressed sensing. In 8th International Conference on Learning Representations, ICLR 2020, 2020.
Bernd André Jung and Matthias Weigel. Spin echo magnetic resonance imaging. Journal of Magnelic Resonance Imaging, 87( 817, 2013.
Sarah Leclere, Erik Smistad, Joao Pedrosa, Andreas Ostvik, Frederic Cervenansky, Florian Espinosa, Tor- vald Espeland, Erik Andreas Rye Berg, Pierre-Marc Jodoin, Thomas Grenier, et al. Deep learning for segmentation using an open large-scale datasct in 2d echocardiography. IEEE transactions on medical imaging, 3899)21198 2210, 2019.
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based lcarning applied to docu- ment recognition. Proceedings of the IEEE, 86(11):2278 2324, 1998.
Dennis V Lindley. On mcasure of the information provided by an experiment. The Annols of Mathematical Slatistics, 27(4)9986 1005, 1956.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEFE international conference on compuler vision, pp. 3730-3738, 3733, 2015.
14

---

Published in Transactions on Machine Learning Research (02/2025)
Michael Lustig and John M Pauly. Spirit: iterative self-consistent parallel imaging reconstruction from arbitrary k-space. Magnelic resonance in medicine, 64(2):457-471, 2010.
Peyman Milanfar and Mauricio Delbracio. Denoising: A powerful building- block for imaging, inverse prob- lems, and machine learning. arXiv preprint arXiv:2409.0 06219, 2024.
Eduardo Pérez, Jan Kirchhof, Fabian Krieg, and Florian Römer. Subsampling approaches for compressed sensing with ultrasound arrays in non-destructive testing. Sensors, 20(23):6734, 2020.
Tom Rainforth, Adam Foster, Desi R Ivanova, and Freddie Bickford Smith. Modern Bayesian experimental design. Stotistical Science, 39(1):100-114, 2024.
Meenu Rani, Sanjay B Dhok, and Raghavendra B Deshmukh. A systematic review of compressive sensing: Concepts, implementations and applications. IEEE access,
2018.
Herbert E. Robbins. An empirical Bayes approach to statistics. In Breakthroughs in Stalistics: Foundations and basic theory, pp. 388 394. Springer, 1992.
Litu Rout, Negin Raoof, Giannis Daras, Constantine Caramanis, Alex Dimakis, and Sanjay Shakkottai. Solving linear inverse problems provably via posterior sampling with latent diffusion models. Advances in Newral Information Processing Systems, 36, 2024.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. arXiv preprinl arXiv:2202.000 00511, 2022.
Thomas Sanchez, Tgor Krawczuk, Zhaodong Sun, and Volkan Cevher. Closed loop deep Bayesian inver- sion: Uncertainty driven acquisition for fast MRI, 2020. URL hatps://openreriex.nettt wwt.:ooriieeeeeeeoouuunorriiid
BJ1PO1BKDB
Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. Post-training quantization diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pottern recognition, pp. 1972 1981, 2023.
Roy Shaul, Itamar David, Ohad Shitrit, and Tammy Riklin Raviv. Subsampled brain MRI reconstruction by generative adversarial neural networks. Medical Image Analysis, 65:101747, 2020.
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz. Preudoiinerse-guided diffusson models for inverse problems. In International Conference on Learning Representations, 2023. URL https:// opwnreview.net.oo new.eww.eeeeeeee htt///oouumffa httpe//fooourddddggeaaM....
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.19456 2020.
Tristan SW Stevens, Nishith Chennakeshava, Frederik J Bruijn, Martin Pekař, and Ruud JG van Sloun. Accelerated intravascular ultrasound imaging using deep reinforcement learning. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp- 1216-1220. IEEE, 2022.
Tristan SW Stevens, Faik C Meral, Jason Yu, Iason Z Apostolakis, Jean-Luc Robert, and Ruud JG Van Sloun. Dehazing ultrasound using diffusion models. IEEE Transactions on Medical Imaging, 2024.
Koen CE van de Camp, Hamdi Joudeh, Duarte J Antunes, and Ruud JG van Sloun. Active subsampling using deep generative models by maximizing expected information gain. In TCASSP 2023-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1-5. IEEE, 2023.
Hans Van Gorp, Tris Huijben, Bastiaan S Vecling, Nicola Pezzotti, and Ruud JG Van Sloun. Active deep probabilistic subsampling. In International Conference on Machine Learning, pp. 10509-10518. PMLR, 2021.
15

---

Published in Transactions on Machine Learning Research (02/2025)
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(():600--0 612, 2004.
Tianwei Yin, Zihui Wu, He Sun, Adrian V Dalca, Yisong Yue, and Katherine L Bouman. End-to-end sequential sampling and reconstruction for MRI In Machine Learning for Health, pp. 261-281. PML.R, 2021.
Peiyu Yu, Sirui Xie, Xiaojian Ma, Baoxiong Jia, Bo Pang, Ruiqi Gao, Yixin Zhu, Song- Chun Zhu, and Ying Nian Wu. Latent diffusion energy-based model for interpretable text modelling. In International Conference Machine Learning, pp. 25702-25722 25720. PMLR, 2022.
Jure Zbontar, Florian Knoll, Anuroop Sriram, Tullie Murrell, Zhengnan Huang, Malthew J Muckley, Aaron Defazio, Ruben Stern, Patricia Johnson, Mary Bruno, et al. fastMRI: An open dataset and benchmarks for accelerated MRI. ar preprint 1xXi::811.008835 2018.
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018.
16

---

Published in Transactions on Machine Learning Research (02/2025)
A Appendix
A.1 Derivation of Equation (10)
Here we show that maximising the policy function does not require computing a set of particles for each possible action in the case where the action a subsampling mask. Because the subsampling mask A, = A Ua only varies in at in the arg max, the elements of each particle y( will remain the same for each possible A, except for those indices selected by dz- We therefore decompose the squared L2 norm into two squared L2 norms, one for the indices in a, and the other for those in A--1- The latter then becomes constant in the argmax, and can be ignored. This results in a formulation in which we only need to compute the squared L2 norms for the set of elements corresponding with az. We use U(A)) to indicate the subsampling matrix containing Is on the diagonal at indices in A,-
N
a3 = arg max de
log
exp
20²
Np
Np
[UAA)) (9) f(A))
013
= arg max
log
exp
20
Σ (1(2( 12 /((())))) )2)
N
N₂
KEA:
= arg max
log
exp
20
ae
Σ (f(())) - (1)2 + Σ
(f(*)) )m f(((6))mm) m²
Ny
Np
arg max de
log
Σ exp
IE0L
m
20
(11)
Σ (f((( - f(x()))))
Σ (f($!")) )m f(x )m²)
N₂
LEa:
mEA:-1
arg max
log
exp
exp
20
y
Σ E(( f(8(())) )1)
N₂
Np
= arg max de
log
exp
LEar
20
Σ (0()))
(8)
800ز
N,
N
arg max
log
exp
IEa,
20
A.2 Training Details
The methods and models are implemented the Keras 3.1 (Chollet et al. 2015) library using the Jax
backend (Bradbury et al. 2018). The DDIM architecture is provided by Keras3 the following URL: Each model was trained using one GeForce RTX 2080
https: //xera///eee.eee 1 examples ht/gg/eeetie/ddii.... /tap///rraaeeeeeeeaaeeeeeeeeeeee Ti (NVIDIA, Santa Clara, CA, USA) with 11 GB of VRAM.
A.2.1 MNIST
The model was trained for 500 epochs with the following parameters: widths=(32, 64, 128), block deck__, diffusion steps=30, ema 0.999, learning 0a00=0, weight. decay=0.0001, loss="mae'
17

---

Published in Transactions on Machine Learning Research (02/2025)
A.2.2
CelebA
The model was trained for 200 epochs with the following parameters: 132t 64, 96, 128]. block 2 diffusion steps=30, emaa 0.999, learning rate=0.0001, weight decay=0.0001, loss= "mae".
A.2.3
FastMRI
The training run of 305 epochs with the following parameters: widths-132, 64, 96, 128), block depth-2, diffusion_ sttrsion_st ema 0.999,
deggtt__deayy000011, bass="mac'.
A.3 MNIST metrics
# Pixels (%)
Random
Data Variance
ADS (Ours)
10 (.97%)
0.231 (.002)
0.197 (.002)
0.207 (.002)
25 (2.44%)
0.190 (.002)
0.125 (.002)
0.124 (.002)
50 (4.88%
0.140 (.002)
0.067 (.001)
0.042 (.001)
100 (9.76%)
0.074 (.001)
0.034 (.001)
0.011 (.000)
250 (24.41%)
0.024 (.000)
0.015 (.000)
0.007 (.000)
500 (48.82%)
0.011 (.000)
0.008 (.000)
0.007 (.000)
# Lines (%)
Random
Data Variance
ADS (Ours)
2 (6.25%)
0.197 (.003)
0.152 (.002)
0.148 (.002)
4 (12.5%)
0.143 (.002)
0.086 (.002)
0.071 (.001)
8 (25%)
0.073 (.001)
0.037 (.001)
0.001 (.000)
16 (50%)
0.022 (.001)
0.013 (.000)
0.000 (.000)
24 (75%)
0.010 (.000) 0.0076 (.000)
0.0074 (.000)
Table 3: Mean and (standard error) for the Mean Absolute Error (4) in reconstruction of MNIST samples, using pixel- and line-based subsampling.
A.4 Comparison with Active Deep Probabilistic Subsampling
In this appendix we provide qualitative and quantitative results comparing ADS to supervised active sampling approach, Active Deep Probabilistic Subsampling (ADPS) by Van Gorp et al. (2021). ADPS performs active sampling by training Long Short-Term Memory (LSTM) model to produce logits for each sampling location at each iteration. This is trained jointly with a 'task model', which performs some task based on the partial obscrvations. This could be, for example, a roconstruction model. We chose to compare ADS to ADPS on the lask of reconstructing MNIST digits to facilitate a qualitative comparison of the masks generated by each method. In order to compare on reconstruction task, we adapted the MNIST classification model provided in the ADPS codebase for reconstruction by replacing their fully-connected classification layers with a large UNet and using an MSE training objective to reconstruct 32x32 pixel MNIST digits from partial observations containing 81 81/1024 x 8% of the pixels. We chose UNet architecture with widths=[32, 64, 128, 256] and block depth=2 as the reconstruction model for ADPS, making it similar to the UNet denoising network used by ADS. Both models were trained on the MNIST training set and evaluated on 1000 unseen samples from the validation set.
The results are presented in Figures g g 10 and I1 indicating that ADS outperforms ADPS in terms of reconstruction error, and generated more adaptive and intuitive masks. One reason for the limited adaptivity in the masks generated by ADPS could be the fact that the reconstruction model is trained jointly with the sampling model, leading to a strategy wherein the reconstruction model can depend on receiving certain inputs and not others. We note, however, that the performance of ADPS could change depending on the choice of reconstruction model, loss function used, and other design choices. In general, the major advantage
Classification
18

---

Published in Transactions on Machine Learning Research (02/2025)
of ADS over ADPS is that it does not need to be re-trained for different choices of forward model subsampling rates: all that's needed is diffusion prior, and sampling schemes can vary freely at inference time.
40
30
20-
t.SNE Component 2
t.SN Component 2
10
0
10-
-10-
-2
-30
-40
-00
20
75 -50
25
0
25
50
1-SNE Component
E-SNE Component
(a) ADS
(b) ADPS
Figure 8: t-SNE plots for the masks generated by both (a) ADS and (b) ADPS for 1000 unseen samples from the MNIST validation set. The t-SNE plot for ADS appears to exhibit stronger clustering between different
digits, indicating that the generated masks more digit-dependent. ADPS creates a distinct mask for the digit '1', but there is much overlap across other digits.
(a) ADS
(b) ADPS
Figure 9: The mean across all masks for each digit generated by (a) ADS and (b) ADPS. ADS has generated masks that trace the digits, whereas ADPS has generated spread-out masks in the central region of the image where digits may appear.
19

---

Published in Transactions on Machine Learning Research (02/2025)
Marl
Teyelt
Maii
Mai MEaI
7
7
7
7
hro
Malt
Tapa::
Mauracon
Trwe
Vark3
Tree
Vareccse
bocarnet
3
3
3
3
9
(a) ADS
(b) ADPS
Figure 10: Random samples from the test set with masks, measurements, and reconstructions generated by (a) ADS and (b) ADPS.
0.10
ADPS
ADS
0.08
Mean MAE
0.06
0.04
0.02
0.00
2
Digit Label
Figure 11: Digit-wise comparison of Mean Absolute Error between targets and reconstructions generated by ADS versus ADPS. The error bars each bar indicate the standard error of the mean.
20

---

Published in Transactions on Machine Learning Research (02/2025)
A.5
CelebA Further Metrics
Table 4: MAE (1) of the posterior mean on targets from the CelebA test set.
Num Samples
Random DPS
Data Variance DPS
ADS
50
19.768 (0.043)
20.625 (0.044)
19.569 (0.060)
100 13.623 (0.028)
14.809 (0.028)
11.731 (0.034)
200
8.909 (0.020)
9.786 (0.020)
6.356 (0.018)
300
6.538 (0.014)
7.507 (0.016)
4.426 (0.012)
500
4.320 (0.009)
5.185 (0.010)
2.825 (0.007)
Table 5: SSIM (1) of the posterior mean on targets from the CelebA test set.
Num Samples
Random DPS
Data Variance DPS
ADS
50
0.577 (0.001)
0.557 (0.001)
0.567 (0.001)
100
0.687 (0.001)
0.652 (0.001)
0.697 (0.001)
200
0.791 (0.001)
0.759 (0.000)
0.831 (0.001)
300
0.851 (0.000)
0.820 (0.000)
0.889 (0.000)
500
0.911 (0.000)
0.881 (0.000)
0.941 (0.000)
Table 6: LPIPS (4)
Zhang et al.
2018 of the posterior mean on targets from the CelebA test set.
Num Samples
Random DPS
Data Variance DPS
ADS
50
0.306 (0.001)
0.320 (0.000)
0.287 (0.001)
100
0.230 (0.000)
0.248 (0.000)
0.190 (0.001)
200
0.152 (0.000)
0.174 (0.000)
0.103 (0.000)
300
0.111 (0.000)
0.131 (0.000)
0.071 (0.000)
500
0.069 (0.000)
0.086 (0.000)
0.043 (0.000)
21

---

Published in Transactions on Machine Learning Research (02/2025)
Table 7: Mean MAE (V) across posterior samples on targets from the CelebA test set.
Num Samples
Random DPS
Data Variance DPS
ADS
50
24.486 (0.048)
25.321 (0.050)
23.895 (0.067)
100
17.436 (0.035)
18.493 (0.034)
14.463 (0.040)
200
11.255 (0.025)
12.337 (0.024)
7.704 (0.021)
300
8.230 (0.018)
9.372 (0.020)
5.225 (0.014)
500
5.389 (0.011)
6.477 (0.013)
3.268 (0.008)
Table 8: Mean PSNR (t) across posterior samples on targets from the CelebA test set.
Num Samples
Random DPS
Data Variance DPS
ADS
50
15.988 (0.016)
15.752 (0.015)
16.971 (0.022)
100
18.169 (0.017)
17.791 (0.016)
20.828 (0.023)
200
21.237 (0.019)
20.621 (0.016)
26.314 (0.025)
300
23.617 (0.018)
22.604 (0.018)
30.160 (0.026)
500
26.795 (0.018)
25.274 (0.018)
34.546 (0.026)
Table 9: Mean SSIM (f) across posterior samples on targets from the CelebA test set.
Num Samples
Random DPS
Data Variance DPS
ADS
50
0.486 (0.001)
0.471 (0.001)
0.481 (0.001)
100
0.599 (0.001)
0.569 (0.001)
0.623 (0.001)
200
0.725 (0.001)
0.691 (0.001)
0.783 (0.001)
300
0.799 (0.000)
0.764 (0.000)
0.859 (0.001)
500
0.877 (0.000)
0.841 (0.000)
0.925 (0.000)
Table 10: Mean LPIPS (1)
Zhang et al.
2018
across posterior samples on targets from the CelebA test set.
Num Samples
Random DPS
Data Variance DPS
ADS
50
0.321 (0.001)
0.329 (0.001)
0.305 (0.001)
100
0.250 (0.000)
0.260 (0.000)
0.208 (0.001)
200
0.172 (0.000)
0.184 (0.000)
0.112 (0.000)
300
0.127 (0.000)
0.141 (0.000)
0.073 (0.000)
500
0.079 (0.000)
0.095 (0.000)
0.041 (0.000)
22

---

Published in Transactions on Machine Learning Research (02/2025)
A.6
Further Results on Ultrasound Scan-Line Subsampling
Table 11: MAE on test samples from the CAMUS dataset.
Num Samples Random DPS
Data Variance DPS
ADS
16
12.198 (0.002)
12.431 (0.002)
9.533 (0.001)
32
8.520 (0.001)
8.599 (0.001)
6.701 (0.001)
Table 12: PSNR on test samples from the CAMUS datasct.
Num Samples
Random DPS
Data Variance DPS
ADS
16
22.643 (0.002)
22.480 (0.002)
25.473 (0.001)
32
25.893 (0.001)
25.824 (0.002)
29.130 (0.001)
Target
Messarementa
Reconstruction
Targel
Memurements
Reconstruction
(a) ADS
(b) Random DPS
Figure 12: Targets, measurements, and reconstructions for 5 random samples from the CAMUS test set by (a) ADS and (b) Random fixed-nask DPS. The Mean Absolute Error for each reconstruction is provided.
23

---

Published in Transactions on Machine Learning Research (02/2025)
A.7 Runtime for CelebA
Execution Time (s)
20
15
DPS (N, = 16) 15.51
ADS (N, 16)
300
Number of Measurements
Runtime of ADS on samples from the CelebA dataset with size 128 128 pixels, using an NVIDIA GeForce RIX 2080 Ti GPU. Note the contribution in runtime from DPS, below the dashed purple line, and the contribution from ADS, above the dashed purple line, which scales linearly with the number of measurements. The execution time presented is the mean execution time over N 20 runs, with I standard deviation indicated by the error bars.
Figure 13:
A.8 Effect of varying the number of particles
14
12
10
MAE
8
6
2
4
16
32
Number particles, No
Figure 14: reconstruction accuracy on N 20 samples from the the CelebA validation set, as measured by the MAE between the target and posterior mean, using a variety of values for Np.
24

---

Published in Transactions on Machine Learning Research (02/2025)
A.9 FastMRI Comparison Methods
A.9.1 PG-MRI
Bakker et (2020] use volicy-gradient methods from Reinforcement Learning to learn a policy function T (az ( that outputs new measurement locations given the current reconstruction. Reconstructions are then generated using a pre-existing U-Net based reconstruction model provided by the fastMRI repository.
A.9.2 LOUPE
LOUPE [Bahadir et al, 2020 introduces a end-to-end learning framework that trains a neural network to output under-sampling masks in combination with an anti-aliasing (reconstuction) model on undersampled full-resolution MRI scans. Their loss function consists of reconstruction term and trick enable sampling mask.
A.9.3 SeqMRI
With SeqMRII Yin et al. (2021) propose an end-to-end differentiable sequential sampling framework. They therefore jointly learn the sampling policy and reconstruction, such that the sampling policy can best fit with the strengths and weaknesses of the reconstruction model, and vice versa.
25

---

Published in Transactions on Machine Learning Research (02/2025)
A.10 ADS Mask Distributions for MNIST and fastMRI
Fraction of masks containing each line
Fraction of masks containing each line
0.8
08
40
0.6
0.6
15
0.4
0.4
20
25
120
0.0
120
0.0
Probability of mask containing each line
Probability of mask containing each line
6555-
1.00
1419
0.75
0.50
0.2
10
15
25
30
0
20
40
80
1000
120
line
Iise
(a) ADS mask distribution for 500 samples from the MNIST test set.
(b) ADS mask distribution for 1851 samples from the fast test set.
The distribution of masks chosen by ADS varies according to the task. We observe that the masks chosen for MNIST are less predictable a priori than those chosen for fastMRI, leading to a stronger performance by ADS relative to fixed-mask approaches. We plot at the bottom of cach plot estimates of the probability that each line will appear in mask generated by ADS Bernoulli variables (either the line is present in the mask, not). To quantify the predictability of these masks, we compute the average entropy over each of these variables, finding that MNIST masks are signficantly less predictable than those for fastMRI.
Figure 15:
26

---

Published in Transactions on Machine Learning Research (02/2025)
A.11
FastMRI SSIM Distributions
n1..6
mean=90.13
std=11.01
std=11.04
80
80
60
60
Number
40
40
20
20
20
100
20
80
(a) 200-bin histogram showing distribution of SSIM scores across the FastMRI knee test set for 4x acceleration using ADS.
(b) 200-bin histogram showing distribution of SSIM scores across the FastMRI knee test set for 4x acceleration using difffusion posterior sampling with fixed s-space masks.
Figure 16: Histograms comparing the distribution of SSIM scores using ADS and diffusion posterior sampling.
27


Reviews:
[{"review_id": "t7Df60odgN", "reviewer": "Reviewer_mJLx", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n- The approach is novel, and the paper is well-written,\n- The use of diffusion models in this context is interesting. \n\n**Weaknesses**\nI think the paper could do a better job of justifying why this scheme is practically relevant. For example, it is true that for MRI, the sampling masks are designed by domain experts keeping in mind the characteristics of typical objects as well as what is physically realizable in a scanner. However, I'm not aware of actual hardware where taking such adaptive samples is possible, for the experiments considered.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes an interesting approach for optimizing the measurement mechanism for computational imaging systems using diffusion-based priors. The idea is that for a specific object $\\mathbf{x}$, given a measurement operator and the corresponding measurements $\\mathbf{y}$, if one is able to model the posterior $p(\\mathbf{x} | \\mathbf{y})$, then one can iteratively update A (by adding more sample locations) such that the mutual information between $\\mathbf{x}$ and $\\mathbf{y}$ is maximized.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would encourage the authors to better justify the practical relevance of this approach, ideally by adding discussion and/or numerical experiment for a case where such a scheme is known to be practicable via hardware design.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "9ekpkWaKit", "reviewer": "Reviewer_LnKL", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\n**1. Novelty**\n- The introduction of **Active Diffusion Subsampling (ADS)** is a significant contribution, combining **diffusion models** with **Bayesian Optimal Experimental Design** to create adaptive subsampling strategies.\n- The method addresses a critical gap by providing a **task-flexible and interpretable approach** to subsampling.\n\n**2. Interpretability**\n- Unlike black-box policy approaches, the **white-box entropy-based policy function** is grounded in Bayesian principles, enhancing transparency and interpretability of the sampling decisions.\n\n**3. Flexibility**\n- ADS is **model-agnostic** and does not require retraining for different tasks, making it broadly applicable across domains such as **image inpainting**, **MRI acceleration**, and more.\n\n**4. Performance**\n- The experimental results show that ADS significantly outperforms fixed-mask baselines and competes well with supervised sampling strategies in tasks like MRI acceleration.\n- Strong results are reported for MNIST and CelebA datasets, where ADS achieves:\n  - **Lower reconstruction error** (Mean Absolute Error, MAE).\n  - **Higher quality metrics** (SSIM, PSNR) compared to baselines.\n\n**5. Computational Efficiency**\n- Efficient use of posterior samples and particle-based modeling minimizes the computational overhead during inference.\n- Results demonstrate ADS can be computationally competitive even in resource-intensive applications like MRI acceleration.\n\n**6. General Applicability**\n- The submission highlights the potential of ADS for various domains by validating it on distinct datasets, suggesting strong generalization capabilities.\n\n**7. Clear Presentation**\n- The theoretical formulation and experimental results are well-documented, with detailed explanations of the methodology and intuitive visualizations to support claims (e.g., subsampling masks, reconstruction outputs).\n\nWeaknesses\n\n**1. Limited Improvement at Low Sampling Rates**\n- ADS shows **diminished performance gains at extremely low subsampling rates** (e.g., < 5%), particularly in MRI acceleration.\n  - Potential causes include limitations in posterior sampling accuracy with very few measurements.\n  - Authors may need to explore techniques to enhance posterior estimation robustness under sparse data conditions.\n\n**2. Inference Time**\n- ADS relies on **diffusion posterior sampling**, which can be computationally expensive, especially with large reverse diffusion steps (e.g., 10k steps in fastMRI).\n  - While competitive in execution time for MRI tasks, future work could explore **accelerated inference techniques**, such as:\n    - Quantization.\n    - Model distillation.\n    - Batch sampling designs.\n\n**3. Overlap with Fixed Strategies**\nIn cases like fastMRI, the designed masks show significant overlap with fixed-mask strategies, reducing ADS's relative performance advantage.\n  - This may limit its novelty in scenarios where data distributions naturally favor fixed sampling patterns.\n\n**4. Lack of Robustness Analysis Across Domains**\n- The submission focuses heavily on image-based tasks. It would be beneficial to evaluate ADS on:\n  - Non-image domains (e.g., time-series, tabular data).\n  - Tasks with **different noise distributions** or measurement models.\n\n**5. Dependency on Posterior Sampling**\n- The accuracy of ADS hinges on the quality of posterior sampling achieved by the diffusion model. Limitations in posterior approximations (e.g., due to model capacity or noise) could undermine its performance.\n  - Investigating **alternative posterior estimation techniques** may strengthen the method's robustness.\n\n**6. Scalability to Higher Dimensions**\nWhile results are promising for datasets like MNIST and CelebA, the scalability to higher-dimensional datasets (e.g., 4K medical images) remains unclear.\n  - Discussing strategies for handling such data (e.g., leveraging parallelism or distributed computation) could enhance the method's appeal.\n\n**7. Comparability with Other Methods**\n- While ADS competes well with existing supervised methods, comparisons are limited to a few baselines. Incorporating additional benchmarks or broader state-of-the-art comparisons would strengthen the submission.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "### Summary of Contributions\n\nThis submission presents Active Diffusion Subsampling (ADS), a novel methodology for designing adaptive sampling strategies in inverse problems using diffusion models. The approach offers interpretability and task flexibility, surpassing fixed-mask baselines in performance while maintaining simplicity. The major contributions and new knowledge are as follows:\n\n**1. Novel Algorithm for Active Subsampling**\n- Introduces **Active Diffusion Subsampling (ADS)**, leveraging diffusion models to adaptively select subsampling masks.  \n- Employs **maximum entropy sampling** to actively minimize uncertainty in the reconstruction target \\(x\\) by choosing measurements \\(y\\) with the highest expected entropy.\n- Operates using **pre-trained diffusion models**, removing the need for task-specific retraining.\n\n**2. White-Box Policy Function**\n- The sampling policy is derived from Bayesian Optimal Experimental Design principles:\n  \\[\n  a^*_t = \\arg \\max_{a_t} \\left[ H(y_t \\mid A_t, y_{t-1}) \\right],\n  \\]\n  where \\(H\\) is entropy, ensuring maximal information gain.\n- Provides transparency in decision-making, avoiding reliance on black-box methods common in prior approaches.\n\n**3. Experimental Validation**\n- **MNIST**: ADS achieves a significant reduction in Mean Absolute Error (MAE), outperforming fixed sampling baselines by factors exceeding \\(2.5\\times\\) at low sampling rates.\n- **CelebA**: Demonstrates improved reconstruction quality in Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) compared to random and data variance strategies, particularly at medium sampling rates.\n- **MRI Acceleration**: Competes with supervised sampling techniques, such as PG-MRI and SeqMRI, achieving a **91.26 SSIM** on the fastMRI dataset with only 25% \\(k\\)-space measurements.\n\n**4. Computational Efficiency**\n- Minimizes inference time by reusing quantities computed during the reverse diffusion process for mask selection.\n- Adopts efficient particle-based modeling to represent posterior distributions, enabling computationally lightweight uncertainty quantification.\n\n**5. Real-World Applicability**\n- Validates performance in high-dimensional imaging tasks, with applications ranging from **image inpainting** to **MRI acceleration**, showcasing adaptability across domains.\n\n**6. Broader Impact and Limitations**\n- Reduces reliance on domain-specific expertise by automatically generating sampling strategies.\n- Proposes directions for accelerating posterior sampling and extending ADS to batch sampling.\n\nExperimental Findings\n- ADS improves task-specific reconstruction performance without retraining, achieving **interpretable and adaptive sampling** policies.\n- Sample-dependent mask designs result in significant improvements, especially in datasets like MNIST, where variance across samples is high.\n- Offers insights into subsampling challenges at extremely low rates, proposing further investigations into diffusion model robustness in such settings.\n\nADS bridges theoretical contributions in Bayesian design with practical advances in adaptive subsampling, setting a new benchmark for efficient, interpretable sampling strategies in inverse problems.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Recommendations for Improvement\n\n1. **Address Sparse Data Challenges**:\n   - Investigate improvements to posterior sampling under very low sampling rates, potentially incorporating prior knowledge or augmenting the reverse diffusion process.\n\n2. **Optimize Inference Efficiency**:\n   - To lower computational costs, explore accelerated inference strategies (quantization, distillation, or reduced diffusion steps).\n\n3. **Expand Evaluation Scope**:\n   - Validate ADS on non-image datasets or domains to demonstrate broader applicability.\n   - Include tasks with diverse noise characteristics or measurement models to test robustness.\n\n4. **Enhance Comparability**:\n   - Benchmark against a wider range of supervised and unsupervised methods across different domains.\n\n5. **Discuss Scalability**:\n   - Address potential challenges in scaling ADS to high-dimensional data and outline strategies to mitigate these issues.", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "nXSzumRowQ", "reviewer": "Reviewer_1Wyn", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n* The work is generally well-presented with sufficient background. The corresponding motivation is clear and the derivation of the ADS method is intuitive from a high level. The work appears to be novel, extending previous subsampling work on end-to-end models and GANs to diffusion models.\n* The experiments are convincing and clearly show that ADS is able to find informative patches of images for reconstruction, in particularly in limited budget scenarios.\n* The training protocol and additional experimental detail look complete and should be reproducible.\n\n**Weaknesses**\n\n* While the abstract details ADS as a method for active subsampling, it mentions the term \"active subsampling\" exactly once, and does not mention the experimental contributions of the work (presumably acceleration during inference). Moreover, the use of the word \"outperform\" is not attached to any metric such as speed or error. Please clarify that ADS is able to greedily find subsamplings that are \"more informative\" as to have better reconstruction.\n\n* (Sec. 4.1) The actual explanation of ADS is a bit unclear. I initially (wrongly) believed that ADS was trying to find an optimal set of subsampled timesteps $S$ such that reconstructed $\\mathbf{x}_0$ would be of high quality. However upon further reading into the experiments section, it appears actually that $|S|$ controls the number of allowable subsamplings, and $N_p$ is a set of possible reconstructions, such as $N_p=4$ in Figure 1. Moreover, the GMM is directly stated to have variances $\\sigma_y^2 I$ centered at the current \"particles\", but it is not clear what $\\sigma_y$ actually is, perhaps it is connected to the current time-step $\\tau \\in S$? I would suggest rewriting Section 4.1 and clarifying the role of $S$ earlier with a sentence.\n\n* Maybe I am still misunderstanding, but it appears that to compute the region of maximum uncertainty, fully-sampled measurements are still required (stated in page 6: \"compute a single set of fully-sampled measurement particles ...\"). Could the authors please clarify this, especially how a fully-sampled measurement fits into the whole pipeline?\n\n* There is a lack of comparison with other learned subsampling methods, notably adaptive in [3]. While I appreciate that the policy is black-box, the end results seems to be a working subsampling mask for a given reconstruction model, and it would be very interesting to see how the derived masks work between different model archetypes.\n\n**Minor**\n\n* Figure 3 is quite hard to understand. Please add a textual description in the caption.\n\n* I would be interested to see how much longer ADS takes compared to the baseline fixed subsampling due to the additional particles required.\n\n* How does the double sum over the number of samples in Equation 10 (sum over $i,j=1,...,N_p$) fare in practice? It seems that in high dimensions, $N_p$ needs to be higher, which would lead to worse performance. Also, how does the method differ for different values of $N_p$?\n\n**Questions/Comments**\n\n* (Sec. 3.3) Above equation 5: \"The (variance-preserving) SDE defining the noising process is as follows:...\"\n\n* Missing reference for Tweedie's formula on page 5, e.g. [1,2]. Maybe a fuller explanation specifying the relation between the MMSE denoiser and the score function would aid the flow.\n\n* What is $N_p$ for the MRI experiment? \n\n* Can the method be extended to finding common subsampling masks for multiple images? It would be interesting to see how the masks found by ADS differ and presumably degrade once a common mask for multiple images is deployed.\n\n* There are some minor typos in the references, \"{mri}\", missing captialisation on names like \"bayesian\" etc.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This work proposes Active Diffusion Subsampling (ADS), which is an greedy adaptive scheme for subsampling measurements in diffusion posterior sampling, such as patches in images. To approximate the entropy and thus select patches that maximize information gain, the authors propose using a Gaussian mixture model to model the uncertainty when reconstructing with incomplete measurements. The experiments demonstrate significantly better reconstruction quality when given a limited sample budget, showing heuristically that it can find areas of interest, and can presumably be extended to work with pre-trained diffusion models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Overall, I am positive on this work, as it clearly demonstrates that it is possible to find areas of interest for diffusion posterior models. The only critical change I request is to clarify the exposition of the proposed ADS method, in particular the roles of all the different variables $N_p, S,y_\\tau, \\sigma_y$ etc.\n\n**Strengthening**\n* Additional experiments as suggested above, such as varying $N_p$, wall-clock comparison to fixed subsamplings, connection with previous work\n* Fix the caption of Figure 3\n* Clarifying the contributions (adaptively deriving a good subsampling regime for a given reconstruction task) in the abstract and introduction\n* Add more comments into Algorithm 1 e.g. what is step 7 doing? step 8: subsampled data fidelity step, *single input* $\\mathbf{x}$\n\n**References**\n\n[1] Efron, Bradley. \"Tweedie’s formula and selection bias.\" Journal of the American Statistical Association 106.496 (2011): 1602-1614.\n\n[2] Robbins, Herbert E. \"An empirical Bayes approach to statistics.\" Breakthroughs in Statistics: Foundations and basic theory. New York, NY: Springer New York, 1992. 388-394.\n\n[3] Van Gorp, H., Huijben, I., Veeling, B. S., Pezzotti, N., & Van Sloun, R. J. (2021, July). Active deep probabilistic subsampling. In International Conference on Machine Learning (pp. 10509-10518). PMLR.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors have addressed all concerns raised by the reviewers. All reviewers recommend acceptance.
Decision: Accept (poster)