Metropolis-CVAE: Bootstrapping Labels for Bayesian Inference via Semi-Supervised Conditional Variational Autoencoders
Authors: Jonathan Doucette
Christian Kames
Alexander Rauscher
Venue: NeurIPS 2021 Submitted (2021)
Keywords: Metropolis-Hastings
CVAE
Bayesian Inference
MCMC
Semi-Supervised

Abstract: In Bayesian parameter estimation, models make simplifying assumptions to make parameter inference feasible. If learned inference methods are trained using data simulated by models, however, distributional differences between simulated and observed data may lead to biased inference results on the observed data. In this work, we introduce a semi-supervised learned Bayesian inference method which makes use of both simulated data -- for which the underlying parameters are known by construction -- and unlabeled data, which may depend on nuissance parameters not captured by the simulation procedure. A conditional variational autoencoder (CVAE) is trained to perform approximate inference simultaneously on the sets of labeled simulated data and unlabeled data, where the unlabeled data is initialized with arbitrary pseudo labels. At each training iteration, new candidate pseudo labels are drawn from the CVAE posterior and the pseudo labels are updated using the Metropolis-Hastings algorithm. This process results in a Markov chain of bootstrapped pseudo labels for each unlabeled datum, effectively performing online Markov chain Monte Carlo (MCMC) inference wherein the proposal distribution is a CVAE informed by labeled simulated data, producing proposals which are increasingly likely to be accepted as training proceeds. The resulting CVAE is able to efficiently produce samples from the posterior distributions of both the simulated and unlabeled data, implicitly marginalizing over nuissance parameters in the unlabeled data. We demonstrate the effectiveness of this method in magnetic resonance imaging (MRI) where MCMC is computationally impractical to due the (3+1)D nature of the images, showing improvement against traditional MCMC inference in both speed and posterior quality.

Metropolis-CVAE: Bootsrrppiing Labels for Bayesian Inference via Semi-Supervised Conditional Variational Autoencoders
Anonymous Author(s)
Affiliation
Address email
Abstract
In Bayesian parameter estimation, models make simplifying assumptions to make parameter inference feasible. If learned inference methods are trained using data simulated by models, however, distributional differences between simulated and observed data may lead to biased inference results on the observed data. In this work, we introduce a semi-supervised learned Bayesian inference method which makes use of both simulated data for which the underlying parameters known by construction and unlabeled data, which may depend on nuissance parameters not captured by the simulation procedure. A conditional variational autoencoder (CVAE) is trained to perform approximate inference simultaneously on the sets labeled simulated data and unlabeled data, where the unlabeled data is initialized with arbitrary pseudo labels. At each training iteration, new candidate pseudo labels are drawn from the CVAE posterior and the pseudo labels updated using the Metropolis-Hastingg algorithm. This process results a Markov chain of bootstrapped pseudo labels for each unlabeled datum, effectively performing online Markov chain Monte Carlo (MCMC) inference wherein the proposal distribution is a CVAE informed by labeled simulated data, producing proposals which are increasingly likely to be accepted as training proceeds. The resulting CVAE is able to efficiently produce samples from the posterior distributions of both the simulated and unlabeled data, implicitly marginalizing over nuissance parameters in the unlabeled data, We demonstrate the effectiveness of this method in magnetic resonance imaging (MRI) where MCMC is computationally impractical to due the (3+1)D nature of the images, showing improvement against traditional MCMC inference in both speed and posterior quality.
3
5
7
11
13
15
16
17
18
19
20
21
22
23
Introduction
24
25 Bayesian parameter estimation methods are robust techniques for quantifying properties of a system that cannot be observed directly [I. In order to estimate such parameters, one first needs to develop model of the phenomena to be studied. This process requires deep domain-specific knowledge. For 28 all but the most basic of systems, a series of simplifying assumptions on the system are required to make parameter inference tractable. Typically, tractable is synonymous with not being unreasonably expensive to compute. Examples of such methodologies include perturbation theory, which models containing Taylor series expansions drop higher order terms; mean-field theory, in which interactions
26
27
30
31
32 involving many degrees of freedom are replaced by averaged approximations; and (stochastic) differential equations. when used continuous limits of discrete stochastic processes. The price one pays for utilizing given approximation is highly problem dependent. When approximate models used for parameter inference, the mismatch between model predictions and data may propagate
34
Submitted to 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Do not distribute.

---

36
through the inference process and lead to bias and other misestimations of the inferred parameters. In 37 this work, we are interested the following research question: can deep learning model be trained
38 to perform inference, while incorporating both labeled data generated from a well-understood model,
39
and unlabeled data which contains additional complex structure which cannot be modeled?
40
Here, we present Metropolis conditional variational autoencoders (Metropolis-CVARE CVAEs combine traditional CVAEs with the Metropolis- inference method. The resulting networks combine the ability of CVAEs to learn to perform rapid approximate Bayesian inference from labeled data with traditional MCMC methodology which requires only a likelihood model and a prior distribution over the parameters of interest. The Metropolis- CVVE initializes the unlabeled data with pseudo labels drawn from the prior distribution and, informed by the labeled data, iteratively improves the pseudo labels throughout training. We demonstrate the effectiveness of the Metropolis-CVAF network compared to traditional MCMC methods on an inference problem from magnetic resonance imaging (MRI) which is inherently computationally challenging due to the (3+1)D nature of the data.
Metropolis- Markov chain monte carlo (MCMC)
41
42
43
44
45
48
47
48
49
Methods
50
51
2.1
Related work
52 In the pioneering work by Sohn et al. 22, conditional variational autoencoders were introduced
53 by considering a variational lower bound to the conditional log-likelihood log p(y((()) of labels
54 given corresponding data x. Ideally, to perform data-driven approximate inference, one would like to train a network to learn to maximize the conditional log-likelihood directly. However, this is 56 known to be an intractable problem. To mitigate this issue, the stochastic gradient variational Bayes (SGVB) framework is employed. In SGVB, latent space which factorizes the marginal likelihood is 58 introduced via p(y |x) = -e dz p(z pyy z, x), as well as a recognition distribution q(z x, y). The conditional log- likelihood is then maximized indirectly via maximizing the variational lower bound
logp(y|x) 2- - KL (1(2/x, x) [l]()))) + Eg(elxy) [logply|x22) 2)]
(1)
:= -
60
where KL is the Kullback-Leibler divergence. The interpretation as a conditional autoencoder is as follows: let p(z) x). (z))x,)), and p(y|x, z) be parameterized by deep neural networks Ej (z|x). Eg Ez(2)x, y). and D(y x, z). respectively. Then, Ej and E3 can be viewed as encoders which map their inputs into distributions over the latent variables D can be viewed stochastic latent representations z and data x into posterior distributions over the labels y. Hence, y is conditionally autoencoded via the mncodr-ddcood pipeline z ~ Ez (z)x,)) + y | z). At inference time, posterior samples are similarly drawn via z ~ Es (z|x) + ~ D(y)x x, z). The CVAE the triplet of networks (Fi, E2, D) - is trained by minimizing L over the CVAE parameters, thereby maximizing the variational lower bound on log pgy((y xx).
61
62
decoder which maps
64
66
67
68
The CVAE approach by Sohn et al. 22 is designed for supervised learning problems. Earlier work of a similar vein by Kingma et al. [3] introduces a semi-supervised framework in which labeled data
70
71 (Xe, y) is used to infer labels for unlabeled data x,, via the minimization of two-term loss function. The first term of the loss function is a supervised loss over (x+y) y) samples, similar to Equation I
72
73 derived from a variational lower bound on the joint log- likelioel log p(xe, y). The second term is an
74 unsupervised loss over X,, samples which treats label inference as a data imputation task. Specifically, which posterior inference is performed; the resulting 76 loss is a variational lower bound on the log-likelihood log p(x.).
the unknown label is treated as a parameter
77 While the semi- uumi-supervised method of Kingma et al. is an elegant approach to discovering labels,
78 it is not quite suitable for inferring labels for out of distribution data. This is due to the implicit assumption that relationships between x; and y learned from the joint distribution p(X;, y) generalize 80 to data from p(x.). In fact, this assumption is made explicit via an extended objective function which
81 adds regularization term (xx() [olog q [xe]] over the empirical distribution Pe (e y) of labeled data. This penalty encourages the learned posterior distribution g(ylx) to generate labels for x,, by
83 extrapolating from the relationships discovers between (Xe, y) pairs. This will naturally lead to
biased labels for x,, when the distribution underlying x,, differs from that of xt-
2

---

Gabbard et al. who made use of CVAEs to accelerate inference for an application in gravitational 85
86 wave astronomy 4. presented an alternate view of Equation Gabbard begin by aiming to minimize the expected cross-entropy
H Bp(x) U dy p(y|x) log log
88
over the data distribution p(x) between the true posterior p(y|x x)) and approximate posterior ply p
89 Employing the SGVB framework and letting p(y(x) =J² dz ((z)x)ppy x), follows from | that
/ ddyypyyss c(x,x)
((x.))))))))))))))))))))))))))))
H S Ep(x))
(3)
Applying Bayes' theorem, equivalently have that
H < Ex(x) (p(s)x) yC( Eply) (pp(yy [C(x,y)]
(4)
(5)
Ep(xy) [C(x, y)]
(6)
Therefore, maximizing the variational lower bound to log p(y(x) [x) over dataset of (x. y) pairs, in
82 Equation [ is equivalent to minimizing the expected cross-entropy via Equation [6] The interpretation of Equations Faand however, will prove useful for remedying the issue of label inference for out of distribution data.
94
95
2.2
Theoretical contributions
96
Equation 5 is a natural framework for using CVAEs to perform inference on simulated data with
37
known labels [4] Suppose labels y ~ p(y) sampled from a prior distribution and p( is subsequently given by a (possibly stochastic) model function x = f(y). Then, Equation corresponds minimizing the average CVAE loss L(x, over pairs of simulated data xx==(()... y)
88
99
100
The novel contribution this work stems from of Equation 4 Using this formulation directly would require sampling from the posterior ply x). which is stated objective. However, this can be circumvented by making the observation that if we could construct Markov chain of pseudo labels
102
during training, such that the stationary distribution of the sequence (ý*)ken p(y x)), then Equation could be approximated as
104
Ep(x) Ep(y))) y)] so Ep(x)
1 Le
Σ i=0
C(x, Fn-1)
105
where (fn-L.+1- ..... Fr) are the Le most recent samples in the Markov chain. This approach to distribution sampling Markov chain Monte Carlo (MCMC) sampling is considered the gold standard parameter inference.
107
108
In this work, we consider the Metropolis- Hectooplis-Hassiinss (MH) algorithm S 6.. In the context of Bayesian inference for recovering labels y from data x, MH sampling begins with a prior distribution p(y).
109
likelihood function p( p(xly), and proposal distribution Q(y'ly) which quantifies the probability of transitioning from y y' the space of possible labels. Given sample yn of Markov chain, the
111
MH update rule is given by

---

MCMC-100
MCMC-3000
Metropolis-CVAE
B
D
10
10
10"
10a
10
10a
109
104
104
104
104
10a
10a
10a
10a
10a
10a
102
102
10
101
10
10
109
100
109
109
-1.0
-0.5
0.0
0.5
1.0
-1,0
-0.5
0.0
0.5
1.0
-1.0
-0.5
0.0
0.5
1.0
-1.0
-0.5
0.0
0.5
1.0
ā
a-a
B
6
E
F
H
10
105
10a
1.00
10a
10
10s
Ideal
0.75
104
103
10a
10a
10a
0.50
10a
10a
102
10
101
10
0.25
100
109
0.00
-1.0
-0.5
0.0
0.5
1.0
-1.0
-0.5
0.0
0.5
-0.3
-0.2
-0.1
0.0
0.1
0.00
0.25
0.50 0.75 1.0
82
loge
loge
logs
logs
P-P plot
Figure 1: (A-G) Comparison of inference results for data simulated using Equation 14 with parameters drawn from Equation [7] Histograms of errors between the means of the empirical distributions and the true labels shown. (H) p-p plot for each label. In all plots, MCMC with 100 samples, MCMC with 3000 samples, and Metropolis-CVAL with 100 samples are shown in red, green, and blue, respectively.
y' -(y'lyn))
min (1, ply' p(xly')
plyn) p(x(yn))
(8)
~ Uniform (0 fy' u uSa a yn otherwise.
Ynt1
The proposal distribution Qyy'ly) is free parameter of the MH algorithm. In general, choosing Q 114 to resemble the true posterior as closely as possible improves the efficiency of the MH algorithm. 115 Therefore, we propose to the approximate posterior p(y|x) of the CVAE itself as the proposal
116 distribution. Recalling that p(y|x) = fr dz x(z())p(y)) x) dz Ei (())) z x), we let
2(ylxm)==Q0 = = dz Eg (z)x))
(9)
L
1
D(y' [2,,
where 2, ~ (2(x).
(10)
I
117 By choosing this proposal function, are able bootstrap pseudo labels ý onto unlabeled data X.-
118 In particular, we minimize the semi-supervised hybrid loss
Laybrid = Luuper + Lieff
(11)
Luper ((xex))<pette [[(x,, y)] Lxell E(xu E(x (xx8) ((xx:)))
(12)
(13)

---

A
B
C
0.3
(x))
0.8
(x,)
Wasserstein distance
Wasserstein distance
0
MH acceptance ratio
0.2
6
0.2
0.6
6g
b
logs logs
loge
logs
0.4
0.1
0.1
0.2
0.0 10
0.0
0 1000 2000 3000 4000 5000 Epoch
10
10a Epoch
10a
10"
10'
10a
10a
Epoch
Figure 2: (A) Pseudo label acceptance rate vs. epochs for the MRI datasets (x) and (x). (B) Wasserstein distance vs. epochs between empirical label distributions and MCMC with 100 posterior samples, and (C) same as (B) but MCMC with 3000 posterior samples.
119 where (r(x. y) and Pu () are the empirical distributions over the pairs of labeled data (xx, y) and unlabeled data with bootstrapped pseudo labels (X.. y), respectively. The pseudolabels are initialized uniformly from the prior space p(y) and are updated according to the MH algorithm at each training iteration.
121
125
Note that in the MH update step, the MH acceptance ratio a can be interepreted as computing a Bayesian goodness of fit check relative to the current label yn. Therefore, Px(x) and (x.(x) need not be identical distributions, merely close enough such that decreasing the supervised loss Luper improves the proposal quality for the self-supervised loss Leer early in training.
124
125
127
2.3
MRI physics application
We consider an application in magnetic resonance imaging. In MRI, data is typically acquired in the form of (3+1)) (3+1)D spatio-temporal grids, with ID magnetic resonance time signals measured in each voxel of the three spatial dimensions. Advanced imaging methods typically involve voxelwise parameter inference for each time signal for the computation of quantitative maps. Modeling the
129
130
131
132
individual time signals for inference, however, is challenging due imperfections the magnetic field generated by the scanner and other sources of signal corruption. Effectively, in MRI there is distributional mismatch problem: simulated data (x). with labels corresponding to well understood physics parameters, does not contain the full distribution of measured data P. (x) which, while generated by the same physics in principle, depends on additional nuissance parameters which cannot be modeled. Machine learning models which intended to generalize to Px.(x) should therefore not be trained only on data from (x).
133
134
196
138
139
In this work, we acquire multi spin-echo (MSE) MRI images. The MSE time signals are modeled using a two-component extended phase graph (EPG) model using the algorithm detailed in Prasloski et al. 7. Using this model, the j-th time point for each signal is given by
140
8, = f(j TE)
(14)
f(t) = Σ A, EPG(t, a, T2, Ti)
8=1
142 where the spin flip angle, B the refocusing control angle, A1, Ag the component amplitudes, T2 < T2,2 the short and long transverse relaxation times, and Ti the longitudinal relaxation time. The EPG(L....) terms are approximately exponentially decaying in I with time constants T2,4, with additional modifications due to MRI physics determined by a. 8, and Ti. The echo time TE is the uniform spacing between time points. We reparameterize A,, Az. T2,1, and T2,2 in terms of the unconstrained parameters 17, 61, and óg as follows: A n7 Ag
143
144
146
77, log T2,1 1 log T7 min
5

---

148
(log m2maax log T2,
m2.m) -0i.)-11...1 and log T2,2 = log min (log 12. llg T2 mim)) + 6
149
where 22,min 10 ms and ma. 1s. The longitudinal relaxation time fixed Ti 1.0s.
150
MRI signal noise can be modeled as Rician g 9. Given data x normalized to have maximum value
the likelihood p(xly) under Rician noise is given by
s. s-((0)g
logp(xly)
Σ
log PRice
Xj
(15)
maxi x(0)x
j=l
6² 5²
v²
EV
where PRice (ElV, 0) = r2
exp
20²
To
0²
(16)
152
the Rician probability density function with location parameter and scale parameter σ; Lo is the modified Bessel function of the first kind with order zero. We have introduced two additional parameters: a scale parameter account for signal normalization, and noise level relative to this
153
154
155
scale. Additionally, we denote 9 = (a, 3, 51. 62) the parameters of the EPG model
In total, there are labels to be inferred: (ax, B, 2 51, 82, log log s). We place the following priors on the parameters:
157
(18000 45% 909,,00 180%)
82 TN((1.00.5,000..11
(180 45", 909900 180")
log 4(lg 10 10ngg.0000000000 1l1o log ~ TV((0.0, 0.5, -2.5,
(17)
(O. 0.5, 1.0)
81 (N.(0.0,0.5 0.5,0,05,000 1.0)
158
where (N,, o, a, the normal distribution with parameters (1, o) truncated the interval [a, 6). and (a, b) is the uniform distribution on [a, bj. The priors were chosen to align with the expectations that: a, 8 are typically near 180% the short component amplitude rj is typically less than the long amplitude 1 17; 81 and 6n should prefer to represent the shortest and longest components; the noise level is chosen uniformly from signal-to-noise ratios between 20 and 100; the scale parameter s should prefer to be 1.
159
180
162
3
Experiments
185
3.1
Data sets
MRI data The MRI data used for this study consists of two anonymized brain scans acquired using a Carr-Pureell-Mibooon
167
(CPMG) [00 II multi spin-echo sequence [12]. The first data set, denoted Pu, (x), contains signals with Nx. 48 samples times i. TE, with echo spacing TE 8 ms, repetition time TR = 1073 ms, matrix size 240 x 240 x 48, and spatial resolution
168
169
0.96 0.96 2.5 mm3 The second data set, denoted (x), contains signals with Nx,2 56
170
samples at times t, = i TE, with echo spacing TE = 7 ins, repetition time TR = 1066 mss matrix size 240 240 x 113, and spatial resolution 1.0 1.0 3.0 mm3 Following the extraction of image volumes containing the brain, 1 (x) and 2u. (xx contain 821 145 and 265 306 signals, respectively. MRI data was acquired on 3 MR system (Ingenia Elition, Philips Medical Systems, Best, The Netherlands) from healthy volunteers giving written and informed consent, and approved by our university ethics board.
172
173
174
175
176
177
Simulated data Using the EPG physics model 14 with Rician noise, we consider two simulated data sets. First, the labeled data set (x) which generated on demand during training using labels y p(y) drawn from the prior distributions 17 Second, precomputed simulated data set Pt..(x) used for validation of the method, where the labels are held out during training. All simulated signals generated with Nx 64 samples and TE m0.
178
179
180
182
MCMC data MCMC is performed using the No-U-Turn Sampler [3 algorithm to generate posterior samples ý p(y(x.) x.) which can be compared with the (Metr posterior

---

CVAE (MCMC-100)
CVAE (MCMC-3000)
Metropolis-CVei CVAE (MCMC-100)
Metropolis CVAE (MCMC-3000)
B
C
D
10a
10a
10a
10a
10a
10a
10a
10a
10
10A
10
10
100
10"
100
109
0.0
0.2
0.4
0.6 0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6 0.8 1.0
0.0 0.1 0.2 0.3 0.4 0.5
Wi (P()). f(a))
W. (#(3),
F((3))
W; (f(M), p()))
W. (515 (())))
G
H
10a
1.00
10a
102.5
Ideal
10a
102.0
10a
0.75
1013
0.50
10a
101.0
10a
100,5
0.25
109
100.0
109
0.00
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.1
0.2
0.3
0.4
0.00
0.01
0.02
0.03
0.00
0.25
0.50 0.75 1.0
Wi (6))), ((6))
Wi (f(loge), F(loge))
W3 (F(log))) F(logs))
9 plot
Figure 3: (A-G) Histograms of Wasserstein distances between (CEr and MCMC empirical distributions for the two MRI data sets. Histograms of errors between the means of the empirical distributions and the true labels are shown. (H) Quantile-quantile plot for each label between global empirical distributions of CVAE samples and MCMC samples. In all plots, comparison with MCMC with 100 samples and 3000 samples are shown in red and green, respectively; CVAE trained only on simulated data shown in dashed lines; Metropolis- Cetropolis-CVAE shown in solid lines.
184 samples. MCMC was performed twice: once to draw 100 posterior samples for every time signal in
Pa.. (x), (x), and (x). and once to draw 3000 posterior samples for subset of 5000 signals from the training, validation, and testing partitions of all three data sets. In total, performing the above MCMC analysis took approximately 72 h using AMD Ryzen 9 3950X 16-Core CPU.
186
188
3.2
Model architecture
CVAE components Let x E R.Vx, y Ry and z E R& be input data, corresponding labels, and latent space samples, respectively. The encoders E₁ and Eg are chosen to be multivariate normal distributions: (z/x) = (Hz, )2,) and E2(z|x, y) = (Hzz) o2,), where Pz., Hz, E No and 2 Tne E RR Similarly, the decoder is given by D(y x, z) TN Ty, 0, 1), where My E BRN, N, and By E + parameterize independent multivariate normal distributions truncated to [0, 1]. The labels y are scaled linearly from the prior domains 17 to [0, 1j8y, in order to better condition the network during training. Similarly, as we are interested only in the relative x values and not their absolute scale, inputs x are normalized to [0, N=
189
191
N=
192
193
194
195
196
197
Each of the EJ. Es. and D networks are composed of fully connected layers with ReLU activation functions and / hidden layers, with hidden dimension Nu 512, for total of H + 4 layers. The dimensions of the data, labels, and latent space are N₂ = 64, Ny = 7, and N₂ = 12, respectively. The encoder networks output μz,, log oz M22. and log Ts vectors. In order to avoid latent space collapse during early training stages that is, one or both of |, log F22| + cc each Ml; was bounded to (-3, 3) using the activation function x + 3 tanh(r)) and each log oz was bounded to (-6,0 0) using the activation function x + 3 tanh(x) - 3. These bounds were determined by observing empirical juz and log T2 values and choosing intervals which clipped only the tails of the distributions. The decoder network outputs My and log σy vectors without further nonlincarities.
198
199
200
201
202
204
7

---

208
Loss functions The terms Super and Seer in Equation I1 are identical apart from their inputs, with each term consisting of the KL-divergence and evidence lower bound (ELBO) terms from the variational lower bound of Equation [] The KT--divergence can be computed in closed closed-form, and is given by
207
209
KL y)
Σ
22n + (Nzzg (my.)
+ log
Uz.. 1
(18)
20 2:d
Tna.) 2
210
Following sampling 22 ~ (z),2 y) and subsequently computing (Ny: log σy) = D(y /x, 22), the ELBO component is approximated
E (x llgg D(y(x, z.] 2
(19)
(0 Myy. )) log Ty.j
Yj My.
Myg
log Φ
- loo
②
Y..j
Tyy
Ty.
1
where
((E)
v 2m
exp
-
2
c2
(C)) = + erf
22
<({) and (() are the probability density and cumulative distribution functions of the standard normal distribution, respectively. Note that each term in the sum of Equation 19 the log-likeldhood of normal distribution with parameters (Py,i, Ty,s) truncated the unit interval [o, 1].
212
213
214
215
3.3
Training
216
All data sets were split into training/validdatiomrrssnngg with proportions 50 %/25 %/25 %. Input data was padded with zeros length Nx 64, if necessary. To support data with differing unpadded lengths, random masking was performed during training: for each x, random integer jmask was sampled from 32-64 and all elements Xソ>jmm were to zero. The ADAM optimizer was used with an initial learning rate of 10-4, The learning rate was decreased every 1000 epochs by factor of vTO. Training was completed after 5000 epochs, where an epoch is defined 100 iterations. Each iteration, a batch of 1024 data and (pseudo-)label pairs are drawn from either labeled or an unlabeled dataset, with equal probability. If labeled data are sampled, the loss component Lemper (xe, y) is descended on. unlabeled data are sampled, the pseudo labels ý corresponding to the sampled x., are updated using Equation g before descending on the loss component Lsett (X., V). In the Metropolis- Hasttiggs update step, we set Le = 1 in equation] 7 and Lg in equation [9] Training was performed on a single Nvidia GeForce RTX 3080 GPU with 10 GB of VRAM; approximately 15 hours required to train for 5000 epochs.
217
218
219
220
222
224
226
228
4 Results and discussion
229
230
In the first experiment, Metropolis-CVAE is trained on online simulated labeled data x (x(x) as well as precomputed simulated data x,, ~ Pa.3 (x) with labels held out during training. Figure | A-G) shows the distributions of prediction errors by the trained network for each label. For each method, the prediction error is defined as the difference between the true label and the mean of the posterior samples. The labels have been normalized [0. 1). The histograms for the Metropolis-CVV CVAA with
232
233
234
235
100 posterior samples are more tightly clustered around zero than MCMC with either 100 or 3000 posterior samples denoted MCMC- 100 and MCMC-3000 - in cases except for one (Figure I
236
Figure H shows p~p plot: the fraction of posterior samples greater than equal to the true label, plotted against the cumulative distribution of p-values across the data set; the Metropolis
238
produces similar curves as MCMC-3000.
8

---

240
In the second experiment, a Metroppoiss-CAEEs similarly trained using labeled simulated X (x), but now with the unlabeled data X,, drawn from the MRI data sets Pu,, (x) and pu.2 For comparison, a second traditional CVAE is trained on the simulated data only. Figure 2 shows the acceptance rate of the proposed pseudo labels y' ~ Q(y') for each data set during training. As the Metropolis-CVAF continually learns from both data sets, we find that the network quickly enters a negative feedback loop in which the acceptance rate continually increases throughout training. Noteworthy is that the acceptance rate of ý converges to value between 0.6 and 0.7 for both datasets. The default target acceptance rate of the No-U-Turn sampler is 0.65, a value which originates from theoretical result pertaining to Hamiltonian Monte Carlo (HMC) 14 Theoretical work would be required to demonstrate a direct connection between Metropolis- -VAisss and the HMC result. However, this illustrates that the network neither accepts rejects more proposals than is typically desired. Figure B-C) shows the average Wasserstein distance between empirical label distributions from the Metropolis-CVAF with MCMC-100 and MCMC-3000, respectively. During training. the Metropolis-C -VAE quickly reaches minimum Wasserstein distances with respect to MCMC-100, Minima are reached with respect to MCMC-3000 similarly quickly. We can extrapolate from this result that the Metropolis-CVAt likely produces higher quality posterior samples than MCMC-3000.
241
243
244
245
247
249
251
253
254
255
256
This may be expected, as the Metropolis-CVAL updates its pseudo labels every training iteration, continuously performing MCMC throughout training using the MH update rule g
257
Figure B[A-G) compares the empirical distributions of traditional CVAE trained only simulated data with a Metropolis- -VAE. Histograms of the Wasserstein distances between the two networks and MCMC-100 do not show large differences, due to the low-quality of MCMC-100. Compared with MCMC-3000, parameters which are traditionally easier to infer, such as a and 33, show little difference
259
261
262
between the networks. The CVAE trained on supervised data alone shows significant deviation from
263
MCMC-3000 for several parameters, particularly 17, oj. and log c. This illustrates the ability of the
264
Metropolis-CVAE generalize well to the unlabeled test data, as unlabeled training data from P.. (x)
has been explicitly incorporated into its training process, Figure HH) shows quantile-quantile plots for distributions of all label samples across the data sets. All methods agree.
265
266
267
Limitations A limitation of this method is the requirement of a likelihood function which is fast to compute in order to perform the Setropolis-Hastinng update step g For physics models, computing the likelihood of a set of parameters often involves costly forward simulations of complex models. Therefore, computationally intensive models which require solving non-trivial integral or differential equations will not be suitable.
268
289
270
271
Another limitation is that the pseudo label sampling procedure may converge to stable local minimum which is far from the globally optimal labels. This is inherent to the MH update step; while MCMC methods often provide convergence guarantees in the limit of large numbers of update steps, one can not predict how many update steps this will require in practice. However, by training the Metropolis-CVAA on both simulated data generated from the prior space as well as real unlabeled data, we have not found this to be a practical concern.
273
274
275
276
277
278
Potential negative societal impacts This work describes a framework for ascribing labels unla- beled data. Potential malicious and unintended uses could occur if this framework were significantly extended beyond the MRI physics inference problem which we considered. For example, this method-
279
280
281
ology could be used to infer missing data generally, with the inferred data then presented under the pretense that it were true data. Further, we have not shown that our method of data inference for
282
283
unlabeled data is inherently fair, nor that the clinical MRI data under which the Metropolis-CVAE
284
model is trained on cannot be recovered from the network weights.

---

References
285
286
[1]
Udo von Toussaint. Bayesian inference in physics. Reviews of Modern Physics, 83(3):943-999 September 2011. doi: Mod///ee
https:/s:://inn aps org/doi/ Publisher: American Physical Society.
288
10. 10.1103/RewNodPhys 83. 943
[2] Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning Structured Output Representation using Deep Conditional Generative Models. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., December 2015. URL
290
292
nwurips. cctpp//e/ccc/pppp///111//// pdf
f 0110/665226602330772277700022203
294
[3] Diederik Kingma, Danilo Rezende, Shakir Mohamed, and Max Welling. Semi-Supervised Learning with Deep Generative Models. 1406.:440..220 [cs, stat], October 2014. URL
296
httt:///axiiv wtgg 5298 Xiv: 1406.5298.
297
[4] Hunter Gabbard, Chris Messenger, Ik Siong Heng, Francesco Tonolini, and Roderick Murray-Smith. Bayesian parameter estimation using conditional variational autoencoders for gravitational-wave astronomy, arXiv: (ate:::999002220 06296 [astro-ph, prysicx:gr-qc), September
296
300
2019. URL htttt://aaxxiii wttp:/o///////////// httt:://aarxv.orrr/aaa///10 06296
arXiv: 1909.06296.
301
[5] Nicholas Metropolis, Arianna W. Rosenbluth, Marshall N. Rosenbluth, Augusta H. Teller, and Edward Teller. Equation of State Calculations by Fast Computing Machines. The Journal of Chemical Physics, 21 1087 June 1953. ISSN
302
doi: 10. (166//11699) 14.
304
URL URI.htttss:/// https://aip sci
1063/1 1699114 Publisher: American In-
305
stitute of Physics.
308
[6] W. K. Hastings. Monte Carlo sampling methods using Markov chains and their applications. Biometrika. 57(1)997-109, April 1970. ISSN 0006-3444. doi: 10. 10.1093/biomeu/57..997 URL
307
308
309
[7] Thomas Prasloski, Burkhard Mädler, Qing-San Xiang, Alex MacKay, and Craig Jones.
310
Applications of stimulated echo correction multicomponent '2 analysis. Magnetic
311
Resonance in Medicine, 670)):18034114, 2012. ISSN 1522-2594.
doi: 10.
312
mrm.23117. URL U wilperrywwllee.. wiley. www//oi//// http//aa//....00 10020/...000/m000 20:00///mm... 23157 _eprint: https///onniiilllaayyyyyyyyyo.od 23100/000..0..000000....333.....
313
(8)
S. O. Rice. Mathematical Analysis of Random Noise. Bell System Technical Journal, 23(3): 282-332, 1944. ISSN 1538-7305. doi: 10. 153007/....3-7770.55. 000.287777 URL/https
315
316
//onlinelibrary. herpiibbrrrrwwlleyye. wiley. wttp:://////////o/////////////// 1002/j 111.:1000//.111-1011000- 1531- 1944. tb00874.
_eprint: http:///oiiiillaaooo...oooomolod
317
99 O. Rice. Mathematical Analysis of Random Noise. Bell System Technical Journal, 24(1): 46-156, 1945. ISSN 1538-7305. doi: 10. htt.5/5888777555999990000030335 URL hhtpps
319
//onlinelibrary. wiley wwt..//////oo///////e/co eprint: http://wwooiiiillooaa//////////p
1022100111900011118385553 1945. tbt04t300.... .x
321
[10]
H. Y. Carr and E. M. Purcell. Effects of Diffuusion on Free Precession in Nuclear Magnetic Resonance Experiments. Physical Review, 94()):30--638 May 1954, doi: 10. 10.1103/PhysRev.44
323
630. URL htttps://iikk aps www.aap........oo.oo.. www.oo/g///oooo//o///...... http://10...100//yhhshe 94. 630 Publisher: American Physical Society.
325
[11] S. Meiboom and D. Gill. Modified Spin-Echo Method for Measuring Nuclear Relaxation Times. Review of Scientific Instruments, 29(8):688-61,, August 1958 ISSN 0034-6748, doi: 10. 10.106//1.11112900 URL https://aaaaa htt.Attpss/aappscaaatioooooooro iewaatiioooooo//aooooooo httgg/doi/abbs. /00 176266
327
329
Publisher: American Institute of Physics.
330
[12] Kenneth P. Whittall, Alex L. Mackay, Douglas A. Graeb, Robert A. Nugent, David K. B. Li, and Donald W. Paty. vivo measurement of T2 distributions and water contents in normal human brain. Magnetic Resonance in Medicinee 37(1):34 1997. ISSN 1522-2594. doi: 10. wtt02//mm.191037001111 URL htttps://omlinellbrraa wiley wwt.doiooomaaeeeeeeeee abbs/10
331
333
1002/ mtt. 1910370107
10

---

335
[13] Matthew D Hoffman and Andrew Gelman. The No-U-Tum sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res., 15()):1593-1623 2014.
336
337
[14] Alexandros Beskos, Natesh Pillai, Gareth Roberts, Jesus-Maria Sanz-Serna, and An-
338
drew Stuart. Optimal tuning of the hybrid Monte Carlo algorithm. Bernoulli, 19 doi: 0..155/12-BB1144
339
(5A):1501-1534, November 2013. ISSN 1350-7265.
340
URL https://projecteuccae
341
Optimal Sesimal-tunnnge twnningninnnnnnnnnn of - the- -eeerhybdddd d Monte- Carlo algori 1ttp:/tth//mm..11//222..2. 15500011111111111..
342
full Publisher: Bernoulli Society for Mathematical Statistics and Probability.
Checklist
343
344
1. For all authors...
345
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes]
348
347
(b) Did you describe the limitations of your work? [Yes] See the Limitations paragraph of the Results and discussion section.
349
(c) Did you diseuss any potential negative societal impacts of your work? [Yes]] See the Potential negative societal impacts paragraph of the Results and discussion section.
350
351
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
352
353
2. If you are including theoretical results...
354
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
355
(b) Did you include complete proofs of all theoretical results? [N/A]
356
3. If you ran experiments..
357
(a) Did you include the code, data, and instructions needed reproduce the main ex- perimental results (either in the supplemental material or as URL)? [Yes] Code to reproduce the main results included with the supplemental material. The MRI data cannot be included, as it is medical data. However, the simulated data is provided in the supplemental material.
358
359
360
361
362
(b) Did you specify all the training details (e.g., data splits. hyperparameters, how they were chosen)? [Yes] See the Training subsection of the Experiments section.
363
364
(c) Did you report error bars (e-g., with respect to the random seed after running experi- ments multiple times)? [Yes] We provide entire histograms of relevant error metrics; see Figures and and 3
366
367
(d) Did you include the total amount of compute and the type of resources used (ype of GPUs, internal cluster, or cloud provider)? [Yes] See the MCMC data paragraph of the Data sets subsection of the Experiments section, as well as the Training subsection of the Experiments section.
388
389
370
371
4. If you are using existing assets code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [N/A]
373
(b) Did you mention the license of the assets? [N/A]
374
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
The trained models are supplied in the supplemental material.
378
(d) Did you discuss whether and how consent was obtained from people whose data you'l 're using/curating? [Yes] See the MRI data paragraph of the Data sets subsection of the Experiments section.
377
378
379
(c) Did you discuss whether the data you are using/curating contains personally identifiable information offensive content? [Yes] The MRI data used in this work is anonymized; see the MRI data paragraph of the Data sets subsection of the Experiments section.
381
382
5. If you used crowdsourcing or conducted research with human subjects...
383
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [No] This was a prospective study; this information was not available.
11

---

385
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [Yes] These MRI scans were acquired with approval from our university ethics board; see the MRI data paragraph of the Data sets subsection of the Experiments section.
386
386
389
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
12


Reviews:
[{"review_id": "utJ2Tv673C", "reviewer": "Reviewer_bwY8", "summary": "Variational inference and distribution sampling are two popular methods for statistical inference. The present paper combines these two approaches in a semi-supervised learning setting, where the distribution of unlabeled data may differ from that of labeled data. Experimental results are provided for an MRI dataset, where the task is to infer the imaging/tissue parameters.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Variational inference and distribution sampling are both powerful approaches, and combining them is likely to be impactful. Here, the authors present a reasonable and novel strategy for semi-supervised learning.\n\n- Overall, I find the method interesting, but experimental justification could be significantly expanded: (i) How would other semi-supervised methods, for instance the cited Kingma et al paper, perform on this task? (ii) Could the authors add 1-2 more tasks that are studied in previous semi-supervised learning papers?\n- Somewhat related to the previous point, how would this method extend to scenarios with discrete labels?\n- The authors already discuss sampling-related distributions of the proposed method. But, I think it would be valuable to put this in a better context: For some of the more popular applications in ML conferences such as image/audio analysis, language processing, recommendation systems, how applicable is this approach? It'd be ok if applicability is low for many tasks, but I believe such discussion would improve the paper.\n- I find the review of previous work somewhat missing. A better survey of the semi-supervised learning literature would significantly improve the better. I think the current flow converges to CVAE very quickly.\n- The presentation of the material is clear.\n\n----------------------------- UPDATE -----------------------------\n\nI like the main idea behind this paper. I also think that the improvement plan proposed by the authors sounds reasonable and interesting. However, considering that it is certainly possible for this set of experiments to reveal no significant practical improvements with respect to baselines, I decided to keep my score at its current level.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "7"}, "extra_text": {}}, {"review_id": "YP6txJ7TvgJ", "reviewer": "Reviewer_ba7Q", "summary": "This paper addresses semi-supervised learning problem. The authors propose to adopt Metropolis-Hastings Markov chain Monte Carlo (MH-MCMC) into the sampling process of posterior in conditional variational autoencoder (CVAE) learning.\nExperimental results on MRI datasets demonstrate that the proposed method performs better than existing methods regarding the number of samplings.", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "### Originality:\n* This work proposes a novel combination of CVAE for semi-supervised learning and MH-MCMC.\n\n### Quality:\n* The problem setting this paper addresses is not clear.\n    - The problem setting is finally described in the experiment section, so I cannot find whether it represents the scope of the proposed approach or the scope of the specific experiments conducted here.\n    - In Section 2.3, it would be better to focus more on the machine learning perspective.\n\n* The motivation to use MH-MCMC is not described in the main text.\n    - The authors do not specify a clear reason/purpose to introduce this combination of CVAE for semi-supervised learning and MH-MCMC.\n    - The specific combination would be novel, but there have been many combinations of other sampling methods including MCMC, and CVAE learning. Therefore, they should differentiate their approach from them.\n    - In Section 2.1 (related work), it would be better to focus on comparing the proposed approach and existing works.\n\n* In experiments, although the authors claim the efficiency of the proposed approach, they did not provide a comparison on actual computational time. Thus, it limits the significance of the analysis. (I acknowledge that they provided the analysis in terms of the number of sampling.)\n\n\n### Clarity:\n* The problem setting and the motivation of the proposed approach are unclear, as stated above.\n\n* Minner comments.\n    - What is the difference between $q$ in Eq.(1) and $\\hat{p}$ in Eq.(2)? Both seem to represent approximate posterior.\n    - In Eq.(10), $L_z$ is not defined.\n\n\n### Significance:\n* Unclear scope of this paper addresses degrades significance.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "They addressed the limitations and potential negative social impact of their work.", "review_text": "", "extra_scores": {"ethical_concerns": "NA.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "YNnpN0Z8Rhs", "reviewer": "Reviewer_QXph", "summary": "This paper introduces an online MCMC framework for semi-supervised learning with variational autoencoders. The idea of the paper stems from physics problems, where one typically wants to infer certain parameters of a data generating process from empirical data. An example of this is quantitative MRI. \n\nThe authors consider the model of data generating process known, and also assume a distribution mismatch between the simulated data, as well as the empirical data, which may include certain confounding factors that may be impossible to model. During the the training of a VAE, the authors progressively improve the psudolabels via a Metropolis-Hastings scheme. The key novelty of the paper is that the proposal distribution for the sampler is an posterior distribution $\\hat p(\\mathbf{y} | \\mathbf{x})$, modeled by the VAE itself.\n", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The most interesting part of this paper is a novel application of an existing machine learning result -- its adaptation to the MR physics problem. The release of the code also makes this work rather strong. The manuscript is well written. However, I believe that the experiments in the paper lack insights, rigor, and comparison to alternative methods.\n\nThings to fix:\n\nEquation (7) : please define what is $n$.\n\nLine 177: use \\eqref to refer to equation (14)\n\n\nExperimental design issues:\n\n-- Lack of baselines and proper comparisons. I believe that in order to truly prove the benefits of the proposed technique, one has to collect a sufficiently large dataset, and show that e.g. proposed methods for inference of quantitative MRI parameters boosts the metric of e.g. prediction of brain conditions. I would argue that acceptance rate of the sampler is not the most appropriate metric and substantially limits the level of experimental validation.\n\n-- In the current context, the authors have only MCMC baseline, but what about machine learning methods? E.g. one off-the-shelf semi-supervised baseline could be to train a simple regressor of parameters in a semi-supervised way using e.g. Pi-model or other techniques. \n\n-- How is the method generalizing from being trained on the first scan to the second scan? \n- One very critical issue that I see with in paper, is the convergence of the sampler under different domain shift levels. One could test this in a simulator, and then analyze when the method actually stops converging. E.g. visualizing Wassestein distance vs. the level of distribution mismatch would definitely add important insights\n\n- The penultimate issue I have with the experiments in the paper, is that the authors validated only one application: brain. qMRI is investigated in the knee imaging domain. I would propose the authors to strengthen this part as well. \n\n-- The final issue is that the method seems to work for any likelihood function. I would suggest the authors to evaluate it also on CIFAR10/100 classification benchmarks, and compare to classical VAE. The authors can conduct the distribution mismatch experiments there as well.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "This work does not have any negative societal impact.\n\nMy main points for improvements are summarized in the main review. Here is the summary of limitations:\n\n- Lack of SSL baselines\n- Lack of clinically-relevant evaluation\n- Generalization of the sampler is not investigated (from scan 1 to scan 2)\n- Narrow domain in evaluation (the method seems to be general though)\n- Lack of insights on distribution shift level\n- Only one application: one can also conduct experiments on image classification, and inference of qMRI parameters from knee images.\n", "review_text": "", "extra_scores": {"ethical_concerns": "N/A", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "Dp8Y2MxJDi2", "reviewer": "Reviewer_J7Vm", "summary": "The authors present a scheme for parameter inference that builds on both simulated/labelled data and real/unlabelled data.\nIt uses MCMC with a CVAE proposal distribution to infer the data labels, and then trains the CVAE on the mixed data, bootstrapping the process.\nThey evaluate their method on an MRI task.", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "# Overall review\n\nI really liked this paper and have tentatively recommended acceptance.\nThe main caveat in my recommendation is that I couldn't fully read Figure 3 - I have asked for clarification below.\nMoreover, because I like what you've done so much, it feels like a shame that you haven't done more to explore the opportunities of the algorithm more generally.\nI've left some thoughts on this below.\nWhether or not the paper is accepted in its current form, I would encourage you to explore this a bit further in order to increase the impact and significance of the work (I think you have a little extra space in the paper that could be used for this).\n\n# Major comments\n\nYour main contribution seems to be a strategy for inferring parameters for generative models in a label-efficient way.\nI like your overall strategy.\nI am mindful of the fact that I think my research background is slightly different than yours (just inferring from the paper), so some of our assumed context may differ.\nI've tried to account for this in my review, but we should be extra careful not to accidentally talk past each other, and if I seem to have misunderstood your points then please help me understand.\n\nI don't have the background to know if your interpretation of MRI imaging as mixing simulated labelled data with unlabelled real data is novel.\nIf it is you should make this more explicit, and if not you should cite the source.\n\nI cannot comment on the correctness of the work between lines 139 and 163.\n\nPersonally, I am most interested in the algorithmic contribution: the mixing of MCMC pseudo-data alongside simulated labelled data.\nHere, it feels like there are some design options that could benefit from more investigation and discussion.\n- What are the trade-offs in performing more than one MCMC step for each new label?\n- What heuristics should be used to balance the gradient-based optimization of the proposal distribution/inferred parameters against the MCMC label inference?\n- Are there advantages to 'warm-starting' the CVAE proposal in various ways?\n- Why do you randomly select labelled or unlabelled points rather than using a steady ratio between them? What might the best ratio be? Does this change during training?\n\nIt seems as though the impact of this work is much greater if the method can be generalized to other settings and data-sets.\nI think I'm sufficiently interested in the method already to tentatively recommend acceptance, but I think you could get much more out of this with more exploration of the algorithm itself.\n\nCertainly, the paper would benefit from an algorithm box.\nIf nothing else, this might force you to enumerate the design assumptions that you make in implementing the algorithm, and give you an opportunity to discuss alternative choices.\n\nCan you also say a little more about the hyperparameter choices made in S3.3? A number of these choices seem quite specific.\nIf you're just using something from a previous implementation that's absolutely fine, but it's nice to know what the origin of the choices is.\n\nThis comment is the most important for me: I cannot read Figure 3 and it seems the most important figure in the paper.\nThis is your opportunity to show that adding the unsupervised data improves your model, which is your main contribution.\nI think this could be improved just by switching colors---unfortunately the line-shape makes it very hard to distinguish dashed lines from full ones.\nI'm going to give benefit of the doubt for the moment that the results show the Metropolis-CVAE helping on at least a few of these.\nIf it is at all possible, please replace this figure with a more legible one during the review stage, as my provisional score is sensitive to this graph showing sufficient difference between the two methods.\n\nIf you can think of anything further that can be done to highlight performance differences between the CVAE and Metropolis-CVAE, these would be most likely to reaffirm my belief that this paper should be accepted.\n\n# Minor remarks\n\nThe abstract could be much shorter.\n\nDoes the use of two different data sources for the MRI data complicate things?\nHow do you deal with the mismatch?\n(Other than the masking - after all the voxels are different sizes etc.)\n\nI usually don't work with VAEs, and I understand that truncating the Gaussian is more usual in that domain.\nIt does, however, mean that the KL divergence estimate in equation (18) is wrong because the approximate posterior which is sampled from is not a Gaussian.\nI know most people ignore this, so I don't hold it against this paper, but it seems worth noting.\nCan I just check also whether you truncate by resampling or by clamping values greater than 1 down to 1?\nThese are quite different distributions.\n\nComputationally, checking each step whether you are using one loss or the other, then waiting for your CPU to return the pseudo-label, and then doing a backwards pass, is very bad (removes most of the advantage of GPU by blocking parallelization).\nI think it's beyond the scope of this work, but it would be valuable to consider possible parallelization strategies or ways to pre-compute the pseudo-labels in batches.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "These were adequately addressed.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper describes an algorithm reminiscent of the dynamics of an Monte-Carlo expectation-maximization (MCEM) algorithm where the E step is replaced by MCMC, here the classical MH algorithm. During the training of a VAE (similar to M step), the authors progressively improve the psudolabels via a Metropolis-Hastings scheme. (The derivation here seems to be pretty similar to and it might be informative if the authors would point out key differences of their approach.) One novelty of the paper is that the proposal distribution for the sampler is a posterior distribution modeled by the VAE itself. 

Overall, the reviewers seem to have liked the key idea but raise major concerns regarding the lack of sufficient experimental evaluation at this stage. I think this is a major concern and I encourage the authors to carefully address the reviewers extensive comments and submit to a future venue.


Decision: Reject