DeepGEM: Generalized Expectation-Maximization for Blind Inversion
Authors: Angela F Gao
Jorge Castillo Castellanos
Yisong Yue
Zachary E Ross
Katherine Bouman
Venue: NeurIPS 2021 Poster (2021)
Keywords: inverse problem
computational imaging
seismic tomography

Abstract: Typically, inversion algorithms assume that a forward model, which relates a source to its resulting measurements, is known and fixed. Using collected indirect measurements and the forward model, the goal becomes to recover the source. When the forward model is unknown, or imperfect, artifacts due to model mismatch occur in the recovery of the source. In this paper, we study the problem of blind inversion: solving an inverse problem with unknown or imperfect knowledge of the forward model parameters. We propose DeepGEM, a variational Expectation-Maximization (EM) framework that can be used to solve for the unknown parameters of the forward model in an unsupervised manner. DeepGEM makes use of a normalizing flow generative network to efficiently capture complex posterior distributions, which leads to more accurate evaluation of the source's posterior distribution used in EM. We showcase the effectiveness of our DeepGEM approach by achieving strong performance on the challenging problem of blind seismic tomography, where we significantly outperform the standard method used in seismology.  We also demonstrate the generality of DeepGEM by applying it to a simple case of blind deconvolution.

DeepGEM: Generalized Expectaton-MMxxinzatton for Blind Inversion
Angela F. Gao'
Jorge C. Castellanos
Yisong Yue!
Zachary E. Ross
Katherine L. Bouman"
"Computing and Mathematical Sciences, California Institute of Technology "Caltech Seismological Laboratory, California Institute of Technology (afgao, jcastellanos, yyue, zross, klbouman) @ caltech. edu
Abstract
Typically, inversion algorithms assume that forward model, which relates source to its resulting measurements, is known and fixed. Using collected indirect measurements and the forward model, the goal becomes recover the source. When the forward model is unknown, or imperfect, artifacts due to model mis- match occur in the recovery of the source. In this paper, we study the problem of blind inversion: solving an inverse problem with unknown or imperfect knowl- edge of the forward model parameters. We propose DeepGEM, a variational Sspectation-Maximie
(EM) framework that can be used to solve for the un- known parameters of the forward model in an unsupervised manner. DeepGEM makes use of a normalizing flow generative network to efficiently capture complex posterior distributions, which leads to more accurate evaluation of the source's s posterior distribution used in EM. We showcase the effectiveness of our DeepGEM approach by achieving strong performance on the challenging problem of blind seismic tomography, where we significantly outperform the standard method used in seismology. We also demonstrate the generality of DeepGEM by applying simple case of blind deconvolution.
Introduction
Physics-based inversion methods typically recover an unknown source from indirect measurements by assuming that the source and measurements are related via known forward model [5] 7 30 For example, non-blind deconvolution algorithms often assume that measured blurry image related to spatially-invariant invariant blur kernel [9] and traditional seismic inversion velocity of the Earth's interior known a priori when solving for an earthquake's 's hypocenter 33). However, these "known" forward models are generally idealized and ignore intricacies of the systems that are either hard capture or simply unknown. Inversion algorithms with forward model mismatch result in biased reconstructions. For instance, bias regularly seen non-blind deconvolution results, where reconstruction artifacts are often present due to the use of an incorrect blur kernel [
its true sharp image via a known spatially-
methods assume that the spatially-varying
In order reduce the effects of model mismatch, in this paper we tackle the problem of blind inversion: solving an inverse problem without knowledge of the underlying forward model parameters. When considering how learning can help, a natural first idea might be to learn direct map from measurements to the desired source via supervised learning. However, such a "model-free" approach generally not practical, due to the lack of available ground truth training data. For example, the blur kernel caused by handheld camera shake cannot be reproduced to get training set of
35th Conference on Neural Information Processing Systems (NcurIPS 2021),

---

Sharp-blurry pairs to train a deconvolution approach. In seismic tomography, synthetic earthquakes cannot be placed densely throughout the interior of the Earth to measure the ground response as a function of hypocenter location. An alternative approach, which we adopt, is to develop unsupervised methods that treat the true forward model as something that unobserved and must be inferred. An additional consideration is that we must solve for the true forward model from a single dataset, without knowledge of the true source that produces the measurements
In this paper, we propose Deep Generalized blind inverse problems. Using the indirect measurements as input, DeepGEM jointly estimates the source and forward model that together produce the observed measurements. DeepGEM is variational inference based framework that makes use of deep learning machinery to easily capture and optimize complex probabilistic distributions that cannot be easily integrated in analytic Expectation- Maximization (EM) solutions. Our proposed framework inversion problems described by differentiable forward models. In Section 4 we showcase the effectiveness of our DeepGEM approach on the challenging problem of blind seismic tomography. where we significantly outperform methods used in seismology. We also demonstrate the generality of DeepGEM by applying it to blind deconvolution in Section
(DeepGEM) for solving
generic and can be applied to blind
Travel Time Measurements (true source locs.)
Baseline
DeepGEM Velocity Recon. (recovered source locs.)
True Earth Velocity
Velocity Recon. (recovered source locs.)
5 km
(a)
(br
(c)
(d)
Velocity km/s
seconds
Occy km/s
Velecity km/s
Figure 1: DeepGEM applied to blind seismic tomography. (a) A simulated cross section of the Earth's interior (velocity structure), along with the locations of receivers on the surface (red triangles) that collect measurements. (b) The time it takes for a wave traveling from source below the surface to reach the specified receiver is visualized for each location in the region of interest. The overlaid dots represent the true locations of simulated earthquakes and indicate the measured travel times that constrain optimization. (c) The subsurface velocity reconstruction obtained using a baseline approach optimized with the help of seismologist. Note that the bright anomaly is missing from this reconstruction. Overlaid dots represent the inferred earthquake locations. (d) DeepGEM reconstructed subsurface velocity and inferred earthquake locations. Note that DeepGEM is able to accurately recover the gradient of the velocity field as well as partially recover the central anomaly.
2 Background and related work
The joint optimization of a forward model with source recovery is a very challenging ill-posed problem, leading to many possible solutions that are hard to disambiguate. For example, in blind deconvolution the blurry image observed can be equivalently explained by a sharp source image convolved with an extended kernel or blurry source image convolved with impulse kerne; to prefer one solution over another, additional information, such as image priors, must be considered. Previous work on inversion in poorly characterized systems focused on limited contexts, such as -patially-invariant blind deconvolution [1] 22 14 [0] and CT with a simple rotational error (35). These methods tend to be highly specialized to each application domain, and cannot be easily generalized. In contrast, we propose a flexible model-based Bayesian framework that can be applied across multiple differentiable blind inversion problems.
2.1 Model-based Bayesian inversion
In model-based inversion, unobserved sources x and observed measurements y are related through a forward model: y = f(x). When the parameters of the underlying model are unknown, one can parameterize the forward model as fo(x) and then solve for the true model parameters 0* common approach is to solve maximum posteriori (MAP) objective: either MAPp.e x or MAPp.

---

MAPp,> 0.2 solves for the optimal point estimate of the pair 2 that maximizes a joint objective:
{0, . arg max [log p(0, a0 x/yg] arg max [log (()(0,a) + log p(0) ()) log)(
(1)
6,8
6,8
In practice, formal probabilistic definitions of p(0) and p(x:) are often unknown and replaced with regularization terms (e.g., total variation, sparsity) [2 21. Although MA provides straightfor- ward approach to solve for 0, the energy landscape of Eq. T is typically poorly behaved due to the ill-posed nature of the problem [22]. As a result, optimization is likely to get stuck in a (bad) local minimum.
MAP, attempts to smooth the energy landscape by reducing the number of parameters that must be optimized. This is done by solving for parameters of the forward model, 8, that perform best under the full volume of possible interpretations:
0 = arg max [log p(0ly)] = arg max log (o(o))) + log log
(2)
Since this marginalization integral is often intractable, Expectation -otecattionnMaxxiiiaatioonn (EM) algorithms have long been used for solving MAP, efficiently [T. EM is an iterative algorithm that alternates between: "E"-Step) calculating the posterior of conditioned on the current estimated forward model parameters ((t-1); and M-Step) updating 0 to maximize the expected value of the log likelihood:
g arg max [[ax-p(elg.0t--1) [log =)] + log p(0))
The advantage of MAPo over MAP,,z optimization for the blind deconvolution problem was de- scribed in [22] and its success demonstrated via EM optimization in [21] However, it is important to note that evaluating the expectation in of Eq. (3) over complex distributions is often intractable. For instance, the authors of [21] were forced to restrict the posterior distribution to a Gaussian distri- bution. Alternatively, stochastic EM methods [3 g bypasses the need to evaluate the expectation directly, approximating it by sampling the distribution. In this paper, we solve MAP, using complex distributions parameterized by deep neural networks.
Forward model parameterization Model-based inversion requires that the parametric form of fo(r) is well matched with the true forward model, which is not always known, Alternatively, neural networks can be used to approximate the forward model, where represents the network weights. This setup is flexible, in that it can be used to approximate arbitrarily complex forward models, with the downside that it is often not interpretable when the parameters have no physical meaning. In this work, we develop and make use of interpretable, physically-motivate deep neural networks to parameterize (.). for the problems blind seismic tomography and blind deconvolution.
2.2 Deep variational distributions
We are interested in solving blind inverse problems using the EM algorithm to optimize the MAP, objective. Optimizing in this fashion requires the use of the posterior distribution p(zly, ((t-1)) in evaluating Eq. 3 As inverse problems are often ill-posed, we expect that the posterior distribution of source x conditioned on the forward model parameters 0 is likely to be complex and perhaps even molti-modal. Therefore, it is important to able to parameterize a flexible family of distributions to best estimate this conditional posterior.
Deep Probabilistic Imaging (DPI) 32 uses a normalizing flow-based generative model, G.))). solve for the uncertainty of an unknown source v given a fixed forward model f(x) and measurements y. DPI solves the variational objective:
$ = arg min KL (=))))(((()))) = arg min Ex-oy (x) log p(yk()) - log p(x) + log q (x)],
(4)
where (x) is the implicit distribution defined by (2) for Rlu (o 1 and log p(y|z) x ll - f(x)]l2 2 + e when there exists i.i.d. Gaussian noise on the measurements, y. After inference, (z) can be efficiently sampled by evaluating Gz(2) for Gaussian sample 2.
The DPI variational objective is equivalent to the Variational Autoencoder with a fixed decoder, fxr) In practice, vanilla VAEs constrain the posterior to be Gaussian
objective, except
|o| is the dimension of
3

---

distribution, relying on the reparameterization trick for tractable optimization. Alternatively, DPI uses flow-based networks to efficiently sample and directly compute go(x) [3 T8. DPI's use
of flow-based network allows for complicated and multi-modal posterior distributions constrained
only by the space of possible invertible network architectures. Our proposed DeepGEM approach
utilizes similar tools to model flexible distributions over 2, while simultaneously learning the forward
model parameters
3 Methods
We propose a deep variational EM approach (DeepGEM) that optimizes the MAP, objective in Eq. 2
to recover the parameters of forward model fo(r) using only single static set of measurements y. Once learned, the updated forward model can then be used to estimate the posterior distribution of
the unknown source, x. DeepGEM iterates between two stages that are inspired by the standard EM algorithm: (1) an -step that learns a variational distribution, Jacu (z:,, to approximate the posterior distribution of x given the current forward model parameters ((t-1), and (2) an M-step (refer to Eq. 3 that solves for ((() that maximizes the expected value of the log likelihood function of f with respect the posterior distribution Ioco (z) estimated in the prior "E"-step. Each step alternated and solved to convergence. Since this is an EM algorithm (up to the variational approximation), our
method inherits properties EM, including convergence to a local minimum of Eq.
3.1 "Expectation" step ('E"-step)
The goal of DeepGEM's "F""-""""""""t"""""""""" "E"-step is to solve for the posterior distribution p(rly, ((t-1) that facilitates optimizing Eq. 3 Because this posterior distribution can be very complex, and even multi-modal, we propose to use flexible variational approach to learn the parameters ø of an approximate posterior distribution 4o (z). The variational distribution qo(x)) can then be used to evaluate Eq. via efficient sampling.
Using DPI we solve for a flexible variational distribution, (x(x) that well approximates the posterior distribution plaly, 0(-1)), DPI uses normalizing flow network, Gø(z), with input = E RR where (z(z) (2( when ~ (0, 1). Normalizing flow networks allow for exact computation of the log-likelihood log qx(x) needed to solve
((()) arg min (L(gg()))l((th) (1(()))))))(( o(c-1)))
N
1
log p(g)()))( 1x,) log p(z.) + log (x())
2 arg min
N
for En = GZ(z))
2n ~ N0,
(5)
(as derived from Eq. 4 for batch size of N where log p(x) is prior on the source and log p(yl ax, o(*)) the data likelihood. When assuming the measurements experience i.i additive Gaussian noise with standard deviation o,, log 1og (((), xn)) 207 |y fac (Ix)) +
3.2 Maximization step (M-step)
The goal of DeepGEM's M-step is to the parameterized approximate posterior distribution, Jate (z), from the prior "E"-step to update 0, the parameters the unknown forward model fo(-).
This is achieved by sampling from the learned normalizing flow network, Goto (-), to stochastically
solve Eq. 3
N
( arg max
Σ
Dogp(y|0,xa))
+ log logr(0) for In G (zn); 2n NN(0 (6)
N
where p(0) is prior on the forward model. This prior can be used to encourage the forward model parameters to remain close to initial model 0 by defining log p(0) ||0 0|12 + c.

---

4 Case study: blind seismic tomography
Two fundamental seismic inverse problems are spatially localizing an earthquake's hypocenter (also referred to source localization) and tomographic reconstruction of the Earth's subsurface [3). These problems are interconnected: source localization relies on knowing how fast waves propagate through different regions of the Earth's interior, referred as the Earth's velocity structure. However, in standard seismological practice, due to difficulties in solving these problems jointly, they are generally treated separately: source localization is performed initially using oversimplified velocity models 30. and then the tomography problem is performed by taking those best-fitting hypocenters as ground truth (66). This approach typically results in the need over-regularize the inverse problem by smoothing out high-freen frequency information [2], and can only be improved by carefully incorporating other forms of information such as waveform-derived quantities []6 23 24 or by performing costly experiments such as controlled explosions. In contrast, we demonstrate our DeepGEM approach on blind seismic tomography, solving for the subsurface velocity (parameterized by 8) when the source hypocenters, 2, are unknown. Measurements, y. used to constrain the inverse problem are the time it takes for the first wave to propagate from its source to a receiver on the Earth's s surface, referred as travel time measurement (refer to Fig. T
4.1 Seismic tomography background
The earthquake source location, also called the hypocen- ter, is the location where the earthquake nucleates [30]. The source location can be triangulated using travel times from multiple receivers. However, when there are very few receivers (< 3D, in 2D), the source localization is ill-posed and there exists multiple equally optimal solutions.
Physics of earthquake source localization:
Physics of travel time tomography: Travel time tomography is method for reconstructing the velocity structure using the absolute arrival times of earthquake waves from the earthquake to the receiver (30. Given perfect knowledge of the earthquake locations æ and receivers r at every position in the ground, the exact velocity can be computed by solving the Eikonal equation V(r) = [V,T(a, r) where T(x,r' r) is the travel time from an earthquake to a receiver. There exists an analytical solution to the Eikonal equation; however, seismologists often use simplifications, such as straight ray tomography, for efficiency [2].
implicitly solves the Eikonal equation 33 by learning mapping from source-receive pair (I, r) to its associated travel time, learning 0 such that fo(x,r) r) Rs T(x, r) where T(x, r) is the true travel time. The velocity structure can then be extracted from the learned EikoNet by solving the Eikonal equation through automatic differentiation.
Deep Learning for travel time tomography: EikoNet 31
In particular, the computed velocity Vsx) at location s is |/V se, ())2 s) Note that this computed velocity depends on the source location æ. Idcally, the velocity should be invariant to the source location; however, in practice, this is only true when EikoNet is trained with semsely-sampled (x, r) pairs. Additionally, the travel time from a source to a receiver, T(x, r), should be identical to the travel time from receiver source, T T(r, 2),, but remains unconstrained by EikoNet.
DeepGEM lg Prior Models
Earthquake Source Localization Visualizations
Homogenous
Gradient
Layer
Rece verls)
True source
Prior mean of source
10
Posteriar conditioned on
Figure 2: (Left) A visualization of the homogeneous, gradient, and layer models used in p(0)'s Ca term. (Right) Visualizations used to describe source configurations and earthquake source posteriors. (a) Visualizations of the true and initialized source locations are plotted as stars (true) connected to circles, which indicate the expected source locations according to p(x:). Note that the expected source locations deviate significantly from the true locations. (b) Visualizations of the learned posterior distribution, qo orly, 8), for each source are plotted as colored histograms and overlaid with stars (of the same color) indicating the true source location.
5

---

Known
DeepGEM with Uncertain Source Locs.
Baselines
True Prior
Source Locs.
No Prier
Prior
Homogeenous
Gradiint
Layer
w/ Gradiennt
we Cracieent In
Straight Ray
sourcss
1.19
1.428
25 sourdes
0 3.5
0.361
1.020 0.303 0.177 2.275
6.967
Figure 3: DeepGEM reconstructions significantly outperform baselines, and improve with more sources and better Lo prior models. Each row corresponds to the reconstructions obtained from single noisy observation of travel times, where the true Earth velocity is shown in Fig. aa). Results shown are simulated using 20 surface receivers and varying number of sources (9, 25, and 49) in a uniform grid. Columns 3-6 show DeepGEM results obtained using different Lo priors. Note that results improve as the Le prior becomes closer the true velocity structure, and as the number of sources increases. As reference, column 2 shows the velocity reconstruction obtained using DeepGEM under fixed, perfectly known source locations. Columns 7-8 show results obtained by the baseline approaches. The velocity reconstruction MSE is included in the top right of each reconstruction. DeepGEM substantially outperforms both straight ray and MAP baselines. Note that DeepGEM also sometimes outperforms the second column, where the source locations are known; this due to the fact that DeepGEM is sometimes able absorb measurement error due to its overparameterization See the supplemental material for more examples.
4.2 DeepGEM setup for blind seismic tomography
For blind seismic tomography, we parameterize the forward model using modification of EikoNet, fo(2, rr), with unknown source location x and known receiver location as inputs and y Rs T(r, r), the absolute travel time between æ and r, as output. In order to solve Eqs. 5 and 6 for an updated forward model, we must define priors p(æ) and p(). The prior source locations, p(z), is often well defined, typically a Gaussian distribution N [z, ow)) with a standard deviation of 0g km and mean T.
We construct prior over the forward model, p(0), that encourages EikoNet learn velocity close (s). Additionally, as discussed above, there are constraints specific seismic tomography that are not explicitly enforced through EikoNet's architecture: (1) velocity reconstruction invariance with respect to the source location, and (2) travel time symmetry between sources and receivers. We augment the prior p(0) to include these constraints, implemented as soft constraints:
Lo logr(0) A9 Σ (1) (
Lv
CT
Σ ERRyER.
[|V+.()) Vo,(s))1z +DT Σ [T((s, r) T(r, s))2.
PER
VER.
#ES
#5S
The velocity constraint represented through Lv and travel time constraint through Lr, with corresponding hyperparameters bv and År. S is of points sampled uniformly from the region of interest, R is the set of all receiver locations, and (s)s [
define realistic prior over unknown source km. We assume the measurements y are sampled from Gaussian distribution with mean ğ true travel times computed using the package eikonalfm [29 33 and realistic standard deviation of o, = 0.2 seconds. To simulate real world passive tomography, we assume receivers are located only along the surface (the top edge of the 2D image) of the region of interest, which is 20 km x20 km, and sources can be anywhere within the region of interest. Before DeepGEM optimization, we first initialize using maximum likelihood estimation assuming is the true source location.
Implementation details: In our experiments
locations as p(x) (ā, oz). where F ~ ((x, σz) and

---

Source Locs.
Source Locs, 2
Source Locs.
Source Locs.
Source Locs, 5
Table Velocity and source
Source LocS w/
0.330
0.364
0.355
0.326
3.303
localization error on random
Known
Gradient E,
source configurations
Velocity
Source
Source Count
Error
Loc, Error
Source Locs N/
0.305
0.795
0.375
(km/s)
[m)
Uncertain
Giodient E.,
0.52
62.88
0.063
21.77
0.42
36.62
25
0.043
4.93
0.44
33.78
Source Locs. Posterior
49
0.181
4.56
0.27
30.38
100
0.021
1.43
Figure DeepGEM consistently recovers prominent features across various earthquake source configurations. Row 2 shows DeepGEM results obtained using different random source configurations, where the true Earth velocity is shown in Fig. Ta). As reference, row 1 shows the velocity reconstruction obtained using DeepGEM under fixed, perfectly known source locations. As can be seen by the table, both velocity and localization error decrease with an increasing number of sources in the region of interest. Each mean and standard deviation is computed using 5 random realizations true source configurations.
The posterior distribution of the source locations, is estimated using Real-NVP network G.()) with 16 affine coupling layers. An updated EikoNet (described in the supplemental material) has been modified to parameterize fx()). This EikoNet is pretrained with samples from the prior p(z) ) as input and the simulated travel time measurements as output. We use Adam as the optimizer [36] with a batch size of 32 and E-step learning rate of 1e-3 and M-step learning rate of 5e-5. Hyperparameters (rr, Xy. Ao) were empirically chosen by inspecting the loss on grid search over hyperparameters. Results presented have been with 10 EM iterations, each with 800 "E"-step epochs and 2000 M-step epochs. Each DeepGEM model takes ~~6 hours on a NVIDIA Quatro RTX 5000.
4.3 Results
Comparison to Baseline Approaches: We compare results from DeepGEM results obtained using a baseline run by seismologist. This iterative baseline alternates between source inversion and straight ray tomography, and standard approach used for blind tomography. Further detail on this baseline is provided in the supplemental material. The gradient model shown in Fig.2li used to perform the initial source inversion for this baseline; nonetheless, we find that the solution quickly diverges. Therefore, the optimization. As seen in Fig. DecpGEM consistently outperforms this human-in mamnann---thh-looo tho-loop baseline across all source configurations. In Fig. 3 we also compare with a MAPa,a solution. MAPs,> consistently outperformed by the DeepGEM MAP, approach across all source configurations. Additional results highlighting the sensitivity of MAPa,z to hyperparameters and initialization shown in the supplemental material.
rely the expertise of the domain expert decide when to terminate
We evaluate DeepGEM's recovery of the true velocity structure shown in Fig. I Jusing one of three different Lo priors shown Fig. 2 homogeneous, gradient, and layer, well as Lo 0. The homogeneous model takes on value of 6.419 km/s, the average velocity value the true velocity structure. The gradient captures the increasing velocity as depth increases, and the layer model represents the true model without the added anomaly. As shown in Fig, 3 the mean squared error tends to decrease with the gradient and layer La prior models, which are closer to the true velocity structure.
Sensitivity to Lo Prior Choice:
Sensitivity to Source Configuration: We evaluate results with sources that are both uniformly and randomly spaced throughout the region of interest. Improved performance is expected when the number sources increased and/or when sources are well distributed. To better understand DeepGEM's performance we introduce an ablation, where the velocity structure is learned by training fo(.) with access to perfect source locations x (i.e., p(x:) is delta function). As shown in Fig 3 even when training with true source locations, the anomaly is not well resolved until ~49 sources.
7

---

100 Sources
961 Sources
0.250
2.496
1.511
source posterior
0.586
Sources:
No
Meas, Noise:
Yes
Yes
Yes
Figure 5: DeepGEM recovery with a single receiver. Velocity reconstructions shown in columns and 5 demonstrate that DeepGEM is able to learn some of the true velocity features (see Fig. D even when limited measurements from a single receiver. However, these reconstructions show clear signs of overfitting to the measurement data. This is demonstrated by observing that the DeepGEM reconstruction with perfectly known source locations significantly better with single receiver when measurement error is included the measurements (comparing columns I and 2). Note that the learned posterior column 4 appears to be non- Gaussan,,,,, but not multimodal.
a)),
True Prior
Source Locs.
Truth
No Cgg. Prior
Gradient Lg
0.366
0.865
Table 2: Velocity and source localization error on random velocity fields
Model 1
Velocity Error (km/s)
Source
Lg
Loc. Error
0.264
0.654
(m)
Model 2
Nome Known
0.57
Source Locs.)
0.25
0.51
24.40
None
0.30
7.18
0.94
21.37
0.243
0.554
Homagenous
0.45
5.34
Model 3
1.07
25.20 1
Gradient
0.53
6.46
Figure 6: Performance of DeepGEM recovery on random velocity fields. Ten random velocity fields were drawn from a GRF-based distribution and used to simulate travel time measurements with 20 receivers and 100 randomly placed sources. (left) Reconstructions obtained for of these configurations are shown. (right) A table lists the mean and standard deviation of velocity and source error obtained across the ten models, each recovered using different Cg priors.
Fig. 3 shows realization of DeepGEM results obtained from different counts of uniformly spaced sources. As expected, the MSE between the reconstructed and true velocity structure tends to decrease the number of sources increases. Fig. 4 shows results obtained using five randomly generated source configurations, with 49 sources each. These results demonstrate that, although the reconstructed velocity structure is somewhat sensitive to the underlying source configuration, the primary features of the true velocity can still be recovered in all cases. Velocity and source shown in the accompanying table.
localization obtained for different random configurations
Sensitivity to Number of Receivers: In the case of a single receiver, there exists an entire ring of source locations that result in the same travel time measurement. Fig. 5 contains results from this challenging one-receiver setting using DeepGEM with/without measurement noise and with/without known source locations. Since there is only overfit. However, perhaps surprisingly, artifacts are more severe when source locations are perfectly constrained. These artifacts caused by the velocity model overfitting noise in the travel time measurements, and are substantially reduced when noise-free travel time measurements are used. Note that the recovered source location posterior (x()) obtained by the "E"-step is non-Gaussian.
receiver, the velocity model is able to easily
Sensitivity to Velocity Structure: Fig. DeepGEM tested on randomly generated velocity fields, each generated from Gaussian random field (GRF) described in the supplemental material.

---

These results show that DeepGEM works well at recovering the primary features across a variety of different velocity structures. As compared when L. 0, the La gradient prior biases the reconstruction towards smoother velocity structures. The accompanying table contains error statistics for ten different randomly generated velocity fields.
5 Case study: blind deconvolution
We apply DeepGEM the problem of blind deconvolution to further demonstrate the generality of our approach. Blind deconvolution is a classic ill- ill-posed imaging problem that aims to reconstruct
sharp image from blurry image with an unknown PSF 19 25 21
22 20 6 27]. Blurry
images, caused by handheld camera shake, can be modeled using single spatially invariant blur kernel:
kn + for E ~~ (0,,0,,)
(8)
where y is the blurry image, represents a 2D convolution, x the true sharp image, ko is the spatially invariant blur kernel, and E is additive Gaussian noise.
5.1 DeepGEM setup for blind deconvolution
For blind deconvolution, we parameterize the forward model fe(-) using a deep network consisting of multiple convolution layers without non-linear activation, as proposed in [3]. Multiple convolutional layers without activation simply overparameterizes linear blur kernel, which has been empirically shown to produce multiple good minima that are easier to converge to. To ensure the blur kernel is non-negative and volume preserving. we use a Softmax layer and normalize the kernel. For an layer network with weights for I,... n. the resulting parameterized forward model is:
Softmax (01 02 ...
fo(x) kg
Softmax (0 02 ... ...0 O,)
Reconstructed Image and Blur Kernel using DeepGFM
True Sharp Image and Measured Blur Kemee
Reconstructed Image using Measured Kernel
Measured Blurry Image
Figure 7: DeepGEM results are comparable to inversion with known blur kernels for simple Fashion MNIST images. Blurry measured images (column are generated using the true sharp image and blur kernel (shown in column 2). The deconvolved images obtained using the true blur kernel are shown column Reconstructed deconvolved images and corresponding inferred blur kernel from DeepGEM are shown in column 4.
Implementation details: We demonstrate DeepGEM using simple Total Variation (T') regulariza- tion place of p(x). We assume Gaussian prior on the noise on the blurry measurements where (0, 0.01) as well sparsity prior on the reconstructed kernel through an lo.s soft constraint.

---

The posterior distribution of the sharp image, is estimated using Real-NVP network Gp(+) with 16 affine coupling layers. We use 5 convolution layers to parameterize kg. We use Adam as the optimizer [36] with batch size of 64 and an E-step learning rate of 5e-4 and M-step learning rate of le-4. Hyperparameters, weights used for sparsity and TV priors, were empirically chosen by grid search over hyperparameters. Results presented have been run with 10 EM iterations, each with 400 "E"-step epochs and 400 M-step epochs, which takes ~ hour on a NVIDIA Tesla V100.
5.2 Results
In Fig. 0 we show results from DeepGEM on three different Fashion-MNIST 34 images. The Fashion MNIST images are 64x64 pixels in size and each blur kernel contained within 15x 15 pixels. We choose to use images from Fashion-MNISS because they are compatible with the TV prior used in place of log p(x). ). In contrast, other natural images such as images from [22] (which were originally curated for deconvolution) are not compatible with this handcrafted prior. While this highlights that handcrafted priors such as TV are often insufficient, we choose to leave this to future work. Please refer to the supplemental material for more details about the performance of TV prior on different types of images.
The blurry images in the first column of Fig. exhibit artifacts from the blur kernel in the form of repeated features, caused by the bi-modal blur kernel. The images recovered via DeepGEM are much sharper and are comparable to the images reconstructed with the known blur kernel. Additionally, DeepGEM reconstructs blur kernels (i.e., kg) that are similar to the true kernel shape, with two lobes along the same diagonal. Note that even when there is perfect knowledge about the shape of the blur kernel, detailed structures on the images are not recovered due to the TV prior.
6 Conclusions
In this paper we present DeepGEM, deep probabilistic framework for tackling blind inverse problems through estimation of the forward model. DeepGEM achieves strong performance in the task of joint seismic tomography and earthquake source localization, substantially outperforming standard approaches currently being used in seismology on synthetic data. The proposed framework flexible and can be applied to different applications that require estimation or fine tuning of forward model parameters. We demonstrate this flexibility by also applying the approach to a simple, but challenging, blind deconvolution problem. Future work includes applying this method real seismic data, extending to other applications, and incorporating aatt-d priors. Our results highlight the benefit of blending physically sound model-based techniques with learning machinery for blind inverse problems.
Broader Impact
DeepGEM can be used to solve for system's model mismatch, which can then help improve our understanding of complex physical systems. However, this tool is not trustworthy enough for safety-critical systems. Nonetheless, this approach can benefit society through a better understanding of fundamental science and advanced earthquake prediction models via seismic imaging.
Acknowledgments and Disclosure of Funding
This research was carried out at the Jet Propulsion Laboratory and the California Institute of Technol- ogy under contract with the National Aeronautics and Space Administration and funded through the President's and Director's Research and Development Fund (PDRDF). This work was sponsored by Beyond Limits, Jet Propulsion Laboratory Award 1669417, NSF Award 2048237, and generous gifts from Luke Wang and Yi Li. Additionally, we would like to thank He Sun for many helpful discussions. We declare no competing interests.
References
[1] Raied Aljadaany, Dipan K Pal, and Marios Savvides. Douhlos-rachford networks: Learning both the image prior and data fidelity terms for blind image deconvolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages
2019.
[2] M. P. Barmin, M. H. Ritzwoller, and A. L. Levshin. A Fast and Reliable Method for Surface Wave Tomography, pages 1351-1375 Birkhäuser Basel, Basel, 2001.
10

---

[3] Sefi Bell-Kligler. Assaf Shocher, and Michal Irani. Blind super-resolution kernel estimation using an internal-gan. CoRR, abs/1909.06588 2019.
[4] Ebru Bozdag, Jeannot Trampert, and Jeroen Tromp. Misfit functions for full waveform inversion based on instantaneous phase and envelope measurements. Geophysical Journal International, 185(2):845 870,
2011.
[5] Olivier Cappé and Eric Moulines. On-line pepectationmmaxxiizzziin algorithm for latent data models. Journal of the Royal Statistical Society; Series B (Statistical Methodology), 71(3):593-00 613, Jun 2009.
[6] Ayan Chakrabarti. A neural approach to blind motion deblurring. In European conference on computer vision, pages 221-235. Springer, 2016.
[7] Chengqian Che, Fujun Luan, Shuang Zhao, Kavita Bala, and Ioannis Gkioulekas. Inverse transport networks. arXiv preprint arXiv: 180..000000000 2018.
[8] Jianfei Chen, Jun Zhu, Yee Whye Teh, and Tong Zhang. Stochastic expectation maximization with variance reduction. In NeurIPS, pages 997887988.. 2018
[9] S. Cho, Jue Wang, and Lee. Handling outliers non-blind image deconvolution. In 2011 International Conference on Computer Vision, pages 195-502, 2011.
[10] Sunghyun Cho and Seungyong Lee. Fast motion deblurring. In ACM SIGGRAPH Asia 2009 Papers, SIGGRAPH Asia '09, New York, NY, USA, 2009. Association for Computing Machinery.
[11] Arthur P Dempster, Nan M Laird, and Donald B Rubin. Maximum likelihood from incomplete data via the algorithm. Journal of the Royal Statistical Sociery: Series B (Methodological), 39(1):1-22, 1977.
[12] Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components estimation, 2015.
[13] Laurent Dinh, Jascha Sohl-Dicksteii and Samy Bengio. Density estimation using real nvp, 2017.
[14] Rob Fergus, Barun Singh, Aaron Hertzmann, Sam T. Roweis, and William T. Freeman. Removing camera shake from a single photograph. In ACM SIGGRAPH 2006 Pupers, SIGGRAPH '06. page 787-794, New York, NY, USA, 2006. Association for Computing Machinery.
[15] Jeffrey Fessler. Model-based image reconstruction for mri. JEEE signal processing magazine, 27 89, 2010.
[16] Andreas Fichtner, Brian LN Kennett, Heiner Igel, and Hans-Peter Bunge. Theoretical background for continental-and global-scale full-waveform inversion in the time-frequency domain. Geophysical Journal International, 175(2)x665-685, 2008.
[17] Diederik Kingma and Max Welling, Auto-encoding variational bayes. arXiv preprint 1r1iv:1312.6114. 2013.
[18] Durk Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible Ixl convolutions. In Advances in neural information processing systems, pages 10215-10224, 2018.
[19] D. Kundur and D. Hatzinakos. Blind image deconvolution. IEEE Signal Processing Magazine, 13()):33644
1996.
[20] Orest Kupyn, Volodymyr Budzan, Mykola Mykhailych, Dmytro Mishkin, and Jifí Matas. Deblurgan: Blind motion deblurring using conditional adversarial networks. In Proceedings of the IEEF conference on computer vision and pattern recognition, pages 8183-8192, 2018.
[21] A. Levin, Y. Weiss, F. Durand, and W. T. Freeman. Efficient marginal likelihood optimization in blind deconvolution, CVPR 2011, pages 2657-2664, 2011.
[22] Anat Levin, Yair Weiss, Fredo Durand, and William Freeman. Understanding and evaluating blind deconvolution algorithms. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages JEEE, 2009.
[23] Jingrui Luo and Ru-Shan Wu. Seismic envelope inversion: reduction of local minima and noise resistance. Geophysical Prospecting, (3(3):597-614 2015.
[24] Yi Luo and Gerard T Schuster. Wave-equation traveltime inversion. Geophysics, 56(5):645 653, 1991.
11

---

[25] D. Pertone and P. Favaro. Total variation blind deconvolution: The devil is in the details. In 2014 IEEE Conference Computer Vision and Pattern Recognition (CVPR), pages 2909-2916, Los Alamitos, CA. USA. jun 2014. IEEE Computer Society.
[26] Nicholas Rawlinson, S Pozgay, and Fishwick. Seismic tomography: window into deep earth. Physics of the Earth and Planetary Interiors, 178034):101-135, 2010.
[27] Dongwei Ren, Kai Zhang, Qilong Wang. Qinghua Hu, and Wangmeng Zuo. Neural blind deconvolution ing deep priors. Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 3341 -3300, 2020.
[28] Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and ap- proximate inference in deep generative models. In International conference on muchine learning, pages 1278-1286, PMLR, 2014.
[29] J. A. Sethian. Fast marching methods. SIAM Review, 41(2):199-22 1999.
[30] Peter M. Shearer. Introduction seismology. Cambridge university press, 2019.
[31] Jonathan D. Smith, Kamyar Arizzadenesheli, and 7achary E. Ross. Eikonet: Solving the eikonal equation with deep neural networks, 2020.
[32] He Sun and Katherine L Bouman. Deep probabilistic imaging: Uncertainty quantification and multi-modal solution characterization computational imaging. Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 2628-2637, 2021.
[33] Eran Treister and Eldad Haber. marching algorithm for the factored eikonal equation. Journal of Computational Physics, 324:210-225, 11 2016.
[34] Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv 2017.
[35] Mingyang Xie, Sun, Jiaming Liu, Brendt Wohlberg, and Ulugbek Kamilov. Joint reconstruction and calibration using regularization by denoising, 2020.
[36] Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander Smola. Dive into deep learning, 2021.
12


Reviews:
[{"review_id": "zFkPynmy7XA", "reviewer": "Reviewer_FVfb", "summary": "This paper proposes a general variational Expectation-Maximization framework aided by a normalizing flow generative network to solve inverse problems for which a forward physical model is known, but important model mismatch exists. The approach is evaluated on a blind seismic tomographic task and showed to quantitatively and qualitatively outperform a state-of-the art baseline through various experiments, that include ablation studies. The versatility of the framework is also illustrated by a small case-study on image blind deconvolution, where the authors report convincing qualitative results on two images.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This is a very good paper. The addressed topic is timely and relevant, the methodology is novel and technically sound, the paper is clearly written, experiments are well designed and the reported results are good and convincing. Some important clarifications are nevertheless required.\n\nSome clarifications needed in section 3:\n- In both eq. (5) and (6) the authors appear to be using an approximation of the expectation via a form of re-parameterization trick (as coined in the original VAE paper by Kingma and Welling), but this is not explicitly stated.\n- Some derivations are missing to explain how the log-likelihood is \"exactly computed\" (line 131)\n- More generally, it is not clear how (5) and (6) are solved in practice, some derivations should be provided.\n- On lines 114-115, the authors explain that the MAP objective will be optimized using \"only measurements y\". It is not clear at this stage and only becomes clear later that a *single* measurement y is used. This is what allows the authors to remove the dependency on y from q_{\\phi}[x) on line 127 and later (otherwise it does not make sense). Please clarify this.\n\nSome clarifications needed in section 4:\n- Lines 201-209: How many parameters do the RealNVP and EikoNet contain? How does this tie in with the number of available observations, which seem relatively small? More generally, did the authors use training/development sets (in particular to tune hyperparameters) and how were they design?\n- For both Fig. 3 and 4: How do you explain that errors get lower as the number of sources increases? shouldn't it be the opposite, as the problem is less and less well-posed? Or is it because of the strong provided priors? please comment on that.\n- Line 247: the authors write that the obtained posterior is non-Gaussian, but is it multimodal? (unclear from figure).\n\nTypos:\n-Line 87: can be used _to_ approximate\n-Line 265: emperically -> empirically\n-Line 294: by also applying the approach to the simple, but challenging, blind deconvolution problem -> to _a_ simple ...\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors correctly address the limitations and societal impact of their work.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2,5"}, "extra_text": {}}, {"review_id": "iSMXfSzpBOT", "reviewer": "Reviewer_xa1o", "summary": "This paper proposes to use variational EM algorithm to simultaneously recover forward model and latent variable in an inverse problem. The proposed method is extensively evaluated on seismic tomography.", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- Novelty: the proposed method consists of using neural nets to parametrize forward models and use variational EM to solve for parameters. Both aspects are well established and does not provide much new insights.\n\n- Experiments: the proposed method is extensively evaluated on blind seismic tomography. However this paper lacks quantitative results and comparison with baseline methods. In blind convolution there are no quantitative comparison. It seems to me that for publication purposes, this paper should be reorganized as a  technical novelty on seismic tomography instead of a general machine learning algorithm, as in its current form the evaluation is not comprehensive enough.\n\n- Scope: this paper discusses model mismatch, however in blind deconvolution experiments, instead of using a mismatch model the authors treat the forward model as blind. To properly deal with model mismatch, the algorithm should be able to utilize the inaccurate initial model to improve performance. This aspect is not reflected in the deconvolution case study.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Have the authors adequately addressed the limitations and potential negative societal impact of their work? If not, please include constructive suggestions for improvement：Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4 hours"}, "extra_text": {}}, {"review_id": "cLBwKUHoGfV", "reviewer": "Reviewer_M56A", "summary": "This paper proposes a normalizing-flow-based framework to solve inverse problems with model mismatch. The main idea is to use a normalizing flow to model the posterior distribution of the unknowns x conditioned on the current model parameters \\theta and the observation y, as part of the expectation-maximization algorithm. The proposed strategy yields results that seem to outperform \"traditional\" methods in seismic tomography and blind deconvolution.\n", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: The paper builds on existing ideas in an original way. It proposes a creative use of normalizing flows to solve inverse problems with model mismatch. On the other hand, the central idea---using flows to model conditional distributions---is not a new one (among other places, it is described in the cited DPI paper).\n\nQuality: The paper is well written and the empirical results seem solid.\n\n- One somewhat philosophical comment is that model mismatch and blind inverse problems (as in blind deconvolution) are not really the same thing. In blind inverse problems we are up front about the fact that some explicit parameter of the forward model is unknown and should be identified as part of the inverse problem. In vanilla model mismatch we don't always have a model for the mismatch, and we often assume that the mismatch is in some sense small. In your examples, \\theta is really a bona fide unknown, like x.\n\n- This is especially evident in that in your seismic tomography experiments (e.g. Figure 3) you make more effort to compare the wave speed models \\theta than the source locations x which are ostensibly the object you're trying to solve for. There is nothing wrong with this but to me it indicates a certain arbitrariness in the split between the unknowns of interest and the model parameters. In this case for the split you leverage independence of x and \\theta (Btw, is this verified in practice? I would expect that the sources are mostly located along some geological features which indicates a dependence between the geology \\theta and the source locations. Or am I wrong here?). Of course, in seismic imaging the split is very natural and it corresponds to well-known inverse problems (tomography vs source localization), but one could also say that the model is perfectly known---it is the wave equation---and you're jointly solving for the parametric RHS and the wave speed.\n\n- I am wondering about the \"seismologist-run\" baseline. Why limit oneself to straight rays? The main difference between the two approaches then seems to be in your use of the EikoNet rather than in the use of flows and the EM algorithm. But EikoNet could be easily used as the forward model in the alternating scheme used by the seismologist, without involving flows. Wouldn't that be a fairer comparison? Otherwise this is not just model mismatch (as in the wave speed being different) but also model class (or physics) mismatch, since regardless of the wave speed your method uses physical, bent rays, while the seismologist's method uses unphysical straight rays. I think that a solid comparison should also use EikoNet for the baseline.\n\n- Somehow related to the above two comments: in several places in the manuscript you advertise the \"generality\" of the approach, but the generality seems to equate the fact that normalizing flows can model conditional distribution which was shown already in the DPI paper. Granted, your application is a bit different. To the contrary, for each inverse problem one wants to address (here tomography and deconvolution), one has to design a differentiable, parameterized forward model (here in both cases a neural net). Thus the proposed method is not an out-of-the-box general method. The bulk of the difficulty is, I expect, in the design of this differentiable forward model. I believe this is a point worth elaborating.\n\n- Out of curiosity, how do you expect your method to scale? It seems that you need to train a normalizing flow at every iteration of the EM algorithm? How long does one iteration of that take? Can you speculate about performance on large-scale problems?\n\nClarity: The papers is very clear and well written.\n\nSignificance: The paper addresses an important problem of machine learning for inverse problems with model mismatch. It is surely a significant problem for the inverse problems / wave imaging community. The machine learning contribution on the other hand is minor (in view of the existing DPI and EikoNet papers), but inverse problems have been gaining traction at NeurIPS over the past couple of years.\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "YROLrHLQ9fN", "reviewer": "Reviewer_AwcU", "summary": "This paper considers the problem of inverting a signal from a given forward model $y=f_\\theta(x^*)+\\epsilon$, while accounting model mismatch from true model $f_{\\theta^*}$. The authors propose to solve this problem by first approximating the distribution of $x$ with $q_\\phi(x) \\sim G_\\phi (z)$ using a deep model and carrying out an expectation over $x\\sim q_\\phi(x)$ of the log-likelihood $p(y|\\theta,x)$. Model parameters $\\theta$ are approximated via maximization over this expectation (EM framework). They call this framework DeepGEM.\n\nThe authors specifically apply this mathematical model to 1) blind seismic tomography, where the task is to estimate the velocity structure of earthquake waves. The velocity computation depends on travel time $T$ (this is the variable observed) parameterized by source location $x$ and receiver location $r$.  In this paper $T(x,s) = f_\\theta(x,s) $ and $y \\approx T(x,s=r)$. 2) Blind deconvolution with $y=x * k_\\theta +\\epsilon$.", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The authors propose to use deep probabilistic models [31] to estimate a-posteriori distribution of signal x, in the context of inverse problems with model mismatch. The approach of using Expectation-Maximization has been used for solving inverse problems with model mismatch, such as blind deconvolution [20]; however this is the first paper to consider deep priors under this framework. Moreover the application of such approach to the domains of blind seismic tomography and blind deconvolution are novel. However there are several concerns.\n\nIt is unclear if this kind of approach will even converge; this paper will benefit from showing training curves showing reduction of loss value. \n\nIn Line 142, why is the prior on $\\theta$ (remains close to initial) reasonable? What if the initialization is far from the true parameters $\\theta^*$. This section requires a comment on how $\\theta$ is initialized.\n\nIn Figures 3 and 4 what are the metrics reported in the top right corner of each figure?\n\nFor the blind deconvolution case study, there is no baseline comparison, even though baseline models have been referenced in literature survey (eg. [20,21]). Since this paper is purely experimental in nature, baseline comparisons are important. \n\nSome notations need to be clarified or introduced as described below.\n\nSome line specific comments are as below:\n\nLine 102: define |x| to be dimension/cardinality\n\nStep 3.1: explain what value theta assumes here. Is it randomly initialized?\n\nLine 132: simplifying KL divergence -- N has not be defined anywhere so far.\n\nLine 135: if y includes gaussian noise with 0 mean, then eps = || y - f(x) ||^2, \n\nLine 142: why should theta stay close to \\tilde(theta)? Why is this a reasonable assumption?\n\nLine 185: $\\bar{x}$ is not defined at this point (it is defined later at 195)\n\nLine 187: What is $\\tilde{V}(s)$ and how does it differ from $V(s)$? \n\nApart from these concerns the paper is otherwise well written in terms of introduction and explanations related to the blind seismic tomography. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations: Line 216: authors rely on domain expert to decide when to terminate the optimization for the baseline, without which the algorithm diverges. It's unclear how to gauge the accuracy of baseline.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6 hours"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The problem studies inverse problems with a partially unknown forward operator. The idea is to use a general variational Expectation-Maximization framework aided by a normalizing flow generative network. The paper is focused on blind seismic tomographic and shows very good performance. The comparisons to baselines was a bit confusing but we understand that there are limited prior works on this problem and that the proposed method will easily outperform methods that do not use deep generative models. 

There was a debate on this paper solving blind inversion versus that mismatch. In any case there is good novelty in the paper which contains novel ideas, is solid in a real and useful application and is very well written. 
Decision: Accept (Poster)