  Multi-segment reconstruction (MSR) is the problem of estimating a signal
given noisy partial observations. Here each observation corresponds to a
randomly located segment of the signal. While previous works address this
problem using template or moment-matching, in this paper we address MSR from an
unsupervised adversarial learning standpoint, named MSR-GAN. We formulate MSR
as a distribution matching problem where the goal is to recover the signal and
the probability distribution of the segments such that the distribution of the
generated measurements following a known forward model is close to the real
observations. This is achieved once a min-max optimization involving a
generator-discriminator pair is solved. MSR-GAN is mainly inspired by CryoGAN
[1]. However, in MSR-GAN we no longer assume the probability distribution of
the latent variables, i.e. segment locations, is given and seek to recover it
alongside the unknown signal. For this purpose, we show that the loss at the
generator side originally is non-differentiable with respect to the segment
distribution. Thus, we propose to approximate it using Gumbel-Softmax
reparametrization trick. Our proposed solution is generalizable to a wide range
of inverse problems. Our simulation results and comparison with various
baselines verify the potential of our approach in different settings.
