  The automation of probabilistic reasoning is one of the primary aims of
machine learning. Recently, the confluence of variational inference and deep
learning has led to powerful and flexible automatic inference methods that can
be trained by stochastic gradient descent. In particular, normalizing flows are
highly parameterized deep models that can fit arbitrarily complex posterior
densities. However, normalizing flows struggle in highly structured
probabilistic programs as they need to relearn the forward-pass of the program.
Automatic structured variational inference (ASVI) remedies this problem by
constructing variational programs that embed the forward-pass. Here, we combine
the flexibility of normalizing flows and the prior-embedding property of ASVI
in a new family of variational programs, which we named cascading flows. A
cascading flows program interposes a newly designed highway flow architecture
in between the conditional distributions of the prior program such as to steer
it toward the observed data. These programs can be constructed automatically
from an input probabilistic program and can also be amortized automatically. We
evaluate the performance of the new variational programs in a series of
structured inference problems. We find that cascading flows have much higher
performance than both normalizing flows and ASVI in a large set of structured
inference problems.
