  We propose Intermediate Layer Optimization (ILO), a novel optimization
algorithm for solving inverse problems with deep generative models. Instead of
optimizing only over the initial latent code, we progressively change the input
layer obtaining successively more expressive generators. To explore the higher
dimensional spaces, our method searches for latent codes that lie within a
small $l_1$ ball around the manifold induced by the previous layer. Our
theoretical analysis shows that by keeping the radius of the ball relatively
small, we can improve the established error bound for compressed sensing with
deep generative models. We empirically show that our approach outperforms
state-of-the-art methods introduced in StyleGAN-2 and PULSE for a wide range of
inverse problems including inpainting, denoising, super-resolution and
compressed sensing.
