  The interest of the deep learning community in image synthesis has grown
massively in recent years. Nowadays, deep generative methods, and especially
Generative Adversarial Networks (GANs), are leading to state-of-the-art
performance, capable of synthesizing images that appear realistic. While the
efforts for improving the quality of the generated images are extensive, most
attempts still consider the generator part as an uncorroborated "black-box". In
this paper, we aim to provide a better understanding and design of the image
generation process. We interpret existing generators as implicitly relying on
sparsity-inspired models. More specifically, we show that generators can be
viewed as manifestations of the Convolutional Sparse Coding (CSC) and its
Multi-Layered version (ML-CSC) synthesis processes. We leverage this
observation by explicitly enforcing a sparsifying regularization on
appropriately chosen activation layers in the generator, and demonstrate that
this leads to improved image synthesis. Furthermore, we show that the same
rationale and benefits apply to generators serving inverse problems,
demonstrated on the Deep Image Prior (DIP) method.
