Year: 2021

Abstract:
The interest of the deep learning community in image synthesis has grown massively in recent years. Nowadays, deep generative methods, and especially Generative Adversarial Networks (GANs), are leading to state-of-the-art performance, capable of synthesizing images that appear realistic. While the efforts for improving the quality of the generated images are extensive, most attempts still consider the generator part as an uncorroborated “black-box”. In this paper, we aim to provide a better understanding and design of the image generation process. We interpret existing generators as implicitly relying on sparsity-inspired models. More specifically, we show that generators can be viewed as manifestations of the Convolutional Sparse Coding (CSC) and its Multi-Layered version (MLCSC) synthesis processes. We leverage this observation by explicitly enforcing a sparsifying regularization on appropriately chosen activation layers in the generator, and demonstrate that this leads to improved image synthesis. Furthermore, we show that the same rationale and benefits apply to generators serving inverse problems, demonstrated on the Deep Image Prior (DIP) method.

heading: 1 INTRODUCTION
text: The use of Generative Adversarial Networks (GANs) for image synthesis is one of the most fascinating outcomes of the emerging deep learning era, e.g., Goodfellow et al. (2014); Radford et al. (2015); Zhu et al. (2017); Ledig et al. (2017); Karras et al. (2018). GAN is a machine-learning apparatus built of two competing neural networks, a generator and a discriminator. The generator produces samples, while its adversary, the discriminator, attempts to distinguish between them and training ones. The resolution and quality of images produced by these intriguing game-theoretic machines have improved significantly in recent years, e.g., Karras et al. (2019); Brock et al. (2019).
Although leading to impressive results, GANs are difficult to train and prone to undesired phenomena, such as mode collapse, failure to converge and vanishing gradients (Thanh-Tung & Tran, 2020). Much of the research in this field has been focusing on mitigating the above difficulties and on stabilizing the training process. Most of this progress has been achieved by heuristically modifying the architectures of the generator and the discriminator, and by exposing new and better behaved training losses (Salimans et al., 2016; Arjovsky et al., 2017; Gulrajani et al., 2017; Mao et al., 2017). As such, while GANs in general have been extensively studied and redesigned, the generator itself still operates as a ”black-box” of unjustified architecture nor meaning.
Image generators within GANs are deep neural networks, comprised of a sequence of convolutionalbased blocks. Distinct from other types of deep learning architectures, generators learn a transfer function from a (typically simple) multivariate source distribution to a target one. To do so, each of the generator’s blocks performs a transitional step from its input to a latent representation. The desired overall mapping is thus achieved by a chain of these incremental steps. As mentioned above, previous work has considered the generator’s mapping as an end-to-end process, with hardly any attempt1 to propose a model that provides a possible explanation to the true nature of it.
Motivated by the sparse modeling literature (Elad, 2010), we aim to propose a novel interpretation that sheds some light on the architecture of image generators and provides a meaningful and effective regularization to such networks. We interpret generators as implicitly relying on sparse models in general, and on the Convolutional Sparse Coding (CSC) and its Multi-Layered (ML-CSC) version in
1A notable exception to this rule is the work reported in Mahdizadehaghdam et al. (2019), which will be described and tied to our contribution in Section 3.
particular (Szlam et al., 2010; Bristow et al., 2013; Chalasani et al., 2013; Grosse et al., 2012; Heide et al., 2015; Papyan et al., 2017b; Papyan et al., 2016; Sulam et al., 2018; Sulam et al., 2018). This observation provides a possible explanation for the generator’s intermediate mappings. We harness this insight by proposing a general model-based approach to regularize image generators, which can be easily applied to various architectures.
More specifically, we refer to the generator as a machine that synthesizes a sparse representation, followed by a multiplication by a dictionary. Accordingly, we propose several versions of sparsityinsipred regularizations, posed both as constraints or penalties, injected into the GAN training process. Under the CSC interpretation, the last layer of the generator consists of the learned synthesis dictionary, built as a set of filters. ML-CSC further assumes that a sequence of such dictionaries is multiplying the representation vector for generating the output image. We validate our proposed view by conducting extensive experiments on a variety of well-known GAN architectures, from relatively simple to up-to-date ones. Experimenting with a wide range of such networks demonstrates the versatility and generality of our approach. All the conducted tests show a performance improvement (assessed via Fréchet Inception Distance and Inception Score), achieved by our simple yet effective method.
We further extend our contribution by demonstrating that the same rationale and improvement are valid for other image generator neural networks. More specifically, we apply the proposed regularizations to the Deep Image Prior (DIP) algorithm (Ulyanov et al., 2018), which utilizes an image generator as an implicit image prior for solving various inverse problems. We examine the effects of our approach, and show that in restoration tasks as well, performance improvement is attained.
Adopting a different perspective towards our contribution, image generators are highly expressive overparametrized neural networks (Balaji et al., 2021) that solve what seems to be a severely illposed problem. Using overparametrized models with a large capacity might lead to a poor generalization and inferior results, and especially so when trained with relatively small datasets. As such, a properly chosen regularization applied to the generator may lead to benefits in both stability and overall performance. Indeed, driven by similar reasons, regularization has been adopted for the discriminator, being overparametrized as well, e.g., Wasserstein and Spectral Normalization GANs (Arjovsky et al., 2017; Gulrajani et al., 2017; Miyato et al., 2018; Zhang et al., 2019). Our work offers to close this gap by introducing a meaningful such regularization for generators, thereby further stabilizing and boosting the synthesis process.
To summarize, this work focuses on deep neural networks that serve as image generators, and its key contributions are the following:
• We propose a novel sparse modeling interpretation of image generators for better understanding them.
• We leverage this interpretation for proposing appropriate sparsity-inspired regularizations to generators.
• We demonstrate the validity of our view by showing a performance boost in image synthesis with various GAN architectures.
• We show that a similar performance improvement is applicable to image generators serving inverse problems, in the context of DIP.

heading: 2 BACKGROUND

heading: 2.1 SPARSE MODELING
text: Sparse Modeling has been proven to be highly effective in signal and image processing applications, e.g., (Dabov et al., 2007; Bruckstein et al., 2009; Yang et al., 2010; Elad, 2010; Dong et al., 2011; 2013; Mairal et al., 2014). This model assumes an underlying linear generative model, according to which, a signal x ∈ RN can be described as a linear combination of few columns from a dictionary D ∈ RN×M , i.e. x = DΓ, where Γ is a sparse vector. The columns of D are referred to as atoms and they may form an overcomplete set, i.e., M > N . Retrieval of a sparse vector Γ, corresponding to a given a signal x and a dictionary D, is referred to as sparse coding, formulated as
min Γ ‖Γ‖0 s.t. DΓ = x, (1)
where ‖Γ‖0 counts the non-zeros in Γ. Thus, synthesizing a signal according to this model is done by generating a sparse representation vector Γ and multiplying it by D.

heading: 2.2 CONVOLUTIONAL SPARSE CODING MODEL
text: When handling images, sparse modeling has been usually deployed on small overlapping patches due to the complexity of handling a general content dictionary. The Convolutional Sparse Coding (CSC) model (Szlam et al., 2010; Grosse et al., 2012; Bristow et al., 2013; Chalasani et al., 2013; Heide et al., 2015; Papyan et al., 2017b) is a global variant of the above model, which bypasses the need to operate on patches. The CSC’s dictionary is structured, being a concatenation of banded circulant matrices containing small support filters, each appearing in all possible shifts. Thus, D ∈ RN×mN where N is the size of the signal x and m is the number of filters, each of length n N . According to this model, a signal x can be expressed by x = DΓ, as described above (see Figure 1).
The CSC has demonstrated superb performance in image processing tasks, such as denoising, separation, fusion, and super-resolution (Gu et al., 2015; Liu et al., 2016; Papyan et al., 2017a; Simon & Elad, 2019; Zisselman et al., 2019). A recent work (Papyan et al., 2017b) has proposed a theoretical analysis of this model, exposing the need for a redefinition of the sparsity measure to be used on Γ, in order to account for local use of atoms instead of globally counting non-zeros. We shall get back to this in the next section, when using the CSC model.

heading: 2.3 MULTI-LAYER CONVOLUTIONAL SPARSE CODING (ML-CSC)
text: ML-CSC (Papyan et al., 2016; Sulam et al., 2018; Sulam et al., 2018) is an extension of the CSC, which generates a cascade of sparse representations {Γi}Li=1, corresponding to CSC dictionaries {Di}Li=1. This model assumes that a signal x ∈ RN can be represented as
x = D1Γ1 s.t. ‖Γ1‖0 ≤ λ1, Γ1 = D2Γ2 s.t. ‖Γ2‖0 ≤ λ2,
... ΓL−1 = DLΓL s.t. ‖ΓL‖0 ≤ λL,
where {λi}Li=1 are sparsity thresholds. Thus, while the first equation perfectly aligns with the regular CSC model, the additional equations add further structure by suggesting that each sparse representation vector is by itself a CSC signal. Note that by substituting the above equations, a signal x can also be described as x = D1 · · ·DiΓi = DeffΓ, 1 ≤ i ≤ L, with intermediate sparse representations.

heading: 3 IMPROVED IMAGE SYNTHESIS VIA GANS
text: Deep generative models are neural network-based architectures that synthesize signals, such that their output follows the probabilistic distribution of a given data source. In computer vision, these generators are used successfully for synthesizing natural images, by training on massive datasets of photos. To this end, they map a given source distribution Pz (typically a simple and canonical multivariate Gaussian) to a data distribution of interest Px,
G(z) = xgen s.t. z ∼ Pz and xgen ∼ Px. (2)
Due to the complexity of image synthesis, the above mapping function is usually modeled by a highly expressive feed-forward deep convolutional neural network, consisting of several consecutive layers. In its simplest and most common form, an image generator with K layers can be rephrased as a feed-forward CNN of the form
GK(...(G1(z))) = xgen, (3)
whereGi represents the ith layer of the generative model, applying convolutions, normalizations and a non-linearity (typically ReLU). Thus, the overall mapping is attained in tandem of K transitional mappings. Despite the incremental nature of the deep image generator, it is treated and trained in an “end-to-end“ mode, without understanding the purpose of the inner activations, nor enforcing a specific structure or properties on the intermediate activations.
In this work, we interpret existing image generators as implicitly relying on sparse modeling, and we propose a novel model-based approach to describe and improve the synthesis process of such architectures. Since these generators are highly expressive and overparametrized, confining them can lead to superior results and facilitate the training process. To do so, we recall the convolutional sparse coding (CSC) and its Multi-Layer version (ML-CSC) models, as discussed in Section 2.
According to the CSC model, an image x can be represented as a multiplication of a sparse representation vector Γ by a convolutional dictionary D, i.e. x = DΓ. The ML-CSC further assumes that D is achieved by a multiplication of L dictionaries: Deff = D1D2 · · · DL. Note, however, that although both are generative models, there is a substantial gap between their description and the process described in Equation (2). Whereas the CSC synthesis starts with a sparse representation vector, the image generation setup in Equation (2) starts with a dense latent random vector z. To bridge this gap, we propose to interpret image generators as performing two consecutive tasks:
1. GS : Map the input vector z to a sparse latent vector Γ (done by the generator’s first K − 1 layers).
2. GI : Multiply Γ by a convolutional dictionary D, i.e., DΓ ∼ Px (performed by the generator’s Kth layer that learns a convolutional dictionary for this purpose).
The second task is exactly the CSC synthesis process. More specifically, if D is a single convolutional dictionary, the above formulation is equivalent to the CSC, whereas if D is comprised of two
or more dictionaries, this becomes the ML version of it. By splitting the generation process into two parts, we identify the role of Γ as the sparse representation in a (ML-)CSC-based model. This way, the image synthesis process can be described as
GS(z) = Γ , GI(Γ) = x
s.t. z ∼ Pz, Γ is sparse, x ∼ Px, namely, the end-to-end mapping can be separated into two consecutive mappings, where each may be easier to learn. This interpretation is further shown in Figure 3. In the ML-CSC context, GI is comprised of two or more global convolutional dictionaries.
We have analyzed the sparsity of Γ in regular adversarial training and found out that it is not sufficient. For the purpose of designing more compatible generators with both the CSC and the ML-CSC models, we encourage GS to map z to a truly sparse representation Γ. To this end, we propose few explicit techniques, serving as regularization forces applied in the training phase:
(1) L1 regularization: Adding an L1 based penalty on the representations Γ to the overall loss of the image generator.
(2) L0 constraint: Eliminating small non-zero entries in Γ to satisfy a predefined sparsity constraint: ‖Γ‖0 ≤ λ. Namely, the amount of non-zero entries in Γ should be less or equal to λ. This can be viewed as a projection to a constraint-satisfying tensor, obtained by zeroing the smallest absolute values of the representation.
(3) L0,∞ inspired constraint: This pseudo-norm is based on a new sparsity measure related to the CSC Papyan et al. (2017b). ‖Γ‖0,∞ is the maximal number of non-zero coefficients affecting any pixel in the image x. Thus, forcing ‖Γ‖0,∞ ≤ λ restricts the number of local atoms to λ. While this constraint is theoretically justified in the context of CSC, projection onto it is known to be challenging (Plaut & Giryes, 2019). To approximate it in a computationally plausible manner, we use a “needle”-based sparsity measure (Papyan et al., 2017a; Zisselman et al., 2019). In the general 2D CSC case, Γ is a 3D tensor, of size H×W×C, where H and W define the image size to be synthesized, and C is the number of filters. We define a needle as 1×1×C tensor, contained in Γ. Hence, Γ contains H×W needles, each contributes to a patch of pixels in a corresponding position in the output image. In this configuration, every pixel is affected by several adjacent needles. This setup is described in Figure 2. We propose to limit the amount of non-zero entries in every such needle. To this end, for a given representation tensor Γ, we zero the smallest absolute values in each needle of the representation to satisfy the relaxed constraint.
Although the above three options all promote sparsity in Γ, they are substantially different, as indeed will become evident in our experiments. While L1 and L0 consider global sparsity, the L0,∞ forces a local balance in the use of the atoms, i.e., limiting the local density in the representation Γ. As for the difference between L0 and L1, the first is deployed as a constraint, while the latter is used as a penalty. Since both L0,∞ and L0 are constraints, we implemented them as projection layers which map an input tensor to a constraint-satisfying one, by zeroing its smallest entries. Figure 4 illustrates the effect of applying the proposed regularization techniques and the differences between them in inducing sparsity.
We should note that a connection between sparse modeling and GANs’ generators has been already proposed in Mahdizadehaghdam et al. (2019). In their work, they introduced sparsity in GANs for boosting performance, and achieved their goal by designing a specific architecture that utilizes a patch-based modeling and a pre-trained dictionary. While inspired by their view, our work differs substantially, as we propose a more general strategy that leverages more advanced models (CSC and ML-CSC), does not require a pre-training of any sort, and can be easily applied to various existing GAN architectures. Indeed, as we shall see in the next section, our theme extends to other generators that go beyond GANs.
To summarize the proposed theme, we argue that learning a direct mapping from random noise to natural images’ distribution is an extremely hard task that can be mitigated by enforcing a model that provides a meaningful regularization. As in every model, the CSC introduces specific structured assumptions about the data. Since this model has been shown to be highly compatible with natural images, we believe that these assumptions empirically hold, and hence, the suggested approach will lead to better results. To verify the above statement, we conduct experiments in various types of tasks, compare and dissect the results of regularized and non-regularized generators.

heading: 4 IMPROVED SOLUTION OF INVERSE PROBLEMS
text: Inverse problems involve the process of recovering data from degraded observations. There are various well-known such problems in image processing, e.g., denoising, deblurring, inpainting and super-resolution, all of which are ill-posed, as they may have no unique solution. In the field of image processing, solving inverse problems is one of the most important and studied topics, and there are thousands of papers dedicated to the derivation of various algorithms and strategies for handling such tasks. In this paper, we focus on one specific and fascinating method, DIP (Ulyanov et al., 2018), which uses an image generator for handling inverse problems.
According to DIP, a deep image generator should be trained (i.e., adapt its parameters θ) to map a fixed random tensor z to a given corrupted image x0, and the solution to the inverse problem would be the generator’s output. Formally,
θ∗ = arg min θ L(Gθ(z)|x0), x∗ = Gθ∗(z), (4)
where L(x|x0) is a task-dependant loss term. This way, much of the information about the images’ distribution Px is derived from the generator’s architecture.
DIP is an appealing unsupervised technique for handling inverse problems, and we find it is ideal for testing our core hypothesis – the main claim advocated in our work is that CSC based models are adequate for generating images. If indeed correct, one would expect that CNN architectures with induced sparsity could serve better as a prior for inverse problems. Recall that both L0 and L0,∞ sparsity constraints are enforced via an architectural modification (a projection layer). DIP puts forward an exact such test, as its essence is a reliance solely on the model’s architecture itself for regularizing inverse problems. Therefore, image restoration via DIP serves as a different yet meaningful setup for verifying our sparse-modeling assumptions. To this end, we aim to inject well-justified sparsity-inspired regularizations to specific activation layers within the generator for improving the final outcome.
Although DIP is a generic concept that can be applied to various generator architectures, we experiment with the same architecture as in the original paper – an encoder-decoder model that maps a latent vector drawn from Pz into an image of the same spatial extent. More specifically, DIP uses a U-Net-like architecture (Ronneberger et al., 2015) with skip-connections. Due to the significant structural difference between the encoder-decoder and standard generators, applying our method in the same way in both is unjustified. To better understand the proper context of sparsity in such architectures, we focus on the encoder part of the overall system. We interpret this part as transforming its input into a set of transitional representations {Γ1, ...,ΓK}, where K is the number of scales in the architecture, as can be seen in Figure 5. These are injected to the decoder, from which it constructs the output image. As these representations are attained by a CNN, which resembles a multi-layered thresholding algorithm (Papyan et al., 2016; Sulam et al., 2018; Sulam et al., 2018), we interpret this process as sparse coding serving the ML-CSC model. According to this perspective, the encoder maps the input random vector to a dense signal and proceeds by performing a multi-layer pursuit to obtain {Γ1, ...,ΓK}, in the spirit of the ML-CSC model. Using this observation, we propose to apply our sparsity-inducing regularization, as described in section 3, on all these representations.

heading: 5 EXPERIMENTS
text: In this section we experimentally examine the proposed method, comparing its performance with non-regularized (“vanilla”) image generator’s architectures. First, we evaluate our method on image synthesis via GANs. We conduct an extensive study on a variety of GAN architectures and explore the effect of applying our suggested regularizations to them. In addition, we study the effect of applying our approach on limited data sources of different sizes. We also show that sparsity-inducing
regularization is versatile and can be applied to more general image generators. To this end, we apply our method to the Deep Image Prior algorithm, which handles restoration problems. In both image synthesis and in solving inverse problems, we dissect the effects of the proposed regularization techniques that induce sparsity, as described in the previous sections.

heading: 5.1 IMPROVED IMAGE SYNTHESIS
text: For experimenting with the suggested regularizations for image synthesis, we apply these on GANs during training. The proposed approach is enforced only on the generator part, as the discriminator remains untouched. We exhaustively evaluate the effect of these regularizations on various GAN variants using the CIFAR-10 dataset (Krizhevsky, 2012), since it is one of the most popular benchmarks for image synthesis.
We apply our method to several GAN architectures, conditional and unconditional, in order to show its compatibility with different synthesis processes. We experiment with a wide range of architectures, considering both the CSC and the ML-CSC variants: DCGAN (Radford et al., 2015), conditional GAN (cGAN) (Mirza & Osindero, 2014), Wasserstein GAN with Gradient Penalty (WGANGP) (Gulrajani et al., 2017), Mode Seeking GAN (MSGAN) (Mao et al., 2019), Spectral Normalization GAN (SNGAN) (Miyato et al., 2018), Self Attention GAN (SAGAN) (Zhang et al., 2019) and Improved Self Supervised GAN with Multiclass minimax game (SSGAN*) (Tran et al., 2019), BigGAN (Brock et al., 2019) and Differentiable Augmentation BigGAN (DiffCRBigGAN)(Zhao et al., 2020).
To evaluate the synthesis results, we use the Fréchet Inception Distance (FID) (Heusel et al., 2017), which is a widely used metric for assessing the quality of generated images. Lower FID scores indicate a better quality of the synthesized images. In addition to the FID, we also evaluate the performance of the conditional BigGAN models using Inception Score (IS) (Salimans et al., 2016), where higher values are better.
The results of the conducted experiments are summarized in Tables 1, 2. The findings attest that using our proposed regularization techniques significantly enhances the performance across all examined GAN models, from simple to more sophisticated up-to-date architectures, which demonstrate the versatility and the generality of the proposed regularizations. Furthermore, the tested ML-CSC approach using two layers (i.e. Deff = D1D2) leads to improved performance, when compared to the single-layered CSC.
We turn to study the effects of our proposed regularizations when operating with limited datasets. To this end, we use 10% (5,000 images) and 20% (10,000 images) of the CIFAR-10 dataset and experiment with a BigGAN architecture. We also examine the effects of applying our method in addition to the differentiable augmentation method (Zhao et al., 2020), which provides state-of-theart results on limited data. Table 3 demonstrates the performance improvement attained by applying our method in both of the setups.

heading: 5.2 IMPROVED SOLUTION OF INVERSE PROBLEMS
text: We proceed by experimenting with our regularizations on standalone image generators, in the context of the DIP algorithm. Our goal is to compare the ability of regularized and non-regularized image generators to serve as an implicit prior for solving inverse problems.
In the conducted experiments, we solve a denoising problem with regularized and non-regularized image generators. As already mentioned in Section 4, we use an U-Net-like “hourglass” architecture with skip-connections, as in DIP (Ulyanov et al., 2018). Excluding the regularization, the compared generators have the same architecture and are being trained with exactly the same hyperparameters and setup. We conduct a similar experiment to the one in DIP, using the standard denoising dataset. 2 In this setup, an observed noisy image x0 is created by adding an additive white Gaussian noise (AWGN) with standard deviation σn (σn = 25 in our experiments). To quantitatively evaluate the denoising performance, we use the Peak signal-to-noise ratio (PSNR).
The results are reported in Table 4, where “Single” refers to the top PSNR value achieved by a single output, and “Average” is the highest PSNR value of an averaged output (obtained by an exponential sliding window over past iterations, as performed in DIP). We run the experiment multiple times to verify the statistical significance of our results.
As can be seen, enforcing sparsity by our proposed methods outperform the non-regularized model. These results demonstrate that the CSC modeling assumption is valid and contributes to better regularizing the inverse problem.

heading: 6 CONCLUSIONS
text: In this work we have described simple yet effective regularization techniques for image generator architectures, which rely on sparse modeling. We demonstrate that such regularizations yield substantial improvements across a wide range of GAN architectures. In addition, we show the versatility of the approach by applying it to image generators for solving inverse problems using DIP. In this context, our regularization improves the denoising results achieved by DIP. The enhanced performance achieved by promoting sparsity in image generators, in general, testifies to the relevance of sparsity-inspired models in image synthesis. We believe that this connection can be further learned and utilized to obtain even more promising results.
2http://www.cs.tut.fi/˜foi/GCF-BM3D/index.html#ref_results

heading: 7 REPRODUCIBILITY STATEMENT
text: In this work we have examined our proposed method both for boosting image synthesis via GANs and in improving the solution of inverse problems via DIP. In both settings, we implemented our approach using the official software in each of the papers we have referred to, and using the same hyperparameters as stated in their experiments. In addition, in all of our tests, we evaluated the performance multiple times using different seeds, so as to factor out randomness in the attained improvement. A complete code package for running all the conducted experiments and reproducing the reported results of this paper will be published upon acceptance of the paper.

heading: B. ADDITIONAL ANALYSIS OF SPARSITY IN GANS
text: We turn to present additional graphs, similar to the one given in Figure 4a, that demonstrate the effect of promoting sparsity in different GAN architectures. As can be seen in Figure B, inducing sparsity consistently improves the performance of the tested models.
Figure B: A visualization of the performance improvement obtained by the proposed sparsityinducing methods across various GAN architectures.

Review summaries:
The paper proposes viewing CNN's through the lens of sparse coding. In practice it mostly boils down to encouraging sparsity in the last layer activations. To this end the paper proposes three different mechanisms that promote sparsity. This is shown to improve the FID scores somewhat in one dataset with a range of different GAN architectures. The ideas are also applied to another generator-like DIP scenario.

This paper proposes a CNN-based (specifically GAN-based) solution to tackle image synthesis using sparse coding concept. The proposed method utilises the generator from a generative adversarial network to synthesise a sparse representation, i.e. the sparse code, for image synthesis. This method proposes to split original ML-CSC into two consecutive function learning. It claims that the two functions are easier to learn. In addition, this paper also proves and adds image regularisation such as DIP into the framework. Extensive experiments demonstrate that the proposed methods outperforms state of the art image synthesis methods including traditional sparse coding methods and deep learning based methods.   

The paper describes image generation as a sparse coding reconstruction process. The authors formulate the sparse coding as a core block of a convolution image generator and test several different approaches for enforcing sparsity in the trained network. The experiments show that the method can improve FID scores of common GAN networks and it can also increase PSNR of image reconstruction using deep prior when compared to the same network architectures without sparsity objective.