On Analyzing Generative and Denoising Capabilities of Diffusion-based Deep Generative Models
Anonymous Author(s)
AffiliationAddress email
Abstract
Diffusion-based Deep Generative Models (DDGMs) offer state-of-the-art performance in generative modeling.
Their main strength comes from their unique setupin which a model (the backward diffusion process) is trained to reverse the forward diffusion process, which gradually adds noise to the input signal.
Although
DDGMs are well studied, it is still unclear how the small amount of noise istransformed during the backward diffusion process.
Here, we focus on analyzingthis problem to gain more insight into the behavior of DDGMs and their denoising and generative capabilities.
We observe a fluid transition point that changesthe functionality of the backward diffusion process from generating a (corrupted)image from noise to denoising the corrupted image to the final sample.
Based onthis observation, we postulate to divide a DDGM into two parts: a denoiser anda generator.
The denoiser could be parameterized by a denoising auto-encoder,while the generator is a diffusion-based model with its own set of parameters.
Weexperimentally validate our proposition, showing its pros and cons.
Introduction
Diffusion-based Deep Generative Models [22] (DDGM) have recently attracted increasing attention,due to the unprecedented quality of generated samples [5, 9, 11].
The general idea behind this set ofmethods is to generate samples using diffusion processes [8, 10, 11, 23, 24].
In the forward diffusionprocess , an image is passed through a number of steps that consecutively add a small portion of noiseto it.
The backward diffusion process is a direct reverse of the forward process, where a generativemodel is trained to gradually denoise the image.
With a sufficient number of the forward diffusionsteps, noisy images approach isotropic Gaussian noise.
Then, generating new examples is possible byapplying the backward diffusion to the noise sampled from the standard Gaussian distribution.
While the performance of DDGMs is impressive, not all of their aspects are fully understood.
Intuitively, a DDGM is trained to remove small amounts of noise from many intermediary corruptedimages.
Although this perspective is reasonable and complies with the interpretation of DDGMsusing stochastic differential equations [10, 24], it is still unclear how the small amount of noiseis removed during the backward diffusion process where images are composed of almost entirelyrandom values.
The more adequate intuition might be that in its initial steps, a diffusion model doesnot only remove noise but also introduces a new signal according to the distribution learned fromthe data.
In this work, we further investigate this observation to understand the balance between thegenerative and denoising capabilities of DDGMs.
In particular, we aim to answer the following three questions in this paper: (i) Is there a transition inthe functionality of the backward diffusion process that switches from generating to denoising?
(ii)
How does this split of functionality affect the performance?
(iii) Does the denoising part in DDGMsgeneralize to other data distributions?
As a result, the contribution of the paper is threefold:
• First, we analyze the noise distribution in the forward diffusion process and how steps of the
diffusion process are correlated with the reconstruction error.
• Second, based on our analysis, we postulate that DDGMs are composed of two parts: a denoiserand a generator .
As a result, we propose a new class of models that consist of a DenoisingAuto-Encoder and a Diffusion-based generator shortened as DAED.
DAED could be considered asa variation of DDGMs with an explicit split into the denoising part and the generating part.
• Third, we empirically assess the performance of DDGMs and DAED on three datasets (FashionMNIST, CIFAR10, CelebA) in terms of data generation and transferability (i
e., how DDGMs behaveon different data distribution).
Background2.1 Diffusion-Based Deep Generative Models (DDGMs)
Model formulation We follow the formulation of the Diffusion-based Deep Generative Models
(DDGMs) as presented in [22, 8].
DDGMs could be seen as infinitely deep hierarchical VAEs with aspecific family of variational posteriors [10, 11, 25, 26], namely, Gaussian diffusion processes [22].
Given a data point x 0 and latent variables x t , . . .
, x T , we want to optimize the marginal likelihoodp ( x 0 ) =
(cid:82) p ( x 0 , . . .
, x T )d x 1 , . . .
, x T .
We define the backward (or reverse ) process as a Markovchain with Gaussian transitions starting with p ( x T ) =
N ( x T ; 0 , I ) , that is:
[Equation]
where p ( x t − 1 | x t ) =
N ( x t − 1 ; µ θ ( x t , t ) , Σ θ ( x t , t )) .
Additionally, we define the forward diffusionprocess as a Markov chain that gradually adds Gaussian noise to the data according to a variance schedule β 1 , ..., β T , namely, q ( x 1 , . . .
, x T | x 0 )
= (cid:81)
Tt =
1 q ( x t | x t − 1 ) , where q ( x t | x t − 1 ) =
N ( x t ; √ 1 − β t x t − 1 , β t I ) .
Let us further define α t = 1 − β t and α t = (cid:81) ti =
α i .
Since the conditionals in the forward diffusion can be seen as Gaussian linear models, we can analytically calculatethe following distributions:
[Equation]
and
[Equation]
3) where ˜ µ ( x t , x 0 ) = √
α
[Equation]
x t , and ˜ β t = 1 − α t − 1 1 − α t β t .
We can use (2) and (3) todefine the variational lower bound as follows:
[Equation]
that we further optimize with respect to the parameters of the backward diffusion.
The conditional likelihood In this paper, we focus on images, thus, data is represented by integersfrom 0 to 255 .
Following [8], we scale them linearly to [ − 1 , 1] .
As a result, to obtain discretelog-likelihoods, we consider the discretized (binned) Gaussian conditional likelihood [8]:
[Equation]
where D is the data dimensionality of x 0 , and i denotes one coordinate of x 0 , and:
[Equation]
Noise scheduling Originally, [8] propose to linearly scale the noise parameters β t ( linear scheduling ), e.g., scaling linearly from β 1 = 10 − 4 to β T = 0 .
In [17], authors suggest to increase thenumber of less noisy steps through cosine scheduling : ¯
[Equation]
t
[Equation]
with clipping the values of β t to 0 .
999 to prevent potential instabilities at the end of the diffusion.
Training details In [8], authors notice that a single part of the variational lower bound is equal to:
[Equation]
where ϵ ∼ N ( 0 , I ) and ϵ θ is a neural network predicting the noise ϵ from x t .
Since we use (3) in thevariational lower bound objective (4), and x t could be sampled from the forward diffusion for givendata, see (2), we can optimize one layer at a time.
In other words, we can randomly pick a specificcomponent of the objective, L t , and update the parameters by optimizing L t without running thewhole forward process from x 0 to x T .
As a result, the training becomes very efficient and learningvery deep models (with hundreds or even thousands of steps) possible.
In [8], it is also proposed to train a simplified objective that is a version of (7) without scaling, namely:
[Equation]
where t is uniformly sampled between 1 and T .
To further reduce computational and memory costs,typically, a single, shared neural network is used for modeling ϵ θ [8, 11, 17] that is parameterizedby an architecture based on U-Net type neural net [18].
The U-Net could be seen as a specificauto-encoder that passes all codes from an encoder to the decoder.
2.2 Denoising Auto-Encoders
Another class of models, Denoising Auto-Encoders (DAEs), is similar to DDGMs in the sensethat they also revert a known corruption process.
However, DAEs are trained to remove the noisein a single pass, and unlike DDGMs, they cannot generate new objects.
Specifically, DAEs areauto-encoders that reconstruct a data point x 0 from its corrupted (noisy) version [1, 2, 4, 28].
Let usdenote the auto-encoder by f φ ( · ) .
Using the same notation as for DDGMs, the Gaussian corruptiondistribution is q ( x 1 | x 0 ) .
Then, a DAE maximizes the following objective function:
[Equation]
9) and, in particular, for the Gaussian distribution with the identity covariance matrix, we get the originalobjective for DAEs [28]: ln p (
[Equation]
An analysis of DDGMs
The core idea behind DDGMs is the gradual noise injection to images as we go forward in timesuch that the final object is a sample from the standard Gaussian distribution.
Then, in the backwarddiffusion process model reverts this procedure and, as a result, generates new objects.
Therefore, understanding the success of DDGMs relies heavily on understanding how the injected noise influencesthe behavior of both training and the model itself.
The noise distribution in the forward diffusion process The first question we ask is how muchcorrupted an image gets after applying a specific noise schedule.
Following [8, 12, 17], we can utilizethe signal-to-noise ratio (SNR), expressed as the squared mean of a signal (here: image) divided
0 t/T − l og
S N R
FashionMNIST CIFAR 0 .0 .1 .
0 t/T − 0 .− 0 .0 .∆
l og
S N R
FashionMNIST CIFAR (a) Linear noise schedule 0 .0 .1 .
0 t/T − l og
S N R
FashionMNIST CIFAR 0 .0 .1 .
0 t/T −− ∆ l og
S N R
FashionMNIST CIFAR
[Figure]
by the variance of a signal, to quantify the amount of noise in x t .
For this purpose, the quantity ofinterest is the forward diffusion for a given x 0 , namely, q ( x t | x 0 ) , that results in the following SNR:
[Equation]
Similarly to [11], we formulate the forward diffusion in such a way that the SNR is strictly monotonically decreasing in time, namely, SNR ( x 0 , t ) <
SNR ( x 0 , s ) for t > s .
This means that an imagebecomes more noisy as we go forward in time.
In Figure 1 (left) we plot the logarithm of the SNR for both linear (Figure 1.a) and cosine (Figure1.b) noise schedules for two datasets (FashionMNIST and CIFAR10).
We average SNR over thex 0 ’s (from the corresponding dataset).
The right column depicts the change of the log SNR, i.e.,its discrete derivative ∆ log SNR ( t ) = log SNR ( x 0 , t ) − log SNR ( x 0 , t − 1) .
First of all, we cannotice a point at which the log-SNR drops below 0 .
This corresponds to the situation of the noiseovershadowing the signal.
In the case of the linear noise schedule, this happens after about 20%of steps, while for the cosine noise schedule, it appears after about 25 − 50% of steps.
However,the transition occurs in both cases.
The biggest changes in the log-SNR are noticeable within thefirst 10% of steps.
This may suggest that the signal is the strongest within the first 10 − 20% of theforward diffusion process steps, and then it starts being overshadowed by the noise.
The reconstruction error of DDGMs Since we know that the signal is not lost within the first10 − 20% of steps, the next question is about the reconstruction capabilities of DDGMs, namely,what is the reconstruction error of x t ∼ q ( x t | x 0 ) .
To be clear, we are not interested in how mucheach step of a DDGM contributes to the final objective (e.g., see Figure 2 in [17]) but rather how wella DDGM reconstructs a noisy image x t .
[Figure]
In Figure 2 we plot the Mean Absolute Error (MAE) and the Multi-Scale Structural
Similarity (MS-SSIM) [29] that both measure the difference between an original image x 0 and a corrupted image at the t th stepx t reversed by the backward diffusion.
Wepresent the values on two datasets (FashionMNIST and CIFAR10) for the first 20% ofsteps.
Apparently, after around 10% of thesteps, the reconstruction error starts growing, and the MAE increases linearly above 0 .
1 (i.e., about 6% of error per pixel).
At the same time, the
MS-SSIM drops below 0 .
9 − 0 .
95 (i.e., the discrepancy between original images and reconstructionsbecomes perceptually evident).
This observation might suggest that DDGMs could be roughly dividedinto two parts: a fraction of steps of a DDGM (e.g., first 10% of the steps) constitute a denoiser thatturns a corrupted image into a clear image, and the remaining steps of the DDGM are responsible forturning noise into a noisy structure (a corrupted image), i.e., a generator that generates meaningfulpatterns.
In other words, we claim that DDGM can be interpreted as a composition of a denoiserand a generator, but the boundary between those two parts is fluid.
Moreover, the denoiser graduallyremoves the noise in a generative manner (i.e., by sampling x t − 1 ∼ p ( x t − 1 | x t ) ).
DDGMs as hierarchical VAEs In this paper, we postulate that DDGMs could be seen as acomposition of parts that serve different purposes.
We can get additional insight into our claim bynoticing a close connection between DDGMs and hierarchical VAEs.
As presented in [10, 11, 25],if we treat all x t ’s with t > 0 as latents, and see the forward diffusion process as a composition of(non-trainable) variational posteriors, DGGMs become a specific formulation of hierarchical VAEs.
On the other hand, we can start with a VAE with a single latent variable, x 1 , for which the variationallower bound is equal to:
[Equation]
Then, similarly to [27, 30], the marginal p ( x 1 ) could be further modeled by a DDGM.
By keepingthe dimensionality of x 1 the same as x 0 , and taking the variational posterior q ( x 1 | x 0 ) to be fixed andpart of the forward diffusion, we get the DDGM model.
This perspective of combining a VAE with a
DDGM opens new possibilities for developing hybrid models.
DEAD: Denoising Auto-Encoder with Diffusion
In this work, we propose a specific combination that distinctly splits the DDGM into generativeand denoising parts.
As noted in the previous section, the signal in the forward diffusion processis the strongest within the first 10 − 20% of steps, and, thus, we postulate to perceive this first partof a DDGM as a denoiser.
Together with the observation about the combination of a VAE witha DDGM-based prior, we consider turning a denoising auto-encoder into a generative model.
Webring a DDGM-based part into DAE for generating corrupted images.
The resulting objective is thefollowing:
[Equation]
where in (13) we introduce additional latent variables and the variational posterior over them, thatyields the variational lower bound.
We call the resulting model DAE with a Diffusion , or DEADfor short.
In a sense, DAED is a DDGM with distinct parameterizations of the part between x 0 andx 1 , and the part for the remaining x ’s. Thus, DEAD is almost identical to a DDGM, but there arethe following differences: (i)
We can control the amount of noise in q ( x 1 | x 0 ) .
It can correspondto the first step of the forward diffusion model, or we can introduce more noise at once that wouldcorrespond to several steps in the DDGM. (ii)
We use two different parameterizations, namely, anauto-encoder (e.g., a U-Net architecture) for f φ ( · ) and a separate, shared U-Net for modeling the DDGM from x 1 to x T .
Since there are two neural networks, the lower bound to the objective ℓ isin fact a composition of two objectives with disjunctive parameters, namely, the objective for thedenoiser , ℓ DAE , and the objective for the generator (i.e., the diffusion-based generative model), ℓ D .(iii)
In the DAED, we introduce the denoiser explicitly and make a clear distinction between thedenoising and the generating parts while, as discussed earlier, this boundary is rather fluid in DDGMs.
By introducing DAED, we can analyze what happens if we distinctly divide those two aspects withtwo separate parametrizations.
Moreover, we hypothesize that the resulting model may better generalize across various data distributions due to decoupling the parameterization of the denoiser and the generator.
The training datasetmay bias a single, shared parameterization in a DDGM, and while denoising an image from a differentdomain, it may add some artifacts from the source.
While with two distinct parameterizations, theremight be a lower chance for that.
We evaluate this hypothesis in the experiments.
Related work
DDGM for image generation Various modifications of DDGMs were recently proposed to improvetheir sampling quality.
This includes simplifying the learning objective and proposing new noise schedulers, which allow DDGMs to achieve state-of-the-art results.
In this work, we show thatsplitting the decoder into two parts, namely, a denoiser and a generator, can benefit the performance,especially when training with the variational lower bound.
Properties of DDGMs
In [8] authors notice that DDGMs can be beneficial for lossy compression,observing (Figure 5 in [8]) that most of the bits are allocated to the region of the smallest distortionthat corresponds to the first steps of a DDGM.
We draw a similar conclusion when discussing thedenoising ability of the diffusion model in Section 3.
However, we base our analysis on the signal-tonoise ratio rather than compression.
On the other hand [21] focus on the computational complexityof DDGM and propose a progressive distillation that iteratively reduces the number of diffusion steps.
The work shows that it is possible to considerably reduce the number of sampling steps withoutlosing performance.
We believe that their results support our intuition that it is reasonable to combineseveral initial steps into a single denoiser model.
In [3], authors evaluate how the diffusion processchanges in time when model is trained with different objectives (Eq. 7 or Eq.
They observe thatthe image generation process differs significantly and that it is more beneficial to switch betweenthose two approaches at different stages of the diffusion.
In this work, we also investigate changes inthe diffusion process, but we focus on the generative and denoising capabilities of the model instead.
Connection to hierarchical Variational Autoencoders Several works have noted the connection of
DDGM to VAEs.
In [10] authors focus on the continuous diffusion models and draw the connectionto the infinitely deep hierarchical VAEs.
In [11] authors further explore this connection, formulate a
VLB objective in terms of the signal-to-noise ratio and propose to learn noise schedule, which bringsthe forward diffusion process even closer to the encoder of a VAE.
Recently a latent score-basedgenerative model (LSGM) was proposed [27], which can be seen as a VAE with the score-basedprior.
We follow a similar direction and propose to see a DDGM as a combination of a denoisingauto-encoder with an additional diffusion-based generator of corrupted images.
Experiments
[Figure] Experimental setup In all the experiments, weuse a U-Net-based architecture with timestep embeddings as proposed in [8, 17].
We train all themodels with a linear β scheduler and uniform stepssampler to simplify the comparison.
All implementation details and hyperparameters are included in the Appendix ( A.4) and code repository.
For DAED, weuse the same architecture for both the diffusion partand the denoising autoencoder.
We run experimentson three standard benchmarks with different complexity: FashionMNIST [31] of gray-scale 28 × 28images, CIFAR-10 [13] of 32 × 32 natural images, and CelebA [14] of 64 × 64 photographs offaces.
We do not use any augmentations during training for any dataset.
We report results for bothvariational lower bound loss (VLB) [22] and simplified objective [8].
Following [17] we evaluate thequality of generations with Fréchet Inception Distance (FID) [6] and distributions Precision (Prec)and Recall (Rec) metrics [19] that disentangle FID score into two aspects: the quality of generatedresults (Precision) and their diversity (Recall).6.1 Is there a transition in functionality of the backward diffusion process that switches fromgenerating to denoising?
In section 3, we investigate how the signal-to-noise ratio and the reconstruction error of a DDGMchange with the increasing number of diffusion steps (see Figure 3).
Based on this analysis, wepostulate that DDGMs can be divided into two parts: a denoiser and a generator .
To determine theswitching point, we propose an experiment that answers the following question:
Is there a denoising part of a DDGM that is agnostic to the signal from the data?
To that end, we refer once more to the analysis of the reconstruction error (e.g., MAE) from differentdiffusion steps.
This time, however, we compare the quality of reconstructions with a single DDGMmodel trained on the CIFAR10 dataset and then evaluated on CIFAR10 and CelebA. The result of thisexperiment is presented in Figure 3.
Interestingly, we notice that for approximately 10% of the initialsteps of the DDGM, there is a negligible difference in the reconstruction error between these twodatasets.
This fact may suggest that, indeed, the model does not require any information about thebackground data signal in the first steps, and it is capable of denoising corrupted images.
However,after this point (about 10% of steps), the reconstruction error starts growing faster for the dataset themodel was not trained on.
This indicates that information about the domain becomes important andaffects performance.
6.2 How does splitting DDGMs into generative and denoising parts affect the performance?
The results so far confirm our claims that DDGMs could be divided into denoising and generativeparts.
Independently of a dataset, there appears to be a transition point at which a DDGM stopsgenerating a corrupted image from noise and starts denoising it in a generative manner.
Here, weaim to verify whether it is possible to do a clear split into a denoising part and a generating part.
Forthis purpose, we use the introduced DAED approach that consists of a DAE part (the denoiser) and a
DDGM (the generator) parameterized by two distinct U-Nets.
[Figure]
) on three different datasets.
First, we consider a situation in whichwe train a DDGM using the simplified objective (8) and then replace thefirst steps with a DAE.
In other words,we train a DAED in two steps: firstthe DDGM and then the DAE.
Thisexperiment aims to check how the decoupling of the DDGM into two partsinfluences the model performance.
In
Figure 4 we present the dependencybetween the log-SNR at the splittingpoint and the FID score.
In all cases,the performance of DAED is comparable to the DDGM if we replace the
DAE with up to the 10% of the stepsthat correspond to log ( SNR ) is equalto around 4 .
For more complicated datasets like CIFAR10 and CelebA, fewer steps could be replaced.
This effect could be explained by the fact that images in these datasets have three channels (RGB), andremoving noise is more problematic.
That outcome reconfirms our presumptions that it is reasonableto split the DDGM since the final performance is not significantly affected by the division for anadequately chosen splitting point.
To get further insight into the qualitative performance, in Figure 5 we demonstrate how the selectionof the splitting point with respect to the Signal to Noise Ratio (SNR) affects the quality of final
[Table]
generations 1 .
We present non-cherry-picked samples from DAED trained in the same manner asdescribed in the previous paragraph.
As expected, the more noise the DAE part (the denoiser) mustdeal with (see the values of β 1 in Figure 5), the fewer details in the generations there are.
Thesesamples again indicate that by replacing some steps with a denoiser, we get a trade-off between”cleaning” the corrupted image or, in fact, further generating details.
It seems that there is a sweetspot for perceptually appealing images that contain details and are ”smooth” at the same time, seeβ 1 = 0 .
025 in Figure 5.
However, as it is typically difficult to provide convincing arguments bystaring at samples, we further propose to analyze quantitative measures.
In Table 1, we compare the performance of DAED against the DDGM on FashionMNIST, CIFAR10,and CelebA in terms of FID, Precision and Recall scores.
We want to highlight that our goal is not toachieve SOTA results on the before-mentioned datasets but to verify whether we can gain some furtherunderstanding and, potentially, some improvement by splitting the denoising and generative parts.
We consider two scenarios, namely, learning a DDGM and DAEDs using either the variational lowerbound (VBL) or the simplified objective (Simple) with various lengths of the diffusion.
Interestingly,
DAED outperforms the DDGM when these models are trained using the VBL loss.
For the simplifiedobjective, DAED trained with the same number of diffusion steps yields slightly lower performancethan standard DDGMs.
As indicated by the Precision/Recall, generations from DAED are as preciseas those from DDGM.
However, they lack certain diversity, probably due to the smoothing effect ofthe DAE part.
Detailed results for other setups are presented in Appendix A. {+
+}
6.3 Does
the noise removal in
generalize to other data distributions The last question we are interested in is the generalizability of DDGMs to other data distributions.
We refer to this concept as transferability for short.
In other words, the goal of this experiment isto determine whether we can reuse a model or its part on new data with as good performance aspossible.
In this experiment, we rely on the results presented in Section 6.1 where roughly the first10% of steps could be seen as the denoising part.
To further strengthen this perspective, we alsoutilize DAED with an explicit division into the denoising and generating parts.
[Table]
[Table]
[Table]
First, we consider the case in which we compare the reconstruction errors measured by the MAEand the MS-SSIM.
In this scenario, we train a DDGM on a source dataset
and then assess it ona target dataset.
We use CIFAR10 or ImageNet (32x32 or 64x64) as source data and CIFAR10,1 Generations for all datasets are presented in Appendix A.
CIFAR100, or CelebA as target data.
For each image from the target dataset, we apply the DAEpart of DAED to obtain the reconstruction or 793 steps of the forward and backward diffusion in thecase of the DDGM, which corresponds to the same level of added noise.
For this experiment, weuse the pre-trained DDGM from [17] that consists of 4000 steps and uses the cosine noise scheduler.
The results are outlined in Table 2.
First of all, there is no significant difference in the performanceof DDGMs trained with either the VBL objective or the simplified objective.
They achieve a quitesatisfactory MAE and MS-SSIM scores.
However, DAED outperforms the DDGMs, obtaining muchbetter transferability.
We explain it by the fact that probably, with each step in the denoising part
DDGM adds details that are typical for source data while DAED focuses on removing noise andproduces a smoother output.
This outcome may further suggest that splitting DDGMs into two partswith two separate parameterizations is reasonable and even beneficial.
To get further insight into the transferability behavior, we present a few (non-cherry-picked) examplesfrom CelebA in Figure 6.a and four toy examples in Figure 6.b.
We use the same setup as explainedin the previous paragraph (i.e., the pre-trained DDGM provided in [17]), and the images are noisedwith β 1 = 0 .
In columns 3–6 in Figure 6.a, we present reconstructions for the DDGM trainedon CelebA, the DDGM trained on ImageNet, DAED trained on CelebA, and DAED trained on
ImageNet, respectively.
It becomes apparent that the DDGM trained on CelebA denoises the imageby generating new details while DAED denoises by smoothing.
Interestingly, DAED performs betterthan the DDGM when we use ImageNet-trained models to denoise CelebA.
In Figure 6.b, we depictseveral toy examples that were denoised with the DDGM and DAED trained on CIFAR10.
We seethat the DDGM adds many details that are artifacts from the source data.
It seems that DAED doesnot suffer from that behavior.
[Figure]
Conclusion
In this work, we investigate the generative and denoising capabilities of the Diffusion-based Deep
Generative Models.
We observe and experimentally validate that it is reasonable to understand
DDGMs as a combination of two parts.
The first one generates noisy samples from the pure noise byinputting more signal from a learned data distribution, while the second one removes the remainingnoise from the signal.
Although for standard DDGMs, the exact switching point between those twoparts is fluid, we propose a new approach dubbed DAED that is explicitly built as a combination of agenerative component (a DDGM) and a denoising one (a DAE).
In the experiments, we observe that
DAED simplifies training with a standard VLB loss function that leads to improved performance.
Onthe other hand, with increasing noise processed by DAE, DAED smoothens the generations resultingin lower performance when training with the simplified objective.
We further show that DDGMs, and
DAED especially, generalize well to unseen data, what opens new possibilities for further research interms of transfer or continual learning of DDGMs.