Autoregressive Generative Modeling with Noise Conditional Maximum Likelihood Estimation
Authors: Henry Li
Yuval Kluger
Venue: Submitted to ICLR 2023 (2023)
Keywords: density estimation
autoregressive models
generative modeling
score-based models
diffusion models

Abstract: We introduce a simple modification to the standard maximum likelihood estimation (MLE) framework. Rather than maximizing a single unconditional likelihood of the data under the model, we maximize a family of \textit{noise conditional} likelihoods consisting of the data perturbed by a continuum of noise levels. We find that models trained this way are more robust to noise, obtain higher test likelihoods, and generate higher quality images. They can also be sampled from via a novel score-based sampling scheme which combats the classical \textit{covariate shift} problem that occurs during sample generation in autoregressive models. Applying this augmentation to autoregressive image models, we obtain 3.32 bits per dimension on the ImageNet 64x64 dataset, and substantially improve the quality of generated samples in terms of the Frechet Inception distance (FID) --- from 37.50 to 12.09 on the CIFAR-10 dataset.

Under review as a conference paper at ICLR 2023
AUTOREGRESSIVE GENERATIVE MODELING WITH NOISE CONDITIONAL MAXIMUM LIKELIHOOD ESTI- MATION
Anonymous authors Paper under double-blind review
ABSTRACT
We introduce a simple modification the standard maximum likelihood estima- tion (MLE) framework. Rather than maximizing a single unconditional model likelihood, we maximize family of noise conditional likelihoods consisting of the data perturbed by a continuum of noise levels. We find that models trained this way are more robust to noise, obtain higher test likelihoods, and generate higher quality images. They can also be sampled from via novel score-based sampling scheme which combats the classical covariate shift problem that occurs during sample generation in autoregressive models. Applying this augmentation to autoregressive image models, we obtain 3.32 bits per dimension on the Ima- geNet 64x64 dataset, and substantially improve the quality of generated samples in terms of the Frechet Inception distance (FID) from 37.50 to 12.09 on the CIFAR-10 dataset.
INTRODUCTION
Maximum likelihood estimation (MLE) is arguably the gold standard for probabilistic model fit- ting. and serves the de facto method for parameter estimation in countless statistical problems Bishop ( across variety of fields. Estimators obtained via MLE enjoy number of theoret- ical guarantees, including consistency, efficiency, asymptotic normality, and equivariance to model the field of density estimation and generative model- ing, MIEE models have played a key role, where autoregressive models and normalizing flows have boasted competitive performance in a bevy of domains, including images Child et al. (2019), text
reparameterizations Van der Vaart 2000
Vaswani et al. (017)) audio Oord et al. (2016), and tabular data Papamakarios et al. (2017).
broadly agreed upon as one of the most rigorous metrics for goodness-of-fit in statistical and generative modeling, models with high likelihoods do not nec- essarily produce samples of high visual quality. This phenomenon has been discussed at length by Theis al. 2015); Huszár (015), and corroborated in empirical studies Grover al. (20T8): Kim et al.] (2022). Autoregressive models suffer an additional affliction: they have notoriously unstable dynamics during sample generation Bengio al. (2015); Lamb al. (2016) due their sequential sampling algorithm, which can cause errors to compound across time steps. Such errors cannot usually be corrected ex post facto due to the autoregressive structure of the model, and can substan- tially affect downstream steps as we find that model likelihoods are highly sensitive to even the most minor of perturbations.
However, while log-likelihood
Score-based diffusion monels|Song et al. (2020b): : Ho et al. (2020) offer different perspective on the data generation process. Even though sampling is also sequential, diffusion models are more robust to perturbations because, in essence, they are trained as denoising functions Ho et al. (2020). More- over, the update direction in each step is unconstrained (unlike token-wise autoregressive models, which can only update one token at a time, and only once), meaning the model can correct emror from previous steps. However, likelihood evaluations have no closed form, requiring ODE/SDE solvers and hundreds to thousands of calls to the underlying network, and rendering the models incapable of being trained via MLE. Diffusion models also do not inherit any of the asymptotic guarantees Hyvvrinen & Dayan
2005 Song al. 22020a of score matching, even though they

---

Under review as a conference paper at ICLR 2023
Generated samples CelebA 64x64 (above) and CIFAR-10 (below). Autoregressive models trained via vanilla maximum likelihood (left) are brittle to sampling errors and can quickly diverge, producing nonsensical results. Those trained by algorithm (right) more robust and ultimately generate more coherent sequences.
Figure 1:
are thusly trained These details make diffusion models theoretically inferior and less viable for many downstream tasks that involve the likelihood, such anomaly and out-of-distribution (OOD)
detection Ren et al. 2019 adversarial defense [Sng et al. 2017). among others. Thus we wonder: there conceptual middle ground?
In this paper, we offer such a framework. We further analyze the likelihood-sample quality mis- match in autoregressive models, and propose techniques inspired by diffusion models to alleviate it. In particular, we leverage the fact that the score function is naturally learned as a byproduct of max- imum likelihood estimation. This allows a novel two-part sampling strategy with noisy sampling and score- score-based refinement.
Our contributions are threefold. 1) We investigate the pitfalls of training and inference under the maximum likelihood estimation framework, particularly regarding sensitivity to noise corruptions. 2) We propose simple sanity test for checking the robustness of likelihood models to minor pertur- bations, and find that many models fail this test. 3) We introduce a novel framework for the training and sampling of MLE models that significantly improves the noise-robustness and generated sample quality. As result, we obtain model that can generate samples quality approaching that of dif- fusion models, without losing the maximum likelihood framework and O(1) likelihood evaluation speed of MLE models.
BACKGROUND AND RELATED WORK
Let our dataset consist of i.i.d. samples drawn from an unknown target density x Pdate (x). The goal of likelihood-based generative modeling is to approximate Pdato via a parametric model po. where samples x ~ Po can be easily obtained.
'Though each conditional score is trained via score matching. the final model depends heavily on the solu- tion of chosen SDE which is not specified by the framework. Thus score matching does not directly produce diffusion model.
2

---

Under review as a conference paper at ICLR 2023
2.1 BACKGROUND
We first discuss fundamental techniques for estimating and sampling from Pa in generative model- ing.
Score-based Modeling technique for generative modeling that has recently grown in popular- ity is score-based modeling, which involves minimizing sum of weighted score matching losses Hyvärinen & Dayan (2005) on noise-corrupted data
arg min Pa.
B log Vx log (()dd,
where p(X(x) describes corruption process and Vx log p(x) is also known the (Stein) score function. Sampling from pe can then be achieved via annealed Langevin dynamics Song & Ermon 2019), variational denoising [H et al. (2020), or reversing diffusion process Song et al. (2020b).
To obtain a proper likelihood value, score-based models must be framed as a diffusion process, where the density is then obtained the solution of a related ordinary differential equation (ODE). Here, each data point is modeled as a function x : [0, + Rd such that x(0) Pdata and xTT Pprior. The forward diffusion process is an Ito stochastic differential equation (SDE)
dx = f(x, t) + g())dw,
for some drift and diffusion terms f and y. where is the standard Wiener process. By result in Anderson (1982), this diffusion can be tractably reversed
dx f(x, t) + g²) + V logpe(x)]dt g())d,,
and depends additionally on the noise-conditional score function, where w is a backwards Wiener process. Sampling then consists of drawing x(T) Pprior, and solving the reverse diffusion pro- cess.
Maximum Likelihood Estimation (MLE) The standard MLE framework consists in solving
arg max E Exx-paara log (ox) arg max
log (x
X
XEX
Likelihood models draw samples pe one of two ways. Normalizing flows Dinh et al. (2014); Rezende Mohamed (2015): Grathwohl et al. (2018) apply a series of invertible transformations to a latent variable E R Pprior. On the other hand, autoregressive models sample x sequentially and wise by drawing from each conditional likelihood.
In this work, we draw an explicit distinction between MLE and other parameter estimation tech- niques that increase the likelihood, such as score-based modeling or variational lower bound maxi- mization, which is sometimes regarded maximum likelihood [Sng et al. 2202 The key differ- ence is that MLE provides theoretical guarantees on the resulting model including consistency, efficiency, asymptotic normality |aan der Vaart 2000 tional lower bound maximization do not.
whereas cose- modeling and varia-
2.2 RELATED WORK
A number of other works combine score- and energy-based modeling with autoregressive archi- tectures. Hoogeboom et al. (202T) propose order-agnostic autoregressive model for simulating discrete diffusions via variational lower bound. Meng et al.) (2020) use unnormalized autoregres- sive models to learn distributions in an augmented score-matching framework. Nash & Durkan (2099) design an energy-based model with an autoregressive structure such that the normalizing constant can be estimated coordinate-win via importance sampling. Similar to our approach, Meng ct al. 202T) decompose the data generation process into a noisy sampling step and a denoising step, but their formulation introduces latent variables and does not support adaptive refinement strategies. Ultimately, each approach relinquishes the ability to compute exact likelihoods in their framework, which is one of the motivating advantages of using autoregressive structures.
A closely related vein of research explores alternative training and inference strategies so as to im- prove sampling stability in autoregressive models. [Bengio al. (2015) propose training models
3

---

Under review as a conference paper at ICLR 2023
Figure 2: Images from CIFAR-10 (top) versus their pr perturbed counterparts for 0.5 (middle)
and π = 1 (bottom). The differences are nearly indistinguishable to the human eye, yet cause
drastic deviations in average log-likelihood for standard likelihood models (Section 3 and Table I.
with mixture of true and generated samples over time, where the proportion of generated samples gradually grows to take up the majority of training sequences. This is initially promising, except Huszár (2015) note that this technique biased and not guaranteed produce solutions that con- verge on the true distribution. Lamb et al. (2016) subsequently suggest incorporating an adversarial loss provided by a discriminator that "teaches" the model produce more realistic samples over multiple sampling steps. Both techniques again depart from the maximum likelihood framework.
Jun et al. (222)) considers alternative forms of data augmentation, which experimentation in Kingma et al. 2 show is synergistic with noise conditional perturbations during training. Perhaps most similar to our approach is Jayaram & Thickstun (2021). l). who also suggest sampling via the score function with Langevin dynamics, but they crucially do not train with noise, which Song & Ermon (2019) found to be essential for stable sampling in a Langevin algorithm. As a result, they are not able to sample images unconditionally.
Ultimately, our approach uniquely provides a principled and generalized framework for modeling stochastic processes (including diffusions) that retains the asymptotic guarantees of maximum like- lihood estimation while producing samples of superior quality.
3 THE PITFALLS OF MAXIMUM LIKELIHOOD ESTIMATION
We first show that density models trained maximize the standard likelioel are surprisingly sensitive to minor perturbations. We then discuss why this bad for generative modeling perfor- mance.
3.1 SIMPLE SANITY TEST
Consider the class of minimally corrupted probability densities discrete k-bit integer space (e.g., 8-bit images, or 16-bit digital audio signals):
24
Pr (x)
Σ Ix (x)x))(xx, x=1
(((xx) = (bttflip(x,) k)
fx -
w.p. 1-T T w.p. π
(5)
where E [0. 1] and bitflip(x, E) is the bit flip operation on the æth bit of x. Pr is minimally corrupted in the sense that p- describes the distribution of points in Pdata that have been perturbed by flipping the least significant bit (LSB) with probability n.
In 8-bit images, the difference between samples drawn from Px and Pdato is imperceptible to the human eye, even for = 1 (see Fig 2). However, for likelihood models, this perturbation drastically increases the negative log-likelihood (Table) failing this test is problematic, especially for autoregressive models.
the next section, we provide three reasons for why
3.2 WHAT THIS MEANS (FOR GENERATIVE MODELING AND DENSITY ESTIMATION)
First, noise is natural and being less robust to noise also means being poorer fit to natural data, especially in the ways that matter to the end user. Outside of the log-likelihode measures of generative success in generative models fall under two categories: qualitative assessments (e.g..
4

---

Under review as a conference paper at ICLR 2023
the no-reference perceptual quality assessment Wang et al. (2002) 'eyeballing' it) and quantitative heuristics (e.g., computing statistics of hidden activations of pretrained CCNNs|Saliman et al. (2016);
Heusel et al. (2017); Sajjadi et al. (2018)) Both strategies either rely directly on the human visual system, or are known to be closely related to it Güşlüü & van Gerven (2015); Yamins et al. (014);; al., (2017); Cichy al. (2016). Therefore, implicit in the use of these criteria is the existence of a human (or human-like -ike) model of images
Khaligh-Razavi & Kriegeskorte (2014); Eickenberg
Qhumans where Ghamon Pdato Huszár (2015). Our sanity test thus shows that we are still quite far from obtaining Iruman- We summarize this observation with the following lemma.
Lemma 3.1. (Informal) Let Quaman be human-like like distribution on images, and pe be a likelihood model that not robust to a visually imperceptible perturbation 9x- Then pa cannot be equal to Qhaman almost surely.
Second, generative sample quality suffers, especially for autoregressive models. Due to the sequen- tial nature of autoregressive sampling and the fact that models are presented only with data from the true distribution during training, autoregressive models are already known to be
to handle the sequences they encounter during sampling Bengio et al. (015); Lamb et al. (2016); Meng et al. (2021). Any sampling error introduces a distributional shift that can affect the modeled dis- tribution on downstream tokens. This will increase the risk of mis- mis-sampling the next token, which, in turn, further affects downstream modeling. This is related to the well-known covariate shift phe- nomenon (Shimodaira (2000). Sensitivity minor perturbations only exacerbates the problem, and Table Table[shows that in vanilla autoregressive likelihood models, mis-sampling pixels by even a single bit can cause drastic changes to the overall likelihood. This can explain why such models commonly produce nonsensical results (Fig
Third, we may need to re-evaluate our research trajectory in image-based density estimation. One way to interpret Table is is via the following lemma.
Lemma 3.2. Let q be a distribution on d-dimensional k-bit data x E {o...., 24.d. Denote x; E {0. 1 as the bit mask containing the ith bit of x. When = 2. we have
Exapp [g(x)] Exx-pasu [9(xx,,..,, x1-)))
where g(x
Sec, g(x)dx is the marginal of q(x) (after marginalizing out xx).
In other words, the expected likelihood with the full k-bit image over p1 is equivalent the expected likelihood with the first k most significant bits over Pdata- T that while there is large variance in likelihoods with æ O, most models achieve between 3.75- 3.85 BPD with π = 0.5. This suggests that most gains so far in likelihood modeling have been focused on modeling the least significant bit, which ultimately bears little significance to the inherent content of the image. Conversely, NCML trained models are significantly more effective at modeling the remaining bits.
For these reasons, we find that improving nobse-nobus is of central importance to likelihood- based generative modeling, and especially likelihood beelihood-based autoregressive generative models.
4 NOISE CONDITIONAL MAXIMUM LIKELIHOOD
we propose a simple modification the standard objective in maximum likelihood estimation. Rather than evaluating single likelihood as in the vanilla formulation, we consider a family of noise conditional likelihoods
To alleviate the problems discussed in Section 3
Exxp,Exp, logpe. (x).
where Pt is a stochastic process indexed by noise scales f modeling a noise-corrupting process on Pdatos and p distribution over such scales. We call this the noise conditional maximum likelihood (NCML) framework. In general, Eq 7 is an all-purpose plug-in objective that can be used with any likelihood model adapted accept a noise conditioning vectof We now explore various perspectives of NCML that may help with reasoning about this framework.
"Though continuous likelihood or a discretization of it (e.g. normalizing flows Dinh et al. 2014 or autoregressive models with non-softmax probabilities (Salimans al. (2017) La & Kluger (2022) necessary for computation of the score function.
5

---

Under review as a conference paper at ICLR 2023
1. Coise-Conditions. Maximum Likelihood Training
argmax Ee (x00))m log (x,((x())
2. Noisy Sampling
x(T), ~ par(((() (Xx(()))
x/O)
dx f(x, () + g())d
X(T)
x(13
(f(x.)) + 500t) Vados(s]]o glflow
3. Score based Refinement
Figure 3: An overview of the NCML generative algorithm. There are three steps: 1) We train the noise conditional density model po. via NCML. We sample from the modeled noisy distribution, ie., P8.r with some > 0. 3) We refine the sample by solving reverse diffusion involving the learned score function V (x).
NCML Regularized Maximum Likelihood When the set of noise scales E T is finitely large, a natural and mathematically equivalent formulation of NCML is as a form data- and model- dependent regularization of the standard maximum likelihood estimation objective:
Etsp Exup, log (x)) x Excepate log (o..oo..))))) (x) +
Σ AxExpp log (x))
(8)
IET
MLE objective
regularization term
where T comprises the of nonzero noise scales and X, p())/p(O). Clearly, the standard likelihood can be considered a special case of our proposed method where X = O for all / E T. Furthermore, since the NCML framework can simply be seen formulating [T] simultaneous and separate MLE problems, it retains all the statistical properties of standard MLE.
Of course, any form of regularization introduces bias the model framework. Whereas LO/L1/L2 regularizations bias towards solutions of minimal or sparse weight norm, our experiments suggest that NCML biases towards solutions that are less sensitive noisy perturbations (see Section 4.1
NCML as Diffusion Model Letting be the time index of diffusion process, our approach becomes elosely related to score-based diffusion models (Song et al. (2020b). albeit with two crucial differences.
First, instead of merely estimating the noise-conditional score Va Pe) for T, we directly estimate itself. However, V. log (x) is still learned by-product of NCML, as it is the derivative of the log the learned quantity. may then access our approximated score via standard backpropagation techniques. Therefore, like diffusion models, we can draw samples via Langevin dynamics. This provides an alternative strategy for sampling from Pe,s, which we explore 4.2
Second, need not design diffusion so that pr approximates the limiting stationary distribution of the process. This necessary in diffusion models as the limiting prior is the only tractable distribution initialize the sampling algorithm with. Since have learned the density itself for all E T, we may initialize from any point of the diffusion, which increases the flexibility of the sampling strategy, and can drastically reduce the steps required to solve the reverse diffusion.
For our models, we consider the three diffusion processes proposed in Song et al. (2020b): variance exploding (VF), variance preserving (VP), sub-variance preserving (sub-VIl VP), and choose y be

---

Under review as a conference paper at ICLR 2023
1.2
00g0
0.
Figure 4: Noise robustness of NCML-trained models with Variance Preserving (VP), sub- Variance Preserving (sub-VP), and Variance Exploding (VE) noise schedules, measured in terms of A logp (defined in ( between Pdate and Px. for x = 0.5 (left) and ग 1 (middle). (Right) shows that robustness is closely related to the average absolute perturbation per pixel of each noise schedule, as function of t. More details in Section 4.1
the uniform distribution over T. Due to space constraints, we refer to the aforementioned paper for more details these SDBs.
4.1 ROBUS-NSS OF NCML MODELS
We find that models trained via NCML are more robust noise, suggesting increased stability during autoregressive sampling and improved modeling of more visually salient features (Section Surprisingly, this not only the case for noise conditions t O, where the model is exposed to noisy samples. Indeed, even in the / = O condition, noise-robustness is improved (Table and our models surpass state-of-the-ard likelihood models on minimally perturbed data (as defined in Eq 5) in terms of average log-likelihood. This is somewhat unexpected: by passing the noise condition, the model should theoretically be able to separate the NCML loss into distinct problems at each noise scale. If this occurs, then the 0 case should simplify to vanilla MLE problem, where behavior would not differ from standard likelihood models.
This lends credence to the regularization perspective provided in 4 In essence, noise robustness is explicitly enforced for t > 0. For t = 0, NCML leverages the limited capacity of the underlying network to implicitly impose robustness. To quantify the noise-robustness at each i, we define the simple measure A log as the absolute difference between the negative model log likelihood (as measured in bits per dimension) evaluated on Pdata minus that on Px. i.e.,
A log [Epaa log Po E.p. log pa
(9)
The left and middle graphs in Fig 4 show A logg function of for ग 0.5 and π 1. respectively. Here, we can clearly see that noise robustness increases with increasing (. Moreover. the regularization effect of NCML enforces greater robustness than competing models even O. as seen by the dotted line showing the next lowest A logp, p, indicating that NCML enforces some degree of noise-robustness as regularization.
As expected, the correlation between noise-robustness and t follows closely to the noise schedules of VP, sub- VP, and VE SDEs in [0, T| respectively. This can be seen in the rightmost plot of Figure which shows the average absolute perturbation per pixel of each SDE over time. The dotted line represents 1, i.e., the absolute perturbation of corrupted images in our sanity test Pa. The point at which each model attains robustness to Pa corrupted noise is more or less the same time the noise schedule begins to perturb each pixel by at least one bit, on average.
The increased noise-robustness of NCML-trained models at larger t motivates our improved autore- gressive sampling algorithm, which we introduce below.
4.2 SAMPLING WITH AUTOREGRESSIVE NCML MODELS
Our framework allows for two sampling strategies. The first is to draw directly from the noise-free distribution Pe.o, in which case the conditional likelihood simplifies to standard (unconditional) likelihood, and sampling is identical to that for a vanilla autoregressive model.
However, discussed in Section this strategy is unstable and tends to quickly accumulate errors. This motivates an alternative two-par woo-part sampling strategy, which involves drawing from po., for t (the noisy sampling phase), then solving reverse diffusion process back to (the score-based
7

---

Under review as a conference paper at ICLR 2023
CIFAR-10
ImageNet 64x64
Model
FID
NLL
NLL
NLL
NLL
NLL
NLL
0
0.5
π 0
0.5
ELBO
VDM
7.41
2.49
3.76
3.96
3.40
3.76
3.87
ScoreFlow
5.40
2.90
3.83
3.98
VDVAE
2.84
3.89
4.10
3.52
3.63
3.82
Likelihood
Flow++
3.09
3.84
4.09
3.69
3.82
3.99
DenseFlow
48.15
2.98
3.79
4.02
3.35
3.69
3.85
+ixeICNN+
55.72
2.92
3.85
4.02
3.52
3.81
3.99
PixelSNAIL
36.62
2.85
3.83
3.99
Sparse Transformer
37.50
2.80
3.82
3.97
3.44
3.73
3.89
NCPN (ML)
46.72
2.91
3,82
3.99
3,49
3.68
3.90
NCPN (NCML-VE)
32.71
2.87
3.73
3.94
3.32
3.67
3.85
NCPN (NCML-subVP)
23.42
2.95
3.68
3.93
3.36
3.66
3.80
NCPN (NCML-VP)
12.09
3.20
3.61
3.89
3.43
3.62
3.76
Table 1: Results CIFAR-I0 and ImageNet 64x64.
Negative log-likelihood (NLL) is in bits per dimension. Lower is better. *NLL with π = 0 equivalent to NLL of the original data.
refinement phase). The tractability of the latter is due to the faet that NCML-trained models learn the score as a byproduct of likelihood estimation. This is identical score-based diffusion models SSnn et al. (2020b), except for the key difference that we need not initialize with the prior distribution, as we can sample from any E T.
the sampling procedure in
This two-part strategy has several benefits. First, we saw in Section 4.1 that NCML-trained models are more robust to noise at higher t, which improves stability during sampling. This is reflected in the improved FID of NCML-trained NCPNs compared to ML- (i.e., maximum likelihood-) trained NCPNs Table I Second, the refinement phase allows the model to make fine-tuned adjustments to the sample, which further improve quality. This is not possible in standard autoregressive models by construction.
EXPERIMENTS
We demonstrate that incorporating noise in the maximum likelihood framework provides signifi- cant improvements in terms of density estimation, sample generation, and anomaly detection. In all experiments, we fix to be one of the variance exploding (VE), variance preserving (VP), or sub-variance preserving (sub-VP) SDEs, and f be the uniform distribution over / T. For architecture, we introduce the noise conditional pixel network (NCPN), which consists of a Pixel- CNN+ Salimans al. (2017) backbone with added attention layers. More experimental details can be found in A.2
Unconditional Modeling We evaluate our models minimally perturbed transformations (see
Section 3.1 of unconditional CIFAR-10 and ImageNet 64x64 for E {0, 1. where we note All noise conditional models. i.e., ours, VDM Kingma et (202T),
that P- = Pdata when and ScoreFlow (Song et al. (2021). are evaluated at = 0. We show our results in Table 1 We additionally evaluate our model on the CelebA 64x64 dataset; autoregressive comparisons on this dataset are limited, so we defer results to the appendix.
On the standard unperturbed dataset = 0, our models attain competitive likelihoods on CIFAR-10 and state-of-thed -he---rtttttt likelihoods on ImageNet 64x64. On all perturbed datasets, models achieve state-of-ther offtthf---atttt likelihoods. Furthermore, we significantly improve on the state-of-the-art in terms of sample quality among MLE models on CIFAR-10, from 37.50 to 12.09 in terms of FID. In general, our results indicate that the NCML framework improves generative modeling performance of the underlying models in terms of both test liklliioooods and sample quality.
8

---

Under review as a conference paper at ICLR 2023
Figure 5: Class-conditional sampling on CIFAR-10 (left). Image completion on CIFAR-10 (right).
See Section 5 for more details.
Class-conditional Modeling We find that NCML-trained models exhibit stable sampling in class-conditional generation. For this experiment, we train a class-conditional model under our framework on the CIFAR-IO dataset. We show results in Figure where each row shows data sampled from a different class of CIFAR-10.
Image Completion We further examine NCML-trained models in a controllable generation context through the image completion task, which involves conditioning an autoregressive model on the first half of an image, and drawing the second half from the modeled conditional distribution. We again use the CIFAR-10 dataset, and compare models trained under our framework against those trained with MLE, All images are taken from the test
minimize data memorization. Our results are in figure 5 (right). The leftmost and rightmost four columns are outputs generated by an MLE-trained model and an NCML-trained model, respectively, with the middle column depicting input to the sampling algorithm. Both models the same architccture. Our model demonstrates improved stability over the course of sampling, and produces completed images with greater realism and fidelity, while maintaining high diversity of sampled trajectories.
masked
-ut-of-distribution (OOD) Detection We show that robustness to minimally perturbed data confers benefits that extend beyond en- hanced sample quality, including marked im- provements anomaly detection. In this task,
Ditu/Medel
PiseCNNN+
GLOW
FRM
AR-CSM
(VP)
SVHN
0.32
0.24
0.63
0.68
0.74
Coest Uniform
0.0
0.0
0.30
0.57
0.68
Uniform
Average
the network trained on in-distribution points, then tasked to produce a statistic that success- fully differentiates in-distribution points from out-of-distributio points. Following Du &
Table 2: AUROC scores of models trained on CIFAR-10 on the OOD detection task.
Mordatch| 2019h; Meng et al. 2020 we train models on CIFAR-10, and let the OOD points be
SVHN Netzer et al. 2011] constant uniform, and random uniform images. We compare the Area
Under the Receiving Operator Curve (AUROC) our model statistic to vanilla PixeICNN+ Sal-
mans et al.
2017), GLOW & Dhariwal 2018 EBM Du & Mordatch (0199) and AR-CSM
Meng et a..
2020 models, and see that our model performs equally or better in every respect (Table

---

Under review as a conference paper at ICLR 2023
6 CONCLUSION AND FURTHER WORK
We proposed a simple sanity test for checking the robustness of likelihoods to visually imperceptible levels of noise, and found that most models are highly sensitive perturbations under this test. We argue that this is further evidence of fundamental disconnect between likelihoods and other sample quality metrics. To alleviate this issue, developed novel framework for training likelihood models that combines autoregressive and diffusion models in a principled manner. Finally, we find that models trained under this setting have substantial improvements in both training and evaluation.
While models trained under the NCML framework show greater invariance to imperceptible noise, they are by no means robust, indicating that the underlying model still differs significantly from the theoretical human model Qhuman proposed in Huszár (2015). We hope that further research can help close this gap, and furnish us with a more intuitive grasp on the maximum likelihood as a framework for assessing goodness-of -f-fit in generative models.
REFERENCES
Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Ap- plications, 12(3):313-326, 1982.
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. Advances in neural information processing systems, 28, 2015.
Christopher M Bishop. Pattern recognition and machine learning. Number 4 in 4. Springer, 2006.
Xi Chen, Nikhil Mishra, Mostafa Rohaninejad, and Picter Abbeel. Pixelsnail: An improved au- toregressive generative model. In International Conference on Machine Learning, pp. 864-872. PMLR, 2018
Rewon Child. Very deep vaes generalize autoregressive models and can outperform them on images. arXiv preprint (xXix2201.10065) 2020.
Rewon Child, Scott Gray, Alec Radford, and Ilya Sutskever. Generating long sequences with sparse transformers. arXiv preprint arXiv: 10509, 2019.
Radoslaw Martin Cichy, Aditya Khosla, Dimitrios Pantazis, Antonio Torralba, and Aude Oliva. Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hicrarchical correspondence. Scientific reports, 6(1):1-13, 2016.
Laurent Dinh, David Krueger, and Yoshua Bengio. Nice: Non-linear independent components esti-
mation. arXiv preprint arXiv:1410.8511 2014.
Yilun Du and Igor Mordatch. Implicit generation and generalization in energy-based models. arXiv preprint arXiv: 100.00993300.89, 2019.
Michael Eickenberg, Alexandre Gramfort, Gaöl Varoquaux, and Bertrand Thirion. Seeing it all: Convolutional network layers map the function of the human visual system. Neurolmage, 152: 184-194, 2017.
Will Grathwohl, Ricky TQ Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. Ffjord: Free-form continuous dynamics for scalable reversible generative models. arXiv preprint arXiv:1810.01367. 2018.
Matej Grcié, Ivan Grubišié, and Siniša Šegvié. Densely connected normalizing flows. Advances in Neural Information Processing Systems, 34:23968 -43229,8-23982, 2021.
Aditya Grover, Manik Dhar, and Stefano Ermon. Flow-gan: Combining maximum likelihood and adversarial learning in generative models. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
Umut Güçlü and Marcel AJ van Gerven. Deep neural networks reveal gradient in the complexity of neural representations across the ventral stream. Journal of Neuroscience, 35(27): 10005-0000111 2015.
10

---

Under review as a conference paper at ICLR 2023
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochre- iter. Gans trained by two time-scale update rule converge to local nash equilibrium. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Gar- nett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Asso- ciates, Inc., 2017. URL http:: htttps:://prooeeedinn www.ingg...ennnueeeeeee www. ips. cct/paperr/c17/f1ee 6811699777ebbffffee88811313 71118837636777626--- Paper. pdf
Jonathan Ho, Xi Chen, Aravind Srinivas, Yan Duan, and Picter Abbeel. Flow++: Improving flow- based generative models with variational dequantization and architecture design. In International Conference Machine Learning, pp. 2722-2732 PMLR, 2019a.
Jonathan Ho, Nal Kalchbrenner, Dirk Weissenborn, and Tim Salimans. Axial attention in multidi- 2019b.
mensional transformers. arXiv preprint
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
Emiel Hoogeboom, Alexey A Gritsenko, Jasmijn Bastings, Ben Poole, Rianne den Berg, and
Tim Salimans. Autoregressive diffusion models. arXiv preprint crXiv:2110.02037,220 2021.
Ferenc Huszár. How (not) to train your generative model: Scheduled sampling, likelihood, adver- sary? arXiv preprint arXiv:/511.05100 2015.
Aapo Hyvärinen and Peter Dayan, Estimation of non-normalized statistical models by score match- ing. Journal of Machine Learning Research, 6(4). 2005.
Vivek Jayaram and John Thickstun. Parallel and flexible sampling from autoregressive models via langevin dynamics. In International Conference Machine Learning, pp. 4807 4818-4818. PMLR, 2021.
Heewoo Jun, Rewon Child, Mark Chen, John Schulman, Aditya Ramesh, Alec Radford, and Ilya Sutskever. Distribution augmentation for generative modeling. In International Conference on Machine Learning, pp. 5006-5015 PMLR, 2020.
Seyed-Mahdi Maaligh-Razavi and Nikolaus Kriegeskorte. Deep supervised, but not unsupervised, models may explain it cortical representation. PLoS computational biology, 10(11):e100311, 2014.
Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, and II-Chul Moon. Soft truncation: A universal training technique of score-based diffusion model for high precision score estimation. In International Conference on Machine Learning, pp. 11201-11221 PMLR, 2022.
Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Ad- vances in neural information processing systems, 34:21696-21700 2021.
Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31, 2018.
Alex M Lamb, Anirudh Goyal ALIAS PARTH GOYAL, Ying Zhang, Saizheng Zhang, Aaron C Courville, and Yoshua Bengio. Professor forcing: new algorithm for training recurrent net- works. Advances in neural information processing systems, 29, 2016.
Henry Li and Yuval Kluger. Neural inverse transform sampler. In International Conference Machine Learning, pp. 12813-12825. PMLR, 2022.
Chenlin Meng, Lantao Yu, Yang Song, Jiaming Song, and Stefano Ermon. Autoregressive score matching. Advances in Neural Information Processing Systems, 33:6673-6683, 2020.
Chenlin Meng, Jiaming Song, Yang Song, Shengjia Zhao, and Stefano Ermon. Improved autore- gressive modeling with distribution smoothing. arXiv preprint arXiv:2103. 15089, 2021.
Charlie Nash and Conor Durkan. Autoregressive energy machines. International Conference on Machine Learning, pp. 1735-1744. PMLR, 2019.

---

Under review as a conference paper at ICLR 2023
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits in natural images with unsupervised feature learning. NIPS Workshop on Deep Learning and Unsupervised Feature Learning. 2011.
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. Wavenet: A generative model for raw audio. arXiv preprint arXiv: 1e0..6609999999 2016.
George Papamakarios, Theo Pavlakou, and lain Murray. Masked autoregressive flow for density estimation. Advances in neural information processing systems, 30, 2017.
Jie Ren, Peter J Liu, Emily Fertig, Jasper Snoek, Ryan Poplin, Mark Depristo, Joshua Dillon, and Balaji Lakshminarayanan. Likelihood ratios for out-of-distributioo detection. Advances in neural information processing systems, 32, 2019.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows Interna- fional conference machine learning, pp. 1530-1538. PMLR, 2015.
Mehdi SM Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly, Assessing generative models via precision and recall. Advances in neural information processing systems, 31, 2018.
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford. Xi Chen, and Xi Chen. Improved techniques for training gans. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett (eds.). Advances in Neural Information Processing Systems, volume 29, Cur- ran Associates, Inc., 2016. URL htttpp:://prroceednngg nwwrips...... cttpppppe.cppapp///16..... File/Ba3363abe75 00000/22222222700000003066600--0
Tim Salimans, Andrej Karpathy, Xi Chen, and Diederik P Kingma. Pixelcnn++: Improving the pixelenn with discretized logistic mixture likelihood and other modifications. arXiv preprint arXiv:1701.05517, 2017.
Hidetoshi Shimodaira. Improving predictive inference under covariate shift by weighting the log- likelihood function. Journal of statistical planning and inference, 90(2):227-244, 2000.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
Yang Song, Taesup Kim, Sebastian Nowozin, Stefano Ermon, and Nate Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. arXiv preprint arXiv: 10766, 2017.
Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. Uncertainty in Artificial Intelligence, pp. 574-584. PMLR, 2020a.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13455 2020b.
Yang Song. Conor Durkan, lain Murray, and Stefano Ermon. Maximum likelihood training of score- based diffusion models. Advances in Neural Information Processing Systems, 34:1415-1428, 2021.
Lucas Theis, Aäron van den Oord, and Matthias Bethge. A note on the evaluation of generative models. arXiv preprint arXiv:/511.0184/ 2015.
Aad W Van der Vaart. Asymptotic statistics, volume Cambridge university press, 2000.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural informa- fion processing systems, 30, 2017.
12

---

Under review as a conference paper at ICLR 2023
Zhou Wang, Hamid R Sheikh, and Alan Bovik. No-reference perceptual quality assessment of jpeg compressed images. In Proceedings. International conference on image processing, volume pp. I-I. IEEE, 2002.
Daniel LK Yamins, Ha Hong, Charles F Cadieu, Ethan A Solomon, Darren Seibert, and James DiCarlo. Performance-optmmized hierarchical models predict neural responses in higher visual cortex. Proceedings of the national academy of sciences, 111(23):8619 8624, 2014.
13

---

Under review as a conference paper at ICLR 2023
A APPENDIX
A.1 PROOFS
Lemma 3.1 (Formal) Let Qloonon and pa be continuous and bounded distributions supported on R. We assume that Quuman is human-like. Therefore, if there exists a perturbation such that samples drawn from Pdota and Px (as defined in 2) are visually indistinguishable to the human eye, then Exupe [log Qhuman (x) Ex-paasa flog Huuman (x)). Further assume that Ex-pp [log pa (x)
Ex-pecto [logpo(x)) pa (x)]. Then po cannot be equal to Quuman almost surely.
Proof. We show this by contradiction. Suppose that Qhuman Po almost everywhere. Then for any distribution V,
Ex [po( Exwolghamon (x)].
Let Pdata and Pr be two distributions, where by assumption
Exxepaee Ihuman (x)] Expux Iahaman (x)],
while
Ex-pasto [x)( o Fxxopo (po (x)).
Thus we have
Exoeaora (x)] = Exppaina Ihammn (x)] = Exape Iquman (x)] Expp, [x)(,)),
(10)
but assumed that Excepeaa [po(x)] # Expex (po (x)), which is contradiction. Thus we have the desired result.
D
Lemma 3.2 Let q be a distribution on d-dimensional k-bit data x {o...., 24.d. Denote x; {0, 1 as the bit mask containing the ith bit of x. When 2. we have
Exapa [a(x)] = Exxpaoi (u , xx-1))
(11)
where q(x1..... xk-1) = Sxy. q(x)dxx is the marginal of g(x) (after marginalizing out xx).
Proof. To see this, we first note that any p(x) can always be decomposed as p(x1, , Thus, letting Pa = B2zp In (x(x) (as defined in 5).
Exap, [a(x)]
E Exngr 4m (x|x)
Ex [x(x)
(12)
E Ex-paacee
Σ q( X2.... ......,,--. ,1 =i)
(13)
Excpaacc [q(x),..,,xx x)-1
(14)
desired.
A.2 ADDITIONAL EXPERIMENTAL DETAILS
Our proposed NCPN architecture consists of the PixeICNN+ backbone Salimans al. (2017) with axial attention layers Ho et al. (2019b) after each residual block. We retain the hyperparameters of PixelCNN++, changing only the dropout on the CIFAR-10 dataset (from 0.5 0.25), which we reduced due to the regularization properties of NCML. For the axial attention layers, we use 8 heads and skip connection rescaling as in Song et al. (2200)). Finally, we add noise conditioning to each residual block via Gaussian Fourier Projection layer, much like Ho et al. 2200 Song et al. (0220b).
For our NCML-trained models, the diffusion times of the VE, VP, and sub-VP SDEs were chosen to be T = 0.3, T = 0.1, and T = 0.025, respectively. The values are selected such that the standard deviation of the per-pix ver-pixel differences between samples in Paato and their noised counterparts in pr was 10 bits. We suspect that further improvements can be made to the empirical results if these numbers chosen more judiciously.
14

---

Under review as a conference paper at ICLR 2023
Model
NLL
NLL
NLL
0*
0.5
1
NCPN (ML)
2.25
3.72
4.35
NCPN (NCML VE)
2.22
3.63
4.21
NPCN (NCML sub-VP)
2.31
3,44
3.98
NCPN (NCML VP)
2.48
3.14
3.67
Table 3: Results on CelebA 64x64. Negative log-likelihood (NLL) is in bits per dimension. Lower better. *NLL with is equivalent NLLL of the original data.
Data/Model
MLE
NCML-VE
NCML-subVP
NCML-VP
SVHN f
0.35
0.43
0.65
0.74
Const Uniform f
0.1
0.56
0.59
0.68
Uniform f
1.0
1.0
1.0
1.0
Average
0.48
0.66
0.73
0.80
CIFAR-10 BPD (æ = 1
3.99
3.94
3.93
3.89
Table 4: AUROC scores of our NCPN models trained on CIFAR-10 on the OOD detection task, with either MIE or different noise schedules.
All NCPN models were trained on RTX 2080 GPUs for 500,000 iterations. This is approximately 1.5 weeks of training. We use the same NCPN architecture and hyperparameters across all datasets (except for dropout, which is set to 0.25 on CIFAR-10 and 0.00 on ImageNet 64x64 and CelebA 64x64). All NCPN models have 73M parameters.
A.2.1 DENSITY ESTIMATION AND GENERATIVE MODELING EXPERIMENTS
For experiments CIFAR-10 and ImageNet 64x64, compare against Kingma al. (2021
Song et al. (202T); Child 2020); Ho et al. 2019a); Greié et al. 2021); Salimans et al. 2011 7 Chen
al. (2018); Child (2019) Some results could not be included due to the irreproducibility of modeling on CelebA 64x64, so do not provide comparisons, however the performance of model summarized in Table
the techniques. There is limited existing work on likelihood-batt
A.2.2 OUT--OF-DISTRIBUTTOO DETECTION EXPERIMENTS
We directly the CIFAR-10 models trained in A.2. and thus retain all hyperparameters from the previous experiment. All NCPN are evaluated with the time condition / = 0. Judicious choice of the statistic is important for the performance of the model. For example, Du & Mordatch| (2019) use the unnormalized energy function log( Z pxx)) where Z = exp f(x) is the partition function, and V log p. where p(x); denotes the ith coordinate of the score. We use Vpp(z) 2 p(x) * V log p(x)) l2. Additionally, we show that performance on the OOD de- tection task is closely associated with robustness to minimally perturbed data (Table in the sense that improved robustness to P- is associated with improved performance on OOD detection. One possible explanation for this correlation is that robustness visually inconsequential high frequency content forces the model to rely more intrinsic features of the image to assign probabilities, which allows the model to better classify OOD images.
Meng et al. 2020)
A.3 ADDITIONAL FIGURES
15

---

Under review as a conference paper at ICLR 2023
Figure 6: Samples from NCPN trained ImageNet 64x64, with Pt variance preserving (VP) diffusion process.
16

---

Under review as a conference paper at ICLR 2023
Figure 7: Samples from NCPN trained on CelebA 64x64, with pt variance preserving (VP) diffusion process.
17

---

Under review as a conference paper at ICLR 2023
Figure 8: Samples from NCPN trained on CIFAR-10, with Pt variance preserving (VP) diffusion process.
18

---

Under review as a conference paper at ICLR 2023
Figure 9: Samples from NCPN trained on CIFAR- 10, with Pt as sub-variance preserving (sub- VP) diffusion process.
19

---

Under review as a conference paper at ICLR 2023
Figure 10: Samples from NCPN trained on CIFAR- 10, with pt variance exploding (VE) diffu-
sion process.
20


Reviews:
[{"review_id": "izedl1q1S8p", "reviewer": "Reviewer_Vbpt", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes to train noise-conditional autoregressive models across a continuum of noise levels, analogously to diffusion models. The authors argue that such models improve density estimation performance under a least-significant-bit perturbation scheme, and also improve sample generation. ", "summary_of_the_review": "I feel the characterization of various generative models is lacking throughout the paper, and the contribution is poorly motivated and limited in experimental evaluation. Overall, I don't think the paper does enough to recommend acceptance. ", "strength_and_weaknesses": "# Strengths\nThe paper demonstrates that training an autoregressive model as the marginal t = 0 of a noise process improves sample generation at no extra cost. \n\n# Weaknesses\n\n**Is likelihood really important?**\nIt is arguable the degree to which we should care about the likelihood (and ability to evaluate the likelihood) in and of itself, despite the paper's arguments. For example:\n- 2.1: 'therefore do not have the same asymptotic guarantees as MLE (e.g., efficiency, functional equivariance, sufficiency).' Score-matching also has asymptotic guarantees. Moreover, how much do we really believe in the guarantees of MLE in practice? For example, the parameters of any neural-network-based model will not be identifiable, meaning most of the guarantees of MLE don't apply.   \n- 2.1: 'Moreover, likelihoods have no closed-form expression, thus requiring approximate ODE/SDE solvers and hundreds to thousands of function evaluations to evaluate.' \nIf we don't need likelihoods to train, why does it matter that they're expensive to evaluate? There is no key functionality presented in the paper that relies on efficient likelihood estimation. \n\nUsually likelihood evaluation, be it exact, approximate, or through a bound, is used as a training objective for generative models. If we don't need it to train, why do we care about it? I feel this isn't well-motivated.   \n\n**Questionable claims about generative models**\nThe paper states more than once that diffusion models cannot be trained by maximum likelihood. \nIntro: 'diffusion models are poor likelihood models, as they cannot be trained via maximum likelihood.'\n2.1: 'However, they cannot be trained by maximum likelihood'\nThis is incorrect. Diffusion models formulated either as latent-variable models or as score-based models can both be fit by maximum likelihood. There is a footnote at the end of page one to say 'at best they can optimize a lower bound of the likelihood', but the main statements claiming (i) diffusion models are poor likelihood models and (ii) diffusion models cannot be trained by maximum likelihood are incorrect. Not only can score-based models be trained by maximum likelihood, but they are also effectively continuous-time normalizing flows, but can be trained more efficiently. \n\n2.1: 'Likelihood models draw samples x ∼ pθ one of two ways.' \nStating that likelihood-based models draw samples either as discrete-time flows or as AR models is reductive. This fails to include continuous-time flows or latent-variable models, and diffusion models can be formulated as either.  \n\n**Experimental results**\nIt is unclear the degree to which the model's robustness to the LSB perturbation scheme is practically useful. Moreover, the proposed score-based sampling scheme is strictly more expensive than standard AR sampling, since sampling requires D passes of the model for the initial noise sample, then taking gradients through a single evaluation of the model for each subsequent score evaluation. As mentioned before, I'm not sure of the usefulness of the likelihood measurements under LSB-perturbation for various models in Table 1, and the single FID value reported for CIFAR-10 along with qualitative samples is unconvincing.  \n\n**Nits**\n- Intro: 'AR models boast state-of-the-art performance in many domains, including images... and audio'. This is certainly not true for images anymore, and is it true for audio? \n\n- The paper repeatedly states the likelihood is a function of the data when it is a function of the parameters of the model. \nAbstract: 'Likelihood of the data under the model'\n2.1: 'the likelihood of each sample'\n3.2: 'the model’s predicted likelihood of downstream tokens'\n\n- 'The goal of likelihood-based generative modeling is to approximate pdata via a parametric model pθ, where samples x ∼ pθ can be easily obtained.' This is the goal of generative modeling, not likelihood-based generative modeling. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "S58Ul1CbvS5", "reviewer": "Reviewer_4nNY", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper investigates a likelihood model for a family of increasingly noisy distributions, with the true data distribution at one end, and noise at the other. This construction is conceptually similar to the popular class of diffusion models, except the authors decided to learn a log density model of the data (and its noisy versions), instead of just learning its derivative (cf. the score-based interpretation of diffusion models). The authors motivate learning density for the whole family of noisy distributions—instead of just for the data—by showing that modern pixel space autoregressive image models are rather brittle to bit flip perturbations that are barely perceptible by human eye. Due to equivalence to diffusion processes, the authors propose a sampling process based on integrating the SDE in reverse as known from the diffusion literature. Their empirical results of their models improve on other explicit log likelihood models in terms test log likelihood, albeit still lack significantly behind the FID scores of SOTA diffusion models.", "summary_of_the_review": "While the empirical results are somewhat lacking behind score-based models, they, I found the paper an insightful (especially the part on brittleness of modern autoregressive likelihood models) and enjoyable read. ", "strength_and_weaknesses": "### strengths\n* interesting observations regarding brittleness of modern pixel-space autoregressive image models\n* proposed model does seem to provide improvements over other models in the same class\n\n### weaknesses \n* the proposed model still lacks significantly behind SOTA diffusion models\n* somewhat weak motivation for why we would want to have an explicit likelihood model", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "CyE7q8Zu-T", "reviewer": "Reviewer_YYAi", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper presents a novel way to train autoregressive models and to sample from them.\nThe authors argue that the standard maximum likelihood approach is overly sensitive to tiny perturbations, assigning low likelihoods to images that have been modified in a way that is Imperceivable to humans.\nAdditionally, they argue sampling results from autoregressive models are inferior due to these sensitivities.\n\nThe proposed method works by training a model to not only the distribution of the training data, but also of noisy versions of the data.\nThe model is conditioned on the noise level.\nThe authors find that their model increases sample quality and is more rout to small perturbations.\n", "summary_of_the_review": "All-in-all, this is an interesting paper with some smaller flaws, that could be fixed in the final version.\nMost importantly, the details on training and architecture have to be provided.\nFinally, I have to admit that I am not an expert on score matching and that somebody with more knowledge of the topic might have a more informed opinion.\n\nI changed my rating regarding the novelty in light of the discussion with other reviewers and the AC.", "strength_and_weaknesses": "Strengths:\n\n* The paper is well written. The problem is excellently introduced and motivated.\n* The idea is interesting and original as far as I know.\n\nWeaknesses:\n\n* As far as I understand, the argument about autoregressive models being overly sensitive to small changes is based on the assumption, that they use discrete distributions (e.g. for 8bit images) over pixel values, right? I wonder if this would hold for models that use parametric continuous distributions for each pixel.\n* I am missing some details in the paper on how exactly the PixelCNN++ architecture was modified.\n* How exactly is the model conditioned on the noise level? I think this could make quite a difference in how much more robust the model will become by forcing it to also describe the noisy data.\n\n\nMinor points:\n\n* The authors write: \"p_π describes the distribution of points in pdata that have had their least\nsignificant bit incremented or decremented with probability π\"\nThis formulation could be more clear. What happens when a 0 valued bit is decremented?\nIs this simply about adding or subtracting one form the pixel value?\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "74VmBJoVeb", "reviewer": "Reviewer_4m9N", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work proposes a noise conditional likelihood-based method for training autoregressive models. Specifically, instead of directly training the model on clean images, the authors propose to train a sequence of autoregressive models on images perturbed with different noise levels using MLE. The authors show with empirical results that the proposed approach is able to improve the sample quality of autoregressive models while also achieving reasonably good likelihoods (in BPD).", "summary_of_the_review": "Although this work considers a very interesting application of autoregressive models by incorporating noise-conditioning schemes from diffusion/score-based models, the technical novelty is limited and the paper presentation will also need to be improved.", "strength_and_weaknesses": "Strength:\n1. The paper considers a very interesting direction for autoregressive models.\n\n2. This work has a decent amount of experiments and the performance of the proposed method is reasonable.\n\nWeaknesses:\n1. Limited novelty: training generative models on images perturbed with multiple noise levels is not a new idea. Similar ideas have been explored for diffusion models [1, 2], normalizing flows [3], and also autoregressive models [4, 5]. Specifically, the main idea is very similar to [4] except that the authors now use autoregressive models to parameterize a normalized density (which can be trained via maximum likelihood estimation), while [4] uses autoregressive models to parameterize \"scores\" and trained the models with score matching on images perturbed with various noise levels. The main idea is also similar to [5] (which is not discussed), except that in [5], only two noise levels are used. The conclusion that training autoregressive models on noisy images improves the sample quality and model robustness was also mentioned in [5]. Given these related works, the novelty of this work is questionable.\n\n2. Some claims are flawed. For instance, in the Introduction, the claim that \"diffusion models are poor likelihood models and cannot be trained via maximum likelihood\" might not be correct. Diffusion models can be understood as hierarchical VAEs and can be trained by optimizing ELBO (which also optimizes likelihoods). Follow-up works [6, 7] also show that diffusion/score-based models can be good likelihood models.\n\n3. Some related works are not discussed or compared. For instance, [5] is a closely related paper and should be discussed and compared. [8, 9] also consider conditioning on different data augmentations which share similarities with the proposed method. It would be good to also compare with these approaches.\n\n4. The equations/notations are not defined properly in section 3.\n\n5. Section 3.2 is not convincing. Only intuitions are provided without rigorous arguments or proof. It is unclear whether the arguments are correct or not.\n\n6. It remains unclear why the noise-robustness experiments (section 4.1) are useful. The proposed sanity test for checking the robustness of likelihood models is not convincing or mathematically-justified It is unclear whether a better score implies better robustness. It would be more convincing to compare on some standard benchmarks for robustness.\n\n7. The empirical performance is not very impressive. For instance,  [6, 7] are able to achieve much better performance.\n\n[1] Denoising Diffusion Probabilistic Models: https://arxiv.org/abs/2006.11239\n\n[2] Score-Based Generative Modeling through Stochastic Differential Equations: https://arxiv.org/abs/2011.13456\n\n[3] Denoising Normalizing Flow: https://proceedings.neurips.cc/paper/2021/file/4c07fe24771249c343e70c32289c1192-Paper.pdf\n\n[4] Improved Autoregressive Modeling with Distribution Smoothing: https://arxiv.org/abs/2103.15089\n\n[5] Autoregressive Score Matching: https://arxiv.org/abs/2010.12810\n\n[6] Variational Diffusion Models: https://arxiv.org/abs/2107.00630\n\n[7] Maximum Likelihood Training of Score-Based Diffusion Models: https://arxiv.org/abs/2101.09258\n\n[8] Distribution Augmentation for Generative Modeling: http://proceedings.mlr.press/v119/jun20a/jun20a.pdf\n\n[9] Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling: https://arxiv.org/abs/1812.01608", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject