Locally Hierarchical Auto-Regressive Modeling for Image Generation
Anonymous Author(s)
AfﬁliationAddress email
Abstract
Auto-Regressive (AR) modeling on image generation provides a scalable solutionto synthesize high-quality examples by directly optimizing the data likelihood,factorized into a product of conditionals in pre-deﬁned generation order.
However,it is computationally expensive to train AR models on the pixels of high-resolutionimages, and many practical approaches decompose a learning process into thefollowing two stages: (a) representing a high-resolution image by a short discretesequence, and (b) learning an AR model on this sequence, rather than on pixels.
This paper points out that every code in the sequence has a receptive ﬁeld of thesame size and most existing works in this framework fail to exploit multi-scaleinformation of images.
To mitigate this limitation, we represent an image by a set ofdiscrete sequences of multiple resolutions and generate images using an AR modelbased on variational autoencoder, which is referred to as Hierarchical-Quantized
Variational AutoEncoder (HQ-VAE).
We propose a Hierarchical-Quantized Transformers (HQ-Transformer) to model the multi-level discrete sequences efﬁcientlyand generate novel images of good quality.
Experiments on class-conditional andtext-conditional generation tasks verify that our approach outperforms previous
AR models, in terms of ﬁdelity of generation samples under a similar computationbudget.
We also demonstrate structure-condition image generation to show theusage of disentangled top and bottom codes.
Extensive ablation studies explainrationales for our architectural design.
Introduction
Deep generative models [11, 15, 35] have become the fundamental basis for synthesizing diverseimages of high-resolution from various forms of input supervision, including class conditions [21, 35],text prompts [26, 27] and segmentation maps [9] to name a few.
Most approaches in this paradigmcould be roughly categorized into two types depending on whether they explicitly maximize thedata likelihood or not.
GANs implicitly estimate the underlying data distribution to generate noveland crispy samples, but it typically suffers from the lack of diversity [11].
In contrast, it is widelyaccepted that likelihood-based models are capable of generating diverse samples, compared to GANs,since they aim to cover all modes of data distribution [28].
In this paper, we focus on auto-regressivemodels among many likelihood-based approaches since the scalability of auto-regressive models isvalidated in several works [25, 27].
Auto-Regressive (AR) models factorize the data likelihood into a product of conditionals in predeﬁned generation order.
Besides the mode-coverage property of AR models, they require onlyminimal changes to architecture and optimization for being applied to various data modalities, suchas audio, natural language, and images [24, 34, 35].
However, regardless of the modality, training
AR models on high-dimensional samples is a very computationally demanding task, since each
[Figure]
conditional needs to process many past observations.
In the context of image generation, VQVAE [36] reduces the complexity of AR models by compressing a continuous RGB signal of highresolution into a discrete sequence of low resolution.
In speciﬁc, VQ-VAE introduces a two-stagelearning process, where stage 1 represents an image by a set of discrete codes, and stage 2 constructs
AR priors [4, 35] on discrete sequences.
To make this two-stage framework effective, the codes needto have sufﬁcient information to reconstruct the original image from the discrete sequence, while thesequence length should be reasonably short to train AR models.
In this study, we postulate that leveraging the multi-scale structure of visual images is an efﬁcient wayto represent an image as discrete codes for AR modeling.
Many approaches to reducing the sequencelength have been proposed to improve the stage 1 architecture [17, 38] or training objectives [9, 18].
These methods, however, adopt the codes with the same receptive ﬁeld size.
AR models in the secondstage, therefore, need to process each code in the sequence equally important.
Suppose that we havemulti-level codes obtained from stage 1, where the high-level code contains the overall structure andthe low-level one encodes the detailed information.
It now becomes possible to treat such codesdifferently, so an efﬁcient solution could be presented in stage 2.
This is because given the codes oflow resolution, a light AR model may be enough to generate the codes of high resolution.
In order to achieve this goal, we propose locally hierarchical auto-regressive modeling extendingtwo-stage AR modeling.
We ﬁrst extend VQ-VAE to produce a set of codes of two levels: the topand bottom codes, where the top contains the high-level information and the bottom encode thelow-level details.
Using these hierarchical discrete representations, an efﬁcient AR model could bedeveloped to generate a novel sample in a locally hierarchical manner.
Speciﬁcally, the top codes aresequentially generated in a pre-deﬁned order, and several bottom codes associated with a top code aredecoded in parallel.
The main contributions of the proposed model are summarized below:
• We propose HQ-VAE (Hierarchical-Quantized Variational Auto-Encoder) to disentangle the
information encoded in the top and bottom codes, without introducing additional losses andlearnable components for disentanglement.•
We propose HQ-Transformer (Hierarchical-Quantized Transformer) that fully exploits thespatial relationship between the top and bottom codes.
Our network is built upon anassumption that there exist several bottom codes conditionally independent given a top code.•
Our model achieves competitive performance on class- and text-conditional image generation, while observing high throughput in sample generation, compared to various types ofgenerative models.
Related Works
In this section, we highlight the relevance and difference between previous approaches and ours.
E n c o d e r
STAGE 1 (HQ-VAE)
Downsample
Quantizer
Upsample
QuantizerD
e c o d e r e top e bot
Image (256x256)
Reconstruction
STAGE 2 (HQ-Transformer) sos
Main Transformer uuu n h 2 h 1 hh n
Input Embedding Transformer t n b (n,1) b (n,2) b (n,3) b (n,4)
Prediction Head Transformer t n+1 t n+1 t n+1 t n+ b (n+1,1) b (n+1,2) b (n+1,3) b (n+1,4) t n+
[Figure]
Causal
[Figure]
Blocks
Bidirectional Attention Blocks
Mixed Attention Blocks t 1 tt 3 tb (1,1) b (1,2) b (1,3) b (1,4)
b (2,1) b (2,2) b (2,3) b (2,4)
[Figure]
(3,1) b (3,2) b (3,3) b (3,4)
[Figure]
[Figure] [Figure]
Discrete representation for autoregressive models Learning AR priors on discrete representationshave been proven to be effective in language modeling [24, 25].
In the case of continuous sampleslike images, we need to encode sufﬁcient information of examples in the codes, while the sequencelength should be short to train AR models.
In order to do that, VQ-GAN [9] shortens the codes byusing the perceptual and adversarial losses to encode the semantic information in the codes as muchas possible.
RQ-VAE [18] further reduces the sequence length by converting an image into a stackof codes.
This is constructed by iteratively quantizing the residual vector between the latent featureand code embedding, effectively reducing the latent feature map resolution without degrading thereconstruction quality.
However, we observe that these approaches do not consider the multi-scalenature of an image, meaning that the receptive ﬁeld of every code has the same size.
Hierarchical representation for autoregressive models To resolve the limitation of these singlescale discrete representation learning methods, many approaches to constructing the codes of multipleresolutions [10, 22, 28] have been proposed.
VQ-VAE 2 [28] and HAM [10] make use of thehierarchical relationship between the codes to generate a novel image in a coarse-to-ﬁne manner,but these require specialized modules to disentangle the information in the code levels.
Hierarchical
VQ-VAE [22] simply encodes the structural and textual information in different types of codes bytraining two separate VQ-VAEs, where the ﬁrst is trained on the original images and the second on theresidual between the original and reconstructed image.
This process can disentangle the informationin the code levels, but how to train AR models based on this set of codes is not studied.
Diffusion models Diffusion models [12, 15, 29] iteratively denoise a sample from the white noiseby seeing the contexts in a bidirectional manner.
Most diffusion models have been implemented ina
U-Net [30] style architecture, meaning that the multi-scale feature of an image is well exploited.
However, they suffer from low throughput in the generation, since the intermediate computationresults between the denoising steps are not reusable, unlike AR models.
Locally Hierarchical Auto-Regressive Models
This section describes the details of the proposed architecture for image generation.
Our approachexploits two levels of codes—top and bottom codes—for the representation of an image contrary toexisting two-stage approaches based on AR models.
The proposed architecture encourages the topcode to model the overall structure of an image while facilitating the bottom code to complement theremaining details of the image.
To this end, we propose Hierarchical-Quantized Variational AutoEncoder (HQ-VAE) for the image vector quantization in the ﬁrst stage followed by HQ-Transformer to learn the prior over a sequence of the top and bottom codes in the second stage.
Note that HQTransformer achieves better perceptual quality and sampling speed using the underlying hierarchicalstructure.
Figure 2 illustrates the two stage procedure of our AR model.
3.1 Stage 1: Hierarchical-Quantized VAE (HQ-VAE)
We present the architecture and the optimization method of the proposed HQ-VAE, for which we ﬁrstreview the vector quantization technique and then discuss the hierarchical quantization module.
3.1.1 Notations for Vector Quantization
Vector quantization [20] maps a d -dimensional continuous vector into a code s .
The mappingselects the nearest code embedding in a codebook, a collection of tuples { ( s, e ( s )) } s ∈ n C , wheree ( s ) ∈ R d e denotes the embedding of code s and n C is the size of codebook C .
Supposing that z ij isa d -dimensional vector from the feature map z ∈ R l × l × d at the spatial location ( i, j ) , the code s ijand its embedding are given by
[Equation]
where C ( · ) returns the code embedding of an input code and s ij =
Q ( z ij ; C ) denotes the quantizationprocess introduced for later use.
Note that, since every element in a feature map has its own code, thespatial resolution of the code map, denoted by s , is the same as that of the feature map.
3.1.2 Hierarchical Vector Quantization
Hierarchical vector quantization H ( · ; r ) transforms a feature map z ∈ R rl × rl × d into a tuple of twocode maps ( t , b ) , where t and b have the shapes of l × l and rl × rl , respectively, with a scaling factorr ≥ 1 .
The objective of this bottleneck process is to encode different levels of information using thetop and bottom codes.
Intuitively, it is desirable for the top codes to hold high-level information of aninput image while low-level details are encoded by the bottom codes.
To accomplish this objective,our hierarchical quantization adopts a two-stream network, with no additional learnable components.
In speciﬁc, given a scaling factor r and a feature map z , the following process extracts the top codeand its code embedding:
[Equation]
where C top denotes the codebook of top codes and Downsample ( · ) is the average pooling function toreduce the spatial resolution of a feature map.
Based on t , the bottom code and its embedding arederived by
[Equation]
where C bot is the codebook of bottom codes and Upsample ( · ) is implemented as an interpolationoperation with the ‘nearest’ option.
Figure 2(left) illustrates this hierarchical quantization process.
Our scheme ﬁrst captures the high-level information of a feature map by quantizing its downsampledversion based on the top codebook.
Hence, the residual vector between the original feature map andthe upsampled top code embedding map only contains high-frequency components in the featuremap.
Figure 3 compares the reconstructed images from each of the top and bottom code embeddings,demonstrating the disentangling performance by the proposed hierarchical vector quantization.
Despite the reasonable performance given by the resampling idea, we observe that the overallstructure reconstructed by the top code embeddings is not strong, as shown in Figure 3a.
Topush more information into the top codes, we have to reduce the information loss caused by thedownsampling operation as much as possible.
A simple yet effective solution to this challenge isto employ pixel-unshufﬂe and -shufﬂe operations [33].
Given the feature map z ∈ R rl × rl × d anda scaling factor r , the pixel-unshufﬂe operation returns a tensor with the shape l × l × r 2 d andmaintains all information in the original feature map.
To compute the residual, we shufﬂe the topcode embedding and restore the original dimensionality of z .
Figure 3 clearly demonstrates that thetop codes constructed by the pixel-shufﬂe contain the primary information of an input image morethan the ones given by pooling operations.
o r
i g i na l
Avg.
Pool
Quantizer
Nearest Interp.
Quantizer
Encoder
Decoder 16 x 16 x K 8 x 8 x K 16 x 16 x K - (a) Average pooling and nearest interpolation o r i g i na l
PixelUnShuffle
Quantizer
PixelShuffle
Quantizer
Encoder
Decoder16 x 16 x K 8 x 8 x 4K 16 x 16 x K (b) Pixel-unshufﬂe and pixel-shufﬂe
3.1.3 HQ-VAE: Architecture & Optimization
Overall structure HQ-VAE consists of three parts: encoder E ( · ; f ) , decoder D ( · ) , and hierarchicalquantizer H ( · ; r ) , as shown in Figure 2, where f is the initial downsampling factor and r is theadditional scaling factor for the top code.
Given an image x ∈ R h × w × 3 , an encoder E ( · ) extractsthe feature map z ∈ R hf × wf × d .
Then, the hierarchical quantizer H ( · ; r ) is applied to the feature map,providing the top and bottom codes ( t , b ) and their associated code embeddings: e top and e bot .
Now,the decoder D ( · ) takes the top and bottom code embeddings to reconstruct the input image.
Objective function To train three parts in HQ-VAE, our objective function is composed of thereconstruction loss introduced in VQ-VAE [36] as well as adversarial and perceptual losses to improvethe ﬁdelity of reconstructed images [9].
Here, we only present the reconstruction loss as below:
[Equation] [Equation]
[Equation]
where sg [ · ] refers to a stop-gradient operation, and β is set to 0.25 as suggested in [36].
The ﬁrst termmeasures L 2 distance between the reconstructed image and the original.
The second and third termsenforce the code embedding close to the inputs of quantizers.
Since we have two types of codes, wesimply sum up the second and third terms of each type.
To update the codebook, we make use ofexponential moving average (EMA), because this accelerates the convergence [36].
Implementation details We adopt the same network structures for encoder and decoder as in
VQ-GAN [9] with two minor modiﬁcations: (a) to increase throughput in training, we set the strideof the ﬁrst convolution layer in the encoder and the last transposed convolution layer in the decoderas two, instead of one in the original implementation.
(b) we replace the batch normalization [16] in adiscriminator by group normalization [37] to achieve the consistent results regardless of a batch size.
3.2 Stage 2: Hierarchical-Quantized Transformer (HQ-Transformer)
In this section, we propose HQ-Transformer (see Figure 2) to autoregressively model the top andbottom codes by fully exploiting their spatial relationships obtained from the ﬁrst stage.
3.2.1 Formulation
Our objective is to maximize the log-likelihood of a tuple of t and b .
Given the scaling factor rfor the top, the resampling operation aligns a single top code t i with r 2 bottom codes b ( i,k ) , where1 ≤ k ≤ r 2 for every i .
We annonate all bottom codes aligned with a single top code t i as b i .
Giventhat the top code map shape is l × l and the sequence order of the top is a typical raster scan order,the likelihood function can be factorized below:
[Equation]
where the notation <
i denotes a collection of j when j < i .
The conditional can be further factorizedinto:
[Equation]
where the last term can be maximized by predicting the current position given the previous top andbottom codes.
For the ﬁrst term, we observe that a reasonable assumption on the bottom codes leadsto more efﬁcient formulation.
Here, we assume that the bottom codes are conditionally independent given the associated top code,since bottom codes contain only detailed local information if the top code has global-level information,which is enforced by our stage 1 model, HQ-VAE.
Therefore, we further expand the ﬁrst term in
Eq. 6 as
[Equation]
3.2.2 Architectural Details
Similar to GPT-like architectures [3, 25], HQ-Transformer consists of three sub-modules—inputembedding, main transformer, and prediction head.
To effectively aggregate the information fromthe top and bottom codes for the main transformer and making a prediction, we design the inputembedding and prediction head by a Transformer instead of linear projection.
Input Embedding Transformer (IET) IET transforms a tuple of ( t i , b i ) into a single embeddingvector u i , for all i .
In order to do that, we ﬁrst construct a local embedding by summing up the codeembedding and positional encoding:
[Equation]
where e top,in i and e bot,in ( i,k ) mean the corresponding learnable code embedding of t
i and b ( i,k ) and PE top [ · ]and PE bot [ · ] denote learnable positional encodings for the top and bottom.
Note that PE bot [0] isintroduced to distinguish between the local embeddings from the top and bottom codes.
Now, theselocal embeddings are reﬁned by a bidirectional transformer:
[Equation]
where the structure is the same with GPT-2 [25].
And. we observe that it is sufﬁcient to employ asingle GPT-2 block (composed of a self-attention and MLP block).
Then, the ﬁnal input embeddingu i for the main Transformer block is simply averaging all updated local embeddings:
[Equation]
Main Transformer
This module updates the outputs of IET by a stack of N main casual self-attentionand MLP blocks:
[Equation]
where the sequence length is the number of top codes and the start-of-sentence token (SOS) is locatedat the ﬁrst of input embedding.
Here, we use the same structure of GPT-2.
Prediction Head Transformer (PHT)
This module predicts the top and bottom codes from thehidden representation of the main transformer, where the bottom codes at the next position aregenerated in parallel based on their associated top code and a hidden representation from the main
Transformer.
In speciﬁc, we ﬁrst deﬁne the input sequence of PHT as below:
[Equation]
where e top,head i means the corresponding learnable code embedding of t i .
This sequence is thenupdated to produce the conditional probabilities of top and bottom codes:
[Equation]
where each probability is deﬁned as
[Equation]
Here, we use the self-attention mask pattern as a mixture of causal and bidirectional mask—thecausal attention from the top to bottom codes and the bidirectional attention between the bottoms.
Compared to full-autoregressively modeling on the bottoms, this parallel decoding achieves fastersampling and even better generation quality.
We will discuss the effect of this parallel decoding inthe experiments section.
3.2.3 Computational complexity of HQ-Transformer
Suppose that we have the top code t ∈
Z l × l and the bottom code b
∈
Z rl × rl , where r is ascaling factor for the top code quantizer.
We denote L top =
l 2 as the length of the top codes and
L local = 1 + r 2 as the number a single top code and all bottom codes associated with the topcode.
The time complexities of input embedding, main, and prediction head transformer are then
O ( N in L top L 2 local ) , O ( N main L 2 top ) and O ( N head L top L 2 local ) , where N in , N main , and N head are the numberof attention blocks.
And, O ( N main L 2 top ) dominates the other terms, since N main is much larger than N inand N head .
Suppose that the code map resolution of VQ-GAN is equal to the bottom code resolutionof HQ-Transformer.
Computational complexity of HQ-Transformer is mainly O ( N main L 2 top ) , whichis much lower than the complexity of VQ-GAN O ( N main L 2 top r 2 ) .
Experiments
In the experiment section, we evaluate our models on class- and text-conditional image generationtasks.
We observe that HQ-Transformer achieves competitive performance to other types of generationmodels, while allowing high throughput in sample generation.
We also present various ablationstudies on our architecture to support the rationale of our choice.
4.1 Conditional Image Generation
Settings We train our models on 1.2M samples from ImageNet [6] trainset for a class-conditionalimage generation task.
In case of a text-conditional task, we use 15M image-text pairs constructed by
Conceptual Caption (CC) [32] and Conceptual-12M [2].
In training phase of both cases, we resizethe short axis of an image to 256 and randomly crop the image to acquire an image of 256 × 256resolution.
In testing, we also resize the short axis to 256 and crop the image at the center.
We setup HQ-VAE by f = 8 and r = 2 to produce the top and bottom codes of 8 × 8 and 16 × 16resolution, respectively.
We instantiate three types of HQ-Transformer: (a) L main = 12 , and L head =
4for the smallest model,
(b) L main = 24 and L head = 4 for the middle-size model, and (c) L main = 42 ,and L head = 6 for our largest model.
In all cases, L in = 1 and other parameters related to the networksize are the same.
We describe more details on the network architecture in the supplementary material.
For evaluation measure, we use Fréchet Inception Distance (FID) [14], precision & recall [31] inclass-conditional generation, and FID and CLIP-score [23] in a text-conditional generation task.
Wemeasure throughput in sample generation on a single Tesla A100 in all cases.
[Table] [Table]
[Figure]
Class-conditional image generation Table 1 shows that our largest model, HQ-Transformer (L),achieves the best FID 7.15 compared to two-stage AR models on the similar sequence length, VQGAN ( 16 × 16 ) and RQ-Transformer ( 8 × 8 × 4 ).
We observe that HQ-Transformers report the highestthroughput in sample generation than that of other models, since ours generate r 2 bottom codes inparallel given the associated top code.
Compared to ADM, our smallest model (S) can generatesamples of similar quality, while achieving 650x larger throughput.
We also conduct rejectionsampling by using ResNet-101 [13] with acceptance rate 0.25 for all our models, and the results aremarked in the parenthesis in Table 1.
Here, our models show competitive performance to the othermethods.
Figure 4 demonstrates top code-conditional image generation to show the disentanglementof top and bottom codes.
Contrary to typical generative models, HQ-Transformer restricts thediversity of generated images within details by sampling only bottom codes from HQ-Transformerwith ﬁxed top code.
Text-conditional image generation Table 2 reports FID and CLIP-score of several models measured on CC3M validation split, and throughput in sample generation.
This shows that our model iscomparable to RQ-Transformer in terms of FID (12.86 vs. 12.33) and CLIP-score (0.26 vs. 0.26),
[Table]
[Table] [Table]
while achieving 18% higher throughput than that of RQ-Transformer.
This result shows that ourmodel behaves similarly regardless of data modality
[Table]
4.2 Detailed Experiments on
HQ
-VAE and
HQ
-Transformer
Comparison with the other stage 1 methods Table 3 compares two types of HQ-VAE, ( f =16 , r = 2) and ( f = 8 , r = 2) , with the existing stage 1 models, where we measure FID between theoriginal and reconstructed image, referred to as rFID.
We also measure the number of codes activatedin reconstruction, and report it in the last column.
rFID from HQ-VAE ( 8 × 8 + 16 × 16 ) is betterthan the ones from baselines with the similar sequence length:
RQ-VAE ( 8 × 8 × 4 ) and RQ-VAE( 8 × 8 × 5 ).
In addition, HQ-VAE ( 16 × 16 + 32 × 32 ) achieves better than that of ViT-VQGAN( 32 × 32 ), while the efﬁcient Transformer can be implemented on the hierarchical code maps, unlike
ViT-VQGAN.
Ablation study on HQ-VAE Table 4 supports the rationale of our choice, showing that resamplingwith pixel-shufﬂe and unshufﬂe outperforms the one with average pooling and nearest interpolationby 0.53 in FID, since our choice prevents from loosing information when downsampling.
We alsoobserve that longer training boosts rFID performance from 3.14 to 2.61.
Ablation study on PHT in HQ-Transformer Table 5 compares the effects on the type of selfattention module and the target label type in Prediction Head Transformer.
In Table 5, the causalattention means that the top and bottom codes are sequentially generated like in typical AR models.
The mixed attention means our procedure; the top code is ﬁrst generated and r 2 bottom codes aredecoded in parallel.
We observe that our parallel generation scheme achieves better FID than afully-autoregressive policy, since error accumulation during sampling is alleviated by our paralleldecoding approach.
We also validate the effects of soft-label as an alternative to one-hot label incross-entropy loss function for training HQ-Transformer as proposed in RQ-Transformer [18].
In ourcase, using soft-labels with temperature τ = 1 improves the performance similarly observed in [18].
Conclusion
We introduce a two-stage auto-regressive model (HQ-VAE and HQ-Transformer) to generate anovel sample in a locally hierarchical manner extending typical two-stage auto-regressive models.
HQ-VAE represents an image by a top and bottom codes with multiple resolutions, encoding theoverall high-level structure of an image in the top codes of low-level and detailed information in thebottom codes of high resolution.
HQ-Transformer models these top and bottom codes efﬁcientlyand generates novel images of high quality.
Experiments conﬁrm that our approach is capable ofgenerating high-quality samples compared to recent architectures, while achieving high throughputin sample generation.
Checklist
The checklist follows the references.
Please read the checklist guidelines carefully for information onhow to answer these questions.
For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .
You are strongly encouraged to include a justiﬁcation to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.
For example:
• Did you include the license to the code and datasets?
[Yes] See Section ?? .
• Did you include the license to the code and datasets?
[No] The code and the data are
proprietary.
• Did you include the license to the code and datasets?
[N/A]
Please do not modify the questions and only use the provided macros for your answers.
Note that the
Checklist section does not count towards the page limit.
In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
For
all authors...
(a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s
contributions and scope?
[Yes]
(b) Did you describe the limitations of your work?
[Yes] We discuss the limitation of our
models on the supplementary material.
(c) Did you discuss any potential negative societal impacts of your work?
[Yes] We discuss
it in the supplementary material
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A]
(b) Did you include complete proofs of all theoretical results?
[N/A]
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experi-
mental results (either in the supplemental material or as a URL)?
[Yes] We report in
both the paper and the supplementary material.
(b) Did you specify all the training details (e
g., data splits, hyperparameters,
how they
were chosen)?
[Yes] We report in both the paper and the supplementary material.
(c) Did you report error bars (e
g., with respect to the random seed after running experi-
ments multiple times)?
[No] The evaluation protocol for conditional generative models
does not report the error bars.
Instead, we ﬁx the random seed and make experiments
reproducible.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] We report in both the paper and the
supplementary material.
If you are using existing assets (e
g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] We cite and report inthe supplementary material.
(b) Did you mention the license of the assets?
[Yes] We report in the supplementary
material.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]
(d) Did you discuss whether and how consent was obtained from people whose data you’reusing/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identiﬁableinformation or offensive content?
[N/A]5.
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, ifapplicable?
[N/A](b)
Did you describe any potential participant risks, with links to Institutional ReviewBoard (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amountspent on participant compensation?
[N/A]
A Appendix
Optionally include extra information (complete proofs, additional experiments and plots) in theappendix.
This section will often be part of the supplemental material.