Original: Procedural Image Programs for Representation Learning
Revised: Procedural Image Programs for Representation Learning

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Learning image representations using synthetic data allows training neural networkswithout some of the concerns associated with real images, such as privacy andbias.
Revised: Learning image representations using synthetic data allows training neural networks without some of the concerns associated with real images, such as privacy and bias.

Original: Existing work focuses on a handful of  generative processes which are hardto integrate together to scale up.
Revised: Existing work focuses on a handful of curated generative processes which require expert knowledge to design, making it hard to scale up.

Original: To overcome this, we propose training with alarge dataset of twenty-one thousand programs, each one generating a diverse setof synthetic images.
Revised: To overcome this, we propose training with a large dataset of twenty-one thousand programs, each one generating a diverse set of synthetic images.

Original: These programs are short code snippets, which are easy tomodify and fast to execute using OpenGL.
Revised: These programs are short code snippets, which are easy to modify and fast to execute using OpenGL.

Original: The proposed dataset can be used forboth supervised and unsupervised representation learning, and reduces the gapbetween pre-training with real and procedurally generated images by 38%.
Revised: The proposed dataset can be used for both supervised and unsupervised representation learning and reduces the gap between pre-training with real and procedurally generated images by 38%.

Revised: Code, models, and datasets are available at: https: //github.com/mbaradad/shaders21k

Original:  Introduction
Revised:  Introduction

Original: Training neural networks using data from hand-crafted generative models is a novel approach thatallows pre-training without access to real data.
Revised: Training neural networks using data from hand-crafted generative models is a novel approach that allows pre-training without access to real data.

Original: This technique has been shown to be effective ondownstream tasks for different modalities, such as images [1, 2], videos [3], and text [4].
Revised: This technique has been shown to be effective on downstream tasks for different modalities, such as images [1, 2], videos [3], and text [4].

Original: Despite itsbig potential, previous work for images is limited, focusing on a handful of generative processes, likefractals [5], textured polygons [3], or dead-leaves and statistical processes [2].
Revised: Despite its big potential, previous work for images focuses on a handful of generative processes, like fractals [5], textured polygons [3], or dead-leaves and statistical processes [2].

Original:  What would happen ifwe scale this up to 21 thousand generative image  programs?
Revised: To achieve good performance with the original set of procedures, expert knowledge is required to make the generated images match simple image statistics. This makes it hard to improve over existing work, as these generative models have already been carefully studied and tuned. What would happen if we used a large set of procedural image programs without curation, instead of focusing on a small set of programs?

Original: To this end, we collect a large-scale set ofimage programs, each producing a diverse set of imagesthat display both shape and texture (see Figures 1 and 2 for examples).
Revised: To this end, we collect a large-scale set of programs, each producing a diverse set of images that display simple shapes and textures (see Figures 1 and 2 for examples).

Original: We then use these programsto generate images and train neural networks with supervised and unsupervised  methods.
Revised: We then use these programs to generate images and train neural networks with supervised and unsupervised representation learning methods.

Original: Thesenetworks can then be used on different downstream tasks, reaching state-of-the-art performance formethods that do not pre-train using real images.
Revised: These networks can then be used for different downstream tasks, reaching state-of-the-art performance for methods that do not pre-train using real images.

Original: The generative programs of this collection are coded in a common language, the OpenGL shadinglanguage, that encapsulates the image generation process in a few lines of code.
Revised: The generative programs of this collection are coded in a common language, the OpenGL shading language, that encapsulates the image generation process in a few lines of code.

Original: Previous work usesad-hoc slow methods for each generative process, which makes it hard to integrate them into thesame framework.
Revised: Previous work uses ad-hoc slow methods for each generative process, which makes it hard to integrate them into the same framework.

Original: Compared to existing approaches, the rendering of our programs is performed athigh throughput (hundreds of frames per second with a single modern GPU), allowing generation onthe fly while training.
Revised: Compared to existing approaches, the rendering of our programs is performed at high throughput (hundreds of frames per second with a single modern GPU), allowing generation on the fly while training.

Original: Furthermore, the dataset consists of a large collection of generative processes allowing trainingparadigms that were not possible with previous methods. Using each code snippet as a latent class,we can learn representations using supervised approaches.
Revised: As the dataset consists of a large collection of generative processes, we can use each code snippet as a latent class, which allows learning representations using supervised approaches.

Original: In contrast, previous methods requiread-hoc techniques for clustering the generative parameters into discrete classes [5].
Revised: In contrast, previous methods require ad-hoc techniques for clustering the generative parameters into discrete classes [5].

Original: Together with the dataset, the main contributions of the paper are:
Revised: Together with the collection of programs, the main contributions of the paper are

Original:  • A study on how performance scales as a function of the number of generative image programs,  supervised 
Revised: : • A study on how performance scales as a function of the number of generative image programs, for supervised and unsupervised approaches.

Original: Our experiments show  downstream  scales logarithmically with  (see Figure 
Revised: Our experiments show that downstream performance scales logarithmically with the number of generative programs (see Figure 3 and Section 4).

Original: • State-of-the-art performance for representation learning without real data.
Revised: • State-of-the-art performance for representation learning without real data.

Original: When the number of programs is high enough, our approach outperforms equivalent methods by a large margin.•
Revised: When the number of programs is high enough, our approach outperforms previous methods by a large margin.

Original:  An analysis of the properties that make for a good image generation program in Section 5.
Revised: • An analysis of the properties that make for a good image generation program in Section 5.

Original: Thisprovides insights on how to design new programs that maximize performance.
Revised: This provides insights on how to design new programs that maximize performance.

Revised: Previous work [2] showed that performance saturates fast when generating more samples from their data distribution (i

Revised: e. more images from their curated set of programs), while our work shows that more samples from our distribution (i

Revised: e. more programs) do not saturate performance at the current scale.

Revised: This points out a clear direction on how to further improve performance: collect or learn to generate more image programs.

Original: To the best of our knowledge this is the first work to pair a large collection of image generationprocesses and short programs that is amenable to experimentation.
Revised: To the best of our knowledge this is the first work to pair a large collection of image-generation processes and short programs that is amenable to experimentation.

Original: This set of procedural programshas the potential to be a testbed to answer key questions linked to these paired modalities, someof which we answer in this paper We will release
Revised: This set of procedural programs has the potential to be a testbed to answer key questions linked to these paired modalities, some of which we answer in this paper.

Original: the

Original: rendering code and training pipeline upon

Original: acceptance.

Original:  Previous work
Revised:  Previous work

Original: Pre-training with synthetic data Pre-training neural networks is a widely used technique to improveperformance and convergence speed during training.
Revised: Pre-training with synthetic data Pre-training neural networks is a well-known technique to improve performance and convergence speed during training.

Original: Typically, pre-trained networks are either usedas a starting point in the optimization or to transfer the knowledge from related tasks to the one ofinterest [7].
Revised: Typically, pre-trained networks are either used as a starting point in the optimization or to transfer the knowledge from related tasks to the one of interest [7].

Original: When available, pre-training uses real data as close as possible to the task of interest.
Revised: When available, pre-training uses real data as close as possible to the task of interest.

Original: If the dataavailable for the task is scarce, a common practice is to train on a dataset of natural images, typically
Revised: If the data available for the task is scarce, a common practice is to train on a dataset of natural images, typically

Original: ImageNet [8].
Revised: ImageNet [8].

Original: These pre-trained models are then used on a wide variety of tasks, achieving state-ofthe-art results on tasks with different degrees of similarity to ImageNet.
Revised: These pre-trained models are then used on a wide variety of tasks, achieving state-ofthe-art results on tasks with different degrees of similarity to ImageNet.

Original: Despite this, it is unclearhow much of the performance gain is due to knowledge transfer from the pre-training task or theoptimization procedure being simpler to engineer when starting from a pre-trained model [9].
Revised: Despite this, it is unclear how much of the performance gain is due to knowledge transfer from the pre-training task or the optimization procedure being simpler to engineer when starting from a pre-trained model [9].

Original: Alternatively, other works achieve state-of-the-art performance by pre-training on synthetic data.
Revised: Alternatively, other works achieve state-of-the-art performance by pre-training on synthetic data.

Original: These approaches generate realistic-looking images paired with ground-truth annotations, which arethen used as pre-training data.
Revised: These approaches generate realistic-looking images paired with ground-truth annotations, which are then used as pre-training data.

Original: Currently, this approach is a standard technique to obtain state-of-theart performance for low-level vision tasks such as flow estimation [10, 11] and depth prediction [12],between others.
Revised: Currently, this approach is a standard technique to obtain state-of-the- art performance for low-level vision tasks such as flow estimation [10, 11] and depth prediction [12], between others.

Original: Contrary to these approaches, recent works have studied whether it is necessary to use real-lookingdata for pre-training.
Revised: Contrary to these approaches, recent works have studied whether it is necessary to use real-looking data for pre-training.

Original: The motivations for doing so are several folds, from targeting domains where

Original: Property Avg. [Table] rendering would require expert knowledge (like medical imaging), deploying agnostic models todownstream tasks, or making the generative process interpretable and optimizable [3].
Revised: The motivations for doing so are several folds, from targeting domains where rendering would require expert knowledge (like medical imaging), deploying agnostic models to downstream tasks, or making the generative process interpretable and optimizable [3].

Original: In this spirit, the work of [1] introduced a way to train image models using renders with fractalstructures.
Revised: In this spirit, the work of [1] introduced a way to train image models using renders with fractal structures.

Original: This procedure has been shown to work with different neural architectures [5] and hasbeen further improved with better sampling and colorization strategies [13].
Revised: This procedure has been shown to work with different neural architectures [5] and has been further improved with better sampling and colorization strategies [13].

Original: In the same spirit, [2]tested different generative processes to learn general image representations.
Revised: In the same spirit, [2] tested different generative processes to learn general image representations.

Original: Their best-performingmodels are based on neural networks, which are not interpretable and require expert knowledge tomanually design.
Revised: Their best-performing models are based on neural networks, which are not interpretable and require expert knowledge to design manually.

Original: Representation learning Representation learning is at a level of maturity that shallow networks ontop of pre-trained models outperform supervised approaches trained from scratch [14, 15].
Revised: Representation learning Representation learning is at a level of maturity that shallow networks on top of pre-trained models outperform supervised approaches trained from scratch [14, 15].

Original: Supervisedapproaches trained on a large corpus with supervision are currently the best representation learners interms of performance [16].
Revised: Training on a large corpus with supervision is currently the best representation of learning practice in terms of performance [16].

Original: Despite this, these models inherit the ethical issues associated with thedatasets they are trained on. Although they outperform models trained with smaller data collections,this comes at the price of more severe ethical issues, as data curation at a large scale is costly. On the other hand, unsupervised approaches allow training with unlabeled data, which is usuallyeasier to acquire than labeled  data  at the same scale by the final users of  the models.
Revised: On the other hand, unsupervised approaches allow training with unlabeled data, which is usually easier to acquire than labeled data. Although both outperform models trained with smaller data collections, this comes at the price of more severe ethical issues, as models inherit the concerns associated with the datasets they are trained on.

Original: Once practitionershave pre-trained a model with unlabeled data, they can fine-tune or learn a shallow network on top,which typically requires a substantially smaller amount of annotated data.

Original: The most widely used method for unsupervised representation learning is contrastive learning [17],which has seen consistent improvements in recent years [14, 18].
Revised: The most widely used method for unsupervised representation learning is contrastive learning [17], which has seen consistent improvements in recent years [14, 18].

Original: Contrastive learning is wellestablished and has been shown to perform well in a wide range of image domains, including medicalimaging [19] and satellite imagery [20].
Revised: Contrastive learning is well established and has been shown to perform well in a wide range of image domains, including medical imaging [19] and satellite imagery [20].

Original: Because of this, in this work, we focus on contrastivelearning as an unsupervised approach to learning image representations.
Revised: Because of this, in this work we focus on contrastive learning as an unsupervised approach to learning image representations.

Original: Alternatively, early work on unsupervised representation learning focused on pretext tasks thatare different from contrastive training, such as context prediction [21] or rotation prediction [22].
Revised: Alternatively, early work on unsupervised representation learning focused on pretext tasks that are different from contrastive training, such as context prediction [21] or rotation prediction [22].

Original: Although theseinitially did not achieve the levels of performance of contrastive approaches, recentwork has shown that one of these pretext tasks, masked  image prediction  withautoencoding, coupledwith Vision Transformers [23], can outperform contrastive training in large-scale settings [24].
Revised: Although these did not achieve the levels of performance of contrastive learning, recent work has shown that one of these pretext tasks, masked autoencoding with an image prediction loss, coupled with Vision Transformers [23], can outperform contrastive training in large-scale settings [24].

Original: Wedo not study these novel approaches as their robustness and broad applicability to different tasks andsmall training budgets is yet to be studied.
Revised: We do not study these novel approaches as their robustness and broad applicability to different tasks and small training budgets is yet to be studied.

Original:  The Shaders-1k/21k dataset
Revised:  The Shaders-1k/21k dataset

Original: The collection of programs we propose consists of a set of generative processes of the form: g iθ : z →x .
Revised: The collection of programs we propose consists of a set of generative processes of the form: g iθ : z → x .

Original: Each generative program corresponds to the latent class g iθ , from which we can produce imagesx by sampling the stochastic variables z from a prior model.
Revised: Each generative program corresponds to the latent class g iθ , from which we can produce images x by sampling the stochastic variables z from a prior model.

Original: The constant i indexes the programsavailable, while θ corresponds to the numerical constants of the generative program.
Revised: The constant i indexes the programs available, while θ corresponds to the numerical constants of the generative program.

Original: In the proposeddataset, each g iθ corresponds to an OpenGL fragment shader.
Revised: In the proposed collection, each g iθ corresponds to an OpenGL fragment shader.

Original: Existing similar methods [1, 2] focuson a single type of program ( i = 1 , with variable θ ), while this work focuses on a large collection ofgenerative procedures ( i ∈ { 1 , ..., 21 k } ) with fixed θ .
Revised: Existing similar methods [1, 2] focus on a single type of program ( i = 1 , with variable θ ), while this work focuses on a large collection of generative procedures ( i ∈ { 1 , ..., 21 k } ) with fixed θ .

Original: In previous work, the generative programs were designed ad hoc in different programming languages,making it hard to scale or mix different approaches.
Revised: In previous work, the generative programs were designed ad hoc in different programming languages, making it hard to scale or mix different approaches.

Original: To overcome this, we fix a common programminglanguage and choose the OpenGL Shading Language, a high-level programming language that allowsspecifying rendering programs with a syntax based on the C programming language.
Revised: To overcome this, we fix a common programming language and choose the OpenGL Shading Language, a high-level programming language that allows specifying rendering programs with a syntax based on the C programming language.

Original: Furthermore, itis amenable to GPU acceleration, making it particularly suitable to quickly produce images using astandard deep learning stack.
Revised: Furthermore, it is amenable to GPU acceleration, making it particularly suitable to quickly produce images using a standard deep learning stack.

Original: 3.1 Program collection
Revised: 3.1 Program collection

Original: To obtain the desired OpenGL fragment shaders, we query the web for code snippets correspondingto fragment shaders.
Revised: To obtain the desired OpenGL fragment shaders, we query the web for code snippets corresponding to fragment shaders.

Original: We obtain them from two sources: Twitter and Shadertoy [25].
Revised: We obtain them from two sources: Twitter and Shadertoy [25].

Original: The programsobtained from Twitter use the TwiGL syntax as described in [26], which allows  extremely compactcode (typically less than 280 characters) by abstracting away standard functionalities. 
Revised: The programs obtained from Twitter use the TwiGL syntax as described in [26], which allows for extremely compact code (typically less than 280 characters) by abstracting away standard functionalities. On the other Property

Original: On the otherhand, Shadertoy is a repository of OpenGL shaders using the standard OpenGL syntax.
Revised: Avg [Table] columns correspond to shaders from -1k hand, Shadertoy is a repository of OpenGL shaders using the standard OpenGL syntax.

Original: Shaders in
Revised: Shaders in

Original: Shadertoy are generally longer (on average 4k characters) and they produce simpler images thanthose from TwiGL, as the latter are typically coded by more expert users.
Revised: Shadertoy are generally longer (on average 4k characters) and they produce simpler images than those from TwiGL, as the latter are typically coded by more expert users.

Original: The only stochastic variable we consider in practice in the collected shaders is the timestep ( z = t ∈115 R ).
Revised: The only stochastic variable we consider in practice in the collected shaders is the timestep ( z = t ∈ R ).

Original: The parameter t allows generating a continuous video stream, which in lots of cases is periodic.
Revised: The parameter t allows generating a continuous video stream, which in lots of cases is periodic.

Original: We sample t at 4 frames per second for as many samples as required through all experiments.
Revised: We sample t at 4 frames per second for as many samples as required through all experiments.

Original: We addan increment to t sampled at uniform ∆ t ∼ U [0 , 0 . 25] , to avoid exact duplicates for the cases wherethe shader is periodic.
Revised: We add an increment to t sampled at uniform ∆ t ∼ U [0 , 0 . 25] , to avoid exact duplicates for the cases where the shader is periodic.

Original: After downloading the code snippets from these two sources, we obtain an initial set of 52 k shadersthat compile with the OpenGL shading language syntax.
Revised: After downloading the code snippets from these two sources, we obtain an initial set of 52 k shaders that compile with the OpenGL shading language syntax.

Original: We first perform duplicate removal byrendering a single image using the same t for all scripts and removing shaders that produce the sameimages.
Revised: We first perform duplicate removal by rendering a single image using the same t for all scripts and removing shaders that produce the same images.

Original: Additionally, we remove scripts for which different t ’s produce the same image (i.e. theydo not produce videos).
Revised: Additionally, we remove scripts for which different t ’s produce the same image (i.e. they produce static images).

Original: With this, we obtain a final set of 1 .
Revised: With this, we obtain a final set of 1 .

Original: 089 fragment shaders from TwiGL and19 .
Revised: 089 fragment shaders from TwiGL and 19 .

Original: 994 from Shadertoy, which are unique and generate a diverse set of images when varying t .
Revised: 994 from Shadertoy, which are unique and generate a diverse set of images when varying t .

Original: In Table 1, we report the main characteristics of the shaders and the images they produce.
Revised: In Table 1, we report the main characteristics of the shaders and the images they produce.

Original: Combiningall the shaders from both sources, images can be rendered at 979 frames per second at a resolution of384 × 384 , using a single modern GPU and including transfer to general memory.
Revised: Combining all the shaders from both sources, images can be rendered at 979 frames per second at a resolution of 384 × 384 , using a single modern GPU and including transfer to general memory.

Original: When stored todisk as JPEG, the average size per image is 54 kB, compared to 70 kB for ImageNet-1k when resizedto the same resolution.
Revised: When stored to disk as JPEG, the average size per image is 54 kB, compared to 70 kB for ImageNet-1k when resized to the same resolution.

Original: Shaders-1k/21k From a qualitative analysis and the compression results of the generated images in
Revised: Shaders-1k/21k From a qualitative analysis and the compression results of the generated images in

Original: Table 1, we further conclude that TwiGL shaders are more curated and of higher quality than thoseobtained from Shadertoy (see Figure 2 for examples of both).
Revised: Table 1, we further conclude that TwiGL shaders are more curated and of higher quality than those obtained from Shadertoy (see Figure 2 for examples of both).

Original: To distinguish and experiment withboth sources, we refer to the dataset composed of only TwiGL shaders as Shaders-1k and the datasetconsisting of all available shaders from both sources as Shaders-21k.
Revised: To distinguish and experiment with both sources, we refer to the dataset composed of only TwiGL shaders as Shaders-1k and the dataset consisting of all available shaders from both sources as Shaders-21k.

Original:  Experiments 
Revised:  Experiments 4.1 Supervised vs Unsupervised representation learning

Original: 4.1 Supervised vs Unsupervised representation learning

Original: To test downstream performance when pre-training with Shaders-1k and Shaders-21k, we performan initial set of experiments with and without latent class supervision.
Revised: To test downstream performance when pre-training with Shaders-1k and Shaders-21k, we perform an initial set of experiments with and without latent class supervision.

Original: We test three representationlearning methodologies: supervised classification with cross-entropy loss (CE), supervised contrastive 
Revised: We test three representation learning methodologies: supervised classification with cross-entropy loss (CE), supervised contrastive learning (SupCon) [27], and unsupervised representation learning (SimCLR) [14].

Original: Dataset
Revised: Dataset

Original: S.CLR
Revised: S.CLR

Original: CE
Revised: CE

Original: S.Con
Revised: S.Con

Original: Random init.
Revised: Random init.

Original: Places 52 .50 .55 .
Revised: Places 52 .50 .55 .

Original: I60 .78 .80 .
Revised: I60 .78 .80 .

Original: S.GAN O.
Revised: S.GAN O.

Original: D.leaves M.
Revised: D.leaves M.

Original: FracDB-1k 33 .26 .32 .
Revised: FracDB-1k 33 .26 .32 .

Original: S-1k 43 .41 .40 .
Revised: S-1k 43 .41 .40 .

Original: S-21k 47 .40 .48 .
Revised: S-21k 47 .40 .48 .

Original: [Figure]
Revised: [Figure]

Original: To compare these training methodologies, we train a ResNet-18 with 100k samples generated for eachdataset at a resolution of 256 × 256 .
Revised: To compare these training methodologies, we train a ResNet-18 with 100k samples generated for each dataset at a resolution of 256 × 256 .

Original: We follow the training procedure and public implementationdescribed in [27] for the three methods.
Revised: We follow the training procedure and public implementation described in [27] for the three methods.

Original: We train for 200 epochs with a batch size of 256 , with therest of the hyperparameters set to those found to work well for ImageNet-100 in the original paper.
Revised: We train for 200 epochs with a batch size of 256 , with the rest of the hyperparameters set to those found to work well for ImageNet-100 in the original paper.

Original: The images after augmentations are fed to the network at a resolution of 64 × 64 .
Revised: The images after augmentations are fed to the network at a resolution of 64 × 64 .

Original: After training theencoder, we evaluate using the linear protocol described in [17], which consists of only training alinear classifier on top of the representations learned.
Revised: After training the encoder, we evaluate using the linear protocol described in [17], which consists of only training a linear classifier on top of the representations learned.

Original: We evaluate on ImageNet-100, a subset of
Revised: We evaluate on ImageNet-100, a subset of

Original: ImageNet-1k [8] defined in [28] using the averaged pooled features from the last convolutional layer,which have a dimensionality of 512.
Revised: ImageNet-1k [8] defined in [28] using the averaged pooled features from the last convolutional layer, which have a dimensionality of 512.

Original: As shown in Table 2, we compare performance using the shader datasets against Places365 [6], whichwas proposed in [2] as an upper bound for real images when testing on ImageNet; two procedurallygenerated datasets, FractalDB [1] and Dead Leaves with mixed shapes [2]; and the state of the art fortraining without real data, StyleGAN Oriented [2].
Revised: As shown in Table 2, we compare performance using the shader datasets against Places365 [6], which was proposed in [2] as an upper bound for real images when testing on ImageNet; two procedurally generated datasets, FractalDB [1] and Dead Leaves with mixed shapes [2]; and the state of the art for training with similar data, StyleGAN Oriented [2].

Original: Shaders-21k using SupCon achieves the best performance over existing methods that do not use realdata, but it is closely followed by the same dataset when using SimCLR.
Revised: Shaders-21k using SupCon achieves the best performance over existing methods that do not use real data, but it is closely followed by the same dataset when using SimCLR.

Original: Both methods performingequally is expected in this case, as the number of class positives per batch decreases with the numberof classes in the dataset.
Revised: Both methods performing similarly is expected in this case, as the number of class positives per batch decreases with the number of classes in the dataset.

Original: In this experiment, the batch size is relatively small (256) compared to thenumber of classes (21k).
Revised: In this experiment, the batch size is relatively small (256) compared to the number of classes (21k).

Original: As the number of classes increases, the SupCon loss tends to be equivalentto SimCLR loss, and in the limit of having as many classes as training samples, they are the same.
Revised: As the number of classes increases, the SupCon loss tends to be equivalent to SimCLR loss, and in the limit of having as many classes as training samples, they are the same.

Original: Using this setting, we test performance for an increasing number of generative programs, sampled atrandom from Shaders-21k.
Revised: Using this setting, we test performance for an increasing number of generative programs, sampled at random from Shaders-21k.

Original: In this experiment, the number of images is kept constant at 100k whilethe number of shaders increases (i.e. with more shaders, there are fewer images per shader).
Revised: In this experiment, the number of images is kept constant at 100k while the number of shaders increases (i.e. with more shaders, there are fewer images per shader).

Original: As seenin Figure 3, we observe that supervision is helpful in the middle regime where there are more than20 shaders but less than 1000 shaders, but fails to outperform unsupervised approaches outside thisregime.
Revised: As seen in Figure 3, we observe that supervision is helpful in the middle regime where there are more than 20 shaders but less than 1000 shaders, but fails to outperform unsupervised approaches outside this regime.

Original: We also note from this sweep that performance is not saturated at 21k shaders, although itscales logarithmically with the number of shaders.
Revised: We also note from this sweep that performance is not saturated at 21k shaders, and it scales logarithmically with the number of shaders.

Original: Finally, we also note from this sweep that using the full set of Shaders-1k outperforms a random subsetof Shaders-21k of the same size.
Revised: Finally, we also note from this sweep that using the full set of Shaders-1k outperforms a random subset of Shaders-21k of the same size.

Original: Shaders-1k achieves higher top-1 accuracy on ImageNet-100 ( 43 . 26 )compared to a random subset of 1089 shaders obtained from Shaders-21k ( 42 . 26 ).
Revised: Shaders-1k achieves higher top-1 accuracy on ImageNet-100 ( 43 . 26 ) compared to a random subset of 1089 shaders obtained from Shaders-21k ( 42 . 26 ).

Original: This furtherverifies that the shaders of Shaders-1k are on average of higher quality than those on Shaders-21k, ashypothesized in Section 3.
Revised: This further verifies that the shaders of Shaders-1k are on average of higher quality than those on Shaders-21k, as hypothesized in Section 3.

Original: Pre-train Dataset
Revised: Pre-train Dataset

Original: I-
Revised: I

Original: [Table]
Revised: [Table]

Original: VTAB Struct.
Revised: VTAB [Table]

Original: Random init
Revised: Random init

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Places365 [6]
Revised: Places365 [6]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: ImageNet-1k
Revised: ImageNet-1k

Original: [Table]
Revised: [Table]

Original: [Table]

Original: StyleGAN
Revised: StyleGAN

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]

Revised: StyleGAN

Original: [Table]
Revised: [Table]

Original: StyleGAN O. [2] (MixUp)
Revised: [2] (MixUp)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: FractalDB

Original: [Table]
Revised: [Table]

Original: FractalDB-1k [1]
Revised: 1k [1]

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: Dead-leaves Mixed [2]
Revised: Dead-leaves Mixed [2]

Original: [Table]
Revised: [Table]

Original: [Table]

Original: [Table]

Original: [Table]
Revised: [Table]

Original: 4.2 Large scale unsupervised training
Revised: 4.2 Large scale unsupervised training

Original: Following the observation that contrastive training performs better than classification to pre-trainfeature extractors, we extend the previous approach to a large-scale setting.
Revised: Following the observation that contrastive training performs better than classification to pre-train feature extractors, we extend the previous approach to a large-scale setting.

Original: In this setting, we traina ResNet-50 using MoCo v2 [18], with images generated at 384 × 384 resolution.
Revised: In this setting, we train a ResNet-50 using MoCo v2 [18], with images generated at 384 × 384 resolution.

Original: We train for200 epochs with 1.3M images, with a batch size of 256 , and set the rest of the hyperparameters tothose found to work best on ImageNet-1k in the original paper.
Revised: We train for 200 epochs with 1.3M images, with a batch size of 256 , and set the rest of the hyperparameters to those found to work best on ImageNet-1k in the original paper.

Original: In Table 3, we show downstreamperformance for ImageNet-1k and ImageNet-100 and the 19 datasets of the Visual Task Adaptation
Revised: In Table 3, we show downstream performance for ImageNet-1k and ImageNet-100 and the 19 datasets of the Visual Task Adaptation

Original: Benchmark [29], averaged over the natural, specialized and structured categories.
Revised: Benchmark [29], averaged over the natural, specialized and structured categories.

Original: When using data directly rendered from Shaders-1/21k, the performance is low compared to existingmethods.
Revised: When using data directly rendered from Shaders-1/21k, the performance is low compared to existing methods.

Original: We observe that the contrastive loss is lower for Shaders-1k compared to ImageNet-1k (6.2and 6.6), while top-1 accuracy for the contrastive task is much worse for Shaders-1k (12.5% and92.2% respectively).
Revised: We observe that the contrastive loss is lower for Shaders-1k compared to ImageNet-1k (6. and 6.6), while top-1 accuracy for the contrastive task is much worse for Shaders-1k (12.5% and 92.2% respectively).

Original: This points out that the network is able to solve the contrastive task properly onaverage (i.e. it is highly confident on most of the 64k negatives in the MoCo buffer), while it is highlyconfused by a few of the negative samples, which correspond to images from the same shader.
Revised: This points out that the network is able to solve the contrastive task properly on average (i.e. it is highly confident on most of the 64k negatives in the MoCo buffer), while it is highly confused by a few of the negative samples, which correspond to images from the same shader.

Original: We hypothesize that this is caused by a large capacity network being able to learn shortcut solutionsfor the contrastive task [30], which results in learned features that do not transfer to downstreamtasks.
Revised: We hypothesize that this is caused by a large capacity network being able to learn shortcut solutions for the contrastive task [30], which results in learned features that do not transfer to downstream tasks.

Original: As can be seen from examples in Figure 2, it is easy for a large capacity network to distinguishdifferent shaders via shortcut solutions, like background color or salient features, even after MoCo v2image augmentations.
Revised: As can be seen from examples in Figure 2, it is easy for a large capacity network to distinguish different shaders via shortcut solutions, like background color or salient features, even after MoCo v image augmentations.

Original: To overcome this, we propose mixing samples from different shaders in order to produce interpolatedsamples in pixel space.
Revised: To overcome this, we propose mixing samples from different shaders in order to produce interpolated samples in pixel space.

Original: This removes some of the shortcut solutions while preserving the complexityof the underlying processes being mixed.
Revised: This removes some of the shortcut solutions while preserving the complexity of the underlying processes being mixed.

Original: We investigate three well-known mixing strategies: CutMix[31], MixUp [32], and producing samples from a GAN [33] that has been trained on the shaders.
Revised: We investigate three well-known mixing strategies: CutMix [31], MixUp [32], and producing samples from a GAN [33] that has been trained on the shaders.

Original: MixUp has been previously studied in the context of unsupervised contrastive learning [34], showingimproved performance in the case where the train time distribution is aligned with the test timedistribution.
Revised: MixUp has been previously studied in the context of unsupervised contrastive learning [34], showing improved performance in the case where the train time distribution is aligned with the test time distribution.

Original: Using FID computed with 50k samples with respect to ImageNet-100, we conclude that MixUpoutperforms the other two strategies (details in Supp.
Revised: Using FID computed with 50k samples with respect to ImageNet-100, we conclude that MixUp outperforms the other two strategies (details in Supp.

Original: Mat.).
Revised: Mat.).

Original: This metric was found to correlate withdownstream performance in [2], and allows searching for good hyperparameters without training.
Revised: This metric was found to correlate with downstream performance in [2], and allows searching for good hyperparameters without training.

Original: For the MixUp strategy, we found that mixing 6 frames with weights sampled from a Dirichletdistribution with α i = 1 yields the best FID, while rendering time is still affordable.
Revised: For the MixUp strategy, we found that mixing 6 frames with weights sampled from a Dirichlet distribution with α i = 1 yields the best FID, while rendering time is still affordable.

Original: To be fair againstprevious methods, MixUp is performed offline before training starts, so that the diversity at trainingtime is the same for all datasets (i.e. 1.3M fixed images, with the same MoCo v2 augmentationsacross all experiments).
Revised: For a fair comparison against previous methods, MixUp is performed offline before training starts, so that the amount of images seen during training time is the same for all datasets (i.e. 1.3M fixed images, with the same MoCo v augmentations applied during training across all experiments).

Original: As seen in Table 3, training with MixUp greatly outperforms the raw shaders.
Revised: As seen in Table 3, training with MixUp greatly outperforms the raw shaders.

Original: Shaders-21k with
Revised: Shaders-21k with

Original: MixUp achieves the best performance overall on ImageNet-1k/100 and outperforms the previous stateof-the-art by a substantial margin: an extra 6.71% top-1 accuracy on ImageNet-1k.
Revised: MixUp achieves the best performance overall on ImageNet-1k/100 and outperforms the previous state-of-the-art by a substantial margin: an extra 6.71% top-1 accuracy on ImageNet-1k.

Original: This represents a38% relative improvement for the gap between real images different than ImageNet-1k (Places365),and the previous state-of-the-art for procedural images, StyleGAN Oriented.
Revised: This represents a 38% relative improvement for the gap between real images different than ImageNet-1k (Places365), and the previous state-of-the-art for procedural images, StyleGAN Oriented [2].

Revised: In

Original: In Figure 4 we show 5nearest-neighbors retrieval on Imagenet-100 for our best model against a random initialized networkand places (additional results for other networks can be found in Supp.
Revised: Figure 4 we show 5 nearest-neighbors retrieval on Imagenet-100 for our best model against a random initialized network and the Places365 network (additional results for other networks can be found in Supp.

Original: Mat.).
Revised: Mat.).

Original: As can be seen, nearestneighbors successfully retrieve good candidates given a query, capturing different perceptual aspectsof the image that are not captured with a randomly initialized model.
Revised: As can be seen, nearest neighbors successfully retrieves good candidates given a query, capturing different perceptual aspects of the image that are not captured with a randomly initialized model.

Original: We note that StyleGAN Oriented [2] does not improve with the MixUp strategy, achieving worsedownstream performance than with the original samples.
Revised: We note that StyleGAN Oriented [2] does not improve with the MixUp strategy, achieving worse downstream performance than with the original samples.

Original: This is consistent with the FID metric, asthe samples with MixUP yield worse (higher) FID than the samples without (38.74 vs 37.74).
Revised: This is consistent with the FID metric, as the samples with MixUP yield worse (higher) FID than the samples without (38.74 vs 37.74).

Original: In the case of VTAB, Shaders-1k/21k with MixUp outperforms the baselines for natural and specialized tasks, in the case of specialized tasks being close to Places performance.
Revised: In the case of VTAB, Shaders-1k/21k with MixUp outperforms the baselines for natural and specialized tasks, in the case of specialized tasks being close to Places performance.

Original: On the other hand,
Revised: On the other hand,

Original: Shaders-1k/21k without mixing outperform their MixUp alternative on structured tasks, althoughperformance for structured tasks is generally low for methods trained with contrastive learning.
Revised: Shaders-1k/21k without MixUp outperform their MixUp counterparts on structured tasks, although performance for structured tasks is generally low for methods trained with contrastive learning.

Original: Thisis because structured tasks require information about counts, position, or size of the elements in theimages, and this information is lost given the invariances that the data augmentations impose [35].
Revised: This is because structured tasks require information about counts, position, or size of the elements in the images, and this information is lost given the invariances that the data augmentations impose [35].

Original: From the results per dataset (see Supp.
Revised: From the results per dataset (see Supp.

Original: Mat.), it can be seen that Shaders-1k performs well (evenbetter than ImageNet) on datasets that resemble shaders, as is the case for dSprites dataset, whichconsists of 2D shapes procedurally generated.
Revised: Mat.), it can be seen that Shaders-1k performs well (even better than ImageNet) on datasets that resemble shaders, as is the case of the dSprites dataset, which consists of 2D shapes procedurally generated.

Original: Finally, we use the fact that the Shaders dataset allows rendering novel samples during train time totrain with the same procedure, but with a live generator (i.e. the samples for each of the 200 epochsare different).
Revised: Finally, we use the fact that the Shaders dataset allows rendering novel samples during train time to train with the same procedure, but with a live generator (i.e. the samples for each of the 200 epochs are different).

Original: This methodology requires twice as much time to train with the same setup, as the GPUis shared by the rendering and the training, while only improving performance by a small margin,achieving a top-1 accuracy on ImageNet-1k/100 of 45.24% and 66.42%, respectively.
Revised: This methodology requires twice as much time to train with the same compute power, as the GPU is shared by the rendering and the training. Despite this, it only improves performance marginally, achieving a top-1 accuracy on ImageNet-1k/100 of 45.24% and 66.42%, respectively.

Original: This showsthat scaling up the number of samples per shader  has limited benefits, as this corresponds to 200times more samples (corresponding to the 200 epochs) than the results presented in Table 3.
Revised: This shows that, similarly to other works, scaling up the number of samples per shader (and not the number of shaders) has limited benefits, as this corresponds to 200 times more samples (corresponding to the 200 epochs) than the results presented in Table 3.

Original:  What makes for a good generative image program?
Revised:  What makes for a good generative image program?

Original: A single shader: To test what properties of a single shader are useful for downstream performance,we train a ResNet-18 with SimCLR as described in Section 4.1 for each shader in Shaders-1k.
Revised: A single shader: To test what properties of a single shader are useful for downstream performance, we train a ResNet-18 with SimCLR as described in Section 4.1 for each shader in Shaders-1k.

Original: Weuse 10k images from each of the programs in Shaders-1k, train the ResNet-18 for 100 epochs, andthen compute performance with linear evaluation for 100 epochs on a random subset of 10k imagesfrom ImageNet-100.
Revised: We use 10k images from each of the programs in Shaders-1k, train the ResNet-18 for 100 epochs, and then compute performance with linear evaluation for 100 epochs on a random subset of 10k images from ImageNet-100.

Original: As shown in Figure 5, there is substantial variability in performance betweeneach of the shaders, and one of the main qualitative drivers  is the diversity between samples. 
Revised: As shown in Figure 5, there is substantial variability in performance between each of the shaders, and one of the main qualitative drivers for performance is the diversity between samples. [Figure]

Original: With the performance computed for each of the shaders in Shaders-1k, we train two shallow networks,consisting of two linear layers with ReLu and a hidden dimension of 32 to predict performance pershader.
Revised: With the performance computed for each of the shaders in Shaders-1k, we train two shallow networks, consisting of two linear layers with ReLu and a hidden dimension of 32 to predict performance per shader.

Original: The inputs to the shallow network are the features extracted with the ResNet-50 trainedwith MoCo v2 for Shaders-1k with MixUp in Section 4.2.
Revised: The inputs to the shallow network are the features extracted with the ResNet-50 trained with MoCo v2 for Shaders-1k with MixUp in Section 4.2.

Original: We train using an L1-loss to regress[Figure] top-1 accuracy for each shader, splitting them 85 - 15 into train-val.
Revised: We train using an L1-loss to regress top-1 accuracy for each shader, splitting them 85 - 15 into train-val.

Original: Of the two trained networks,one gets information from 50 frames (the concatenated average, minimum, and maximum of theactivations) while the other only takes a single image as input.
Revised: Of the two trained networks, one gets information from 50 frames (as the concatenated average, minimum, and maximum of the activations) while the other only takes a single image as input.

Original: We use these networks for the rest ofthe experiments in this section, depending on whether the phenomena studied requires predictingperformance from a single or multiple frames.
Revised: We use these networks for the rest of the experiments in this section, depending on whether the phenomena studied requires predicting performance from a single or multiple frames.

Original: With the network that gets information from 50 frames, we rank all the shaders in Shaders-21k.
Revised: With the network that gets information from 50 frames, we rank all remaining shaders in Shaders21k.

Original: In

Original:  Figure 7 we show the shader with the best-predicted performance over all Shaders-21k, which: 1)qualitatively is more  diverse than the top-performing in Shaders-1k (seen in Figure 5 2) its top-1accuracy with the comparable training setup is 36%, considerably higher than the 26% achieved forthe best shader  Shaders-1k.
Revised: In Figure 7 we show the shader with the best predicted performance over all Shaders-21k, which is more qualitatively diverse than the top-performing in Shaders-1k (see in Figure 5)and its top-1 empirical accuracy is 36%, considerably higher than the 26% achieved for the best shader in Shaders-1k.

Original: Additionally, in Figure 6 we show scatter plots for several simple properties of the generated imagesfor each shader against their performance.
Revised: Additionally, in Figure 6 we show scatter plots for several simple properties of the generated images for each shader against their performance.

Original: As can be seen, the metric that best correlates withperformance ( r = 0 . 46 ) is LPIPS intra-image distance, matching the findings in [2].
Revised: As can be seen, the metric that best correlates with performance ( r = 0 . 46 ) is LPIPS intra-image distance, matching the findings in [2].

Original: LPIPS [36] isa perceptual similarity metric that has been found to align better with human judgment than otheralternatives.
Revised: LPIPS [36] is a perceptual similarity metric that has been found to align better with human judgment than other alternatives.

Original: LPIPS intra-image is defined as the average LPIPS distance for two random cropscovering 50% of a given image, and was first proposed in [37].
Revised: LPIPS intra-image is defined as the average LPIPS distance for two random crops covering 50% of a given image and was first proposed in [37].

Original: Consequently, we conclude thatshaders that produce images with high self-similarity tend to perform poorly.
Revised: Consequently, we conclude that shaders that produce images with high self-similarity tend to perform poorly.

Original: On the other hand, although FID and gzip compression have a low correlation with performance, theplots show that there is a Pareto frontier for both metrics, where shaders cannot achieve high accuracywith low compression or high FID.
Revised: On the other hand although FID and gzip compression have a low correlation with performance, the plots show that there is a Pareto frontier for both metrics where shaders cannot achieve high accuracy with low compression or high FID.

Original: Performance as a function of rendering time also suggest thatthere is a big margin to improve it, as the correlation between rendering time and performance isweak and there are lots of shaders that perform well while their rendering time is low.
Revised: Performance as a function of rendering time also suggests that there is a big margin for improvement, as the correlation between rendering time and performance is weak and there are lots of shaders that perform well while their rendering time is low.

Original: A collection of shaders: In the same spirit, we consider what makes a collection of shaders performwell or bad when treated as a set, and whether it is possible to achieve increasingly better performance with a fixed number of shader programs.
Revised: A collection of shaders: In the same spirit, we consider what makes a collection of shaders perform well or bad when treated as a set, and whether it is possible to achieve increasingly better performance with a fixed number of shader programs.

Original: With the trained performance predictor per shader as in Section we compute the expected performance for all shaders in Shaders-21k that are not part of Shaders-1k. In Figure 7 we compare using a variable number of shaders selected either by predictedperformance or at random.
Revised: In Figure 7 we compare using a variable number of shaders selected either by predicted performance (with the same predictor as before that takes 50 frames) or at random.

Original: As seen in the figure, the greedy approach of  selecting thebest-predictedperforming shader each time outperforms by a big margin random selection.
Revised: As seen in Figure 7, the greedy approach of each time selecting the shader with the best-predicted performance outperforms by a big margin random selection

Original: Finally, as a qualitative experiment of the properties of the Shaders-1k dataset that make for a goodshader we explore the space of a StyleGAN v2 trained with Shaders-1k. We use the shallow predictorthat only takes features from a single image to do so.
Revised: Finally, as a qualitative experiment of the properties of the Shaders-1k dataset that make for a good shader we explore the space of a StyleGAN v2 trained with Shaders-1k, with the shallow predictor that only takes features from a single image.

Original: Then we explore StyleGAN space by tracingwalks in the embedding space z , in the directions that minimize or maximize performance.
Revised: We explore StyleGAN space by tracing walks in the embedding space z , in the direction that minimizes or maximizes performance.

Original: As Figure8 shows, samples that improve performance are more colorful and fill the canvas.
Revised: As Figure 8 shows, samples that improve performance are more colorful and fill the canvas.

Original: On the other hand,samples obtained after traversing z in the direction that minimizes performance tend to collapse torepetitive shapes, gray-scale images, or big regions of the imagebeing ofa solid color.
Revised: On the other hand, samples obtained after traversing z in the direction that minimizes performance tend to collapse to repetitive shapes, gray-scale images, or big regions of the image of solid color.

Original:  Conclusions
Revised:  Conclusions

Original: In this paper, we have proposed using a large collection of procedural image programs to learndata-free representations.
Revised: This paper proposes using a large collection of procedural image programs to image representations without real images.

Original: As experiments show, training with shaders outperforms existing methodsthat do not use real data.
Revised: As experiments show, training with shaders outperforms existing methods that do not use real data.

Original: Using these procedural image models is competitive against natural images,especially in domains far away from the natural image domain, as demonstrated in the specializedtasks in VTAB.
Revised: Using these procedural image models is competitive against natural images, especially in domains far away from the natural image domain, as demonstrated in the specialized tasks in VTAB.

Original: Checklist
Revised: Acknowledgement

Revised: Manel Baradad was supported by the LaCaixa Fellowship, and this research was supported by a grant from the MIT-IBM Watson AI lab.

Revised: Experiments were partially conducted using computation resources from the Satori cluster donated by IBM to MIT, and the MIT’s Supercloud cluster.

Original: The checklist follows the references. Please read the checklist guidelines carefully for information onhow to answer these questions.
Revised: This paper was prepared for information purposes by the teams of researchers from the various institutions identified above, including the Global Technology Applied Research group of JPMorgan Chase Bank, N.A..

Original: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] . You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description. For example: • Did you include the license to the code and datasets?
Revised: This paper is not a product of the Research Department of JPMorgan Chase Bank,

Original: [Yes] See Section ?? .
Revised: N.A. or its affiliates.

Original: • Did you include the license to the code and datasets?
Revised: Neither JPMorgan Chase Bank, N.A. nor any of its affiliates make any explicit or implied representation or warranty and none of them accept any liability in connection with this paper, including, but limited to, the completeness, accuracy, reliability of information contained herein and the potential legal, compliance, tax or accounting effects thereof.

Original: [No] The code and the data areproprietary.• Did you include the license to the code and datasets? [N/A] Please do not modify the questions and only use the provided macros for your answers.
Revised: This document is not intended as investment research or investment advice or a recommendation, offer or solicitation for the purchase or sale of any security, financial instrument, financial product or service or to be used in any way for evaluating the merits of participating in any transaction.

Original: Note that the Checklistsection does not count towards the page limit. In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes](d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
Revised: [Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A](b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [No] We will release them upon acceptance. (b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
Revised: [Yes] https: //github.com/mbaradad/shaders21k (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] For novel techniques, we report hyperparameters.
Revised: [Yes] For novel techniques, we report hyperparameters.

Original: For well-known methods, we use the default set of hyperparameters, as described in the references
Revised: For well-known methods, we use the default set of hyperparameters, as described in the references

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [No] We run large scale experiments (ImageNet-1k scale), and multiple runs are computationally unfeasible (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: [No] We run large scale experiments (ImageNet-1k scale), and multiple runs are computationally unfeasible (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] We include total amount of computeto generate the data.
Revised: [Yes] We include total amount of compute to generate the data.

Original: For well-known methods, we use standard hyperparameters, andthe compute requiered is the same as the original methods. 
Revised: For well-known methods, we use standard hyperparameters, and the compute requiered is the same as the original methods. 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] We use code-snippets available online,and we will only release the references to the available websites.
Revised: [Yes] We use code-snippets available online, and we will only release the references to the available websites.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: [No] (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [Yes] We use code-snippets available online, and we will only release the references to the available websites.
Revised: [Yes] We use code-snippets available online, and we will only release the references to the available websites.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable[Table] information or offensive content?
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [Yes] We havemanually checked the output of thescripts to ensure there is no offensive content.[Table]
Revised: [Yes] We have checked the output of the generative image programs through our experiments and we have not found offensive content.

Original: If you used crowdsourcing or conducted research with human subjects...[Table] (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable? [N/A]
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [Table]

Original:  (b) Did you describe any potential participant risks, with links to Institutional Review[Table] Board (IRB) approvals, if applicable?[N/A]
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [Table] (c) Did you include the estimated hourly wage paid to participants and the total amount[Table] spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: [Table]