Original: Procedural Image Programs for Representation Learning
Revised: Procedural Image Programs for Representation Learning

Original: Abstract
Revised: Abstract

Original: Learning image representations using synthetic data allows training neural networks without some of the concerns associated with real images, such as privacy and bias.
Revised: Learning image representations using synthetic data allows training neural networks without some of the concerns associated with real images, such as privacy and bias.

Original: Existing work focuses on a handful of curated generative processes which require expert knowledge to design, making it hard to scale up.
Revised: Existing work focuses on a handful of curated generative processes which require expert knowledge to design, making it hard to scale up.

Original: To overcome this, we propose training with a large dataset of twenty-one thousand programs, each one generating a diverse set of synthetic images.
Revised: To overcome this, we propose training with a large dataset of twenty-one thousand programs, each one generating a diverse set of synthetic images.

Original: These programs are short code snippets, which are easy to modify and fast to execute using OpenGL.
Revised: These programs are short code snippets, which are easy to modify and fast to execute using OpenGL.

Original: The proposed dataset can be used for both supervised and unsupervised representation learning and reduces the gap between pre-training with real and procedurally generated images by 38%.
Revised: The proposed dataset can be used for both supervised and unsupervised representation learning and reduces the gap between pre-training with real and procedurally generated images by 38%.

Original: Code, models, and datasets are available at: https: //github.com/mbaradad/shaders21k
Revised: Code, models, and datasets are available at: https: //github.com/mbaradad/shaders21k

Original:  Introduction
Revised:  Introduction

Original: Training neural networks using data from hand-crafted generative models is a novel approach that allows pre-training without access to real data.
Revised: Training neural networks using data from hand-crafted generative models is a novel approach that allows pre-training without access to real data.

Original: This technique has been shown to be effective on downstream tasks for different modalities, such as images [1, 2], videos [3], and text [4].
Revised: This technique has been shown to be effective on downstream tasks for different modalities, such as images [1, 2], videos [3], and text [4].

Original: Despite its big potential, previous work for images focuses on a handful of generative processes, like fractals [5], textured polygons [3], or dead-leaves and statistical processes [2].
Revised: Despite its big potential, previous work for images focuses on a handful of generative processes, like fractals [5], textured polygons [3], or dead-leaves and statistical processes [2].

Original: To achieve good performance with the original set of procedures, expert knowledge is required to make the generated images match simple image statistics.
Revised: To achieve good performance with the original set of procedures, expert knowledge is required to make the generated images match simple image statistics.

Original: This makes it hard to improve over existing work, as these generative models have already been carefully studied and tuned.
Revised: This makes it hard to improve over existing work, as these generative models have already been carefully studied and tuned.

Original: What would happen if we used a large set of procedural image programs without curation, instead of focusing on a small set of programs?
Revised: What would happen if we used a large set of procedural image programs without curation, instead of focusing on a small set of programs?

Original: To this end, we collect a large-scale set of programs, each producing a diverse set of images that display simple shapes and textures (see Figures 1 and 2 for examples).
Revised: To this end, we collect a large-scale set of programs, each producing a diverse set of images that display simple shapes and textures (see Figures 1 and 2 for examples).

Original: We then use these programs to generate images and train neural networks with supervised and unsupervised representation learning methods.
Revised: We then use these programs to generate images and train neural networks with supervised and unsupervised representation learning methods.

Original: These networks can then be used for different downstream tasks, reaching state-of-the-art performance for methods that do not pre-train using real images.
Revised: These networks can then be used for different downstream tasks, reaching state-of-the-art performance for methods that do not pre-train using real images.

Original: The generative programs of this collection are coded in a common language, the OpenGL shading language, that encapsulates the image generation process in a few lines of code.
Revised: The generative programs of this collection are coded in a common language, the OpenGL shading language, that encapsulates the image generation process in a few lines of code.

Original: Previous work uses ad-hoc slow methods for each generative process, which makes it hard to integrate them into the same framework.
Revised: Previous work uses ad-hoc slow methods for each generative process, which makes it hard to integrate them into the same framework.

Original: Compared to existing approaches, the rendering of our programs is performed at high throughput (hundreds of frames per second with a single modern GPU), allowing generation on the fly while training.
Revised: Compared to existing approaches, the rendering of our programs is performed at high throughput (hundreds of frames per second with a single modern GPU), allowing generation on the fly while training.

Original: As the dataset consists of a large collection of generative processes, we can use each code snippet as a latent class, which allows learning representations using supervised approaches.
Revised: As the dataset consists of a large collection of generative processes, we can use each code snippet as a latent class, which allows learning representations using supervised approaches.

Original: In contrast, previous methods require ad-hoc techniques for clustering the generative parameters into discrete classes [5].
Revised: In contrast, previous methods require ad-hoc techniques for clustering the generative parameters into discrete classes [5].

Original: Together with the collection of programs, the main contributions of the paper are
Revised: Together with the collection of programs, the main contributions of the paper are

Original: : • A study on how performance scales as a function of the number of generative image programs, for supervised and unsupervised approaches.
Revised: : • A study on how performance scales as a function of the number of generative image programs, for supervised and unsupervised approaches.

Original: Our experiments show that downstream performance scales logarithmically with the number of generative programs (see Figure 3 and Section 4).
Revised: Our experiments show that downstream performance scales logarithmically with the number of generative programs (see Figure 3 and Section 4).

Original: • State-of-the-art performance for representation learning without real data.
Revised: • State-of-the-art performance for representation learning without real data.

Original: When the number of programs is high enough, our approach outperforms previous methods by a large margin.
Revised: When the number of programs is high enough, our approach outperforms previous methods by a large margin.

Original: • An analysis of the properties that make for a good image generation program in Section 5.
Revised: • An analysis of the properties that make for a good image generation program in Section 5.

Original: This provides insights on how to design new programs that maximize performance.
Revised: This provides insights on how to design new programs that maximize performance.

Original: Previous work [2] showed that performance saturates fast when generating more samples from their data distribution (i
Revised: Previous work [2] showed that performance saturates fast when generating more samples from their data distribution (i

Original: e. more images from their curated set of programs), while our work shows that more samples from our distribution (i
Revised: e. more images from their curated set of programs), while our work shows that more samples from our distribution (i

Original: e. more programs) do not saturate performance at the current scale.
Revised: e. more programs) do not saturate performance at the current scale.

Original: This points out a clear direction on how to further improve performance: collect or learn to generate more image programs.
Revised: This points out a clear direction on how to further improve performance: collect or learn to generate more image programs.

Original: To the best of our knowledge this is the first work to pair a large collection of image-generation processes and short programs that is amenable to experimentation.
Revised: To the best of our knowledge this is the first work to pair a large collection of image-generation processes and short programs that is amenable to experimentation.

Original: This set of procedural programs has the potential to be a testbed to answer key questions linked to these paired modalities, some of which we answer in this paper.
Revised: This set of procedural programs has the potential to be a testbed to answer key questions linked to these paired modalities, some of which we answer in this paper.

Original:  Previous work
Revised:  Previous work

Original: Pre-training with synthetic data Pre-training neural networks is a well-known technique to improve performance and convergence speed during training.
Revised: Pre-training with synthetic data Pre-training neural networks is a well-known technique to improve performance and convergence speed during training.

Original: Typically, pre-trained networks are either used as a starting point in the optimization or to transfer the knowledge from related tasks to the one of interest [7].
Revised: Typically, pre-trained networks are either used as a starting point in the optimization or to transfer the knowledge from related tasks to the one of interest [7].

Original: When available, pre-training uses real data as close as possible to the task of interest.
Revised: When available, pre-training uses real data as close as possible to the task of interest.

Original: If the data available for the task is scarce, a common practice is to train on a dataset of natural images, typically
Revised: If the data available for the task is scarce, a common practice is to train on a dataset of natural images, typically

Original: ImageNet [8].
Revised: ImageNet [8].

Original: These pre-trained models are then used on a wide variety of tasks, achieving state-ofthe-art results on tasks with different degrees of similarity to ImageNet.
Revised: These pre-trained models are then used on a wide variety of tasks, achieving state-ofthe-art results on tasks with different degrees of similarity to ImageNet.

Original: Despite this, it is unclear how much of the performance gain is due to knowledge transfer from the pre-training task or the optimization procedure being simpler to engineer when starting from a pre-trained model [9].
Revised: Despite this, it is unclear how much of the performance gain is due to knowledge transfer from the pre-training task or the optimization procedure being simpler to engineer when starting from a pre-trained model [9].

Original: Alternatively, other works achieve state-of-the-art performance by pre-training on synthetic data.
Revised: Alternatively, other works achieve state-of-the-art performance by pre-training on synthetic data.

Original: These approaches generate realistic-looking images paired with ground-truth annotations, which are then used as pre-training data.
Revised: These approaches generate realistic-looking images paired with ground-truth annotations, which are then used as pre-training data.

Original: Currently, this approach is a standard technique to obtain state-of-the- art performance for low-level vision tasks such as flow estimation [10, 11] and depth prediction [12], between others.
Revised: Currently, this approach is a standard technique to obtain state-of-the- art performance for low-level vision tasks such as flow estimation [10, 11] and depth prediction [12], between others.

Original: Contrary to these approaches, recent works have studied whether it is necessary to use real-looking data for pre-training.
Revised: Contrary to these approaches, recent works have studied whether it is necessary to use real-looking data for pre-training.

Original: The motivations for doing so are several folds, from targeting domains where rendering would require expert knowledge (like medical imaging), deploying agnostic models to downstream tasks, or making the generative process interpretable and optimizable [3].
Revised: The motivations for doing so are several folds, from targeting domains where rendering would require expert knowledge (like medical imaging), deploying agnostic models to downstream tasks, or making the generative process interpretable and optimizable [3].

Original: In this spirit, the work of [1] introduced a way to train image models using renders with fractal structures.
Revised: In this spirit, the work of [1] introduced a way to train image models using renders with fractal structures.

Original: This procedure has been shown to work with different neural architectures [5] and has been further improved with better sampling and colorization strategies [13].
Revised: This procedure has been shown to work with different neural architectures [5] and has been further improved with better sampling and colorization strategies [13].

Original: In the same spirit, [2] tested different generative processes to learn general image representations.
Revised: In the same spirit, [2] tested different generative processes to learn general image representations.

Original: Their best-performing models are based on neural networks, which are not interpretable and require expert knowledge to design manually.
Revised: Their best-performing models are based on neural networks, which are not interpretable and require expert knowledge to design manually.

Original: Representation learning Representation learning is at a level of maturity that shallow networks on top of pre-trained models outperform supervised approaches trained from scratch [14, 15].
Revised: Representation learning Representation learning is at a level of maturity that shallow networks on top of pre-trained models outperform supervised approaches trained from scratch [14, 15].

Original: Training on a large corpus with supervision is currently the best representation of learning practice in terms of performance [16].
Revised: Training on a large corpus with supervision is currently the best representation of learning practice in terms of performance [16].

Original: On the other hand, unsupervised approaches allow training with unlabeled data, which is usually easier to acquire than labeled data.
Revised: On the other hand, unsupervised approaches allow training with unlabeled data, which is usually easier to acquire than labeled data.

Original: Although both outperform models trained with smaller data collections, this comes at the price of more severe ethical issues, as models inherit the concerns associated with the datasets they are trained on.
Revised: Although both outperform models trained with smaller data collections, this comes at the price of more severe ethical issues, as models inherit the concerns associated with the datasets they are trained on.

Original: The most widely used method for unsupervised representation learning is contrastive learning [17], which has seen consistent improvements in recent years [14, 18].
Revised: The most widely used method for unsupervised representation learning is contrastive learning [17], which has seen consistent improvements in recent years [14, 18].

Original: Contrastive learning is well established and has been shown to perform well in a wide range of image domains, including medical imaging [19] and satellite imagery [20].
Revised: Contrastive learning is well established and has been shown to perform well in a wide range of image domains, including medical imaging [19] and satellite imagery [20].

Original: Because of this, in this work we focus on contrastive learning as an unsupervised approach to learning image representations.
Revised: Because of this, in this work we focus on contrastive learning as an unsupervised approach to learning image representations.

Original: Alternatively, early work on unsupervised representation learning focused on pretext tasks that are different from contrastive training, such as context prediction [21] or rotation prediction [22].
Revised: Alternatively, early work on unsupervised representation learning focused on pretext tasks that are different from contrastive training, such as context prediction [21] or rotation prediction [22].

Original: Although these did not achieve the levels of performance of contrastive learning, recent work has shown that one of these pretext tasks, masked autoencoding with an image prediction loss, coupled with Vision Transformers [23], can outperform contrastive training in large-scale settings [24].
Revised: Although these did not achieve the levels of performance of contrastive learning, recent work has shown that one of these pretext tasks, masked autoencoding with an image prediction loss, coupled with Vision Transformers [23], can outperform contrastive training in large-scale settings [24].

Original: We do not study these novel approaches as their robustness and broad applicability to different tasks and small training budgets is yet to be studied.
Revised: We do not study these novel approaches as their robustness and broad applicability to different tasks and small training budgets is yet to be studied.

Original:  The Shaders-1k/21k dataset
Revised:  The Shaders-1k/21k dataset

Original: The collection of programs we propose consists of a set of generative processes of the form: g iθ : z → x .
Revised: The collection of programs we propose consists of a set of generative processes of the form: g iθ : z → x .

Original: Each generative program corresponds to the latent class g iθ , from which we can produce images x by sampling the stochastic variables z from a prior model.
Revised: Each generative program corresponds to the latent class g iθ , from which we can produce images x by sampling the stochastic variables z from a prior model.

Original: The constant i indexes the programs available, while θ corresponds to the numerical constants of the generative program.
Revised: The constant i indexes the programs available, while θ corresponds to the numerical constants of the generative program.

Original: In the proposed collection, each g iθ corresponds to an OpenGL fragment shader.
Revised: In the proposed collection, each g iθ corresponds to an OpenGL fragment shader.

Original: Existing similar methods [1, 2] focus on a single type of program ( i = 1 , with variable θ ), while this work focuses on a large collection of generative procedures ( i ∈ { 1 , ..., 21 k } ) with fixed θ .
Revised: Existing similar methods [1, 2] focus on a single type of program ( i = 1 , with variable θ ), while this work focuses on a large collection of generative procedures ( i ∈ { 1 , ..., 21 k } ) with fixed θ .

Original: In previous work, the generative programs were designed ad hoc in different programming languages, making it hard to scale or mix different approaches.
Revised: In previous work, the generative programs were designed ad hoc in different programming languages, making it hard to scale or mix different approaches.

Original: To overcome this, we fix a common programming language and choose the OpenGL Shading Language, a high-level programming language that allows specifying rendering programs with a syntax based on the C programming language.
Revised: To overcome this, we fix a common programming language and choose the OpenGL Shading Language, a high-level programming language that allows specifying rendering programs with a syntax based on the C programming language.

Original: Furthermore, it is amenable to GPU acceleration, making it particularly suitable to quickly produce images using a standard deep learning stack.
Revised: Furthermore, it is amenable to GPU acceleration, making it particularly suitable to quickly produce images using a standard deep learning stack.

Original: 3.1 Program collection
Revised: 3.1 Program collection

Original: To obtain the desired OpenGL fragment shaders, we query the web for code snippets corresponding to fragment shaders.
Revised: To obtain the desired OpenGL fragment shaders, we query the web for code snippets corresponding to fragment shaders.

Original: We obtain them from two sources: Twitter and Shadertoy [25].
Revised: We obtain them from two sources: Twitter and Shadertoy [25].

Original: The programs obtained from Twitter use the TwiGL syntax as described in [26], which allows for extremely compact code (typically less than 280 characters) by abstracting away standard functionalities.
Revised: The programs obtained from Twitter use the TwiGL syntax as described in [26], which allows for extremely compact code (typically less than 280 characters) by abstracting away standard functionalities.

Original: On the other
Revised: On the other

Original: Property
Revised: Property

Original: Avg
Revised: Avg

Original: [Table]
Revised: [Table]

Original: columns correspond to shaders from
Revised: columns correspond to shaders from

Original: -1k hand, Shadertoy is a repository of OpenGL shaders using the standard OpenGL syntax.
Revised: -1k hand, Shadertoy is a repository of OpenGL shaders using the standard OpenGL syntax.

Original: Shaders in
Revised: Shaders in

Original: Shadertoy are generally longer (on average 4k characters) and they produce simpler images than those from TwiGL, as the latter are typically coded by more expert users.
Revised: Shadertoy are generally longer (on average 4k characters) and they produce simpler images than those from TwiGL, as the latter are typically coded by more expert users.

Original: The only stochastic variable we consider in practice in the collected shaders is the timestep ( z = t ∈ R ).
Revised: The only stochastic variable we consider in practice in the collected shaders is the timestep ( z = t ∈ R ).

Original: The parameter t allows generating a continuous video stream, which in lots of cases is periodic.
Revised: The parameter t allows generating a continuous video stream, which in lots of cases is periodic.

Original: We sample t at 4 frames per second for as many samples as required through all experiments.
Revised: We sample t at 4 frames per second for as many samples as required through all experiments.

Original: We add an increment to t sampled at uniform ∆ t ∼ U [0 , 0 . 25] , to avoid exact duplicates for the cases where the shader is periodic.
Revised: We add an increment to t sampled at uniform ∆ t ∼ U [0 , 0 . 25] , to avoid exact duplicates for the cases where the shader is periodic.

Original: After downloading the code snippets from these two sources, we obtain an initial set of 52 k shaders that compile with the OpenGL shading language syntax.
Revised: After downloading the code snippets from these two sources, we obtain an initial set of 52 k shaders that compile with the OpenGL shading language syntax.

Original: We first perform duplicate removal by rendering a single image using the same t for all scripts and removing shaders that produce the same images.
Revised: We first perform duplicate removal by rendering a single image using the same t for all scripts and removing shaders that produce the same images.

Original: Additionally, we remove scripts for which different t ’s produce the same image (i.e. they produce static images).
Revised: Additionally, we remove scripts for which different t ’s produce the same image (i.e. they produce static images).

Original: With this, we obtain a final set of 1 .
Revised: With this, we obtain a final set of 1 .

Original: 089 fragment shaders from TwiGL and 19 .
Revised: 089 fragment shaders from TwiGL and 19 .

Original: 994 from Shadertoy, which are unique and generate a diverse set of images when varying t .
Revised: 994 from Shadertoy, which are unique and generate a diverse set of images when varying t .

Original: In Table 1, we report the main characteristics of the shaders and the images they produce.
Revised: In Table 1, we report the main characteristics of the shaders and the images they produce.

Original: Combining all the shaders from both sources, images can be rendered at 979 frames per second at a resolution of 384 × 384 , using a single modern GPU and including transfer to general memory.
Revised: Combining all the shaders from both sources, images can be rendered at 979 frames per second at a resolution of 384 × 384 , using a single modern GPU and including transfer to general memory.

Original: When stored to disk as JPEG, the average size per image is 54 kB, compared to 70 kB for ImageNet-1k when resized to the same resolution.
Revised: When stored to disk as JPEG, the average size per image is 54 kB, compared to 70 kB for ImageNet-1k when resized to the same resolution.

Original: Shaders-1k/21k From a qualitative analysis and the compression results of the generated images in
Revised: Shaders-1k/21k From a qualitative analysis and the compression results of the generated images in

Original: Table 1, we further conclude that TwiGL shaders are more curated and of higher quality than those obtained from Shadertoy (see Figure 2 for examples of both).
Revised: Table 1, we further conclude that TwiGL shaders are more curated and of higher quality than those obtained from Shadertoy (see Figure 2 for examples of both).

Original: To distinguish and experiment with both sources, we refer to the dataset composed of only TwiGL shaders as Shaders-1k and the dataset consisting of all available shaders from both sources as Shaders-21k.
Revised: To distinguish and experiment with both sources, we refer to the dataset composed of only TwiGL shaders as Shaders-1k and the dataset consisting of all available shaders from both sources as Shaders-21k.

Original:  Experiments 4.1 Supervised vs Unsupervised representation learning
Revised:  Experiments 4.1 Supervised vs Unsupervised representation learning

Original: To test downstream performance when pre-training with Shaders-1k and Shaders-21k, we perform an initial set of experiments with and without latent class supervision.
Revised: To test downstream performance when pre-training with Shaders-1k and Shaders-21k, we perform an initial set of experiments with and without latent class supervision.

Original: We test three representation learning methodologies: supervised classification with cross-entropy loss (CE), supervised contrastive learning (SupCon) [27], and unsupervised representation learning (SimCLR) [14].
Revised: We test three representation learning methodologies: supervised classification with cross-entropy loss (CE), supervised contrastive learning (SupCon) [27], and unsupervised representation learning (SimCLR) [14].

Original: Dataset
Revised: Dataset

Original: S.CLR
Revised: S.CLR

Original: CE
Revised: CE

Original: S.Con
Revised: S.Con

Original: Random init.
Revised: Random init.

Original: Places 52 .50 .55 .
Revised: Places 52 .50 .55 .

Original: I60 .78 .80 .
Revised: I60 .78 .80 .

Original: S.GAN O.
Revised: S.GAN O.

Original: D.leaves M.
Revised: D.leaves M.

Original: FracDB-1k 33 .26 .32 .
Revised: FracDB-1k 33 .26 .32 .

Original: S-1k 43 .41 .40 .
Revised: S-1k 43 .41 .40 .

Original: S-21k 47 .40 .48 .
Revised: S-21k 47 .40 .48 .

Original: [Figure]
Revised: [Figure]

Original: To compare these training methodologies, we train a ResNet-18 with 100k samples generated for each dataset at a resolution of 256 × 256 .
Revised: To compare these training methodologies, we train a ResNet-18 with 100k samples generated for each dataset at a resolution of 256 × 256 .

Original: We follow the training procedure and public implementation described in [27] for the three methods.
Revised: We follow the training procedure and public implementation described in [27] for the three methods.

Original: We train for 200 epochs with a batch size of 256 , with the rest of the hyperparameters set to those found to work well for ImageNet-100 in the original paper.
Revised: We train for 200 epochs with a batch size of 256 , with the rest of the hyperparameters set to those found to work well for ImageNet-100 in the original paper.

Original: The images after augmentations are fed to the network at a resolution of 64 × 64 .
Revised: The images after augmentations are fed to the network at a resolution of 64 × 64 .

Original: After training the encoder, we evaluate using the linear protocol described in [17], which consists of only training a linear classifier on top of the representations learned.
Revised: After training the encoder, we evaluate using the linear protocol described in [17], which consists of only training a linear classifier on top of the representations learned.

Original: We evaluate on ImageNet-100, a subset of
Revised: We evaluate on ImageNet-100, a subset of

Original: ImageNet-1k [8] defined in [28] using the averaged pooled features from the last convolutional layer, which have a dimensionality of 512.
Revised: ImageNet-1k [8] defined in [28] using the averaged pooled features from the last convolutional layer, which have a dimensionality of 512.

Original: As shown in Table 2, we compare performance using the shader datasets against Places365 [6], which was proposed in [2] as an upper bound for real images when testing on ImageNet; two procedurally generated datasets, FractalDB [1] and Dead Leaves with mixed shapes [2]; and the state of the art for training with similar data, StyleGAN Oriented [2].
Revised: As shown in Table 2, we compare performance using the shader datasets against Places365 [6], which was proposed in [2] as an upper bound for real images when testing on ImageNet; two procedurally generated datasets, FractalDB [1] and Dead Leaves with mixed shapes [2]; and the state of the art for training with similar data, StyleGAN Oriented [2].

Original: Shaders-21k using SupCon achieves the best performance over existing methods that do not use real data, but it is closely followed by the same dataset when using SimCLR.
Revised: Shaders-21k using SupCon achieves the best performance over existing methods that do not use real data, but it is closely followed by the same dataset when using SimCLR.

Original: Both methods performing similarly is expected in this case, as the number of class positives per batch decreases with the number of classes in the dataset.
Revised: Both methods performing similarly is expected in this case, as the number of class positives per batch decreases with the number of classes in the dataset.

Original: In this experiment, the batch size is relatively small (256) compared to the number of classes (21k).
Revised: In this experiment, the batch size is relatively small (256) compared to the number of classes (21k).

Original: As the number of classes increases, the SupCon loss tends to be equivalent to SimCLR loss, and in the limit of having as many classes as training samples, they are the same.
Revised: As the number of classes increases, the SupCon loss tends to be equivalent to SimCLR loss, and in the limit of having as many classes as training samples, they are the same.

Original: Using this setting, we test performance for an increasing number of generative programs, sampled at random from Shaders-21k.
Revised: Using this setting, we test performance for an increasing number of generative programs, sampled at random from Shaders-21k.

Original: In this experiment, the number of images is kept constant at 100k while the number of shaders increases (i.e. with more shaders, there are fewer images per shader).
Revised: In this experiment, the number of images is kept constant at 100k while the number of shaders increases (i.e. with more shaders, there are fewer images per shader).

Original: As seen in Figure 3, we observe that supervision is helpful in the middle regime where there are more than 20 shaders but less than 1000 shaders, but fails to outperform unsupervised approaches outside this regime.
Revised: As seen in Figure 3, we observe that supervision is helpful in the middle regime where there are more than 20 shaders but less than 1000 shaders, but fails to outperform unsupervised approaches outside this regime.

Original: We also note from this sweep that performance is not saturated at 21k shaders, and it scales logarithmically with the number of shaders.
Revised: We also note from this sweep that performance is not saturated at 21k shaders, and it scales logarithmically with the number of shaders.

Original: Finally, we also note from this sweep that using the full set of Shaders-1k outperforms a random subset of Shaders-21k of the same size.
Revised: Finally, we also note from this sweep that using the full set of Shaders-1k outperforms a random subset of Shaders-21k of the same size.

Original: Shaders-1k achieves higher top-1 accuracy on ImageNet-100 ( 43 . 26 ) compared to a random subset of 1089 shaders obtained from Shaders-21k ( 42 . 26 ).
Revised: Shaders-1k achieves higher top-1 accuracy on ImageNet-100 ( 43 . 26 ) compared to a random subset of 1089 shaders obtained from Shaders-21k ( 42 . 26 ).

Original: This further verifies that the shaders of Shaders-1k are on average of higher quality than those on Shaders-21k, as hypothesized in Section 3.
Revised: This further verifies that the shaders of Shaders-1k are on average of higher quality than those on Shaders-21k, as hypothesized in Section 3.

Original: Pre-train Dataset
Revised: Pre-train Dataset

Original: I
Revised: I

Original: [Table]
Revised: [Table]

Original: VTAB
Revised: VTAB

Original: [Table]
Revised: [Table]

Original: Random init
Revised: Random init

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Places365 [6]
Revised: Places365 [6]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: ImageNet-1k
Revised: ImageNet-1k

Original: [Table]
Revised: [Table]

Original: StyleGAN
Revised: StyleGAN

Original: [Table] [Table]
Revised: [Table] [Table]

Original: StyleGAN
Revised: StyleGAN

Original: [Table]
Revised: [Table]

Original: [2] (MixUp)
Revised: [2] (MixUp)

Original: [Table] [Table]
Revised: [Table] [Table]

Original: FractalDB
Revised: FractalDB

Original: [Table]
Revised: [Table]

Original: 1k [1]
Revised: 1k [1]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Dead-leaves Mixed [2]
Revised: Dead-leaves Mixed [2]

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: 4.2 Large scale unsupervised training
Revised: 4.2 Large scale unsupervised training

Original: Following the observation that contrastive training performs better than classification to pre-train feature extractors, we extend the previous approach to a large-scale setting.
Revised: Following the observation that contrastive training performs better than classification to pre-train feature extractors, we extend the previous approach to a large-scale setting.

Original: In this setting, we train a ResNet-50 using MoCo v2 [18], with images generated at 384 × 384 resolution.
Revised: In this setting, we train a ResNet-50 using MoCo v2 [18], with images generated at 384 × 384 resolution.

Original: We train for 200 epochs with 1.3M images, with a batch size of 256 , and set the rest of the hyperparameters to those found to work best on ImageNet-1k in the original paper.
Revised: We train for 200 epochs with 1.3M images, with a batch size of 256 , and set the rest of the hyperparameters to those found to work best on ImageNet-1k in the original paper.

Original: In Table 3, we show downstream performance for ImageNet-1k and ImageNet-100 and the 19 datasets of the Visual Task Adaptation
Revised: In Table 3, we show downstream performance for ImageNet-1k and ImageNet-100 and the 19 datasets of the Visual Task Adaptation

Original: Benchmark [29], averaged over the natural, specialized and structured categories.
Revised: Benchmark [29], averaged over the natural, specialized and structured categories.

Original: When using data directly rendered from Shaders-1/21k, the performance is low compared to existing methods.
Revised: When using data directly rendered from Shaders-1/21k, the performance is low compared to existing methods.

Original: We observe that the contrastive loss is lower for Shaders-1k compared to ImageNet-1k (6. and 6.6), while top-1 accuracy for the contrastive task is much worse for Shaders-1k (12.5% and 92.2% respectively).
Revised: We observe that the contrastive loss is lower for Shaders-1k compared to ImageNet-1k (6. and 6.6), while top-1 accuracy for the contrastive task is much worse for Shaders-1k (12.5% and 92.2% respectively).

Original: This points out that the network is able to solve the contrastive task properly on average (i.e. it is highly confident on most of the 64k negatives in the MoCo buffer), while it is highly confused by a few of the negative samples, which correspond to images from the same shader.
Revised: This points out that the network is able to solve the contrastive task properly on average (i.e. it is highly confident on most of the 64k negatives in the MoCo buffer), while it is highly confused by a few of the negative samples, which correspond to images from the same shader.

Original: We hypothesize that this is caused by a large capacity network being able to learn shortcut solutions for the contrastive task [30], which results in learned features that do not transfer to downstream tasks.
Revised: We hypothesize that this is caused by a large capacity network being able to learn shortcut solutions for the contrastive task [30], which results in learned features that do not transfer to downstream tasks.

Original: As can be seen from examples in Figure 2, it is easy for a large capacity network to distinguish different shaders via shortcut solutions, like background color or salient features, even after MoCo v image augmentations.
Revised: As can be seen from examples in Figure 2, it is easy for a large capacity network to distinguish different shaders via shortcut solutions, like background color or salient features, even after MoCo v image augmentations.

Original: To overcome this, we propose mixing samples from different shaders in order to produce interpolated samples in pixel space.
Revised: To overcome this, we propose mixing samples from different shaders in order to produce interpolated samples in pixel space.

Original: This removes some of the shortcut solutions while preserving the complexity of the underlying processes being mixed.
Revised: This removes some of the shortcut solutions while preserving the complexity of the underlying processes being mixed.

Original: We investigate three well-known mixing strategies: CutMix [31], MixUp [32], and producing samples from a GAN [33] that has been trained on the shaders.
Revised: We investigate three well-known mixing strategies: CutMix [31], MixUp [32], and producing samples from a GAN [33] that has been trained on the shaders.

Original: MixUp has been previously studied in the context of unsupervised contrastive learning [34], showing improved performance in the case where the train time distribution is aligned with the test time distribution.
Revised: MixUp has been previously studied in the context of unsupervised contrastive learning [34], showing improved performance in the case where the train time distribution is aligned with the test time distribution.

Original: Using FID computed with 50k samples with respect to ImageNet-100, we conclude that MixUp outperforms the other two strategies (details in Supp.
Revised: Using FID computed with 50k samples with respect to ImageNet-100, we conclude that MixUp outperforms the other two strategies (details in Supp.

Original: Mat.).
Revised: Mat.).

Original: This metric was found to correlate with downstream performance in [2], and allows searching for good hyperparameters without training.
Revised: This metric was found to correlate with downstream performance in [2], and allows searching for good hyperparameters without training.

Original: For the MixUp strategy, we found that mixing 6 frames with weights sampled from a Dirichlet distribution with α i = 1 yields the best FID, while rendering time is still affordable.
Revised: For the MixUp strategy, we found that mixing 6 frames with weights sampled from a Dirichlet distribution with α i = 1 yields the best FID, while rendering time is still affordable.

Original: For a fair comparison against previous methods, MixUp is performed offline before training starts, so that the amount of images seen during training time is the same for all datasets (i.e. 1.3M fixed images, with the same MoCo v augmentations applied during training across all experiments).
Revised: For a fair comparison against previous methods, MixUp is performed offline before training starts, so that the amount of images seen during training time is the same for all datasets (i.e. 1.3M fixed images, with the same MoCo v augmentations applied during training across all experiments).

Original: As seen in Table 3, training with MixUp greatly outperforms the raw shaders.
Revised: As seen in Table 3, training with MixUp greatly outperforms the raw shaders.

Original: Shaders-21k with
Revised: Shaders-21k with

Original: MixUp achieves the best performance overall on ImageNet-1k/100 and outperforms the previous state-of-the-art by a substantial margin: an extra 6.71% top-1 accuracy on ImageNet-1k.
Revised: MixUp achieves the best performance overall on ImageNet-1k/100 and outperforms the previous state-of-the-art by a substantial margin: an extra 6.71% top-1 accuracy on ImageNet-1k.

Original: This represents a 38% relative improvement for the gap between real images different than ImageNet-1k (Places365), and the previous state-of-the-art for procedural images, StyleGAN Oriented [2].
Revised: This represents a 38% relative improvement for the gap between real images different than ImageNet-1k (Places365), and the previous state-of-the-art for procedural images, StyleGAN Oriented [2].

Original: In
Revised: In

Original: Figure 4 we show 5 nearest-neighbors retrieval on Imagenet-100 for our best model against a random initialized network and the Places365 network (additional results for other networks can be found in
Revised: Figure 4 we show 5 nearest-neighbors retrieval on Imagenet-100 for our best model against a random initialized network and the Places365 network (additional results for other networks can be found in

Original: Supp.
Revised: Supp.

Original: Mat.).
Revised: Mat.).

Original: As can be seen, nearest neighbors successfully retrieves good candidates given a query, capturing different perceptual aspects of the image that are not captured with a randomly initialized model.
Revised: As can be seen, nearest neighbors successfully retrieves good candidates given a query, capturing different perceptual aspects of the image that are not captured with a randomly initialized model.

Original: We note that StyleGAN Oriented [2] does not improve with the MixUp strategy, achieving worse downstream performance than with the original samples.
Revised: We note that StyleGAN Oriented [2] does not improve with the MixUp strategy, achieving worse downstream performance than with the original samples.

Original: This is consistent with the FID metric, as the samples with MixUP yield worse (higher) FID than the samples without (38.74 vs 37.74).
Revised: This is consistent with the FID metric, as the samples with MixUP yield worse (higher) FID than the samples without (38.74 vs 37.74).

Original: In the case of VTAB, Shaders-1k/21k with MixUp outperforms the baselines for natural and specialized tasks, in the case of specialized tasks being close to Places performance.
Revised: In the case of VTAB, Shaders-1k/21k with MixUp outperforms the baselines for natural and specialized tasks, in the case of specialized tasks being close to Places performance.

Original: On the other hand,
Revised: On the other hand,

Original: Shaders-1k/21k without MixUp outperform their MixUp counterparts on structured tasks, although performance for structured tasks is generally low for methods trained with contrastive learning.
Revised: Shaders-1k/21k without MixUp outperform their MixUp counterparts on structured tasks, although performance for structured tasks is generally low for methods trained with contrastive learning.

Original: This is because structured tasks require information about counts, position, or size of the elements in the images, and this information is lost given the invariances that the data augmentations impose [35].
Revised: This is because structured tasks require information about counts, position, or size of the elements in the images, and this information is lost given the invariances that the data augmentations impose [35].

Original: From the results per dataset (see Supp.
Revised: From the results per dataset (see Supp.

Original: Mat.), it can be seen that Shaders-1k performs well (even better than ImageNet) on datasets that resemble shaders, as is the case of the dSprites dataset, which consists of 2D shapes procedurally generated.
Revised: Mat.), it can be seen that Shaders-1k performs well (even better than ImageNet) on datasets that resemble shaders, as is the case of the dSprites dataset, which consists of 2D shapes procedurally generated.

Original: Finally, we use the fact that the Shaders dataset allows rendering novel samples during train time to train with the same procedure, but with a live generator (i.e. the samples for each of the 200 epochs are different).
Revised: Finally, we use the fact that the Shaders dataset allows rendering novel samples during train time to train with the same procedure, but with a live generator (i.e. the samples for each of the 200 epochs are different).

Original: This methodology requires twice as much time to train with the same compute power, as the GPU is shared by the rendering and the training.
Revised: This methodology requires twice as much time to train with the same compute power, as the GPU is shared by the rendering and the training.

Original: Despite this, it only improves performance marginally, achieving a top-1 accuracy on ImageNet-1k/100 of 45.24% and 66.42%, respectively.
Revised: Despite this, it only improves performance marginally, achieving a top-1 accuracy on ImageNet-1k/100 of 45.24% and 66.42%, respectively.

Original: This shows that, similarly to other works, scaling up the number of samples per shader (and not the number of shaders) has limited benefits, as this corresponds to 200 times more samples (corresponding to the 200 epochs) than the results presented in Table 3.
Revised: This shows that, similarly to other works, scaling up the number of samples per shader (and not the number of shaders) has limited benefits, as this corresponds to 200 times more samples (corresponding to the 200 epochs) than the results presented in Table 3.

Original:  What makes for a good generative image program?
Revised:  What makes for a good generative image program?

Original: A single shader: To test what properties of a single shader are useful for downstream performance, we train a ResNet-18 with SimCLR as described in Section 4.1 for each shader in Shaders-1k.
Revised: A single shader: To test what properties of a single shader are useful for downstream performance, we train a ResNet-18 with SimCLR as described in Section 4.1 for each shader in Shaders-1k.

Original: We use 10k images from each of the programs in Shaders-1k, train the ResNet-18 for 100 epochs, and then compute performance with linear evaluation for 100 epochs on a random subset of 10k images from
Revised: We use 10k images from each of the programs in Shaders-1k, train the ResNet-18 for 100 epochs, and then compute performance with linear evaluation for 100 epochs on a random subset of 10k images from

Original: ImageNet-100.
Revised: ImageNet-100.

Original: As shown in Figure 5, there is substantial variability in performance between each of the shaders, and one of the main qualitative drivers for performance is the diversity between samples.
Revised: As shown in Figure 5, there is substantial variability in performance between each of the shaders, and one of the main qualitative drivers for performance is the diversity between samples.

Original: [Figure]
Revised: [Figure]

Original: With the performance computed for each of the shaders in Shaders-1k, we train two shallow networks, consisting of two linear layers with ReLu and a hidden dimension of 32 to predict performance per shader.
Revised: With the performance computed for each of the shaders in Shaders-1k, we train two shallow networks, consisting of two linear layers with ReLu and a hidden dimension of 32 to predict performance per shader.

Original: The inputs to the shallow network are the features extracted with the ResNet-50 trained with MoCo v2 for Shaders-1k with MixUp in Section 4.2.
Revised: The inputs to the shallow network are the features extracted with the ResNet-50 trained with MoCo v2 for Shaders-1k with MixUp in Section 4.2.

Original: We train using an L1-loss to regress top-1 accuracy for each shader, splitting them 85 - 15 into train-val.
Revised: We train using an L1-loss to regress top-1 accuracy for each shader, splitting them 85 - 15 into train-val.

Original: Of the two trained networks, one gets information from 50 frames (as the concatenated average, minimum, and maximum of the activations) while the other only takes a single image as input.
Revised: Of the two trained networks, one gets information from 50 frames (as the concatenated average, minimum, and maximum of the activations) while the other only takes a single image as input.

Original: We use these networks for the rest of the experiments in this section, depending on whether the phenomena studied requires predicting performance from a single or multiple frames.
Revised: We use these networks for the rest of the experiments in this section, depending on whether the phenomena studied requires predicting performance from a single or multiple frames.

Original: With the network that gets information from 50 frames, we rank all remaining shaders in Shaders21k.
Revised: With the network that gets information from 50 frames, we rank all remaining shaders in Shaders21k.

Original: In Figure 7 we show the shader with the best predicted performance over all Shaders-21k, which is more qualitatively diverse than the top-performing in Shaders-1k (see in Figure 5)and its top-1 empirical accuracy is 36%, considerably higher than the 26% achieved for the best shader in
Revised: In Figure 7 we show the shader with the best predicted performance over all Shaders-21k, which is more qualitatively diverse than the top-performing in Shaders-1k (see in Figure 5)and its top-1 empirical accuracy is 36%, considerably higher than the 26% achieved for the best shader in

Original: Shaders-1k.
Revised: Shaders-1k.

Original: Additionally, in Figure 6 we show scatter plots for several simple properties of the generated images for each shader against their performance.
Revised: Additionally, in Figure 6 we show scatter plots for several simple properties of the generated images for each shader against their performance.

Original: As can be seen, the metric that best correlates with performance ( r = 0 . 46 ) is LPIPS intra-image distance, matching the findings in [2].
Revised: As can be seen, the metric that best correlates with performance ( r = 0 . 46 ) is LPIPS intra-image distance, matching the findings in [2].

Original: LPIPS [36] is a perceptual similarity metric that has been found to align better with human judgment than other alternatives.
Revised: LPIPS [36] is a perceptual similarity metric that has been found to align better with human judgment than other alternatives.

Original: LPIPS intra-image is defined as the average LPIPS distance for two random crops covering 50% of a given image and was first proposed in [37].
Revised: LPIPS intra-image is defined as the average LPIPS distance for two random crops covering 50% of a given image and was first proposed in [37].

Original: Consequently, we conclude that shaders that produce images with high self-similarity tend to perform poorly.
Revised: Consequently, we conclude that shaders that produce images with high self-similarity tend to perform poorly.

Original: On the other hand although FID and gzip compression have a low correlation with performance, the plots show that there is a Pareto frontier for both metrics where shaders cannot achieve high accuracy with low compression or high FID.
Revised: On the other hand although FID and gzip compression have a low correlation with performance, the plots show that there is a Pareto frontier for both metrics where shaders cannot achieve high accuracy with low compression or high FID.

Original: Performance as a function of rendering time also suggests that there is a big margin for improvement, as the correlation between rendering time and performance is weak and there are lots of shaders that perform well while their rendering time is low.
Revised: Performance as a function of rendering time also suggests that there is a big margin for improvement, as the correlation between rendering time and performance is weak and there are lots of shaders that perform well while their rendering time is low.

Original: A collection of shaders: In the same spirit, we consider what makes a collection of shaders perform well or bad when treated as a set, and whether it is possible to achieve increasingly better performance with a fixed number of shader programs.
Revised: A collection of shaders: In the same spirit, we consider what makes a collection of shaders perform well or bad when treated as a set, and whether it is possible to achieve increasingly better performance with a fixed number of shader programs.

Original: In Figure 7 we compare using a variable number of shaders selected either by predicted performance (with the same predictor as before that takes 50 frames) or at random.
Revised: In Figure 7 we compare using a variable number of shaders selected either by predicted performance (with the same predictor as before that takes 50 frames) or at random.

Original: As seen in Figure 7, the greedy approach of each time selecting the shader with the best-predicted performance outperforms by a big margin random selection
Revised: As seen in Figure 7, the greedy approach of each time selecting the shader with the best-predicted performance outperforms by a big margin random selection

Original: Finally, as a qualitative experiment of the properties of the Shaders-1k dataset that make for a good shader we explore the space of a StyleGAN v2 trained with Shaders-1k, with the shallow predictor that only takes features from a single image.
Revised: Finally, as a qualitative experiment of the properties of the Shaders-1k dataset that make for a good shader we explore the space of a StyleGAN v2 trained with Shaders-1k, with the shallow predictor that only takes features from a single image.

Original: We explore StyleGAN space by tracing walks in the embedding space z , in the direction that minimizes or maximizes performance.
Revised: We explore StyleGAN space by tracing walks in the embedding space z , in the direction that minimizes or maximizes performance.

Original: As Figure 8 shows, samples that improve performance are more colorful and fill the canvas.
Revised: As Figure 8 shows, samples that improve performance are more colorful and fill the canvas.

Original: On the other hand, samples obtained after traversing z in the direction that minimizes performance tend to collapse to repetitive shapes, gray-scale images, or big regions of the image of solid color.
Revised: On the other hand, samples obtained after traversing z in the direction that minimizes performance tend to collapse to repetitive shapes, gray-scale images, or big regions of the image of solid color.

Original:  Conclusions
Revised:  Conclusions

Original: This paper proposes using a large collection of procedural image programs to image representations without real images.
Revised: This paper proposes using a large collection of procedural image programs to image representations without real images.

Original: As experiments show, training with shaders outperforms existing methods that do not use real data.
Revised: As experiments show, training with shaders outperforms existing methods that do not use real data.

Original: Using these procedural image models is competitive against natural images, especially in domains far away from the natural image domain, as demonstrated in the specialized tasks in VTAB.
Revised: Using these procedural image models is competitive against natural images, especially in domains far away from the natural image domain, as demonstrated in the specialized tasks in VTAB.

Original: Acknowledgement
Revised: Acknowledgement

Original: Manel Baradad was supported by the LaCaixa Fellowship, and this research was supported by a grant from the MIT-IBM Watson AI lab.
Revised: Manel Baradad was supported by the LaCaixa Fellowship, and this research was supported by a grant from the MIT-IBM Watson AI lab.

Revised: Rogerio Feris was supported in part by Darpa LwLL.

Original: Experiments were partially conducted using computation resources from the Satori cluster donated by IBM to MIT, and the MIT’s Supercloud cluster.
Revised: Experiments were partially conducted using computation resources from the Satori cluster donated by IBM to MIT, and the MIT’s Supercloud cluster.

Original: Disclaimer
Revised: Disclaimer

Original: This paper was prepared for information purposes by the teams of researchers from the various institutions identified above, including the Global Technology Applied Research group of JPMorgan
Revised: This paper was prepared for information purposes by the teams of researchers from the various institutions identified above, including the Global Technology Applied Research group of JPMorgan

Original: Chase Bank, N.A..
Revised: Chase Bank, N.A..

Original: This paper is not a product of the Research Department of JPMorgan Chase Bank,
Revised: This paper is not a product of the Research Department of JPMorgan Chase Bank,

Original: N.A. or its affiliates.
Revised: N.A. or its affiliates.

Original: Neither JPMorgan Chase Bank, N.A. nor any of its affiliates make any explicit or implied representation or warranty and none of them accept any liability in connection with this paper, including, but limited to, the completeness, accuracy, reliability of information contained herein and the potential legal, compliance, tax or accounting effects thereof.
Revised: Neither JPMorgan Chase Bank, N.A. nor any of its affiliates make any explicit or implied representation or warranty and none of them accept any liability in connection with this paper, including, but limited to, the completeness, accuracy, reliability of information contained herein and the potential legal, compliance, tax or accounting effects thereof.

Original: This document is not intended as investment research or investment advice or a recommendation, offer or solicitation for the purchase or sale of any security, financial instrument, financial product or service or to be used in any way for evaluating the merits of participating in any transaction.
Revised: This document is not intended as investment research or investment advice, or a recommendation, offer or solicitation for the purchase or sale of any security, financial instrument, financial product or service, or to be used in any way for evaluating the merits of participating in any transaction.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] (c) Did you discuss any potential negative societal impacts of your work?
Revised: [Yes] (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: [Yes] (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] https: //github.com/mbaradad/shaders21k (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: [Yes] https: //github.com/mbaradad/shaders21k (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] For novel techniques,
Revised: [Yes] For novel techniques,

Original: we report hyperparameters.
Revised: we report hyperparameters.

Original: For well-known methods, we use the default set of hyperparameters, as described in the references
Revised: For well-known methods, we use the default set of hyperparameters, as described in the references

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [No] We run large scale experiments (ImageNet-1k scale), and multiple runs are computationally unfeasible (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: [No] We run large scale experiments (ImageNet-1k scale), and multiple runs are computationally unfeasible (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] We include total amount of compute to generate the data.
Revised: [Yes] We include total amount of compute to generate the data.

Original: For well-known methods, we use standard hyperparameters, and the compute requiered is the same as the original methods. 4.
Revised: For well-known methods, we use standard hyperparameters, and the compute requiered is the same as the original methods. 4.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] We use code-snippets available online, and we will only release the references to the available websites.
Revised: [Yes] We use code-snippets available online, and we will only release the references to the available websites.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whether
Revised: [No] (d) Did you discuss whether

Original: and how consent was obtained from people whose data you’re using/curating?
Revised: and how consent was obtained from people whose data you’re using/curating?

Original: [Yes] We use code-snippets available online, and we will only release the references to the available websites.
Revised: [Yes] We use code-snippets available online, and we will only release the references to the available websites.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [Yes] We have checked the output of the generative image programs through our experiments and we have not found offensive content.
Revised: [Yes] We have checked the output of the generative image programs through our experiments and we have not found offensive content.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]