Original: U NDERSTANDING P URE CLIP G
Revised: U NDERSTANDING P URE CLIP G

Original: UIDANCE FOR V OXEL
Revised: UIDANCE FOR V OXEL

Original: G RID N E RF M ODELS
Revised: G RID N E RF M ODELS

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: [Figure]
Revised: [Figure]

Original: A BSTRACT
Revised: A BSTRACT

Original: We explore the task of text to 3D object generation using CLIP.
Revised: We explore the task of text to 3D object generation using CLIP.

Original: Specifically, we use CLIP for guidance without access to any datasets, a setting we refer to as pure
Revised: Specifically, we use CLIP for guidance without access to any datasets, a setting we refer to as pure

Original: CLIP guidance.
Revised: CLIP guidance.

Original: While prior work has adopted this setting, there is no systematic study of mechanics for preventing adversarial generations within CLIP.
Revised: While prior work has adopted this setting, there is no systematic study of mechanics for preventing adversarial generations within CLIP.

Original: We illustrate how different image-based augmentations prevent the adversarial generation problem, and how the generated results are impacted.
Revised: We illustrate how different image-based augmentations prevent the adversarial generation problem, and how the generated results are impacted.

Original: We test different CLIP model architectures and show that ensembling different models for guidance can prevent adversarial generations within bigger models and generate sharper results.
Revised: We test different CLIP model architectures and show that ensembling different models for guidance can prevent adversarial generations within bigger models and generate sharper results.

Original: Furthermore, we implement an implicit voxel grid model to show how neural networks provide an additional layer of regularization, resulting in better geometrical structure and coherency of generated objects.
Revised: Furthermore, we implement an implicit voxel grid model to show how neural networks provide an additional layer of regularization, resulting in better geometrical structure and coherency of generated objects.

Original: Compared to prior work, we achieve more coherent results with higher memory efficiency and faster training speeds.
Revised: Compared to prior work, we achieve more coherent results with higher memory efficiency and faster training speeds.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Text to image generation has seen major recent advances with the release of DALLE (Ramesh et al., 2021) and diffusion models such as DALLE2 (Ramesh et al., 2022), CogView2 (Ding et al., 2022) and Latent Diffusion (Rombach et al., 2022).
Revised: Text to image generation has seen major recent advances with the release of DALLE (Ramesh et al., 2021) and diffusion models such as DALLE2 (Ramesh et al., 2022), CogView2 (Ding et al., 2022) and Latent Diffusion (Rombach et al., 2022).

Original: A natural next step is the task of generating 3D objects from text input.
Revised: A natural next step is the task of generating 3D objects from text input.

Original: However, supervised methods relying on paired image-text data are less suited to text to 3D generation as large-scale paired text and 3D datasets are less available.
Revised: However, supervised methods relying on paired image-text data are less suited to text to 3D generation as large-scale paired text and 3D datasets are less available.

Original: Thus, the regime of little to no 3D data training supervision is beneficial.
Revised: Thus, the regime of little to no 3D data training supervision is beneficial.

Original: While this might seem daunting, recent work in text to 3D generation showed promising results without using large-scale datasets by bridging the gap using guidance from pretrained vision-language models such as CLIP (Radford et al., 2021).
Revised: While this might seem daunting, recent work in text to 3D generation showed promising results without using large-scale datasets by bridging the gap using guidance from pretrained vision-language models such as CLIP (Radford et al., 2021).

Original: At the same time, advances in differentiable neural rendering and the development of NeRF (Mildenhall et al., 2020) now allow for direct optimization of a 3D representation to match input images.
Revised: At the same time, advances in differentiable neural rendering and the development of NeRF (Mildenhall et al., 2020) now allow for direct optimization of a 3D representation to match input images.

Original: Combining these approaches with CLIP guidance, we can generate 3D representations from text
Revised: Combining these approaches with CLIP guidance, we can generate 3D representations from text

Original: Supplementary Website: directly without paired text–3D data, by optimizing the similarity of the text to the rendered images.
Revised: Supplementary Website: directly without paired text–3D data, by optimizing the similarity of the text to the rendered images.

Original: Work that leverages CLIP for text to 3D generation can be grouped by the amount of 3D data required.
Revised: Work that leverages CLIP for text to 3D generation can be grouped by the amount of 3D data required.

Original: The first set of methods train a generative model on a 3D dataset, and then optimize a mapping network from text to the latent space of the generative model using CLIP guidance and differentiable rendering.
Revised: The first set of methods train a generative model on a 3D dataset, and then optimize a mapping network from text to the latent space of the generative model using CLIP guidance and differentiable rendering.

Original: The second set of methods utilizes no 3D or text supervision and has access to only the pretrained CLIP model.
Revised: The second set of methods utilizes no 3D or text supervision and has access to only the pretrained CLIP model.

Original: We refer to this latter regime as pure CLIP guidance .
Revised: We refer to this latter regime as pure CLIP guidance .

Original: Given the scarcity of text–3D pair datasets, we focus on this regime.
Revised: Given the scarcity of text–3D pair datasets, we focus on this regime.

Original: A prominent example of the pure CLIP guidance regime is Dream Fields (Jain et al., 2022) which uses Mip-NeRF (Barron et al., 2021) and CLIP to guide the 3D optimization process for every new input text prompt.
Revised: A prominent example of the pure CLIP guidance regime is Dream Fields (Jain et al., 2022) which uses Mip-NeRF (Barron et al., 2021) and CLIP to guide the 3D optimization process for every new input text prompt.

Original: Unfortunately, this approach requires significant computational resources and exhibits poor quality generation with low-density artifacts when using direct voxel grid optimization (see appendix of original paper).
Revised: Unfortunately, this approach requires significant computational resources and exhibits poor quality generation with low-density artifacts when using direct voxel grid optimization (see appendix of original paper).

Original: We also find that the quality of the results in Dream Fields is largely attributable to the LiT (Zhai et al., 2022) guidance model.
Revised: We also find that the quality of the results in Dream Fields is largely attributable to the LiT (Zhai et al., 2022) guidance model.

Original: When using the vanilla CLIP models as in our work, results are far worse.
Revised: When using the vanilla CLIP models as in our work, results are far worse.

Original: Optimizing the CLIP similarity is also prone to adversarial examples where generated images with high similarity according to CLIP have little perceived resemblance to the text description for a human (Liu et al., 2021).
Revised: Optimizing the CLIP similarity is also prone to adversarial examples where generated images with high similarity according to CLIP have little perceived resemblance to the text description for a human (Liu et al., 2021).

Original: Recent text to 3D methods use image-based augmentations as regularization to prevent these issues.
Revised: Recent text to 3D methods use image-based augmentations as regularization to prevent these issues.

Original: However, there has been no systematic study of which of these regularizations matters and how much.
Revised: However, there has been no systematic study of which of these regularizations matters and how much.

Original: In addition, there are several possible design choices for the NeRF and CLIP modules, including the use of explicit voxel grids without any neural networks vs implicit neural representations.
Revised: In addition, there are several possible design choices for the NeRF and CLIP modules, including the use of explicit voxel grids without any neural networks vs implicit neural representations.

Original: We systematically compare these and other factors that impact generation quality, and show that it is possible to generate highly detailed 3D representations with voxel grids alone.
Revised: We systematically compare these and other factors that impact generation quality, and show that it is possible to generate highly detailed 3D representations with voxel grids alone.

Original: Our main contributions are: 1) We conduct a systematic study of augmentations and their effect on text to 3D generation results with pure CLIP guidance; 2) We compare different CLIP backbones for guidance as well as model ensembles for finer 3D object detail; 3)
Revised: Our main contributions are: 1) We conduct a systematic study of augmentations and their effect on text to 3D generation results with pure CLIP guidance; 2) We compare different CLIP backbones for guidance as well as model ensembles for finer 3D object detail; 3)

Original: We compare the regularization effects on geometry of explicit vs implicit voxel grids; and 4) We demonstrate generation of highresolution grids using CLIP guidance only.
Revised: We compare the regularization effects on geometry of explicit vs implicit voxel grids; and 4) We demonstrate generation of highresolution grids using CLIP guidance only.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Text to Image.
Revised: Text to Image.

Original: Recent text to image generation work has shown impressive results, from autoregressive methods (Ramesh et al., 2021;
Revised: Recent text to image generation work has shown impressive results, from autoregressive methods (Ramesh et al., 2021;

Original: Ding et al., 2021; Yu et al., 2022) to diffusion methods (Nichol et al., 2021; Ramesh et al., 2022; Rombach et al., 2022; Saharia et al., 2022; Ding et al., 2022).
Revised: Ding et al., 2021; Yu et al., 2022) to diffusion methods (Nichol et al., 2021; Ramesh et al., 2022; Rombach et al., 2022; Saharia et al., 2022; Ding et al., 2022).

Original: However, these methods require significant computational resources, and data supervision which is harder to obtain in large quantities in the case of 3D objects and corresponding text prompts.
Revised: However, these methods require significant computational resources, and data supervision which is harder to obtain in large quantities in the case of 3D objects and corresponding text prompts.

Original: To alleviate this problem, several works use CLIP and a pretrained image generator for image generation or manipulation without explicit supervision of corresponding pairs.
Revised: To alleviate this problem, several works use CLIP and a pretrained image generator for image generation or manipulation without explicit supervision of corresponding pairs.

Original: VQGAN-CLIP (Crowson et al., 2022) passes a randomly initialized image through a pretrained VQGAN encoder to get the latent vector.
Revised: VQGAN-CLIP (Crowson et al., 2022) passes a randomly initialized image through a pretrained VQGAN encoder to get the latent vector.

Original: The latent vector is then fed through the decoder, and after applying several augmentations, CLIP similarity is calculated to use as a loss to optimize the latent vector.
Revised: The latent vector is then fed through the decoder, and after applying several augmentations, CLIP similarity is calculated to use as a loss to optimize the latent vector.

Original: Fuse Dream (Liu et al., 2021) shows that CLIP similarity scores are prone to adversarial attacks and that applying Diff
Revised: Fuse Dream (Liu et al., 2021) shows that CLIP similarity scores are prone to adversarial attacks and that applying Diff

Original: Augment (Zhao et al., 2020) results in more robust CLIP scores that can be used for optimization.
Revised: Augment (Zhao et al., 2020) results in more robust CLIP scores that can be used for optimization.

Original: Text to 3D with CLIP Guidance.
Revised: Text to 3D with CLIP Guidance.

Original: Initial work in text to 3D shape generation relied on paired 3D shape and text data for supervised training of joint 3D-text embedding spaces (Chen et al., 2018;
Revised: Initial work in text to 3D shape generation relied on paired 3D shape and text data for supervised training of joint 3D-text embedding spaces (Chen et al., 2018;

Original: Jahan et al., 2021; Liu et al., 2022).
Revised: Jahan et al., 2021; Liu et al., 2022).

Original: Large pretrained image-text embeddings and differentiable rendering led to recent work demonstrating that CLIP and NeRF enable 3D object generation (Sanghi et al., 2022; Jain et al., 2022), manipulation (Michel et al., 2022; Wang et al., 2022; Youwang et al., 2022), and even 3D human animation generation (Hong et al., 2022) without direct supervision of text and 3D corresponding pairs.
Revised: Large pretrained image-text embeddings and differentiable rendering led to recent work demonstrating that CLIP and NeRF enable 3D object generation (Sanghi et al., 2022; Jain et al., 2022), manipulation (Michel et al., 2022; Wang et al., 2022; Youwang et al., 2022), and even 3D human animation generation (Hong et al., 2022) without direct supervision of text and 3D corresponding pairs.

Original: Here we distinguish between two different levels of supervision in these works.
Revised: Here we distinguish between two different levels of supervision in these works.

Original: Although there are no corresponding text and 3D examples in the first category, a dataset of 3D objects is available.
Revised: Although there are no corresponding text and 3D examples in the first category, a dataset of 3D objects is available.

Original: CLIP-Forge (Sanghi et al., 2022) first trains an implicit (occupancy) autoencoder model on the ShapeNet dataset.
Revised: CLIP-Forge (Sanghi et al., 2022) first trains an implicit (occupancy) autoencoder model on the ShapeNet dataset.

Original: Then in a second stage, a normalizing flow model is trained with multi-view images and CLIP to project from the CLIP latent space onto the latent space of the autoencoder.
Revised: Then in a second stage, a normalizing flow model is trained with multi-view images and CLIP to project from the CLIP latent space onto the latent space of the autoencoder.

Original: CLIP-NeRF (Wang et al., 2022) uses a similar approach by training a conditional NeRF model and a mapping network to predict updates for the conditional codes according to the input text.
Revised: CLIP-NeRF (Wang et al., 2022) uses a similar approach by training a conditional NeRF model and a mapping network to predict updates for the conditional codes according to the input text.

Original: Text2Mesh (Michel et al., 2022) takes an input base mesh matching the semantic class of the input text and trains color and displacement prediction for the base mesh according to the text prompt using CLIP similarity as the primary loss.
Revised: Text2Mesh (Michel et al., 2022) takes an input base mesh matching the semantic class of the input text and trains color and displacement prediction for the base mesh according to the text prompt using CLIP similarity as the primary loss.

Original: CLIP-Mesh (Khalid et al., 2022) starts from a sphere mesh initialized with random normal and texture maps, and optimizes mesh vertices and maps through differentiable rendering and CLIP similarity between text and current object.
Revised: CLIP-Mesh (Khalid et al., 2022) starts from a sphere mesh initialized with random normal and texture maps, and optimizes mesh vertices and maps through differentiable rendering and CLIP similarity between text and current object.

Original: Dream Fields (Jain et al., 2022) utilizes a NeRF representation instead to generate views of the 3D object from a set of cameras and uses a similar optimization process.
Revised: Dream Fields (Jain et al., 2022) utilizes a NeRF representation instead to generate views of the 3D object from a set of cameras and uses a similar optimization process.

Original: As shown in FuseDream (Liu et al., 2021), CLIP is very prone to adversarial generations.
Revised: As shown in FuseDream (Liu et al., 2021), CLIP is very prone to adversarial generations.

Original: In the first class of methods using 3D dataset supervision, generations are constrained with a prior over 3D objects making the adversarial generation problem less severe.
Revised: In the first class of methods using 3D dataset supervision, generations are constrained with a prior over 3D objects making the adversarial generation problem less severe.

Original: Since no such datasets are present in the second regime, augmentation techniques are utilized to prevent such adversarial generations in the case of Dream Fields and CLIP-Mesh.
Revised: Since no such datasets are present in the second regime, augmentation techniques are utilized to prevent such adversarial generations in the case of Dream Fields and CLIP-Mesh.

Original: CLIP-Forge, CLIP-Mesh, and Dream Fields are text-to-3D generation methods, while CLIP-NeRF and Text2Mesh are text-to-3D manipulation methods.
Revised: CLIP-Forge, CLIP-Mesh, and Dream Fields are text-to-3D generation methods, while CLIP-NeRF and Text2Mesh are text-to-3D manipulation methods.

Original: Also,
Revised: Also,

Original: Text2Mesh, Dream Fields, and CLIP-Mesh require per-prompt training, while the other works do not require training/tuning for new text prompts during inference.
Revised: Text2Mesh, Dream Fields, and CLIP-Mesh require per-prompt training, while the other works do not require training/tuning for new text prompts during inference.

Original: Compared to previous work without 3D supervision, we systematically study different image augmentations, CLIP architectures, and ensembling of CLIP models, and the corresponding impact on generation results.
Revised: Compared to previous work without 3D supervision, we systematically study different image augmentations, CLIP architectures, and ensembling of CLIP models, and the corresponding impact on generation results.

Original: Hybrid Representation NeRFs.
Revised: Hybrid Representation NeRFs.

Original: In the original NeRF (Mildenhall et al., 2020), a coordinate-based
Revised: In the original NeRF (Mildenhall et al., 2020), a coordinate-based

Original: MLP continuously represents a 3D scene compared to traditional explicit representations such as voxel grids.
Revised: MLP continuously represents a 3D scene compared to traditional explicit representations such as voxel grids.

Original: However, slow and memory-intensive computation is required to render even a single image due to the number of forward passes required for sampled points along rays before plugging into volumetric rendering.
Revised: However, slow and memory-intensive computation is required to render even a single image due to the number of forward passes required for sampled points along rays before plugging into volumetric rendering.

Original: Other work (Liu et al., 2020; Yu et al., 2021; Garbin et al., 2021;
Revised: Other work (Liu et al., 2020; Yu et al., 2021; Garbin et al., 2021;

Original: Hedman et al., 2021; Reiser et al., 2021) have utilized a hybrid implicit-explicit representation to speed up rendering.
Revised: Hedman et al., 2021; Reiser et al., 2021) have utilized a hybrid implicit-explicit representation to speed up rendering.

Original: However, such methods still use coordinates and Fourier embeddings to represent the input.
Revised: However, such methods still use coordinates and Fourier embeddings to represent the input.

Original: More recent work propose learning the input features stored directly in voxel grid structures.
Revised: More recent work propose learning the input features stored directly in voxel grid structures.

Original: Due to learning better input representations, the MLPs used in these models can be smaller, and density values can be used to prune calculations for voxels on the grid.
Revised: Due to learning better input representations, the MLPs used in these models can be smaller, and density values can be used to prune calculations for voxels on the grid.

Original: DVGO (Sun et al., 2022) represents the scene as an explicit density grid and learns feature grid along with a view-dependent
Revised: DVGO (Sun et al., 2022) represents the scene as an explicit density grid and learns feature grid along with a view-dependent

Original: MLP to model color.
Revised: MLP to model color.

Original: TensoRF (Chen et al., 2022) factorizes the voxel grids as vectors and matrices, allowing for a smaller memory footprint.
Revised: TensoRF (Chen et al., 2022) factorizes the voxel grids as vectors and matrices, allowing for a smaller memory footprint.

Original: InstantNGP (M ¨ uller et al., 2022) utilizes multiresolution hierarchies of hash tables to represent the learned feature grids and another MLP to predict density and color.
Revised: InstantNGP (M ¨ uller et al., 2022) utilizes multiresolution hierarchies of hash tables to represent the learned feature grids and another MLP to predict density and color.

Original: Plenoxels (Fridovich-Keil et al., 2022) learns a sparse voxel grid of density values and spherical harmonic coefficients for color.
Revised: Plenoxels (Fridovich-Keil et al., 2022) learns a sparse voxel grid of density values and spherical harmonic coefficients for color.

Original: ReLU Fields (Karnewar et al., 2022) also utilizes explicit density grids with spherical harmonics for color and shows that using ReLU activation after interpolating query point in the grid result in competitive results.
Revised: ReLU Fields (Karnewar et al., 2022) also utilizes explicit density grids with spherical harmonics for color and shows that using ReLU activation after interpolating query point in the grid result in competitive results.

Original: Our work builds on DVGO.
Revised: Our work builds on DVGO.

Original: In the explicit voxel grid model, the density and color grids are optimized directly with no neural networks.
Revised: In the explicit voxel grid model, the density and color grids are optimized directly with no neural networks.

Original: This is akin to learning alpha and RGB values in an image directly in the 2D case.
Revised: This is akin to learning alpha and RGB values in an image directly in the 2D case.

Original: For the implicit version, we use neural networks to predict the density and color grids.
Revised: For the implicit version, we use neural networks to predict the density and color grids.

Original:  B ACKGROUND
Revised:  B ACKGROUND

Original: CLIP.
Revised: CLIP.

Original: Radford et al.
Revised: Radford et al.

Original: (2021) implement CLIP as an image encoder Enc I and text encoder Enc T trained using contrastive losses.
Revised: (2021) implement CLIP as an image encoder Enc I and text encoder Enc T trained using contrastive losses.

Original: The two encoders project image x I and text x T into vectors in an aligned latent space v I
Revised: The two encoders project image x I and text x T into vectors in an aligned latent space v I

Original: =
Revised: =

Original: Enc I ( x I ) ∈ R n z × 1 , v T
Revised: Enc I ( x I ) ∈ R n z × 1 , v T

Original: =
Revised: =

Original: Enc T ( x T ) ∈
Revised: Enc T ( x T ) ∈

Original: R n z × 1 .
Revised: R n z × 1 .

Original: Since the latent space is aligned between two modalities, the similarity between the image and text is given by the cosine similarity v I · v T / ∥ v
Revised: Since the latent space is aligned between two modalities, the similarity between the image and text is given by the cosine similarity v I · v T / ∥ v

Original: I ∥∥ v T ∥ .
Revised: I ∥∥ v T ∥ .

Original: The model is pretrained on a large-scale dataset collected by OpenAI consisting of more than 400 million image text pairs.
Revised: The model is pretrained on a large-scale dataset collected by OpenAI consisting of more than 400 million image text pairs.

Original: We refer to different pretrained CLIP models by the architecture of the image encoder such as “ViT-B/32”, “ViT-B/16” etc.
Revised: We refer to different pretrained CLIP models by the architecture of the image encoder such as “ViT-B/32”, “ViT-B/16” etc.

Original: Dream Fields.
Revised: Dream Fields.

Original: Jain et al.
Revised: Jain et al.

Original: (2022) introduced the use of pure CLIP guidance for 3D object generation.
Revised: (2022) introduced the use of pure CLIP guidance for 3D object generation.

Original: For the backbone 3D representation, they use an implicit NeRF representation, specifically
Revised: For the backbone 3D representation, they use an implicit NeRF representation, specifically

Original: Mip-NeRF(Barron et al., 2021) which uses conical frustums over sampled points to calculate an integrated positional encoding (IPE) as opposed to the positional encoding (PE) in the original NeRF.
Revised: Mip-NeRF(Barron et al., 2021) which uses conical frustums over sampled points to calculate an integrated positional encoding (IPE) as opposed to the positional encoding (PE) in the original NeRF.

Original: CLIP similarity between the rendered image and input text prompt is used to facilitate training.
Revised: CLIP similarity between the rendered image and input text prompt is used to facilitate training.

Original: They use a background augmentation scheme to prevent adversarial generations.
Revised: They use a background augmentation scheme to prevent adversarial generations.

Original: Direct Voxel Grid Optimization.
Revised: Direct Voxel Grid Optimization.

Original: In DVGO (Sun et al., 2022), voxel grids are used to represent the density and color in NeRF.
Revised: In DVGO (Sun et al., 2022), voxel grids are used to represent the density and color in NeRF.

Original: The density grid V ( density )
Revised: The density grid V ( density )

Original: ∈ R 1 × N x × N y × N z is modeled explicitly.
Revised: ∈ R 1 × N x × N y × N z is modeled explicitly.

Original: The color grid is either modeled explicitly as V ( rgb ) ∈ R 3 × N x × N y × N z , or with a learnable feature grid
Revised: The color grid is either modeled explicitly as V ( rgb ) ∈ R 3 × N x × N y × N z , or with a learnable feature grid

Original: [Figure]
Revised: [Figure]

Original: 𝑧𝑧
Revised: 𝑧𝑧

Original: [Figure]
Revised: [Figure]

Original: Raw Density
Revised: Raw Density

Original: Raw Color
Revised: Raw Color

Original: Explicit Voxel Grid traditional japanesebuildings; trending on artstation.
Revised: Explicit Voxel Grid traditional japanesebuildings; trending on artstation.

Original: Activation 𝐿𝐿 𝐾𝐾𝐾𝐾 𝐿𝐿 𝑇𝑇𝑇𝑇 𝐿𝐿 𝑇𝑇𝑇𝑇
Revised: Activation 𝐿𝐿 𝐾𝐾𝐾𝐾 𝐿𝐿 𝑇𝑇𝑇𝑇 𝐿𝐿 𝑇𝑇𝑇𝑇

Original: CLIPText
Revised: CLIPText

Original: Encoder
Revised: Encoder

Original: CLIPImageEncoder 𝐿𝐿 𝐶𝐶𝐾𝐾𝐶𝐶𝐶𝐶 𝐿𝐿 𝑇𝑇𝑇𝑇
Revised: CLIPImageEncoder 𝐿𝐿 𝐶𝐶𝐾𝐾𝐶𝐶𝐶𝐶 𝐿𝐿 𝑇𝑇𝑇𝑇

Original: Diff
Revised: Diff

Original: Back
Revised: Back

Original: Perspective
Revised: Perspective

Original: V
Revised: V

Original: [Equation]
Revised: [Equation]

Original: ∈ R D × N x × N y × N z and shallow MLP to predict color for query points implicitly where D is the feature length.
Revised: ∈ R D × N x × N y × N z and shallow MLP to predict color for query points implicitly where D is the feature length.

Original: The original DVGO had two stages: a stage to the in the grid, and a fine stage.
Revised: The original DVGO had two stages: a stage to the in the grid, and a fine stage.

Original: There is no ground surface in our task, so we do not require a coarse stage.
Revised: There is no ground surface in our task, so we do not require a coarse stage.

Original: Volumetric rendering follows the quadrature rule (Max, 1995).
Revised: Volumetric rendering follows the quadrature rule (Max, 1995).

Original: Concretely, Eq
Revised: Concretely, Eq

Original: 1 is used to render the
Revised: 1 is used to render the

Original: C (
Revised: C (

Original: ) for a ray where δ is the between adjacent
Revised: ) for a ray where δ is the between adjacent

Original: , α i is the compositing value, is the number of queried points along the ray, and is the
Revised: , α i is the compositing value, is the number of queried points along the ray, and is the

Original: ˆ C ( r ) =
Revised: ˆ C ( r ) =

Original: K (cid:88) i =
Revised: K (cid:88) i =

Original: Tr i α
Revised: Tr i α

Original: i c i , where α i = alpha( σ i , δ i )
Revised: i c i , where α i = alpha( σ i , δ i )

Original: = 1 − exp( − σ i δ i ) , and Tr i = i − 1 (cid:89) j =
Revised: = 1 − exp( − σ i δ i ) , and Tr i = i − 1 (cid:89) j =

Original: [Equation]
Revised: [Equation]

Original: The density σ and color c are interpolated from voxel grid values at the sample point coordinate and passed through a shallow MLP for implicit feature grids.
Revised: The density σ and color c are interpolated from voxel grid values at the sample point coordinate and passed through a shallow MLP for implicit feature grids.

Original: The sigmoid function is applied to the raw color value.
Revised: The sigmoid function is applied to the raw color value.

Original: For the density, a post-activation scheme is used with the softplus function on the raw density, α ( post ) =
Revised: For the density, a post-activation scheme is used with the softplus function on the raw density, α ( post ) =

Original: alpha(softplus(interp( x, V ( density ) )) , δ ) .
Revised: alpha(softplus(interp( x, V ( density ) )) , δ ) .

Original:  M ETHOD
Revised:  M ETHOD

Original: Model architecture.
Revised: Model architecture.

Original: For our NeRF model, we implement two variations of the voxel grid representation: Vox Exp , an explicit voxel grid representation, and Vox Imp , an implicit version that uses
Revised: For our NeRF model, we implement two variations of the voxel grid representation: Vox Exp , an explicit voxel grid representation, and Vox Imp , an implicit version that uses

Original: MLPs to predict the density and color.
Revised: MLPs to predict the density and color.

Original: Vox Exp explicitly models the two voxel grids consisting of the density V ( density ) ∈ R 1 × N x × N y × N z and color V
Revised: Vox Exp explicitly models the two voxel grids consisting of the density V ( density ) ∈ R 1 × N x × N y × N z and color V

Original: ( rgb ) ∈ R 3 × N x × N y × N z .
Revised: ( rgb ) ∈ R 3 × N x × N y × N z .

Original: Vox Imp is an implicit coordinate-based MLP voxel grid representation with positional encodings (Mildenhall et al., 2020) of the grid vertex coordinates formulated as V ( PE ) ∈
Revised: Vox Imp is an implicit coordinate-based MLP voxel grid representation with positional encodings (Mildenhall et al., 2020) of the grid vertex coordinates formulated as V ( PE ) ∈

Original: R L × N x × N y × N z , where L is the channel size after positional encoding of grid vertex coordinates.
Revised: R L × N x × N y × N z , where L is the channel size after positional encoding of grid vertex coordinates.

Original: Separate density and color MLPs are applied on the positional encodings to obtain the density and color predictions.
Revised: Separate density and color MLPs are applied on the positional encodings to obtain the density and color predictions.

Original: We base our voxel grid model implementations on DVGO.
Revised: We base our voxel grid model implementations on DVGO.

Original: However, note that our positional encoding feature grid V ( PE ) is fixed and not learnable like the DVGO feature grid V ( feat ) for color.
Revised: However, note that our positional encoding feature grid V ( PE ) is fixed and not learnable like the DVGO feature grid V ( feat ) for color.

Original: Our overall model illustration can be found in Fig.
Revised: Our overall model illustration can be found in Fig.

Original: The trainable parameters for the explicit model are the parameters of the explicit grids and the bias term in the softplus activation of the density value.
Revised: The trainable parameters for the explicit model are the parameters of the explicit grids and the bias term in the softplus activation of the density value.

Original: For the implicit voxel grid the trainable parameters are the density and color MLPs and the bias term in the softplus function.
Revised: For the implicit voxel grid the trainable parameters are the density and color MLPs and the bias term in the softplus function.

Original: During training we also add progressive scaling of the voxel grid resolution as in DVGO.
Revised: During training we also add progressive scaling of the voxel grid resolution as in DVGO.

Original: Augmentations.
Revised: Augmentations.

Original: We study the impact of combining three augmentation schemes: Background augmentation (BackAug) from Dream Fields Jain et al.
Revised: We study the impact of combining three augmentation schemes: Background augmentation (BackAug) from Dream Fields Jain et al.

Original: (2022), Diff augment (DiffAug)
Revised: (2022), Diff augment (DiffAug)

Original: Zhao et al.
Revised: Zhao et al.

Original: (2020), and perspective augmentations (PerspAug) from Text2Mesh Michel et al.
Revised: (2020), and perspective augmentations (PerspAug) from Text2Mesh Michel et al.

Original: BackAug consists of alpha compositing checkerboard, textures or gaussian noise backgrounds to the image.
Revised: BackAug consists of alpha compositing checkerboard, textures or gaussian noise backgrounds to the image.

Original: DiffAug contains several image augmentations, including color jittering, image translation, and cutout.
Revised: DiffAug contains several image augmentations, including color jittering, image translation, and cutout.

Original: Liu et al.
Revised: Liu et al.

Original: (2021) used it for text to image generation and showed it prevents adversarial generations.
Revised: (2021) used it for text to image generation and showed it prevents adversarial generations.

Original: PerspAug denotes random perspective transformations applied to the image.
Revised: PerspAug denotes random perspective transformations applied to the image.

Original: Losses.
Revised: Losses.

Original: We combine the CLIP and transmittance losses introduced in Dream Fields, with losses from DVGO to reduce noise and promote smoothness.
Revised: We combine the CLIP and transmittance losses introduced in Dream Fields, with losses from DVGO to reduce noise and promote smoothness.

Original: In addition, we introduce a spherical prior loss term to encourage a coherent object.
Revised: In addition, we introduce a spherical prior loss term to encourage a coherent object.

Original: For a model parameterized by θ , the CLIP loss (Eq. 2) enforces the cosine similarity between the NeRF generated image
Revised: For a model parameterized by θ , the CLIP loss (Eq. 2) enforces the cosine similarity between the NeRF generated image

Original: I ( θ, p ) for a camera pose p and the input caption x T to be high in CLIP space.
Revised: I ( θ, p ) for a camera pose p and the input caption x T to be high in CLIP space.

Original: The transmittance loss (Eq. 3) prevents the scene from being overcrowded by applying a loss when the average transmittance is over the threshold τ and Tr( θ, p ) is the transmittance image.
Revised: The transmittance loss (Eq. 3) prevents the scene from being overcrowded by applying a loss when the average transmittance is over the threshold τ and Tr( θ, p ) is the transmittance image.

Original: [Equation]
Revised: [Equation]

Original: To encourage centered objects and uniform size, we introduce a spherical prior (Eq. 4) where the probability is 1 for coordinates q within a sphere of radius 1 .
Revised: To encourage centered objects and uniform size, we introduce a spherical prior (Eq. 4) where the probability is 1 for coordinates q within a sphere of radius 1 .

Original: We calculate the KL divergence between the spherical prior and the density voxel grid with the sampled point coordinates q from grid vertices (Eq. 5).
Revised: We calculate the KL divergence between the spherical prior and the density voxel grid with the sampled point coordinates q from grid vertices (Eq. 5).

Original: This loss serves the same purpose as ray shifting in Dream Fields.
Revised: This loss serves the same purpose as ray shifting in Dream Fields.

Original: However, it is not trivial to shift and scale the voxel grid directly.
Revised: However, it is not trivial to shift and scale the voxel grid directly.

Original: Therefore, we promote centering and uniform size through this loss instead.
Revised: Therefore, we promote centering and uniform size through this loss instead.

Original: [Equation]
Revised: [Equation]

Original: In our model, we enable the ensembling of different CLIP models by adding a second similarity loss using a different CLIP model as shown in Eq.
Revised: In our model, we enable the ensembling of different CLIP models by adding a second similarity loss using a different CLIP model as shown in Eq.

Original: 6, where Enc I 

Revised: 6, where Enc I

Original: and Enc T
Revised: and Enc T

Original: [Equation]
Revised: [Equation]

Original: are the image and text encoders for the second CLIP model respectively.
Revised: are the image and text encoders for the second CLIP model respectively.

Original: [Equation]
Revised: [Equation]

Original: Following DVGO, we also add a total variation loss L TV to reduce noise and promote smoothness and a background entropy loss L σ to encourage density values be either 0 or 1 .
Revised: Following DVGO, we also add a total variation loss L TV to reduce noise and promote smoothness and a background entropy loss L σ to encourage density values be either 0 or 1 .

Original: Our complete loss is shown in Eq.
Revised: Our complete loss is shown in Eq.

Original: 7, where the λ s are the weights for the different loss terms.
Revised: 7, where the λ s are the weights for the different loss terms.

Original: [Equation]
Revised: [Equation]

Original: We find that during optimization, it is important to schedule the L TV and L KL s loss terms so that they are turned off toward the end of the optimization.
Revised: We find that during optimization, it is important to schedule the L TV and L KL s loss terms so that they are turned off toward the end of the optimization.

Original: Please see Appendix A.1 for more discussion and details of other hyperparameter values.
Revised: Please see Appendix A.1 for more discussion and details of other hyperparameter values.

Original:  E XPERIMENTS 5. I MPLEMENTATION
Revised:  E XPERIMENTS 5. I MPLEMENTATION

Original: We sample camera poses following Dream Fields with a fixed elevation of 30 ◦ , azimuth of 360 ◦ around the scene, and a radius of 4 between the camera and origin.
Revised: We sample camera poses following Dream Fields with a fixed elevation of 30 ◦ , azimuth of 360 ◦ around the scene, and a radius of 4 between the camera and origin.

Original: For the augmentations, unless otherwise specified, we turn on all three of them.
Revised: For the augmentations, unless otherwise specified, we turn on all three of them.

Original: For DiffAug, we use the same settings as in the original implementation 1 .
Revised: For DiffAug, we use the same settings as in the original implementation 1 .

Original: For BackAug, we use the same JAX implementation from Dream
Revised: For BackAug, we use the same JAX implementation from Dream

Original: Fields 2 .
Revised: Fields 2 .

Original: For PerspAug, we set the distortion to 0 .
Revised: For PerspAug, we set the distortion to 0 .

Original: 6 with a probability of being applied to 1 . 0 .
Revised: 6 with a probability of being applied to 1 . 0 .

Original: The augmentations are applied in the order of (1) DiffAug, (2) BackAug, and (3) PerspAug.
Revised: The augmentations are applied in the order of (1) DiffAug, (2) BackAug, and (3) PerspAug.

Original: For most experiments, eight variations are applied for each image with both DiffAug and BackAug and only one for PerspAug, resulting in 64 images per optimization iteration.
Revised: For most experiments, eight variations are applied for each image with both DiffAug and BackAug and only one for PerspAug, resulting in 64 images per optimization iteration.

Original: For larger models trained with
Revised: For larger models trained with

Original: CLIP ViT-L/14 and Vox Imp , only four variations of DiffAug is applied for a given image resulting in 32 images after augmentation.
Revised: CLIP ViT-L/14 and Vox Imp , only four variations of DiffAug is applied for a given image resulting in 32 images after augmentation.

Original: Training is conducted with image resolution 224 2 during training using the Vox Exp model with the CLIP ViT-B/32 model for 15 k iterations unless otherwise specified.
Revised: Training is conducted with image resolution 224 2 during training using the Vox Exp model with the CLIP ViT-B/32 model for 15 k iterations unless otherwise specified.

Original: The MLPs used in our implicit voxel grid are 3 layers deep with hidden layer dimensions of 128 .
Revised: The MLPs used in our implicit voxel grid are 3 layers deep with hidden layer dimensions of 128 .

Original: Diff:  Back:  Perspective:  KL:  Diff:  Back:  Perspective:  KL: 
Revised: Diff:  Back:  Perspective:  KL:  Diff:  Back:  Perspective:  KL: 

Original: Diff: 
Revised: Diff: 

Original: Back:  Perspective:  KL: 
Revised: Back:  Perspective:  KL: 

Original: Diff: 
Revised: Diff: 

Original: Back:  Perspective  KL:
Revised: Back:  Perspective  KL:

Original:  Diff:  Back:  Perspective:  KL: 
Revised:  Diff:  Back:  Perspective:  KL: 

Original: Diff: 
Revised: Diff: 

Original: Back:  Perspective:  KL: 
Revised: Back:  Perspective:  KL: 

Original: Diff: 
Revised: Diff: 

Original: Back:  Perspective  KL:  "Bouquet of flowers sitting in a clear glass vase.
Revised: Back:  Perspective  KL:  "Bouquet of flowers sitting in a clear glass vase.

Original: " Diff:  Back:  Perspective:  KL: 
Revised: " Diff:  Back:  Perspective:  KL: 

Original: [Figure]
Revised: [Figure]

Original: "A pork dish with onions and peppers on a white plate."
Revised: "A pork dish with onions and peppers on a white plate."

Original: [Figure]
Revised: [Figure]

Original: "A dish with and
Revised: "A dish with and

Original: "Apples on tree ready to pick in garden area."
Revised: "Apples on tree ready to pick in garden area."

Original: [Figure]
Revised: [Figure]

Original: 
Revised: 

Original: [Figure]
Revised: [Figure]

Original: 5. A BLATION
Revised: 5. A BLATION

Original: We compare the effect of different augmentations and the KL loss term on Vox Exp .
Revised: We compare the effect of different augmentations and the KL loss term on Vox Exp .

Original: In Appendix A.3, we show the augmentation ablation results on Vox Imp .
Revised: In Appendix A.3, we show the augmentation ablation results on Vox Imp .

Original: Prior KL Loss Ablation.
Revised: Prior KL Loss Ablation.

Original: We show the effect of the L KL s loss term on the generation results in Fig.3.
Revised: We show the effect of the L KL s loss term on the generation results in Fig.3.

Original: Blank images indicate that the densities in the entire scene are near zero.
Revised: Blank images indicate that the densities in the entire scene are near zero.

Original: When all augmentations and the KL loss is turned off (first column) the generated scene contains floating clouds of noncoherent artifacts.
Revised: When all augmentations and the KL loss is turned off (first column) the generated scene contains floating clouds of noncoherent artifacts.

Original: If augmentations are turned on without the KL loss (2nd, 3rd columns), the voxel grid will fail to produce any densities.
Revised: If augmentations are turned on without the KL loss (2nd, 3rd columns), the voxel grid will fail to produce any densities.

Original: We hypothesize this is because the CLIP similarity loss landscape is more discontinuous when we add augmentations, making it hard for densities to emerge since there are no gradients in any direction.
Revised: We hypothesize this is because the CLIP similarity loss landscape is more discontinuous when we add augmentations, making it hard for densities to emerge since there are no gradients in any direction.

Original: The gradients are near zero for these cases shown in the plot below the images.
Revised: The gradients are near zero for these cases shown in the plot below the images.

Original: When the KL loss is turned on (last 3 columns) the densities are more concentrated and doesn’t disappear even when all augmentations are turned on.
Revised: When the KL loss is turned on (last 3 columns) the densities are more concentrated and doesn’t disappear even when all augmentations are turned on.

Original: This could mean that the KL loss term helps to force densities emerge in the beginning allowing for meaningful gradients.
Revised: This could mean that the KL loss term helps to force densities emerge in the beginning allowing for meaningful gradients.

Original: Therefore, in addition to regularizing the size and centering of the object generated, the
Revised: Therefore, in addition to regularizing the size and centering of the object generated, the

Original: KL loss term also plays a vital role in generating coherent results.
Revised: KL loss term also plays a vital role in generating coherent results.

Original: Augmentation Ablation.
Revised: Augmentation Ablation.

Original: We show the effect of the augmentations: (1) DiffAug, (2) BackAug, and
Revised: We show the effect of the augmentations: (1) DiffAug, (2) BackAug, and

Original: PerspAug and their combinations in Fig.
Revised: PerspAug and their combinations in Fig.

Original: When all three augmentations are turned off (first col-
Revised: When all three augmentations are turned off (first col-

Original: [Figure]
Revised: [Figure]

Original: RN50x
Revised: RN50x

Original: ViT-B/
Revised: ViT-B/

Original: ViT-B/ "A pizza that is covered in a lot of toppings."
Revised: ViT-B/ "A pizza that is covered in a lot of toppings."

Original: "matte painting of a bonsai tree; trending on artstation."
Revised: "matte painting of a bonsai tree; trending on artstation."

Original: ViT-L/
Revised: ViT-L/

Original: ViT-B/32 + ViT-L/14 ViT-B/ 16 + ViT-L/ "Bouquet of flowers sitting in a clear glass vase." umn), we can see the results are very noisy and incoherent for the text prompt.
Revised: ViT-B/32 + ViT-L/14 ViT-B/ 16 + ViT-L/ "Bouquet of flowers sitting in a clear glass vase." umn), we can see the results are very noisy and incoherent for the text prompt.

Original: With only DiffAug on (col 2), textures start appearing, but the densities are sparsely distributed like point clouds.
Revised: With only DiffAug on (col 2), textures start appearing, but the densities are sparsely distributed like point clouds.

Original: For results with only BackAug (col 3), the object is condensed and looks more like a contiguous object.
Revised: For results with only BackAug (col 3), the object is condensed and looks more like a contiguous object.

Original: When both are turned on (col 4), the objects more clearly match the input prompt.
Revised: When both are turned on (col 4), the objects more clearly match the input prompt.

Original: Turning on PerspAug (col 5) helps to get rid of unwanted background textures in the apple tree prompt.
Revised: Turning on PerspAug (col 5) helps to get rid of unwanted background textures in the apple tree prompt.

Original: However, it may create an overly smooth image or almost cartoonish effect in the plate of food.
Revised: However, it may create an overly smooth image or almost cartoonish effect in the plate of food.

Original: While we observe that turning off PerspAug has better qualitative results for many prompts, it is infeasible to perform per-prompt tuning, so we leave PerspAug on for following experiments.
Revised: While we observe that turning off PerspAug has better qualitative results for many prompts, it is infeasible to perform per-prompt tuning, so we leave PerspAug on for following experiments.

Original: For the Vox Imp model, we observe that the model can produce clean results without background textures even without PerspAug (see Appendix A.3).
Revised: For the Vox Imp model, we observe that the model can produce clean results without background textures even without PerspAug (see Appendix A.3).

Original: 5. D IFFERENT CLIP M ODEL E XPERIMENTS
Revised: 5. D IFFERENT CLIP M ODEL E XPERIMENTS

Original: Backbone Networks.
Revised: Backbone Networks.

Original: We compare generation results from 5 pretrained CLIP models with different backbone networks in Fig.
Revised: We compare generation results from 5 pretrained CLIP models with different backbone networks in Fig.

Original: Overall the ResNet (He et al., 2016) models (ResNet50, RN50x4) are very noisy, but some features within the generations vaguely resemble the text prompt.
Revised: Overall the ResNet (He et al., 2016) models (ResNet50, RN50x4) are very noisy, but some features within the generations vaguely resemble the text prompt.

Original: It seems that these models have a hard time separating the main subject of the text prompts from other textures.
Revised: It seems that these models have a hard time separating the main subject of the text prompts from other textures.

Original: For ViT-B/32 and ViT-B/16 the generations are more coherent and have good separation from unwanted textures compared to the ResNet models.
Revised: For ViT-B/32 and ViT-B/16 the generations are more coherent and have good separation from unwanted textures compared to the ResNet models.

Original: It can also be seen that ViT-B/16 can generate more details, as seen in the bonsai plant and pizza .
Revised: It can also be seen that ViT-B/16 can generate more details, as seen in the bonsai plant and pizza .

Original: This has also been shown in the Dream Fields paper.
Revised: This has also been shown in the Dream Fields paper.

Original: However, when we use the biggest model, ViT-L/14, the results are very noisy and non-coherent.
Revised: However, when we use the biggest model, ViT-L/14, the results are very noisy and non-coherent.

Original: We hypothesize that as the models increase to a certain point, it becomes easier to find adversarial images within the CLIP space.
Revised: We hypothesize that as the models increase to a certain point, it becomes easier to find adversarial images within the CLIP space.

Original: This could suggest that ViT-L/14 is overfit on the texture features for images, not the objects’ overall structure.
Revised: This could suggest that ViT-L/14 is overfit on the texture features for images, not the objects’ overall structure.

Original: Discounting the ViT-L/14 model, vision transformers (Dosovitskiy et al., 2021) show much better results than traditional convolutional architectures.
Revised: Discounting the ViT-L/14 model, vision transformers (Dosovitskiy et al., 2021) show much better results than traditional convolutional architectures.

Original: We attribute this to the tokenization of image patches, which discretizes the image into regions and may help to disentangle the overall structure and textures of the image.
Revised: We attribute this to the tokenization of image patches, which discretizes the image into regions and may help to disentangle the overall structure and textures of the image.

Original: CLIP Model Ensemble.
Revised: CLIP Model Ensemble.

Original: Fig.
Revised: Fig.

Original: 5 (last two columns) shows the results of different CLIP ensembles, namely ViT-B/32+ViT-L/14 and ViT-B/16+ViT-L/14.
Revised: 5 (last two columns) shows the results of different CLIP ensembles, namely ViT-B/32+ViT-L/14 and ViT-B/16+ViT-L/14.

Original: We see that for both ensemble models, we can generate much finer details than using one guidance model alone (see example for the flowers and bonsai plant ).
Revised: We see that for both ensemble models, we can generate much finer details than using one guidance model alone (see example for the flowers and bonsai plant ).

Original: This suggests that using a smaller CLIP model to initialize the generation process helps prevent the bigger model ViT-L/14 from adversarial generations and may even help to generate
Revised: This suggests that using a smaller CLIP model to initialize the generation process helps prevent the bigger model ViT-L/14 from adversarial generations and may even help to generate

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: more fine grain details.
Revised: more fine grain details.

Original: This is likely due to the bigger model size of ViT-L/14 as well as the smaller patch sizes it takes in as tokens.
Revised: This is likely due to the bigger model size of ViT-L/14 as well as the smaller patch sizes it takes in as tokens.

Original: However, we also observe that some generation results are worse, such as the pizza having fewer details than just using ViT-B/32 or ViT-B/16.
Revised: However, we also observe that some generation results are worse, such as the pizza having fewer details than just using ViT-B/32 or ViT-B/16.

Original: We attribute this to the two CLIP models having different maximums for the pizza , which may cause conflicting similarity losses and generate less sharp objects.
Revised: We attribute this to the two CLIP models having different maximums for the pizza , which may cause conflicting similarity losses and generate less sharp objects.

Original: Explicit vs Implicit Voxel Grids vs Dream Fields.
Revised: Explicit vs Implicit Voxel Grids vs Dream Fields.

Original: We show the results of Vox Exp , Vox Imp and
Revised: We show the results of Vox Exp , Vox Imp and

Original: Dream Fields in Fig.
Revised: Dream Fields in Fig.

Original: All three models were trained with image resolution 168 2 and 10 k iterations using the CLIP ViT-B/16 model.
Revised: All three models were trained with image resolution 168 2 and 10 k iterations using the CLIP ViT-B/16 model.

Original: Note that we turn off PerspAug for Vox Imp as it is unnecessary for the implicit model (see Appendix A.3).
Revised: Note that we turn off PerspAug for Vox Imp as it is unnecessary for the implicit model (see Appendix A.3).

Original: The Vox Exp model is able to generate textures that corresponds with the text prompt.
Revised: The Vox Exp model is able to generate textures that corresponds with the text prompt.

Original: However, the geometry does not conform with objects in the real world.
Revised: However, the geometry does not conform with objects in the real world.

Original: The Vox Imp model in comparison produces objects with better geometrical structure.
Revised: The Vox Imp model in comparison produces objects with better geometrical structure.

Original: We see this with the honey jar , where the Vox Imp model generates a cohesive 3D object whereas we see an incomplete jar in the Vox Exp model.
Revised: We see this with the honey jar , where the Vox Imp model generates a cohesive 3D object whereas we see an incomplete jar in the Vox Exp model.

Original: The coherency for Vox Imp is also better with respect to the text prompt.
Revised: The coherency for Vox Imp is also better with respect to the text prompt.

Original: For example, in the text prompt involving a bag of trash and a park bench , the Vox Imp model successfully generates both items whereas there is only a trash bag for Vox Exp .
Revised: For example, in the text prompt involving a bag of trash and a park bench , the Vox Imp model successfully generates both items whereas there is only a trash bag for Vox Exp .

Original: However, we notice that for complex objects like animals ( cat , zebra ) all models fail at producing correct geometry.
Revised: However, we notice that for complex objects like animals ( cat , zebra ) all models fail at producing correct geometry.

Original: This is likely due to the limitation of the guidance model used.
Revised: This is likely due to the limitation of the guidance model used.

Original: This failure mode was described as “repeated patterns on multiple sides of the object” by Dream Fields.
Revised: This failure mode was described as “repeated patterns on multiple sides of the object” by Dream Fields.

Original: Overall this is still a very hard task and we find several prompts where the models failed to generate cohesive results as can be seen in the example with the orange bike and aircraft door .
Revised: Overall this is still a very hard task and we find several prompts where the models failed to generate cohesive results as can be seen in the example with the orange bike and aircraft door .

Original: Dream Fields had poor results overall and many generations are unrecognizable with respect to the text prompt.
Revised: Dream Fields had poor results overall and many generations are unrecognizable with respect to the text prompt.

Original: This is in contrast to results shown in their paper using the LiT ViT-B/32 (Zhai et al., 2022) model trained on billions of higher resolutions ( 288 2 ) captioned images.
Revised: This is in contrast to results shown in their paper using the LiT ViT-B/32 (Zhai et al., 2022) model trained on billions of higher resolutions ( 288 2 ) captioned images.

Original: For quantitative comparison, we follow Jain et al.
Revised: For quantitative comparison, we follow Jain et al.

Original: and report the R-Precision@1 retrieval scores for the three models in Tab. 1.
Revised: and report the R-Precision@1 retrieval scores for the three models in Tab. 1.

Original: We calculate R-Precision@1 retrieval scores for the text and images of the objects on a validation set of 153 text prompts used in the Dream Fields paper.
Revised: We calculate R-Precision@1 retrieval scores for the text and images of the objects on a validation set of 153 text prompts used in the Dream Fields paper.

Original: The images are rendered at held-out poses of elevation 45 ◦ following Dream Fields.
Revised: The images are rendered at held-out poses of elevation 45 ◦ following Dream Fields.

Original: We report the retrieval scores for both ViT-B/16, which the models were optimized on, and a different CLIP model,
Revised: We report the retrieval scores for both ViT-B/16, which the models were optimized on, and a different CLIP model,

Original: ViT-B/32.
Revised: ViT-B/32.

Original: All models are able to achieve high retrieval scores when evaluated on the CLIP model that they were trained on.
Revised: All models are able to achieve high retrieval scores when evaluated on the CLIP model that they were trained on.

Original: Overall, we see that the Vox Imp model achieves the best quantitative results.
Revised: Overall, we see that the Vox Imp model achieves the best quantitative results.

Original: The Vox Exp model outperforms Dream Fields considerably when evaluated with a different
Revised: The Vox Exp model outperforms Dream Fields considerably when evaluated with a different

Original: CLIP-model.
Revised: CLIP-model.

Original: We note that training at lower resolutions ( 168 2 ) result in quality degradation compared to models trained at higher resolutions in Fig.
Revised: We note that training at lower resolutions ( 168 2 ) result in quality degradation compared to models trained at higher resolutions in Fig.

Original: 1 and that training at higher resolutions ( 224 2 ) may result in even better results while still using less memory (see Appendix A.5) compared to
Revised: 1 and that training at higher resolutions ( 224 2 ) may result in even better results while still using less memory (see Appendix A.5) compared to

Original: Dream Fields.
Revised: Dream Fields.

Original:  D ISCUSSION
Revised:  D ISCUSSION

Original: 3D Representation.
Revised: 3D Representation.

Original: In this paper we mainly explored how pure CLIP guidance affects NeRF models.
Revised: In this paper we mainly explored how pure CLIP guidance affects NeRF models.

Original: However, it would be interesting to see if findings here also apply to methods based on explicit representations such as CLIP-Mesh (Khalid et al., 2022).
Revised: However, it would be interesting to see if findings here also apply to methods based on explicit representations such as CLIP-Mesh (Khalid et al., 2022).

Original: Since they start with a primitive mesh for optimization the possible topology and complexity of possible generations are limited compared to
Revised: Since they start with a primitive mesh for optimization the possible topology and complexity of possible generations are limited compared to

Original: NeRFs.
Revised: NeRFs.

Original: It would be worth exploring whether this helps to regularize CLIP guidance.
Revised: It would be worth exploring whether this helps to regularize CLIP guidance.

Original: Augmentations.
Revised: Augmentations.

Original: Augmentations are essential for preventing adversarial generations from CLIP guidance.
Revised: Augmentations are essential for preventing adversarial generations from CLIP guidance.

Original: We showed how different augmentations affect the generation results and that sometimes it can even be harmful to the coherency of the results.
Revised: We showed how different augmentations affect the generation results and that sometimes it can even be harmful to the coherency of the results.

Original: This was shown with perspective augmentation.
Revised: This was shown with perspective augmentation.

Original: It can help eliminate unwanted textures in the background for some prompts, but can also overly smooth the textures for others.
Revised: It can help eliminate unwanted textures in the background for some prompts, but can also overly smooth the textures for others.

Original: While prior work in this task has not put as much emphasis on testing different augmentations, we think it is important to explore more variations to find the best augmentations for different use cases.
Revised: While prior work in this task has not put as much emphasis on testing different augmentations, we think it is important to explore more variations to find the best augmentations for different use cases.

Original: Guidance Models.
Revised: Guidance Models.

Original: While we may try all sorts of augmentations to improve coherency, the main constricting factor lies within the guidance model such as CLIP.
Revised: While we may try all sorts of augmentations to improve coherency, the main constricting factor lies within the guidance model such as CLIP.

Original: If the guidance model is not able to learn the correct structure and geometry of objects, no augmentations will help bring those out in the results.
Revised: If the guidance model is not able to learn the correct structure and geometry of objects, no augmentations will help bring those out in the results.

Original: Therefore, it is important to also improve the guidance models.
Revised: Therefore, it is important to also improve the guidance models.

Original: We see that while Dream
Revised: We see that while Dream

Original: Fields had worse results when using vanilla CLIP models for guidance, the LiT (Zhai et al., 2022) model used in their paper had a huge impact on making their results coherent.
Revised: Fields had worse results when using vanilla CLIP models for guidance, the LiT (Zhai et al., 2022) model used in their paper had a huge impact on making their results coherent.

Original: This suggests the fine-tuning process in LiT can be helpful in learning better features for guidance.
Revised: This suggests the fine-tuning process in LiT can be helpful in learning better features for guidance.

Original: We also find using alternative CLIP models such as OpenCLIP Ilharco et al.
Revised: We also find using alternative CLIP models such as OpenCLIP Ilharco et al.

Original: (2021) can result in high quality generation (see Appendix A.4).
Revised: (2021) can result in high quality generation (see Appendix A.4).

Original: It may also be helpful to look into more recent text-image contrastive models such as SLIP (Mu et al., 2021) and UniCL (Yang et al., 2022).
Revised: It may also be helpful to look into more recent text-image contrastive models such as SLIP (Mu et al., 2021) and UniCL (Yang et al., 2022).

Original: Our model can serve as a proxy qualitative measure for text-image similarity models to see how robust the features learned are and whether the models have learned underlying structures for objects that can be used to construct geometry that conforms with the real world.
Revised: Our model can serve as a proxy qualitative measure for text-image similarity models to see how robust the features learned are and whether the models have learned underlying structures for objects that can be used to construct geometry that conforms with the real world.

Original: Dataset Supervision.
Revised: Dataset Supervision.

Original: Another avenue to help with adversarial generations is to have a 3D dataset to help provide a good geometric prior for generators.
Revised: Another avenue to help with adversarial generations is to have a 3D dataset to help provide a good geometric prior for generators.

Original: Many current text to 3D works leverages this to generate higher quality results, albeit at the cost of text prompt freedom.
Revised: Many current text to 3D works leverages this to generate higher quality results, albeit at the cost of text prompt freedom.

Original: Our work is orthogonal to theirs, and the techniques proposed in this paper can also be applied to their works.
Revised: Our work is orthogonal to theirs, and the techniques proposed in this paper can also be applied to their works.

Original: Resource Usage.
Revised: Resource Usage.

Original: Overall, our models take less memory and are faster to train than Dream Fields (see Appendix A.5 for details).
Revised: Overall, our models take less memory and are faster to train than Dream Fields (see Appendix A.5 for details).

Original: Our smallest model (Vox Exp with CLIP ViT-B/32) uses 7GB and can be optimized on a RTX 

Revised: Our smallest model (Vox Exp with CLIP ViT-B/32) uses 7GB and can be optimized on a RTX

Original: Ti in roughly 20 minutes.
Revised: Ti in roughly 20 minutes.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper we have showed that it is possible to train an explicit voxel grid NeRF model with pure
Revised: In this paper we have showed that it is possible to train an explicit voxel grid NeRF model with pure

Original: CLIP guidance to generate coherent results with a combination of image-based augmentations.
Revised: CLIP guidance to generate coherent results with a combination of image-based augmentations.

Original: We also provide empirical evaluations for how different augmentations as well as implicit voxel grids prevent adversarial examples within CLIP.
Revised: We also provide empirical evaluations for how different augmentations as well as implicit voxel grids prevent adversarial examples within CLIP.

Original: Our model performs better than prior work when using the same CLIP model for guidance.
Revised: Our model performs better than prior work when using the same CLIP model for guidance.

Original: It is also less compute and memory intensive while also being faster to train.
Revised: It is also less compute and memory intensive while also being faster to train.

Original:  E THICS S TATEMENT
Revised:  E THICS S TATEMENT

Original: We acknowledge there are potential risks with using the CLIP (Radford et al., 2021) and OpenCLIP (Ilharco et al., 2021) models for guidance pertaining to deceptive and discriminatory content.
Revised: We acknowledge there are potential risks with using the CLIP (Radford et al., 2021) and OpenCLIP (Ilharco et al., 2021) models for guidance pertaining to deceptive and discriminatory content.

Original: CLIP was trained by OpenAI with an in-house dataset of 400 million text-image pairs.
Revised: CLIP was trained by OpenAI with an in-house dataset of 400 million text-image pairs.

Original: While the dataset is not publicly released, models that leverage CLIP for learning have demonstrated biases for applications like text-to-image generation (Struppek et al., 2022).
Revised: While the dataset is not publicly released, models that leverage CLIP for learning have demonstrated biases for applications like text-to-image generation (Struppek et al., 2022).

Original: OpenCLIP was trained on the
Revised: OpenCLIP was trained on the

Original: LAION (Schuhmann et al., 2021) dataset which contains offensive and explicit images as shown by Birhane et al. (2021).
Revised: LAION (Schuhmann et al., 2021) dataset which contains offensive and explicit images as shown by Birhane et al. (2021).

Original: Our model inherits these biases as we use these models as priors to generate 3D objects.
Revised: Our model inherits these biases as we use these models as priors to generate 3D objects.

Original: On the one hand, text-to-3D models provide more accessible tools for the general public.
Revised: On the one hand, text-to-3D models provide more accessible tools for the general public.

Original: On the other hand, they can also be used to produce harmful content by malicious actors to spread deceptive and hateful information.
Revised: On the other hand, they can also be used to produce harmful content by malicious actors to spread deceptive and hateful information.

Original: In addition, other potential ethical concerns raised by text-to-image (Nichol et al., 2021;
Revised: In addition, other potential ethical concerns raised by text-to-image (Nichol et al., 2021;

Original: Ramesh et al., 2022; Rombach et al., 2022; Saharia et al., 2022) models also apply.
Revised: Ramesh et al., 2022; Rombach et al., 2022; Saharia et al., 2022) models also apply.

Original: More systematic studies are needed in the space of text-to-3D models as higher quality generations are becoming possible and the potential risks need to be assessed and safeguards put in place to prevent misuse.
Revised: More systematic studies are needed in the space of text-to-3D models as higher quality generations are becoming possible and the potential risks need to be assessed and safeguards put in place to prevent misuse.

Original: A A PPENDIX A. H YPERPARAMETERS AND S CHEDULING
Revised: A A PPENDIX A. H YPERPARAMETERS AND S CHEDULING

Original: Our density and color MLPs are both 3-layer MLPs.
Revised: Our density and color MLPs are both 3-layer MLPs.

Original: The input dimension for both are 63 which includes the raw coordinates and fourier features.
Revised: The input dimension for both are 63 which includes the raw coordinates and fourier features.

Original: The hidden feature size is 128 for all layers and the output size is 1 and 3 for density and color MLPs respectively.
Revised: The hidden feature size is 128 for all layers and the output size is 1 and 3 for density and color MLPs respectively.

Original: The loss weight λ s for each of the terms can be found in Tab. 2.
Revised: The loss weight λ s for each of the terms can be found in Tab. 2.

Original: λ CLIP 2 is only set for models trained with two CLIP guidance models, otherwise it is set to 0 .
Revised: λ CLIP 2 is only set for models trained with two CLIP guidance models, otherwise it is set to 0 .

Original: [Table]
Revised: [Table]

Original: The other important hyperparameters can be found in Tab. 3.
Revised: The other important hyperparameters can be found in Tab. 3.

Original: Here the number of voxels decides the resolution of both the density and color grids.
Revised: Here the number of voxels decides the resolution of both the density and color grids.

Original: The grids are rectangular shapes with the proportions decided by the camera frustums and each side has integer number of voxels scaled so that the volume roughly has the amount of voxels as shown in the table.
Revised: The grids are rectangular shapes with the proportions decided by the camera frustums and each side has integer number of voxels scaled so that the volume roughly has the amount of voxels as shown in the table.

Original: We also employ progressive scaling as in the original DVGO.
Revised: We also employ progressive scaling as in the original DVGO.

Original: For each scaling iteration the number of voxels is doubled with the starting number of voxels ⌊ N x × N y × N z 2 N pg ⌋ , where N pg is the number of progressive scaling iterations.
Revised: For each scaling iteration the number of voxels is doubled with the starting number of voxels ⌊ N x × N y × N z 2 N pg ⌋ , where N pg is the number of progressive scaling iterations.

Original: We find that it is important to schedule the L TV and L KL s so that they are only turned on for some iterations.
Revised: We find that it is important to schedule the L TV and L KL s so that they are only turned on for some iterations.

Original: The TV loss L TV is applied only during the iterations defined in the table and turned off otherwise.
Revised: The TV loss L TV is applied only during the iterations defined in the table and turned off otherwise.

Original: Since the density values are near zero in the beginning, we find that the TV loss that encourages smoothness will prevent any densities from appearing.
Revised: Since the density values are near zero in the beginning, we find that the TV loss that encourages smoothness will prevent any densities from appearing.

Original: Also TV loss will make the object too smooth so we also turn it off towards the end to help the formation of sharp textures.
Revised: Also TV loss will make the object too smooth so we also turn it off towards the end to help the formation of sharp textures.

Original: For the KL loss L KL , we turn it on only in the beginning iterations.
Revised: For the KL loss L KL , we turn it on only in the beginning iterations.

Original: This is because the KL loss term helps to make the densities emerge, but may also encourage noisy densities appear outside the object within the sphere prior.
Revised: This is because the KL loss term helps to make the densities emerge, but may also encourage noisy densities appear outside the object within the sphere prior.

Original: For the ‘CLIP ensemble iterations’, it is only applied to the second CLIP guidance model L CLIP 2 for ensembled guidance models.
Revised: For the ‘CLIP ensemble iterations’, it is only applied to the second CLIP guidance model L CLIP 2 for ensembled guidance models.

Original: [Table]
Revised: [Table]

Original: Other Hyperparameters
Revised: Other Hyperparameters

Original: [Table]
Revised: [Table]

Original: In Fig.
Revised: In Fig.

Original: 7 we illustrate the effects of both KL and TV losses.
Revised: 7 we illustrate the effects of both KL and TV losses.

Original: For KL loss we see a very clear circular hue around the generated object where the sphere prior locates.
Revised: For KL loss we see a very clear circular hue around the generated object where the sphere prior locates.

Original: We turn this off in the later iterations to eliminate some noise densities around the object.
Revised: We turn this off in the later iterations to eliminate some noise densities around the object.

Original: For TV loss when it is turned on it helps to smooth the object.
Revised: For TV loss when it is turned on it helps to smooth the object.

Original: Again we turn it off towards the end of training.
Revised: Again we turn it off towards the end of training.

Original: It can be seen that when it is turned off textures and edges appear more clearly to make the object look sharper.
Revised: It can be seen that when it is turned off textures and edges appear more clearly to make the object look sharper.

Original: A. A UGMENTATION , A RCHITECTURE R-P RECISION
Revised: A. A UGMENTATION , A RCHITECTURE R-P RECISION

Original: Augmentation Ablation
Revised: Augmentation Ablation

Original: The R-Precision for augmentation ablations can be found in Tab. 4.
Revised: The R-Precision for augmentation ablations can be found in Tab. 4.

Original: Here
Revised: Here

Original: Vox Exp is trained using the ViT-B/16 CLIP model.
Revised: Vox Exp is trained using the ViT-B/16 CLIP model.

Original: Note that we add a variant of ViT-B/32 that was
Revised: Note that we add a variant of ViT-B/32 that was

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: add DiffAug, the R-Precision value goes up significantly.
Revised: add DiffAug, the R-Precision value goes up significantly.

Original: This indicates the texture of the object plays a large role in the coherency of the object according to CLIP.
Revised: This indicates the texture of the object plays a large role in the coherency of the object according to CLIP.

Original: When just adding BackAug, the scores do not go up by much.
Revised: When just adding BackAug, the scores do not go up by much.

Original: However, when we add both DiffAug and BackAug the R-Precision scores are the best.
Revised: However, when we add both DiffAug and BackAug the R-Precision scores are the best.

Original: PerspAug decreases the R-Precision score when added, this is probably due to the reduction in texture sharpness we saw in Fig.
Revised: PerspAug decreases the R-Precision score when added, this is probably due to the reduction in texture sharpness we saw in Fig.

Original: However, qualitatively for some text prompts it helps to remove noise and unwanted background textures making the object have better geometric coherency.
Revised: However, qualitatively for some text prompts it helps to remove noise and unwanted background textures making the object have better geometric coherency.

Original: We notice that for many objects with broken geometry as long as the texture resembles the text, the R-Precision will still be high.
Revised: We notice that for many objects with broken geometry as long as the texture resembles the text, the R-Precision will still be high.

Original: This is a big limitation of CLIP R-Precision as it is not a good metric for whether or not the generated object has good geometric structure.
Revised: This is a big limitation of CLIP R-Precision as it is not a good metric for whether or not the generated object has good geometric structure.

Original: Architecture Ablation
Revised: Architecture Ablation

Original: The R-Precision for Vox Exp and Vox Imp trained with different CLIP architectures can be found in Tab.
Revised: The R-Precision for Vox Exp and Vox Imp trained with different CLIP architectures can be found in Tab.

Original: For Vox Exp all augmentations are turned on, for Vox Imp DiffAug and BackAug are turned on.
Revised: For Vox Exp all augmentations are turned on, for Vox Imp DiffAug and BackAug are turned on.

Original: We refer to the ViT-B/32(OpenCLIP) column to compare methods as the models are trained either using ViT-B/32 or ViT-B/16.
Revised: We refer to the ViT-B/32(OpenCLIP) column to compare methods as the models are trained either using ViT-B/32 or ViT-B/16.

Original: The Vox Exp model trained with ViT-B/
Revised: The Vox Exp model trained with ViT-B/

Original: Retrieval model R-Precision
Revised: Retrieval model R-Precision

Original: Method (CLIP)
Revised: Method (CLIP)

Revised: ViT

Original: [Table]
Revised: [Table]

Original: OpenCLIP)
Revised: (OpenCLIP)

Original: Vox
Revised: Vox

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Vox
Revised: Vox

Original: [Table]

Original: [Table]
Revised: [Table]

Original: Vox Exp (
Revised: Vox Exp (ViT

Original: [Table]
Revised: [Table]

Original: + 
Revised: + ViT

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: L/14) ( 71 . 90 ) 61 .61 .

Original: Vox Imp (
Revised: Vox Imp (ViT

Original: [Table]
Revised: [Table]

Original: Vox [Table] [Table]
Revised: Vox Imp (ViT

Original: [Table]
Revised: [Table]

Original: ( 98 . 04 ) 85 ."Apples on tree ready to pick in garden area."
Revised: ( 98 . 69 ) 85 ."Apples on tree ready to pick in garden area."

Original: "A pork dish with onions and peppers on a white plate."
Revised: "A pork dish with onions and peppers on a white plate."

Original: Diff: 
Revised: Diff: 

Original: Back: 
Revised: Back: 

Original: Perspective: 
Revised: Perspective: 

Original: Diff: 
Revised: Diff: 

Original: Back: 
Revised: Back: 

Original: Perspective: 
Revised: Perspective: 

Original: Diff: 
Revised: Diff: 

Original: Back: 
Revised: Back: 

Original: Perspective: 
Revised: Perspective: 

Original: Diff: 
Revised: Diff: 

Original: Back: 
Revised: Back: 

Original: Perspective: 
Revised: Perspective: 

Original: Diff: 
Revised: Diff: 

Original: Back: 
Revised: Back: 

Original: Perspective:  has the highest score for explicit models.
Revised: Perspective:  has the highest score for explicit models.

Original: We see that the ensemble model (ViT-B/32+ViT-L/14) has much lower scores compared to the other explicit models.
Revised: We see that the ensemble model (ViT-B/32+ViT-L/14) has much lower scores compared to the other explicit models.

Original: Qualitatively, the ensemble models have higher texture details for most objects.
Revised: Qualitatively, the ensemble models have higher texture details for most objects.

Original: However, we also notice that for some prompts, subjects mentioned in the input text are not present in the generated object of the ensemble model that was otherwise present when just using ViT-B/32 or ViT-B/16 for guidance.
Revised: However, we also notice that for some prompts, subjects mentioned in the input text are not present in the generated object of the ensemble model that was otherwise present when just using ViT-B/32 or ViT-B/16 for guidance.

Original: For Vox Imp , the model trained with ViT-B/32 had the highest performance.
Revised: For Vox Imp , the model trained with ViT-B/32 had the highest performance.

Original: Qualitatively, objects generated by Vox Imp trained with
Revised: Qualitatively, objects generated by Vox Imp trained with

Original: ViT-B/16 can generate more detailed geometry.
Revised: ViT-B/16 can generate more detailed geometry.

Original: In Dream Fields, their ViT-B/32 model also outperformed their ViT-B/16 model in terms of R-Precision.
Revised: In Dream Fields, their ViT-B/32 model also outperformed their ViT-B/16 model in terms of R-Precision.

Original: CLIP R-Precision While the R-Precision is correlated with the faithfulness of the generated object with respect to the prompt, higher scores do not always yield the best qualitative results.
Revised: CLIP R-Precision While the R-Precision is correlated with the faithfulness of the generated object with respect to the prompt, higher scores do not always yield the best qualitative results.

Original: The RPrecision score itself is also prone to adversarial examples.
Revised: The RPrecision score itself is also prone to adversarial examples.

Original: This suggests that quantitative evaluation for Text-to-3D models is still very challenging and better metrics are required.
Revised: This suggests that quantitative evaluation for Text-to-3D models is still very challenging and better metrics are required.

Original: A. V OX I MP A UGMENTATION A BLATION
Revised: A. V OX I MP A UGMENTATION A BLATION

Original: We show the augmentation ablation for the Vox Imp model with KL loss on in Fig. 8.
Revised: We show the augmentation ablation for the Vox Imp model with KL loss on in Fig. 8.

Original: Overall we find that compared to Vox Exp , the Vox Imp model is less noisy even in cases where less augmentations are on.
Revised: Overall we find that compared to Vox Exp , the Vox Imp model is less noisy even in cases where less augmentations are on.

Original: There seem to be no unwanted background textures even when just Diff and Back augmentations are on.
Revised: There seem to be no unwanted background textures even when just Diff and Back augmentations are on.

Original: Since the perspective augmentation can sometimes hurt the texture sharpness we turn it off for Vox Imp as it seems unnecessary.
Revised: Since the perspective augmentation can sometimes hurt the texture sharpness we turn it off for Vox Imp as it seems unnecessary.

Original: [Figure]
Revised: [Figure]

Original: OpenCLIPViT-B
Revised: OpenCLIPViT-B

Original: [Figure]
Revised: [Figure]

Original: OpenCLIPViT-B
Revised: OpenCLIPViT-B

Original: [Figure]
Revised: [Figure]

Original: CLIP
Revised: CLIP

Original: ViT-B/ "medieval people celebrating a festival with many stalls; trending on artstation."
Revised: ViT-B/ "medieval people celebrating a festival with many stalls; trending on artstation."

Original: “Tokyo city; trending on artstation."
Revised: “Tokyo city; trending on artstation."

Original: "steampunk city; trending on artstation.
Revised: "steampunk city; trending on artstation.

Original: “a fantasy castle; trending on artstation.
Revised: “a fantasy castle; trending on artstation.

Original: " “a souls like dark fantasy cityscape;trendingon artstation."
Revised: " “a souls like dark fantasy cityscape;trendingon artstation."

Original: In this section we tested out another implementation of CLIP, namely OpenCLIP Ilharco et al.
Revised: In this section we tested out another implementation of CLIP, namely OpenCLIP Ilharco et al.

Original: (2021) which was trained on the LAION Schuhmann et al.
Revised: (2021) which was trained on the LAION Schuhmann et al.

Original: (2021) dataset.
Revised: (2021) dataset.

Original: Particularly we use the
Revised: Particularly we use the

Original: ViT-B/32 variant trained on LAION-2B and the ViT-B/16 variant trained on LAION-400M. We use the Vox Imp model and the image resolution is set to 224 2 during training.
Revised: ViT-B/32 variant trained on LAION-2B and the ViT-B/16 variant trained on LAION-400M. We use the Vox Imp model and the image resolution is set to 224 2 during training.

Original: Our results are shown in
Revised: Our results are shown in

Original: Fig.
Revised: Fig.

Original: Overall, we see that there is not one model that performed the best across the board.
Revised: Overall, we see that there is not one model that performed the best across the board.

Original: For some prompts, all three models generated coherent results and the variation between them can be desirable for creative applications.
Revised: For some prompts, all three models generated coherent results and the variation between them can be desirable for creative applications.

Original: It would be interesting to explore further how varying datasets as well as size can have an impact on the quality and diversity of the generated results.
Revised: It would be interesting to explore further how varying datasets as well as size can have an impact on the quality and diversity of the generated results.

Original: A. R ESOURCE U SAGE
Revised: A. R ESOURCE U SAGE

Original: In Tab. 6 the memory usage and time consumption is shown across the devices we conducted training on.
Revised: In Tab. 6 the memory usage and time consumption is shown across the devices we conducted training on.

Original: All of our model variations consumed less memory compared to Dream Fields and was faster to train.
Revised: All of our model variations consumed less memory compared to Dream Fields and was faster to train.

Original: Their model takes a lot of memory because of the 8-layer MLP with residual connection used for the NeRF model.
Revised: Their model takes a lot of memory because of the 8-layer MLP with residual connection used for the NeRF model.

Original: Since we mostly used RTX A6000s with 48 GB of VRAM we didn’t optimize hyperparameters such as voxel grid resolution much unless we ran out of memory.
Revised: Since we mostly used RTX A6000s with 48 GB of VRAM we didn’t optimize hyperparameters such as voxel grid resolution much unless we ran out of memory.

Original: For code release we will provide both Vox Exp and Vox Imp model settings that will run on more commodity GPUs.
Revised: For code release we will provide both Vox Exp and Vox Imp model settings that will run on more commodity GPUs.

Original: It is worth noting that the Vox Imp model trained at image resolutions 168 2 and 224 2 consumes about the same amount of memory.
Revised: It is worth noting that the Vox Imp model trained at image resolutions 168 2 and 224 2 consumes about the same amount of memory.

Original: This is because we evaluate the forward pass on the entire V ( PE ) grid and since both variations has the same voxel grid resolution their memory consumption is about the same.
Revised: This is because we evaluate the forward pass on the entire V ( PE ) grid and since both variations has the same voxel grid resolution their memory consumption is about the same.

Original: We plan to update this in the future so that regions of the voxels that are not hit by rays will be masked out from the calculation of the forward pass.
Revised: We plan to update this in the future so that regions of the voxels that are not hit by rays will be masked out from the calculation of the forward pass.

Original: We were also not able to run Dream Fields at higher resolutions in their paper because of availability issues with TPU v3-8.
Revised: We were also not able to run Dream Fields at higher resolutions in their paper because of availability issues with TPU v3-8.

Original: [Table]
Revised: [Table]