Original: Generative Neural Articulated Radiance Fields
Revised: Generative Neural Articulated Radiance Fields

Original: computationalimaging.org publications gnarf
Revised: computationalimaging.org publications gnarf

Original: Abstract
Revised: Abstract

Original: Unsupervised learning of 3D-aware generative adversarial networks (GANs) using only collections of single-view 2D photographs has very recently made much progress.
Revised: Unsupervised learning of 3D-aware generative adversarial networks (GANs) using only collections of single-view 2D photographs has very recently made much progress.

Original: These 3D GANs, however, have not been demonstrated for human bodies and the generated radiance ﬁelds of existing frameworks are not directly editable, limiting their applicability in downstream tasks.
Revised: These 3D GANs, however, have not been demonstrated for human bodies and the generated radiance ﬁelds of existing frameworks are not directly editable, limiting their applicability in downstream tasks.

Original: We propose a solution to these challenges by developing a 3D GAN framework that learns to generate radiance ﬁelds of human bodies or faces in a canonical pose and warp them using an explicit deformation ﬁeld into a desired body pose or facial expression.
Revised: We propose a solution to these challenges by developing a 3D GAN framework that learns to generate radiance ﬁelds of human bodies or faces in a canonical pose and warp them using an explicit deformation ﬁeld into a desired body pose or facial expression.

Original: Using our framework, we demonstrate the ﬁrst high-quality radiance ﬁeld generation results for human bodies.
Revised: Using our framework, we demonstrate the ﬁrst high-quality radiance ﬁeld generation results for human bodies.

Original: Moreover, we show that our deformation-aware training procedure signiﬁcantly improves the quality of generated bodies or faces when editing their poses or facial expressions compared to a 3D GAN that is not trained with explicit deformations.
Revised: Moreover, we show that our deformation-aware training procedure signiﬁcantly improves the quality of generated bodies or faces when editing their poses or facial expressions compared to a 3D GAN that is not trained with explicit deformations.

Original:  Introduction
Revised:  Introduction

Original: Unsupervised learning of 3D-aware generative adversarial networks (GANs) using large-scale datasets of unstructured single-view images is an emerging research area.
Revised: Unsupervised learning of 3D-aware generative adversarial networks (GANs) using large-scale datasets of unstructured single-view images is an emerging research area.

Original: Such 3D GANs have recently been demonstrated to enable photorealistic and multi-view consistent generation of radiance ﬁelds representing human faces [1–7].
Revised: Such 3D GANs have recently been demonstrated to enable photorealistic and multi-view consistent generation of radiance ﬁelds representing human faces [1–7].

Original: These approaches, however, have not been shown to work with human bodies, partly because learning the body pose distribution is much more challenging given the signiﬁcantly higher diversity in articulations compared to facial expressions.
Revised: These approaches, however, have not been shown to work with human bodies, partly because learning the body pose distribution is much more challenging given the signiﬁcantly higher diversity in articulations compared to facial expressions.

Original: Yet, generative 3D models of photorealistic humans have signiﬁcant utility in a wide range of applications, including visual effects, computer vision, and virtual or augmented reality.
Revised: Yet, generative 3D models of photorealistic humans have signiﬁcant utility in a wide range of applications, including visual effects, computer vision, and virtual or augmented reality.

Original: In these scenarios, it is critical that the generated people are editable to support interactive applications, which is not necessarily the case for existing 3D GANs.
Revised: In these scenarios, it is critical that the generated people are editable to support interactive applications, which is not necessarily the case for existing 3D GANs.

Original: While variations of linear blend skinning [8] have been adopted to articulate radiance ﬁelds for single-scene scenarios [9–21], it is unclear how to efﬁciently apply such deformation methods to generative models.
Revised: While variations of linear blend skinning [8] have been adopted to articulate radiance ﬁelds for single-scene scenarios [9–21], it is unclear how to efﬁciently apply such deformation methods to generative models.

Original: With our work, dubbed generative neural articulated radiance ﬁelds or GNARF, we propose solutions to both of these challenges.
Revised: With our work, dubbed generative neural articulated radiance ﬁelds or GNARF, we propose solutions to both of these challenges.

Original: Firstly, we demonstrate generation of high-quality 3D (i.e., multi-view consistent and geometry aware) human bodies using a GAN that is trained in an unsupervised manner on datasets containing single-view images.
Revised: Firstly, we demonstrate generation of high-quality 3D (i.e., multi-view consistent and geometry aware) human bodies using a GAN that is trained in an unsupervised manner on datasets containing single-view images.

Original: To this end, we adopt the recently proposed tri-plane
Revised: To this end, we adopt the recently proposed tri-plane

Original: [Figure]
Revised: [Figure]

Original: feature representation [1], which is extremely efﬁcient for training and rendering radiance ﬁelds, while being compatible with conventional 2D CNN–based generators, such as StyleGAN [22].
Revised: feature representation [1], which is extremely efﬁcient for training and rendering radiance ﬁelds, while being compatible with conventional 2D CNN–based generators, such as StyleGAN [22].

Original: While this framework has been successfully demonstrated for faces in prior work, we are the ﬁrst to adapt it to generating radiance ﬁelds of full human bodies.
Revised: While this framework has been successfully demonstrated for faces in prior work, we are the ﬁrst to adapt it to generating radiance ﬁelds of full human bodies.

Original: Secondly, we tackle the editability of the generated radiance ﬁelds by introducing an explicit radiance ﬁeld deformation step as part of our GAN training procedure.
Revised: Secondly, we tackle the editability of the generated radiance ﬁelds by introducing an explicit radiance ﬁeld deformation step as part of our GAN training procedure.

Original: This step ensures that the generator synthesizes radiance ﬁelds of people in a canonical body pose, which is then explicitly warped according to the body pose distribution of the training data.
Revised: This step ensures that the generator synthesizes radiance ﬁelds of people in a canonical body pose, which is then explicitly warped according to the body pose distribution of the training data.

Original: We show that this new approach generates high-quality, editable, multi-view-consistent human bodies and that our approach can also be applied to editing faces, increasing the controllability of existing generative models for this task (see Fig. 1).
Revised: We show that this new approach generates high-quality, editable, multi-view-consistent human bodies and that our approach can also be applied to editing faces, increasing the controllability of existing generative models for this task (see Fig. 1).

Original: To summarize, the contributions of our approach are:
Revised: To summarize, the contributions of our approach are:

Original: • We present a 3D-aware GAN framework for the generation of editable radiance ﬁelds of human bodies.
Revised: • We present a 3D-aware GAN framework for the generation of editable radiance ﬁelds of human bodies.

Original: To our knowledge, this is the ﬁrst approach of its kind.
Revised: To our knowledge, this is the ﬁrst approach of its kind.

Original: • Our framework introduces an efﬁcient neural representation for articulated objects, including bodies and heads, that combines the recently proposed tri-plane feature volume representation with an explicit feature volume deformation that is guided by a template shape.
Revised: • Our framework introduces an efﬁcient neural representation for articulated objects, including bodies and heads, that combines the recently proposed tri-plane feature volume representation with an explicit feature volume deformation that is guided by a template shape.

Original: • We demonstrate high-quality results for unconditional generation and animation of human bodies using the SURREAL and AIST++ datasets and faces using the FFHQ dataset.
Revised: • We demonstrate high-quality results for unconditional generation and animation of human bodies using the SURREAL and AIST++ datasets and faces using the FFHQ dataset.

Original:  Related Work
Revised:  Related Work

Original: Articulated 3D Representations.
Revised: Articulated 3D Representations.

Original: Parametric shape templates are one of the most common types of articulated 3D representations adopted in recent neural scene representation and rendering approaches.
Revised: Parametric shape templates are one of the most common types of articulated 3D representations adopted in recent neural scene representation and rendering approaches.

Original: These templates, including faces [24, 25], bodies [26], hands [27], or a combination of these parts [28], and even animals [29], have been widely utilized for pose estimation and reconstruction, e.g. [30–34].
Revised: These templates, including faces [24, 25], bodies [26], hands [27], or a combination of these parts [28], and even animals [29], have been widely utilized for pose estimation and reconstruction, e.g. [30–34].

Original: Volume deformation is also commonly used in computer graphics, for example using mean value coordinates (MVC) [35] or biharmonic coordinates [36] for shape deformation and editing [37–40].
Revised: Volume deformation is also commonly used in computer graphics, for example using mean value coordinates (MVC) [35] or biharmonic coordinates [36] for shape deformation and editing [37–41].

Original: GNARF combines parametric template shapes, such as FLAME [25] for heads and SMPL [26] for bodies, with an intuitive surface-driven volume deformation approach that is both computationally efﬁcient and qualitatively comparable to or better than both skinning and MVC-based deformation.
Revised: GNARF combines parametric template shapes, such as FLAME [25] for heads and SMPL [26] for bodies, with an intuitive surface-driven volume deformation approach that is both computationally efﬁcient and qualitatively comparable to or better than both skinning and MVC-based deformation.

Original: This provides intuitive editing control for articulated radiance ﬁeld deformation.
Revised: This provides intuitive editing control for articulated radiance ﬁeld deformation.

Original: Neural Radiance Fields.
Revised: Neural Radiance Fields.

Original: Coordinate networks, also known as neural ﬁelds [41], have emerged as a powerful tool that enable differentiable representations of 3D scenes [42–55] and learning viewdependent neural radiance ﬁelds [56–82].
Revised: Coordinate networks, also known as neural ﬁelds [42], have emerged as a powerful tool that enable differentiable representations of 3D scenes [43–56] and learning viewdependent neural radiance ﬁelds [57–83].

Original: While initial proposals have focused on static scenarios, recent work has demonstrated successful representations of dynamic scenes [83–90].
Revised: While initial proposals have focused on static scenarios, recent work has demonstrated successful representations of dynamic scenes [84–91].

Original: Articulated neural radiance ﬁelds further extend these approaches by providing editability for neural representations of human heads [91–96] and bodies [9–21, 97, 98] or animals [99], often by deforming the underlying radiance ﬁelds using traditional 3D morphable models or skeleton-based parameterizations, or alternatively conditioning the radiance ﬁeld decoder with pose-related parameters.
Revised: Articulated neural radiance ﬁelds further extend these approaches by providing editability for neural representations of human heads [92–97] and bodies [9–21, 98, 99] or animals [100], often by deforming the underlying radiance ﬁelds using traditional 3D morphable models or skeleton-based parameterizations, or alternatively conditioning the radiance ﬁeld decoder with pose-related parameters.

Original: A more detailed survey of static, dynamic, and articulated neural radiance ﬁelds can be found in the recent state-of-the-art report by Tewari et al.
Revised: A more detailed survey of static, dynamic, and articulated neural radiance ﬁelds can be found in the recent state-of-the-art report by Tewari et al.

Original: Note that all of these techniques are supervised with scene-speciﬁc multi-image data and focus on representing, i.e., “overﬁtting”, a single scene.
Revised: Note that all of these techniques are supervised with scene-speciﬁc multi-image data and focus on representing, i.e., “overﬁtting”, a single scene.

Original: Therefore, it is not easily possible to train these models using unstructured 2D image data and then apply them to generate and edit new and unseen objects or humans.
Revised: Therefore, it is not easily possible to train these models using unstructured 2D image data and then apply them to generate and edit new and unseen objects or humans.

Original: Generative 3D-aware Radiance Fields.
Revised: Generative 3D-aware Radiance Fields.

Original: Building on the success of 2D image-based GANs [22, 101–103], recent efforts have focused on training 3D-aware multi-view consistent GANs from collections of single-view 2D images in an unsupervised manner.
Revised: Building on the success of 2D image-based GANs [22, 102–104], recent efforts have focused on training 3D-aware multi-view consistent GANs from collections of single-view 2D images in an unsupervised manner.

Original: Achieving this challenging goal requires a combination of a neural scene representation and differentiable rendering algorithm.
Revised: Achieving this challenging goal requires a combination of a neural scene representation and differentiable rendering algorithm.

Original: Recent work in this domain builds on representations using meshes [104, 105], dense [106–111] or sparse [112] voxel grids, multiple planes [2], fully implicit networks [3–7], or a combination of lowresolution voxel grids combined with 2D CNN-based image upsampling layers [113, 114].
Revised: Recent work in this domain builds on representations using meshes [105, 106], dense [107–112] or sparse [113] voxel grids, multiple planes [2], fully implicit networks [3–7], or a combination of lowresolution voxel grids combined with 2D CNN-based image upsampling layers [114, 115].

Original: Our 3D
Revised: Our 3D

Original: GAN architecture is most closely related to the recent work by Chan et al.
Revised: GAN architecture is most closely related to the recent work by Chan et al.

Original: [1], which uses an efﬁcient tri-plane-based volume representation combined with neural volume rendering.
Revised: [1], which uses an efﬁcient tri-plane-based volume representation combined with neural volume rendering.

Original: We extend this work by including an explicit deformation ﬁeld in our GAN architecture to model diverse articulations, which allows the generator to synthesize radiance ﬁelds of human bodies or heads in a canonical pose while being supervised by 2D image collections that contain arbitrary pose distributions.
Revised: We extend this work by including an explicit deformation ﬁeld in our GAN architecture to model diverse articulations, which allows the generator to synthesize radiance ﬁelds of human bodies or heads in a canonical pose while being supervised by 2D image collections that contain arbitrary pose distributions.

Original: Explicitly disentangling radiance ﬁeld generation and deformation enables us to drastically improve the quality of generated human bodies and faces when their poses or facial expressions are edited.
Revised: Explicitly disentangling radiance ﬁeld generation and deformation enables us to drastically improve the quality of generated human bodies and faces when their poses or facial expressions are edited.

Original: HeadNeRF [115] is related to our approach in that they generate 3D heads conditioned on various attributes.
Revised: HeadNeRF [116] is related to our approach in that they generate 3D heads conditioned on various attributes.

Original: Both HeadNeRF and GNARF condition on identity and facial expression independently, with the former also conditioning on illumination and albedo.
Revised: Both HeadNeRF and GNARF condition on identity and facial expression independently, with the former also conditioning on illumination and albedo.

Original: However, to disentangle individual attributes they need to acquire training images of the same person performing various expressions in different lighting conditions.
Revised: However, to disentangle individual attributes they need to acquire training images of the same person performing various expressions in different lighting conditions.

Original: In contrast, and similar to other 3D GANs, our approach only requires single-view images of different people and can therefore work with readily available 2D image collections.
Revised: In contrast, and similar to other 3D GANs, our approach only requires single-view images of different people and can therefore work with readily available 2D image collections.

Original: The recent work by Grigorev et al.
Revised: The recent work by Grigorev et al.

Original: [116] also generates human bodies.
Revised: [117] also generates human bodies.

Original: However, their work proposes a 2D GAN that generates textures which are used in combination with a standard articulated template mesh whereas we aim at generating and editing radiance ﬁelds using a 3D GAN.
Revised: However, their work proposes a 2D GAN that generates textures which are used in combination with a standard articulated template mesh whereas we aim at generating and editing radiance ﬁelds using a 3D GAN.

Original: The concurrent work of Noguchi et al.
Revised: The concurrent work of Noguchi et al.

Original: [117] is closest to ours as the method also includes a radiance ﬁeld deformation step in a tri-plane-based 3D GAN.
Revised: [118] is closest to ours as the method also includes a radiance ﬁeld deformation step in a tri-plane-based 3D GAN.

Original: Our evaluation shows superior generation quality and, more importantly, while their approach ties the network architecture to the speciﬁc choice of skeleton, our surface-driven volume deformation is agnostic to the particular choice of template and can be used with human bodies, faces, or other object types.
Revised: Our evaluation shows superior generation quality and, more importantly, while their approach ties the network architecture to the speciﬁc choice of skeleton, our surface-driven volume deformation is agnostic to the particular choice of template and can be used with human bodies, faces, or other object types.

Original:  Generative Articulated Neural Radiance Fields
Revised:  Generative Articulated Neural Radiance Fields

Original: GNARF is a novel general framework to train 3D-aware GANs for deformable objects that have a parametric template mesh, e.g. human bodies and faces.
Revised: GNARF is a novel general framework to train 3D-aware GANs for deformable objects that have a parametric template mesh, e.g. human bodies and faces.

Original: It builds on the efﬁcient tri-plane feature representation [1] for the generated neural radiance ﬁeld, but additionally applies an explicit deformation which alleviates the requirement for the generator to learn a complicated distribution of articulations.
Revised: It builds on the efﬁcient tri-plane feature representation [1] for the generated neural radiance ﬁeld, but additionally applies an explicit deformation which alleviates the requirement for the generator to learn a complicated distribution of articulations.

Original: As a result, the generator automatically learns to generate radiance ﬁelds of objects in the canonical pose, which are then warped explicitly to produce target body poses and facial expressions in a fully controllable and interpretable manner.
Revised: As a result, the generator automatically learns to generate radiance ﬁelds of objects in the canonical pose, which are then warped explicitly to produce target body poses and facial expressions in a fully controllable and interpretable manner.

Original: 3.1 Modeling Articulated Radiance Fields
Revised: 3.1 Modeling Articulated Radiance Fields

Original: We ﬁrst discuss our approach to modeling and rendering articulated radiance ﬁelds, before describing how this is integrated into the 3D GAN in Sec. 3.2.
Revised: We ﬁrst discuss our approach to modeling and rendering articulated radiance ﬁelds, before describing how this is integrated into the 3D GAN in Sec. 3.2.

Original: Scene representation.
Revised: Scene representation.

Original: To represent an object, we leverage the recently proposed tri-plane feature representation [1].
Revised: To represent an object, we leverage the recently proposed tri-plane feature representation [1].

Original: This representation uses three axis-aligned 2D feature planes, each with resolution
Revised: This representation uses three axis-aligned 2D feature planes, each with resolution

Original: N × N × C , where N and C denote the spatial resolution and number of channels.
Revised: N × N × C , where N and C denote the spatial resolution and number of channels.

Original: The feature of any
Revised: The feature of any

Original: Deformation
Revised: Deformation

Original: [Figure]
Revised: [Figure]

Original: 3D point x ∈ R 3 is queried by projecting x onto the planes, retrieving three feature vectors via bilinear interpolation, and aggregating the vectors by summation, i.e., F ( x ) =
Revised: 3D point x ∈ R 3 is queried by projecting x onto the planes, retrieving three feature vectors via bilinear interpolation, and aggregating the vectors by summation, i.e., F ( x ) =

Original: F xy ( x ) + F yz ( x ) + F xz ( x ) where F ij : R 3 (cid:55)→ R C is a function mapping 3D coordinates to features on the ij plane via projection and interpolation.
Revised: F xy ( x ) + F yz ( x ) + F xz ( x ) where F ij : R 3 (cid:55)→ R C is a function mapping 3D coordinates to features on the ij plane via projection and interpolation.

Original: Articulated deformation.
Revised: Articulated deformation.

Original: We use the deformation function D : R 3 (cid:55)→ R 3 (detailed later) to warp a coordinate x from the target (deformed) space into the canonical space.
Revised: We use the deformation function D : R 3 (cid:55)→ R 3 (detailed later) to warp a coordinate x from the target (deformed) space into the canonical space.

Original: Using a small multilayer perceptron MLP : R C (cid:55)→ R 4 , we convert the deformed 3D feature volume into a neural ﬁeld of spatially varying RGB colors c and volumetric densities σ as
Revised: Using a small multilayer perceptron MLP : R C (cid:55)→ R 4 , we convert the deformed 3D feature volume into a neural ﬁeld of spatially varying RGB colors c and volumetric densities σ as

Original: [Equation]
Revised: [Equation]

Original: There are many possible choices for how to specify the deformation ﬁeld in an intuitive manner.
Revised: There are many possible choices for how to specify the deformation ﬁeld in an intuitive manner.

Original: For example, linear blend skinning can be used to deform the entire volume “rigged” by a skeleton (see e.g. [8]).
Revised: For example, linear blend skinning can be used to deform the entire volume “rigged” by a skeleton (see e.g. [8]).

Original: While skinning is popular for human body articulations, it cannot explain subtle deformation due to varying facial expressions.
Revised: While skinning is popular for human body articulations, it cannot explain subtle deformation due to varying facial expressions.

Original: Another option is to use the object-speciﬁc template mesh as a cage and apply cage-based deformation for the entire volume using mean value coordinates (MVC) [35].
Revised: Another option is to use the object-speciﬁc template mesh as a cage and apply cage-based deformation for the entire volume using mean value coordinates (MVC) [35].

Original: However, the high computational cost of evaluating MVCs on the full-resolution grid (see Tab. 1) is prohibitive for GAN training and, more critically, this approach generally leads to severe artifacts when the template mesh (accidentally) includes self-intersections (see supplement).
Revised: However, the high computational cost of evaluating MVCs on the full-resolution grid (see Tab. 1) is prohibitive for GAN training and, more critically, this approach generally leads to severe artifacts when the template mesh (accidentally) includes self-intersections (see supplement).

Original: To alleviate these problems, we use an intuitive surface-driven deformation method, which we label as the Surface Field (SF) method.
Revised: To alleviate these problems, we use an intuitive surface-driven deformation method, which we label as the Surface Field (SF) method.

Original: This method only requires canonical and target template meshes with correspondences, which are readily available for faces [25] and bodies [26].
Revised: This method only requires canonical and target template meshes with correspondences, which are readily available for faces [25] and bodies [26].

Original: These template shapes, in turn, can be driven using skeletons, manual editing, or using keypoints or landmarks that could be detected in and transfered from videos of other people.
Revised: These template shapes, in turn, can be driven using skeletons, manual editing, or using keypoints or landmarks that could be detected in and transfered from videos of other people.

Original: Therefore, the SF method is generally sufﬁcient to apply to different body parts and it can be intuitively edited in a number of ways, resulting in accurate volume deformation for our class of volumetric models.
Revised: Therefore, the SF method is generally sufﬁcient to apply to different body parts and it can be intuitively edited in a number of ways, resulting in accurate volume deformation for our class of volumetric models.

Original: The SF approach assigns each 3D coordinate x to its nearest triangle t D x = [ v 0 , v 1 , v 2 ] ∈ R 3 × 3 on the target (deformed) mesh.
Revised: The SF approach assigns each 3D coordinate x to its nearest triangle t D x = [ v 0 , v 1 , v 2 ] ∈ R 3 × 3 on the target (deformed) mesh.

Original: We compute the barycentric coordinates [ u, v, w ] of the coordinate projected onto this triangle and ﬁnd the corresponding triangle on the canonical mesh t C x and its normal n C t x .
Revised: We compute the barycentric coordinates [ u, v, w ] of the coordinate projected onto this triangle and ﬁnd the corresponding triangle on the canonical mesh t C x and its normal n C t x .

Original: The deformed coordinate can then be computed as
Revised: The deformed coordinate can then be computed as

Original: [Equation]
Revised: [Equation]

Original: The SF approach is very fast to compute and mitigates artifacts from self-intersections of the template shape, thereby combining the beneﬁts of linear blend skinning and MVC-based approaches for the task of radiance ﬁeld deformation.
Revised: The SF approach is very fast to compute and mitigates artifacts from self-intersections of the template shape, thereby combining the beneﬁts of linear blend skinning and MVC-based approaches for the task of radiance ﬁeld deformation.

Original: Rendering deformed radiance ﬁelds.
Revised: Rendering deformed radiance ﬁelds.

Original: We render the radiance ﬁeld using (neural) volume rendering [61, 118].
Revised: We render the radiance ﬁeld using (neural) volume rendering [62, 119].

Original: For this purpose, the aggregated feature F ( r ) of a ray r is computed by integrating the volumetric features f and density σ as
Revised: For this purpose, the aggregated feature F ( r ) of a ray r is computed by integrating the volumetric features f and density σ as

Original: [Equation]
Revised: [Equation]

Original: where t n and t f indicate near and far bounds along the ray r ( t )
Revised: where t n and t f indicate near and far bounds along the ray r ( t )

Original: =
Revised: =

Original: o + t d pointing from its origin o into direction d .
Revised: o + t d pointing from its origin o into direction d .

Original: The volume rendering equation (eq. 3) is typically approximated using numerical methods, such as the quadrature rule [118].
Revised: The volume rendering equation (eq. 3) is typically approximated using numerical methods, such as the quadrature rule [119].

Original: 3.2 3D GAN Framework
Revised: 3.2 3D GAN Framework

Original: An overview of our 3D pipeline is shown in Fig.
Revised: An overview of our 3D pipeline is shown in Fig.

Original: Several components, including the StyleGAN generator, the tri-plane representation, the volume rendering, the CNN-based image super-resolution module, and (dual) discrimination are directly adopted from the EG3D framework [1].
Revised: Several components, including the StyleGAN generator, the tri-plane representation, the volume rendering, the CNN-based image super-resolution module, and (dual) discrimination are directly adopted from the EG3D framework [1].

Original: Instead of directly generating the radiance ﬁeld with the target body pose or facial expression, however, GNARF is unique in generating the radiance ﬁeld in a canonical pose and then applying the deformation ﬁeld discussed in the previous section to warp the feature volume.
Revised: Instead of directly generating the radiance ﬁeld with the target body pose or facial expression, however, GNARF is unique in generating the radiance ﬁeld in a canonical pose and then applying the deformation ﬁeld discussed in the previous section to warp the feature volume.

Original: We additionally remove the pose conditioning on the generator, and only use camera pose and body pose conditioning in the discriminator.
Revised: We additionally remove the pose conditioning on the generator, and only use camera pose and body pose conditioning in the discriminator.

Original: This removes the ability for the generator to incorporate any knowledge about the ﬁnal view or pose in the canonical radiance ﬁeld generation, ensuring that the generated results will be robustly animatable beyond just the image rendered at training time.
Revised: This removes the ability for the generator to incorporate any knowledge about the ﬁnal view or pose in the canonical radiance ﬁeld generation, ensuring that the generated results will be robustly animatable beyond just the image rendered at training time.

Original: Thus, the generator depends only on the latent code controlling identity, which is input into a StyleGAN2 generator.
Revised: Thus, the generator depends only on the latent code controlling identity, which is input into a StyleGAN2 generator.

Original: This architectural choice takes advantage of the state-of-the-art 2D generative model architectures by using them to generate the tri-plane 3D representation.
Revised: This architectural choice takes advantage of the state-of-the-art 2D generative model architectures by using them to generate the tri-plane 3D representation.

Original: The discriminator having access to the camera and body poses ensure that the GAN learns to generate warping accurate to a target pose rather than just being in the correct distribution.
Revised: The discriminator having access to the camera and body poses ensure that the GAN learns to generate warping accurate to a target pose rather than just being in the correct distribution.

Original: Finally, we adopt a radiance ﬁeld rendering strategy which samples along each ray inside of an expanded template mesh.
Revised: Finally, we adopt a radiance ﬁeld rendering strategy which samples along each ray inside of an expanded template mesh.

Original: This ensures that the integration samples are taken in regions of the radiance ﬁeld with the most detail and not taken in empty space, simultaneously improving the quality of the generated results and speeding up training.
Revised: This ensures that the integration samples are taken in regions of the radiance ﬁeld with the most detail and not taken in empty space, simultaneously improving the quality of the generated results and speeding up training.

Original: Additional implementation details, source code, and pre-trained models can be found in the supplement or on our website.
Revised: Additional implementation details, source code, and pre-trained models can be found in the supplement or on our website.

Original:  Experiments
Revised:  Experiments

Original: We ﬁrst evaluate the proposed deformation ﬁeld by overﬁtting a single representation on a single dynamic full body scene.
Revised: We ﬁrst evaluate the proposed deformation ﬁeld by overﬁtting a single representation on a single dynamic full body scene.

Original: Then we apply this deformation method in a GAN training pipeline for both bodies (AIST++ [23] and SURREAL [119]) and faces (FFHQ) [103].
Revised: Then we apply this deformation method in a GAN training pipeline for both bodies (AIST++ [23] and SURREAL [120]) and faces (FFHQ) [104].

Original: Training details and hyper-parameters are discussed in the supplement.
Revised: Training details and hyper-parameters are discussed in the supplement.

Original: AIST++ is a large dataset consisting of 10.1M images capturing 30 performers in dance motion.
Revised: AIST++ is a large dataset consisting of 10.1M images capturing 30 performers in dance motion.

Original: Each frame is annotated with a ground truth camera and ﬁtted SMPL body model.
Revised: Each frame is annotated with a ground truth camera and ﬁtted SMPL body model.

Original: SURREAL contains 6M images of synthetic humans created using SMPL body models in various poses rendered in indoor scenes.
Revised: SURREAL contains 6M images of synthetic humans created using SMPL body models in various poses rendered in indoor scenes.

Original: FFHQ is a large dataset of high-resolution images of human faces collected from Flickr.
Revised: FFHQ is a large dataset of high-resolution images of human faces collected from Flickr.

Original: All images have licenses that allow free use, redistribution, and adaptation for non-commencial use.
Revised: All images have licenses that allow free use, redistribution, and adaptation for non-commencial use.

Original: 4.1 Single-scene Overﬁtting
Revised: 4.1 Single-scene Overﬁtting

Original: We compare the proposed surface-driven deformation method, SF, with two alternative methods,
Revised: We compare the proposed surface-driven deformation method, SF, with two alternative methods,

Original: MVC and skinning, in a single-scene overﬁtting task.
Revised: MVC and skinning, in a single-scene overﬁtting task.

Original: MVCs require a set of weights (called the mean value coordinates) to be computed w.r.t.
Revised: MVCs require a set of weights (called the mean value coordinates) to be computed w.r.t.

Original: every vertex of the target mesh M D for every sample point.
Revised: every vertex of the target mesh M D for every sample point.

Original: The sample point is then deformed into the canonical pose by linearly combining the vertices of the canonical mesh M C with these computed weights.
Revised: The sample point is then deformed into the canonical pose by linearly combining the vertices of the canonical mesh M C with these computed weights.

Original: In skinning, the sampling points are deformed to the canonical pose by the rigid transformation of the closest bone as measured by point to line-segment distance.
Revised: In skinning, the sampling points are deformed to the canonical pose by the rigid transformation of the closest bone as measured by point to line-segment distance.

Original: We ﬁnd this simpliﬁed deﬁnition of skinning effective in avoiding blending between two topologically distant body parts (e.g., hand and pelvis) if the starting pose brings them to a geometric proximity.
Revised: We ﬁnd this simpliﬁed deﬁnition of skinning effective in avoiding blending between two topologically distant body parts (e.g., hand and pelvis) if the starting pose brings them to a geometric proximity.

Original: We select a multi-view video sequence from the AIST++ dataset [23] and optimize tri-plane features in the canonical pose using a subset of the views and frames for supervision.
Revised: We select a multi-view video sequence from the AIST++ dataset [23] and optimize tri-plane features in the canonical pose using a subset of the views and frames for supervision.

Original: We then evaluate the quality of the estimated radiance ﬁeld warped into these training views and poses but also into held-out test views and poses.
Revised: We then evaluate the quality of the estimated radiance ﬁeld warped into these training views and poses but also into held-out test views and poses.

Original: We apply several modiﬁcations to the tri-plane architecture to reduce overﬁtting; details regarding these changes as well as the selection of training and evaluation views are provided in the supplemental material.
Revised: We apply several modiﬁcations to the tri-plane architecture to reduce overﬁtting; details regarding these changes as well as the selection of training and evaluation views are provided in the supplemental material.

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: To speed up MVC and SF computation, we decimate the source and deformed SMPL mesh using
Revised: To speed up MVC and SF computation, we decimate the source and deformed SMPL mesh using

Original: Quadric Error Metric Decimation [120] in the Open3D library [121] from the original 13,776 faces to 1,376 faces, while tracking the correspondence between the source and deformed meshes.
Revised: Quadric Error Metric Decimation [121] in the Open3D library [122] from the original 13,776 faces to 1,376 faces, while tracking the correspondence between the source and deformed meshes.

Original: Nonetheless computing MVC for each deformed pose is still prohibitively expensive for online training (3.7 s per example).
Revised: Nonetheless computing MVC for each deformed pose is still prohibitively expensive for online training (3.7 s per example).

Original: We thus precompute the deformation for training and test body poses on a ﬁxed 16 3 grid and retrieve the deformation for arbitrary sampling points using trilinear interpolation.
Revised: We thus precompute the deformation for training and test body poses on a ﬁxed 16 3 grid and retrieve the deformation for arbitrary sampling points using trilinear interpolation.

Original: As shown in Tab. 1, our SF method outperforms the others for both training and test images.
Revised: As shown in Tab. 1, our SF method outperforms the others for both training and test images.

Original: MVC performs worst, partially due to the grid approximation, which is essential in practice.
Revised: MVC performs worst, partially due to the grid approximation, which is essential in practice.

Original: The skinning method is comparable to SF in terms of image quality but it is 3 × slower.
Revised: The skinning method is comparable to SF in terms of image quality but it is 3 × slower.

Original: Moreover, skinning cannot sufﬁciently deform subtle facial expressions.
Revised: Moreover, skinning cannot sufﬁciently deform subtle facial expressions.

Original: Therefore, the SF approach is the most ﬂexible among these deformation methods by being compatible with different human body parts while also offering computational and memory efﬁciency.
Revised: Therefore, the SF approach is the most ﬂexible among these deformation methods by being compatible with different human body parts while also offering computational and memory efﬁciency.

Original: 4.2 Human Body Generation and Animation
Revised: 4.2 Human Body Generation and Animation

Original: We now use our SF approach as the deformation method for feature volumes generated by GNARF.
Revised: We now use our SF approach as the deformation method for feature volumes generated by GNARF.

Original: Our method is trained and evaluated on the captured AIST++ [23] and the synthetic SURREAL [119] datasets.
Revised: Our method is trained and evaluated on the captured AIST++ [23] and the synthetic SURREAL [120] datasets.

Original: For both datasets, our method generates high-quality multi-view consistent human bodies in diverse poses that closely match the target pose.
Revised: For both datasets, our method generates high-quality multi-view consistent human bodies in diverse poses that closely match the target pose.

Original: Baselines & Evaluation.
Revised: Baselines & Evaluation.

Original: Because GNARF is the ﬁrst method to learn a generative model of radiance ﬁelds representing bodies, we propose a baseline where we use the original EG3D trained without deformation to generate a feature volume (not in the canonical pose) then warp it into various target poses during inference time using the proposed SF deformation method.
Revised: Because GNARF is the ﬁrst method to learn a generative model of radiance ﬁelds representing bodies, we propose a baseline where we use the original EG3D trained without deformation to generate a feature volume (not in the canonical pose) then warp it into various target poses during inference time using the proposed SF deformation method.

Original: Without the feature
Revised: Without the feature

Original: [Table]
Revised: [Table]

Original: volume deformation, the generator is forced to learn to model both identity and pose in its latent space.
Revised: volume deformation, the generator is forced to learn to model both identity and pose in its latent space.

Original: As a result, the tri-plane features no longer represent a human body in a consistent canonical pose, but rather match the distribution of poses in the dataset.
Revised: As a result, the tri-plane features no longer represent a human body in a consistent canonical pose, but rather match the distribution of poses in the dataset.

Original: The animation of generated bodies is similar to our proposed approach, except that the generated (arbitrarily posed) human body is used as the canonical pose, for which we obtain a SMPL mesh by applying the human shape reconstruction method SPIN [31].
Revised: The animation of generated bodies is similar to our proposed approach, except that the generated (arbitrarily posed) human body is used as the canonical pose, for which we obtain a SMPL mesh by applying the human shape reconstruction method SPIN [31].

Original: Additionally, we included the concurrent work ENARF-GAN and its variant
Revised: Additionally, we included the concurrent work ENARF-GAN and its variant

Original: ENARF-VAE [117] in the evaluation whenever a fair and standardized comparison is possible.
Revised: ENARF-VAE [118] in the evaluation whenever a fair and standardized comparison is possible.

Original: We compare FID scores to evaluate the quality and diversity of generated images as well as the
Revised: We compare FID scores to evaluate the quality and diversity of generated images as well as the

Original: Percentage of Correct Keypoints (PCK) metric to evaluate the quality of the animation.
Revised: Percentage of Correct Keypoints (PCK) metric to evaluate the quality of the animation.

Original: PCK computes the percentage of 2D keypoints detected on a generated rendered image that are within an error threshold (half the size of the head in the case of PCKh@0.5) of keypoints on a ground truth image in the same pose and view.
Revised: PCK computes the percentage of 2D keypoints detected on a generated rendered image that are within an error threshold (half the size of the head in the case of PCKh@0.5) of keypoints on a ground truth image in the same pose and view.

Original: We use an off-the-shelf body keypoint estimator [122] trained on MPII [123] publically available on the MMPose Project [124] to estimate keypoints from a corresponding ground truth and generated image.
Revised: We use an off-the-shelf body keypoint estimator [123] trained on MPII [124] publically available on the MMPose Project [125] to estimate keypoints from a corresponding ground truth and generated image.

Original: AIST++.
Revised: AIST++.

Original: AIST++ is a challenging dataset as the body poses are extremely diverse.
Revised: AIST++ is a challenging dataset as the body poses are extremely diverse.

Original: We collect 30 frames per video as our training data after ﬁltering out frames whose camera distance is above a threshold or the human bounding box is partially outside the image.
Revised: We collect 30 frames per video as our training data after ﬁltering out frames whose camera distance is above a threshold or the human bounding box is partially outside the image.

Original: Then we extract the human body by cropping a 600 × 600 patch centered at the pelvis joint, and resize these frames to 256 × 256 .
Revised: Then we extract the human body by cropping a 600 × 600 patch centered at the pelvis joint, and resize these frames to 256 × 256 .

Original: Since this dataset does not provide ground truth segmentation masks, we use a pre-trained segmentation model [125] to remove backgrounds to stabilize the GAN training.
Revised: Since this dataset does not provide ground truth segmentation masks, we use a pre-trained segmentation model [126] to remove backgrounds to stabilize the GAN training.

Original: To speed up training, we use
Revised: To speed up training, we use

Original: GAN transfer learning [126].
Revised: GAN transfer learning [127].

Original: Rather than initializing our network weights randomly, we begin training from a pre-trained EG3D [1] model.
Revised: Rather than initializing our network weights randomly, we begin training from a pre-trained EG3D [1] model.

Original: Fine-tuning allows for quicker convergence and saves computational resources during training.
Revised: Fine-tuning allows for quicker convergence and saves computational resources during training.

Original: As shown in Tab. 2, we can see that our method outperforms the naive reanimation of EG3D by a large margin (see the 4th row vs. the 5th row).
Revised: As shown in Tab. 2, we can see that our method outperforms the naive reanimation of EG3D by a large margin (see the 4th row vs. the 5th row).

Original: Furthermore, our animated method also generates better images than the
Revised: Furthermore, our animated method also generates better images than the

Original: EG3D baseline that does not support animations.
Revised: EG3D baseline that does not support animations.

Original: This is likely due to
Revised: This is likely due to

Original: GNARF allowing the generator to focus on generating a speciﬁc identity in a canonical pose instead of learning both identity and complex pose distribution in a combined latent space.
Revised: GNARF allowing the generator to focus on generating a speciﬁc identity in a canonical pose instead of learning both identity and complex pose distribution in a combined latent space.

Original: Similarly, our model enables high-quality re-animation, as demonstrated by the PCKh@0.5 metric.
Revised: Similarly, our model enables high-quality re-animation, as demonstrated by the PCKh@0.5 metric.

Original: In
Revised: In

Original: Fig.
Revised: Fig.

Original: 3, our method produces signiﬁcantly better qualitative results than those generated by the baseline.
Revised: 3, our method produces signiﬁcantly better qualitative results than those generated by the baseline.

Original: The baseline results using re-warped
Revised: The baseline results using re-warped

Original: EG3D are signiﬁcantly degraded, since it is difﬁcult to accurately estimate the SMPL mesh from the generated images, which we use as the canonical pose in the deformation function.
Revised: EG3D are signiﬁcantly degraded, since it is difﬁcult to accurately estimate the SMPL mesh from the generated images, which we use as the canonical pose in the deformation function.

Original: Additionally, ﬂoating artifacts which exist in the radiance ﬁeld outside of camera views and make no difference in the conventionally rendered images become visible after being warped can cause false occlusions.
Revised: Additionally, ﬂoating artifacts which exist in the radiance ﬁeld outside of camera views and make no difference in the conventionally rendered images become visible after being warped can cause false occlusions.

Original: In Fig.
Revised: In Fig.

Original: 1, we show that our method generates bodies in a canonical pose with diverse identities.
Revised: 1, we show that our method generates bodies in a canonical pose with diverse identities.

Original: Additionally, we show that by changing the SMPL parameters, we can drive each radiance ﬁeld to a desired target pose and render at an arbitrary novel view.
Revised: Additionally, we show that by changing the SMPL parameters, we can drive each radiance ﬁeld to a desired target pose and render at an arbitrary novel view.

Original: SURREAL.
Revised: SURREAL.

Original: We also test our method on the SURREAL dataset.
Revised: We also test our method on the SURREAL dataset.

Original: The training data is extracted from the ﬁrst frame of each video in SURREAL’s training split.
Revised: The training data is extracted from the ﬁrst frame of each video in SURREAL’s training split.

Original: Each frame is cropped based
Revised: Each frame is cropped based

Original: FID (500) ↓
Revised: FID (500) ↓

Original: FID (50k) ↓
Revised: FID (50k) ↓

Original: AED ↓
Revised: AED ↓

Original: APD ↓
Revised: APD ↓

Original: ID-Consistency [Table]
Revised: ID-Consistency ↑

Original: EG3D (+3DMM est.
Revised: EG3D (+3DMM est.

Original: & re-warping)
Revised: & re-warping)

Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table] 
Revised: [Table] [Figure] [Table] [Figure]

Original: PIRenderer [128]
Revised: PIRenderer [129]

Original:  [Table]
Revised: [Figure] [Table] [Figure] [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: 3D GAN inversion [129] 31.
Revised: 3D GAN inversion [130] 31.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: GNARF
Revised: GNARF

Original: [Table]
Revised: [Table]

Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: on the provided segmentation mask from head-to-toe, and resized to 128 × 128 .
Revised: on the provided segmentation mask from head-to-toe, and resized to 128 × 128 .

Original: We ﬁlter out the backgrounds and set them to be black.
Revised: We ﬁlter out the backgrounds and set them to be black.

Original: The SURREAL dataset provides ground truth SMPL parameters and camera intrinsics and extrinsics for each frame.
Revised: The SURREAL dataset provides ground truth SMPL parameters and camera intrinsics and extrinsics for each frame.

Original: Similarly to AIST++, we use transfer learning from a pre-trained EG3D model at the appropriate resolution.
Revised: Similarly to AIST++, we use transfer learning from a pre-trained EG3D model at the appropriate resolution.

Original: In Tab. 2, we see that our method trained with deformation produces improved FID scores as the version trained without SF feature volume deformation (5th vs 3rd row).
Revised: In Tab. 2, we see that our method trained with deformation produces improved FID scores as the version trained without SF feature volume deformation (5th vs 3rd row).

Original: Attempting to deform generated radiance ﬁelds results in an immense degradation in quality, shown by both the FID and
Revised: Attempting to deform generated radiance ﬁelds results in an immense degradation in quality, shown by both the FID and

Original: PCKh@0.5 metrics (4th row).
Revised: PCKh@0.5 metrics (4th row).

Original: Additional qualitative results and evaluations are included on the website.
Revised: Additional qualitative results and evaluations are included on the website.

Original: 4.3 Human Face Generation and Editing
Revised: 4.3 Human Face Generation and Editing

Original: Thanks to the surface-driven deformation method, SF, our method can directly utilize expressive parametric face models to drive subtle deformations.
Revised: Thanks to the surface-driven deformation method, SF, our method can directly utilize expressive parametric face models to drive subtle deformations.

Original: We use the FLAME [25] head model and apply the state-of-the-art facial reconstruction method DECA [127] to estimate the ﬂame parameters (100D identity vectors, 6D joint position vectors and 50D expression vectors) from the training dataset.
Revised: We use the FLAME [25] head model and apply the state-of-the-art facial reconstruction method DECA [128] to estimate the ﬂame parameters (100D identity vectors, 6D joint position vectors and 50D expression vectors) from the training dataset.

Original: Since DECA does not account for eyeball movement, we remove the eyeballs from the original
Revised: Since DECA does not account for eyeball movement, we remove the eyeballs from the original

Original: FLAME template.
Revised: FLAME template.

Original: Additionally, we add triangles to close the holes at the neck and mouth area in the
Revised: Additionally, we add triangles to close the holes at the neck and mouth area in the

Original: FLAME template, which improves the consistency of the SF deformation.
Revised: FLAME template, which improves the consistency of the SF deformation.

Original: These preprocessing steps produce a mesh with 3,741 vertices and 7,478 faces.
Revised: These preprocessing steps produce a mesh with 3,741 vertices and 7,478 faces.

Original: Finally we apply the same decimation method as in the body experiments to obtain a coarse mesh (2,500 triangles), which we use during the GAN training for faster SF deformation.
Revised: Finally we apply the same decimation method as in the body experiments to obtain a coarse mesh (2,500 triangles), which we use during the GAN training for faster SF deformation.

Original: To train GNARF, we start with a conventional EG3D model pre-trained using the FFHQ dataset (see [1] for details on the training details and data processing).
Revised: To train GNARF, we start with a conventional EG3D model pre-trained using the FFHQ dataset (see [1] for details on the training details and data processing).

Original: We then ﬁne-tune this model using the proposed framework, which includes the SF deformation module.
Revised: We then ﬁne-tune this model using the proposed framework, which includes the SF deformation module.

Original: Note that we apply a global scaling and translation to the FLAME template mesh to roughly align it to the faces generated by the pre-trained EG3D model.
Revised: Note that we apply a global scaling and translation to the FLAME template mesh to roughly align it to the faces generated by the pre-trained EG3D model.

Original: For evaluation, our ﬁrst baseline is the original EG3D model with re-warping at inference time, as described in Sec.
Revised: For evaluation, our ﬁrst baseline is the original EG3D model with re-warping at inference time, as described in Sec.

Original: We also include two state-of-the-art facial reenactment methods, PIRenderer [128] and 3D GAN inversion [129].
Revised: We also include two state-of-the-art facial reenactment methods, PIRenderer [129] and 3D GAN inversion [130].

Original: Several metrics are used by our quantitative evaluation.
Revised: Several metrics are used by our quantitative evaluation.

Original: FID (500) follows the evaluation protocol of Lin et al.
Revised: FID (500) follows the evaluation protocol of Lin et al.

Original: [129], where the ground truth dataset consists of 500 randomly sampled identities and the test dataset is constructed by animating the ground truth using randomly sampled target poses.
Revised: [130], where the ground truth dataset consists of 500 randomly sampled identities and the test dataset is constructed by animating the ground truth using randomly sampled target poses.

Original: FID (50k) follows the protocol in EG3D, where the entire
Revised: FID (50k) follows the protocol in EG3D, where the entire

Original: FFHQ dataset is treated as the ground truth and the test dataset includes 50k generated images using randomly sampled latent vectors, camera poses and FLAME facial parameters.
Revised: FFHQ dataset is treated as the ground truth and the test dataset includes 50k generated images using randomly sampled latent vectors, camera poses and FLAME facial parameters.

Original: Following [129], we evaluate the ﬁdelity of the animation with the Average Expression Distance (AED) and the Average
Revised: Following [130], we evaluate the ﬁdelity of the animation with the Average Expression Distance (AED) and the Average

Original: Pose Distance (APD) computed using DECA estimation, as well as the identity consistency based on a face recognition model [130].
Revised: Pose Distance (APD) computed using DECA estimation, as well as the identity consistency based on a face recognition model [131].

Original: As shown in Tab. 3, our method is superior to the baseline methods in both FIDs and in the edited ﬁdelity, and comparable to the baseline EG3D in terms of identity consistency.
Revised: As shown in Tab. 3, our method is superior to the baseline methods in both FIDs and in the edited ﬁdelity, and comparable to the baseline EG3D in terms of identity consistency.

Original: The advantage of our method in editing ability is clearly demonstrated in Fig.
Revised: The advantage of our method in editing ability is clearly demonstrated in Fig.

Original: 5: our method reproduces the target expression more accurately when compared to the face reenactment methods and mitigates the warping artifacts present in the baseline EG3D. In Fig.
Revised: 5: our method reproduces the target expression more accurately when compared to the face reenactment methods and mitigates the warping artifacts present in the baseline EG3D. In Fig.

Original: 6, we further demonstrate the quality of our results.
Revised: 6, we further demonstrate the quality of our results.

Original: Notice that even though the FLAME model does not include teeth, the SF deformation guides the neural radiance ﬁeld to construct teeth consistently at the correct location.
Revised: Notice that even though the FLAME model does not include teeth, the SF deformation guides the neural radiance ﬁeld to construct teeth consistently at the correct location.

Original:  Discussion
Revised:  Discussion

Original: Limitations and future work.
Revised: Limitations and future work.

Original: Our work is not without limitations.
Revised: Our work is not without limitations.

Original: The level of detail in the generated bodies, for example, is relatively low.
Revised: The level of detail in the generated bodies, for example, is relatively low.

Original: This is partly due to the limited resolution of the training data in the SURREAL and AIST++ datasets, but also due to the limited resolution that the tri-plane representation offers for any one body part, such as the face.
Revised: This is partly due to the limited resolution of the training data in the SURREAL and AIST++ datasets, but also due to the limited resolution that the tri-plane representation offers for any one body part, such as the face.

Original: An interesting avenue of future work could include the exploration of adaptive radiance ﬁeld resolution for human bodies, allocating more resolution to salient parts (see e.g. [131]).
Revised: An interesting avenue of future work could include the exploration of adaptive radiance ﬁeld resolution for human bodies, allocating more resolution to salient parts (see e.g. [132]).

Original: The image quality of the generated bodies is currently on par with simpler approaches that only need to generate an RGB texture on the SMPL mesh [116].
Revised: The image quality of the generated bodies is currently on par with simpler approaches that only need to generate an RGB texture on the SMPL mesh [117].

Original: Yet, details in faces and hair cannot be handled by a texture-generating approach and we expect the quality of generative radiance ﬁelds to surpass that of simpler alternatives with increasing and perhaps adaptive radiance ﬁeld resolutions.
Revised: Yet, details in faces and hair cannot be handled by a texture-generating approach and we expect the quality of generative radiance ﬁelds to surpass that of simpler alternatives with increasing and perhaps adaptive radiance ﬁeld resolutions.

Original: Another bottleneck of our current framework is the challenge of being able to work with large-scale datasets showing a diversity of visible humans.
Revised: Another bottleneck of our current framework is the challenge of being able to work with large-scale datasets showing a diversity of visible humans.

Original: In-the-wild datasets, such as MSCOCO [132], do contain this diversity but also contain a signiﬁcant amount of occlusion and detailed backgrounds, requiring the generator to spend capacity on modeling the distribution of backgrounds and occlusions themselves.
Revised: In-the-wild datasets, such as MSCOCO [133], do contain this diversity but also contain a signiﬁcant amount of occlusion and detailed backgrounds, requiring the generator to spend capacity on modeling the distribution of backgrounds and occlusions themselves.

Original: An interesting avenue of future work consists of explicitly modeling occlusion and background in the GAN training pipeline independently from identity and pose.
Revised: An interesting avenue of future work consists of explicitly modeling occlusion and background in the GAN training pipeline independently from identity and pose.

Original: Currently, the background is not modeled in the generative framework and requires pre-processing of the dataset to separate foreground from background.
Revised: Currently, the background is not modeled in the generative framework and requires pre-processing of the dataset to separate foreground from background.

Original: Additionally, pose estimation from in-the-wild images is still not very accurate, degrading the quality of our deformation function.
Revised: Additionally, pose estimation from in-the-wild images is still not very accurate, degrading the quality of our deformation function.

Original: On the other hand, large-scale custom curated datasets such as the one used in InsetGAN [131] are not publicly available.
Revised: On the other hand, large-scale custom curated datasets such as the one used in InsetGAN [132] are not publicly available.

Original: Finally, the deformation we use as part of the 3D GAN training is limiting in several ways: it does not allow for topology changes, it does not prevent solid parts like teeth or eyeglasses from being stretched, and the deformation quality degrades for points far from the surface.
Revised: Finally, the deformation we use as part of the 3D GAN training is limiting in several ways: it does not allow for topology changes, it does not prevent solid parts like teeth or eyeglasses from being stretched, and the deformation quality degrades for points far from the surface.

Original: In general, using the surface of a parametric mesh to guide an volume may not be the optimal choice for more complex volumetric scenes.
Revised: In general, using the surface of a parametric mesh to guide an volume may not be the optimal choice for more complex volumetric scenes.

Original: More advanced deformation methods, perhaps including kinematic constraints, could be an interesting direction of future research.
Revised: More advanced deformation methods, perhaps including kinematic constraints, could be an interesting direction of future research.

Original: Ethical considerations.
Revised: Ethical considerations.

Original: GANs could be misused for generating edited imagery of real people.
Revised: GANs could be misused for generating edited imagery of real people.

Original: Such misuse of image synthesis techniques poses a societal threat, and we do not condone using our work with the intent of spreading disinformation.
Revised: Such misuse of image synthesis techniques poses a societal threat, and we do not condone using our work with the intent of spreading disinformation.

Original: We also recognize a potential lack of diversity in our results, stemming from implicit biases of the datasets we process.
Revised: We also recognize a potential lack of diversity in our results, stemming from implicit biases of the datasets we process.

Original: Conclusion.
Revised: Conclusion.

Original: Our work takes important steps towards photorealistic 3D-aware image synthesis of articulated human bodies and faces with applications in visual effects, virtual or augmented reality, and teleconferencing among others.
Revised: Our work takes important steps towards photorealistic 3D-aware image synthesis of articulated human bodies and faces with applications in visual effects, virtual or augmented reality, and teleconferencing among others.

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: Alexander W. Bergman was supported by a Stanford Graduate Fellowship.
Revised: Alexander W. Bergman was supported by a Stanford Graduate Fellowship.

Original: Gordon Wetzstein was supported by NSF Award 1839974, Samsung, Stanford HAI and a PECASE from the ARO.
Revised: Gordon Wetzstein was supported by NSF Award 1839974, Samsung, Stanford HAI and a PECASE from the ARO.

Original: We thank Connor Lin for helping standardize comparisons to [129].
Revised: We thank Connor Lin for helping standardize comparisons to [130].

Original: We thank Atsuhiro Noguchi and the authors of [117] for helping in standardization of the comparisons of our methods
Revised: We thank Atsuhiro Noguchi and the authors of [118] for helping in standardization of the comparisons of our methods

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reﬂect the paper’s contributions and scope?

Original: [Yes] (b) Did you describe the limitations of your work?
Revised: [Yes] (b) Did you describe the limitations of your work?

Original: [Yes] See Section 5.
Revised: [Yes] See Section 5.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] See Section 5.
Revised: [Yes] See Section 5.

Original: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: [Yes] 2.
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [N/A] (b) Did you include complete proofs of all theoretical results?
Revised: [N/A] (b) Did you include complete proofs of all theoretical results?

Original: [N/A] 3.
Revised: [N/A] 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [No] We plan to release the code completely, but have not yet with submission.
Revised: [No] We plan to release the code completely, but have not yet with submission.

Original: Details on the architecture, training, and data are available in the supplemental material.
Revised: Details on the architecture, training, and data are available in the supplemental material.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See supplemental document and sections 4.2 and 4.3.
Revised: [Yes] See supplemental document and sections 4.2 and 4.3.

Original: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [No] GNARF models require a signiﬁcant amount of computational resources (see supplemental document), and thus averaging training over many seeds would be computationally infeasible.
Revised: [No] GNARF models require a signiﬁcant amount of computational resources (see supplemental document), and thus averaging training over many seeds would be computationally infeasible.

Original: However, we evaluate our trained models with a large number of generated images, demonstrating the robustness of our method.
Revised: However, we evaluate our trained models with a large number of generated images, demonstrating the robustness of our method.

Original: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] See supplemental document.
Revised: [Yes] See supplemental document.

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] See Section 4.
Revised: [Yes] See Section 4.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [No] (d) Did you discuss whetherand how consent
Revised: [No] (d) Did you discuss whether

Original:  was obtained from people whose data you’re using/curating?
Revised: and how consent was obtained from people whose data you’re using/curating?

Original: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?
Revised: [N/A] (e) Did you discuss whether the data you are using/curating contains personally identiﬁable information or offensive content?

Original: [N/A] 5.
Revised: [N/A] 5.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]