Original: L IGHT S AMPLING F IELD AND BRDF R EPRESENTATION FOR P HYSICALLY - BASED N EURAL R ENDERING
Revised: L IGHT S AMPLING F IELD AND BRDF R EPRESENTATION FOR P HYSICALLY - BASED N EURAL R ENDERING

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: Physically-based rendering (PBR) is key for immersive rendering effects used widely in the industry to showcase detailed realistic scenes from computer graphics assets.
Revised: Physically-based rendering (PBR) is key for immersive rendering effects used widely in the industry to showcase detailed realistic scenes from computer graphics assets.

Original: A well-known caveat is that producing the same is computationally heavy and relies on complex capture devices.
Revised: A well-known caveat is that producing the same is computationally heavy and relies on complex capture devices.

Original: Inspired by the success in quality and efficiency of recent volumetric neural rendering, we want to develop a physically-based neural shader to eliminate device dependency and significantly boost performance.
Revised: Inspired by the success in quality and efficiency of recent volumetric neural rendering, we want to develop a physically-based neural shader to eliminate device dependency and significantly boost performance.

Original: However, no existing lighting and material models in the current neural rendering approaches can accurately represent the comprehensive lighting models and BRDFs properties required by the PBR process.
Revised: However, no existing lighting and material models in the current neural rendering approaches can accurately represent the comprehensive lighting models and BRDFs properties required by the PBR process.

Original: Thus, this paper proposes a novel lighting representation that models direct and indirect light locally through  light sampling strategy in a learned light sampling field.
Revised: Thus, this paper proposes a novel lighting representation that models direct and indirect light locally through a light sampling strategy in a learned light sampling field.

Original: We also propose BRDF models to separately represent surface/subsurface scattering details to enable complex objects such as translucent material (i.e., skin, jade).We then practice our proposed representations with an end-to-end physically-based neural face skin shader, which takes a standard face asset (i.e., geometry, albedo map, and normal map) and an HDRI for illumination as inputs and generates a photo-realistic rendering as output.
Revised: We also propose BRDF models to separately represent surface/subsurface scattering details to enable complex objects such as translucent material (i.e., skin, jade). We then implement our proposed representations with an end-to-end physically-based neural face skin shader, which takes a standard face asset (i.e., geometry, albedo map, and normal map) and an HDRI for illumination as inputs and generates a photo-realistic rendering as output.

Original: Extensive experiments showcase the quality and efficiency of our PBR face skin shader, indicating the effectiveness of our proposed lighting and material representations.
Revised: Extensive experiments showcase the quality and efficiency of our PBR face skin shader, indicating the effectiveness of our proposed lighting and material representations.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Physically-based rendering (PBR) provides a shading and rendering method to accurately represent how light interacts with objects in virtual 3D scenes.
Revised: Physically-based rendering (PBR) provides a shading and rendering method to accurately represent how light interacts with objects in virtual 3D scenes.

Original: Whether working with a real-time rendering system in computer graphics or film production, employing a PBR process will facilitate creating images that look like they exist in the real world for a more immersive experience.
Revised: Whether working with a real-time rendering system in computer graphics or film production, employing a PBR process will facilitate the creation of images that look like they exist in the real world for a more immersive experience.

Original: Industrial PBR pipelines take the guesswork out of authoring surface attributes like transparency since their methodology and algorithms are based on physically accurate formulas and resemble real-world materials.
Revised: Industrial PBR pipelines take the guesswork out of authoring surface attributes like transparency since their methodology and algorithms are based on physically accurate formulae and resemble real-world materials.

Original: This process relies on onerous artist tuning and high computational power in a long production cycle.
Revised: This process relies on onerous artist tuning and high computational power in a long production cycle.

Original: In recent years, academia has shown incredible success using differentiable neural rendering in extensive tasks such as view synthesis (Mildenhall et al., 2020), inverse rendering (Zhang et al., 2021a), and geometry inference (Liu et al., 2019).
Revised: In recent years, academia has shown incredible success using differentiable neural rendering in extensive tasks such as view synthesis (Mildenhall et al., 2020), inverse rendering (Zhang et al., 2021a), and geometry inference (Liu et al., 2019).

Original: Driven by the efficiency of neural rendering, a natural next step would be to marry neural rendering and PBR pipelines.
Revised: Driven by the efficiency of neural rendering, a natural next step would be to marry neural rendering and PBR pipelines.

Original: However, none of the existing neural rendering representations supports the accuracy, expressiveness, and quality mandated by the industrial PBR process.
Revised: However, none of the existing neural rendering representations supports the accuracy, expressiveness, and quality mandated by the industrial PBR process.

Original: A PBR workflow models both specular reflections, which refers to light reflected off the surface, and diffusion or subsurface scattering, which describes the effects of light absorbed or scattered internally.
Revised: A PBR workflow models both specular reflections, which refers to light reflected off the surface, and diffusion or subsurface scattering, which describes the effects of light absorbed or scattered internally.

Original: Pioneering works of differentiable neural shaders such as Softras (Liu et al., 2019) adopted the Lambertian model as BRDF representation, which only models the diffusion effects and results in low-quality rendering.
Revised: Pioneering works of differentiable neural shaders such as Softras (Liu et al., 2019) adopted the Lambertian model as BRDF representation, which only models the diffusion effects and results in low-quality rendering.

Original: NeRF (Mildenhall et al., 2020) proposed a novel radiance field representation for realistic view-synthesis under an emit-absorb lighting transport assumption without explicitly modeling BRDFs or lighting, and hence is limited to a fixed static scene with no scope for relighting.
Revised: NeRF (Mildenhall et al., 2020) proposed a novel radiance field representation for realistic view-synthesis under an emit-absorb lighting transport assumption without explicitly modeling BRDFs or lighting, and hence is limited to a fixed static scene with no scope for relighting.

Original: In follow-up work, NeRV (Srinivasan et al., 2020) took one more step by explicitly modeling directional light, albedo, and visibility maps to make the fixed scene relightable.
Revised: In follow-up work, NeRV (Srinivasan et al., 2020) took one more step by explicitly modeling directional light, albedo, and visibility maps to make the fixed scene relightable.

Original: The indirect illumination was achieved by ray-tracing under the assumption of one bounce of incoming light.
Revised: The indirect illumination was achieved by ray tracing under the assumption of one bounce of incoming ∗ Equal contributions.

Original:  However, this lighting model is extremely computationally heavy for real-world environ- ment illumination when more than one incoming directional lights exist.
Revised: light. However, this lighting model is extremely computationally heavy for real-world environment illumination when more than one incoming directional lights exist.

Original: To address this problem,
Revised: To address this problem,

Original: NeRD (Boss et al., 2020) and PhySG (Zhang et al., 2021a) employ a low-cost global environment illumination modeling method using spherical Gaussian (SG) to extract parameters from HDRIs.
Revised: NeRD (Boss et al., 2020) and PhySG (Zhang et al., 2021a) employ a low-cost global environment illumination modeling method using spherical gaussian (SG) to extract parameters from HDRIs.

Original: Neural-PIL (Boss et al., 2021) further proposed a pre-trained light encoding network for a more detailed global illumination representation.
Revised: Neural-PIL (Boss et al., 2021) further proposed a pre-trained light encoding network for a more detailed global illumination representation.

Original: However, it is still a global illumination representation assuming the same value for the entire scene, which is not true in the real world, where illumination is subjected to shadows and indirect illumination bouncing off of objects in different locations in the scene.
Revised: However, it is still a global illumination representation assuming the same value for the entire scene, which is not true in the real world, where illumination is subjected to shadows and indirect illumination bouncing off of objects in different locations in the scene.

Original: Thus it’s still an approximation but not an accurate representation of the environment illumination.
Revised: Thus it’s still an approximation but not an accurate representation of the environmental illumination.

Original: Regarding material (BRDF)
Revised: Regarding material (BRDF)

Original: modeling, all the current works adopt the basic rendering parameters (such as albedo, roughness, and metalness) defined in the rendering software when preparing the synthetic training data.
Revised: modeling, all the current works adopt the basic rendering parameters (such as albedo, roughness, and metalness) defined in the rendering software when preparing the synthetic training data.

Original: However, they will fail in modeling intricate real-world objects such as participating media (e.g., smoke, fog) and translucent material (organics, skins, jade), where high scattering and subsurface scattering cannot be ignored.
Revised: However, they will fail in modeling intricate real-world objects such as participating media (e.g., smoke, fog) and translucent material (organics, skins, jade), where high scattering and subsurface scattering cannot be ignored.

Original: Such objects require more effort and hence attract more interest in research in their traditional PBR process.
Revised: Such objects require more effort and hence attract more interest in research in their traditional PBR process.

Original: In this work, we aim to design accurate, efficient lighting/ illumination and BRDF representations to enable the neural PBR process, which will support high-quality and photo-realistic rendering in a fast and lightweight manner.
Revised: In this work, we aim to design accurate, efficient lighting/ illumination and BRDF representations to enable the neural PBR process, which will support high-quality and photo-realistic rendering in a fast and lightweight manner.

Original: To achieve this goal, we propose a novel lighting representation - light sampling field to model both the direct and indirect illumination from HDRI environment maps.
Revised: To achieve this goal, we propose a novel lighting representationa Light Sampling Field to model both the direct and indirect illumination from HDRI environment maps.

Original: Our light sampling field representation faithfully captures the direct illumination (incoming from light sources) and indirect illumination (summary of all indirect incoming lighting from surroundings) given an arbitrary sampling location in a continuous field.
Revised: Our Light Sampling Field representation faithfully captures the direct illumination (incoming from light sources) and indirect illumination (summary of all indirect incoming lighting from surroundings) given an arbitrary sampling location in a continuous field.

Original: Accordingly, we propose BRDF representations in the format of surface specular, surface diffuse, and subsurface scattering for modeling real-world object material.
Revised: Accordingly, we propose BRDF representations in the format of surface specular, surface diffuse, and subsurface scattering for modeling real-world object material.

Original: This paper mainly evaluates the proposed representations with a novel volumetric neural physically-based shader for human facial skin.
Revised: This paper mainly evaluates the proposed representations with a novel volumetric neural physically-based shader for human facial skin.

Original: We trained with an extensive high-quality database, including real captured ground truth images as well as synthetic images for illumination augmentation.
Revised: We trained with an extensive high-quality database, including real captured ground truth images as well as synthetic images for illumination augmentation.

Original: We also introduce a novel way of integrating surface normals into volumetric rendering for higher fidelity.
Revised: We also introduce a novel way of integrating surface normals into volumetric rendering for higher fidelity.

Original: Coupled with proposed lighting and BRDFs models, our light transport module delivers pore-level realism in both on- and underneath-surface appearance unprecedentedly.
Revised: Coupled with proposed lighting and BRDFs models, our light transport module delivers pore-level realism in both on- and underneath-surface appearance unprecedentedly.

Original: Experiments show that our light sampling field is robust enough to learn illumination by local geometry, and such an effectcan usually  only be modeled by ray tracing.
Revised: Experiments show that our Light Sampling Field is robust enough to learn illumination by local geometry. Such an effect usually can only be modeled by ray tracing.

Original: Therefore, our method compromises neither efficiency nor quality with the light sampling field when compared to ray tracing.
Revised: Therefore, our method compromises neither efficiency nor quality with the Light Sampling Field when compared to ray tracing.

Original: The main contributions of this paper are as follows: 1) A novel volumetric lighting representation that accurately encodes the direct and indirect illumination positionally and dynamically given an environment map.
Revised: The main contributions of this paper are as follows: 1) A novel volumetric lighting representation that accurately encodes the direct and indirect illumination positionally and dynamically given an environment map.

Original: Our local representation enables efficient modeling of complicated shading effects such as inter-reflectance in neural rendering for the first time as far as we are aware.
Revised: Our local representation enables efficient modeling of complicated shading effects such as inter-reflectance in neural rendering for the first time as far as we are aware.

Original: A

Original:  BRDF measurement representation that supports the PBR process by modeling specular, diffuse, and subsurface scattering separately.
Revised: A BRDF measurement representation that supports the PBR process by modeling specular, diffuse, and subsurface scattering separately.

Original: A novel and lightweight neural PBR face shader that takes facial skin assets and environment maps (HDRIs) as input and efficiently renders photo-realistic, high-fidelity, and accurate images comparable to industrial traditional PBR pipelines such as Maya.
Revised: A novel and lightweight neural PBR face shader that takes facial skin assets and environment maps (HDRIs) as input and efficiently renders photo-realistic, highfidelity, and accurate images comparable to industrial traditional PBR pipelines such as Maya.

Original: Our face shader is trained with an image database consisting of extensive identities and illuminations.
Revised: Our face shader is trained with an image database consisting of extensive identities and illuminations.

Original: Once trained, our models will extract lighting models and BRDFs from input assets, which works well for novel subjects/ illumination maps.
Revised: Once trained, our models will extract lighting models and BRDFs from input assets, which works well for novel subjects/ illumination maps.

Original: Experiments show that our PBR face shader significantly outperforms the state-of-the-art neural face rendering approaches regarding quality and accuracy, which indicates the effectiveness of the proposed lighting and material representations.
Revised: Experiments show that our PBR face shader significantly outperforms the state-of-the-art neural face rendering approaches with regard to quality and accuracy, which indicates the effectiveness of the proposed lighting and material representations.

Original:  R ELATED W ORK
Revised:  R ELATED W ORK

Original: Volumetric Neural Rendering Volumetric rendering models the light interactions with volume densities of absorbing, glowing, reflecting, and scattering materials (Max, 1995).
Revised: Volumetric Neural Rendering Volumetric rendering models the light interactions with volume densities of absorbing, glowing, reflecting, and scattering materials (Max, 1995).

Original: A neural volumetric shader trains a model from a set of images and queries rendered novel images.
Revised: A neural volumetric shader trains a model from a set of images and queries rendered novel images.

Original: The recent state-of-the-art was summarized in a survey (Tewari et al., 2020).
Revised: The recent state-of-the-art was summarized in a survey (Tewari et al., 2020).

Original: In addition, Zhang et al.
Revised: In addition, Zhang et al.

Original: (2019) introduced  radiance field as a differentiable theory of radiative transfer.
Revised: (2019) introduced the radiance field as a differentiable theory of radiative transfer.

Original: Neural Radiance Field (NeRF) (Mildenhall et al., 2020) further described scenes as differentiable neural representation and the following raycasting integrated color in terms of the transmittance factor, volume density, and the voxel diffuse color.
Revised: Neural Radiance Field (NeRF) (Mildenhall et al., 2020) further described scenes as differentiable neural representation and the following raycasting integrated color in terms of the transmittance factor, volume density, and the voxel diffuse color.

Original: Extensions to NeRF were developed for better image encoding (Yu et al., 2020), ray marching (Bi et al., 2020a), network efficiency (Lombardi et al., 2021;
Revised: Extensions to NeRF were developed for better image encoding (Yu et al., 2020), ray marching (Bi et al., 2020a), network efficiency (Lombardi et al., 2021;

Original: Yariv et al., 2020), realistic shading (Suhail et al., 2022) and volumetric radiative decomposition (Bi et al., 2020b; Rebain et al., 2020; Zhang et al., 2021c; Verbin et al., 2021).
Revised: Yariv et al., 2020), realistic shading (Suhail et al., 2022) and volumetric radiative decomposition (Bi et al., 2020b; Rebain et al., 2020; Zhang et al., 2021c; Verbin et al., 2021).

Original: In particular, NeRV (Srinivasan et al., 2020),
Revised: In particular, NeRV (Srinivasan et al., 2020),

Original: NeRD (Boss et al., 2020; 2021) decomposed the reconstructed volume into geometry, SVBRDF and illumination given a set of images even under varying lighting condition.
Revised: NeRD (Boss et al., 2020; 2021) decomposed the reconstructed volume into geometry, SVBRDF, and illumination given a set of images even under varying lighting conditions.

Original: RNR (Chen et al., 2020b) assumed the environmental illumination as distant light and is able to decompose the scene into an albedo map with a 10-order spherical harmonics  of incoming directions.
Revised: RNR (Chen et al., 2020b) assumed the environmental illumination as distant light and is able to decompose the scene into an albedo map with a 10-order spherical harmonics (SH) of incoming directions.

Original: Portrait and Face Relighting Early single-image relighting techniques utilize CNN-based image translation (Nalbach et al., 2017; Thies et al., 2019).
Revised: Portrait and Face Relighting Early single-image relighting techniques utilize CNN-based image translation (Nalbach et al., 2017; Thies et al., 2019).

Original: Due to the lack of 3D models, image translation approaches cannot recover surface materials or represent realistic high-fidelity details, thus neural volumetric relighting approaches are widely adopted recently.
Revised: Due to the lack of 3D models, image translation approaches cannot recover surface materials or represent realistic high-fidelity details, thus neural volumetric relighting approaches are widely adopted recently.

Original: Ma et al.
Revised: Ma et al.

Original: (2021) proposed a lightweight representation to decode only the visible pixels during rendering.
Revised: (2021) proposed a lightweight representation to decode only the visible pixels during rendering.

Original: Bi et al.
Revised: Bi et al.

Original: The face relighting utilized strong priors.
Revised: The face relighting utilized strong priors.

Original: Zhou et al.
Revised: Zhou et al.

Original: (2019) fitted a 3D face model to the input image and obtains refined normal to help achieve relighting.
Revised: (2019) fitted a 3D face model to the input image and obtained refined normal to help achieve relighting.

Original: Chen et al.
Revised: Chen et al.

Original: (2020a)
Revised: (2020a)

Original: relit the image by using spherical harmonics lighting on a predicted 3D face.
Revised: relit the image by using spherical harmonics lighting on a predicted 3D face.

Original: Hou et al.
Revised: Hou et al.

Original: (2022) introduced a shadow mask estimation module to achieve novel face relighting with geometrically consistent shadows.
Revised: (2022) introduced a shadow mask estimation module to achieve novel face relighting with geometrically consistent shadows.

Original: With high-quality volumetric capture in lightstage (Debevec et al., 2000) to obtain training data, this trend achieved the following: regression of a one-light-at-a-time (OLAT) image for relighting (Meka et al., 2019), encoding the feature tensors for a Phong shading into UV space and relighting using an
Revised: With high-quality volumetric capture in lightstage (Debevec et al., 2000) to obtain training data, this trend achieved the following: regression of a one-light-at-a-time (OLAT) image for relighting (Meka et al., 2019), encoding the feature tensors for a Phong shading into UV space and relighting using an

Original: HDRI map (Meka et al., 2020), a neural renderer that can predict the non-diffuse residuals (Zhang et al., 2021b).
Revised: HDRI map (Meka et al., 2020), a neural renderer that can predict the non-diffuse residuals (Zhang et al., 2021b).

Original: Bi et al.
Revised: Bi et al.

Original: (2021) proposed neural networks to learn relighting implicitly but lack of modeling both surface and subsurface reflectance properties following physical light transport.
Revised: (2021) proposed neural networks to learn relighting implicitly but lacked modeling both surface and subsurface reflectance properties following physical light transport.

Original: Similar to our approach most, Sun et al.
Revised: Similar to our approach most, Sun et al.

Original: (2021) inferred both light transport and density, and enabled relighting and view synthesis from a sparse set of input images.
Revised: (2021) inferred both light transport and density, and enabled relighting and view synthesis from a sparse set of input images.

Original:  M ETHODS 3. P RELIMINARIES
Revised:  M ETHODS 3. P RELIMINARIES

Original: We use the rendering equation Kajiya (1986) to estimate the radiance L at a 3D point x with the outgoing direction ω
Revised: We use the rendering equation Kajiya (1986) to estimate the radiance L at a 3D point x with the outgoing direction ω

Original: o : L ( x , ω o ) = (cid:82) ω
Revised: o : L ( x , ω o ) = (cid:82) ω

Original: i ∈
Revised: i ∈

Original: Ω + f ( x , ω
Revised: Ω + f ( x , ω

Original: o , ω i )
Revised: o , ω i )

Original: L i ( x , ω i )( ω i · n x ) d ω i , where f ( x , ω o , ω i ) is the BRDF representation and L i ( x , ω i ) measures the radiance of incident light with direction ω i .
Revised: L i ( x , ω i )( ω i · n x ) d ω i , where f ( x , ω o , ω i ) is the BRDF representation and L i ( x , ω i ) measures the radiance of incident light with direction ω i .

Original: Generally, the incident light could be categorized as direct vs. indirect light.
Revised: Generally, the incident light could be categorized as direct vs. indirect light.

Original: Fig.
Revised: Fig.

Original: 1 illustrates an example of direct light (path (a), (b)) and indirect light (path (c)) in human face skin, where subsurface scattering happens in the deeper dermis layer, causing the nearby area to receive indirect light.
Revised: 1 illustrates an example of direct light (path (a), (b)) and indirect light (path (c)) in human face skin, where subsurface scattering happens in the deeper dermis layer, causing the nearby area to receive indirect light.

Original: Therefore, the rendering formulation can be split into separate components with direct and indirect lighting:
Revised: Therefore, the rendering formulation can be split into separate components with direct and indirect lighting:

Original: [Equation]
Revised: [Equation]

Original: where f s , f ss represent the BRDF evaluation of surface and subsurface, respectively.
Revised: where f s , f ss represent the BRDF evaluation of surface and subsurface, respectively.

Original: Following the render equation, we design a lightweight physically-based rendering method by learning different
Revised: Following the render equation, we design a lightweight physically-based rendering method by learning different

Original: BRDF representations and modeling the direct and indirect lights, as we will introduce in the next few sections.
Revised: BRDF representations and modeling the direct and indirect lights, as we will introduce in the next few sections.

Original: 3. I LLUMINATION AND B IDIRECTIONAL R EFLECTANCE D ISTRIBUTION L EARNING
Revised: 3. I LLUMINATION AND B IDIRECTIONAL R EFLECTANCE D ISTRIBUTION L EARNING

Original: We propose a lighting model and material model to construct the light sampling field and estimate the BRDF representations, as will be introduced in the next several sections.
Revised: We propose a lighting model and material model to construct the Light Sampling Field and estimate the BRDF representations, as will be introduced in the next several sections.

Original: 3.2. L IGHTING M ODEL
Revised: 3.2. L IGHTING M ODEL

Original: Considering the interaction between light and different skin layers, we propose a novel light modeling method using HDRIs that decomposes lighting into direct illumination and indirect illumination.
Revised: Considering the interaction between light and different skin layers, we propose a novel light modeling method using HDRIs that decomposes lighting into direct illumination and indirect illumination.

Original: For direct illumination, We use light importance sampling to simulate external light sources, such as light bulbs.
Revised: For direct illumination, We use light importance sampling to simulate external light sources, such as light bulbs.

Original: And we implement ray tracing for the specular reflectance effect.
Revised: And we implement ray tracing for the specular reflectance effect.

Original: For indirect illumination, we introduce a light sampling field that models location-aware illumination using SH.
Revised: For indirect illumination, we introduce a Light Sampling Field that models location-aware illumination using SH.

Original: This learned local light-probe models subsurface scattering and inter-reflectance effects.
Revised: This learned local light-probe models subsurface scattering and inter-reflectance effects.

Original: [Figure]
Revised: [Figure]

Original: Importance Light Sampling for Direct Illumination.
Revised: Importance Light Sampling for Direct Illumination.

Original: Direct radiance comes from  HDRI map directly.
Revised: Direct radiance comes from the HDRI map directly.

Original: In order to compute  contribution of pixels in HDRI to different points in our radiance field, we use a SkyDome to represent direct illumination by projecting the HDRI environment map onto a sphere.
Revised: In order to compute the contribution of pixels in HDRI to different points in our radiance field, we use a SkyDome to represent direct illumination by projecting the HDRI environment map onto a sphere.

Original: Each pixel on the sphere is regarded as a distant directional light source.
Revised: Each pixel on the sphere is regarded as a distant directional light source.

Original: Hence, direct lighting is identical at all locations in the radiance field.
Revised: Hence, direct lighting is identical at all locations in the radiance field.

Original: Such representation preserves specular reflection usually achieved by ray-tracing methods.
Revised: Such representation preserves specular reflection usually achieved by ray tracing methods.

Original: We take two steps to construct the representation.
Revised: We take two steps to construct the representation.

Original: Firstly, we uniformly sample a point grid of size N = 800 on the sphere, where each point is a light source candidate.
Revised: Firstly, we uniformly sample a point grid of size N = 800 on the sphere, where each point is a light source candidate.

Original: Secondly, we apply our importance light sampling method to filter valid candidates by two thresholds: 1) an intensity threshold that clips  intensity of outliers with extreme values; 2) an importance threshold that filters out outliers in textureless regions.
Revised: Secondly, we apply our importance light sampling method to filter valid candidates by two thresholds: 1) an intensity threshold that clips the intensity of outliers with extreme values; 2) an importance threshold that filters out outliers in textureless regions.

Original: Fig.
Revised: Fig.

Original: 2a illustrates this process.
Revised: 2a illustrates this process.

Original: [Figure]
Revised: [Figure]

Original: Light Sampling Field for Indirect Illumination.
Revised: Light Sampling Field for Indirect Illumination.

Original: Indirect illumination models the incoming lights reflected or emitted from surrounding objects, which is achieved by ray-tracing with the assumption of limited bounce times in the traditional PBR pipeline.
Revised: Indirect illumination models the incoming lights reflected or emitted from surrounding objects, which is achieved by ray tracing with the assumption of limited bounce times in the traditional PBR pipeline.

Original: Inspired by the volumetric lightmaps used in Unreal Engine (Karis & Games, 2013), which stores precomputed lighting in sampled points and use them for interpolation at runtime for modeling indirect lighting of dynamic and moving objects. We adopt a continuous light sampling field for accurately modeling the illumination variation in different scene positions.
Revised: Inspired by the volumetric lightmaps used in Unreal Engine (Karis & Games, 2013), which stores precomputed lighting in sampled points and use them for interpolation at runtime for modeling indirect lighting of dynamic and moving objects, We adopt a continuous Light Sampling Field for accurately modeling the illumination variation in different scene positions.

Original: We use Spherical Harmonics (SH) to model the total incoming lights at each sampled location separately.
Revised: We use Spherical Harmonics (SH) to model the total incoming lights at each sampled location separately.

Original: We compute the local SH by multiplying fixed Laplace’s SH basis with predicted SH parameters.
Revised: We compute the local SH by multiplying fixed Laplace’s SH basis with predicted SH parameters.

Original: Specifically, we use SH of degree l = 1 for each color channel (RGB).
Revised: Specifically, we use SH of degree l = 1 for each color channel (RGB).

Original: Therefore, we acquire 3 (color channels) × 4 (basis) = 12 -D vector as local SH representation.
Revised: Therefore, we acquire 3 (color channels) × 4 (basis) = 12 -D vector as local SH representation.

Original: We downsample the HDRI map to 100 × 150 resolution and project it to a sphere.
Revised: We downsample the HDRI map to 100 × 150 resolution and project it to a sphere.

Original: Each pixel on the map is considered as a input lighting source.
Revised: Each pixel on the map is considered an input lighting source.

Original: We use the direction and color of each pixel as the lighting embedding to feed into a light field sampling network for inference of coefficients of local SH.
Revised: We use the direction and color of each pixel as the lighting embedding to feed into a light field sampling network for inference of coefficients of local SH.

Original: We visualize our light sampling field with selected discrete sample points in Fig.
Revised: We visualize our Light Sampling Field with selected discrete sample points in Fig.

Original: 2b.
Revised: 2b.

Original: Learning Light Sampling Field.
Revised: Learning Light Sampling Field.

Original: We design a network (Fig. 3b) to predict the spherical Harmonics coefficient C mk of a continuous light field.
Revised: We design a network (Fig. 3b) to predict the spherical Harmonics coefficient C mk of a continuous light field.

Original: The inputs of this network are the lighting embedding z l , the positional encoding of 3D location x  and view direction ω .
Revised: The inputs of this network are the lighting embedding z l , the positional encoding of 3D location x , and view direction ω .

Original: Conditioned on the lighting representations, the network succeeds in predicting the accurate and location-aware lighting.
Revised: Conditioned on the lighting representations, the network succeeds in predicting the accurate and location-aware lighting.

Original: Fig.
Revised: Fig.

Original: 6b evaluates our lighting model.
Revised: 6b evaluates our lighting model.

Original: 3.2. MATERIAL MODEL
Revised: 3.2. MATERIAL MODEL

Original: The choice of reflectance parameters usually relies on artists’ tuning and therefore requires high computational power and results in a long production cycle, but sometimeseven not  ideal.
Revised: The choice of reflectance parameters usually relies on artists’ tuning and therefore requires high computational power and results in a long production cycle, but sometimes not very ideal.

Original: To tackle the problem, we propose a lightweight material network (Fig. 3a) to estimate the BRDF parameters including specular strength γ ∈ R and skin scattering η ∈ R 3 through learnable parameters.
Revised: To tackle the problem, we propose a lightweight material network (Fig. 3a) to estimate the BRDF parameters including specular strength γ ∈ R and skin scattering η ∈ R 3 through learnable parameters.

Original: The parameters are crucial to represent the reflectance property of both surface and subsurface.
Revised: The parameters are crucial to represent the reflectance property of both surface and subsurface.

Original: Together with input albedo ρ , we can construct a comprehensive BRDF to model surface reflection and subsurface scattering.
Revised: Together with input albedo ρ , we can construct a comprehensive BRDF to model surface reflection and subsurface scattering.

Original: It consists of a surface specular component f ss , a surface diffuse component f sd , and a subsurface scattering component f sss .
Revised: It consists of a surface specular component f ss , a surface diffuse component f sd , and a subsurface scattering component f sss .

Original: Refer to Sec.
Revised: Refer to Sec.

Original: 3.3 for a detailed light transport and Fig.
Revised: 3.3 for a detailed light transport and Fig.

Original: for an evaluation of modeling subsurface scattering.
Revised: for an evaluation of modeling subsurface scattering.

Original: 3. L IGHT T RANSPORT
Revised: 3. L IGHT T RANSPORT

Original: Light transport defines a light path from the luminaire to the receiver.
Revised: Light transport defines a light path from the luminaire to the receiver.

Original: We introduce the light transport that connects our lighting model and material model in Fig.
Revised: We introduce the light transport that connects our lighting model and material model in Fig.

Original: We also detail the rendering equation in this section to matching the light transport along the light path.
Revised: We also detail the rendering equation in this section to match the light transport along the light path.

Original: R
Revised: R

Original: [Table]
Revised: [Table]

Original: [Figure]

Original: Volume Casting.
Revised: Volume Casting.

Original: Our lighting model includes direct illumination and indirect illumination.
Revised: Our lighting model includes direct illumination and indirect illumination.

Original: Light transports from explicit lights and casts along the light path.
Revised: Light transports from explicit lights and casts along the light path.

Original: We therefore define the light transport along the light path, p , in the volume by the following:
Revised: We, therefore, define the light transport along the light path, p , in the volume by the following:

Original: [Equation]
Revised: [Equation]

Original: where L ( ˆx , ω o ) is the total radiance at ˆx along the light path direction ω
Revised: where L ( ˆx , ω o ) is the total radiance at ˆx along the light path direction ω

Original: o .
Revised: o .

Original: σ ( x ) is the volume density, which is converted from the input geometry.
Revised: σ ( x ) is the volume density, which is converted from the input geometry.

Original: τ ( x ) = exp( − (cid:82) x 0 σ ( p ( t )) dt ) is the visibility that indicates whether the location x is visible on the light path, where p ( t ) represents the 3D location on the light path along ω o from ˆx and defined as p ( t ) =
Revised: τ ( x ) = exp( − (cid:82) x 0 σ ( p ( t )) dt ) is the visibility that indicates whether the location x is visible on the light path, where p ( t ) represents the 3D location on the light path along ω o from ˆx and defined as p ( t ) =

Original: ˆx + t ω o .
Revised: ˆx + t ω o .

Original: We use explicit geometry to construct a more reliable density field.
Revised: We use explicit geometry to construct a more reliable density field.

Original: More specifically, we locate the intersection coordinates, x 0 , between any arbitrary light path and the input geometry.
Revised: More specifically, we locate the intersection coordinates, x 0 , between any arbitrary light path and the input geometry.

Original: For any point x along the light path ω , we define the density σ as the following central Gaussian distribution function: σ ( x ) =
Revised: For any point x along the light path ω , we define the density σ as the following central Gaussian distribution function: σ ( x ) =

Original: α σ · exp ( − d G ( x ) 2 / (2 δ 2 )) , where d G ( x ) is the distance between x and the intersection x 0 , α σ and δ are two scalars that determine the magnitude and standard deviation of the gaussian distribution.
Revised: α σ · exp ( − d G ( x ) 2 / (2 δ 2 )) , where d G ( x ) is the distance between x and the intersection x 0 , α σ and δ are two scalars that determine the magnitude and standard deviation of the gaussian distribution.

Original: Material Scattering.
Revised: Material Scattering.

Original: The light transport between the scene objects and light sources is characterized by  rendering equation.
Revised: The light transport between the scene objects and light sources is characterized by the rendering equation.

Original: Eqn.
Revised: Eqn.

Original: 1 introduce the rendering covering direct illumination and indirect illumination.
Revised: 1 introduce the rendering covering direct illumination and indirect illumination.

Original: Also, classified by reflection location, the reflected radiance L ( x , ω o ) has two componenets: surface reflectance and subsurface volume scattering.
Revised: Also, classified by reflection location, the reflected radiance L ( x , ω o ) has two components: surface reflectance and subsurface volume scattering.

Original: To have a comprehensive light transport representation, we further develop the equation with the dissection of light as well as the specialized BSSRDF components:
Revised: To have a comprehensive light transport representation, we further develop the equation with the dissection of light as well as the specialized BSSRDF components:

Original: [Equation]
Revised: [Equation]

Original: [Figure] [Equation] here L d i ( x , ω i ) and L id i ( x , ω i ) are the incoming direct and indirect radiance from direction ω
Revised: here L di ( x , ω i ) and L idi ( x , ω i ) are the incoming direct and indirect radiance from direction ω

Original: i at point x , respectively.
Revised: i at point x , respectively.

Original: f ss , f sd  and f sss are the different counterparts of light transport parameterized by material representations.
Revised: f ss , f sd , and f sss are the different counterparts of light transport parameterized by material representations.

Original: We show the complete light transport in Eqn.
Revised: We show the complete light transport in Eqn.

Original: 4 consists of our light and material representation.
Revised: 4 consists of our light and material representation.

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Revised: reflectance

Original: [Table]
Revised: [Table]

Original: direct 
Revised: direct light [Equation] [Table] reflectance

Original: [Table]
Revised: [Table]

Revised: direct light

Original: [Equation]
Revised: [Equation]

Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: where γ x and η x are the predicted specular strength and scattering from material network at x .
Revised: where γ x and η x are the predicted specular strength and scattering from material network at x .

Original: R ( ω i , n x ) denotes the reflection direction of ω i at the surface with normal n x and e denotes the specular exponent.
Revised: R ( ω i , n x ) denotes the reflection direction of ω i at the surface with normal n x and e denotes the specular exponent.

Original: Also, ρ s x and ρ ss x are surface and subsurface albedo at x sampled from the input albedo map correspondingly with geometry.
Revised: Also, ρ s x and ρ ss x are surface and subsurface albedo at x sampled from the input albedo map correspondingly with geometry.

Original:  I MPLEMENTATION D ETAILS
Revised:  I MPLEMENTATION D ETAILS

Original: In order to construct the density field σ , we set α σ and δ to be 10 and 0 . 5 , respectively.
Revised: In order to construct the density field σ , we set α σ and δ to be 10 and 0 . 5 , respectively.

Original: We further compare the rendering results of other values and visualize them in  Appendix.
Revised: We further compare the rendering results of other values and visualize them in the Appendix.

Original: In the constructed radiance field, to sample rays, we draw 1024 random rays per batch.
Revised: In the constructed radiance field, to sample rays, we draw 1024 random rays per batch.

Original: Along each ray, we sample 64 points for the shading model.
Revised: Along each ray, we sample 64 points for the shading model.

Original: The low-frequency location of 3D points and direction of rays are transformed to high-frequency input via positional encoding and directional encoding respectively (Mildenhall et al., 2020).
Revised: The low-frequency location of 3D points and direction of rays are transformed to high-frequency input via positional encoding and directional encoding respectively (Mildenhall et al., 2020).

Original: The length of encoded position and view direction is 37 andrespectively inboth the material network and light sampling field network.
Revised: The length of encoded position and view direction is 37 andrespectively in the material network and the Light Sampling Field network.

Original: Also, importance light sampling takes 800 light samples z ∈ R 3 from the HDRI input for direct lighting, we further downsample the input HDRI and embedded all pixels as a light embedding z l ∈ R 6 × 15000 to feed into the light sampling field network.
Revised: Also, importance light sampling takes 800 light samples z ∈ R 3 from the HDRI input for direct lighting. We further downsample the input HDRI and embedded all pixels as a light embedding z l ∈ R 6 × 15000 to feed into the Light Sampling Field network.

Original: We use an 8-layer MLP with 256 neurons in each layer for both networks.
Revised: We use an 8-layer MLP with 256 neurons in each layer for both networks.

Original: For the material network, encoded sample locations are fed in the first layer of the MLP, while the encoded view direction are later fed in layer 4.
Revised: For the material network, encoded sample locations are fed in the first layer of the MLP, while the encoded view direction is later fed in layer 4.

Original: The output of material MLP for each queried 3D point is specular strength γ ∈ R , and scattering η ∈ R 3 .
Revised: The output of material MLP for each queried 3D point is specular strength γ ∈ R , and scattering η ∈ R 3 .

Original: The light sampling field network have a similar network structure, but also have direct light embedding in layer 4 as input and outputs encoded spherical coefficients C m k ∈ R 12 for indirect lighting.
Revised: The Light Sampling Field network has a similar network structure but also has direct light embedding in layer 4 as input and outputs encoded spherical coefficients C mk ∈ R 12 for indirect lighting.

Original: During light transport, we obtain a weighted value for each ray based on τ distribution among the sampled points along the ray.
Revised: During light transport, we obtain a weighted value for each ray based on τ distribution among the sampled points along the ray.

Original: After introducing values from pre-processed albedo and normal maps, the value of each component on rays is gathered and visualized as an image with pixels representing their intensities following our rendering Eqn. 4.
Revised: After introducing values from pre-processed albedo and normal maps, the value of each component on rays is gathered and visualized as an image with pixels representing their intensities following our rendering Eqn. 4.

Original: Finally, the rendered RGB values are constrained with ground truth by an MSE loss.
Revised: Finally, the rendered RGB values are constrained with ground truth by an MSE loss.

Original: In our application, MLP modules can converge in 50 , 000 iterations ( 2 . 6 hour) on a single Tesla V100, with decent results on the same level of detail as reference images.
Revised: In our application, MLP modules can converge in 50 , 000 iterations ( 2 . 6 hour) on a single Tesla V100, with decent results on the same level of detail as reference images.

Original:  E XPERIMENTS AND E VALUATION 5. T RAINING D ATASET
Revised:  E XPERIMENTS AND E VALUATION 5. T RAINING D ATASET

Original: Our training dataset is composed of a synthetic image dataset and a Lightstage-scanned image dataset.
Revised: Our training dataset is composed of a synthetic image dataset and a Lightstage-scanned image dataset.

Original: In synthetic dataset , we used a professionally-tuned Maya face shader to render 40-view colored images under all combinations between 21 face assets and 101 HDRI+86 OLAT illumination.
Revised: In synthetic dataset , we used a professionally-tuned Maya face shader to render 40-view colored images under all combinations between 21 face assets and 101 HDRI+86 OLAT illumination.

Original: Lightstage-scan dataset consists of 16-view captured colored images of 48 subjects inexpressions under white illumination.
Revised: Lightstage-scan dataset consists of 16-view captured colored images of 48 subjects inexpressions under white illumination.

Original: We carefully selected subjects in both dataset preparation to cover diverse age, skin color and gender.
Revised: We carefully selected subjects in both dataset preparation to cover diverse ages, skin colors and gender.

Original: Further details can be found in Appendix.
Revised: Further details can be found in Appendix.

Original: A.
Revised: A.

Original: [Figure]

Original:  E VALUATION AND A NALYSIS
Revised: 5. E VALUATION AND A NALYSIS

Original: [Figure]
Revised: [Figure]

Original: Material Model Evaluation.
Revised: Material Model Evaluation.

Original: We conduct a comparison experiment with only surface scattering (BRDF) in Fig.
Revised: We conduct a comparison experiment with only surface scattering (BRDF) in Fig.

Original: 5, which presents two BRDF materials (middle two columns) and our proposed material with layer-by-layer decomposition (right four columns).
Revised: 5, which presents two BRDF materials (middle two columns) and our proposed material with layer-by-layer decomposition (right four columns).

Original: From the comparison, Cook-Torrance BRDF has  more specular effect than Lambertian yet saving rubber-alike appearance from  uncanny valley.
Revised: From the comparison, CookTorrance BRDF has a more specular effect than Lambertian yet saves rubber-alike appearance from the uncanny valley.

Original: Apart from surface scattering, our method also predicts subsurface scatteringin material to achieve  vivid looking around  nose tip and ears by depicting red blood cell and vessel color.
Revised: Apart from surface scattering, our method also predicts subsurface scattering to achieve a vivid look around the nose tip and ears by depicting red blood cells and vessel color.

Original: [Figure]
Revised: [Figure]

Original: (a) Rendering results using proposed materials and location-aware local SH as lighting model.
Revised: (a) Rendering results using proposed materials and location-aware local SH as lighting model.

Original: (b) Rendering results using proposed materials with global SH.
Revised: (b) Rendering results using proposed materials with global SH.

Original: (c) Rendering results with global SH and without using  subsurface scattering layer in  material.
Revised: (c) Rendering results with global SH and without using the subsurface scattering layer in the material.

Original: We also demonstrate the inter-reflection effects in the zoom-in boxes.
Revised: We also demonstrate the inter-reflection effects in the zoom-in boxes.

Original: Compare to (c) and (b), (a) achieves soft shadows.
Revised: Compare to (c) and (b), (a) achieves soft shadows.

Original: Lighting Model Evaluation.
Revised: Lighting Model Evaluation.

Original: We show our captured real data under a real fixed HDRI illumination in Fig.
Revised: We show our captured real data under a real fixed HDRI illumination in Fig.

Original: 6a.
Revised: 6a.

Original: Fig.
Revised: Fig.

Original: 6b illustrates uniform white ball illumination within a scene rendered by SH lighting (degree l = 1 ), Spherical Gaussian (SG) Neural-PIL(Boss et al., 2021) and our method, respectively.
Revised: 6b illustrates uniform white ball illumination within a scene rendered by SH lighting (degree l = 1 ), Spherical Gaussian (SG) Neural-PIL(Boss et al., 2021) and our method, respectively.

Original: Compared with other models, our proposed method delivers the highest fidelity of illumination with the widest spectrum of light as well as a lighting field sensitive to lighting distribution.
Revised: Compared with other models, our proposed method delivers the highest fidelity of illumination with the widest spectrum of light as well as a lighting field sensitive to lighting distribution.

Revised: We further provide an extensive ablation study to validate our light sampling for modeling direct illumination in Fig. 15.

Original: Evaluation of Light and Material Modeling in indirect illumination.
Revised: Evaluation of Light and Material Modeling in indirect illumination.

Original: We evaluate our light and material components in Fig.
Revised: We evaluate our light and material components in Fig.

Original: In (C), we use pre-calculated SH of degree l = 1 to model the global illumination and albedo as a diffuse scattering to render the face, resulting in a face image with strong shadows and an unnatural appearance.
Revised: In (c), we use pre-calculated SH of degree l = 1 to model the global illumination and albedo as a diffuse scattering to render the face, resulting in a face image with strong shadows and an unnatural appearance.

Original: In (b), we introduce learned subsurface scattering in material but still use the same pre-calculated SH of degree l =as global illumination, which results in color shift and artifacts.
Revised: In (b), we introduce learned subsurface scattering in material but still use the same pre-calculated SH of degree l =as global illumination, which results in color shift and artifacts.

Original: In (c), we further introduce local SH and infer a light sampling field to replace the pre-calculated SH.
Revised: In (c), we further introduce local SH and infer a light sampling field to replace the pre-calculated SH.

Original: Together with the full spectrum of material layers, we achieve realistic rendering effects.
Revised: Together with the full spectrum of material layers, we achieve realistic rendering effects.

Original: In particular, we demonstrate interreflection effects in the zoom-in box.
Revised: In particular, we demonstrate interreflection effects in the zoom-in box.

Original: The shadow is softened by modeling the sss and positional illumination.
Revised: The shadow is softened by modeling the scattering and positional illumination.

Original: Inverse Rendering.
Revised: Inverse Rendering.

Original: Our method can also achieve highfidelity inverse rendering withonly multi-view images (under various illumination) instead of geometry and texture maps as input asset.
Revised: Our method can also achieve highfidelity inverse rendering with multi-view images (under various illumination) instead of geometry and texture maps as input.

Original: To make this possible, we additionally implemented MLP to predict a density field.
Revised: To make this possible, we additionally implemented MLP to predict a density field.

Original: We present our results under novel illumination in Fig. 8.
Revised: We present our results under novel illumination in Fig

Original: 5. Q UALITATIVE R ESULTS
Revised: 5. Q UALITATIVE R ESULTS

Revised: Rendering on General Object Assets.

Revised: We picked orange and meat in addition to face subjects to show that our method is generalizable on diverse objects with multi-layered structures in Fig.

Revised: Through testing on different organic materials, we show consistent sharpness and realistic appearance, especially in accomplishing specular reflection on orange and accurate soft shadow on meat.

Original: Qualitative Comparison.
Revised: Maya Comparison.

Revised: We compare our renderings with Maya, an industry-level rendering engine, under HDRI or OLAT (one-light-at-a-time) illumination and present zoom-in pore-level details in Fig.

Original: In Fig.
Revised: The zoom-in inspections show comparable or even better [Figure] rendering results in their sharpness or illumination. Skin wrinkles and forehead specularity are particularly rich and sharp from the proposed method.

Original: 10, we compare the  rendering results of FRF, NeLF, SIPR,
Revised: At testing time, with the same rendering assets and queries (2500 × 2500 in resolution), our method requires the training images with only 800 × 800 in resolution.

Original: Neural-PIL and our methods under HDRI and OLAT (one-light-at-a-time) illuminations.
Revised: Under OLAT settings, our method casts hard shadows and soft shadows as accurately as Maya.

Original: We also show results using multi-view images instead of geometry in input. Our method not only casts clear hard shadow due to full occlusion by face geometry, but also showcases soft shadow caused by indirect illumination. Neural-PIL and NeLF neither model directional light nor were trained on OLAT data so we compare them under HDRI illumination only.
Revised: More comparisons and qualitative results can be found in Fig.

Original: Regarding functionality, SIPR is an image-based relighting method that models scene in 2D; FRF, Neural-PIL and NeLF model in 3D. SIPR cannot be queried from novel viewpoints. Neural-PIL inherits per-scene-per-train manner as NeRF and thus, is not generalizable on different subjects.
Revised: 17 and Fig.

Original: We also provide rendering by Maya – a top-notch industrial renderer – as reference. We present more qualitative results in Fig. 15, zoom- [Figure]
Revised: Qualitative Comparison.

Original: HDRI

Original: [Figure] HDRI HDRI NeLF
Revised: In Fig.

Original: SIPR

Original: Neural-PIL

Original: Maya

Original: Ours

Original: Ours* (a) HDRI illumination

Original: OLAT

Original: OLAT

Original: OLAT OLAT FRF  SIPR 
Revised: 16, we compare the rendering results of FRF (Tewari et al., 2021), NeLF (Sun et al., 2021), SIPR (Sun et al., 2019),

Original: Ours Maya ( [Figure] [Figure] illumination in comparison for both HDRI and OLAT illumination in Fig.
Revised: Neural-PIL (Boss et al., 2021), and our methods under HDRI and OLAT illuminations.

Original: 16 and testing performance of our trained-once models on other novel subjects in other datasets in Fig.17.
Revised: We present more testing performance of our trainedonce models on other novel subjects in other datasets in Fig.19.

Original: Rendering on General Object Assets.

Original: We picked orange and meat in addition to face subjects to show that our method is generalizable on diverse objects with multi-layered structure in Fig.

Original: Through testing on different organic material, we show consistent sharpness and realistic appearance, especially in accomplishing difficult specular reflection on orange and accurate soft shadow on meat.

Original: 5. Q UANTITATIVE R ESULTS
Revised: 5. Q UANTITATIVE R ESULTS

Original: We evaluated PSNR, SSIM and LPIPS in quantitative measurements (Table 1) with Maya rendering as  benchmark.
Revised: We evaluated PSNR, SSIM, and LPIPS in quantitative measurements (Table 1) with Maya rendering as a benchmark.

Original: Specifically, LPIPS evaluates  perception on  VGG-19 network.
Revised: Specifically, LPIPS evaluates the perception on the VGG-19 network.

Original: Ours outperforms other baseline methods in all three metrics.
Revised: Ours outperforms other baseline methods in all three metrics.

Revised: Moreover, We do not provide the SSIM score of

Revised: NeLF due to the slight view difference.

Original: Baseline
Revised: Baseline

Revised: FRF

Original: [Table]
Revised: [Table]

Revised: Neural

Original: [Table]
Revised: [Table]

Original: [Table] 
Revised: [Table] [Table] [Table] [Table]

Original: [Table]
Revised: [Table]

Original: LPIPS (VGG)
Revised: SSIM [Table] [Table] [Table]

Revised: N/A 0.

Original: [Table]
Revised: [Table]

Original: due to slight view difference.
Revised: LPIPS (VGG) ↓ 0.

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: We demonstrate that the prior neural rendering representation for physically-based rendering fail to accurately model environment lighting or capture subsurface details.
Revised: We demonstrate that the prior neural rendering representation for physically-based rendering fails to accurately model environment lighting or capture subsurface details.

Original: In this work, we propose a differentiable light sampling field network that models dynamic illumination and indirect lighting in a lightweight manner.
Revised: In this work, we propose a differentiable light sampling field network that models dynamic illumination and indirect lighting in a lightweight manner.

Original: In addition, we propose a flexible material network that models subsurface scattering for complicated materials such as  human face.
Revised: In addition, we propose a flexible material network that models subsurface scattering for complicated materials such as the human face.

Original: Experiments on both synthetic and realworld datasets demonstrate that our light sampling field and material network collectively improve the rendering quality under complicated illumination compared with prior works.
Revised: Experiments on both synthetic and realworld datasets demonstrate that our light sampling field and material network collectively improve the rendering quality under complicated illumination compared with prior works.

Original: In the future, we will focus on modeling more complicated materials such as translucent materials and participated media.
Revised: In the future, we will focus on modeling more complicated materials such as translucent materials and participated media.

Original: We will also collect datasets based on general objects and apply them for extensive tasks such as inverse rendering. 
Revised: We will also collect datasets based on general objects and apply them for extensive tasks such as inverse rendering. A.  [Figure]

Revised: Face Capture.

Revised: We use the Light Stage (Debevec, 2012) to capture data for training purposes.

Revised: The

Original: A. T RAINING D ATASET
Revised: Light Stage features controllable lights and cameras, allowing us to capture multiview images using polarized spherical gradient illumination (Ma et al., 2007; Ghosh et al., 2011). By decomposing the specular and diffuse surface reflections from the captured images, we can generate image space diffuse albedo and high-frequency normals. We further fit the reconstructed 3D face to our template mesh for consistent UV space textures. OLAT Mapping.

Revised: To simulate OLAT illumination, we approximate a directional light source by using an area light source positioned on a sphere.

Revised: We then convert this area light into a High

Revised: Dynamic Range Image (HDRI) map using equirectangular mapping.

Revised: We illustrate the segmentations and corresponding OLAT mapping in Fig.

Revised: A. T RAINING D ATASET

Original: Our training dataset consists of 1) a Light Stage-scanned multi-view colored image dataset under white illumination (Fig.
Revised: Our training dataset consists of 1) a Light Stage-scanned multi-view colored image dataset under white illumination shown in Fig.

Original: 12 (a)), 2) a synthetic multi-view colored image dataset rendered via a professionally-tuned Maya face shader (Fig.
Revised: 12 (a), 2) a synthetic multi-view colored image dataset rendered via a professionally-tuned Maya face shader shown in Fig.

Original: 12 (b)).
Revised: 12 (b).

Original: In the following sections, we introduce data composition and settings for synthetic data rendering and lightstage scanning.
Revised: In the following sections, we introduce data composition and settings for synthetic data rendering and lightstage scanning.

Original: Synthetic Image Dataset Our input rendering assets to Maya renderer are 101 HDRI environment maps, 86 OLAT environment maps and 21 face assets: the HDRI data covers various illuminations from outdoor open area to small indoor area; OLAT environment map cover 86 different regions in SkyDome as directional light; 21 subjects cover a variety of skin color, age and gender.
Revised: Synthetic Image Dataset Our input rendering assets to Maya renderer are 101 HDRI environment maps, 86 OLAT environment maps, and 21 face assets: the HDRI data covers various illuminations from outdoor open areas to small indoor areas; OLAT environment map cover 86 different regions in SkyDome as directional light; 21 subjects cover a variety of skin color, age, and gender.

Original: Each face asset consists of a coarse mesh, an albedo map, and a high-frequency normal map.
Revised: Each face asset consists of a coarse mesh, an albedo map, and a high-frequency normal map.

Original: We renderedfixed-view RGBA-space images under all combinations of illumination and face assets.
Revised: We rendered 40 fixed-view RGBA-space images under all combinations of illumination and face assets.

Original: In total, we acquired 37 , 160 images in 800 × 800 -pixel resolution for our synthetic image dataset.
Revised: In total, we acquired 37 , 160 images in 800 × 800 -pixel resolution for our synthetic image dataset.

Original: Lightstage-scanned Real Image Dataset Synthetic data is insufficient to train a model with output rendering close to the real world.
Revised: Lightstage-scanned Real Image Dataset Synthetic data is insufficient to train a model with output rendering close to the real world.

Original: We utilized lightstage to additionally capture multi-view images for 48 subjects under uniform white illumination.
Revised: We utilized lightstage to additionally capture multi-view images for 48 subjects under uniform white illumination.

Original: Besides  diverse skin colors, gender, and age of subjects, the lightstage-scanned dataset consists of 27 expressions of each subject.
Revised: Besides the diverse skin colors, gender, and age of subjects, the lightstage-scanned dataset consists of 27 expressions of each subject.

Original: We set up 16 fixed cameras covering different viewpoints of the frontal view.
Revised: We set up 16 fixed cameras covering different viewpoints of the frontal view.

Original: In total, we acquired 20 ,images in 3008 × 4096 -pixel resolution for our real image dataset.
Revised: In total, we acquired 20 ,images in 3008 × 4096 -pixel resolution for our real image dataset.

Original: B M ORE RESULTS AND ANALYSIS
Revised: B M ORE RESULTS AND ANALYSIS

Original: Wepresent
Revised: We

Original: Density Field Construction.
Revised: Density Field Construction.

Original: We show how α σ and δ affect the constructed density field in Fig. 13.
Revised: We show how α σ and δ affect the constructed density field in Fig. 13.

Original: Larger δ results  a more evenly distributed density at samples along the light path.
Revised: Larger δ results in a more evenly distributed density at samples along the light path.

Original: The constructed density field hence present a coarse boundary around the input geometry.
Revised: The constructed density field hence presents a coarse boundary around the input geometry.

Original: Therefore, the rendered results tend to have  blurry appearance.
Revised: Therefore, the rendered results tend to have a blurry appearance.

Original: However, smaller δ approaches Dirac delta distribution, where only samples close to the intersections have valid density values.
Revised: However, smaller δ approaches Dirac delta distribution, where only samples close to the intersections have valid density values.

Original: The rendered results thus have black stripes.
Revised: The rendered results thus have black stripes.

Original: Larger α σ alleviates the false density construction and results  accurate density values around the input geometry.
Revised: Larger α σ alleviates the false density construction and results in accurate density values around the input geometry.

Original: More Qualitative Results.
Revised: High-frequency Surface Normal. To investigate the effectiveness of high-frequency surface normal as input, we compare results by replacing high-frequency input normal maps with a low-frequency one extracted from coarse geometry (Fig. 14). Given other input assets unchanged, rendered results with low-frequency normal maps show the incapability of extracting high-frequency information from input images. Therefore, high-frequency normal maps are crucial in achieving pore-level sharpness even though training data contains high-frequency details on images.

Revised: Ablation Study on Light Sampling

Revised: We validate the Importance Light Sampling for simulating various external light sources with an extensive ablation study in Fig.

Revised: Under the same illumination conditions, the Importance

Revised: Light Sampling can generate soft and appropriate diffuse reflection while preserving the accurate lighting distribution from the input HDRI.

Revised: It ensures that the rendered images maintain the high degree of realism and fidelity to the original lighting conditions.

Revised: In contrast, Uniform

Revised: Spherical Sampling, while capable of representing the lighting environment with the same number of sampled lights, tends to produce hard shadows and may result in less detailed and overexposed images.

Revised: Methods

Revised: Input

Revised: Output

Revised: Novel

Revised: Subjects

Revised: Novel

Revised: Viewpoint Views

Revised: Known

Revised: Lighting

Revised: Directional

Revised: Light

Revised: HDRI

Revised: SIPR

Revised: Single-view

Revised: SH ✓

Revised: FRF

Revised: Single-view

Revised: OLAT ✓ ✓ ✓

Revised: Neural-PIL

Revised: Single-view

Revised: OLAT ✓ ✓ ✓ ✓

Revised: NeLF

Revised: Multi-view ✓ ✓

Revised: Ours

Revised: Multi-view*

Revised: HDRI ✓ ✓ ✓ ✓

Revised: Qualitative Comparison.

Revised: In Fig.

Revised: 16, we compare the rendering results of FRF, NeLF, SIPR,

Revised: Neural-PIL, and our methods under HDRI and OLAT (one-light-at-a-time) illuminations, as well as using multi-view images as input instead of geometry.

Revised: To be more concrete, we identify different methods’ input, output, and functionality in Table 2.

Revised: Our method stands out for its ability to produce clear hard shadows resulting from full occlusion by face geometry, and soft shadows caused by indirect illumination.

Revised: In contrast, Neural-PIL and NeLF do not model directional light and were not trained on OLAT data, so we compare them only under HDRI illumination.

Revised: SIPR is an imagebased relighting method that models the scene in 2D, and cannot be queried from novel viewpoints.

Revised: FRF, Neural-PIL, and NeLF, on the other hand, models in 3D. Neural-PIL inherits a per-scene-pertrain manner as NeRF and, thus is not generalizable on different subjects.

Revised: In addition, we provide rendering results from Maya, a top-notch industrial renderer, as a reference for comparison.

Revised: More Qualitative Results.

Original: In Fig.
Revised: Fig.

Original: 15, we provide comprehensive  qualitative results.
Revised: 18 presents a comprehensive collection of qualitative results.

Original: For each row, we exhibit rendering inputs including  geometry, albedo map,object normal map and  followed by six rendered results.
Revised: Each row showcases the rendering inputs, including the geometry, albedo map, and normal map, followed by six rendered images.

Original: The first three are different  expressions under all white illumination, and the last three are neutral expression under different illuminations.
Revised: The first three show different facial expressions under all-white illumination, while the last three display neutral expressions under different lighting conditions.

Original: With the proposed photo-realistic neural renderer, we can perform rendering at anyarbitrary resolution while preserving the rendering quality.
Revised: By utilizing our photo-realistic neural renderer, we are able to render images at any resolution without compromising quality.

Original: Maya Comparison.

Original: We further compare our rendering with Maya, industry-level rendering engine, under HDRI or OLAT illumination and present zoom-in pore-level details in Fig.

Original: The zoom-in inspections show comparable or even better rendering results in their sharpness or illumination.

Original: Skin wrinkles and forehead specularity are particularly rich and sharp from the proposed method.

Original: At testing time, with the same rendering assets and queries (2500 × 2500 in resolution), our method requires the training images with only 800 × 800 in resolution.

Original: Under OLAT settigns,

Original: Our method casts hard shadow and soft shadow as accurate as Maya.

Original: Rendering Speed.
Revised: Rendering Speed.

Original: In addition to the rendering results, we also compare the rendering speed in
Revised: In addition to the rendering results, we also compare the rendering speed in

Original: Table 2.
Revised: Table 3.

Original: With the same specification of output and environment, our method is able to achieve up to 47-49 times faster with engineering acceleration (e.g. multi-thread processing) 
Revised: With the same specification of output and environment, our method is able to achieve up to 47-49 times faster with engineering acceleration (e.g. multi-thread processing) FRF [Figure] HDRI

Revised: HDRI

Original: Resolution
Revised: HDRI NeLF SIPR Neural-PIL

Revised: Maya

Revised: Ours

Revised: Ours* (a) HDRI illumination

Revised: OLAT

Revised: OLAT

Revised: OLAT

Revised: OLAT

Original: [Table]
Revised: FRF SIPR Ours Maya (b) OLAT illumination Resolution 800 ×1440 ×2880 ×5760 ×

Revised: Maya 130s 370s 1857s 7848s

Revised: Ours 3s 8s 31s 158s

Original: Testing on Other Datasets.
Revised: Testing on Other Datasets.

Original: To demonstrate that high-quality training data does not determine  robustness of  method, we evaluated our trained model over other available resources in Fig.
Revised: To demonstrate that high-quality training data does not determine the robustness of the method, we evaluated our trained model over other available resources in Fig.

Original: We converted three datasets to match our input as following,
Revised: We converted three datasets to match our input as follows,

Original: • Triplegangers.
Revised: • Triplegangers.

Original: We used FaceX 3DMM to fit and align geometries; albedo map was directly transferred from source; normal map was inferred using (Wang et al., 2018).
Revised: We used FaceX 3DMM to fit and align geometries; albedo map was directly transferred from source; normal map was inferred using ( Wang et al., 2018)

Original: • 3D Scan Store.
Revised: • 3D Scan Store.

Original: We used 3DMM to fit and align geometries; albedo and normal maps were provided and not further processed.
Revised: We used 3DMM to fit and align geometries; albedo and normal maps were provided and not further processed.

Original: • FaceScape.
Revised: • FaceScape.

Original: Geometries and albedo maps were provided and not further processed.
Revised: Geometries and albedo maps were provided and not further processed.

Original: Normal map was unavailable and not used.
Revised: The normal map was unavailable and not used.

Original: Our method delivers promising fidelity on all three testing datasets.
Revised: Our method delivers promising fidelity on all three testing datasets.

Original: First, our method is capable of adapting to different mesh topologies.
Revised: First, our method is capable of adapting to different mesh topologies.

Original: For example, meshes in the Triplegangers dataset has denser vertices in the front face than behind while meshes in FaceScape have more uniform density, but our trained-once model performs equally well in both testing datasets.
Revised: For example, meshes in the Triplegangers dataset has denser vertices in the front face than behind while meshes in FaceScape have more uniform density, but our trained-once model performs equally well in both testing datasets.

Original: Second, our method does not sacrifice fidelity when input normal is unavailable during testing.
Revised: Second, our method does not sacrifice fidelity when input normal is unavailable during testing.

Original: Thanks to explicit geometry volume and robust model, pixels in the albedo map are precisely mapped onto the surface and therefore, yield no noise in rendering.
Revised: Thanks to explicit geometry volume and robust model, pixels in the albedo map are precisely mapped onto the surface and therefore, yield no noise in rendering.

Original: Finally, we acquired and tested our pre-trained model over GeneratedPhotos dataset containing only low-frequency albedo and normal map in Fig.
Revised: Finally, we acquired and tested our pre-trained model over the Generated Photos dataset containing only low-frequency albedo and normal map in Fig.

Original: The dataset is generated by some anonymous method with only  single-view online image as input.
Revised: The dataset is generated by some anonymous method with only a single-view online image as input.

Original: Our method not only outputs  clear silhouette but also shows realistic pre-level detail when high-frequency input is unavailable.
Revised: Our method not only outputs a clear silhouette but also shows realistic pre-level detail when high-frequency input is unavailable.

Original: High-frequency Surface Normal.

Original: To investigate the effectiveness of high-frequency surface normal as input, we compare results by replacing high-frequency input normal maps with a lowfrequency one extracted from coarse geometry (Fig. 14).

Original: Given other input assets unchanged, rendered results with low-frequency normal maps show the incapability of extracting high-frequency information from input images.

Original: Therefore, high-frequency normal maps are crucial in achieving pore-level sharpness even though training data contains high-frequency details on images.

Original: [Figure]
Revised: [Figure]