Original: NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo 
Revised: NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo Anonymous Author(s)

Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: Uncalibrated photometric stereo (UPS) is challenging due to the inherent ambiguitybrought by unknown light.
Revised: Uncalibrated photometric stereo (UPS) is challenging due to the inherent shapelight.

Original: Existing solutions alleviate the ambiguity by eitherexplicitly associating reflectance to light conditions or resolving light conditions ina supervised manner.
Revised: Existing solutions alleviate this ambiguity by either explicitly associatingreflectance to light estimation or resolving light conditions in a supervised manner.

Original: This paper establishes an implicit relation between light cluesand light estimation and  solves UPS in an unsupervised manner.
Revised: This paper establishes an implicit relation between light clues and light estimationand solves UPS in an unsupervised manner.

Original: The key idea is torepresent the reflectance as four neural intrinsics fields, i.e. , position, light, specular,and shadow, based on which the neural light field is implicitly associated withlight clues of specular reflectance and cast shadow.
Revised: The key idea is to represent thereflectance as four neural intrinsics fields, i.e , specular, shadow, light, and position,based on which the neural light field is implicitly associated to light clues ofspecular reflectance and cast shadow.

Original: The unsupervisedoptimizationof neural intrinsics fields can be free from training data bias and fully exploitsall observed pixel values for UPS.
Revised: The unsupervised training of neural intrinsicsfields fully exploits all observed pixel values, which facilitates solving UPS.

Original: Our method achieves a superior performanceadvantage over state-of-the-art UPS methods on public datasets and promisingresults under the challenging setting of sparse UPS.
Revised: Ourmethod demonstrates a superior performance advantage over state-of-the-art UPSmethods on public datasets and promising results under the challenging setting ofsparse UPS.

Original: The code will be releasedsoon.
Revised: We will release our codes and pre-trained models once the paper wasaccepted.

Original:  Introduction
Revised:  Introduction

Original: Photometric stereo (PS) [50] aims at recovering the surface normal from several light-varying imagescaptured at a fixed viewpoint.
Revised: Photometric stereo (PS) [47] aims at recovering the surface normal from several light-varying imagescaptured at a fixed viewpoint.

Original: As compared with other  approaches ( e g. , multi-view stereo [41], activesensor-based solutions [58]), photometric stereo is excellent at recovering fine-detailed surfaces andhas been widely used for Hollywood movies [11], industrial quality inspection [49] and biometrics[55].
Revised: As compared with other stereo approaches ( e g. , multi-view stereo [38],active sensor-based solutions [55]), photometric stereo is excellent at recovering fine-detailed surfacesand has been widely used for Hollywood movies [9], industrial quality inspection [46] and biometrics[52].

Original: Calibrating accurate lighting directions is crucial to the performance of photometric stereomethods [54].
Revised: Calibrating accurate lighting directions is crucial to the performance of photometric stereomethods [51].

Original: However, lighting calibration is often tedious, dramatically restricting the applicabilityin the  real-world.
Revised: However, lighting calibration is often tedious, dramatically limiting the applicability inthe real-world.

Original: To this end, researchers develop uncalibrated photometric stereo (UPS) methodsthat estimate surface normal with unknown lights.
Revised: To this end, researchers develop uncalibrated photometric stereo (UPS) methods thatestimate surface normal with unknown lights.

Original: Uncalibrated photometric stereo suffers from General Bas-Relief (GBR) ambiguity [6] for an integrable surface.
Revised: Uncalibrated photometric stereo suffers from General Bas-Relief (GBR) ambiguity [6] for an integrable surface.

Original: Early solutions address the ambiguity by explicitly associating reflectance to light,i.e. , adopting analytic reflectance models ( e.g. , Lambertian reflectance [4], [37], parametric specularreflection [20], specular spikes [56], inter-reflection [12]) or imposing priors from reflectance properties [3, 25, 24, 44].
Revised: Early solutions address the ambiguity by explicitly associating reflectance to light, i.e ,adopting analytic reflectance models ( e g. , Lambertian reflectance [4], [34], parametric specular reflection [18], specular spikes [53], inter-reflection [10]) or imposing priors from reflectance properties[3, 22, 21, 41].

Original: Thus,due to the strong reliance on reflectance assumption, these methods can beless effective for unknown reflectance.
Revised: Thus, these methods could be less effective for unknown reflectance due to the strongreliance reflectance assumption.

Original: Besides, these methods ignore the clue of cast shadow andeven fail in shadow regions due to the shadow-free surface assumption.
Revised: Besides, these methods ignore the clue of cast shadow and even failfor shadow regions due to the shadow-free surface assumption.

Original: Further, most of them requirethe light intensity to be identical for robust estimation.
Revised: Further, most of them require thelight intensity to be identical for robust estimation.

Original: Recently, deep learning-based approachesaddress the ambiguity by estimating light direction and intensity before recovering surface normal [13,15, 27, 40].
Revised: Recently, deep learning-based approaches addressthe ambiguity by estimating light direction and intensity before recovering surface normal [12, 13, 25,37].

Original: They train a light estimation network using a large-scale amount of rendered data in asupervised manner.
Revised: They train a light estimation network using a large-scale amount of rendered data in a supervisedmanner.

Original: However, the training data bias [33] can hardly be eliminated and can produceunexpected estimation for real-world data.
Revised: However, the training data bias [30] can hardly be eliminated and could produce unexpectedestimation for real-world data.

Original: Because rendered training data inevitably contains thedomain gap from the  real ones and  scarcely cover all surfaces with different geometry and reflectancein the real-world.
Revised: Because rendered training data inevitably contain the domain gapfrom real ones and could scarcely cover all surfaces with different geometry and reflectance in thereal-world.

Original: Besides, such a two-step solution can bring accumulating errors for surface normal
Revised: Besides, such a two-step solution could bring accumulating errors for surface normal

Original: Neural Shadow Field
Revised: Neural Shadow Field

Original: P o s i t i on
Revised: P o s i t i on

Original: Light direction ( ) + × × = × × × · · · 𝒏 ! 𝑘 !" 𝜌 !" 𝑘 ! # Neural Position Field
Revised: Light direction ( ) + × × = × × × · · · 𝒏 𝑖 𝑘 𝑖 𝑑 𝜌 𝑖 𝑑 𝑘 𝑖 𝑠 Neural Position Field

Original: P o s i t i on
Revised: P o s i t i on

Original: Neural Specular Field
Revised: Neural Specular Field

Original: S
Revised: S

Original: u r f a c e no r m a l
Revised: u r f a c e N o r m a l

Original: Light direction
Revised: Light direction

Original: Neural Light Field
Revised: Neural Light Field

Original: O b s e r v ed i m age s
Revised: O b s e r v ed I m age s

Original: Light direction · · · 𝜌 !" (𝒍 # ,𝑒 # ) 𝑠 ! # 𝑘 !" 𝑘 !$ 𝜌 !
Revised: Light direction · · · 𝜌 𝑖𝑑 (𝒍 𝑗 ,𝑒 𝑗 ) 𝑠 𝑖𝑗 𝑘 𝑖𝑑 𝑘 𝑖𝑠 𝜌 𝑖𝑗𝑠 𝒏 𝑖 𝑚 𝑖𝑗 · · · · · · · · · · · · · · · · · · position-dependent light- dependent 𝜌 𝑖𝑑 ✓  𝑘 𝑖 𝑠 ✓  𝑘 𝑖𝑑 ✓  𝒏 𝑖 ✓  𝒍 𝒋  ✓ 𝑒 𝑗  ✓ 𝜌 𝑖𝑗𝑠 ✓ ✓ 𝑠 𝑖𝑗 ✓ ✓ estimation.

Original: #$ 𝒏 !

Original: 𝑚 !

Original: Position-Dependent

Original: Light-

Original: Dependent 𝜌 !"

Original: ü û 𝑘 !

Original: # ü û 𝑘 ! "

Original: ü û 𝒏 !

Original: ü û 𝒍 𝒋 û ü 𝑒 % û ü 𝜌 !

Original: # ü ü 𝑠 ! % ü ü estimation. Further, all these methods assume the light intensity distributed in a pre-defined range( i.e. , [0 . 2 , 2] ), restricting their applicability.
Revised: Further, all these methods assume the light intensity to distribute in a pre-defined range( i.e , [0 . 2 , 2] ), limiting their applicability.

Original: To this end, we propose NeIF , representing general reflectance as Ne ural I ntrinsics F ields foruncalibrated photometric stereo.
Revised: To this end, we propose NeIF , representing general reflectance as Ne ural I ntrinsics F ields foruncalibrated photometric stereo.

Original: Our method differs from previous methods in three aspects: 1)it fully considers clues of specular reflectance and cast shadow from each observed pixel for lightestimation so that it is expected to produce accurate estimation for both light conditions and surfacenormal; 2) it does not make explicit assumptions about the reflectance or light so that it works withgeneral surface reflectance and flexible light settings; 3) it infers light and surface normal in anunsupervised manner so that it is free from training data bias and achieves stable performance fordata from different sources.
Revised: Our method differs from previous methods in three aspects: 1)it fully considers clues of specular reflectance and cast shadow from each observed pixel for lightestimation so that it is expected to produce accurate estimation for both light conditions and surfacenormal; 2) it does not make explicit assumptions about the reflectance or light so that it could workwith general surface reflectance and flexible light settings; 3) it infers light and surface normal in anunsupervised manner so that it could be free from training data bias and achieve stable performancefor data from difference sources.

Original: Our key idea is to represent the general reflectance as four neural intrinsics fields ( i.e. , position,light, specular, and shadow,  see Fig. 1), implemented by four multi-layer perceptrons (MLPs).
Revised: Our key idea is to represent the general reflectance as four neural intrinsics fields ( i.e , specularreflectance, cast shadow, light, and position, see Fig. 1), implemented by four multi-layer perceptrons(MLPs).

Original: Thesefour fields are connected based on the implicit relation (or dependence) of these intrinsics so thatno explicit assumption is imposed, e.g. , we take the estimated light as the input to recover specularreflectance and cast shadow instead of explicitly exploiting them for light estimation.
Revised: The connection of these four fields is based on the implicit relation (or dependence) of theseintrinsics so that no explicit assumption is imposed, e.g. , we take the estimated light as the input torecover specular reflectance and cast shadow instead of explicitly exploiting them for light estimation.

Original: These intrinsicsfields are optimized to reconstruct each pixel value from observed images, which fully exploitmutual information across different dimensions, as shown in Fig.
Revised: These intrinsics fields are optimized to reconstruct each pixel value from observed images, whichfully exploit mutual information across different dimensions, as shown in Fig.

Original: The reconstruction error isbackpropagated to the neural light field through neural specular and shadow fields so that clues ofspecular and shadow can be implicitly and fully considered for light estimation.
Revised: The reconstructionerror backpropagates to the neural light field through neural specular and shadow fields, so that cluesof specular and shadow can be implicitly and fully considered for light estimation.

Original: Our contributionsare summarized as:
Revised: Our contributionsare summarized as

Original: • We represent general reflectance as four intrinsic neural fields to implicitly associate per-
Revised: • We represent general reflectance as four intrinsic neural fields to implicitly associate per-

Original: pixel reflectance to light, whichsolves uncalibrated photometric stereo by fully considering clues of specular reflectance and cast shadow for light estimation.
Revised: pixel reflectance to light, which fully considers clues of specular reflectance and cast shadow for light estimation.

Original: • We propose the NeIF, an uncalibrated photometric stereo method trained in an unsupervised
Revised: • We propose the NeIF, an uncalibrated photometric stereo method trained in an unsupervised

Original: manner, which works with general surface reflectance and flexible light settings, and is freefrom training data bias.
Revised: manner, which works with general surface reflectance, flexible light settings and trainingdata bias.

Original: • We show that our method achieves  superior performance over uncalibrated and unsupervisedmethods.
Revised: • We show that our method achieves a superior performance over uncalibrated and unsupervised methods.

Original: We also demonstrate its excellent generalization capacity to data from differentsources and promising performance with the challenging setup of sparse uncalibrated photometric stereo.
Revised: We also demonstrate its excellent generalization capacity to data from different sources and promising performance with the challenging setup of sparse uncalibrated photometric stereo.

Original:  Related Work
Revised:  Related Work

Original: This section mainly reviews the latest works in neural reflectance representation, and related workson unsupervised PS methods and UPS methods.
Revised: In this section, we mainly review the latest works in neural reflectance representation, and relatedworks unsupervised calibrated methods and uncalibrated photometric stereo methods.

Original: Readers may refer to [45] and [13] for a morecomprehensive summary.
Revised: Readers mayrefer to [39] and [12] for a more comprehensive summary.

Original: 2.1 Neural Reflectance Representation
Revised: 2.1 Neural Reflectance Representation

Original: Neural Radiance Fields (NeRFs)
Revised: Neural Radiance Fields (NeRFs)

Original: [34] focus on the 3D geometries without explicitly modeling theinteraction between light and objects via the image formation model.
Revised: [31] focuses on the 3D geometries without explicitly modeling theinteraction between light and objects via photometric image formation model.

Original: Many subsequent worksexplore its application in various computer vision problems, such as relighting [16, 46], photometricstereo [29], and multi-view  stereo [7, 9, 60, 28, 61].
Revised: Many subsequentworks explore its application in various computer vision problems, such as relighting [14], photometricstereo [26], and multi-view photometric stereo [24].

Original: These works  require known lighting conditions [7,46, 61, 29], adopt simple reflectance model [59], or leverage multi-view information [7, 9, 60, 28, 61].
Revised: These works either require known lightingconditions [ bi2022neural , 43, 58, 26] or adopt simple reflectance model [56]. There are alsosome works [7, 57] considering general reflectance, but they require multi-view information anddoes not explicitly consider cast shadow effects. Kaya et al.

Revised: [24] implemented NeRF in multiview photometric stereo, but their work also leverages multi-view information.

Original: Different from these methods, our method considers general reflectance, cast-shadow effects, andunknown light conditions for uncalibrated photometric stereo by taking images captured at a singleviewpoint.
Revised: Different from thesemethods, our method considers general reflectance, cast-shadow effects, and unknown light conditionsfor uncalibrated photometric stereo by taking images captured at a single viewpoint.

Original: 2.2 Unsupervised  Photometric Stereo
Revised: 2.2 Unsupervised Calibrated Photometric Stereo

Original: Classical methods solve the calibrated photometric stereo problem without knowing the groundtruth surface normal.
Revised: Classical methods solved the calibrated photometric stereo problem without knowing the groundtruth surface normal.

Original: Therefore, we classify them as unsupervised methods.
Revised: Therefore, we classify them as unsupervised methods.

Original: The least squarebased algorithm [50] provides the simplest solution, which assumes the object to be Lambertian.
Revised: The least square basedalgorithm [47] provides the simplest solution, which assumes the object to be Lambertian.

Original: It is generally served as a baseline method due to its stability, but its strong assumption on thereflectance model makes it fail for non-Lambertian surface.
Revised: Thealgorithm’s performance is robust in a wide range of datasets, but its strong assumption on thereflectance model remains much space for improvement.

Original: The following works either regard thenon-Lambertian reflectance components as the outliers [5, 18, 36, 51, 26, 52] or apply analyticreflectance models including Torrance-Sparrow [20], the Ward model [18], a mixture of multiple Ward models [21], [1] etc. to consider the non-Lambertian effects.
Revised: The following works either regard thenon-Lambertian reflectance components as the outliers [5, 16, 33, 48, 23, 49] or apply analyticreflectance models including Torrance-Sparrow[18], the Ward model[16], a mixture of multiple Wardmodels [19], [1] etc. to consider the non-Lambertian effects.

Original: However, the performance ofthose methods can only deal with limited types of materials.
Revised: However, the performance of thosemethods can only deal with limited types of materials.

Original: There are also more advanced methodsthat utilize the general refletance features such as reciprocity, isotropy [2], and monotonicity [24].
Revised: There are also more advanced methods thatutilize the general refletance features such as reciprocity, isotropy [2], and monotonicity [21].

Original: Those methods give a reliable estimation for objects with a broad range of materials.
Revised: Thosemethods give a reliable estimation for objects with a broad range of materials.

Revised: Inspired by them, ourapproach considers the general reflectance properties.

Original: With the progress of deep learning, many learning-based frameworks have been proposed for calibrated photometric stereo.
Revised: With the progress of deep learning, many learning-based frameworks have been proposed for calibrated photometric stereo.

Original: Taniai et al.
Revised: Taniai et al.

Original: [47] proposed the first unsupervised learning-based photometric stereo method through a rendering equation.
Revised: [44] propose the first unsupervised learning-based photometricstereo method through a rendering equation.

Original: However, their reflectance model does not separatelyconsider shadow, specular highlights, and diffuse components.
Revised: However, their reflectance model does not separatelyconsider shadow, specular highlights, and diffuse components.

Revised: Different from these methods that need the known lights, we address the uncalibrated photometricstereo.

Original: We also train our method in an unsupervised manner.
Revised: We also train our method in an unsupervised manner.

Original: Different from previous methods, we addressthe challenging problem of UPS and separately model cast shadow, specular reflectance, and diffusereflectance.
Revised: Because it is expected to be free fromtraining data bias and thus obtain a stable estimation across data from different sources.

Original: 2.3 Uncalibrated Photometric Stereo
Revised: 2.3 Uncalibrated Photometric Stereo

Original: Previous works hold the  Lambertian assumption and address the ambiguity brought by a 3 × 3transformation matrix.
Revised: Previous works study the phenomenon where Lambertian objects under arbitrary directions cangenerate identical images when similar transformations are applied to both the object’s normal andthe light’s direction.

Original: Belhumeur et al.
Revised: Belhumeur et al.

Original: [6] reduce the dimension of the transformation to a threeparameters GBR transformation by considering integrability constraints.
Revised: [6] reduce the dimension of the transformation to a threeparameters GBR transformation by considering integrability constraints.

Original: Based on that, extra cluesfrom reflectance, such as half-vector symmetry [31], albedo clustering [42], specular spikes [56], orassumptions of light source distribution, such as ring light [63], symmetry light [35], or uniformdistributed light sources [57, 4, 42, 53, 37, 31], are used to resolve the GBR ambiguity.
Revised: Based on that, extra clueson reflectance, such as half-vector symmetry [28], albedo clustering [42], specular spikes [53], orassumptions on light source distribution, such as ring light [60], symmetry light [32], or uniformdistributed light sources [54, 4, 42, 50, 34, 28], are used to resolve the GBR ambiguity.

Original: However,all these methods require the light intensity to be identical, which is inapplicable in the real-worlddatasets such as D
Revised: However,all these methods require the light intensity to be identical, which is inapplicable in the real-worlddatasets such as D

Original: I L I G EN T [45], A PPLE & G OURD [2], and L IGHT S TAGE D ATA G ALLERY
Revised: I L I G EN T [39], A PPLE & G OURD [2], and L IGHT S TAGE D ATA G ALLERY

Original: Cho et al.
Revised: Choet al.

Original: [17] put up a semi-calibrated method to deal with non-uniform light intensity, but theyassume the light directions to be known.
Revised: [15] put up a semi-calibrated method to deal with non-uniform light intensity, but they assumedthe light directions to be known.

Original: Quéau et al.
Revised: Quéau et al.

Original: [39] address the photometric stereo problemunder inaccurate lighting calibration, while the accuracy can significantly drop when non-Lambertiancomponents become dominant.
Revised: [36] address the photometric stereo problem underinaccurate lighting calibration, while the accuracy could significantly drop when non-Lambertiancomponents become dominant.

Original: Recently, many deep learning methods have been proposed for uncalibrated photometric stereo.
Revised: Recently, many deep learning methods have been proposed for uncalibrated photometric stereo.

Original: Chen et al.
Revised: Chen et al.

Original: [13] propose a supervised uncalibrated framework, SDPS-Net, which can simultaneouslyestimate the light conditions (intensity and direction) and the surface normal.
Revised: [12] propose a supervised uncalibrated framework, SDPS-Net, which can simultaneouslyestimate the light conditions (intensity and direction) and the surface normal.

Original: They suggest treatinglight estimation as a classification problem and separating the normal and light prediction to reducethe complexity.
Revised: They suggest treatinglight estimation as a classification problem and separate the normal and light prediction to reduce thecomplexity.

Original: Their following work, GC-Net [15], improves the performance of SDPS-Net by adding shading as an extra channel to the input of the light estimation network.
Revised: Their following work, GC-Net[13], improved the performance of SDPS-Net by addingshading as an extra channel to the input of the light estimation network.

Original: However, as acommon problem for all supervised methods, an over-fitting problem may occur due to the trainingdata bias [33].
Revised: However, as a commonproblem for all supervised methods, an over-fitting problem may occur due to the training data bias[30].

Original: In contrast, unsupervised methods do not have such a concern.
Revised: In contrast, unsupervised methods do not have such a concern.

Original: Another  benefit isthat there is no need to synthesize training sets for unsupervised network training.
Revised: Another obvious benefit is that there is no need to synthesize training sets for unsupervised network training.

Original: To utilize theadvantage of unsupervised methods, Kaya et al.
Revised: To utilize theadvantage of unsupervised methods, Kaya et al.

Original: [27] propose a compromised method that trains thelight estimation network in a supervised manner (similar to [13]), but estimate the surface normal inan unsupervised way (similar to [47]).
Revised: [25] propose a compromised method that trains thelight estimation network in a supervised manner (similar to [12]), but estimate the surface normal inan unsupervised way (similar to [44].

Original: However, they still suffer from  training data bias during lightestimation.
Revised: However, they still suffer from the training data bias duringlight estimation.

Original: Besides, all these methods make a strict assumption that the light intensity distributed in apre-defined range ( i.e. , [0.2, 2]) and suffers from accumulating error due to their two-step frameworks. In contrast, our method neither makes a strict assumption on reflectance nor needs special light sourcedistribution and jointly solves light conditions and surface normal in an unsupervised manner.
Revised: Compared to previous works, our method neither makes a strict assumption onreflectance nor needs special light source distribution. Instead, we solve the uncalibrated photometricstereo problem in an unsupervised manner for general reflectance surfaces with unknown light sourceinformation.

Original:  Method
Revised:  Method

Original: 3.1 Problem Formulation
Revised: 3.1 Problem Formulation

Original: Given a set of observations I ≜ ( I 0 , I 1 , ..., I f ) of a static surface, illuminated by f unknowndirectional illuminations distributing on the upper-hemisphere, uncalibrated photometric stereo aimsat recovering light directions L ≜ ( l 0 , l 1 , ..., l f ) , light intensities E ≜ ( e 0 , e 1 , ..., e f ) , and surfacenormal N ≜ { n i | i ∈ P } , P is the set of all positions on the surface.
Revised: Given a set of observations I ≜ ( I 0 , I 1 , ..., I f ) of a static surface, illuminated by f unknowndirectional illuminations distributing on the upper-hemisphere, uncalibrated photometric stereo aimsat recovering light directions L ≜ ( l 0 , l 1 , ..., l f ) , light intensities E ≜ ( e 0 , e 1 , ..., e f ) , and surfacenormal N ≜ { n i | i ∈ P } , P is the set of all positions on the surface.

Original: The solution is achieved bysolving the optimization problem,
Revised: The solution is achieved bysolving the optimization problem,

Original: [Equation]
Revised: [Equation]

Original: (1) where ¯ m ij ∈
Revised: ) where ¯ m ij ∈

Original: I j is the observed pixel intensity at position i , # P is the number of elements in P 1 , m ijis the corresponding rendered pixel intensity, D ( · , · ) is a metric describing their difference.
Revised: I j is the observed pixel intensity at position i , # P is the number of elements in P 1 , m ijis the corresponding rendered pixel intensity, D ( · , · ) is a metric describing their difference.

Original: Under anorthographic camera with linear radiometric response, m ij is formulated as (simplified in a per-pixelform),
Revised: Under anorthographic camera with linear radiometric response, m ij is formulated as (simplified in a per-pixelform),

Original: [Equation]
Revised: [Equation]

Original: where v = [0 , 0 , 1] is the view direction pointing toward the viewer, ρ ij describes the generalreflectance, max( n ⊤ i l j , 0) represents the attach shadow.
Revised: where v = [0 , 0 , 1] is the view direction pointing toward the viewer, ρ ij describes the generalreflectance, max( n ⊤ i l j , 0) represents the attach shadow.

Original: Unknown light brings two ambiguities when solving Eq.
Revised: Unknown light brings two ambiguities when solving Eq.

Original: (1), i.e. , shape-light ambiguity, which isdenoted as an invertable matrix G ∈ R 3 × 3 , and reflectance-light ambiguity, which is denoted as anon-zero scalar c j ∈ R ,
Revised: (1), i.e , shape-light ambiguity, which isdenoted as an invertable matrix G ∈ R 3 × 3 , and reflectance-light ambiguity, which is denoted as anon-zero scalar c j ∈ R ,

Original: [Equation]
Revised: [Equation]

Original: 3.2 Neural Intrinsics Fields
Revised: 3.2 Neural Intrinsics Fields

Original: General reflectance decomposition.
Revised: General reflectance decomposition.

Original: To exploit clues of specular reflectance and cast shadow forlight estimation, we decompose the general reflectance as the cast shadow term s ij multiplying thebidirectional reflectance term,
Revised: To exploit clues of specular reflectance and cast shadow forlight estimation, we decompose the general reflectance as the cast shadow term s ij multiplying thebidirectional reflectance term,

Original: [Equation]
Revised: [Equation]

Original: where subscript ‘ i ’ and ‘ j ’ indicate  position- (or normal-) and light-dependent factors, respectively;the cast shadow term s ij is either 0 or 1; ρ di , ρ sij represent the diffuse and specular reflectance, and k diand k si are coefficients that balance out the effects of specular and diffuse reflectance 2 .
Revised: where subscript ‘ i ’ and ‘ j ’ indicate pixel position- (or normal-) and light-dependent factors, respectively, the cast shadow term s ij is either 0 or 1, ρ di , ρ sij represent the diffuse and specular reflectance,and k di and k si are coefficients that balance out the effects of specular and diffuse reflectance 2 .

Original: Neural fields of specular reflectance and cast shadow.
Revised: Neural fields of specular reflectance and cast shadow.

Original: Previous methods identify specular featuresfrom specific pixels and associate an explicit relation to light for light estimation.
Revised: Previous methods identify specular featuresfrom specific pixels and associate an explicit relation to light for light estimation.

Original: However, thisscheme fails for the surface where the specular features are invisible or the explicit relation is violated.
Revised: However, thisscheme could fail for the surface where the specular features are invisible or the explicit relationis violated.

Original: Besides, leaving out the clue of cast shadow can obstruct producing competitive performance.
Revised: Besides, leaving out the clue of cast shadow could obstruct producing competitiveperformance.

Original: In contrast, we leverage both clues of specular and shadow for light estimation, which is achieved bybuilding the neural specular field and neural shadow field, and associating the fields to light conditions.[Figure] PositionNet PE
Revised: In contrast, we leverage both clues of specular and shadow for light estimation, which is achieved bybuilding the neural specular field and neural shadow field and associating the fields to light conditions.

Original: LightNet

Original: [Figure]

Original: SpecularNet

Original: PE

Original: [Figure]

Original: Positional Encode 𝒉 !

Original: =

Original: 𝒍 !

Original: + 𝒗 ||𝒍 !

Original: + 𝒗|| MLPs CNN Encoder 𝑚 "!

Original: = 𝑒 ! 𝑠 "!

Original: 𝑘 "# 𝜌 "# + 𝑘 "$ 𝜌 "!

Original: $ max 𝒏 "% 𝒍 ! , (𝑥,𝑦) 𝒍 !

Original: ShadowNet

Original: PE 𝑘 "$ 𝑘 "# 𝜌 "# 𝒏 " 𝜌 "!

Original: $ 𝒏 "% 𝒉 !

Original: 𝒗 % 𝒉 ! 𝒍 !

Original: 𝑒 ! , ( ) ( [Figure] 𝑠 "! These neural fields make the utmost of all observed pixels, and exploit the mutual information acrossdifferent normal-light (specular) and position-light (shadow) for light estimation.
Revised: These neural fields make the utmost of all observed pixels, which fully exploits the mutual informationacross different normal-light (specular) and depth-light (shadow).

Original: Two MLPs, namely SpecularNet and ShadowNet , implement these neural fields, respectively.
Revised: Two MLPs, namely SpecularNetand ShadowNet , implement these neural fields, respectively.

Original:  we take the estimatedlight direction as the input of these networks to achieve their implicit association to light conditions.
Revised: Instead of estimating light conditions from SpecularNet or ShadowNet, we take the light direction as the input of these networks to achievetheir implicit association to light conditions.

Original: Since the specular reflectance and cast shadow are normal- and  position-dependent, we also feed theestimated surface normal and the positional code to them, respectively.
Revised: Since the specular reflectance and cast shadow arenormal- and pixel position-dependent, we also feed the estimated surface normal and the positionalcode to them, respectively.

Original: SpecularNet and ShadowNetoutput ρ sij and s ij , respectively, as shown inFig.
Revised: SpecularNet and ShadowNet output ρ sij and s ij , respectively, as shown in

Original: Neural light field.
Revised: Neural light field.

Original: There is mutual information across different observed images, i.e. , observedimages with similar appearances are expected to be illuminated by similar lights.
Revised: There is mutual information across different observed images, i.e , observedimages with similar appearances are expected to be illuminated by similar lights.

Original: To fully considerand exploit this mutual information, we build the neural light field by concatenating a CNN encoderto an MLP, namely LightNet .
Revised: To fully considerand exploit this mutual information, we build the neural light field by concatenating a CNN encoderto an MLP, namely LightNet .

Original: The encoder extracts a light code from each observed image 3 , and the LightNet infers the corresponding light conditions ( i.e. , e j , l j ) from the light code, as shown in Fig.
Revised: The encoder extracts a light code from each observed image 3 , andthe LightNet outputs the corresponding light conditions ( i.e , e j , l j ) from the light code, as shown in Fig.

Original: Neural position field.
Revised: Neural position field.

Original: There is mutual information across different positions on a surface, i.e. , theconsistency of shape and diffuse reflectance in the spatial domain.
Revised: There is mutual information across different positions on a surface, i.e , theconsistency of shape and diffuse reflectance in the spatial domain.

Original: To fully consider and exploit thismutual information, we establish a neural position field, namely PositionNet , implemented by an MLP.
Revised: To fully consider and exploit thismutual information, we establish a neural position field, namely PositionNet , implemented by an MLP.

Original: The neural position field outputs position-dependent, light-independent factors 4 , i.e. , n i , ρ di , k di , k si .
Revised: The neural position field output position-dependent, light-independent factors 4 , i.e , n i , ρ di , k di , k si .

Original: The PositionNet takes the positional code as the input, as shown in Fig. 2.
Revised: The PositionNet takes the positional code as the input, as shown in Fig. 2.

Original: 3.3 Optimizing Neural Intrinsics Fields
Revised: 3.3 Optimizing Neural Intrinsics Fields

Original: We adopt the reconstruction loss function with the ℓ 1 metric to optimize our NeIF, 
Revised: We adopt the reconstruction loss function with the ℓ 1 metric to optimze our NeIF, [Equation]

Original: [Equation]
Revised: [Equation]

Original: Silhouette constraint.
Revised: Silhouette constraint.

Original: L rec cannot resolve the shape-light ambiguity in an unsupervised manner dueto the inherently severe ill-posedness.
Revised: L rec cannot resolve the shape-light ambiguity in an unsupervised manner dueto the inherently severe ill-posedness.

Original: Therefore, we introduce the silhouette constraint (similar tothose in [23, 15]) to stabilize the training of PositionNet.
Revised: Therefore, we introduce the silhouette constraint (similar tothose in [ 8758025 , 13]) to stabilize the training of PositionNet.

Original: To be specific, we use polynomial fittingwith a moving window block to traverse and pre-compute the contour’s normal of the given objects,represented as ˆ N si ≜ { ˆ n si k ∈ R 2 × 1 , k ∈ S } , S is the point set of the contour.
Revised: To be specific, we use polynomialfitting with a moving window block to traverse and pre-compute the contour’s normal of the givenobjects, represented as ˆ N si ≜ { ˆ n si k ∈ R 2 × 1 , k ∈ S } , S is the point set of the contour.

Original: We consider ˆ N si can guide the prediction of the azimuth of boundary normal (at the same positions) and introduce thesilhouette loss function,
Revised: We considerˆ N si could guide the prediction of the azimuth of boundary normal (at the same positions) and introduce the silhouette loss function,

Original: [Equation]
Revised: [Equation]

Original: where n k ∈ R 1 × 3 represents the estimated surface normalat the positions of silhouette from PositionNet, C ( · ) cuts off the 3rd dimension of n k ( i.e. , C ( n k ) ∈
Revised: where n k ∈ R 1 × 3 represents the estimated surface normal from PositionNet, C ( · ) cuts off the 3rddimension of n k ( i.e , C ( n k ) ∈

Original: R 1 × 2 ), and Nor ( · ) is the vectornormalization operation.
Revised: R 1 × 2 ), and Nor ( · ) is the vector normalization operation.

Original: Warm-up loss functions.
Revised: Warm-up loss functions.

Original: To avoid local minimum and achieve faster convergence, we warm upthe NeIF in early-stage during training with three additional loss functions.
Revised: To avoid local minimum and achieve faster convergence, we warm upthe NeIF in early-stage during training with three additional loss functions.

Original: We use the azimuth oflighting direction estimated by YS97 [57] to guide the training of LightNet,
Revised: We use the azimuth oflighting direction estimated by YS97 [54] to guide the training of LightNet,

Original: [Equation]
Revised: [Equation]

Original: where l az j are estimated light directions by [57].
Revised: where l az j are estimated light directions by [54].

Original: We adopt the gradient penalty [22] L gp to stabilizethe training of SpecularNet,
Revised: We adopt the gradient penalty [20] L gp to stabilizethe training of SpecularNet,

Original: [Equation]
Revised: [Equation]

Original: The intuition is from Blinn-Phong model [8], where the specular reflectance is monotonicallyincreasing w.r.t the n ⊤ i h j , i.e. , ∇ n ⊤ i
Revised: The intuition is from Blinn-Phong model, where the specular reflectance is monotonically increasingw.r.t the n ⊤ i h j , i.e , ∇ n ⊤ i

Original: h
Revised: h

Original: j ρ sij > 0 .
Revised: j ρ sij > 0 .

Original: We also supervise the training of ShadowNet usingpseudo shadow maps ˆ S j , j = 1 , 2 , ..., f ,
Revised: We also supervise the training of ShadowNet using pseudoshadow maps ˆ S j , j = 1 , 2 , ..., f ,

Original: [Equation]
Revised: [Equation]

Original: The pseudo shadow maps are obtained by binarizing the observed images, i.e. , considering anobserved pixel to be cast shadow if its intensity value is smaller than 0.2 × the mean intensity value ofthis image.
Revised: The pseudo shadow maps are obtained by binarizing the observed images, i.e , considering an observedpixel to be cast shadow if its intensity value is smaller than 0 . 2 × the mean intensity values of thisimage.

Original: After early-stage training, we discard these loss functions for a broad range of reflectance.
Revised: After early-stage training, we discard these loss functions for a broad range of reflectance.

Original: NeIF training.
Revised: NeIF training.

Original: We train NeIF with the warm-up loss function in first 10 epochs,
Revised: We train NeIF with the warm-up loss function in first 10 epochs,

Original: [Equation]
Revised: [Equation]

Original: where λ si = 5 , λ az = 0 .
Revised: where λ si = 5 , λ az = 0 .

Original: 1 , λ gp =10 , λ shadow = 10 .
Revised: 1 , λ gp = , λ shadow = 10 .

Original: We then train NeIF until 500 epochs or convergingwith the loss function,
Revised: We then train NeIF untill 500 epochs or convergingwith the loss function,

Original: [Equation]
Revised: [Equation]

Original: where L recShadow is the another shadow map supervision loss to train the ShadowNet.
Revised: where L recShadow is the another shadow map supervison loss to train the ShadowNet.

Original: The loss functionis the same to L shadow .
Revised: The loss functionis the same to L shadow .

Original: The only difference is the calculation of ˆ S j .
Revised: The only difference is the calculation of ˆ S j .

Original: For L recShadow , we calculate ˆ S jby rendering a depth map with the estimated l j .
Revised: For L recShadow , we calculate ˆ S jby rendering a depth map with the estimated l j .

Original: The depth map is reconstructed from the estimatedsurface normal map N by method [10].
Revised: The depth map is reconstructed from the estimatedsurface normal map N by method [8] 5 .

Original: L recShadow is used to align outputs of ShadowNet to those of
Revised: L recShadow is used to align outputs of ShadowNet to those of

Original: PositionNet.
Revised: PositionNet.

Original: Implementation details.
Revised: Implementation details.

Original: We generate the positional code from the coordinate (in the image plane)by the same method in [34].
Revised: We generate the positional code from the coordinate (in the image plane)by the same method in [31].

Original: With the assumption of isotropic reflectance, we simplify the input of SpecularNet from { v ⊤  l j ,v ⊤ n i , n ⊤ i  l  j  } to { v ⊤  h j ,n ⊤ i h  j  } 5 [31] for easier  training.
Revised: With the assumption of isotropic reflectance, we simplify the inputof SpecularNet from { n ⊤ i l j , n ⊤ i v , l ⊤ j v } to { n ⊤ i h j , h ⊤ j v } 6 [28] for easier model training.

Original: Similar to the Cook-Torrance reflectance model [19], we assume k di + k si = 1 to reduce the number of unknowns.
Revised: Similarto the Cook-Torrance reflectance model [17], we assume k di + k si = 1 to reduce the numbers ofunknowns.

Original: The CNN decoder with declining channels processes the down-sampled images with a dimension of256 × 256 .
Revised: The CNN decoder with declining channels processes the down-sampled images with adimension of 256 × 256 .

Original: The LightNet takes flatten features to predict l j , e j in two different branches.
Revised: The LightNet takes flatten features to predict l j , e j in two different branches.

Original: For the ShadowNet, we also generate the positional code for l j and concatenate it to the feature in the 9thlayer.
Revised: For the ShadowNet, we also generate the positional code for l j and concatenate it to the feature in9-th layer.

Original: The output of ShadowNet is either 0 or 1, which is realized by a step function with a similarimplementation in [38]. v ∥ .
Revised: The output of ShadowNet is either 0 or 1, which is realized by a step function with asimilar implementation in [35] 7 .

Original: This table summarizes comparison methods. ’

Original: N.A.’ represents not applicable as calibrated PS is with known ℓ and e .

Original: ‘Semi’ indicates certain method leverage partial information of light.

Original: ‘ ✓ ’ (or ‘ ✗ ’) represents that certain methods (do not) adopt supervised learning for the estimation of surface normal n , light direction l , or light intensity e .

Original: ‘Identical’ means certain methods require the light intensity of different illuminating to be identical

Original: [Table]

Original:  Experiments
Revised:  Experiments

Original: Training details.
Revised: Training details.

Original: Our main framework is implemented in PyTorch, while the pre-calculations(method [57] and silhouette fitting) are implemented in MATLAB.
Revised: Our main framework is implemented in PyTorch, while the pre-calculations(method [54] and silhouette fitting ) are implemented in MATLAB.

Original: We use Adam as the optimizerwith a learning rate α = 5 × 10 − 4 to train our framework in 500 epochs for each scene separately, andthe warm-up stage takes up 10 epochs.
Revised: We use Adam as the optimizer v ∥ . Please refer to our supplementary material for more details [Table] with a learning rate α = 5 × 10 − 4 to train our framework in 500 epochs for each scene separately, andthe warm-up stage takes up ten epochs.

Original: The last 100 epochs use a lower learning rate α = 5 × 10 − 5for fine-tunning.
Revised: The last 100 epochs use a lower learning rate α = 5 × 10 − 5for fine-tunning.

Original: The batch size is 32 for lighting and 256 for spatially random sampling.
Revised: The batch size is 32 for lighting and 256 for spatially random sampling.

Original: After everyepoch, the depth map is reconstructed according to the predicted normal map by method [10].
Revised: After everyepoch, the depth map is reconstructed according to the predicted normal map by method [8].

Original: Eachscene takes from 2 hours to 6 hours on one RTX 2080Ti 12GB GPU, depending on the resolution ofthe objects.
Revised: Eachscene takes from 2 hours to 6 hours on one RTX 2080Ti 12GB GPU, depending on the resolution ofthe objects.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: We adopt the same metric in [13], the scale-invariant relative error, to measurethe accuracy of recovered light intensity as,
Revised: We adopt the same metric in [12], the scale-invariant relative error, to measurethe accuracy of recovered light intensity as

Original: [Equation]
Revised: [Equation]

Original: where, η is calculated by solving arg minη (cid:80) fj
Revised: where, η is calculated by solving arg min

Original: =1 (
Revised: η

Original: [Equation]
Revised: [Equation]

Original: ˜ e j ) 2 by least squares minimization.
Revised: ˜ e j ) 2 by least squares minimization.

Original: Themetric to measure the accuracy of the predicted light directions and surface normal is the widely usedmean angle error (MAE) in degree.
Revised: Themetric to measure the accuracy of the predicted light directions and surface normal is the widely usedmean angle error (MAE) in degree.

Original: 4.1 Evaluation on Public Datasets
Revised: 4.1 Evaluation on Public Datasets

Original: Since the proposed NeIF is an unsupervised uncalibrated photometric stereo method, we compare itsperformance with state-of-the-art uncalibrated and unsupervised photometric stereo methods.
Revised: Since the proposed NeIF is an unsupervised uncalibrated photometric stereo method, we compare itsperformance with state-of-the-art uncalibrated and unsupervised photometric stereo methods.

Original: Threereal-world datasets, including the  D I L
Revised: Threereal-world datasets, including the benchmark dataset D I L

Original: I G EN T benchmark dataset [45], A PPLE & G OURD dataset [2],and  L IGHT S TAGE D ATA G ALLERY dataset [11], are used for evaluation.
Revised: I G EN T [39], A PPLE & G OURD dataset [2],and the L IGHT S TAGE D ATA G ALLERY [9], are used for evaluation.

Original: Quantitative comparison for normal map.
Revised: Quantitative comparison for normal map.

Original: Table 1 lists relevant works for a comprehensive surfacenormal estimation comparison.
Revised: Table 1 lists relevant works for a comprehensive surfacenormal estimation comparison.

Original: As summarized in Table 1, our method is the only method thataddresses UPS without the supervision of N , L , or E .
Revised: As summarized in Table 1, our method is the only method thataddresses UPS without the supervision of N , L , or E .

Original: However, our method achieves the bestperformance and maintains a considerable advantage over other competitors [13, 15] ( e.g. , 7.86 for
Revised: However, our method achieves the bestperformance and maintains a considerable advantage over other competitors[12, 13] ( i.e , 7.86 for

Original: NeIF vs. 8.71 for [15]).
Revised: NeIF v.s. 8.71 for [13]).

Original: Handling the objects in D I L I G EN T dataset [45] with different shapes andreflectance, numbers from our method are either best or competitive (about 1 ◦ as compared with thebest performing UPS method), which shows its good generalization capacity to general reflectanceand various shapes.
Revised: Objects in D I L I G EN T differ from shapes and reflectance. The numbers ofour method are either best or competitive (less than 1 ◦ as compared with the best performing UPSmethod), which shows its good generalization capacity to general reflectance and various shapes.

Original: This is because our NeIF fully considers mutual information by building upintrinsics fields and the implicit modeling facilitates  general reflectance modeling.
Revised: Because our NeIF fully considers mutual information by building up intrinsics fields and such implicitmodeling facilitates modeling general reflectance.

Original: Visual quality comparison forthe normal map.
Revised: Visual quality comparison for normal map.

Original: Fig.
Revised: Fig.

Original: 3 illustrates the visual quality comparison interms of recovered surface normal maps and corresponding error maps on D I L I G EN T dataset [45]. [Figure] [Figure]
Revised: 3 illustrates the visual quality comparison in termsof recovered surface normal maps and corresponding error maps on D I L I G EN T.

Original: GO URD

Original: GO URD

Original: [Figure]

Original: S T A ND

Original: I N [Figure] [Figure] The comparison is conducted with two state-of-the-art UPS methods [13, 15] and a state-of-the-artunsupervised PS method [47].
Revised: The comparisonis conducted with two state-of-the-art UPS methods[12, 13] and a state-of-the-art unsupervised PSmethod [44].

Original: Our method is less sensitive to spatially-varying albedo due to theper-pixel manner (red boxes in the left subfigure of Fig. 3).
Revised: Our method is less sensitive to spatially-varying albedo due to the per-pixel manner(red boxes in the left subfigure of Fig. 3).

Original: However, although the positional codeconsiders the  global shape effect, this per-pixel manner is less effective in modeling complex shapeinformation as compared with the all-pixel one.
Revised: However, although the positional code considers the [Figure] global shape effect, this per-pixel manner is less effective in modeling complex shape information ascompared with the all-pixel one.

Original: It fails for regions with cast shadow (or overexposure)under most light directions (red boxes in the right subfigure of Fig. 3).
Revised: It could fail for regions with cast shadow (or overexposure) undermost light directions (red boxes in the right subfigure of Fig. 3).

Original: We also show the visualquality results for A PPLE & G OURD dataset [2] and the L IGHT S TAGE D ATA G ALLERY dataset [11]in Fig.4.
Revised: We also show the visual qualityresults for A PPLE & G OURD dataset [2] and the L IGHT S TAGE D ATA G ALLERY [9] in Fig.

Original: Our method produces reliable estimation for most regions, thanks to the full exploitation ofmutual information across different dimensions.
Revised: Ourmethod could produce reliable estimation for most regions, thanks to the full exploitation of mutualinformation across different dimensions.

Original: Quantitative comparison for light directions and intensities.
Revised: Quantitative comparison for light directions and intensities.

Original: As can be observed in Table 2and Table 3, our method achieves a superior performance advantage over unsupervised methods(YS97 [57] and PF14 [37]) while maintaining competitive performance as compared with supervisedmethods (CH19 [13] andCW20 [15]). These supervised methods adopt two-step solutions andsuffer from accumulating error. Therefore, even though they achieve similar performance in terms oflight conditions accuracy, they are less effective on estimating surface normal as compared with ourmethod (see Table 1). [Table]
Revised: As can be observed in Table 2 and

Original: B [Table] B UDDHA C AT
Revised: Table 3, our method achieves a superior performance advantage over unsupervised methods[34, 4,42, 50, 29, 28] while maintaining competitive performance as compared with supervised methods.

Original: C OW G OBLET H ARVEST P OT
Revised: Although the supervised methods pre-define the intensity range, they produce less accurate estimationas compared with our approach due to the possible overfitting of supervised learning.

Original: P OT R EADING [Table] [Table]
Revised: In contrast,our LightNet builds the neural light field, which could fully consider the mutual information amongdifferent observed images, resulting in the accurate estimation of the scale-invariant light intensity.

Original: 4.2 Evaluation on Sparse Uncalibrated Photometric Stereo
Revised: 4.2 Evaluation on Sparse Uncalibrated Photometric Stereo

Original: As compared with other per-pixel PS methods ( e

Original: [Table] g. , ZJ19 [62] and LL19 [30]) [Table] the proposed NeIFexploits much more constraints to estimate the normal at a point ( i.e. , # P × f vs. f ).
Revised: As compared with other per-pixel PS methods ( e.g. , [59, 27]), the proposed NeIF exploits much moreconstraints to estimate the normal at a point, i.e , # P × f v.s. f .

Original: Because itlearns fields instead of regressing intensity profile [43] to surface normal.
Revised: Because it learns fields instead ofregressing intensity profile [40] to surface normal.

Original: Therefore, we investigatethe effectiveness of our  method under the challenging setting of sparse UPS.
Revised: Therefore, we investigate the effectiveness of our [Table] [Table] method under the challenging setting of sparse UPS.

Original: We randomly select10 or 16 images illuminated by different lights from D I L I G EN T dataset [45] andtest our method using these images.
Revised: We randomly select 16 images illuminatedby different lights from D I L I G ENT and test our method using these images.

Original: We repeat this process 30 times, similar to [62].
Revised: We repeat this process30 times, similar to [59].

Original: We compare theperformance with that from the classical UPS method PF14 [37], a deep learning based per-pixelapproach ZJ19 [62], and an all-pixel approach CH18 [14].
Revised: We compare the performance with that from the classical PS method PF14 [34], as deep learning-based approaches ( e.g. , [12, 13]) perform poorly without re-training.

Original: Due to the fewer reconstruction terms totrain our NeIF, we slightly increase the dimension of the positional encoding module from 4 to 6 thatincreases the frequency [48] to stabilize the training, i.e. , strengthening the role of positional code toreduce the variance of estimated intrinsics.
Revised: Due to the fewer reconstruction terms to train our NeIF [Table] we slightly increase the dimension of thepositional encoding module from 4 to 6 by increasing the frequency [45] to stabilize the training, i.e ,strengthening the role of positional code to reduce the variance of estimated intrinsics.

Original: Besides, we drop L az during early stage warm-up because YS97 [57] fails for the sparse inputs.
Revised: Besides, wedrop L az during early stage warm-up because YS97 [54] fails for objects with centralized specularcomponents ( e.g. , B ALL ).

Original: We report the mean results over 30 trails in Table 4.
Revised: We report the mean results over 30 trails in Table 1.

Original: As can be observed, our method achievescompetitive performance onnormal estimation as compared with state-of-the-art sparse PS methodswith known light( i.e. , 11.62 for NeIF vs. 9.82 for ZJ19 [62] under 10 lights, and10.38 for NeIF vs.9.00 for CH18 [14] under 16 lights). Kindly note that we do not require any ground truth normal forsupervision. As compared with the unsupervised UPS method (PF14 [37]), we achieve a superiorperformance advantage. The results validate the effectiveness of building up neural intrinsics fieldsthat fully exploit the mutual information across different dimensions.
Revised: As can be observed, our method achievespromising performance on light and normal estimation for sparse UPS.

Original:  Conclusion
Revised:  Conclusion

Original: This paper proposes NeIF for UPS.
Revised: This paper proposes NeIF for UPS.

Original: By representing the general reflectance as four neural intrinsicsfields, it implicitly imposes the light clues of specular reflectance and cast shadow for light estimation,which facilitates solving UPS with general reflectance.
Revised: By representing the general reflectance as four neural intrinsicsfields, it implicitly associates the light clues of specualr reflectance and cast shadow to light estimation,which faciliate solving UPS with general reflectance.

Original: The proposed NeIF can fully exploit mutualinformation from all observed pixel values so that it produces stable estimation  forPS and sparse PS.
Revised: The proposed NeIF could fully exploit mutualinformace from all observed pixel values so that it could produce stable estimation even for sparse UPS.

Original: The unsupervised training manner of NeIF is beneficial to the generalization capacity of data fromdifferent sources.
Revised: The unsupervised training manner of NeIF is beneficial to the generalization capacity to datafrom different sources.

Original: Limitations.
Revised: Limitations.

Original: Although our method produces promising results for light conditions and surfacenormal estimation, it has several limitations.
Revised: Although our method produces promising results for light condition and surface normalestimation, it has several limitations.

Original: First, the estimated intrinsics, such as shadow, specularand diffuse reflectance, and their balancing coefficients are less accurate for objects with complicatedgeometries, as shown in Fig.
Revised: First, the estimated intrinsics such as shadow, specular anddiffuse reflectance, and their balancing coefficients are less accurate, as shown in Fig.

Original: Second, we need to balance the role of positional code and estimatedintrinsics when applying our method for sparse UPS.
Revised: Second,we need to balance the role of positional code and estimated intrinsics when applying our methodfor sparse UPS.

Original: However, how to find an optimal balancingstrategy is unknown.
Revised: However, how to find an optimal balancing strategy is unknown.

Original: Third, as shown in Fig. 3, our method is less effective on global effects,especially for regions with cast shadow (or overexposure) under most light directions due to our
Revised: Third, as shownin Fig. 3, our method is less effective on global effects, especially for regions with cast shadow (oroverexposure) under most light directions due to the per-pixel manner.

Revised: Besides, the inference of ourmethod is inefficient due to the unsupervised training manner and the per-pixel estimation, which isinapplicable for real-time applications.

Original: [Table]
Revised: Checklist

Revised: The checklist follows the references.

Revised: Please read the checklist guidelines carefully for information onhow to answer these questions.

Revised: For each question, change the default [TODO] to [Yes] , [No] , or[N/A] .

Revised: You are strongly encouraged to include a justification to your answer , either by referencingthe appropriate section of your paper or providing a brief inline description.

Revised: For example:

Revised: • Did you include the license to the code and datasets?

Revised: [Yes] See Section ?? .•

Revised: Did you include the license to the code and datasets?

Revised: [No] The code and the data areproprietary.•

Revised: Did you include the license to the code and datasets?

Revised: [N/A]

Revised: Please do not modify the questions and only use the provided macros for your answers.

Revised: Note that the

Revised: Checklist section does not count towards the page limit.

Revised: In your paper, please delete this instructionsblock and only keep the Checklist section heading above along with the questions/answers below.

Revised: For all authors...

Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s

Revised: contributions and scope?

Revised: [Yes] See Line 9-13,

Revised: (b) Did you describe the limitations of your work?

Revised: [Yes] See Line 287-

Revised: (c) Did you discuss any potential negative societal impacts of your work?

Revised: [No]

Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Revised: them?

Revised: [Yes]

Revised: If you are including theoretical results...

Revised: (a) Did you state the full set of assumptions of all theoretical results?

Revised: [Yes]

Revised: (b) Did you include complete proofs of all theoretical results?

Revised: [N/A]

Revised: If you ran experiments...

Revised: (a) Did you include the code, data, and instructions needed to reproduce the main ex-

Revised: perimental results (either in the supplemental material or as a URL)?

Revised: [Yes] See

Revised: Supplementary Material

Original: per-pixel manner.
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]

Revised: (c) Did you report error bars (e

Revised: g., with respect to the random seed after running experi-

Revised: ments multiple times)?

Revised: [Yes]

Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., type

Revised: of GPUs, internal cluster, or cloud provider)?

Revised: [Yes]

Revised: If you are using existing assets (e

Revised: g., code, data, models) or curating/releasing new assets...

Revised: (a) If your work uses existing assets, did you cite the creators?

Revised: [No]

Revised: (b) Did you mention the license of the assets?

Revised: [No]

Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Revised: [No]

Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Revised: using/curating?

Revised: [No]

Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Revised: information or offensive content?

Original: Besides, the inference of our method is inefficient due to the unsupervised trainingmanner and the per-pixel estimation, which is inapplicable for real-time applications.
Revised: [No] If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, ifapplicable? [N/A]

Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Revised: Board (IRB) approvals, if applicable?

Revised: [N/A]

Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Revised: spent on participant compensation?

Revised: [N/A]