Proceedings of IC-NIDC 2023

NEURAL BRDF PLUGIN FOR UNSUPERVISED PHOTOMETRIC STEREO
Heng Guo††

Boxin Shi‡

Osaka University

ABSTRACT
Recovering surface normal under unknown general reflectanceis challenging for photometric stereo. Supervised learningbased methods implicitly handle general reflectances bylearning from training data. However, it suffers from a demand for a large number of rendered images as training data
and also from a domain gap due to the biased shape distribution in the training set. Unsupervised methods mitigate
these issues by assessing image re-rendering losses, but theirperformance is limited primarily due to the fact that multiple
combinations of reflectance parameters can explain the same
appearance. We combine the merits of these two approaches
by introducing a BRDF plugin, a neural network-based BRDF
dictionary that is trained using a measured BRDF dataset,offering a data prior to effectively constrain reflectances.
Specifically, it maps each measured BRDF to a unique latentcode, forming a latent BRDF space that serves as the dataprior for reflectances. Building upon the BRDF plugin, wedevelop a new unsupervised photometric stereo method torecover both surface normal and reflectance, which exhibits
superior accuracy than previous methods in experiments on
synthetic and real-world datasets.
Key Words— Photometric stereo; Neural BRDF; Unsupervised learning
1. INTRODUCTION
Photometric stereo aims at recovering surface normal and reflectance from a set of images captured with a fixed camera
under varying illuminations. One of the key challenges inphotometric stereo is to handle general surface reflectances,which are described via bidirectional reflectance distributionfunctions (BRDFs). Early photometric stereo methods assume an ideal Lambertian model [1], which is unfortunately
not flexible enough to represent real-world reflectances.
To deal with general BRDFs, supervised learning-based
photometric stereo methods, such as DPSN [2] have been developed, which implicitly address general BRDFs by learning data prior from synthetic image datasets rendered with diverse shapes and BRDFs. Subsequent works use altered network structures, including all-pixel based branches (e.g., PSFCN [3], NormAttentionPSN [4]), per-pixel based branches(e.g., CNN-PS [5], PX-Net [6]), and hybrid approaches (e.g.,

Yasuyuki Matsushita†‡

Peking University

GPS-Net [7]). More comprehensive reviews are found in recent surveys [8, 9] on learning-based photometric stereo. Supervised photometric stereo methods require extensive training datasets, which can lead to significant time and memory
costs, potential domain gaps, and biases in real-world applications due to specific shape and BRDF distributions. They
also limit methods’ applicability to specific lighting conditions, e.g., PS-FCN [3] trained on distant-light datasets cannot
deal with near-light settings.
To bypass these training dataset-related issues in supervised photometric stereo, unsupervised photometric stereo
methods based on image re-rendering losses have been introduced [10, 11], which exhibit flexibility in dealing withdiverse shapes and lighting conditions. However, current
unsupervised photometric stereo methods could suffer fromthe fact that multiple combinations of parameters can explain
the same appearance, as they estimate surface reflectances
purely from image cues without the additional constraints
offered by training data that are used in supervised methods. Consequently, the estimated surface normals may notalign well with the ground-truth (GT) even when low image
re-rendering errors are achieved.
We address the above problem by bringing the data priorof supervised photometric stereo to unsupervised methods.
We observe that the general BRDFs in photometric stereo are
the primary component requiring the data prior for accurate
modeling. Motivated by the observation, we propose a BRDF
plugin, a neural network-based BRDF dictionary trained onreal-world measured BRDF datasets, such as MERL [12].
In our BRDF plugin, each measured BRDF is mapped to aunique high-dimensional latent code, creating a latent BRDF
space. Once trained, our BRDF plugin can be applied to various tasks associated with general BRDF modeling, including
unsupervised photometric stereo.
To verify the effectiveness of our BRDF plugin, we develop an unsupervised photometric stereo method built upon
our BRDF plugin to predict surface normal and reflectance simultaneously, in which the unknown BRDF of the target surfaces is assumed to reside within the latent BRDF space. In
other words, the latent BRDF code of the reflectance, whenmapped via our BRDF plugin, can be described by a linear
combination of the latent codes of the measured BRDFs. This
strategy allows the data prior by the BRDF plugin to enhance
unsupervised photometric stereo by providing additional con-

𝒙

…

BRDF
𝑟
ReLU

𝒕1 : {𝜸1 , 𝜷1 }𝒕0 : {𝜸0 , 𝜷0 }

ReLU

Input

LinearReLU

LinearFrequency Phase

𝑹

∙

+

𝜸

𝜷

Sine

Output𝜙𝑑Linear𝜃𝑑

FiLM SIREN

Rusinkiewic
z coordinatesmapping

Linear𝝎𝑜Linear

𝜃ℎ 𝜃𝑑
𝜙𝑑

𝜃ℎ

FiLM SIREN

Sampling Network 𝑓𝑠 (𝒙, 𝒕; 𝜽𝑠 )𝒏 𝒉𝝎𝑖Linearstraints on reflectance estimation. In addition, as our BRDF
plugin is supervised only on BRDFs but not shapes, we eliminate the need for time and memory-intensive rendering processes associated with building large-scale image datasets andavoid shape-related biases. By evaluating our unsupervised
photometric stereo method on both synthetic and real-world
datasets, we showcase the superior performance of our approach over previous methods.

sin(𝜸 ⋅ input + 𝜷)

Projecting Network Latent
BRDF
𝑓𝑝 (𝑹; 𝜽𝑝 )code 𝒛Database [13]Mapping Network
𝑓𝑚 (𝒛; 𝜽𝑚 )FiLM SIREN layer

2. PROPOSED METHOD
Fig. 1: Network structure of our BRDF plugin.
We begin with modeling general BRDFs with the proposed
BRDF plugin and then introduce our unsupervised photometric stereo method powered by the BRDF plugin.
2.1. BRDF plugin
A BRDF R is a scalar function of surface normal n andincident-outgoing light directions (ωi , ωo ), i.e., R(n, ωi , ωo ).
As shown in Fig. 1, an isotropic BRDF can be represented in amore compact manner using the Rusinkiewicz coordinate [13]x = [θh , θd , φd ]> , where θh ∈ [0◦ , 90◦ ] is the angle between
surface normal and a halfway vector h = (ωi + ωo )/2,
θd ∈ [0◦ , 90◦ ] and φd ∈ [0◦ , 180◦ ] are the spherical coordinates of incident light direction ωi in the coordinate system
where h is at the north pole. Based on Rusinkiewicz coordinate, we represent BRDF as R(x) = R(θh , θd , φd ). There
are two challenges in compactly representing BRDFs: 1)
BRDFs greatly vary across different materials; 2) the domainof BRDF functions is continuous. Therefore, we use a conditional coordinated-based multilayer perceptron (MLP) for
enabling the BRDF plugin.
As shown in Fig. 1, our BRDF plugin consists of threemodules: a projection network, a mapping network, and asampling network, with their network parameters labeled asθp , θm , and θs , respectively. The projection network fp maps
a BRDF to its latent code z serving as an identifier for diverse
BRDFs, i.e.,z = fp (R; θp ),(1)

where R ∈ R90×180×180 is a 3D measured BRDF tensor
from MERL dataset [12], indexed by the three variables of
the Rusinkiewicz coordinates. Thus, we use a 3D CNN [14]in the projection network.
The mapping network and sampling network in our
BRDF plugin produce a BRDF value r sampled at a specific
Rusinkiewicz coordinate x controlled by a material latentcode z. To achieve this, we design a conditional coordinatebased MLP following π-GAN [15]. Specifically, the mapping
network fm takes latent code z as input and outputs latentfeature t for the sampling network, i.e.,t = [γ, β] = fm (z; θm ),(2)

where γ and β are frequency and phase terms in the featurewise linear modulation (FiLM) layer [15], which is embeddedin the sampling network for controlling the BRDF output fora specific material. The sampling network adopts the SIREN
network structure [16] with a periodic sine activation function. Given latent feature t, the sampling network fs outputs
a BRDF value r at a sampled Rusinkiewicz coordinate x as
r = fs (x, t; θs ).

(3)

Combining Eqs. (1) to (3), we optimize the network parameters of the BRDF plugin by minimizing L1 loss between predicted and measured BRDF values of MERL dataset [12],which contains k = 100 diverse BRDF tensors and p =1458000 Rusinkiewicz coordinate samples, i.e.,∗θp∗ , θm, θs∗ = argmin
θp ,θm ,θs= argminθp ,θm ,θs

p
k X
X

p
k X
X

krij − fs (xj , ti ; θs )k1

i=1 j=1

krij − fs (xj , fm (fp (Ri ; θp ) ; θm ) ; θs )k1 .

i=1 j=1

where rij is the BRDF value of the i-th MERL BRDF tensor
Ri sampled at the j-th coordinate xj .
To better fit specular highlights, we compute the loss
in the logarithmic domain using µ-law 1 , where BRDF values are mapped as r̂ij = log(1 + µrij )/ log(1 + µ). After
training, we save the optimized network parameters. Thek latent codes z along with their corresponding latent features t trained from MERL dataset [12] construct a latentcode dictionary Z ∈ Rk×dz and a latent feature dictionary
T ∈ Rk×dt , where td and tz are the dimensions of the latentcode and feature, respectively. The BRDF plugin networkand the latent dictionaries build a data prior from real-world
measured BRDFs, which can be easily embedded into ourunsupervised photometric stereo method.
2.2. Unsupervised photometric stereo with BRDF plugin
As shown in Fig. 2, our unsupervised photometric stereo takes
image observations and the corresponding light directions as
1 µ-law: https://www.cisco.com/c/en/us/support/docs/
voice/h323/8123-waveform-coding.html

(4)

MAX

𝒍𝑓

𝒍1

Share-weighted
CNN

𝒕 = 𝑻𝒘

𝑴 ∈ ℝℎ×𝑤×𝑓
𝒏
𝑧

𝑢

𝒑

∇

𝑣

𝒙𝑓

(𝑢, 𝑣)

Image
coordinates

𝒙1
𝒙2
𝒙3

Geometric Network

Surface normal𝑵𝑓𝑔 (𝒑; 𝜽𝑔 )BRDF
Plugin𝑓𝑠 (𝒙, 𝒕; 𝜽𝑠 )(Fix weight)

𝑟1
𝑟2
𝑟3
⋮
𝑟𝑓

30
25
20
15
10
5
0

LL22
PS-FCNOurs𝒓

gold-metallic-paint2
alum-bronze
specular-maroon-phenolicsilver-paintspecular-violet-phenolic
green-plastic
maroon-plastic
red-phenolic
specular-green-phenolicsilver-metallic-paint2
green-acrylic
blue-acrylic
two-layer-goldorange-paintfruitwood-241
blue-metallic-paintgold-paintviolet-rubber
pearl-paintgray-plastic
specular-blue-phenolic
pvc
light-red-paintyellow-paintaventurnine
dark-red-painttwo-layer-silver
gold-metallic-painthematitepink-fabric2
dark-specular-fabricsilver-metallic-paintcolonial-maple-223
teflonpink-fabricblack-oxidized-steel
cherry-235
delrinwhite-fabricgold-metallic-paint3
pure-rubber
natural-209
blue-rubber
white-fabric2
neoprene-rubber
green-fabricdark-blue-paintwhite-acrylic
pink-felt
ipswich-pine-221
violet-acrylic
white-diffuse-bball
red-fabric2
green-metallic-paint
red-plastic
color-changing-paint2
polyurethane-foamyellow-matte-plastic
green-latex
beige-fabricalumina-oxide
red-fabricspecular-orange-phenolic
purple-paintyellow-plastic
white-marble
specular-red-phenolic
specular-white-phenolic
white-paintsilicon-nitrade
specular-yellow-phenolicpink-jaspernylon
pink-plastic
polyethylene
pickled-oak-260
blue-fabricred-specular-plastic
aluminium
special-walnut-224
grease-covered-steel
black-soft-plastic
blue-metallic-paint2
red-metallic-paintyellow-phenoliclight-brown-fabricblack-fabricbrass
specular-black-phenolicnickelsteel
ss440
tungsten-carbide
green-metallic-paint2
chromechrome-steel
color-changing-paint1
black-phenoliccolor-changing-paint3
black-obsidian𝒍1

𝒘 Latent dictionary 𝑻

Mean angular error

Image encoder 𝑓𝑒 (𝑴, 𝑳; 𝜽𝑒 )Input

Reflectance 𝒓

Fig. 2: Network structure of unsupervised photometric stereo.
input and produces surface normal map N and per-pixel reflectance vector r. These outputs allow us to reconstruct theimage observations and calculate an image re-rendering loss
with the input images. Specifically, our unsupervised photometric stereo contains three modules: a geometric network,
an image encoder, and the embedded BRDF plugin.
We follow the neural surface model proposed in [17] torepresent scene geometry, where depth is represented as aneural function of image coordinates, and surface normal canbe obtained simultaneously based on the analytical derivativesof the neural depth function w.r.t coordinate p, i.e,

MethodLL22 [11]PS-FCN [3]Ours

Avg. MAE

5.18

4.55

2.41

Fig. 3: Evaluation on the MERLSphere dataset [3] measured
by MAE in degree, with material names labeled in x-axis.
bining Eqs. (1), (7) and (8), we predict the reflectancerp = fs (Xp , t; θs ) = fs (Xp , Tfe (M, L; θe ); θs ) .

Together with the surface normal from the geometric network,
we can re-render the image observations at p as m̂p , i.e.,m̂p = rpmax(Lnp , 0),= fs (Xp , Tfe (M, L; θe ); θs )zp = fg (p; θg ),
>

(5)
>

np = ([∇z, −1]) = ([∇fg (p; θg ), −1]) ,(6)

xdenotes unit normalization. For a scene
where (x) = kxk
point at p, given its surface normal np and f input light
directions, we compute its Rusinkiewicz coordinates Xp ={x1 , . . . , xf } for the f incident-outgoing light ray pairs.
These coordinates are fed into the sampling network of our
BRDF plugin to predict the corresponding reflectance rp .
To import data prior from real-world measured BRDFs,
we fix the weight of the sampling network fs (·) in our BRDF
plugin and assume the latent BRDF code of the target object
can be linearly represented by our latent dictionaries Z or T.
Following the practice of π-GAN [15], we optimize the latent
BRDF of the target object in the feature space (using latentfeature dictionary T instead of latent code dictionary Z) to
avoid the complex mapping network fm (·), i.e.,t = Tw,(7)

where w represents a latent weight vector that remains uniform across all scene points. As w is reflectance dependent,
we design an image encoder following the shared-weighted
structure of PS-FCN [3] to predict it from image observations
and the corresponding light directions, i.e.,
w = fe (M, L; θe ),(8)

where M ∈ Rh×w×f contains f input images whose resolution is h × w, L ∈ Rf ×3 stacks the f light directions. Com-

(9)max(L([∇fg (p; θg ), −1])> , 0),(10)where denotes element-wise product. The unknown network parameters θe and θg are optimized by minimizing theL1 loss between observed and re-rendered images m and m̂:Xθe∗ , θg∗ = argmin
kmp − m̂p k1 .
(11)
θe ,θgp

With optimized network parameters, we can recover surface normal and depth map based on Eq. (6), and surface
reflectance based on Eq. (9).
3. EXPERIMENT
We assess our BRDF plugin-embedded unsupervised photometric stereo method using synthetic and real-world datasets.
We select two notable supervised photometric stereo methods, PS-FCN [3] and CNN-PS [5], and a cutting-edge unsupervised method, LL22 [11], as baselines. We evaluate theirsurface normal recoveries using mean angular error (MAE)
between the ground truth and the estimated surface normals.
3.1. Synthetic experiment
As shown in Fig. 3, we evaluate our methods and baselines
on a synthetic photometric stereo dataset MERLSphere [3],which includes 100 spheres covered by MERL BRDFs [12].
Benefited from learned data prior in our BRDF plugin, oursurface normal estimates achieve less than half of the MAE
from LL22 [11] on the average of 100 scenes, especially on

PS-FCN (16.91/15.91)
20
22
29
22
20
24
23
18
32
15

14
15
16
19
20
22
21
24
37
19

5.1
10
12
13
18
12
13
13
19
10

12
15
12
21
22
22
21
29
38
22

45
43
47
34
20
31
36
19
31
16

3.1 4.7 5.1 13 5.6 7.8
13 8.6 14 6.6 13 11
11 8.5 13 7.6 10 11
19 8.1 21 6.2 16 10
20 8.2 23 5.6 19 8.1
26 11 27 10 21 13
19 9.2 20 9.2 17 11
27 13 32 10 23 13
43 20 43 17 38 26
23 10 26 8.3 20 10

6
9.5
11
10
11
14
12
12
25
11

20
17
23
20
19
16
22
12
25
12

17
12
12
15
9.6
13
16
10
23
9

21
16
23
17
12
17
22
10
22
10

25
28
32
24
20
21
24
14
28
15

45

L
AC
RY
LIC

EE

Cu

12 13 24
9.2 11 31
10 10 38
11 11 15
9.8 5.7 26
12 27 1412 10 17
11 10 20
25 29 30
8.6 9.3 9.3
ST

12
11
13
13
11
13
16
13
50
11

E
Al

AB
S

LIT
KE

LO
N
NY

M
PO

L
AC
RY
LIC

Cu

12
13
11
17
19
20
19
19
34
18

Ours (15.70/13.00)
14
14
14
13
17
19
16
17
30
12
EE

14
12
11
14
9.8
17
13
15
29
13

ST

11
14
14
14
11
14
14
19
30
12

E
Al

LIT

AB
S

5.9
12
11
11
13
16
14
15
29
13
KE

7
14
12
17
19
25
20
28
44
19

BA

PV
C

LO
N
NY

M
PO

PP

13
SPIKE 12
NUT 20SQUARE 21
PENGTAGON 25
HEXAGON 22
PROPELLER 27
TURBINE 42
BUNNY 23GOLF

4.4 9.8 3.5
9.8 16 7.1
9.7 14 9.3
11 22 7.9
23 22 9.8
13 24 1112 22 1014 29 1125 37 19
11 24 9.7

PP

TM18 (17.93/14.07)
BALL 5.1

9.4
9.3
8.9
11
14
13
13
12
17
9.5

Table 1: Evaluation on DiLiGenT [18] dataset under 10sparse lights, where the smallest and the second smallest
MAE are highlighted in bold and underlined, respectively.
The top and bottom parts of the table group supervised andunsupervised photometric stereo methods.
Method

Ball

Cow

Bear

Cat

Pot1

Pot2

BuddhaGobletReading

HarvestAvg.

LS [1]
CNN-PS [5]PS-FCN [3]
GPS-Net [7]LMPS [20]
SPLINE-Net [21]

4.58
8.21
4.35
4.33
3.97
4.96

26.48
13.83
9.97
9.34
10.19
8.8

9.84
11.89
5.7
6.34
8.73
5.99

8.9
9
8.24
6.81
6.69
7.52

9.59
12.79
8.38
7.5
7.3
8.77

15.65
15.04
10.37
8.38
9.74
11.79

16.02
13.39
10.54
8.87
11.36
10.07

19.23
15.74
11.21
10.79
10.46
10.43

19.37
16.07
14.34
15
14.37
16.13

31.32
19.36
18.82
16.92
17.33
19.05

16.1
13.53
10.19
9.43
10.01
10.35

TM18 [10]LL22 [11]
Ours

2.12
2.16
1.35

8.87
6.26
9.79

6.92
5.69
5.67

6.58
5.83
5.56

7.14
8.13
6.18

9.61
7.17
6.41

11.41
12.65
10.11

14.99
10.93
9.67

13.7
31.64
14.77

26.55
24.05
28.63

10.79
11.45
9.814

Sphere

GT. N

Pot1

GT. N

0

Fig. 4: Evaluation of surface normal by MAE on
DiLiGenT102 [19] indexed by shape and material, with(mean/median) MAE over 100 scenes shown on the top.

metallic paints and metals such as ‘red-metallic-paint’ and‘steel’. Our method also outperforms the supervised photometric stereo method PS-FCN [3]. One possible reason is PSFCN [3] is influenced by the shape distribution in its photometric stereo training image dataset, while our method avoids
such a problem by only training on a measured BRDF dataset.
3.2. Real-world experiment
Besides the synthetic evaluations, we also test our method on
real-world datasets DiLiGenT [18] and DiLiGenT102 [19].
DiLiGenT102 . As shown in Fig. 4, we show surface normal
estimation errors from existing methods and ours indexed by
the shapes and materials in DiLiGenT102 [19], where CNNPS [5] and TM18 [10] are the state-of-the-art supervised andunsupervised photometric stereo method correspondingly,
based on the benchmark evaluation on DiLiGenT102 [19].
Compared to these baselines, our method achieves the smallest mean and median MAE errors on the 100 scenes covering
diverse shapes and materials. Similar to previous methods,
our approaches also cannot handle the shape ‘TURBINE’
which introduces severe cast shadows and inter-reflections
that are not explicitly modeled in our method. The translucent material ‘ACRYLIC’ is also challenging to us as our
BRDF plugin is trained on the MERL dataset [12], whichcontains no translucent BRDFs. Our approach could handle
this material given additional translucent BRDFs for training.
DiLiGenT. Photometric stereo under sparse lights is of greatinterest for efficient capture. We investigate the normal estimation error of existing methods and ours on DiLiGenT [18]under 10 sparse lights. As shown in Table 1, LMPS [20]and SPLINE-Net [21] are designed for sparse-light photomet-

Est. N15 17
9.8 1410 18
13 14
5.5 8.713 16
12 19
9.8 9.9
22 21
8 12

BA

17
12
16
16
14
15
20
17
25
12

PV
C

14
SPIKE 13
NUT 20SQUARE 22
PENGTAGON 28
HEXAGON 21
PROPELLER 29
TURBINE 54
BUNNY 25GOLF

7.1 4.9 5.3 7.6 8.1
8.5 12 7.4 15 9.9
11 13 13 13 11
9.8 20 7.6 17 8.4
17 24 8.7 20 8.7
11 28 11 23 10
10 22 9.4 19 30
13 36 9.4 24 11
20 52 16 41 20
12 28 8.4 21 9.4

10Err. map

CNN-PS (16.66/14.74)
BALL 5.9

LL22 [11]Ours

LL22 [11]Ours

Fig. 5: Comparison with LL22 [11] under 10 sparse lights on
two objects from the DiLiGenT [18] dataset.
ric stereo based on supervised learning. Compared with existing methods, our method achieves the smallest MAEs on
6 out of 10 objects of DiLiGenT [18], the smallest average
MAE among unsupervised photometric stereo methods, andthe second smallest average MAE among existing photometric stereo methods. Figure 5 further shows the estimated surface normals of LL22 [11] and ours on two DiLiGenT [18]objects. Benefiting from the data prior via our BRDF plugin,
our normal estimates are closer to the GT. The angular error maps primarily display errors in areas with cast shadows,
which our method does not explicitly address.
4. CONCLUSION
In this paper, we enhance unsupervised photometric stereo by
introducing a BRDF plugin that imposes data prior to constraint surface reflectance. Our BRDF plugin is designed as aconditional coordinate-based MLP trained on real-world measured BRDFs, where BRDF data is projected into a learned
latent space. Real-world and synthetic experiments show thatby leveraging this plugin, our proposed method achieves moreaccurate surface normal estimates compared to other unsupervised photometric stereo approaches, bypassing the limitations associated with the large-scale training datasets requiredin supervised photometric stereo methods.

05. ACKNOWLEDGEMENTS
This work and the first author were supported by JSPS KAKENHI (Grant No. JP23H05491). Boxin Shi was supported by
National Natural Science Foundation of China under Grant
No. 62136001.
6. REFERENCES[1] Robert J. Woodham, “Photometric method for determining surface orientation from multiple images,” Opticalengineering, 1980. 1, 4
[2] Hiroaki Santo, Masaki Samejima, Yusuke Sugano,Boxin Shi, and Yasuyuki Matsushita, “Deep photometric stereo network,” in Proc. of International Conference
on Computer Vision Workshops (ICCVW), 2017. 1[11] Junxuan Li and Hongdong Li, “Neural reflectance forshape recovery with shadow handling,” arXiv preprint
arXiv:2203.12909, 2022. 1, 3, 4[12] Wojciech Matusik, Hanspeter Pfister, Matt Brand, andLeonard McMillan, “A data-driven reflectance model,”in Proc. of SIGGRAPH, 2003. 1, 2, 3, 4[13] Szymon M Rusinkiewicz, “A new change of variables
for efficient brdf representation,” Rendering techniques,
vol. 98, pp. 11–22, 1998. 2[14] Shuiwang Ji, Wei Xu, Ming Yang, and Kai Yu, “3dconvolutional neural networks for human action recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 221–231, 2012. 2[3] Guanying Chen, Kai Han, and Kwan-Yee K. Wong,“PS-FCN: A flexible learning framework for photometric stereo,” in Proc. of European Conference on Computer Vision (ECCV), 2018. 1, 3, 4[15] Eric R Chan, Marco Monteiro, Petr Kellnhofer, Jiajun Wu, and Gordon Wetzstein, “pi-gan: Periodic implicit generative adversarial networks for 3d-aware image synthesis,” in Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp.
5799–5809. 2, 3

[4] Yakun Ju, Boxin Shi, Muwei Jian, Lin Qi, Junyu Dong,and Kin-Man Lam, “NormAttention-PSN: A highfrequency region enhanced photometric stereo networkwith normalized attention,” International Journal ofComputer Vision, 2022. 1[16] Vincent Sitzmann, Julien Martel, Alexander Bergman,David Lindell, and Gordon Wetzstein, “Implicit neural representations with periodic activation functions,”
Proc. of Annual Conference on Neural Information Processing Systems (NeurIPS), 2020. 2[5] Satoshi Ikehata, “CNN-PS: CNN-based photometricstereo for general non-convex surfaces,” in Proc. of European Conference on Computer Vision (ECCV), 2018.
1, 3, 4[17] Heng Guo, Hiroaki Santo, Boxin Shi, and Yasuyuki
Matsushita, “Edge-preserving near-light photometric stereo with neural surfaces,”
arXiv preprint
arXiv:2207.04622, 2022. 3[6] Fotios Logothetis, Ignas Budvytis, Roberto Mecca, andRoberto Cipolla, “PX-NET: simple and efficient pixelwise training of photometric stereo networks,” in Proceedings of the IEEE/CVF International Conference on
Computer Vision, 2021. 1[18] Boxin Shi, Zhipeng Mo, Zhe Wu, Dinglong Duan, SaiKit Yeung, and Ping Tan, “A benchmark dataset andevaluation for non-Lambertian and uncalibrated photometric stereo,” IEEE Transactions on Pattern Analysis
and Machine Intelligence, 2019. 4[7] Zhuokun Yao, Kun Li, Ying Fu, Haofeng Hu, and Boxin
Shi, “GPS-Net: Graph-based photometric stereo network,” Advances in Neural Information Processing Systems, 2020. 1, 4

[19] Jieji Ren, Feishi Wang, Jiahao Zhang, Qian Zheng,
Mingjun Ren, and Boxin Shi, “DiLiGenT102 : A photometric stereo benchmark dataset with controlled shapeand material variation,” in Proc. of IEEE Conference
on Computer Vision and Pattern Recognition (CVPR),
2022. 4[8] Yakun Ju, Kin-Man Lam, Wuyuan Xie, Huiyu Zhou,
Junyu Dong, and Boxin Shi, “Deep learning methods
for calibrated photometric stereo and beyond: A survey,” arXiv preprint arXiv:2212.08414, 2022. 1[9] Qian Zheng, Boxin Shi, and Gang Pan, “Summary studyof data-driven photometric stereo methods,” Virtual Reality & Intelligent Hardware, 2020. 1[10] Tatsunori Taniai and Takanori Maehara, “Neural inverse
rendering for general reflectance photometric stereo,”in Proc. of the International Conference on Machine
Learning (ICML). PMLR, 2018, pp. 4857–4866. 1, 4[20] Junxuan Li, Antonio Robles-Kelly, Shaodi You, and Yasuyuki Matsushita, “Learning to minify photometricstereo,” in Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019. 4[21] Qian Zheng, Yiming Jia, Boxin Shi, Xudong Jiang,
Ling-Yu Duan, and Alex C. Kot, “SPLINE-Net: Sparse
photometric stereo through lighting interpolation andnormal estimation networks,” in Proc. of International
Conference on Computer Vision (ICCV), 2019. 4