IW-FCV2022, The 28th International Workshop on Frontiers of Computer Vision

Online Illumination Planningfor Shadow-Robust Photometric Stereo
Hirochika Tanikawa1 , Ryo Kawahara1[0000−0002−9819−3634] , andTakahiro Okabe1[0000−0002−2183−7112]
Department of Artiﬁcial Intelligence,
Kyushu Institute of Technology
680-4 Kawazu, Iizuka, Fukuoka 820-8502, Japanokabe@ai.kyutech.ac.jp

Abstract. Photometric stereo is a technique for estimating normals ofan object surface from its images taken under diﬀerent light source directions. In general, photometric stereo suﬀers from shadows, because
almost no information on surface normals is available from shadowed pixels. In this paper, we propose an illumination planning for shadow-robust
Lambertian photometric stereo; it optimizes the light source directions
adaptively for an object of interest, because cast shadows depend on theentire shape of the object. More speciﬁcally, our proposed method iteratively adds the optimal light source for surface normal estimation bytaking the visibility and linear independence of light source directions
into consideration on the basis of the previously captured images of theobject. We implemented our illumination planning with a programmable
light source in an online manner, and achieve shadow-robust surface normal estimation from a small number of images.
Keywords: photometric stereo · illumination planning · attached shadow· cast shadow · visibility

1Introduction

Photometric stereo is a technique for estimating surface normals of an object ofinterest from its images taken from a ﬁxed viewpoint but under diﬀerent lightsource directions. The classical photometric stereo [18, 15] assumes the Lambertmodel and known light sources, and then estimates surface normals from atleast three images. The extension of the classical photometric stereo to nonLambertian surfaces, unknown light sources, and so on is still one of the mostactively studied topics in computer vision [1, 14].
In this paper, we focus on shadows, i.e. another aspect of the extension of theclassical photometric stereo, and propose photometric stereo robust to shadows.
In general, shadows are classiﬁed into attached shadows and cast shadows; attached shadow is local eﬀect and occurs when the angle between a surface normal
and a light source direction is obtuse, but cast shadow is global eﬀect and occurs
when a light source is blocked by another part of an object surface. Because- 94 -

almost no information on surface normals is available from shadowed pixels 1 ,we need to illuminate an object surface by using appropriate light sources sothat each pixel on the surface is suﬃciently illuminated.
Optimizing light sources, so-called illumination planning [11] is studied also
in the context of photometric stereo. Drbohlav and Chantler [6] address the
optimal light source conﬁgurations for the Lambertian photometric stereo on
the basis of the noise propagation analysis. For example, they show that, when
the number of light sources is three, the three light sources should be placed sothat those directions are orthogonal to each other. Unfortunately, however, theiranalysis does not take shadows into consideration, although attached shadows
occur even on convex objects in practice. More importantly, they ﬁnd the optimal
light source directions a priori independent of the shape of an object. Actually,
cast shadows occur on non-convex objects depending on the entire shapes of theobjects.
Accordingly, we propose an online illumination planning for shadow-robust
Lambertian photometric stereo. Our proposed method estimates surface normals of an object of interest by iteratively adding a novel image of the object
taken under the optimal light source direction. More speciﬁcally, we take thevisibility and linear independence of light source directions into consideration,and propose a method for optimizing the light source direction adaptively for an
object of interest on the basis of the previously captured images of the object.
We implemented our illumination planning by using a Liquid Crystal Display(LCD) as a programmable light source, and conﬁrmed the eﬀectiveness of our
method through a number of experiments using synthetic and real images.
The main contributions of this study are twofold. First, we propose a novel
illumination planning for shadow-robust Lambertian photometric stereo. Our
proposed method iteratively adds the optimal light source according to an objectof interest on the basis of the previous observations of the object. Second, we
implemented our illumination planning with a programmable light source in an
online manner, and achieve shadow-robust surface normal estimation from asmall number of images.

2

Related Work

Conventional Approach: Outlier Removal
Both attached shadows and cast shadows are considered as outliers, because
the pixel values in those shadows deviate from the Lambert model. Therefore, anumber of methods are proposed for detecting shadows in the images for photometric stereo and removing them as outliers from surface normal estimation.
For example, the shadows in photometric stereo images are detected by selecting the optimal three light sources out of four [2], by using RANdom SAmple
Consensus (RANSAC) [13, 10, 16], graph cut [3], and low-rank and sparse decomposition [19, 8]
1

It is known that surface normals can be recovered from attached shadows if a large
number of images taken under varying light source directions are given [12].

- 95 -

Our objective is to achieve shadow-robust photometric stereo too. In contrast
to the above methods, however, our focus is not on detecting shadows but on
illuminating an object surface in the context of active illumination. Speciﬁcally,
we illuminate an object surface so that each pixel is suﬃciently illuminated forsurface normal estimation.
Learning-Based Approach
The learning-based approach is getting more common also in photometricstereo. There are a number of learning-based methods for non-Lambertian photometric stereo and uncalibrated photometric stereo [5, 17, 4, 20]. It is certainthat the learning-based approach is suitable for dealing with non-Lambertian
BRDFs because the reﬂectance is local eﬀect. Unfortunately, however, cast shadows depend on the entire shape of an object, and then they are global eﬀects.
Therefore, eﬃciently handling cast shadows is diﬃcult for the learning-based
approach; a huge amount of training data is required in general.
Recently, Ikehata [7] and Li et al. [9] propose methods for eﬃciently handlingcast shadows in the learning-based approach by incorporating cast shadows into
so-called observation map. Similar to the conventional approach, their objective
is to remove the impact of cast shadows from surface normal estimation implicitly. On the other hand, our focus is on illuminating an object surface so thatsuﬃcient information is available for surface normal estimation.
Illumination Planning
Drbohlav and Chantler [6] address the optimal light source conﬁgurations
for the Lambertian photometric stereo. Speciﬁcally, they reveal the relationshipbetween the noises in pixel values and the expected errors of estimated surface
normals on the basis of noise propagation analysis, and derive the optimal lightsource conﬁgurations for given numbers of light sources. Unfortunately, however,their analysis does not take shadows into consideration and ﬁnds the optimal
light source directions a priori independent of the shape of an object.
In contrast to their method, we take account of shadows for optimizing
light source directions in the Lambertian photometric stereo. In particular, ourmethod optimizes the light source directions adaptively for an object of intereston the basis of the previously captured images of the object in an online manner,because cast shadows are global eﬀects depending on the entire shapes of theobjects.

3

Proposed Method

3.1

Overview

Our proposed method aims to illuminate an object surface from the optimal lightsource directions so that each pixel on the surface is suﬃciently illuminated. To
this end, our method iteratively ﬁnds the optimal light source direction on thebasis of the previously captured images of the object, and then capture a novel
image of the object under the optimal light source direction. The pipeline of our
method is as follows; our proposed illumination planning- 96 -

1. randomly selects three initial light source directions, and then captures three
images of the object under each of those light source directions,
2. ﬁnds out the pixel with the worst accuracy in surface normal estimation,
3. ﬁnds out the optimal light source direction for improving the accuracyof surface normal estimation at the worst pixel, and then captures a novel
image under the optimal light source direction,
4. repeats 2 and 3 a number of times,
and ﬁnally estimates surface normals from all of the captured images.
In Section 3.2, we describe how to estimate surface normals from images withshadows. Then, we explain the way of ﬁnding the worst pixel and the optimal
light source direction in Sections 3.3 and 3.4 respectively.
3.2

Estimating Surface Normals from Images with Shadows

Here, we explicitly formulate the Lambertian photometric stereo from images
with shadows. Let us denote the pixel value at the p-th pixel (p = 1, 2, 3, ..., P )on an object surface under the l-th light source direction (l = 1, 2, 3, ..., L) by
ipl . According to the Lambert model, the pixel value ipl is represented by

ipl = vpl sl np .

(1)Here, sl is the product of the direction sl and intensity of the l-th light source,
and np is the product of the surface normal np and albedo at the p-th pixel.
Both sl and np are unit vectors. We denote the visibility of the p-th pixel fromthe l-th light source direction by vpl ; vpl = 0 if the p-th pixel is in attachedshadow or cast shadow and vpl = 1 otherwise2 .
We can rewrite eq.(1) at the p-th surface point under the L diﬀerent lightsource directions by using matrices as⎞⎛⎞ ⎛⎞⎛   ⎞⎛
s1
vp1 s1
vp1 0 · · · 0
ip1
⎜ ⎟
 ⎟
⎜ ip2 ⎟ ⎜
⎟
⎜
0
v
·
·
·
0
s2 ⎟ 
p2⎜⎟ ⎜ vp2 s2 ⎟⎟⎜⎟ np = ⎜(2)⎟⎜⎜
⎜ .. ⎟ = ⎜
. ⎟
..
..
⎟ np ,⎟⎜⎠⎝⎝ . ⎠ ⎜.
⎠⎝ .. ⎠⎝.

0 0 · · · vpLipLsLvpL sL
ipL = VpL SL np .

(3)

Therefore, we can compute the scaled surface normal np by using the pseudoinverse matrix (VpL SL )† asnp = (VpL SL )† ipL = {(VpL SL ) (VpL SL )}−1 (VpL SL ) ipL ,(4)

if {(VpL SL ) (VpL SL )} is a regular matrix. Note that the surface normal np
with unit length is given by np = np /|np |. When VpL is the L × L identity
matrix, the estimation of the surface normal in eq.(4) results in the estimationfrom images without shadows.
2

The visibility is often used for representing cast shadows, but we use it for representing both attached shadows and cast shadows in this paper.

- 97 -

3.3

Finding Pixel with Worst Accuracy

In a similar manner to Drbohlav and Chantler [6], we can estimate the accuracyof the estimated surface normal via noise propagation analysis. Let us denote theestimated and ground truth scaled surface normals by np and n̄p respectively.
Then, we study the variance-covariance matrix Σ of the surface normal deﬁnedbyΣ = E (np − n̄p )(np − n̄p ) ,(5)
where E[ ] stands for the expectation value.
Suppose that the observed pixel value ĩpl is contaminated by additive noiseδpl and deviates from the ideal pixel value ipl 3 asĩpl = ipl + δpl ,(6)

we can deriveΣ = E {(VpL SL )† δpL }{(VpL SL )† δpL )}
= σ 2 (VpL SL )† {(VpL SL )† }
= σ 2 {(VpL SL ) (VpL SL )}− .

(7)Here, δpL = (δp1 , δp2 , δp3 , · · · , δpL ) , and we assume independent and identically
distributed noises whose mean and variance are 0 and σ 2 respectively. Since theMean Square Error (MSE) is proportional to the trace of the variance-covariancematrix, we obtain the MSE of the estimated surface normal as
MSE ∝ Tr {(VpL SL ) (VpL SL )}−1 .
(8)Therefore, our proposed method ﬁnds out the pixel p̂ with the worst accuracy
in surface normal estimation as(9)p̂ = arg max Tr {(VpL SL ) (VpL SL )}−1 .
pNote that the MSE depends on the surface points via the visibility matrix VpL 4 .
Intuitively, the MSE is larger for the pixel that is illuminated by a smaller numberof light sources with smaller linear independence. When VpL is the L×L identity
matrix, i.e. when we do not take account of shadows, the MSE results in thatof Drbohlav and Chantler [6].
3.4

Finding Optimal Light Source Direction

Suppose that the L images of an object are captured and the p̂-th pixel isselected as the pixel with the worst accuracy according to eq.(9). Our proposed3

Note that not the noise δpl but the visibility vpl is used for representing shadows asshown in eq.(1).
4
When multiple pixels have the same visibility matrix and therefore have the same
MSE, we randomly select one of the pixels as the worst pixel.

- 98 -

method ﬁnds the optimal light source direction, i.e. the (L + 1)-th light source
direction under which a novel image of the object is captured for improving
the accuracy of the estimated surface normal at the p̂-th pixel. Our clue tothe optimal light source direction is the visibility and linear independence; theoptimal light source direction should be visible from the p̂-th pixel and havethe largest linear independence from the previous light source directions withvp̂l = 1.
As mentioned in Section 1, our implementation makes use of an LCD as aprogrammable light source. Then, we represent the direction of a light sourceby the 2-D coordinate system of the display. We assume that our setup is geometrically calibrated in advance, and represent a light source by using the 2-D
coordinate (x, y) of the display and the 3-D vector s(x, y) interchangeably. Our
proposed method maximizes the cost function C(x, y) deﬁned as
C(x, y) = Cvis (x, y) × Clin (x, y)(10)

with respect to (x, y). Here, Cvis (x, y) and Clin (x, y) are the cost functions forthe visibility and linear independence of the light source s(x, y) respectively.
The cost function for the visibility has larger value if the light source direction(x, y) is likely visible from the p̂-th pixel and vice versa. More speciﬁcally, if
the direction (xl , yl ) of the l-th light source is visible from the p̂-th pixel, its
neighboring directions are also considered to be visible. On the other hand, if
the direction of the l-th light source is invisible, its neighboring directions arealso considered to be invisible. Therefore, we deﬁne the cost function for thevisibility by the linear combination of the Gaussian distributions asL(x−xl )2 +(y−yl )2(x−xc )2 +(y−yc )2
1
2w2
2w2
Cvis (x, y) =+(2vp̂l − 1)e−e−
. (11)
2πw2
l=1Here, the ﬁrst term corresponds to the direction of a camera (xc , yc ), and thesecond term corresponds to the previous L light sources. The coeﬃcient (2vp̂l −1) = 1 for visible light sources from the p̂-th pixel, and (2vp̂l − 1) = −1 forinvisible light sources. Because we consider an object surface visible from thecamera, the coeﬃcient of the ﬁrst term is 1. Since the cost function for thevisibility is considered as a likelihood, we adopt the upper and lower limits; weset Cvis (x, y) = 1 if Cvis (x, y) > 1, and Cvis (x, y) = −1 if Cvis (x, y) < −15 .
The cost function for the linear independence has larger value if the lightsource direction (x, y) has larger linear independence from the previous lightsource directions with vp̂l = 1. Therefore, we compute the principal directionsof those previous light sources, and deﬁne the cost function for the linear independence asClin (x, y) = max[s(12)
min s(x, y), −smin s(x, y)].
Here, we denote the 3-D eigenvector corresponding to the smallest eigenvalue of(Vp̂L SL ) by smin , and max[ ] returns the maximum argument.
5

We set the lower limit not to 0 but to 1 so that the gradient of the cost function
C(x, y) does not vanish for gradient-based optimization.

- 99 -

30

120

90

20

60
10
30
0

0
30

30
60

60
90
120

60

30

90

120

90
120

(a)

60

30

90

120

(b)

Fig. 1. Two objects used in the experiments with synthetic images: (a) a slit and (b)a wave. The camera and display are located above the objects.

Since the cost function in eq.(10) is nonlinear with respect to (x, y), theoptimization depends on the initial conditions. In our current implementation,
we used the interior point algorithm (fmincon in MATLAB), and tested multiple
random initial conditions and then selected the best solution that maximizes thecost function.

4

Experiments

To conﬁrm the eﬀectiveness of our proposed method, we conducted a numberof experiments using both synthetic and real images. In the experiments usingsynthetic images, we assumed a similar setup to our prototype system for theexperiments using real images shown in Fig. 8. We assumed that the visibility
vpl is 0 if the pixel value ipl is smaller than a threshold depending on the noiselevel of images, and vpl is 1 otherwise.
We changed√ the standard deviation w of the Gaussian distributions in eq.(11)as w = w0 / L so that the cost function for the visibility can represent ﬁner
details as the number of light sources L increases. As shown in Fig. 8, we used
an ultra wide display and normalized the scale of the coordinate system as theheight of the display is 1, i.e. the range of x is [0, 1], and we empirically set
w0 = 0.7.
4.1

Synthetic Images

Visibility
First, we show how the cost function for the visibility behaves actually. We
used a slit shown in Fig. 1(a) as an object of interest. For the sake of simplicity,
we investigated the visibility at a ﬁxed point on the lower plane of the object, andthe light source directions are randomly selected instead of using our illumination
planning.
Fig. 2 visualizes the cost function for the visibility in the display domain(x, y): (a) the ground truth and the visibilities when the numbers of light sources- 100 -

(a)(b)

precision:
65.35%(c)

precision:
75.08%(d)precision:
93.45%

Fig. 2. The cost function for the visibility in the display domain (x, y): (a) the groundtruth and the visibilities when the numbers of light sources L are (b) 10, (c) 30, and(d) 100 respectively.

(a)

(b)

(c)

(d)

Fig. 3. The cost function for the linear independence for diﬀerent sets of 10 light source
directions.

L are (b) 10, (c) 30, and (d) 100 respectively. Here, the visibility vpl is 1/0 at
white/black pixels in (a) the ground truth, and Cvis is 1/-1 at white/black pixels
from (b) to (d). The selected light source directions are superimposed in thoseimages: orange/light blue points correspond to vpl = 1/0 respectively. We cansee that the cost function for the visibility gets closer to the ground truth as
the number of light sources increases. In addition, we binarize the cost function
for the visibility to 1/0 when Cvis ≥ 0/ < 0, and computed the precision of thevisibility, i.e. the fraction of the light source directions with vpl = 1 among thosewith Cvis ≥ 0, when L =10, 30, and 100. We can see quantitatively that our
cost function for the visibility works well; the precision gets closer to 100% as
the number of light sources increases.
Linear Independence- 101 -

ground
truth

L=5

L=10

L=15
(a)

(b)

(c)

(d)

(e)

(f)

L=5

L=10

L=15

Fig. 4. The qualitative results of the slit when σ = 0.02: the ground truth surface
normals and (a) the estimated surface normals, (b) the errors of the surface normals,and (c) the selected light sources by using our proposed illumination planning, and (d)(e) (f) those of the random selection when L = 5, 10, and 15.

Second, we show how the cost function for the linear independence behaves
actually. For the sake of simplicity, we investigated the linear independence byusing dummy data. More speciﬁcally, we randomly selected 10 light source directions and assumed that a surface point of attention is shadowed under someof them.
Fig. 3 visualizes the cost function for the linear independence for diﬀerent
sets of 10 light source directions. Here, the linear independence Clin is 1/0 at
white/black pixels in the images. The selected light source directions are superimposed in those images: orange/light blue points correspond to vpl = 1/0
respectively. We can see that the cost function for the linear independence takes
larger values at the orthogonal directions to the visible light source directions
illuminating the surface point of attention as expected.
Surface Normal Estimation
Third, we compare the performance of our proposed illumination planningwith the random selection of light source directions for surface normal estimation.
We tested two objects: a slit and a wave in Fig. 1. The camera and display are- 102 -

ground
truth

L=5

L=10

L=15
(a)

(b)

(c)

(d)

(e)

(f)

L=5

L=10

L=15

Fig. 5. The qualitative results of the wave when σ = 0.02: the ground truth surface
normals and (a) the estimated surface normals, (b) the errors of the surface normals,and (c) the selected light sources by using our proposed illumination planning, and (d)(e) (f) those of the random selection when L = 5, 10, and 15.

located above the objects. In order to investigate the robustness to noises in
pixel values, we added zero-mean Gaussian noises, whose standard deviations
σ are 0.01, 0.02, and 0.04 for pixel values normalized to [0, 1], to the synthetic
images.
Fig. 4 shows the qualitative results of the slit when σ = 0.02: the groundtruth surface normals and (a) the estimated surface normals, (b) the errors of
the surface normals, and (c) the selected light sources by using our proposed
illumination planning, and (d) (e) (f) those of the random selection when L = 5,
10, and 15. The whiter the pixels in (b) and (e) are, the larger the errors in surface
normal estimation are. We can see that our illumination planning works better
than the random selection; our illumination planning achieves better accuracy
by using smaller number of light sources than the random selection. We can also
see that the light source directions selected by our illumination planning are
diﬀerent from those of the random selection. Here, the light blue points standfor the initial three light source directions, and the orange points stand for theiteratively added light source directions.

- 103 -

20

V 
V 
V 
V 

15

Angular error

Angular error

20

10
5
0

V 
V 
V 
V 

15
10
5

5

10
15
Number of light sources

0

20

5

10
15
Number of light sources

20

(b)

(a)

Fig. 6. The quantitative results of the slit; the angular error in degree vs. the numberof light sources under various noise levels: (a) our proposed illumination planing and(b) the random selection of light source directions.

V 
V 
V 
V 

30
20
10
0V 
V 
V 
V 

40
Angular error

Angular error

40

30
20
10

5

10
15
Number of light sources

20

0

5

10
15
Number of light sources

20

(b)

(a)

Fig. 7. The quantitative results of the wave; the angular error in degree vs. the numberof light sources under various noise levels: (a) our proposed illumination planing and(b) the random selection of light source directions.

Fig. 6 shows the quantitative results of the slit; the angular error in degree
vs. the number of light sources under various noise levels: (a) our proposed
illumination planing and (b) the random selection of light source directions.
We can see that the angular errors of our illumination planning converges tosmaller values for smaller number of light sources than the random selection.
Those results quantitatively show that our illumination planning works better
than the random selection.
The qualitative and quantitative results of the wave are shown in Fig. 5 andFig. 7 respectively. We can see that we obtain the similar results to those of
the slit. Those results for the wave also support that our proposed illumination
planning works better than the random selection. It is interesting that the lightsource directions selected by our illumination planning for the slit and the wave
are diﬀerent from each other. This is because our method ﬁnds out the optimal
light source directions depending on the shape of an object of interest. Actually,- 104 -

LCD

cameraobjectFig. 8. Our prototype system for the experiments using real images.

(a)

(b)

(c)

(d)

(e)

Fig. 9. The qualitative results using real images: (a) the images of target objects underambient light, the estimated surface normals when (b) L = 3, (c) 10, and (d) 20, and(e) the selected optimal light source directions: light blue points (L = 1 to 3) andorange points (L = 4 to 20).

the optimal light source directions for the wave mainly distribute around thecenter column of the display in order to illuminate the deep valleys of the wave.

4.2

Real Images

Fig. 8 shows our prototype system for the experiments using real images. It
consists of an ultra wide LCD 439P9H1/11 from PHILIPS and a polarimetriccamera BFS-U3-51S5P-C from FLIR. We assume suﬃciently smaller objects
than the distance between the display and the objects, and consider the small
white spots displayed on the LCD as distant light sources. The directions andintensities of those light sources are calibrated in advance by using the imagesof a mirror sphere and a diﬀuse reﬂector.

- 105 -

Our proposed online illumination planning, consisting of controlling the LCD
and camera, separating reﬂection components, ﬁnding the worst pixel, and ﬁnding the optimal light source, is implemented by using MATLAB. We separated
the diﬀuse and specular reﬂection components in the captured images on the basis of polarization, and used the former for the Lambertian photometric stereo.
The optimization of the cost function in eq.(10) takes about 0.05 sec for a single initial condition. Therefore, we can test 80 initial conditions during typical
exposure time of 4 sec.
We qualitatively show how our proposed illumination planning behaves forreal images, since the ground truth surface normals are unknown for real objects.
We tested three objects; a toy bench made of plastics, a milk pitcher and a cup
made of ceramic. Fig. 9 shows (a) the images of those objects under ambient
light, the estimated surface normals when (b) L = 3, (c) 10, and (d) 20, and (e)the selected optimal light source directions: light blue points (L = 1 to 3) andorange points (L = 4 to 20). Here, the orange points in (a) the images of theobjects stand for the selected worst pixels from L = 4 to 20.
The estimated surface normals qualitatively show that the accuracy signiﬁcantly improves as the number of light sources increases from (b) L = 3 to (d)20. Interestingly, we can see that (a) the worst pixels are often selected from
non-convex areas, and (e) the optimal light source directions are selected so thatthey can illuminate those pixels. Those results demonstrate that our proposed
illumination planning behaves as expected, and estimates surface normals froma small number of images.

5

Conclusion and Future Work

We proposed an illumination planning for shadow-robust Lambertian photometric stereo. Our proposed illumination planning takes the visibility and linear
independence of light source directions into consideration, and optimizes thelight source directions adaptively for an object of interest. We implemented ourillumination planning with a programmable light source in an online manner,and achieve shadow-robust surface normal estimation from a small number ofimages.
One direction of our future study is to incorporate interreﬂections into our
proposed illumination planning, because they are often observed on non-convex
surfaces but our illumination planning does not take account of them. The otherdirection is the integration with view planning; it optimizes both the light source
directions and the viewing directions for modeling an object of interest.
Acknowledgement: This work was supported by JSPS KAKENHI Grant Number JP20H00612.

References
1. Ackermann, J., Michael, G.: A Survey of Photometric Stereo Techniques. Now
Foundations and Trends (2015)- 106 -

2. Barsky, S., Petrou, M.: The 4-source photometric stereo technique for threedimensional surfaces in the presence of highlights and shadows. IEEE Trans. PAMI
25(10), 1239–1252 (2003)
3. Chandraker, M., Agarwal, S., Kriegman, D.: ShadowCuts: Photometric stereo withshadows. In: Proc. IEEE CVPR2007. pp. 1–8 (2007)
4. Chen, G., Han, K., Shi, B., Matsushita, Y., Wong, K.Y.K.: Self-calibrating deep
photometric stereo networks. In: Proc. IEEE CVPR2019. pp. 8739–8747 (2019)
5. Chen, G., Han, K., Wong, K.Y.K.: PS-FCN: A ﬂexible learning framework forphotometric stereo. In: Proc. ECCV2018. pp. 3–19 (2018)
6. Drbohlav, O., Chantler, M.: On optimal light conﬁgurations in photometric stereo.
In: Proc. IEEE ICCV2005. vol. 2, pp. 1707–1712 (2005)
7. Ikehata, S.: CNN-PS: CNN-based photometric stereo for general non-convex surfaces. In: Proc. ECCV2018. pp. 3–18 (2018)
8. Ikehata, S., Wipf, D., Matsushita, Y., Aizawa, K.: Robust photometric stereo using
sparse regression. In: Proc. IEEE CVPR2012. pp. 318–325 (2012)
9. Li, J., Robles-Kelly, A., You, S., Matsushita, Y.: Learning to minify photometricstereo. In: IEEE CVPR2019. pp. 7568–7576 (2019)
10. Mukaigawa, Y., Ishii, Y., Shakunaga, T.: Analysis of photometric factors based on
photometric linearization. JOSA A 24(10), 3326–3334 (2007)
11. Murase, H., Nayar, S.K.: Illumination planning for object recognition using parametric eigenspaces. IEEE Trans. PAMI 16(12), 1219–1227 (1994)
12. Okabe, T., Sato, I., Sato, Y.: Attached shadow coding: Estimating surface normals
from shadows under unknown reﬂectance and lighting conditions. In: Proc. IEEE
ICCV2009. pp. 1693–1700 (2009)
13. Okabe, T., Sato, Y.: Object recognition based on photometric alignment using
RANSAC. In: Proc. IEEE CVPR2003. pp. 221–228 (2003)
14. Shi, B., Mo, Z., Wu, Z., Duan, D., Yeung, S.K., Tan, P.: A benchmark dataset
and evaluation for non-Lambertian and uncalibrated photometric stereo. In: Proc.
IEEE CVPR2016. pp. 3707–3716 (2016)15. Silver, W.M.: Determining shape and reﬂectance using multiple images. Master’s
thesis, MIT (1980)
16. Sunkavalli, K., Zickler, T., Pﬁster, H.: Visibility subspaces: uncalibrated photometric stereo with shadows. In: Proc. ECCV2010. pp. 251–264 (2010)
17. Taniai, T., Maehara, T.: Neural inverse rendering for general reﬂectance photometric stereo. In: Proc. ICML2018. pp. 4857–4866 (2018)
18. Woodham, R.J.: Photometric method for determining surface orientation frommultiple images. Optical Engineering 19(1), 139–144 (1980)
19. Wu, L., Ganesh, A., Shi, B., Matsushita, Y., Wang, Y., Ma, Y.: Robust photometricstereo via low-rank matrix completion and recovery. In: Proc. ACCV2010. pp. 703–
717 (2010)
20. Zheng, Q., Jia, Y., Shi, B., Jiang, X., Duan, L.Y., Kot, A.C.: SPLINE-Net: Sparse
photometric stereo through lighting interpolation and normal estimation networks.
In: Proc. IEEE ICCV2019. pp. 8549–8558 (2019)- 107 -