Self-calibrating Photometric Stereo
Boxin Shi1∗ Yasuyuki Matsushita2

Yichen Wei2

Chao Xu1

Ping Tan3

1

Key Lab of Machine Perception (MOE), Peking University
2
Microsoft Research Asia
3
Department of Electrical & Computer Engineering, National University of Singapore

Abstract
We present a self-calibrating photometric stereo method.
From a set of images taken from a fixed viewpoint under
different and unknown lighting conditions, our method automatically determines a radiometric response function andresolves the generalized bas-relief ambiguity for estimating
accurate surface normals and albedos. We show that colorand intensity profiles, which are obtained from registeredpixels across images, serve as effective cues for addressing these two calibration problems. As a result, we develop
a complete auto-calibration method for photometric stereo.
The proposed method is useful in many practical scenarios
where calibrations are difficult. Experimental results validate the accuracy of the proposed method using various
real-world scenes.

1. Introduction
Photometric stereo [23] recovers surface orientations andalbedos from a set of images taken from a fixed viewpoint
under different lighting directions. There are two necessary
calibrations for a standard photometric stereo algorithm towork correctly. First, the camera needs to be radiometrically calibrated to obtain the scene irradiance from measured pixel values. Second, lighting directions and intensities need to be known to uniquely determine the surface.
With these two calibrations, surface orientations and albedos can be estimated uniquely from three images for Lambertian scenes.
Typically, these calibrations are performed separately
before applying photometric stereo algorithms. The radiometric camera calibration is often performed as a completely independent preprocessing step by, for example, using multiple images [15] captured from a fixed viewpoint
under a static lighting configuration with different exposure
times. In practice, however, this involves an additional datameasurement that is not always possible. The lighting direc∗ This work was done while the first author was visiting Microsoft Research Asia.

tions and intensities are often recorded by putting a mirror
sphere and a Lambertian sphere in the scene. These additional calibration objects make the process complicatedand could even introduce interreflections that eventually increase error. On the other hand, if the lighting parameters
are unknown, surface normals of a Lambertian scene canonly be recovered up to a 3 × 3 linear ambiguity [10]. If
the surface is known to be integrable, this ambiguity can be
reduced to the generalized bas-relief (GBR) ambiguity [3].
Without additional prior knowledge about the scene, or thelighting, this shape/lighting ambiguity cannot be resolved.
Hence, these two calibrations limit the practical applications of photometric stereo.
In this paper, we address these two calibration problems
without using any additional input data, or calibration objects. From a set of images taken from a fixed viewpoint under unknown and varying lightings with an unknown camera response function, our method automatically recovers
both the camera’s radiometric response function and the unknown lighting directions and intensities. Our method uses
color and intensity profiles to achieve this goal. A color
profile is defined as the set of measured RGB colors at a
pixel across images. A color profile draws a 3D curve in
the RGB color space. An intensity profile is defined as
an ordered sequence of scene irradiance at a pixel acrossimages. An intensity profile corresponds to a 2D curve in
the irradiance-time space, similar to the appearance profiles
shown by Koppal and Narasimhan [12]. Our method uses
color profiles to determine the radiometric response function, and intensity profiles to resolve the GBR ambiguity.
Putting them together, we build a self-calibrating photometric stereo method.
The rest of the paper is organized as follows. After discussing prior work, we describe the problem and overview
of our approach in Section 2. Section 3 describes the proposed radiometric calibration method based on the color
profiles, and Section 4 explains the method for GBR disambiguation using intensity profiles. Section 5 shows theexperimental results followed by discussions.

Sequence

Surface normal

Radiometriccalibration

GBR disambiguation

Linearizationof color profiles

Surface normal groupingby intensity profiles
I

B
R

Albedogrouping

G
t

Scene

Depth

Figure 1. Overview of the proposed method. The proposed method automatically performs radiometric self-calibration and GBR disambiguation from the input images and gives estimated surface normals and depth maps as output.

1.1. Related works
Our work is related to both radiometric calibration and
GBR disambiguation. Several methods have been proposed to estimate camera response functions. One of
the most widely used methods is Mitsunaga and Nayar’s
method [15], which uses multiple images of a static scene
with multiple different known exposure times. More recent methods work under more general imaging conditions.
Lin et al. [13] proposed a method that only requires a single color image. Later they extended it to work with a single gray-scale image [14]. These methods are based oncolor/intensity mixtures on edge boundaries. There are a
few works that allow scene/camera movements during data
capturing. Kim and Pollefeys [11] computed pixel correspondence across image frames for radiometric calibration.
Grossberg and Nayar [8] used intensity histograms of two
images of different exposure rather than exact pixel correspondence. Wilburn et al. [22] showed the response function can be recovered from a motion blur in a single image. Unlike all of these methods, our radiometric calibration method uses photometric stereo images captured by astatic camera under varying lighting conditions.
The GBR ambiguity is an intrinsic shape/lighting ambiguity for uncalibrated photometric stereo with unknown
lighting parameters. This ambiguity arises because different
combinations of shapes and lightings can produce the same
appearance. To resolve the GBR ambiguity, additional cues
are necessary. Prior knowledge about surface albedo can beused to resolve the GBR [10, 1]. Chandraker et al. [4] usedinterreflections in a Lambertian scene to resolve the ambiguity. It has been shown that the GBR can also be resolved
if the surface reflectance is known to be Lambertian compounded with a Torrance-Sparrow specular reflection [7], orwith specular reflections containing specular spikes [6, 5].
More recently, Tan et al. [20, 21] showed that the GBR canbe reduced and resolved in a stratified manner for isotropic
surfaces. Besides resolving the GBR ambiguity, Basri et al.
[2] proposed a general solution under environment lighting.

Furthermore, to handle various BRDFs, Sato et al. [19] suggested using the similarity in radiance changes and Okabe etal. [17] utilized the attached shadow. Our method takes adifferent approach from these. As we will see later in thispaper, our method can achieve a high-accuracy estimation
using the intensity profile as a cue.
The radiometric calibration and GBR disambiguation
have been discussed separately in prior works. To achieve
a complete auto-calibration in photometric stereo, we solve
the mixture of these problems. We take a fundamentallydifferent approach to achieve this goal using the color andintensity profiles.

2. Problem statement
We perform photometric stereo with images taken froma fixed viewpoint under varying and unknown directional
lighting (both intensities and directions are unknown) using a camera where the response function is also unknown.
We address the two calibration problems in this setting, i.e.,radiometric camera calibration and GBR disambiguation.
In this paper, we use the following imaging model:
M = f (I) = f (ρn) · (El) ,(1)

where M is the recorded pixel value, and ρ and n are thesurface reflectance (or albedo) and the surface normal vector at a pixel. E and l are the intensity and direction of thedirectional lighting. I is the irradiance, and f is the camera’s radiometric response function. Since the albedo ρ andsurface normal vector n are location dependent and lighting intensity E and direction l are image dependent, we use
subscripts i and j to index pixels and images respectively.
For multiple images under varying directional lighting, theimaging model can be written as
Mij = f (Iij ) = f (ρi ni ) · (Ej lj ) .

(2)

Our goal is to estimate ρi , ni , Ej , lj and f from the observations Mij .

Radiometric response function The goal of radiometriccalibration is to estimate the radiometric response function
f that maps the irradiance I to a pixel intensity M . As the
response function f is a monotonic function, there is always
a unique inverse mapping g(= f −1 ). Radiometric calibration amounts to estimating the inverse response function g.
The generalized bas-relief ambiguity As in previous
work [10], the photometric stereo problem is formulated ina matrix form:
I = SL,(3)

where I is a P × F matrix containing irradiance values
from all images, and P and F are the number of pixels
and images respectively. Each row of I corresponds to a
pixel position in an image, and each column corresponds
to a different image. The surface matrix S ∈ RP ×3 represents surface properties, i.e., the i-th row of S encodes
the surface normal at the i-th pixel scaled by its albedo asSi∗ = ρi ni . The lighting matrix L ∈ R3×F represents thelighting directions and intensities, i.e., the j-th column ofthe matrix L corresponds to the lighting direction in the jth image scaled by its intensity as L∗j = Ej lj . When L is
unknown, S and L can be estimated only up to a GBR transformation G by enforcing integrability over the surface as
I = SL = (S0 G)(G−1 L0 ). The GBR transformation G isa three-parameter matrix:




λ
0 0
1 0 0
1
λ 0 . (4)G =  0 1 0  , G−1 =  0
λ−µ −ν 1µ ν λThe overview of the proposed method is illustrated inFig. 1. Our method uses color profiles for radiometric calibration (Section 3) and intensity profiles for resolving GBR
ambiguity (Section 4).

3. Radiometric calibration using color profiles
This section describes our method for determining theradiometric response function using color profiles. A color
profile is a set of RGB color values at the same coordinate
in a set of photometric stereo images. Our method capitalizes on the fact that color profiles form straight lines in the
RGB color space when the radiometric response function f
is linear. On the other hand, if f is nonlinear, color profiles
form nonlinear curves in the RGB space.
To understand this relationship between color profiles
and the radiometric response function f , let us consider theratio between red, green and blue colors at a pixel. In acolor image, Eq. (2) is evaluated at each color channel, i.e.,M r = f (I r ), M g = f (I g ) and M b = f (I b ), where superscripts r, g, and b correspond to the RGB color channels. If the radiometric response function f is linear, i.e.,

0.5
0.4

B

0.3
02
0.2
0.1
0
0.8

0.6
G

0.4

0.2

0.2

0

0.4

0.8

0.6
R

Figure 2. Color profiles of a real-world scene taken by a camera with a non-linear response function. The profiles show clear
curves in the RGB space.

f (x) = kx, the color ratio at a Lambertian point is(M r : M g : M b ) = (kI r : kI g : kI b ) = (ρr : ρg : ρb ), (5)which is constant over the photometric stereo image sequences. Hence, in this case, a color profile is a straight linewith direction (ρr , ρg , ρb ) passing through the origin in the
RGB space. On the other hand, if f is nonlinear, this ratio
will change over the image sequence. As a result, the color
profile bends to a nonlinear curve in the RGB space. This
result is consistent with the empirical observation in [18].
Examples of color profiles obtained from a real-world scene
are shown in Fig. 2.
The correct inverse response function g (g(M ) = I)should linearize the curved color profiles. Using this property as a constraint, we estimate the inverse response function g that maps curved color profiles to straight lines. We
first define a nonlinearity measurement for color profiles toassess how the profiles are bended. We then formulate theestimation problem in an energy minimization framework
to derive the inverse response function g.
Degree of nonlinearity We assess the degree of nonlinearity of a transformed color profile by its line fitting error.
Let us denote the RGB triplet of the measured color value
as M and use g(M ) to represent that the inverse response
function g is applied to each element of M independently.
For a color profile at the i-th pixel, it is computed as
Di (g) =X k(X 0 − X 1 ) × (g(M ij ) − X 1 )ki

ji

kX 0i − X 1i k

i

,

(6)

where X 0i and X 1i are two points in the RGB space thatdefine the line best fits to the transformed color profileg(M i∗ ). Intuitively, D measures the summed Euclidean
distance between g(M ij ) to the fitted line. It approaches tozero when g is the correct inverse function.

I = g(M ) =K
X

ck M k .

(7)

k=0

We estimate the coefficients c = {c0 , ..., cK } and fittedlines X = {X 0i , X 1i |i ∈ Ω} by minimizing the summed
nonlinearity for each color profile i ∈ Ω:
1 X
Di (c),{ĉ, X̂ } = argminc,X |Ω| i∈Ω(8)where |Ω| is the number of color profiles used for the computation. The minimization is performed with normalizedcolor values in the range [0, 1].
For minimization, we put two constraints on the inverse
radiometric response function g, namely monotonicity andboundary conditions, which are general for any inverse response functions. The monotonicity constraint is represented as ∂g/∂M > 0, and the boundary conditions aredescribed as g(0) = 0 and g(1) = 1. We use these boundary conditions
as a hard constraint by P
setting c0 = 0 and
PK−1K
cK = 1 − k=1 ck because g(1) = k=1 ck = 1. The
final formulation of the energy minimization becomesX  ∂g(t) 
1 X{ĉ, X̂ } = argmin
Di (c) + λ
H −
. (9)
∂M
c,X |Ω|
ti∈ΩHere, H(·) is the Heaviside step function (H(x) = 1 whenx ≥ 0, and H(x) = 0, otherwise). The derivative ∂g/∂M
are assessed at various t in the range of [0, 1].
Optimization of Eq. (9) is performed with the NelderMead simplex method [16] due to its simplicity, but othersimilar methods could be used as well. We initialize g as alinear function and X as least square fitted lines from color
profiles.

4. GBR disambiguation using intensity profiles
This section describes the proposed method for resolving
the GBR ambiguity using intensity profiles. By Hayakawa’s
singular value decomposition (SVD) method [10], the irradiance I in Eq. (3) can be factorized into a surface component S and a lighting component L, up to a linear ambiguity. If albedos are known at 6 different normals or lighting
intensity is known at 6 images, this linear ambiguity canbe reduced to a rotational ambiguity. On the other hand,

0.5Group 1
Group
p2

04
0.4

Irraddiance

Energy function We compute the inverse response function g from a set of color profiles Ω by minimizing the nonlinearity. We use a polynomial representation for the inverse response function to reduce the computational complexity as in [15]. Thus, the inverse response function g is
described by a small number of unknown coefficients ck in
the form of K-order polynomial as

0.3
0.2
0.1
0
1

2

3

4

5

6
7
8
Frame number

9

10

11

12

Figure 3. Intensity profiles from a real-world scene. Group 1 andGroup 2 represent two different surface normal groups. Withineach group, the intensity profiles bear the similar shapes.

Yuille et al. [25] showed the original linear ambiguity canbe reduced to a GBR ambiguity by enforcing the integrability constraint.
We observe that the intersection of the rotational groupand the GBR transformation group contains only the identity matrix. Hence, if both constraints from [10] and [25]can be used, the original linear ambiguity is fully resolved.
To do so, we first apply the integrability constraint to derivea result up to the GBR ambiguity. We then resolve the GBR
ambiguity by identifying pixels that have different normals
but with the same albedo.
Suppose we are given pixels that have the same albedo
a but different normals. The GBR transformation should
satisfy ||si G|| = a, i.e.,si CsTi = a2 ,

(10)

where


1
C = GGT =  0
µ

0
1
ν


µ
,
ν
µ2 + ν 2 + λ2

(11)e = µ2 + ν 2 +and si is the i-th row vector of S. Let λ2
λ be an auxiliary variable. The constraint in Eq. (10) is
e a2 }. Therefore, given at
linear in all 4 unknowns {µ, ν, λ,least 4 pixels with different normals but with the common
albedo, the GBR ambiguity can be resolved. Our goal is
to automatically identify these pixels to resolve the GBR
ambiguity.
Surface normal grouping Following Koppal andNarasimhan’s work [12], we use intensity profiles to cluster
surface normals. As shown in Fig. 3, pixels with the same
surface normal show strong correlation in their intensity
profiles, while pixels with different surface normals becomeless correlated. Since we work in the irradiance domain,
instead of using the extrema of the profiles as in [12], we
simply use the correlation of the entire profiles. This allows
us to use fewer images for surface normal clustering. The
Pearson’s correlation rii0 between two intensity profiles Ii

OWL (12)CAT (12)SHEEP (12)OWL (12)PILLOW (15)SHEEP (12)Figure 5. Images of the real-world scene for experiments with thenumber of images used.

Input

Normal grouping

Albedo groupingsame albedo and the same surface normal. Since we aim

Figure 4. Surface normal grouping (middle) and albedo groupingat finding pixels with the same albedo but different surface(right) on the C AT scene. One of the input images is shown onPDF
the createdwith pdfFactory Pro trial version www.pdffactory.com
normals,
only one pixel from each subgroup is selected.
left. Different colors indicate different cluster labels.

and Ii0 is assessed byPF
rii0 =

j=1 (Iij − I i )(Ii j − I i )0(F − 1)σi σ

0

,

(12)

i0

NormalAlbedoofgrouping
I i is the mean value
ofgrouping
the i-th row vector
matrixwhere Input
I, and σi is its standard deviation. Using the correlation as
a distance, we perform K-means clustering to partition images into a set of groups that have similar surface normals.
Hence, by selecting from different groups, we can obtaincreated with pdfFactory
Pro trial
versionwww.pdffactory.com
pointswithdifferent
normal directions. For this purpose,
we use a large K. An example of the surface normal grouping result is shown in Fig. 4 (middle).

Albedo grouping Our albedo grouping aims at identifying pixels with the same albedo. As we have discussed in
Section 3, the RGB ratio of a Lambertian scene point should
be constant (ρr : ρg : ρb ) across images in the irradiancedomain. Hence, we can approximately measure the similarity of pixel albedos by their difference of chromaticity.
To reduce the influence of shadows and noise, we computethe chromaticity of each pixel from its averaged irradiance¯ The chromaticity is defined as a vector of two elvalue I.
ements [I¯r /(I¯r + I¯g + I¯b ), I¯g /(I¯r + I¯g + I¯b )]. We thencluster pixels in the chromaticity domain to obtain pixels
with similar albedo. Similar to the surface normal grouping, we apply K-means clustering using the Euclidean distance of chromaticity vectors. As a result, we obtain albedo
groups as shown in Fig. 4 (right).
Pixel selection We combine the two grouping results toautomatically select pixels with the same albedo but different surface normals.
First, our method selects reliable albedo groups. All thealbedo groups are sorted by their variances, and half of thegroups with larger variances are eliminated. After that, ineach albedo group, we further remove pixels that have alarge distance from the group center in the chromaticity domain. Specifically, pixels that have a distance larger than
the average are discarded. The selected groups are split into
subgroups based on the surface normal grouping result. The
resulting subgroups become the groups of pixels with the

To implement this pixel selection algorithm, we havetested two different approaches. One is random selection
from subgroups, and the other is an optimization approach.
The second approach selects pixels that minimize the summation of the surface normal similarity among selected pixels. This optimization problem is discrete and can be efficiently solved using a belief propagation algorithm [24].
Although the optimization-based method theoretically guarantees to select pixels with diverse surface normals, wefind that the random sampling produces as good result as
the optimization-based method when the number of surface
normal groups is high. For this reason, we use the random
sampling method in this work.
Finally, a set of pixels are selected from R albedo groupsof unknown albedo values {a1 , a2 , . . . , aR }. From thesepixels, we obtain multiple linear equations Eq. (10) withe a2 , · · · , a2 } to resolve the GBR ambiunknowns {µ, ν, λ,1R
guity by a least square solution.

5. Experiments
We use four real-world scenes, C AT, OWL (courtesyof Dan Goldman and Steven Seitz1 ), P ILLOW and S HEEP
scenes, to verify our method. The scenes are shown in
Fig. 5. C AT and OWL are radiometrically calibrated, while
P ILLOW and S HEEP are recorded by a Nikon D70 andCanon 20D, respectively. Both of these cameras have nonlinear radiometric response functions. In this experiment,
we first quantitatively evaluate the radiometric calibration
part. Second, we use the C AT and OWL scenes to assess
the proposed GBR disambiguation method. Finally, weevaluate the entire pipeline with both radiometric calibration and GBR disambiguation. We implement our method
using Matlab. The optimization of Eq. (9) can be implemented simply by a Matlab built-in function “fminsearch”.
Throughout our experiments, we set λ = 5000, |Ω| = 100(100 color profiles), K = 200 for the surface normal grouping, and K = 20 for the albedo grouping in two K-means
stages. With an Intel E6550 (2.33GHz) CPU, the entire process took 10 minutes without optimization.
1 http://www.cs.washington.edu/education/courses/csep576/05wi//projects/project3/project3.htm

InputNormal grouping

Input

Normal grouping

Albedo grouping

Inverse response function
1
0.9

Estimated
Ground truth

Nikon D70

Normalized irrradiance

0.8
0.7

RMSE:
0.02520.6
0.5

Disparity:
0.03870.4
0.3

Std. dev.:
0.0131

0.2
0.1
0
0

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Our method

1

Normali ed measurement
Normalized
meas rement

Minimum
entropy [1]Inverse response function
1
0.9

Estimated
Ground truth

Canon 20D

Normalizedd irradiance

0.8
0.7

RMSE:
0.02540.6
05
0.5

Disparity:
0.04590.4
0.3

Std. dev:
0.0286

0.2
0.1
0
0

Figure 7. Surface normal grouping (middle) and albedo grouping(right) on the S HEEP scene. One of the input images is shown on
the left. Different colors indicate different cluster labels.

Angular error
Mean
Std. dev.
Mean
Std. dev.

PDF created with pdfFactory Pro trial version www.pdffactory.com

0.1

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Normalized measurement

Figure 6. Result of radiometric calibration. The mean curve of
estimated inverse response functions with standard deviation are
shown in comparison with the ground truth.

Radiometric calibration result Using a Nikon D70 andCanon 20D, we take 50 datasets for testing the radiometriccalibration method. From each dataset, we randomly select pixels to produce color profiles. The result is shownin Fig. 6 in comparison with the ground truth obtained by
Mitsunaga and Nayar’s method [15]. The averages of root
mean squared error (RMSE) and disparity (the maximum
difference) are computed from all the 50 datasets. The
statistics show that our method consistently works well withvarious scenes and different cameras.
Surface normal and albedo grouping result We show
an example of the grouping result of the S HEEP scene
in Fig. 7. It depicts the accuracy of the grouping result for
the following GBR disambiguation.
GBR disambiguation result We evaluate the proposed
GBR disambiguation method using the C AT and OWL

Albedo grouping

C AT
6.15
2.83
15.82
6.15

OWL
10.47
4.75
27.10
6.08

Table 1. GBR disambiguation results. Mean and standard deviation of the angular error of the estimated normal maps. Our
method gives lower errors compared to the minimum entropy
method in these two scenes.

scenes. Fig. 8 and Fig. 9 show the results.2 On the top
row, the color coded normal maps (x, y, z components arelinearly mapped in the RGB channel) are shown. On thebottom row is a depth map integrated from the normal
map by Poisson solver. For a validation, we show theground truth obtained by the calibrated photometric stereo
on the left. We compare our method with the minimum entropy method [1]. Because our method uses surface normalgroups in addition to the albedo information, it performs
well in GBR disambiguation. The average angular errors
are summarized in Table 1. Comparing the results between
the C AT and OWL scenes, because of the smoothly varying
albedo of OWL scene, the performances of both our method
and the minimum entropy method degrade.
Self-calibrating photometric stereo result To test theentire pipeline, we use the P ILLOW and S HEEP scenes taken
by a Nikon D70 and Canon 20D, respectively. The results
are shown in Fig. 10 and Fig. 11. In the figures, from left
to right, we show the calibrated result (known lighting andresponse function), the result of our method with and without the radiometric calibration step. These results show that
our method produces results that are close to the calibrated
data, although we only use the input images taken under unknown lighting parameters with an unknown response function. The importance of radiometric calibration can also beseen clearly in the side-by-side comparison. Table 2 shows
the quantitative comparison. Our self-calibration method
significantly improves the accuracy. Fig. 12 shows the rendering of the reconstructed surface.
2 We recommend viewing the electronic version of this paper for better
visualization.

NormalDepth

NormalCalibrated

Our method

Min-entropy

NormalDepth

Calibrated
Our method W/o radiometric calib.
Figure 10. Result of the proposed method for P ILLOW scene. Fromleft to right, calibrated, our method with and without radiometriccalibration.

NormalDepth

Figure 8. GBR disambiguation result for C AT scene. From left PDF
to created with pdfFactory Pro trial version www.pdffactory.com
right, calibrated, our method and minimum entropy method [1] areshown. Top row shows the computed surface normal, and the bottom row shows the integrated depth maps. In the depth maps, thered color represents nearer, and blue color corresponds to further
depth values. The reference sphere shows the normal directions.

Calibrated
Our method W/o radiometric calib.
Figure 11. Result of the proposed method for S HEEP scene. Fromleft to right, calibrated,
our methodPillow
with and withoutSheepradiometric
Angular error
calibration.

Depth

PDF created with pdfFactory Pro trial version www.pdffactory.com

Without
radiometric calib.

mean

16.08

20.18

variance

7.35

6.86

Our
complete solution

mean

5.63

7.30

variance

3.71

3.02

PDF created with pdfFactory Pro trial version www.pdffactory.com

Calibrated

Our method

Min-entropy

Figure 9. GBR disambiguation result for OWL scene. From left
to right, calibrated, our method and minimum entropy method [1]Angularare shown. Top row showstheerrorcomputedCat
surface normal, andOwl
the
Min-entropy
mean
15.82
27.10
bottom row shows the integrated depth maps.
std. dev.
Our method

6.15

mean
Angular error 6.15
P ILLOW

6.08
10.47
S HEEP
4.75
7.30
3.02
20.18
6.86

Figure 12. The surfaces of CAT and PILLOW scenes.

6. Conclusion and discussion

Auto-calibration for photometric stereo is important forphotometric stereo to be applicable in various settings.
Std. dev.
3.71
In this paper, we propose a self-calibrating photometricWithout
Mean
16.08
stereo method that handles both radiometric calibration andradiometric calib.
Std. dev.
7.35
GBR disambiguation. The mixture of these problems issolved by analyzing color/intensity profiles in the RGB and
PDF created
Pro trialerror
versionwww.pdffactory.com
Table with
2. pdfFactory
Mean angularandstandard deviation of selfirradiance-time domains. We show that radiometric calibracalibrating photometric stereo result, and the result without radiotion can be performed from images from a fixed viewpoint
metric calibration.
under varying lightings. We also develop a method to automatically resolve the GBR ambiguity from at least four
PDF created with pdfFactory Pro trial version www.pdffactory.com
pixels with different normals but the same albedo.
Our method

std. dev.Mean

2.83 5.63

Limitations Our radiometric calibration method has acouple of limitations. First, it cannot work with gray-scale
images and scenes. Second, it cannot handle a certain classof response functions f where f (x)/f (ax) = const. holds.
This is a special case when the response function is nonlinear while the color profiles remain straight. This classof function includes f (x) = xn , n ∈ R, when boundary
conditions f (0) = 0 and f (1) = 1 is imposed.
To assess the practical limitation of our method, we
tested our method using the database of response functions(DoRF) [9] with a simulation dataset. We have confirmedthat all the response functions in the database, including
gamma correcting functions, bend the color profiles in the
RGB color space. Among the degenerate case where the response function is in the form of f (x) = xn , only the linear
response function f (x) = x is likely dominant in practice.3
Therefore, by setting the initial guess of the response function as a linear function, our method works well for a wide
variety of real-world camera response functions.

Acknowledgement
The authors would like to thank Neil Alldrin for providing the code to compare the results. Boxin Shi and Chao Xu
was partially supported by the Chinese and Beijing Natural Science Foundation (Grant No.60975014 and 4102024).
Ping Tan was supported by Singapore MOE Grant R-263000-555-112.

References[1] N. Alldrin, S. P. Mallick, and D. J. Kriegman. Resolving thegeneralized bas-relief ambiguity by entropy minimization. In
Proc. of Computer Vision and Pattern Recognition, pages 1–
7, 2007. 2, 6, 7
[2] R. Basri, D. Jacobs, and I. Kemelmacher. Photometric stereo
with general, unknown lighting. Int’l Journal of Computer
Vision, 72(3):239–257, 2007. 2[3] P. Belhumeur, D. J. Kriegman, and A. L. Yuille. The basrelief ambiguity. Int’l Journal of Computer Vision, 35(1):33–
44, 1999. 1[4] M. Chandraker, F. Kahl, and D. J. Kriegman. Reflections on
the generalized bas-relief ambiguity. In Proc. of Computer
Vision and Pattern Recognition, pages 788–795, 2005. 2[5] O. Drbohlav and M. Chantler. Can two specular pixels calibrate photometric stereo? In Proc. of Int’l Conf. on Computer Vision, pages 1850–1857, 2005. 2[6] O. Drbohlav and R. Sara. Specularities reduce ambiguity of
uncalibrated photometric stereo. In Proc. of European Conf.
on Computer Vision, pages 46–62, 2002. 2[7] A. S. Georghiades. Incorporating the torrance and sparrowmodel of reflectance in uncalibrated photometric stereo. In
Proc. of Int’l Conf. on Computer Vision, pages 816–823,
2003. 2[8] M. D. Grossberg and S. K. Nayar. Determining the camera
response from images: What is knowable? IEEE Trans.
3 The linear response as well as other response functions in the form off (x) = xn does not exist in the DoRF.

on Pattern Analysis and Machine Intelligence, 25(11):1455–
1467, 2003. 2[9] M. D. Grossberg and S. K. Nayar. Modeling the space of
camera response functions. IEEE Trans. on Pattern Analysis
and Machine Intelligence, 26(10):1272–1282, 2004. 8[10] H. Hayakawa. Photometric stereo under a light source with
arbitrary motion. Journal of the Optical Society of America,
11(11):3079–3089, 1994. 1, 2, 3, 4
[11] S. Kim and M. Pollefeys. Radiometric alignment of image
sequences. In Proc. of Computer Vision and Pattern Recognition, pages 645–651, 2004. 2[12] S. J. Koppal and S. G. Narasimhan. Clustering appearance
for scene analysis. In Proc. of Computer Vision and Pattern
Recognition, pages 1323–1330, 2006. 1, 4
[13] S. Lin, J. Gu, S. Yamazaki, and H. Shum. Radiometric calibration from a single image. In Proc. of Computer Vision
and Pattern Recognition, pages 938–945, 2004. 2[14] S. Lin and L. Zhang. Determining the radiometric response
function from a single grayscale image. In Proc. of Computer
Vision and Pattern Recognition, pages 66–73, 2005. 2[15] T. Mitsunaga and S. K. Nayar. Radiometric self-calibration.
In Proc. of Computer Vision and Pattern Recognition, pages
374–380, 1999. 1, 2, 4, 6
[16] J. A. Nelder and R. Mead. A simplex method for function
minimization. Computer Journal, 7(4):308–313, 1965. 4[17] T. Okabe, I. Sato, and Y. Sato. Attached shadow coding: estimating surface normals from shadows under unknown reflectance and lighting conditions. In Proc. of Int’l Conf. on
Computer Vision, pages 1693–1700, 2009. 2[18] I. Omer and M. Werman. Color lines: Image specific color
representation. In Proc. of Computer Vision and Pattern
Recognition, pages 946–953, 2004. 3[19] I. Sato, T. Okabe, Q. Yu, and Y. Sato. Shape reconstruction
based on similarity in radiance changes under varying illumination. In Proc. of Int’l Conf. on Computer Vision, pages
1–8, 2007. 2[20] P. Tan, S. P. Mallick, L. Quan, D. J. Kriegman, and T. Zickler.
Isotropy, reciprocity and the generalized bas-relief ambiguity. In Proc. of Computer Vision and Pattern Recognition,
pages 1–8, 2007. 2[21] P. Tan and T. Zickler. A projective framework for radiometricimage analysis. In Proc. of Computer Vision and Pattern
Recognition, pages 2977–2984, 2009. 2[22] B. Wilburn, H. Xu, and Y. Matsushita. Radiometric calibration using temporal irradiance mixtures. In Proc. of Computer Vision and Pattern Recognition, pages 1–7, 2008. 2[23] R. Woodham. Photometric method for determining surface orientation from multiple images. Optical Engineering,19(1):139–144, 1980. 1[24] J. S. Yedidia, W. T. Freeman, and Y. Weiss. Understanding
belief propagation and its generalizations. In Proc. Int’l Joint
Conf. on Artificial Intelligence, pages 239–269, 2001. 5[25] A. L. Yuille, D. Snow, R. Epstein, and P. N. Belhumeur.
Determining generative models of objects under varying illumination: Shape and albedo from multiple images using
svd and integrability. Int’l Journal of Computer Vision,
35(3):203–222, 1999. 4