Near-light Photometric Stereo
with Symmetric Lights
Lilika Makabe, Student Member, IEEE, Heng Guo, Hiroaki Santo, Member, IEEE,
Fumio Okura, Member, IEEE, and Yasuyuki Matsushita, Senior Member, IEEE
Abstract—This paper describes a linear solution method for near-light photometric stereo by exploiting symmetric light source
arrangements. Unlike conventional non-convex optimization approaches, by arranging multiple sets of symmetric nearby light source
pairs, our method derives a closed-form solution for surface normal and depth without requiring initialization. In addition, our method
works as long as the light sources are symmetrically distributed about an arbitrary point even when the entire spatial offset isuncalibrated. Experiments showcase the accuracy of shape recovery accuracy of our method, achieving comparable results to thestate-of-the-art calibrated near-light photometric stereo method while significantly reducing requirements of careful depth initializationand light calibration.
Index Terms—Near-light photometric stereo, Linear solution, Symmetric light distribution, Uncalibrated near-light position✦

1

I NTRODUCTIONPHOTOMETRIC stereo aims to estimate surface normals fromimages captured under varying lighting conditions. Most
conventional settings employ the distant light source assumption, where point light sources are infinitely far away. Under aLambertian assumption, this simplification allows a linear image
formation model, leading to a closed-form solution for surface
normals given three or more images taken under different lighting
conditions [1]. In practice, light sources are located near thetarget scene, and the distant light source assumption breaks down,
resulting in a near-light photometric stereo problem. Near-light
photometric stereo remains challenging due to the variation ofincident light directions and light fall-off effect, both dependingon the scene point location. Even with the Lambertian reflectanceassumption, the problem becomes non-linear since the surface
normals and surface point locations need to be jointly determined.
Prior approaches to near-light photometric stereo rely on nonconvex optimization due to the non-linear nature of the problem [7], [8], [9], [10], [14], [16]. These methods commonly require
the initial guess of the scene point locations or a depth map witha calibrated camera. While these methods yield plausible surface
shape recovery, the result is naturally susceptible to the initial
guess because of the non-convexity. Liu et al. [11] proposesa ring-light setting for near-light photometric stereo to obtain areliable initial guess of the scene point locations. However, evenwith the improved initial guess, the issue of being affected by the
initial guess still remains. Thus, it is wanted a stable near-light
photometric stereo method that is unaffected by the initial guess.
This paper shows that a symmetric configuration of nearby
point light sources yields novel constraints for making the problem
tractable. Specifically, by positioning light sources on a plane thatis perpendicular to the optical axis of the camera and symmetrically around an arbitrary central point on the plane, as shown•

L. Makabe, H. Guo, H. Santo, F. Okura, and Y. Matsushita arewith Graduate School of Information Science and Technology, Osaka
University, Japan.
E-mail:{lilika.makabe,heng.guo,
santo.hiroaki,
okura,
yasumat}@ist.osaka-u.ac.jp

Target surface
cameralight

Symmetric light pairs
Light arrangement examples…
Fig. 1: Examples of light source arrangement for our method. At
least three pairs of symmetric point light sources are arranged tothe image plane with a shared center. Our near-light photometricstereo achieves global minima in shape recovery using the proposed symmetric near-light configuration.
in Fig. 1, we derive a set of linear constraints for determining
surface normals and scene point locations based on the differencesof the symmetric image measurements. More specifically, weshow that three pairs of symmetric point lights, with at least one
pair having a different radius than others (example configurations
shown in Fig. 1 bottom), yield the constraints that are useful for
uniquely reaching the optimal solution. Together with a linear
relaxation of light fall-off effects [12], we develop a linear solution
method for surface normals, scene point locations, and surface
albedos, which is free from the initial guess. In addition, with
our method, the light sources are only required to be partiallycalibrated, i.e., their absolute positions are unneeded, but only theratios of radii of symmetric light source pairs and angles between

TABLE 1: Comparison of photometric stereo methods under Lambertian reflectance. Our method assumes a more practical light model,
requires less light calibration effort, and provides convex and linear optimization to achieve the global optima of surface normal anddepth.
Light model

MethodLight calibration

Light arrangement

Optimization

Distant[1][2][3], [4], [5][6]Calibrated
Partially calibrated
Partially calibrated
Uncalibrated

N.A
Axis-aligned symmetric
Ring
N.A.

ConvexConvexConvexConvexN.A
Constant albedo
Integrability
Integrability, diffuse maxima detection[7], [8], [9], [10]
[11]
[12]
[13]
[14]
Ours

Calibrated
Calibrated
Calibrated
Uncalibrated
Uncalibrated
Partially calibrated

N.A
Ring
N.A.
N.A.
N.A.
Symmetric pairs

Non-convex
Non-convex
Non-convex
Non-convex
Non-convex
Convex/Closed-form

Depth initialization
Depth initialization
Relaxation of light fall-off
Depth initialization, normal integration
Data prior
Relaxation of light fall-off[15]

Uncalibrated

N.A.

Non-convex

Near

Universal

them are assumed to be known while the offset from the camera’s
optical center can be unknown. One of our application scenarios
is taking the devices from place to place for recovering different
target surfaces, which requires frequent assembly and disassemblyof the capture setup. Despite maintaining the relative positionof lights, inevitable shifts in the absolute positions can occur,
requiring careful light calibration. Our method bypasses the recalibration step by using a symmetric light setup, which can be
widely found in capture devices in previous works.
To summarize, this paper contributes to near-light photometricstereo by introducing a linear closed-form solution method that
achieves global minima in surface shape recovery under thelight fall-off relaxation, utilizing symmetric point light pairs.
Furthermore, our method reduces the calibration effort of nearlight position, as it only requires the radii ratios and angles ofthe symmetric light source pairs, rather than precise point light
positions.

2

R ELATED WORK

Our method is related to near-light Lambertian photometric stereo
methods and also photometric stereo methods that exploit specificlight arrangements. We provide a brief summary of related works
in these two categories, as summarized in Table 1. For nonLambertian photometric stereo methods, we recommend referring
to recent surveys and benchmark evaluations, such as DiLiGenT [17] and DiLiGenT102 [18] for distant light configurations,and LUCES [19] for near-light configurations.
2.1

Near-light photometric stereo

Near-light photometric stereo has gained interest since Iwahori etal. [20] due to its practical and accurate modeling of nearby
point lights compared to the conventional distant-light photometricstereo [1]. However, under a near-light condition, different surface
points are illuminated by different light directions and attenuation (i.e., light fall-off) depending on the relative positions between
point lights to the surface points, leading to a non-linear relationship between the image observations and the surface normals. Dueto this non-linearity, the problem of near-light photometric stereo
is inherently non-convex, and there is no closed-form solution.
Under calibrated light settings, variational methods [7], [8],[9], [10] formulate the problem as non-linear PDEs based onimage ratios (ratio of two images under different illuminations)and iteratively update depth and its partial derivatives. Alternative
approaches involving optimization of surface normals and depth

Additional requirements

Data prior

are widely applied in [21], [22], [23], [24], [25] as well as an
uncalibrated near-light photometric stereo method [13]. Surface
normals are first calculated from image observations with depth
initialization, and depth is then recovered through normal integration [26]. Due to the non-convex nature of near-light photometricstereo, these iterative optimization pipelines typically require acareful depth initialization. This depth initialization is also required in the most recent learning-based near-light photometricstereo methods [14], [15], [27], [28] addressing non-Lambertianreflectance.
To avoid depth initialization in non-convex optimization,
Sakaue and Sato [12] introduce light fall-off relaxation and derivea linear solution for near-light photometric stereo. However, their
method treats surface normal and depth as independent variables,
leading to inconsistent normal and depth map estimates. Our
method differs from previous non-convex and light calibrationreliant approaches, providing a linear solution for near-light photometric stereo using partially-calibrated symmetric near lights
and light fall-off relaxation [12]. Our approach ensures that theestimated surface normal and depth achieve global optima underlight fall-off relaxation.
2.2

Photometric stereo with specific light arrangement

Reducing the effort of light calibration is highly desirable inphotometric stereo. While existing uncalibrated photometric stereo
methods, such as Favaro and Papadhimitri [6], do not require
known light directions, they suffer from ambiguity in their estimation. As a result, additional constraints like surface integrability
and detection of local diffuse reflectance maxima [6] are necessary
to resolve the ambiguity. Few studies [14], [15] have explored
uncalibrated photometric stereo under near-light settings. They
adopt a learning-based approach, which eliminates the need forlight source calibration. However, these methods require a large
number of lights to obtain stable estimations.
An alternative approach to minimizing geometric light calibration effort is the utilization of specific light arrangements, whichis referred to as “partially-calibrated” in Table 1.
Under distant light setting, Zhou and Tan [3], adopt ring lightto disambiguate the GBR ambiguity [29] in the normal estimates,
where the light directions lie on a camera view-centered cone.
Chandraker et al. [4] further extends the ring light setting to differential light and achieves surface normal estimation for surfaces
with isotropic reflectances. Minami et al. [2] proposes to use a
simplified version of the ring light, where four directional lights

Light-to-surface distance �∗Target surface
�Light-to-surface
distance �∗�∗ ∝ �∗ − � 22
�∗

3D position �
3D position �
Estimated �∗
as radius�4�∗Fig. 2: Overview of the proposed method. We consider thedifferences and summations of observations under the symmetriclight pairs and construct a linear optimization system to estimate
light-to-surface (scaled) distance e∗ . From the scaled distance e∗ ,we compute the surface position (up to scale if radii are unknown)
and normal.

are fixed in vertical and horizontal symmetry (i.e., axis-aligned),
which is a special case of the symmetric light arrangement. The
observations obtained under symmetric lights provide azimuth
angles of surface normals without assuming a given radius forthe lights. To uniquely determine the full surface normal, theirapproach requires at least two scene points with constant albedo.
Few methods explore light arrangement in near-light photometric stereo. Liu et al. [11] utilize a ring point light configuration
to obtain a reasonable depth initialization for their non-linear
photometric stereo optimization; however, light calibration forthese ring point lights remains necessary. Our approach introduces a symmetric near-light arrangement, offering more flexible
configurations, as shown in Fig. 1, compared to the distant andaxis-aligned symmetric light arrangement [2]. By employing the
proposed symmetric light pairs, we not only eliminate the need forgeometric light calibration but also enable the resolution of nearlight photometric stereo using linear and convex optimization.

3

P ROBLEM STATEMENT

We start by presenting the image formation model under a nearlight setting. We then outline our approach for estimating the scene
shape and albedos using the proposed symmetric near point lightpairs.
Assuming Lambertian reflectance and the equal light radiancefor all the lights, the intensity measurement m ∈ R+ of a surface
point x ∈ R3 illuminated by a point light source can be described
as:

m=ρ

1
2
∥s − x∥2T(s − x) n,∥s − x∥2T

As shown in Fig. 2, instead of directly solving for all theunknowns, our strategy is to first determine the scaled distancee by exploiting the constraints derived by our symmetric light
configuration. Once the scaled distance e is obtained in a per-pixel
manner, we can then solve for surface normal n and surface point
x, and albedo ρ up to scale. In the following, we first introduce thelinear constraint derived from the symmetric light arrangement,
then discuss the surface normal and depth estimation.

4.1

Symmetric-light constraints

As illustrated in Fig. 3, we have a pair of origin symmetric point
lights whose 3D coordinates, sθr+ and sθr− , are:(Tsθr+ = [+r sin θ, +r cos θ, 0] + soTsθr− = [−r sin θ, −r cos θ, 0] + so,

(3)

in which r ∈ R+ is the radius of the symmetric lights,T
so = [sx , sy , sz ] is the offset of the set of symmetric lights.
Hereafter, we call this origin-symmetric light pair parameterized
by the same radius and angle as the symmetric pair.
The observations under the symmetric pair of lights can be
written using Eq. (2) as:(eθr+ mθr+eθr− mθr−

Target surface==

Tsθr+ − x n,Tsθr− − x n

where
surfacenormal
T
n
=
[n
,
n
,
n
]
,
surface
x
y
z
�
T
���−
position x = [x, y, z] ,and scaled distances e∗��are unknown. In a similar×manner to [2], we consider� �
the difference of observations
�
��
���+
under symmetric pair andconstructequationsfor
unknown scaled distancesFig. 3: Light coordinate system. e∗ . First, we consider thedifference of observations
under one pair to obtain the following equation:�(1)where ρ ∈ R+ is the albedo of the surface point, which is scaledby the strength of the light source, s ∈ R3 is the point lightsource position, and n ∈ S2 ⊂ R3 is a surface normal. The
1
factor ∥s−x∥
2 represents the light fall-off effect proportional to2
the squared distance from light source s to surface point x. For thesimplicity of notations, we define d ≜ ∥s − x∥2 and e ≜ ρ−1 d3 .
The Eq. (1) can be rewritten as

em = (s − x) n.

P ROPOSED METHOD(2)

The scalar e can be interpreted as a light-to-surface distance scaledby the albedo and light fall-off, and we hereafter call it a scaled
distance. Given the measurements {m} under a set of point lightsources {s}, our goal is to determine the surface normal n, surface
point x, and albedo ρ at each surface point.

T
T


eθr+ mθr+ − eθr− mθr− = sθr+ − sθr− n = 2 sθr+ − so n.
(4)
When
we
have
at
least
two
�
�
� �
−�
symmetric pair of lights, whoseangles are different from each�other, any other relative vectorss∗∗ − so can be written as a linear
combination of the two bases as:� � −��θsψ
βr+ − so = s sr+ − soFig. 4: Relative vector relation.
+ t sφαr+ − so ,(5)where the ratio of the radii α and β are known in our setting,and coefficients s, t ∈ R can be parameterized by the known

ratio α : β and angles θ, φ, and ψ . By substituting Eq. (5) into
Eq. (4), we can obtain the following equation:ψψψeψ
βr+ mβr+ − eβr− mβr−
Tn
= 2 sψ
βr+ − soTT= 2s sθr+ − so n + 2t sφαr+ − so n
= s eθr+ mθr+ − eθr+ mθr+ + t eφαr+ mφαr+ − eφαr− mφαr− .
(6)When we have npairs pairs of the symmetric light sources, we canobtain npairs − 2 equations using two basis pairs, e.g., (r, θ) and(αr, φ). See the supplementary material for further detail on thenumber of linearly independent constraints.
Further, by considering the summation of the symmetric pairs,
we can obtain the following equations:eθr+ mθr+ + eθr− mθr−T

= 2 (so − x) n
= eφαr+ mφαr+ + eφαr− mφαr− ,(7)which leads to another linear constraints on e∗ . Given npairs pairsof symmetric lights, we can obtain npairs − 1 linear independentequations.
To sum up, we have two kinds of equations from the difference and summation of the symmetric pairs (Eqs. (6) and2nT(7)) for unknown scaled distances e = [e∗ ] ∈ R+ pairs . Let
A ∈ Rneqs ×2npairs be a coefficient matrix formed by putting
together Eqs. (6) and (7), where neqs is the total number of
equations. If we can solve the homogeneous system,

Ae = 0,(8)

for the scaled distance vector e, it can be converted to thesurface position x as will be described in Sec. 4.3. Unfortunately
though, the right null space of the matrix A is three-dimensional,i.e., rank (A) = 2npairs − 3 in the general case. To determine
the scaled distance vector e up to scale, it requires two more
independent constraints so that rank (A) = 2npairs − 1.
4.2

Linear constraints by light fall-off relaxation

To derive additional constraints, we approximate the light fall-off
proportional to the distance, instead of using the squared distance.
Existing studies [12], [13] use the same approximation and showthat its merit in alleviating difficult nonlinearity overtakes theeffect of the approximation error. With the relaxation, the scaled
distance e is approximated as

e = ρ−1 d3 ≈ ρ−1 d2 = ρ−1 ∥s − x∥22 .

(9)

Given this approximated scaled distance, we can obtain four
additional linear constraints on the scaled distance vector e,derived from its differences and sums, similar to the previous
section.
First, Eq. (9) of the symmetric pair can be rewritten using ashifted surface point x′ = x − so as:2eθr+ = ρ−1 sθr+ − x


2





2

θ
−1

sr+ − so − (x − so )
=ρ
2
(10)


 .

−1
2
′T ′
′T
θ


=
ρ
r
+
x
x
−
2x
s
−
s
o

r+







T
T
θ
2
′
′
′
θ
−1
e = ρ
sr− − sor + x x − 2xr−

By summing e∗ , we haveTeθr+ + eθr− = 2ρ−1 r2 + x′ x′ .

(11)

Similarly, by taking a difference we can eliminate the quadratic
terms asT(12)eθr+ − eθr− = −4ρ−1 x′ sθr+ − so .
Additional constraint 1A: When we have at least three symmetricpairs whose radii are different from each other, the following
homogeneous constraints can be obtained from Eq. (11) by eliminating albedo scaled radius:



eψ
+ eψ
− eθr+ + eθr−
βr+
βr−
−1 2
ρ r =
2 (β 2 − 1)


(13)

eφαr+ + eφαr− − eθr+ + eθr−=
2 (α2 − 1)for npairs − 2 combinations.
Additional constraint 1B: As a special case of the above constraint, when we have symmetric pairs whose radii are the sameas each other, we can obtain a constraint from Eq. (11),eθr+ + eθr− = eφr+ + eφr− ,(14)

for npairs − 1 combinations.
Additional constraint 2A: Since Eq. (12) includes the relative
light position sθr+ − so similarly in Eq. (4), we obtain the following equations by similar manner to Eq. (6):




ψ
φ
φ
θ
θ
eψ
−
e
=
s
e
−
e
+
t
e
−
e
(15)
αr+
αr− ,
r+
r−
βr+
βr−
ψ

assuming sβr+ can be represented using the two basis sθr+
φand sαr+ (Eq. (5)). Considering all the possible pairs leads tonpairs − 2 constraints, similar to Eq. (6).

Additional constraint 2B: The above three constraints, Eqs. (13)(15), hold for arbitrary global offset so . In the specific casewhere the lights only have the global offset along z -axis,
i.e., so = [0, 0, sz ], there are additional homogeneous constraints
instead of Eq. (15) as:!
sθr+ − so  φTpeαr+ − eφαr−r!
, (16)
sφαr+ − so  θθTer+ − er−=prwhere sθr+ − so /r = [sin θ, cos θ, 0], and p = [u′ , v ′ , 1] is anormalized camera coordinate, which is parameterized asTu − cx v − cy
(17)
,
,1 ,
p=
fx
fy
where (u, v), (fx , fy ), and (cx , cy ) are the pixel coordinate, focal length, and optical center in a pixel unit,
respectively. This constraint comesfrom Eq. (12) and
TTx′ sθr+ − so = z ′ pT sθr+ − so , where x′ ≜ [x′ , y ′ , z ′ ] .
Considering all the combinations of the symmetric pairs,
npairs − 1 linearly independent constraints can be obtained.

In summary, the equations in Eqs. (13)-(16) are the additional constraints obtained by the relaxation. Putting them in a′matrix A′ ∈ Rneqs ×2npairs , we have a new homogeneous system′
A e = 0, where n′eqs is the number of all the equations from therelaxation. Putting the constraints together, we solve the following
minimization problem by vertically stacking constraint matrices
A and A′ :


2
A
∗
(18)
e = argmin
s.t. eT e = 1.
e
′
A
e
2
The problem is equivalent to finding the right null space of thestacked constraint matrix. We solve the problem by the singularvalue decomposition (SVD) and ensure that the sign of theestimated e becomes positive because they are scaled distances.
While conventional SVD does not guarantee e ≥ 0, we haveempirically observed that the elements of the estimated vector ealways have the same signs.
4.3

Surface normal and position estimation

From scaled distances e estimated in the previous section, we here
estimate the surface position x up to scale and surface normal n. If
we know the radius r of the symmetric lights, the surface position
x can be recovered without the scaling ambiguity.
From Eq. (13), we can parameterize ρ−1 r2 using e, which weknow the exact value from the estimated e. Let us define a scaled
and shifted surface position x′r asx − sx y − sy z − sz Tx′T
, (19)= [x′r , yr′ , zr′ ] =
,
,
x′r ≜
r
r
r
r
Since Eq. (12) can be rewritten using r and θ asTTeθr+ − eθr− = −4ρ−1 r2 x′r [sin θ, cos θ, 0] ,we can derive x′r from Eq. (12) as
eθr+ − eθr− α cos φ − eφαr+ − eφαr− cos θ.
x′r =−4ρ−1 r2 α sin (θ − φ)Similarly, y ′ can be obtained aseθr+ − eθr− α sin φ − eφαr+ − eφαr− sin θ.
yr′ =−4ρ−1 r2 α sin (φ − θ)

(20)

(21)

(22)

Substituting x′ , y ′ , and ρ−1 r2 into Eq. (10) yieldsv!
u θ ′ Tu er+sθr+ − so
x
′
t
′2
′2
zr =
,
− (xr + yr + 1) + 2
2ρ−1 r2rr(23)
assuming the camera is looking toward the +z direction. Note thatthe z element of the relative light position sθr+ − so is always 0,
and thus the third term of Eq. (23) can be parameterized by x′r , yr′
and the known angle θ. The same can be computed for the othercombination of lights. In our method, we take the mean estimate
from using all the equations from all the light pairs, and as a result,
we have the scaled and shifted surface point x′r .
From the relaxed image formation model introduced in theprevious section, we haveTm∗ ≈ ρ

(s − x) n2∥s − x∥2= ρr−1T(s′r − x′r ) n

2 ,∥s′r − x′r ∥2

(24)

φ

T

where s′r = (s − so )/r, e.g., sαr+ = α [sin φ, cos φ, 0] , which
are the known relative positions of symmetric lights. Since only
unknowns are the scalar ρr−1 and surface normal n, and the norm
2of normal ∥n∥2 should be always 1, once the scaled and shiftedsurface point x′r is obtained, the surface normal estimate can beobtained by closed-form solution as classic calibrated photometricstereo [1].
4.4

Optimality under relaxation

Our two-step approach can be summarized as follows:e∗ = argmin L(e) s.t. eT e = 1



e




2
A
,
e
L(e)
=
′


A

2


e = g(x)(25)

where g(x) : x 7→ e denotes the mapping function that corresponds to Eq. (10). Our approach can be viewed as optimizing
a surface point x that minimizes the objective function L(g(x)).
From Eqs. (21)-(23) in Sec. 4.3, the mapping from e to x, hereafter
denoted as g −1 : e 7→ x, is a one-to-one mapping function. Since
the globally optimal e∗ can be obtained by convex optimizationand g −1 is an one-to-one mapping function, the obtained surface
point x∗ = g −1 (e∗ ) gives the global minimum of the objectiveL(g(x)). Surface normal can also be derived as a closed-form
solution using the optimal x∗ .
4.5

Valid light arrangements

In this section, we analyze the solvability of our problem to present
the practical light arrangements derived from the constraints. We
here consider the general case where all the lights are not on thesame line and discuss the special case later.
Scaled distance estimation: We first present the solvability on scaled distance estimation in all possible combinations.
Throughout this section, we establish the coefficient matrices
A′ 1A , A′ 1B , A′ 2A , and A′ 2B by assembling the four constraints
derived by the light fall-off relaxation (Eqs. (13)-(16)), respectively. Considering the linearly independent constraints, the rankof these matrices is as follows:
rank (A)= 2npairs − 3rank (A′ 1A ) = npairs − 2 (non-ring light)rank (A′ 1B ) = npairs − 1 (ring light)
.

′

rank
(A
)
=
n
−
2
(xyz
-axis
offset
)

2A
pairs


rank (A′ 2B ) = npairs − 1 (z -axis offset)

Here, A′ 1A and A′ 1B are both obtained by the elementary operation on Eq. (11), and thus A′ 1A is a subset of A′ 1B . Similarly,
A′ 2A is a subset of A′ 2B . Overall, we have four possible cases
where (i) non-ring light with the global offset along xyz -axis,(ii) non-ring light with the offset along z -axis, (iii) ring light withthe offset along xyz -axis, and (iv) ring light with the offset alongz -axis, which lead to the corresponding additional constraints of
1A + 2A, 1A + 2B, 1B + 2A, and 1B + 2B, respectively.
Since A, A′1A/1B and A′2A/2B are linearly independentof each other, we can estimate e for all the possible lightarrangements if the sum of the rank of constraints from
A, A′1A/1B and A′2A/2B is at least 2npairs − 1; namely, we canestimate e with npairs ≥ 3 for (i)∼ (iii) cases and npairs ≥ 2for (iv) case. In summary, the minimum setting for scaled distanceValidInvalid× scaled distance estimation✓ depth estimation

�pai

=

Special case✓ scaled distance estimation
× depth estimation✓ scaled distance estimation✓ depth estimation✓ scaled distance estimation(with camera intrinsics)
× depth estimation

�pai

=

(i) At least pairs
have the same radii(ii) All pairs havedifferent radii≥✓ scaled distance estimation✓ depth estimation

All the pair shares
the same radii✓ scaled distance estimation
× depth estimation

�pai

…

All the pair shares
the same radii

At least 1 pair havethe different radii

Fig. 5: Examples of valid/invalid light arrangements.
�pai
.

°

=

�pai

=

�pai

=

.

Fig. 6: Light arrangements used in the synthetic experiments. In
every case, we set the inner and outer radii as 1 and 2.

estimation for non-ring light cases i.e., (i) and (iii) is that wehave at least npairs = 3. Although the scaled distance can be
estimated in the case of a ring light, the depth cannot be estimatedas described in the following section.
Surface position estimation: We then consider the solvability ofsurface point estimation given scaled distances. At the end of thispart, we show that the following condition should be satisfied.
Condition 1. At least two pairs with different radii must exist forthe surface position estimation.
In short, this is due to the fact that the denominator of ρ−1 r2
in Eq. (13) goes to 0 when all radii are equal, so the surface
point obtained from Eqs. (21)-(23), which includes ρ−1 r2 , cannot
be recovered either. The condition can also be explained by its
geometric meaning.
Apollonius shows that the set of points that have a given
ratio of distances to the fixed points lies in the circle [30]. The

aforementioned theorem also holds in Rn case (see Appendix:
Proposition 1.); i.o.w., with the two fixed points ∈ R3 and its
corresponding ratios, we get an Apollonius sphere.
oandNow we have the known relative light positions s∗ −sr
the estimated distance ratios e∗ . Therefore, given two randomly
selected lights, we can obtain an Apollonius sphere, where theunknown surface point x′r lies. Thus, the problem of solving
for a surface point, given known relative light positions andtheir distance to the estimated shifted point, can be cast into the
problem of estimating the intersection of Apollonius spheres using
combinations of two lights chosen from 2npairs . Consequently, apoint x can be recovered if the spheres have a unique intersection.
More details along with the proof of each proposition can be foundin the supplementary material.
To facilitate the discussion, we present the following proposition, the proof of which is given in the appendix.
Proposition 2. When the radii of the symmetric lights r∗ ofall pairs are the same, all the possible Apollonius spheres are
centered on a line, whose direction is [−y ′ , x′ , 0].
Assuming that the estimation of the scaled distances has noerror, then the Apollonius spheres always have some common
intersections. If we let all 3D spheres lie on a line, the intersection
obviously has an ambiguity along the 2D circle, and thus theunique surface point x′ cannot be recovered. That is, if all the lightpairs have a unique radius, e.g., arranged on a ring, our relativelycalibrated near-light photometric stereo problem cannot be solved.
In contrast, we empirically observed symmetric light arrangements
sharing the same center with at least two different radii candisambiguate the problem. The examples of valid/invalid lightarrangements are shown in Fig. 5, which satisfies the condition
for both scaled distance estimation and surface point estimation.
Special case: Finally, we consider the solvability condition when
all lights are on the same line. In this case, we can estimate thesurface points with ≥ 4 lights and the known camera intrinsics,
assuming that the x− and y− components of the light offset so
are all 0. With a similar rank analysis as in the general case, we
can confirm that ≥ 4 lights are needed for the scaled distanceestimation. From the scaled distances we have an inhomogeneous
constraint on x′r and yr′ from Eq. (21). Additionally, when so =T[0, 0, sz ] , the ratio yr′ /x′r can be obtained using a normalized
camera coordinate p as v ′ /u′ . From the above two constraints we
can estimate x′r and yr′ , and zr′ can also be obtained by following
Eq. (23).

5

E XPERIMENTS

In this section, we evaluate our method using both synthetic andreal-world scenes. We first outline our experimental settings andthen present the results of our evaluations.
5.1

Experimental settings

Baselines: For comparison, we evaluate our method with threestate-of-the-art techniques: Calibrated [7], FastNFPS [14], andUniversalPS [15]. Calibrated is a near-light calibrated photometricstereo method that assumes Lambertian surfaces.FastNFPS andUniversalPS are both learning-based approaches. FastNFPS isa near-light uncalibrated photometric stereo method for nonLambertian scenes, and UniversalPS is an uncalibrated method
designed for both near and distant light conditions.

npairs = 3
Calibrated [7] fastNFPS [14] UniversalPS [15]

Ours

GT

Ours

npairs = 4
Calibrated [7] fastNFPS [14] UniversalPS [15]Error map

�

4.974

4.590

16.312

0.026

0.032

0.031

4.907

7.384

15.5140.024
Depth error

0.058

0.037

17.719

4.897

3.896

14.934

0.017

0.031

0.029

4.886

6.196

14.451

0.013

0.055

0.038

�

17.009Error map

22.17620.439Error map

Depth map

C RAB

Normal map

Error map

Depth map

B UNNY

Normal map

GT/Input

Normal error
0

10◦ 0

0.01

Fig. 7: Estimation results of our method and comparison methods, with the global offset along z -axis so = [0.0, 0.0, 0.5] added tothe center of the symmetric light pairs. The left-hand side and right-hand side columns show the estimations from npairs = 3 andnpairs = 4 cases, respectively. We show the normal maps and depth maps along with the corresponding error maps, as well as a mean
angular (for normal)/relative absolute errors (for depth) at the bottom of each error map.
For Calibrated, we use the ground-truth light positions as
input. Since Calibrated requires initial guesses of the depth, we
use the median value of the ground-truth depth for synthetic
experiments and the approximate distance between the camera andthe target object for real-world experiments. Since UniversalPS
only outputs normal maps, we evaluate its performance using theestimated normal maps.

error for surface normal, and a relative absolute error for depth
as evaluation metrics. Relative absolute error is computed as theratio of raw absolute error over the ground-truth depth.

Evaluation procedures: For quantitative evaluation in the synthetic experiment, we align the estimated depth maps from eachmethod by scaling and shifting so that the estimated depth fits
the ground truth. This is necessary because FastNFPS and our
method estimate depth maps with scale and shift ambiguities.
Although Calibrated outputs absolute depth, the results can shiftdepending on the initialization. To ensure a fair comparison, weapply the same alignment to all methods. We use an angularDataset: We render synthetic scenes using two Lambertian objects, B UNNY with spatially-varying albedo and C RAB with uniform albedo. We use a camera whose focal length, sensor size,
and resolution are set to 85 mm, 36 × 24 mm, and 720 × 480,
respectively. In the image rendering process, the physically-correct
light fall-off, i.e., proportional to squared distance, is included,
and global illumination effects, such as cast shadows and interreflections, are not included. We also mask the attached shadow

5.2

Evaluation with synthetic scenes

In this section, we quantitatively evaluate the baseline method andours on a synthetic dataset.

�

Ours

npairs = 3
Calibrated [7] fastNFPS [14] UniversalPS [15]

GT

Ours

npairs = 4
Calibrated [7] fastNFPS [14] UniversalPS [15]Error map

7.416

4.704

18.487

0.077

0.033

0.042

8.555

7.375

17.692

0.088

0.058

0.037

18.089

5.116

4.122

16.641

0.025

0.033

0.042

5.504

5.612

16.257

0.030

0.050

0.038

17.536

Error map

20.70818.873

Error map

Depth map

C RAB

Normal map

Error map

Depth map

B UNNY

Normal map

GT/Input

Normal error
0Depth error
10◦ 0

0.01

Fig. 8: Estimation results of our method and comparison methods, with the global offset along all the xyz -axis so = [0.3, 0.4, 0.5]
added to the center of the symmetric light pairs.
area only in the synthetic experiments. The rendered images are
stored in single-precision.
Figure 6 shows the light arrangements used in our synthetic experiments with varying the number of symmetric light pairs from
2 to 4. For each light arrangement, we evaluate the results witha global offset along the z -axis and along all three axes x, y, z axis with respect to the camera’s optical center. The proposed andthe uncalibrated methods assume unknown offsets, while the othermethods use the ground truth light positions. The object is put 6
distance away from the camera center, where the inner radius of
the symmetric pairs is 1.
Results: Figure 7 shows the estimated results of proposed andcomparison methods for our synthetic scenes with the global
offset along the z -axis. For both scenes with different numbersof symmetric light pairs, the proposed method achieves resultsthat are comparable to those of the Calibrated in terms of bothnormal and depth map estimation. One possible reason for the
relatively large errors observed in FastNFPS and UniversalPSis the small number of light sources. Furthermore, we observethat the Calibrated, which assumes a continuous surface, fails toaccurately recover the depth map for scenes with discontinuities,
such as the limbs of the C RAB. In contrast, the proposed method,which employs per-pixel estimation, achieves better results inhandling these discontinuities.
Figure 8 shows the estimated results with the global offset
along all three axes x, y, z - axis. In the scenes with npairs = 3,
although our method also achieves comparable results, it fails
depth recovery in some areas, e.g., right side of the B UNNY’s
face. This is because the npairs = 3 case is the minimal condition
required to solve our homogeneous system, and the solution
should be sensitive to the approximation error from light falloff relaxation. In contrast, in the scenes with npairs = 4, theproposed method achieves almost similar accuracy compared toGT/Input

Ours

Calibrated [7]fastNFPS [14]the axis-aligned symmetric light method, which assumes distantlights, due to its explicit consideration of the near-light effects.

Depth map

B UNNY

Error map

0.01

0

0.038

0.055

0.050

Error map

Depth map

C RAB

0.01

0

0.031

0.059

0.037

Fig. 9: Depth estimation results with npairs = 2 case. We add theglobal offset along the z -axis as so = [0.0, 0.0, 0.5]. We show amean relative absolute error at the bottom of each error map.
the Calibrated although the proposed method assumes an unknown
global offset.
Figure 9 shows the estimated results for npairs = 2, where
the proposed method can only estimate depth maps. The proposed
method outperforms Calibrated even though it uses the given light
positions and initial depth, due to the difficulty in solving the nonconvex optimization problem with a small number of lights.
5.3

Effect of varying light-to-surface distance

The light fall-off relaxation may fail when the light-to-surface
distance is too short. We evaluate the effect of varying light-tosurface distances using synthetic scenes. In this experiment, we
use npairs = 4 lights as shown in Fig. 6, and we change thepositions of light sources along the z -axis. Figure 10 shows theestimated results. While the shorter cases have a larger error, we
can obtain comparable results when we use approximately l ≥ 4,
where l is the distance from the light sources to the target object.
This result suggests that, for example, if we use the radius of
the inner symmetric light pair s ≈ 12.5 cm, we should put thetarget object more than 50 cm far away from the lights, which iscomparable to the conclusion in the supplementary material. The
error by the relaxation is attributed to the difference in the input
images, i.o.w., the difference in the light fall-off terms. To assess
the further analysis of relaxation-accuracy relations, we evaluate
the relative error of the light fall-off terms in the supplementarymaterial. Please refer to it for more details.
5.4

Comparison with axis-aligned method

We further evaluate the estimation results of our method and theaxis-aligned symmetric light method [2]. The light arrangement
used in this experiment and the estimated normal maps by bothmethods are shown in Fig. 11. The proposed method outperforms

5.5 Noise/Shadow effectTo evaluate the method in a more realistic setting, we render theimages with settings as close as possible to real-world experiments, with quantization of 12 bit unsigned integer same as thereal-world setting, global illumination effects enabled including
cast shadows and inter-reflections, and a shot and readout noise
following existing work [31] as synthetic noises. The parameters
for shot noise and the standard deviation for readout noise are
fitted using real-world images captured by the same camera usedin real-world experiments. Figure 14 shows the estimated resultswith noisy images captured under npairs = 4 symmetric lights.
Our surface normal estimates outperform the uncalibrated PS
methods, in most of the area. The depth map from fastNFPS has aflattened shape, while ours can estimate a reasonable shape except
for the boundary area.
5.6 Experiment with real-world scenes
In this section, we show evaluations using our real-world dataset.
To capture the dataset, we use the device shown in Figure 12,
which consists of a CCD camera1 equipped with 25 mm lensand eight LEDs2 . The LEDs are mounted on a board precisely
manufactured by a CNC machine, ensuring the accurate placementof each LED. We employ a constant current circuit in our device
for all light sources to emit uniform radiance. For further detail,please refer to the supplementary material. Since the camera is
fixed to the board via rigs, it is difficult to align the optical centerof the symmetric light pairs with the camera. Therefore, in thisexperiment, the proposed method assumes the existence of an
unknown global offset and solves for it using four symmetric lightpairs. The camera’s intrinsics are calibrated using OpenCV3 forcomparison methods. For the target objects, we use nine objects
with an almost diffuse surface.
Results: Figure 13 shows the estimated normal maps by thecomparison and proposed methods. Overall, the proposed method
achieves almost comparable results to Calibrated, even though theproposed method does not use the exact light positions. Both thelearning-based results in over-smoothed results for most of theobjects. Both the Calibrated and our method output reasonableshape estimations for almost pure Lambertian scenes, such as C AT
and S HEEP, and, for the other scenes, both the Calibrated and ours
are affected by the specular observations due to the deviation fromthe Lambertian assumption. For example, in S ANTA’s lower body,
we can see that both methods failed to recover the correct shape.
Other than C AT, the surfaces contain spatially-varying albedo,
and the proposed method can naturally handle it because of our
per-pixel formulation. Comparing the proposed and comparison
methods, we can see that the proposed method is affected by castshadows more than the calibrated method, for example, aroundthe R ABBIT’s ear part. This is because the proposed method
computes the differences of symmetric pair observations, which issensitive to the pair that one observation is shadowed. The resultsof the proposed method are slightly noisy because of the per-pixel
estimation. This can be improved by considering the smoothness
across the neighboring pixels.
1. BFS-U3-28S5C-C, Teledyne FLIR LLC
2. XLamp CXA1304, Cree, Inc.
3. OpenCV 4.5, https://opencv.org, Retrieved Apr. 14, 2023.

Input image

Median of light-to-surface distance l←−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−
l=6
l=5
l=4
l=3
l=2

Calibrated [7]Ours

Calibrated [7]Ours

Calibrated [7]Ours

Calibrated [7]Ours

Calibrated [7]Normal map

Ours

Error map

10◦

0
4.814

3.900

5.021

3.981

5.374

3.859

6.013

3.908

7.429

4.443

Fig. 10: Estimation results with varying light-to-surface distances. We show the estimated surface normal with angular error maps,
where the distance from the light to the object is set to 2, 3, 4, 5, and 6 under the setting of r = 1.
Light arrangement

GT/Input

Ours

Axis-aligned [2]Symmetric light pairs.

.

Camera

°

6.264

8.860

60 cm

°

Fig. 11: (Left) Axis-aligned symmetric light arrangement used inthis experiment. (Right) Estimated normal maps from the proposed
method (Ours) and axis-aligned symmetric light method [2].

6

C ONCLUSION

We have proposed a near-light photometric stereo method using
origin symmetric lights. Unlike the previous studies dealing with
non-convex optimization problems, we show the symmetric arrangements along with the light fall-off relaxation casts the nearlight photometric stereo problem to a per-pixel linear estimation,
which results in a closed-form solution for surface normal anddepth recovery that is globally optimal under light fall-off relaxation. The experiments show that our method enables comparable
results to the state-of-the-art calibrated near-light photometricstereo method, without explicit calibration or initial depth input.

ACKNOWLEDGMENTS

TargetFig. 12: Our capturing setup. We put the object approximately
60 cm away from the camera. Inner and outer radii are 12.5 cm
and 27.5 cm, respectively. We use the images taken under each of
the lights shown in the figure.
[2]
[3]
[4]

[5]

This work was supported by JSPS KAKENHI Grant Number
JP23H05491.

[6]

R EFERENCES

[7]

[1]

R. J. Woodham, “Photometric method for determining surface orientationfrom multiple images,” Optical Engineering, vol. 19, no. 1, pp. 139–144,
1980.

[8]

K. Minami, H. Santo, F. Okura, and Y. Matsushita, “Symmetric-light
photometric stereo,” in Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2022, pp. 317–325.
Z. Zhou and P. Tan, “Ring-light photometric stereo,” in Proceedings ofthe European Conference on Computer Vision (ECCV), 2010, pp. 265–
279.
M. Chandraker, J. Bai, and R. Ramamoorthi, “On differential photometricreconstruction for unknown, isotropic brdfs,” IEEE Transactions on
Pattern Analysis and Machine Intelligence (PAMI), vol. 35, no. 12, pp.
2941–2955, 2013.
R. Shiradkar, P. Tan, and S. H. Ong, “Auto-calibrating photometric stereo
using ring light constraints,” Machine Vision and Applications, vol. 25,
no. 3, p. 801–809, Apr. 2014.
P. Favaro and T. Papadhimitri, “A closed-form solution to uncalibrated
photometric stereo via diffuse maxima,” in Proceedings of the Conference
on Computer Vision and Pattern Recognition (CVPR), 2012, pp. 821–
828.
Y. Quéau, B. Durix, T. Wu, D. Cremers, F. Lauze, and J.-D. Durou, “Ledbased photometric stereo: Modeling, calibration and numerical solution,”
Journal of Mathematical Imaging and Vision, vol. 60, 2018.
R. Mecca, A. Wetzler, A. M. Bruckstein, and R. Kimmel, “Near field
photometric stereo with point light sources,” SIAM Journal on Imaging

S HEEP

SV-C ATS AILOR S ANTAB UNNYB UDDHAD ONUT

BALL

UniversalPS [15]FastNFPS [14]Calibrated [7]Ours

Input image

C AT

Fig. 13: Estimated normal maps for our real-world dataset. For each scene, we show one of the input images and estimate normal mapsby the proposed method (Ours) and the comparison methods.
Ours

Surface normal
Calibrated [7]15.532/ 9.706

6.034/ 4.452

fastNFPS [14] UniversalPS [15]

GT

Ours

Depth
Calibrated [7]25.184/ 5.4110.549/ 0.470

fastNFPS [14] UniversalPS [15]Error map

Estimated

GT/Input

22.095/ 23.127

14.240/ 12.084

INVALID

Depth error

Normal error
0

3.306/ 4.282

10

◦

0

0.01

Fig. 14: Estimation results of our method and comparison methods on our synthetic scenes, which are rendered under the realistic
image setting described in Sec. 5.5. We show a mean angular (for normal)/relative absolute errors (for depth) at the bottom of eacherror map. The two values separated by “/” indicate the errors without and with masking the pixels where our depth estimation errors
are larger than 0.01.
Sciences, vol. 7, no. 4, pp. 2732–2770, 2014.
[9]

R. Mecca, E. Rodolà, and D. Cremers, “Realistic photometric stereo
using partial differential irradiance equation ratios,” Computers & Graphics, vol. 51, pp. 8–16, 2015.

[10] R. Mecca and Y. Quéau, “Unifying diffuse and specular reflections for thephotometric stereo problem,” in Proceedings of the Winter Conference on
Applications of Computer Vision (WACV), 2016.
[11] C. Liu, S. G. Narasimhan, and A. W. Dubrawski, “Near-light photometricstereo using circularly placed point light sources,” in Proceedings ofIEEE International Conference on Computational Photography (ICCP),

2018.
[12] F. Sakaue and J. Sato, “A new approach of photometric stereo fromlinear image representation under close lighting,” in Proceedings of theInternational Conference on Computer Vision Workshop (ICCVW), 2011,
pp. 759–766.
[13] T. Papadhimitri and P. Favaro, “Uncalibrated near-light photometricstereo,” in Proceedings of the British Machine Vision Conference(BMVC), 2014.
[14] D. Lichy, S. Sengupta, and D. W. Jacobs, “Fast light-weight near-field
photometric stereo,” in Proceedings of the Conference on Computer
Vision and Pattern Recognition (CVPR), 2022.
[15] S. Ikehata, “Universal photometric stereo network using global lighting
contexts,” in Proceedings of the Conference on Computer Vision and
Pattern Recognition (CVPR), 2022, pp. 12 591–12 600.
[16] F. Logothetis, I. Budvytis, R. Mecca, and R. Cipolla, “A CNN based
approach for the near-field photometric stereo problem,” in Proceedingsof the British Machine Vision Conference (BMVC), 2020.
[17] B. Shi, Z. Mo, Z. Wu, D. Duan, S. Yeung, and P. Tan, “A benchmark
dataset and evaluation for non-lambertian and uncalibrated photometricstereo,” IEEE Transactions on Pattern Analysis and Machine Intelligence(PAMI), vol. 41, no. 2, pp. 271–284, 2019.
[18] J. Ren, F. Wang, J. Zhang, Q. Zheng, M. Ren, and B. Shi, “Diligent102: A
photometric stereo benchmark dataset with controlled shape and material
variation,” in Proceedings of the Conference on Computer Vision and
Pattern Recognition (CVPR), 2022, pp. 12 581–12 590.
[19] R. Mecca, F. Logothetis, I. Budvytis, and R. Cipolla, “LUCES: A dataset
for near-field point light source photometric stereo,” in Proceedings ofthe British Machine Vision Conference (BMVC), 2021.
[20] Y. Iwahori, H. Sugie, and N. Ishii, “Reconstructing shape from shading
images under point light source illumination,” in Proceedings of theInternational Conference on Pattern Recognition (ICPR), 1990.
[21] J. Ahmad, J. Sun, L. Smith, and M. Smith, “An improved photometricstereo through distance estimation and light vector optimization fromdiffused maxima region,” Pattern Recognition Letters, vol. 50, pp. 15–
22, 2014.
[22] A. Bony, B. Bringier, and M. Khoudeir, “Tridimensional reconstruction
by photometric stereo with near spot light sources,” in European Signal
Processing Conference, 2013.
[23] T. Collins and A. Bartoli, “3D reconstruction in laparoscopy with closerange photometric stereo,” in International Conference on Medical Image
Computing and Computer-Assisted Intervention, 2012.
[24] X. Huang, M. Walton, G. Bearman, and O. Cossairt, “Near light correction for image relighting and 3D shape recovery,” in Digital Heritage,
2015.
[25] Y. Nie and Z. Song, “A novel photometric stereo method with nonisotropic point light sources,” in Proceedings of the International Conference on Pattern Recognition (ICPR), 2016.
[26] Y. Quéau, J.-D. Durou, and J.-F. Aujol, “Normal integration: a survey,”
Journal of Mathematical Imaging and Vision, vol. 60, no. 4, pp. 576–593,
2018.
[27] F. Logothetis, R. Mecca, I. Budvytis, and R. Cipolla, “A cnn based
approach for the point-light photometric stereo problem,” International
Journal of Computer Vision, pp. 1–20, 2022.
[28] H. Santo, M. Waechter, and Y. Matsushita, “Deep near-light photometricstereo for spatially varying reflectances,” in Proceedings of the European
Conference on Computer Vision (ECCV), 2020.
[29] M. K. Chandraker, F. Kahl, and D. J. Kriegman, “Reflections on thegeneralized bas-relief ambiguity,” in Proceedings of the Conference on
Computer Vision and Pattern Recognition (CVPR), vol. 1, 2005, pp. 788–795.
[30] C. S. Ogilvy, Excursions in Geometry. Courier Corporation, 1990.
[31] Y. Cao, M. Liu, S. Liu, X. Wang, L. Lei, and W. Zuo, “Physics-guided
iso-dependent sensor noise modeling for extreme low-light photography,”in Proceedings of the IEEE/CVF Conference on Computer Vision and
Pattern Recognition, 2023, pp. 5744–5753.

Lilika Makabe received her B.S. and M.S. degrees in computer science from Osaka University, Japan, in 2020 and 2022, respectively,
where she is currently working toward a Ph.D
degree. Her research interests include computational photography and physics-based vision.

Heng Guo received his B.E. and M.S. degrees
in signal and information processing from University of Electronic Science and Technology ofChina, and Ph.D. degree from Osaka University, in 2015, 2018, and 2022. He is currently
a specially-appointed assistant professor at Osaka University. His research interests include
physics-based vision and machine learning.

Hiroaki Santo received his M.S. and Ph.D. degrees in information science from Osaka University, Japan, in 2018 and 2021, respectively. He
is currently an assistant professor with the Department of Multimedia Engineering, Graduate
School of Information Science and Technology,
Osaka University. His research interests include
computer vision and machine learning.

Fumio Okura received the M.S. and Ph.D. degrees in engineering from the Nara Institute of
Science and Technology, in 2011 and 2014,
respectively. He has been an assistant professor with the Institute of Scientific and Industrial
Research, Osaka University, until 2020. He is
now an associate professor with the Graduate
School of Information Science and Technology,
Osaka University. His research interest includes
the boundary domain between computer visionand computer graphics.

Yasuyuki Matsushita received his B.S., M.S.
and Ph.D. degrees in EECS from the Universityof Tokyo in 1998, 2000, and 2003, respectively.
From April 2003 to March 2015, he was with
Visual Computing group at Microsoft Research
Asia. In April 2015, he joined Osaka University
as a professor. His research area includes computer vision, machine learning and optimization.
He is/was an Editor-in-Chief for International
Journal of Computer Vision and on the editorial
board of IEEE Transactions on Pattern Analysis
and Machine Intelligence (TPAMI), The Visual Computer journal, IPSJ
Transactions on Computer Vision Applications (CVA), and Encyclopediaof Computer Vision. He served/is serving as a Program Co-Chair of
PSIVT 2010, 3DIMPVT 2011, ACCV 2012, ICCV 2017, and a General
Co-Chair for ACCV 2014 and ICCV 2021. He is a senior member ofIEEE.