BRDF Reconstruction from Real Object using Reconstructed
Geometry of Multi-view Images
Taishi Ono

Hiroyuki Kubo

Nara Institute of Science and Technology

Nara Institute of Science and Technology

Takuya Funatomi

Yasuhiro Mukaigawa

Nara Institute of Science and Technology

Nara Institute of Science and Technology

Figure 1: Rendering of the reconstructed BRDFs using our method.

ABSTRACTKEYWORDS

In this paper, we present an effective approach to reconstruct thebidirectional reflectance distribution function (BRDF) from a real
object without any special devices and complex setups. Multi-view
images are commonly used to reconstruct the three-dimensional(3D) shape of an object, and this is known as photogrammetry. For
the purpose of photo-realistic rendering of an object, our approachis practical because it is highly integrated into the photogrammetry pipeline. Our method requires to acquire multi-view images
for 3D reconstruction and a few additional images to obtain thereflectance distribution. We use the reconstructed geometry frommulti-view images to determine the region of optimal light and view
directions according to the existing analytic methods. However, thereconstructed geometry and surface normal tend to contain artifacts. Therefore, we introduced assumptions that are the main
contribution of our study to extract the reliable regions from thereconstructed geometry. The results demonstrate that our method
effectively acquires a plausible BRDF.

BRDF reconstruction, multi-view images

CCS CONCEPTS• Computing methodologies → Appearance and texture representations;
Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
SA ’17 Technical Briefs, November 27–30, 2017, Bangkok, Thailand© 2017 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-5406-6/17/11.
https://doi.org/10.1145/3145749.3149426

ACM Reference format:
Taishi Ono, Hiroyuki Kubo, Takuya Funatomi, and Yasuhiro Mukaigawa.
2017. BRDF Reconstruction from Real Object using Reconstructed Geometryof Multi-view Images. In Proceedings of SIGGRAPH Asia 2017 Technical Briefs,
Bangkok, Thailand, November 27–30, 2017 (SA ’17 Technical Briefs), 4 pages.
https://doi.org/10.1145/3145749.3149426

1

INTRODUCTION

Recently, automatic three-dimensional (3D) shape reconstruction
from multi-view images has become more popular. These reconstructed 3D objects are used in many movies and video games
because of their highly detailed geometry. However, the appearance is not sufficiently plausible because the reflectance distribution
property is not observed from the target object.
In this paper, we present an effective approach to BRDF reconstruction from a real object without any special devices or complex
setups. We use multiple-images that are originally prepared for thephotogrammetry, and a few additional images for BRDF acquisition.
Prior to obtaining the reflectance property, we reconstruct the 3D
shape from multi-view images, however, the reconstructed shapetends to contain geometric artifacts. Therefore, we determine thereliable region which contains less artifact of optimal light and view
directions according to the method of [Nielsen et al. 2015]. Our
main contribution is to introduce a few assumptions to extract thereliable region of the reconstructed geometry. The advantage of ourwork is its practicability because our approach is highly integrated
into the existing photogrammetry pipeline.

SA ’17 Technical Briefs, November 27–30, 2017, Bangkok, Thailand

2

RELATED WORK

There are a lot of researches on reconstructing the geometry of an
object from multiple-images. Chandraker et al. [2013] acquired theobject’s shape from its motion with unknown isotropic BRDF. They
proved that four different motions are sufficient to yield the depth
and surface gradient.
To reconstruct the 3D shape and BRDF from a real object, Müller
et al. [2005] developed a measurement device that used camera
and light arrays. However, BRDF sampling requires millions ofmeasurements. Therefore, these methods are not practical because
they need a great number of expensive devices and complex setups.
Focusing on the sampling costs, Nielsen et al. [2015] significantly
reduced the number of samples to recounstruct BRDF. They extracted the principal components of the set of BRDFs from the MERLBRDF database [Matusik et al. 2003] which contains 100 real BRDFs,
and optimized the sampling directions to minimize the condition
number of the related acquisition matrix. One of the disadvantages
is that the method can only be applied to the known shape geometry
and homogeneous object because the method requires the accurate
surface normal direction of the object.
In this paper, we introduce a method to obtain both the shapeand BRDF from multi-view images of the real object.

3

represents the peak of the specular reflection. In our experiment,
we implement these assumptions as constraints using the following
steps:• CurvatureFirst, we calculate the principal curvatures κu and κv whichcorrespond to principal directions u and v in the image space
proposed by Vergne et al. [2009]. We use H = (κu + κv )/2
as the curvature of each pixel. Finally, we remove unreliable
pixels that had a high curvature according to the threshold.
• Monotonicity
Given the lighting direction l, view direction v, surface norl+v , we assume that BRDF r at
mal n, and half vector h = ∥l+v∥
pixels i and j satisfies the following equation:
ni · hi > nj · hj ⇔ r i > r j

Reliable Region Extraction to Avoid theReconstruction Artifact of the Normal
Direction

To estimate the BRDF accurately, it is necessary to prepare the accurate normal direction of the object. Figure 3(c) shows the normal
map of the reconstructed geometry and Figure 3(d) shows the error
map of the angle between the reconstructed and ground truth normal direction. This shows that several regions of the reconstructednormal direction contain the artifact, unfortunately. If we ignore
these errors of the surface normal, the reconstructed BRDF will
also be significantly inconsistent with the ground truth, as shownin Fig. 3(e).
Thus, we introduce a few assumptions to extract the reliable
region from the reconstructed geometry. We assume that the errorof the surface normal has a strong correlation with the curvatureof the reconstructed object because the reconstructed geometry
using the multi-view images tends to be blunted. This means thatthe normal direction of the high curvature region is not reliable.
We also assume that the reflectance distribution should satisfy themonotonicity with respect to the cosine between the surface normal
and half vector. This assumption is introduced in [Higo et al. 2010],
which describes a photometric stereo that works with a wide rangeof BRDFs. Finally, we also assume that the highest intensity region(1)

We avoid using the unreliable pixels, which does not satisfy
Eq. (1) for BRDF reconstruction.
• Region with the highest intensity
For the region k with the highest intensity, we assume thatthe directions of the half vector h and surface normal n
are completely equal, which is explained by the following
equation.
nk · hk = 1(2)
We determine the pixel with the highest intensity among all
pixels in the distant light images, and then modify its surface
normal to satisfy Eq. (2).

BRDF RECONSTRUCTION

Figure 2 shows the overview of our method. Prior to reconstructing
the BRDF, we obtain multi-view images in ambient light (Fig. 3(a)),
and then, reconstruct the geometry from these images. This process
is the photogrammetry pipeline. We also acquire a few images witha single distant light, as shown in Fig. 3(b). From these images, we
determine regions of optimal light and view directions to reconstruct the BRDF. In the following subsection, we briefly describe
the process for our approach.

3.1

T. Ono et al.

3.2

Choosing Optimal Pixels

After extracting reliable region using these assumptions, we determine the appropriate pixels for the sampling directions according
to the method of [Nielsen et al. 2015]. We determine pixels in thereliable region to minimize the condition number of the acquisition
matrix obtained by [Nielsen et al. 2015] and reconstruct the BRDF
from these pixels.
Even under the proposed constraints, the normal directions ofseveral pixels are inaccurate. Therefore, we randomly extract pixels
from the reliable region and execute BRDF reconstruction several
times. We render images using the reconstructed shape and the reconstructed BRDF from the same viewpoint as the captured images.
We acquire the KL divergences of the intensity distribution between
these images, and define the reconstruction with the minimum KL
divergence as the most plausible result.

4

EXPERIMENTAL RESULTS

We demonstrate the effectiveness of our method in simulated environments and also apply this method to real objects.

4.1

Simulation

We executed four simulations on the MERL BRDF database and thefully measured geometry (Stanford Bunny and Dragon, by Stanford
Computer Graphics Laboratory) as the ground truth.
For this simulated experiment, Figure 3(a) shows one of the rendered multi-view images for 3D reconstruction. In this case werendered 55 images with 3, 000 × 2, 000px resolution rendered bypbrt[Pharr and Humphreys 2010]. These images covered spherically of the target object. Figure 3(b) shows one of the images

BRDF Reconstruction from Multi-view Images

SA ’17 Technical Briefs, November 27–30, 2017, Bangkok, Thailand

multi-view images

reconstructed geometry
light and view direction

distant light images

extracted reliable region

Geometry
Reconstruction

BRDF
Reconstruction
reflectance of optimal pixels
in the reliable region

reconstructed BRDF

Figure 2: Overview of our method. First we acquire multi-view images with ambient light, and also acquire a few images with
only one distant light. Next, we reconstruct the 3D shape and estimate the light and view directions of multi-view images. Then
we analytically determine the reliable region of optimal light and view directions, and obtain the BRDF at the corresponding
pixels. Finally, we acquire a plausible BRDF.
rendered with a virtual single distant light. These images were used
to calculate the BRDF at chosen pixels. In this case, we used 11
images for BRDF reconstruction. Each of these images was rendered from the same viewpoint as one of 55 multi-view images. For
the 3D reconstruction, we used Agisoft Photoscan (Agisoft LLC,
www.agisoft.com). In this case, we set the threshold of the curvature to 0.2, and used six pairs of lights and view directions as the
optimal directions for BRDF reconstruction.
Figure 4 shows the results of reconstructed BRDFs. The graphson the left column show the comparison between the reconstructed
BRDFs and ground truth. The pictures on the right column show therendered images using the reconstructed BRDFs and ground truth.
In comparison with Fig. 3(e), we observe a noticeable improvement
in reconstruction quality, in particular around the specular direction.
Additionally, the appearance of both the rendered images using thereconstructed BRDFs and ground truth BRDFs seems to be quite
similar. Figure 1 shows the rendered image of the reconstructed
BRDFs with detailed geometry and environment lighting (Grace
Cathedral, ©1998, 1999 Paul Debevec [Debevec 1998]). This also
indicates that our method can acquire the plausible appearance.

4.2

(a)

(d)

(b)

(c)

(e)

Figure 3: (a) One of the multi-view images used for 3D reconstruction. (b) One of the images rendered with a distantlight for acquiring the BRDF at optimal pixels. (c) Reconstructed normal map. (d) Relation of the angle[rad] between
reconstructed surface normal and ground truth. (e) Comparison between the reconstructed BRDFs without any constraints(solid) and ground truth (dashed).

Real Environment

We applied our method to real objects. In this case, we reconstructed
the BRDF of a hammer. The experimental setup is shown in Fig. 5.
We used an SLR camera (Nikon D5500) to capture images and
SIGMAKOKI SLA-100A as the distant light. We obtained 73 multiview images under room lighting for 3D reconstruction and eight
images with the distant light. Using a clear coat black sphere, weestimated the direction toward the distant light. In this experiment,
we regarded the hammer as composed of two materials, thus, we
individually reconstructed the BRDF from the head and body of thehammer. Figure 6 shows the rendered images with the reconstructed
BRDF from the head and body of the hammer.
We also reconstructed BRDF of an object located in outdoor, as
shown in Fig. 7. In this case, We assumed the sunlight direction

by the time and place of pictures. We separate the pole into twoareas, the central metallic area and the other parts, and individuallyreconstructed BRDF. Figure 7 shows the rendered images withthe reconstructed BRDF. Our method can work well even in theoutside, and this also represents the practicability of our method
because photogrammetry is often applied to outside objects. In eachexperiment, the reconstructed appearance is plausible.

5

CONCLUSION AND FUTURE WORKIn this paper, we developed a truly practical method of BRDF acquisition, which is highly integrated with the production pipelineof photogrammetry, that required a few distant light images in
addition to multi-view images for 3D reconstruction. The mainSA ’17 Technical Briefs, November 27–30, 2017, Bangkok, Thailand

T. Ono et al.

Figure 6: Rendered images using the reconstructed BRDF
with two environment lightings (©1998, 1999 Paul Debevec).

Figure 7: Left: One of the input multi-view images. Middle
and Right: Rendered images with reconstructed BRDF.

Figure 4: Reconstructed BRDF for four simulations: alumbronze, fruitwood-241, pink-fabric2, and green-latex from
the MERL BRDF database. Left: Comparison between the reconstructed BRDFs (solid) and ground truth (dashed). Incident light is at 135 degrees. Upper right: Rendered spherewith ground truth. Lower right: Rendered sphere with reconstructed BRDFs.

Figure 5: Left: Experimental setup. Right upper: One of themulti-view images for 3D reconstruction. Right middle: Picture with black sphere for obtaining the light direction.
Right lower: One of the distant light images

technical contribution is to introduce some assumptions to extract
more reliable regions and reconstruct a more accurate BRDF, andour results show that these constraints provide plausible outputs.
To reconstruct a more accurate BRDF from a real object, weshould consider the textures of materials. We should also discuss
the number of sampling points and threshold of curvature in moredetail because, currently, we empirically determine these values.

ACKNOWLEDGEMENTS
This work is partly supported by JSPS KAKENHI JP15K16027,
JP26700013 and JP15H05918.

REFERENCES
M. Chandraker, D. Reddy, Y. Wang, and R. Ramamoorthi. 2013. What Object Motion
Reveals about Shape with Unknown BRDF and Lighting. In 2013 IEEE Conference
on Computer Vision and Pattern Recognition. 2523–2530. https://doi.org/10.1109/
CVPR.2013.326
Paul Debevec. 1998. Rendering Synthetic Objects into Real Scenes: Bridging Traditionaland Image-based Graphics with Global Illumination and High Dynamic Range
Photography. In Proceedings of the 25th Annual Conference on Computer Graphics
and Interactive Techniques (SIGGRAPH ’98). ACM, New York, NY, USA, 189–198.
https://doi.org/10.1145/280814.280864
T. Higo, Y. Matsushita, and K. Ikeuchi. 2010. Consensus photometric stereo. In 2010 IEEE
Computer Society Conference on Computer Vision and Pattern Recognition. 1157–1164.
https://doi.org/10.1109/CVPR.2010.5540084
Wojciech Matusik, Hanspeter Pfister, Matthew Brand, and Leonard McMillan. 2003.
Efficient Isotropic BRDF Measurement. In Proceedings of the 14th Eurographics Workshop on Rendering (EGRW ’03). Eurographics Association, Aire-la-Ville, Switzerland,
Switzerland, 241–247. http://dl.acm.org/citation.cfm?id=882404.882439
Gero Müller, Gerhard H. Bendels, and Reinhard Klein. 2005. Rapid Synchronous Acquisition of Geometry and Appearance of Cultural Heritage Artefacts. In Proceedingsof the 6th International Conference on Virtual Reality, Archaeology and Intelligent
Cultural Heritage (VAST’05). Eurographics Association, Aire-la-Ville, Switzerland,
Switzerland, 13–20. https://doi.org/10.2312/VAST/VAST05/013-020
Jannik Boll Nielsen, Henrik Wann Jensen, and Ravi Ramamoorthi. 2015. On Optimal,
Minimal BRDF Sampling for Reflectance Acquisition. ACM Trans. Graph. 34, 6,
Article 186 (Oct. 2015), 11 pages. https://doi.org/10.1145/2816795.2818085
Matt Pharr and Greg Humphreys. 2010. Physically Based Rendering, Second Edition:
From Theory To Implementation (2nd ed.). Morgan Kaufmann Publishers Inc., San
Francisco, CA, USA.
Romain Vergne, Romain Pacanowski, Pascal Barla, Xavier Granier, and Christophe
Schlick. 2009. Light Warping for Enhanced Surface Depiction. ACM Trans. Graph.
28, 3, Article 25 (July 2009), 8 pages. https://doi.org/10.1145/1531326.1531331