2011 IEEE/RSJ International Conference onIntelligent Robots and Systems
September 25-30, 2011. San Francisco, CA, USA

Denoising of Range Images using a Trilateral Filter
and Belief Propagation
Shuji Oishi, Ryo Kurazume, Yumi Iwashita, and Tsutomu Hasegawa
Abstract— Two denoising techniques using reflectivity fornoisy range images are proposed: range image smoothing by trilateral filter and range image inpainting by belief propagation.
The trilateral filter makes use of reflectivity as well as spatial
and intensity information so that geometric features, such as
jump and roof edges, are preserved while smoothing. The
range image inpainting technique based on belief propagation
recovers a deteriorated range image using not only the adjacent
range values but also the continuity of the reflectance image. We
conduct simulations and experiments using synthesized images
and actual range images taken by a laser scanner and verify
that the proposed techniques suppress noise while preserving
jump and roof edges and repair deteriorated range images.

I. INTRODUCTION
In recent years, high-precision three-dimensional (3D)
laser scanners, such as RIEGL VZ-400 (RIEGL GmbH),
Leica Scan Station 2 (Leica Geosystems AG), and TOPCON GLS-1500 (TOPCON), have been widely used forlandscape surveying or digital 3D modeling. In addition, a
low-cost, high-resolution laser measurement systems using
two-dimensional laser scanners (SICK LMS151 (SICK AG)
and HOKUYO TOP-URG (HOKUYO)) and a rotary table
have been proposed for 3D environmental map building formobile robot navigation [1]. These LIDAR (light detection
and ranging) sensors acquire high resolution and precise
range images. However, range images often suffer from noisedue to the reflectance property of objects’ surfaces or electrical and mechanical disturbances. For example, one sigma
accuracy of RIEGL VZ-400 is 3mm per 100 meters, thus
a flat surface is measured as a slightly uneven plate. Metal
surface with strong specular reflection or black color cannot
be measured by standard laser scanners. Therefore, denoisingtechniques for range images taken by laser scanners still
remains as a critical problem.
In the present paper, we propose two denoising techniques
which focus on the reflectivity [2]. When we measure range
data by laser scanners, the reflectivity, which indicates thestrength of the reflected laser can be obtained as a by-productof range data. Note that all of the pixels in the range image
have corresponding reflectance values. In other words, therange image and the reflectance image are precisely andfundamentally aligned.
Using the reflectance image, we firstly propose a newsmoothing technique using the trilateral filter by Choudury etS. Oishi is with Graduate School of Information Science and ElectricalEngineering, Kyushu University, 744 Motooka, Nishi-ku, Fukuoka 8190395, Japan oishi@irvs.is.kyushu-u.ac.jp
R. Kurazume, Y. Iwashita, and T. Hasegawa are with Graduate Facultyof Information Science and Electrical Engineering, Kyushu University, 744
Motooka, Nishi-ku, Fukuoka 819-0395, Japan

978-1-61284-455-8/11/$26.00 ©2011 IEEE

al. [3] and reflectivity as a denoising technique of an image.
In the proposed method, the trilateral filter is applied for not
only the range image but also the corresponding reflectanceimage. By taking account of the properties of range andreflectance images, the proposed method can smooth range
images while preserving geometric features such as jump androof edges.
Next, we propose a new inpainting technique of a range
image using a reflectance image and belief propagation. In
this method, the deteriorated range values in a range image
are recovered using not only the adjacent range values but
also the continuity of the reflectance image.
In Section 2, an overview of the previous approaches will
be presented. In Sections 3 and 4, we will propose two
new denoising techniques for range images using reflectanceimages, that is, range image smoothing by the trilateral filter
and range image inpainting by belief propagation. In Section
5, simulations and experiments using a laser scanner will bereported for the purpose of verifying the performance of the
proposed techniques.
II. R ELATED R ESEARCH
Smoothing techniques for range images are classified into
two categories: pixel-based or point-based techniques [4],[5], [6], [7] and mesh-based techniques [8], [9], [10], [11].
Raw range data acquired by range sensors is composed of agroup of 3D points called a point-cloud. Pixel or point-based
methods denoise the range image or the point-cloud directly
without taking the continuity of pixels into account explicitly.
On the other hand, mesh-based methods are applied tostructured meshes, such as triangular patches, by considering
the continuity of the vertexes in the structured meshes.
For the case in which a high-resolution gray-scale image and a low resolution range image are simultaneously
captured from a range sensor, Diebel et al. [5] proposeda technique for estimating high-resolution range images byconsidering the Markov Random Field in high- and lowresolution range images and adjusting smoothing parameters
according to the gradient of the high-resolution gray-scale
image. Crabb et al. [12] and Chan et al. [6] also proposed
an up-sampling technique using the joint bilateral filter[13]. Bohme et al. [11] proposed the denoising techniquefor a range image using the shape-from-shading technique[14]. They introduced an energy function consisting of thedifference of the observed intensity and its estimation basedon the Lambertian reflectance model and the continuity of
the range image and the intensity. Then, the energy function2020

is minimized by the non-linear conjugate gradient method sothat the noise in the range image is suppressed.
On the other hand, several techniques based on the bilateral filter [15], which was developed as an edge-preserving
filter for gray-scale images, have been proposed [9], [10], [4],[16]. Fleishman et al. [10] proposed a 3D edge preserving
filter by applying the bilateral filter for the distance from apoint to its adjacent points projected on a tangential plane(tangential component) and the distance from the adjacentpoints to the tangential plane (normal component). Joneset al. [9] proposed a similar technique using triangularmeshes instead of tangential planes. However, these smoothing techniques are applied after converting from the pointcloud to the meshes and it is difficult to obtain the normal
vectors stably from meshes that contain a great deal ofnoise. Moreover, in some cases, the construction of structured
meshes from a noisy point-cloud is not a simple and trivial
problem.
Miropolsky [4] proposed the geometric bilateral filter,
which uses the distances from the adjacent points and thedifference of normal directions for each point in the pointcloud. However, a stable solution of normal vectors from anoisy point-cloud has not yet been found.
While these methods can be considered as a simple
extension of the bilateral filter for a gray-scale image to arange image, the technique proposed herein uses a reflectanceimage that corresponds one-to-one to the range image forsmoothing the range image. Note that although we assume
that the range and reflectance images have the same resolution, the proposed method can be applied to images having
various resolutions using the joint bilateral filter [13].
For the case in which there are several holes in range data
due to the occlusion or specular or weak reflection, Kawai etal. [17] proposed a completion technique of the 3D surfaces.
They define an energy function based on the similarity ofshapes, and select the best match which minimizes the energyfunction to fill in the holes of 3D geometry. Becker et al.
[18] proposed a completion method using an additional color
image of the same scene from a different viewpoint. Xu etal. [19] also proposed the technique which estimates missing
geometry by learning association of surface normals to image
patches in calibrated images.
On the other hand, several inpainting techniques basedon belief propagation have been proposed [20], [21]. Pedro
et al. [20] proposed an image completion method whichtakes the continuity of pixels into account by applying belief
propagation. Komodakis et al. [21] proposed an exemplarbased inpainting technique using belief propagation. They
introduced the Priority-BP that extends standard belief propagation for priority-based message scheduling and dynamic
label pruning.

As mentioned above, conventional smoothing techniques
for range data are mainly applied for a range image directly.
On the other hand, we focus on a reflectance image thatis acquired as a by-product of the range image for mostrange sensors. By taking the properties of both the range andreflectance images into account, the proposed technique cansuppress noise in a range image while preserving geometricfeatures such as jump and roof edges.
In the following sections, we introduce the conventional
bilateral filter and a reflectance image captured by range
sensors. The trilateral filter using a reflectance image is thendescribed in detail.
A. Reflectance image
Optical range sensors, such as a laser scanner, obtain
range data by measuring the round-trip time of a laser pulse
reflected by an object. Figure 2(a) shows an example of arange image acquired by a 3D laser scanner (Fig. 1 [1]). On
the other hand, most optical range sensors can measure thestrength of the reflected laser pulse (reflectivity). Figure 2(b)shows a reflectance image that depicts reflectance values as agray-scale image. As mentioned above, a unique reflectancevalue is determined for each pixel in the range image. In
other words, the range image and the reflectance image are
precisely and fundamentally aligned.
2D Laser range finder
LMS 200, SICK

Parent robot
Rotating table

Child robot

Child robot

Fig. 1.

Acquisition system of a panoramic range image [1]Roof edge
Jump edge(a) Range image
Roof edge
Jump edge(b) Reflectance image

III. S MOOTHING RANGE IMAGE USING THE TRILATERAL

Fig. 2.

FILTER

In this section, we propose a new technique for smoothinga range image using the trilateral filter [3] and a reflectanceimage.

Range and reflectance images

B. Bilateral filter
The bilateral filter [15] is an edge-preserving smoothingfilter that extends the Gaussian filter so that not only the2021spatial relation but also the variation of the pixel intensity is
considered. In the Gaussian filter, it is assumed that adjacentpixels may have similar intensities, and the weighted sumof neighbor pixels defined by a Gaussian distribution is
calculated for each pixel as follows:j∈S wx (xi , xj )fj(1)
gi =  ij∈Si wx (xi , xj )wx (xi , xj ) = √|x −x |
1
− i j
e 2σx 2
2πσx

2

(2)

where xi is the location of pixel i, gi is a smoothed intensity
value of pixel i, and fj is the original intensity value of the
pixel j, which is a neighbor Si of pixel i. Here, wx (xi , xj )is the weight function determined by a Gaussian function
with a variance of σx2 , and |xi − xj | is the two-dimensional
spatial distance between pixels i and j.
In addition to the Gaussian filter, the bilateral filter takes
the variation of the intensity into consideration for image
smoothing. More precisely, the bilateral filter determines theweight of the neighbor pixels according to not only the twodimensional spatial relation but also the similarity of theintensity, as follows:j∈S wx (xi , xj )wf (fi , fj )fj(3)
gi =  ij∈Si wx (xi , xj )wf (fi , fj )D. Trilateral filter with a reflectance image
The method mentioned above is a straightforward extension of the bilateral filter to a range image. However, as
shown in Fig. 2(a), although abrupt changes of range values,
such as a jump edge, are easily detected in a range image,moderate changes, such as a roof edge, are quite difficult to
detect. Miropolsky [4] introduced the directional variation of
normal vectors in order to emphasize these moderate changes
in the range image. However, if we observe the reflectanceimage shown in Fig.2(b), it is easy to see that these moderatechanges are clearly detected in the reflectance image.
Based on the above consideration, we propose a new filter
that uses reflectance and range images simultaneously forsmoothing a range image, as follows:jgi = 

wx (xi , xj )wf (fi , fj )wd (di , dj )fi
j wx (xi , xj )wf (fi , fj )wd (di , dj )wx (xi , xj )=|x −x |
1
− i j√
e 2σx 22πσxwf (fi , fj )=wd (di , dj )=|d −d |
1
− i j√
e 2σd 2
2πσd

(8)

2

(9)

|f −f |2

− i j2
1
e 2σf
2πσf

(10)
2

(11)

|f −f |2

− i j2
1
wf (fi , fj ) = 
e 2σf
2πσf

(4)

where wf (fi , fj ) is the weight function for the intensity
determined by a Gaussian function with a variance of σf2
and |fi − fj | is the difference in intensity of pixels i and j.
Since the bilateral filter takes the difference in intensity into
account, it is possible to preserve abrupt changes in intensity,
such as roof and jump edges, which are blurred by Gaussian
filters.
C. Extension of the bilateral filter to a range image
Let us apply the bilateral filter for a gray-scale image to arange image. In the same manner as Eqs. (2),(3) and (4), thebilateral filter for a range image can be defined as follows:
j wx (xi , xj )wf (fi , fj )fi(5)gi = 
j wx (xi , xj )wf (fi , fj )wx (xi , xj )=|x −x |
1
− i j√
e 2σx 22πσxwf (fi , fj )=where fi and di are the range and reflectance values in
pixel i, and wx (xi , xj ), wf (fi , fj ), and wd (di , dj ) are
Gaussian functions in the two-dimensional spatial, range,
and reflectance domains with variances of σx2 , σf2 , and σd2 ,respectively.
The filter given by Eq. (8) takes into account three kindsof information in range and reflectance images for smoothinga range image. In other words, it is an extension of thetrilateral filter for images [3] so that it takes the variation of
the reflectivity into consideration for range image smoothing.
Thanks to a variety of properties in range and reflectanceinformation, the proposed trilateral filter enables not only
jump edges but also roof edges to be preserved in a range
image, and the trilateral filter is expected to have higher
performance for edge preservation than the simple expansionof the bilateral filter given by Eq. (5).
Consequently, the proposed smoothing technique for arange image is summarized as follows.

2(6)|f −f |2

− i j2
1
e 2σf
2πσf

(7)

where gi and fi are new and original range values at pixel i,and wx (xi , xj ) and wf (fi , fj ) are Gaussian functions for
two-dimensional spatial and range information with variances of σx2 and σf2 , respectively.
2022

1) Acquire range and reflectance information by a timeof-flight range sensor.
2) Create range and reflectance images in which thevalues of each pixel in range and reflectance images
are proportional to the range and reflectance values.
3) Apply the trilateral filter given by Eq. (8) using range
and reflectance images and obtain a smoothed range
image.
4) Construct a 3D model consisting of meshes from thesmoothed range image.

IV. R ANGE IMAGE INPAINTING BY BELIEF PROPAGATIONIn the previous section, we proposed a range image
smoothing technique using the trilateral filter and a reflectance image. Although this method is effective for range
images that are corrupted by noise, a deteriorated range
image that is missing part of the original image due to
specular reflection or weak reflectivity of the laser pulse
is difficult to repair. For recovering a range image that is
missing part of the original image, this section proposes animage inpainting technique using belief propagation and areflectance image.
A. Loopy belief propagation
Let us consider a graph P consisting of multiple nodes
connected by multiple arcs. We assign label fp to node p sothat the following energy function is minimized.

Dp (fp ) +W (fp , fq )(12)E(f ) =p∈P(p,q)∈N

where Dp (fp ) is a cost term for assigning label fp to nodep, and W (fp , fq ) is a penalty term if labels fp and fq are
assigned to nodes p and q, respectively. Here, N indicatesthe neighbor nodes of node p.
In the framework of belief propagation, the following
messages are repeatedly exchanged between the adjacentnodes in order to determine the optimum label fp thatminimizes the energy function:
mtp→q (fq ) =⎛⎞⎠ (13)
mt−1
min ⎝Dp (fp ) + W (fp , fq ) +s→p (fp )fps∈N (p)\q

After T iterations, optimum label fq∗ is determined so as tominimize the following cost function:
mTp→q (fq )(14)bq (fq ) = Dq (fq ) +p∈N (q)B. Range image inpainting using a reflectance image
We apply belief propagation to a deteriorated range image
and repair the image using a reflectance image. When wemeasure range data using a laser scanner, it often occursthat part of the range image is lost due to saturation of
the reflectivity by specular reflection or a weak laser pulse
reflected on a black surface. In most cases, not only therange information but also the reflectance information inthis region is lost. The proposed inpainting technique forthe range image consists of two steps. First, we repair thereflectance image by belief propagation in Section IV-A,
because the reflectance image clearly contains roof and jumpedges and the restoration of the reflectance image is easierthan the restoration of the range image. Then, we apply belief
propagation to the range image using the repaired reflectanceimage. In Section V, this two-step algorithm is demonstrated
to be able to inpaint the range image more precisely thandirectly applying belief propagation to the range image.

Since belief propagation requires a huge memory and large
calculation cost, the range image is first converted to a 256level gray-scale image. Therefore, the number of labels tobe assigned is 256, as expressed by integers from 0 to 255.
We define the cost term Dp (fp ) for assigning label fp topixel p as
Dp (fp ) = 0(15)
for lost regions andDp (fp ) =| fp − Lp |(16)

for other regions, where Lp is the original label of pixel p.
In addition, we consider the four-neighbor q of pixel p anddefine the cost function for assigning labels fp and fq as
W (fp , fq ) = g(rp , rq )(fp − fq )2(17)

where rp and rq are the intensity values of pixels p and q
in the reflectance image, and g(rp , rq ) is a gain term thatindicates the effect of the reflectance image.
g(rp , rq ) = αe−β(rp −rq )

2

(18)

Equation (17) indicates that the neighboring pixel, which
has a similar reflectance value is preferentially selected
to repair a lost pixel in the deteriorated range image. In
contrast, a pixel having a reflectance value that is changed
discontinuously affects the repair of the range image only
slightly.
V. E XPERIMENTThis section introduces the results of the preliminary experiments for image smoothing using the trilateral filter andimage inpainting by belief propagation using simulated andactual range images. We conducted experiments with various
parameters selected manually, and determined parameters
used for the following experiments.
A. Range image smoothing by the trilateral filter
1) Simulation using a synthesized image: First, we performed the simulation experiments using the synthesizedimage shown in Fig. 3, which is a scene of a square box
having sides of 1 meter in a room. A gray-scale image (Fig.
3(a)) is used instead of a reflectance image, and we added arandom noise of 1 % of the range value to the range image.

(a) Grayscale image

Fig. 3.

(b) Range image

Synthesized images used in the simulation experiment

Figure 4 shows the results obtained using the Gaussian
filter, the bilateral filter, and the trilateral filter, respectively.
Table I shows the RMS errors of the range images after
applying these filters. In the experiment, the kernel size ofeach filter is 9×9 pixels, and the ranges of the range data and2023

(a) 1% noise in depth values(c) Bilateral filter
Fig. 4.
filterRange image

Range image

Reflectance image

Reflectance image(a) Scene 1(b) Scene 2(b) Gaussian filter(d) Trilateral filter
Fig. 6.

Denoised images by Gaussian filter, Bilateral filter and Trilateral

the reflectance data are 13,293 to 17,128[mm] and 0 to 255,
respectively. The variances are set as σx = 4.0, σf = 0.4,
and σd = 6.
As shown in Table I, the RMS error of the proposed
trilateral filter is the smallest, and the proposed trilateralfilter is verified to have high performance for range image
smoothing and edge preservation.

Range and reflectance images

Cross section(a) Original range image(b) Gaussian filter

TABLE IRMS ERROR

Original image
Gaussian filter
Bilateral filter
Trilateral filter (proposed)

RMS [mm]
45.8
17.8
14.1
11.7

(c) Bilateral filter
Fig. 7.

2) Experiments with LIDAR: Next, we performed theexperiments using the 3D laser measurement robot CPS-V
shown in Fig. 1 [1]. The robot enables the surrounding range
data to be captured by rotating the laser scanner (SICK,LMS200) by means of a rotary table. The image size is
200 × 721 pixels.
Figure 5 shows the two experimental conditions: a simple
environment consisting mainly of roof edges (scene 1) anda more complex environment in which a human and otherobjects exist (scene 2) and a number of jump edges areobserved. Figure 6 shows the range and reflectance imagesof these scenes captured by the measurement robot.

(a) Scene 1

Fig. 5.

(b) Scene 2

Experimental setup(d) Trilateral filter

Experimental results for a simple environment

Figure 7 shows the results for scene 1 (Fig. 5(a)). In theexperiment, we set the kernel size of the filters to be 9 × 9
pixels, and the ranges of the range data and the reflectancedata are 275 to 8,191 [mm] and 0 to 255, respectively. The
variances are σx = 0.8, σf = 0.1, and σd = 7 for thenormalized range image.
Figure 7(a) is a 3-D model constructed from the original
range image before applying smoothing filters. Several unexpected bumps appear on the surfaces of the walls and objects
due to the noise in the range image. Figures 7(b), 7(c), and7(d) show the images smoothed by the Gaussian filter, thebilateral filter, and the trilateral filter, respectively. Thesefigures show that the surfaces of the walls are smoothed by
the Gaussian filter and the bilateral filter. However, the edgesof the box and the window frame are blurred. On the other
hand, the trilateral filter can smooth the surfaces of the walls
while preserving the edges of the box and the window frame.
Next, the results for scene 2 (Fig. 5(b)) are shown in Fig.
9. Similar to the experimental results for the scene 1, theGaussian filter and the bilateral filter smooth the surfacesof the walls. In particular, the bilateral filter preserves the

2024

0.7
0.6
Trilateral

Height [m]

0.5
0.4

Gaussian & Bilateral
0.3
Original shape
0.2

Original

Gaussian filter(a) Original range image(b) Gaussian filter(c) Bilateral filter(d) Trilateral filter

Bilateral filter
0.1

Trilateral filter

0
2.6

2.7

2.8

2.9

3

Depth [m]Fig. 8.

Comparison of cross-section shape of the box

Fig. 10.

(a) Original range image(c) Bilateral filter
Fig. 9.

(b) Gaussian filter(d) Trilateral filter

Experimental results for a complex environment

jump edges, such as the shape of the monitor, which isindicated by the arrow. However, the roof edges of walls, theface of the person, and the ruck of the clothes are blurred.
On the other hand, the trilateral filter smoothes the range
image successfully while preserving the jump and roof edges
appropriately, as shown in Fig. 9(d).

Next, we performed the experiments using actual range
and reflectance images taken by the laser scanner on the
CPS-V robots (Fig.1). In this experiment, we use α = 0.75
and β = 0.5. Figure 12 shows the 3-D models restored by
two techniques, that is, the simple belief propagation forrange images and the proposed two-step algorithm usingreflectivity. Missing parts are recovered appropriately byboth techniques and there is not big difference in terms of
the image quality. To emphasise the difference of the twotechniques, we prepared deteriorated range and reflectanceimages manually by cutting a part of a wall, and applied thesimple belief propagation and the proposed technique. Figure
13 shows the restored wall after applying these techniques,
and each RMS error is shown in Table III.
From these results, the range image inpainting is successfully carried out using the two-step algorithm with belief
propagation and the reflectance image.
TABLE IIRMS ERROR FOR RANGE IMAGE INPAINTING

iteration
1230
50

B. Range image inpainting by belief propagation
We performed the simulation for range image inpainting
by belief propagation, as described in Section IV. In theexperiment, we prepared deteriorated reflectance and range
images, which have a small missing region. The size of theimage is 320 × 240 pixels, and the size of the missing partis 20 × 20 pixels. In this experiment, we use α = 0.75 andβ = 1.0.
Figures 11(a) and 11(b) show the original and deteriorated
range images, and Figs. 11(c) and 11(d) show the deteriorated and repaired reflectance images. The inpainted range
images after applying belief propagation 30 times are shownin Figs. 11(e) and 11(f). These images are repaired with andwithout the reflectance image, respectively. The RMS errors
for these repaired images are compared in Table II.

Partial enlarged views

RMS[mm]Without reflectanceWith reflectance
36.24
29.01
28.99
10.68
29.44
10.64

TABLE IIIRMS ERROR IN EXPERIMENTAL RESULTS FOR THE PERFORMANCEEVALUATION

Without reflectanceWith reflectance (2step)

RMS [mm]
7.68
1.44

VI. C ONCLUSION
In the present paper, we proposed two denoising techniques of range images using reflectivity, namely, range2025

(a) Original range image (b)Deteriorated range image(a) Original 3D mesh model with a missing region(c) Deteriorated reflectance (d) Inpainted reflectance
image
image

(b) The 3D mesh model after applying belief propagation(e) Inpainted range image (f) Inpainted range image
without reflectance image using reflectance image(g) Standard BP
Fig. 11.

(h) Proposed method(c) The 3D mesh model after applying our two-step algorithm

Range image inpainting by Belief Propagation

Fig. 12. Experimental results for the range and reflectance images takenby the laser scanner

image smoothing by the trilateral filter and range image
inpainting by belief propagation. By taking into account theproperties of range and reflectance images, the proposed
trilateral filter can suppress noises in range images while
preserving geometric features such as jump and roof edges.
Belief-propagation-based range image inpainting was also
proposed to recover deteriorated range images using not
only the adjacent range values but also the continuity of
the reflectance image. We conducted experiments using a
synthesized image and actual range images and verified
that the proposed denoising techniques successfully suppress
noises and repair deteriorated range images.
Since the reflectance image is obtained as a by-productof range data, the proposed method has several advantages.
For example, no additional measurements or instruments
are required, and, unlike conventional camera images, thereflectance image is not affected by lighting conditions.
In the future, we will discuss the optimum parameters forthe proposed technique and perform quantitative evaluation
for a variety of scenes.
2026

R EFERENCES[1] R. Kurazume, Y. Noda, Y. Tobata, K. Lingemann, Y. Iwashita, andT. Hasegawa, “Laser-based geometric modeling using cooperative
multiple mobile robots,” in Proc. IEEE International Conference on
Robotics and Automation, 2009, pp. 3200–3205.
[2] R. Kurazume, K. Noshino, Z. Zhang, and K. Ikeuchi, “Simultaneous
2D images and 3D geometric model registration for texture mapping
utilizing reflectance attribute,” in Proc. of Fifth Asian Conference on
Computer Vision (ACCV), 2002, pp. 99–106.
[3] P. Choudhury and J. Tumblin, “The trilateral filter for high contrast
images and meshes,” in Eurographics Symposium on Rendering, 2003,
pp. 1–11.
[4] A. Miropolsky and A. Fischer, “Reconstruction with 3D geometricbilateral filter,” in SM ’04: Proceedings of the ninth ACM symposium
on Solid modeling and applications.
Aire-la-Ville, Switzerland,
Switzerland: Eurographics Association, 2004, pp. 225–229.
[5] J. Diebel and S. Thrun, “An application of Markov random fields torange sensing,” in Proceedings of Conference on Neural Information
Processing Systems (NIPS). Cambridge, MA: MIT Press, 2005.
[6] D. Chan, H. Buisman, C. Theobalt, and S. Thrun, “A noise-aware filter
for real-time depth upsamplingw,” in Workshop on Multi-camera andMulti-modal Sensor Fusion Algorithms and Applications, 2008.
[7] M. Lindner, M. Lambers, and A. Kolb, “Sub-pixel data fusion andedge-enhanced distance refinement for 2D/3D images,” International

energy based on similarity of shape,” in Proc. IEEE Int. Conf. on Image
Processing (ICIP2008), 2008, pp. 1532–1535.
[18] J. Becker, C. Stewart, and R. J.Radke, “Lidar inpainting from a single
image,” in In Proceedings of the IEEE International Workshop on 3-D
Digital Imaging and Modeling, 2009.
[19] S.Xu, A.Georghiades, H.Rushmeier, J.Dorsey, and L.McMillan, “Image guided geometry inference,” in Proc. 3rd Int. Symp. on 3DPVT,
2006, pp. 310–317.
[20] P. F. Felzenszwalb and D. P. Huttenlocher, “Efficient belief propagationfor early vision,” International Journal of Computer Vision, vol. 70,
no. 1, 2006.
[21] N. Komodakis and G. Tziritas, “Image completion using global
optimization,” in Proc. IEEE Conf. on Computer Vision and Pattern
Recognition, 2006, pp. 442–452.

(a) Original 3D mesh model with a missing region(b) The 3D mesh model after applying belief propagation(c) The 3D mesh model after applying our two-step algorithm
Fig. 13.

Experimental results for the performance evaluation

Journal on Intelligent Systems Technology and Application, vol. 5, no.
3/4, pp. 344–354, 2008.
[8] M. Desbrun, M. Meyer, P. Schro”der, and A. H. Barr, “Implicit fairingof irregular meshes using diffusion and curvature flow,” in SIGGRAPH’99, 1999, pp. 317–324.
[9] T. R. Jones, F. Durand, and M. Desbrun, “Non-iterative, featurepreserving mesh smoothing,” in SIGGRAPH ’03: ACM SIGGRAPH
2003 Papers. New York, NY, USA: ACM, 2003, pp. 943–949.
[10] S. Fleishman, I. Drori, and D. Cohen-Or, “Bilateral mesh denoising,”
ACM Transactions on Graphics, vol. 22, no. 3, pp. 950–953, 2003.
[11] M. Bohme, M. Haker, T. Martinetz, and E. Barth, “Shading constraint
improves accuracy of time-of-flight measurements,” in Computer Vision and Pattern Recognition Workshops, 2008. CVPRW ’08. IEEE
Computer Society Conference on, 23-28 2008, pp. 1 –6.
[12] R. Crabb, C. Tracey, A. Puranik, and J. Davis, “Real-time foregroundsegmentation via range and color imaging,” in In Proc. of CVPR
Workshop on Time-of-flight Computer Vision, 2008, pp. 1–5.
[13] J. Kopf, M. Cohen, D. Lischinski, and M. Uyttendaele, “Joint bilateral upsampling,” ACM Transactions on Graphics (Proceedings of
SIGGRAPH 2007), vol. 26, no. 3, p. 96, 2007.
[14] J.-D. Durou, M. Falcone, and M. Sagona, “Numerical methods forshape-from-shading: A new survey with benchmarks,” Comput. Vis.
Image Underst., vol. 109, no. 1, pp. 22–43, 2008.
[15] C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color
images,” in ICCV ’98: Proceedings of the Sixth International Conference on Computer Vision. Washington, DC, USA: IEEE Computer
Society, 1998, pp. 839–846.
[16] S. Yoshizawa, A. Belyaev, and H. peter Seidel, “Smoothing by
example: Mesh denoising by averaging with similarity based weights,”
in In Proceedings of the IEEE International Conference on Shape
Modeling and Applications (2006). IEEE, 2006, pp. 38–44.
[17] N. Kawai, T. Sato, and N. Yokoya, “Surface completion by minimizing

2027