33

Object recognition by a laser scannerusing multimodal information
Shuji Oishi, Naoaki Kondo

Yumi Iwashita, Ryo Kurazume

Graduate School of Information Scienceand Electrical Engineering
Kyushu University, Fukuoka, Japan

Graduate Faculty of Information Scienceand Electrical Engineering
Kyushu University, Fukuoka, Japan

Abstract—This paper presents a new object recognition technique using multimodal information by a laser scanner. A ToF
laser scanner captures a three dimensional (3D) shape of a target
object precisely by measuring the round-trip time of a laserpulse. In addition, a laser scanner also collects the strength of thereﬂected laser (reﬂectivity) as side-product of range measurement.
Focusing on the fact that the reﬂectivity contains rich appearance
information of the target object, the proposed technique utilizes
not only range but also reﬂectance information to improve therecognition performance. To evaluate the validity of the proposedtechnique, we applied it for the classiﬁcation of moving objects
on a road as a trafﬁc census application.

I.

I NTRODUCTION

Laser scanners have been widely used for a variety ofapplications such as 3D digital modeling, obstacle detection formobile robots, and people tracking. A laser scanner measuresa range data from the sensor to a target sequentially by around-trip time of a laser pulse, and captures the 3D shapeof the target object. Many object recognition/classiﬁcation
techniques based on range information have been proposedso far [1][2][3][4].
However, it is quite difﬁcult to distinguish objects belonging to different classes but with similar 3D shapes such asa remote control and a cell phone using range data only. To
improve the recognition performance, appearance information
taken by a camera is quite useful. Mohottala et al.[5] developeda recognition system that utilizes a laser scanner and a camera.
This system offered excellent results by acquiring textured 3D
information of an object. However, since range and texture
images are obtained by individual sensors, a calibration process
between these sensors is necessary.

Fig. 1.

Trafﬁc census with a laser scannermultimodal information. In section 3, experimental results fora trafﬁc census application are shown, and the effectiveness ofthe proposed technique is veriﬁed.
II.

P ROPOSED METHOD

On the other hand, laser reﬂectance, which is the power oflaser pulse reﬂected on a surface of a target object, is available
for most laser scanners. A reﬂectance image that depicts
reﬂectance values as a gray-scale image contains appearance
information of the target, and range and reﬂectance images
are fundamentally aligned since both images are obtained bya laser scanner simultaneously. Focusing on the reﬂectivity,
we propose an object recognition technique by a laser scannerusing multimodal information: structure from range image andappearance from reﬂectance image. We apply the proposed
technique to a trafﬁc census application, and show that proposed method improve the recognition accuracy.

In this section, we describe the procedure of the proposed
method, especially about the reﬂectance image, the segmentation technique by background subtraction based on a range
data, and the feature extraction for multimodal information.
Note that we assume that range and reﬂectance images ofmoving objects are obtained from a 2D slit-type laser scannerplaced on a roadside (Figure 1). In this system, the 2D slit laser
beam is scanned along the vertical direction and line imagesof the range and reﬂectance of an object moving in front of
the laser scanner are acquired sequentially.

The rest of this paper is organized as follows. In section
2, we propose a new object recognition technique utilizing

ToF optical range sensors acquire range data by measuring
a round-trip time of a laser pulse reﬂected on an object surface.

䚷 䚷 䚷 䚷 䚷 䚷 © QCAV2013

A. Reﬂectance image

(a) Range image(a) Variances of range values in each region (k = 2)(b) Reﬂectance image
Fig. 2.

(b) Height of a foreground object

Range and reﬂectance images
Fig. 4.

(a) Range image
Fig. 3.

Features of a range image(b) Reﬂectance image

Extracted range and reﬂectance images

Fig. 5. Histogram of oriented gradients (HOG) feature for a reﬂectance image

Figure 2(a) shows an example of a range image acquired bya laser scanner (Fig.1). On the other hand, most of opticalrange sensors record the strength of the reﬂected laser pulse(reﬂectivity).
Reﬂectance value indicates an intensity on the surfaces
points of the targets under a single-frequency light source.
Based on the dichromatic reﬂection model, the reﬂectancevalue consists of diffuse and specular reﬂection. Assuming thespecular component can be ignored and Lambertian reﬂection
is used as a model for diffuse reﬂection, a reﬂectance value isexpressed as follows[6]:on a roadside and moving targets can be extracted easily
as foreground regions by background subtraction techniquefor range images. In addition, the appearance information isavailable by referring the same region in a corresponding
reﬂectance image (Fig.3). Figure 7 shows example range andreﬂectance images of moving objects.
C. Feature extraction

Iq
I = kd 2 cosα(1)rwhere kd is the diffuse reﬂection coefﬁcient, Iq is the power ofthe light source, r is the distance of the light source toward thetarget, and α is the incident angle against the surface normal.
Figure 2(b) shows an example of a reﬂectance image that
depicts reﬂectance values as a gray-scale image. As mentionedabove, the reﬂectance value is determined uniquely for eachpixel in the range image. In other word, the range image andthe reﬂectance image are fundamentally aligned.

We extract two kinds of features from segmented range andreﬂectance images respectively. In order to describe the structure of a target object, we ﬁrst divide the range image into k×k
areas, and then extract a variance of range values for each area.
A height of the target object in the range image is also added
to the feature vector. Consequently, we extract k 2 + 1 featuresfrom a range image as shown in Figure 4. For appearance
information, we calculate Histogram of Oriented Gradients(HOG) features[7] from the reﬂectance image. HOG featuresdescribe local object appearances robustly by extracting thehistogram of gradient orientation and the intensity (Fig.5).

B. Segmentation by background subtraction of range data

D. Proposed method

To extract features of a target object, we ﬁrst cut out
an object region from range and reﬂectance images. To doso, we utilize a range image since the laser scanner is ﬁxed

Consequently, the proposed recognition technique is summarized as follows. We use Support Vector Machine (SVM)
to identify classes of moving objects.

(a) Range images(b) Reﬂectance images
Fig. 7.

Categories of moving objects

TABLE II.

Images
Detected
Correct
Precision
RecallBus1
2424
2292%92%

Range and Reﬂectance images (Proposed)
Bus2
Car
Motorbike
Track
Human
1372
43
1650
1372
43
1650
1172
43
1650
85%
100%100%100%100%85%100%100%100%100%

TABLE I.

2)
3)
4)

Bus2
13171059%
77%

Range image
Car
Motorbike
7243
69
4068
37
99%
93%94%
86%

Track16191684%
100%

Human
50504590%
90%

270 [ °]
50 [Hz]0.5 [ °]
0.5 - 50 [m]± 30 [mm]± 12 [mm]Infrared (905 [nm])Acquire continuous range and reﬂectance image
through a laser scanner.
Segment a region of a object by background subtraction based on a range image.
Extract structural features from the range image andappearance features from the reﬂectance image.
Recognize a target object using SVM.
III.

Bus1
24231878%
75%

F EATURES OF LMS151

Field of view
Frequency
Resolution
Operating range
Systematic error
Statistical errorLight source1)

R ECOGNITION RESULTS

E XPERIMENTS

In this section, we introduce the experimental results of theproposed technique for the trafﬁc census application. FigureFig. 6.

Experimental setup

6 shows experimental setup. We use a laser scanner (SICK,LMS151) that has a ﬁeld of view of 270 °and a scanning frequency of 50 Hz with a resolution of 0.5 °(Table I). Sequential
range and reﬂectance image are obtained simultaneously. In theexperiments, range and reﬂectance images are normalized from0 to 255. The HOG feature consists of 7200 dimensions, andthe parameter is set as k = 4. In this experiment, moving objects

ACKNOWLEDGMENT
This work was supported in part by JSPS KAKENHI Grant
Number 246404 and Grant-in-Aid for Scientiﬁc Research(24656173).

Recall100%
98%

R EFERENCES

96%
94%

[1] A. Johnson and M. Hebert, “Using spin images for efﬁcient object
recognition in cluttered 3d scenes,” in IEEE TRANSACTIONS ON
PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 1999, vol. 21.
[2] R. Triebel, K. Kersting, and W. Burgard, “Robust 3d scan pointclassiﬁcation using associative markov networks,” in IEEE International
Conference on Robotics and Automation, 2006.
[3] B. Steder, R. B. Rusu, K. Konolige, and W. Burgard, “Point featureextraction on 3d range scans taking into account object boundaries,” inIEEE International Conference on Robotics and Automation, 2011.
[4] Volker Steinhage Jens Behley and Armin B. Cremers, “Performanceof histogram descriptors for the classiﬁcation of 3d laser range data inurban environments,” in IEEE International Conference on Robotics andAutomation, 2012.
[5] Masataka Kagesawa Shirmila Mohottala, Shintaro Ono and Katsushi
Ikeuchi, “Fusion of a camera and a laser range sensor for vehicle
recognition,” in Computer Vision and Pattern Recognition Workshops,
2009.
[6] M.Bohme, M.Haker, T.Martinetz, and E.Barth, “Shading constraint
improves accuracy of time-of-ﬂight measurements,” Computer Vision
and Image Understanding, vol. 114, pp. 1329–1335, 2010.
[7] Navneet Dalal and Bill Triggs, “Histograms of oriented gradients forhuman detection,” in Computer Vision and Pattern Recognition, 2005,
vol. Vol.1.

92%90%88%

Fig. 8.

050100
HOG dimensions

150200Recognition results after applying PCA to HOG features

TABLE III.

R ECOGNITION RESULTS USING 50 DIMENSIONS OF HOGFEATURES

Images
Detected
Correct
Precision
Recall

Bus1
24272281%92%

Range and Reﬂectance images (Proposed)
Bus2CarMotorbike
Track
Human
1372
43
1650
1270
431650
1070
43
1650
83%
100%100%100%100%
77%
97%
100%
100%
100%

Total
218
211
97%

consist of 6 classes, that is, Bus1, Bus2, Car, Motorbike, Track,and Human. We consider Bus1 and Bus2 which have slightly
different shapes as different classes since these two buses
belong to different companies.
Table II shows the classiﬁcation results of the moving
objects with and without the reﬂectance image, respectively.
As shown in this table, the recognition accuracy is improvedby using multimodal information. Especially, the classiﬁcation
accuracy for Bus1 and Bus2 is enhanced using the laser
reﬂectivity. Next, we applied Principal Component Analysis(PCA) for HOG features to reduce the dimension. The results
using PCA are shown in Figure 8 and Table III. From theseresults, we veriﬁed that the proposed method works wellalthough the dimension of HOG features is reduced.

IV.

C ONCLUSION

This paper proposed a new object recognition methodby a laser scanner utilizing multimodal information. Using areﬂectance image which is obtained as a side-product of range
scanning, the proposed technique improves the recognition
accuracy even if targets have similar 3D shapes. We conducted
experiments for the trafﬁc census application, and veriﬁedthat the proposed technique enable to classify six kinds ofmoving objects with higher accuracy than the case using range
information only. In the future, we will discuss about theoptimum parameters for the proposed technique, and apply
the proposed technique for a variety of moving objects.

