3D Models of Outdoor Scenes
3D Modeling of Outdoor Environments byIntegrating Omnidirectional Range and Color Images

Application fields
 Site simulation
 Mixed reality
 Virtual walk-throughToshihiro ASAI

These models are mademanually with high cost

Virtual walk-throughVision and Media Computing Lab.
2005/3/23

Automatic 3D modeling for outdoor scenes
has been widely investigated.

12

Objective

Method of 3D Modeling

3D modeling of wide area outdoor scenes

 Estimating 3D shape from multiple images○ Easy to acquire data
× Difficult to apply for wide area with high accuracy

ApproachOmnidirectional sensors are used to acquire range
and color images efficiently.
efficiently.
 RTKRTK-GPS and gyroscope are used to estimate
position and orientation of the sensor system.
 Measuring environments by a laser

rangefinder○ Accurate range data
× Necessary to register multiple range data

RTK-GPS

Omnidirectional
Multi-camera System(OMS)

Using a rangefinder is suitable for accurate
modeling of outdoor scenes.
3

Omnidirectionallaser rangefindergyroscopeSensor system on a vehicle

4

Data Acquisition

Procedure of Modeling

Range and Color Image

1. Data Acquisition
・ Data are acquired at multiple points in outdoor scenes.
2. Registration of multiple range data
・ Acquired data by RTK-GPS and gyroscope are used
as initial value.
・ Sensor position and orientation of range data are
optimized by using improved ICP algorithm.
3. Texture-mapping of color images
・ The highest-resolution suitable texture is selected.

Omnidirectional rangefinder(Riegle, LMS-Z360)

OMS(Point Grey Research, Ladybug)

omnidirectional range image

omnidirectional color image

Maximum measurable range
approx. 200m

Six camera units in a camera block
More than 75% of full spherical view

Measurement accuracy
within 12mm561

Data Acquisition

Position and Orientation

Sensor system

Position

Orientation

RTK-GPS(Nikon-Trimble, LogPakII)

gyroscope(TOKIMEC, TISS-5-40)

Accuracy : approx. 3cm

Accuracy : approx. ±2°

 Geometrical relationships among these sensor
coordinate systems are fixed.
 These coordinate systems are aligned to world
coordinate system.
OMS
Antenna of RTKRTK-GPS
Measurement point RTK-GPS and gyroscope are interlocked with each other.

Alt. Rangefinder Two coordinate systems are aligned automatically.
 Accumulative error of gyroscope is corrected byLon.

measured value of RTK-GPS.
 Origin of the gyroscope coordinate system andmeasurement point of the RTK-GPS are considered thesame point.

gyroscope
Lat.

7

Registration of multiple range data

World coordinate system(GPS coordinate system)

8

Procedure of registering range datastart Position and orientation acquired by sensors arePlane detection from each range data

used as initial values for registration.

Transformation of range data to global coordinate system Whole data are optimized simultaneously by using

Search of the plane which corresponds
to each point of the plane part

ICP algorithm.
 Many plane parts exist in outdoor environments.

Estimation of transformation matrix fromcorresponding points and planes

 The rangefinder can measure environment

omnidirectionally.

noPlane based registration by overlapping
the plane parts of different range data

Is solution converged?
yes

9

Procedure of registering range data
start

10

end

Search of corresponding plane
The plane correspond to a point is searched fromdifferent range data.

Plane detection from each range data
Transformation of range data to global coordinate system

range data
Search of the plane which corresponds
to each point of the plane part

plane parts of range datanormal vector
correspondence

Estimation of transformation matrix fromcorresponding points and planesnoz A plane which corresponds to the point is searched

toward a normal vector direction of the plane.

Is solution converged?

z Correspondences of a point and a plane are calculated

yes
end

11

among whole data.

122

Procedure of registering range datastart

1. Maximization of sum of Inner product of thenormal vectors. (positions are fixed)

Plane detection from each range data

2. Minimization of sum of distance between
corresponding a point and a plane. (orientationsare fixed)

Transformation of range data to global coordinate system
Search of the plane which corresponds
to each point of the plane part

Sensor position of range data

Estimation of transformation matrix fromcorresponding points and planesnoEstimation of transformation matrix

1.

Is solution converge?

2.

yes
13

end

Texture-mapping of color images

14

Experiments
 Environment：our campus

 Texture from the image which gives the highest

 Data acquisition ：68 points (about 50m interval)

resolution.
resolution
 Occlusions are detected from generated 3D shape.

 Required time : about 5 hours◆acquisition position

Triangular patch
Millennium
hallcorrect

incorrect
Camera 2
Camera 1

camera2
camera1
Considering
only resolution

Information
scienceabout250m
Biologicalsciencecamera2
camera1
Consideringresolution and occlusion
15

Result of registration

Materials
scienceabout 300m

16

Result of 3D modeling Acquired data Resolution of range image :1024 x 512
 Search process of corresponding plane Use of a cluster consisting of 24 PCs (CPU: Pentium41.7Ghz, Memory: 1024MB)

 Optimization process Use of single PC (CPU: Pentium4 1.8Ghz, Memory: 1024MB)

Ppm0001.m1v

Required time for registration is about 7 days.
17

Generated model of our campus

183

2D CAD data overlaid ongenerated 3D model

Summary

Generated model has no large distortion.

3D modeling of outdoor scenes Range and color images are acquired efficiently

by using two omnidirectional sensors.
sensors
 Position and orientation are acquired by using
RTKRTK-GPS and gyroscope.
gyroscope− Using as initial value for registration.

Future work Reduction of holes (non-measured portions) of thegenerated model.
19

Continuous scanning

Approach Continuous measurement during the movement

input・ Position and orientation of rangefinder during movement.
(acquired by RTK-GPS + INS sensor)+・ Range data acquired during movement.

of rangefinder.
 The non-measured portions which exist sparsely in environmentby stop-and-go scanning are measured efficiently.

fittingintegrating・ Registered model from stop-and-go scanning data.

+Range data acquired bya rangefinder during movement.

20

Registered model fromstop-and-go scanning data.

outputOptimized position and orientation of rangefinder
during movement.
21

224