UAV
applications
with GNSS
for 3D mappingindex
1. Basics of UAS/UAVs
1. Points to be considered during UAV survey

2. Use of UAVs for 3D mapping
1. Georeferencing Image Data
2. Software
3. Applications

3. Use of Low-cost RTK for UAV

2018.01.24 wed
Yuichi S. Hayakawa / 早川裕弌

about myself

23

1. Type of receivers and antenna
2. How to setup and log data
3. How to process data

about myself

4

Yuichi S. Hayakawa 早川 裕弌

Yuichi S. Hayakawa 早川 裕弌

• Associate Professor atCenter for Spatial Information Science (CSIS),The University of Tokyo(co: Dept. Socio-Cultural Environmental
Studies, Graduate School of Frontier Sicences)• lectureships at Keio SFC, Rikkyo U, Kyoritsu WU,
Ochanomizu WU, Kawamura WU• field: Geomorphology, Geospatial Information
Science, Geoarchaeology• Topics– distribution and processes of WATERFALLS– High-Definition Topography & 3D – measurement
and analysis of landforms and land surface objects– palaeoenvironment around archaeological sites in
West/Central Asia

field science & geospatial analysis

Center for Spatial Information Sciencehigher-resolutionof land surface information
Missions, Aims:• Creating, Developing, and Spreading Spatial Information• Developing Spatial Databases for Research Purposes• Promoting Joint Industry-Government-University Research
http://www.csis.u-tokyo.ac.jp/

getting higher resolution

7

v2D imagery
satellites

aerial photos

v3D topography/land-surface items
radarairphotolaser

and now…

ultra-high resolution

ultra-high resolution

resolution: 0.001–0.01 m

resolution: 0.001–0.01 m

UAS

UAS

unmanned aircraft system

unmanned aircraft science

UAV – unmanned aerial vehicle
multicopter / drone
RPAS – remotely piloted aircraft
system

UAV – unmanned aerial vehicle
multicopter / drone
RPAS – remotely piloted aircraft
systemWhat is UAS/UAV?

What is UAS/UAV?

13- unmanned aerial vehicle- various size and types

Popular UAV

military
RQ-1 Predator
W 14.8 m

Agricultural
RMAX
L 3.6 mAerialphoto
shooting(U.S. Air Force photo)

hobby
Flying Ball
H 11 cm(YAMAHA)

Aerial photo
Phantom 3
W 59 cm(DJI)

Size: ：ca. 20–100 cm
Weight: ca. 500–2000 g
Price: ca. 500–10,000 USD(FLYING BALL)(DJI)

mechanisms of UAVs

Rapid spread forindustry, research,and personal hobby

Relatively cheap, light, easymechanisms of UAV
Reasons for the stability#1: multi-rotor

multirotor

gyro

GNSS

stableflightaccelerometerbarometer

easy
operation(in case of 4 rotors)

Single rotor→inverse rotation ofbody (anti-torque)

4 rotors→paired right and left
rotations dismiss theanti-torquesafetycompass(http://plaza.rakuten.co.jp/mmbirdland/)(http://minizbar.blog100.fc2.com/)mechanisms of UAV
Reasons for the stability#2: built-in GNSS receivermechanisms of UAV
Reasons for the stability#2: built-in GNSS receiver- Stay (hover) at the same position if
without any operation- Automatically correct its positioneven in the wind
- Programmed not to enter thepredefined prohibited areas (airport,
etc.)

Aircraft recognizes its position

Global Navigation Satellite System(GNSS)

Positioning of the
UAV body- Automatically come back “home”
when any problem occurs (lowbattery, radio disconnection, etc.)

Record the takeoff point(https://commons.wikimedia.org/wiki/File:ConstellationGPS.
gif )(http://www.wirelessdictionary.com/wireless_dictionary_GPS_definition.htm(http://crazybrain.jp/SHOP/DJI-Drone-F550-NAZA-V2RTF.html)

mechanisms of UAV
Reasons for the stability#3: built-in barometermechanisms of UAV
Reasons for the stability#4: built-in e-compass

Terrestrial magnetism

Low pressure in higher altitudes

Air pressures areused to identify theaircraft’s relative height

Aircraft identifies its
orientation(http://www.tdk.co.jp/techmag/inductive/200912/)

Altitude can be
maintained during flight

Keep its rightorientation(http://www.tokyocompass.co.jp/mame.htm)

pros and cons of UAVs

mechanisms of UAV
Reasons for the stability#5: built-in accelerometer and gyroadvantages• unmanned•

Aircraft identifies its
inclination anddirection of
movement•••

Keep the right course•

remote area can be surveyed without apersondisadvantages• limited area
• low altitude

simple, long-term, repeated operations•

automatic

high mobility

freedom of operating location

size flexibility

flexible design for various purposes

high resolution••

close-range sensing

development

high potential in industry and market(https://commons.wikimedia.org/wiki/File:Gyroscope_wheel-text.png)

for the appropriate use of UAVs

~150 m above groundslower speed

10–80 km/h

too high resolution

too much datadifficulty in sharing
the space withmanned aircrafts

needs good knowledge and moral of
operators
limited rules after disaster occurrences
no way to contact to manned aircrafts

for the appropriate use of UAVs

24

Alfred Bernhard Nobel

dangerous if
wrongly usedgovernmental
activations of various
UAV applications“There is nothing that is not being exploited in
the world. Advances in science andtechnology’s back-to-back and always
dangerous. It’s the first time can contribute to the
future of mankind and to overcome it.”Soichiro Honda“The right thought of human beings is prior toscience and technology. The evaluation ofhumans having science and technology isundertaken by how they use them and how they
contribute to the society.”applications of UAVs

deliverysecurity

disasterprevention

rescueagriculture

Archaeology

archaeology

entertainment

measurements

UAV specifications
Product name
Aircraft class
Dimensions
WeightMax flight speed
Max ascent/descent speed
Communication distance
Max flight time
Payload25

Operation of UAS

27®

DJI PHANTOM 2
Quadcopter
350 x 350 x 190 mm
1000 g
15 m/s6 m/s1000 m
25 mins.
400 g (empirical value)

Flight height, speed, and resolution

29

28

Camera specifications
Product name
PixelsImage sensor
Dimensions
WeightInterval shooting
Maximum number ofDuration of battery

NIKON COOLPIX A
16.2 million
23.6 x 15.6 mm
111.0 x 64.3 x 40.3 mm
299 g
More than 1 second
More than 6,000 photos (SDXC64GB)
Approx. 70 min.

GNSS unit specification
Product nameTracking channels
Update rate
Geodesics
Accuracy
Dimensions
WeightNIKON GP-1A
18 (SBAS compliant)Once per second
WGS84
Horizontal 10 m RMS
45.5 x 25.5 x 50 mm
24 g

exposureISO®

shutter
speed

aperture30(by Inoue et al., 2014)

Ground resolution(cm/px)

0.5

1.3

2.6

3.9

Updating UAV survey system
Up to now
Phantom2& Coolpix A with GPS unit

LatestPhantom3

31

Phantom2 &CoolpixA
Flight time

15 min.

23 min.

GPS modeGPS

GPS/GLONASS

Communication distance1000 mFPV

×(additional parts)

Auto pilot

×(additional parts)

2000 m○○Real time battery
monitoring

×(additional parts)○

23.6×15.6 mm

6.2×4.7 mm (1/2.3”)

Sensor size
Pixel number
Interval shot

Effective for aerial survey?

Phantom3(Professional)(Coolpix A)

16.2 M (Coolpix A)
1s- (Coolpix A)

12.4 M
5s-

Gimbal×

3-axisPrice

130,000 JPY

175,000 JPY

32

What is photogrammetry?

34

A technology to measure the size, shape,and location of an object based on
photographic imagesa technology to measure size, shapeand location of objects based onstereo-pair camera images• Aerial photogrammetry– Ground surface images using platforms in theair (or satellites)

PHOTOGRAMMETRY• Ground-based photogrammetry– 3D objects on ground from handheld or otherplatforms

Photo shooting for photogrammetry

Nature of photographic images- Large at close range, small at far- Radial inclination from the central point- The higher, the more inclinatedDistortion within a image
based on the object
shape and location

Change camera
location

Left

Reconstruct the surface
morphology from thedistortion

Right(Tsuru and Murai, 2011)(Tsuru and Murai, 2011)

triangulation

37

Angle of two corners and distance between them▶ definition of the triangle
Target point

Triangulation for aerial photogrammetry
baseline
focal length

angle
angle

en
twe
e be

nc

Dista

stereopsis

twoheighters
corn（長谷川ほか，2002）

Stereo photography — 1950s

paralax（長谷川ほか，2002）

http://artwelner.1000words.kodak.com/default.asp?item=220108

3D digital camera FUJIFILM FinePix REAL 3D

registrations

42• camera calibration• internal registration• external registration (georeferencing)Jul 2009(Hasegawa et al., 2002)

data types by photogrammetry

43

44
Structure-from-Motion (SfM)
Multi-View Stereo (MVS) photogrammetry

SfM: reconstruction of camera positions by automatic
detection of tie-points on a stationary target• 3D point cloud• Orthorectified image• DEM（Digital Elevation Model）• 3D mesh model / bird-eye view(texture mapping)

SfM-MVS photogrammetry

45

46
Structure-from-Motion (SfM)
Multi-View Stereo (MVS) photogrammetry

MVS: further photogrammetric process to generate
dense point cloud

SfM: structure from motion• Developed in computer vision(Ullman, 1979; Szeliski, 2010)• Reconstruction 3D structure from 2D images• Sparce point cloud from image matching
MVS: multi-view stereo photogrammetry• Photogrammetry from already aligned 2D
images• Dense point cloud (PMVS/CMVS)

Appropriate photo shooting for SfM-MVS

revolution by UAS
小型無人航空機による革命– much higher, wider & faster –(Agisoft, 2014)

UAS-based SfM-MVS

Characteristics of SfM-MVS

topographic data

50• Various platforms

UAS

– UAV (UAS), balloon, kite
– pole, handheld– undersea etc…

digital camerapoint cloud
aerial photos

• Low cost: <1,000 USD• Multiscale: mm – km
• High resolution: >1M points/faces• Textured images

SfM
3D TIN– Orthorectified image
50

textured model

Software51• SfM and MVS (free)– Bundler• basic SfM reconstruction– PMVS2 and CMVS

• Patch-based Multi-view Stereo• Clustering Views for Multi-view Stereo• dense point cloud reconstruction– other derivatives (Bundler + PMVS/CMVS)• VisualSFM• Bundler Photogrammetry Package• Python Photogrammetry Toolbox• commercial– Agisoft PhotoScan– PIX4D

GNSS for SfM-MVS photogrammetry52• Aircraft positions– Built-in receiver on aircraft– Camera positions can be estimated by SfM
AND GNSS positions• GCP positions– Optional points can be set in the model– Distribution of GCPs affects the model
accuracy• web-based services– Microsoft Photosynth– Autodesk 123D CatchGNSS for SfM-MVS photogrammetry53

GNSS for SfM-MVS photogrammetry54• Aircraft positions• GCP positions– Built-in receiver for
many aircrafts• low in accuracy (1–10 m)• ready to use– separately measured– Wide (unbiased) distribution for the appropriate
model calibration– Optional antenna• potentially highin accuracy (~10 cm)• either realtime or
post-processed(Agisoft, 2014)

GCP distribution and model accuracy

Lower accuracy apart from GCPs

(Obnawa et al. 2014)(Obnawa et al. 2014)

GCP and data quality

casestudies(James et al. 2017)

TLS
UAS
otherstsunamiboulders(NASA Blue Marble)

introduction

62
62• tsunami boulders – key features to reconstructpast tsunamis (e.g., Imamura et al., 2008, JGR;
Goto, 2012, JSSJ)

tsunami boulders insouthern islands ofJapaninsights from UAV and TLS surveys

study sites• accurate volume measurements in the field
have been a challenging issue (e.g., Spiske etal., 2008, EPSL; Watt et al., 2010, USGS-OFR,
Gienko and Terry, 2014, ESPL)• high-definition topographic measurements areapplied to obtain the volume of tsunami
boulders in southwestern Japan

How to measure
the shape of boulders?

?
(Goto et al. 2013 Geology)(Goto et al. 2013 Geology)

methods individual rockwide area

TLS

methods: TLS – terrestrial laser scanning66

UAS

SfM

digital camera

Trimble TX5

aerial photos

SfM

point clouds

terrestrial laser
scanning (TLS)

topographic data

point cloud

cleaning / extraction

3D TIN

textured modela short-range scanner– max. distance: 120 m– max. frequency: 900,000 pts/s– range accuracy: 2 mm @25 m– weight: 5 kg
Post processing by RealWorks &CloudCompare– cloud-based ICP registration

poisson surface reconstruction

3D meshvolume measurement

methods: UAS – unmanned aerial system 67

case studies in southwestern Japan6868

DJI hantom 3 Professional/Advanced• Miyakojima Island (Cape Higashi-Henna)• Shimoji Island• Ishigaki Island• Kuroshima Island

UAS

digital camera

aerial photos

SfM

post processing• SfM-MVS photogrammetryby PhotoScan• positioning accuracy: GCP by
PPK-GNSS (Trimble GeoXH), fix
solution

topographic data

point cloud

3D TIN(GSI Maps)

textured modeltidal waves vs tsunami waves

one of the largest tsunami boulders in
the world: ”Obi-iwa” in Shiomji Is.
tsuna
mi

TLS and volume

71

bould
e

rà

SfM by UAS and ground-based camera72

internal registration

field
measurement
filtering / extraction

V=1658.6 m3

70

UAS: DJI Phantom 3; ground-based camera: RICOH GR IIà entire 3D model generated

3D puzzle

73

UAS (DJI Phantom 3) & TLS (Trimble TX5)

SOUTHERN COAST OF
KURO-SHIMA ISLAND– SOLUTION FOR A WIDER AREA

photographs taken

tsunami boulders targeted

TLS measurement of the boulder #0377

78
3D mesh model around boulder #03 (TLS)• registration error: 6.66 mm• 3D point cloudhttps://skfb.ly/QJC9• https://skfb.ly/QMKC

point cloud

mesh of the boulder #03: flip-side

comparison: TLS and UAS

TLS 72.58 m3

UAS 73.15 m3<1% of difference!
••https://skfb.ly/QJRX
upside down https://skfb.ly/QJSA

volumes for the other boulders

#02

#01

manual (bounding box) vs TLS volumes 82
volume by TLS measurement (m3)

• #01: 24.78 m3• #02: 5.60 m3• #03: 73.15 m3• #04: >15.33 m3• #05: 40.44 m3• #06 (possible): >44.45 m3

81
#05

#06

#04
#03

1,800
1,600
1,400

y = 0.575 x
R² = 0.992

1,200
1,000
800
600
400
200
0

0

500

1,000

1,500

2,000

2,500

3,000

volume by manual measurement (m3)concluding remarks

83
83• successful volume measurements using 3D
point clouds derived from either TLS or UASSfM– ca. x0.6 of manual measurements– TLS is good for accessible individual boulder,
while UAS is also good enough for inaccessible
remote boulders• time consuming: manual filtering• possibility: shape analysis– identification of source location

Repeated measurementsof erosion in a small
coastal island by TLS and
UAS-SfM photogrammetry

volume ofrocky coasterosionstudy site: Suzumejima Island

86(Obanawa et al., 2014)

Northà

coastal proteàction

� cavetform
shore plaedà
being form
87
natural processes of rocky coast erosion
at Suzumejima Island-likeactiveerosion�changes of the island for 50 years
1967

1972

1977

1982

1987

1992

1997

2002

2007

2012

• a rare place: natural processes of coastal erosion– out of the coastal protection– oceanic waves directly attacks the bedrock cliff• rapidly shrinking for decades
1974

2014

35 m

33 m

20 m
32 m a.s.l.

35 m

Obanawa et al. 2015

• constant erosion• shore platform formation• reported erosion rates around thissite: 1 m/y•(1960–1966 by 1:1,000 topo maps;
Horikawa and Sunamura 1967)methods: TLS – terrestrial laser scanners

methods: UAS – unmanned aerial system

TLS #1: TOPCON GLS-1500

DJI Phantom 2 + NIKON COOLPIX A/ Phantom 3 Professional/Advanced

TLS #2: Trimble TX5

post processing• SfM-MVS photogrammetryby PhotoScan• 500-1,000 photos for each time
• positioning accuracy 1: camera-mounted GNSS, >1 m
• positioning accuracy 2: GCP by PPK-GNSS (fix solution)(Trimble GeoXH), 13.4–14.9 mm (14.4 mm RMS)

1 photo / 2 sec
ca. 7 min
205 photos© H. Obanawa

3D modela medium-range scanner– max. distance: 500 m– max. frequency: 30,000 pts/s– range accuracy: 4 mm @150 m– weight: 16 kg (body) + batteries

a short-range scanner– max. distance: 120 m– max. frequency: 900,000 pts/s– range accuracy: 2 mm @25 m– weight: 5 kg

© H. Obanawa

fusion of UAS & TLS

93

fusion of UAS & TLS

94an example of data processing

96• further aligned by ICP– UAS dense cloud à TLS cloud– CloudCompare / Trimble RealWorks– cloud-based registration (ICP), errors: 25.1–39.7mm

an example of data processing

16022395160618C2C distance(detectable distance:
3~10 cm or more)mesh with holes
invert normals forthe “after” dataset

3D mesh polygons

poisson surface
reconstruction

remove noise meshsolid mesh (w/o holes)3D volume calculation

97

an example of data processing

98

calculating volume99

temporal changes in erosion volume
0.3
0.25
0.2
0.15
0.1

160618–161029

160223–160618

period151015–160223

150617–151015

150211–150617

141031–150211

0.05
0volume and rates of erosional changes• by UAS data only
• the volume varies, 10.6 – 178.5 m3 per 4 months

10
0

0.35

140624–141031

erosion rates (m/y)

0.4160618–161029

160223–160618

period151015–160223

150617–151015

150211–150617

141031–150211

200
180
160
140
120
100
80
60
40
20
0

140624–141031(per 4 months)

erosion volume (m3)temporal changes in erosion volume

volume and rates of erosional changes• approx. area of bedrock planes: 1,436 m2• equivalent annual erosion rates: 0.03 – 0.35 m/yremeshing for FEM analysis

101

FEM preliminary results

FEM preliminary results

103

educational/outreach applications104
online 3D viewer

https://skfb.ly/QUwS

102

3D print

3D print examples

2014624

2016618

18

19©地理院地図educational/outreach applications107
cardboard 3D builder

mini workshop at
Kashiwa Open Campus 2017

10
8

UAS-based SfM-MVS photogrammetry

110• acquisition of aerial photos by UAS– 100–300 photos per each flight(typically 15–20 min)– coverage area can be adjusted
by the flight height and distancearchaeologicalapplications• On-site geomorphological mapping– import data
– tracing topographic features using
iPad (GoodNotes app)– realtime matching of the map-derived
features with field observation

Uchu-Kurbu and surroundings (Issyk-Kul)coverage area

coast of Issyk-Kulà– import as spatial data layers– orthorectified images– DEM (digital elevation model) tohillshade, slope, etc.
– extraction of topographic profiles– export as TIFF or PDF images

11
1

on-site dataprocessing

excavation site

Tosor R
iver

• GIS data processing– software: PhotoScan by Agisoft– tie point (sparse cloud)– dense cloud– triangular mesh
– image texture– export orthophoto and DEM

ca. 6 kmspatial data management• SfM-MVS processing

ca. 3 kmapproximate flight coverageUchu-Kurbu and surroundings (Issyk-Kul)cross profiles of the valley
1900

approximate elevation (m)

geomorphological classification (landform mapping)Uchu-Kurbu and surroundings (Issyk-Kul)transverse profile

1850

1800

1750

1700

1650

1600
0

500

1000

1500

2000

2500

longitudinal profile

3 km

3 km

Uchu-Kurbu and surroundings (Issyk-Kul)
geomorphological classification (landform mapping)
T: terrace
F: alluvial fan

ultra-rapid
measurement

for archaeology

400 m

1-hour (!) measurement(Issyk-Kul, Kyrgyz)

orthorectified imageca. 1.5 km

what if w/o
UAS?
(background: © Google)

ca. 1 kmhillshade + 1-m contours

pole camera system if w/o UAS
necessary time on site:
only minutes

orthorectified image
resolution = 1 mm, grid interval = 1 m

topographic contour lines

hillshade image

1 m (red), 10 cm (black) and 1 cm (gray)resolution = 1 cm

test for two tombs

124• 102 photos• processing time: ca. 45 min• w/o GCPs

– but the coordinates of camera positions taken
by Garmin Oregon 300 gives rough estimate ofglobal location of the data

14.0217

TOMBS AT BAT, OMAN

mesh (TIN)

pole-camera for a wider area:
Aigurzhal II/III sites (Naryn)

textured modelSparse point cloud generated by SfM from the photographs
taken in and around the site. Camera positions of thephotographs are shown as blue rectangles.

pole-camera for a wider area:
Aigurzhal II/III sites
573900.000000

574000.000000

574100.000000

574200.000000

574300.000000

Orthorectified image shown in Google Earth.

4587000.000000

500 m

Ã
573800.000000

573900.000000

574000.000000

0574100.000000

50

100

200 m

574200.000000

574300.000000

4586900.000000

4587000.000000
4586900.000000

pole-camera for a wider area:
Aigyrzhal II/III sites

4587100.000000

4587200.000000

hillshade image, colored by elevation.

4587100.000000

4587200.000000

573800.000000

500 m

supports for archaeological landscapereconstruction

visualization
https://vimeo.com/138025938

aerial panorama of Sannai Maruyama
Site, Aomori, northern Japan

Aomori Bay
Aomori PlainààThe Okidate River

North Mound
MuseumSouth Mound

yà

Sou

alle
th V

West Mound

3D model construction by UAS-SfM

topographic map of Sannai Maruyama Site
474400474500474600

North

Valleyà4518000

4743004517900

North Mound

4517800

Soutth MoundWest Mound
S ou

th V

y
alle

à

100 m

orthorectified image

hillshade image of 25-cm resolutionDEM1-m contour lines

before filtering

after filtering

hillshade before filtering

hillshade after filtering

3D virtual tour

https://sketchfab.com/models/85f309f1204e4115ae4a3d4dbe885069

exhibition at The Musesum of Kyoto

highprecision
GNSS forUAS

Tateyama volcano

Reach RTK on Phantom 3© emlid

Shutter sync via hot-shoe(not for this case, though)

Reach RTK DF13 connector

2018.01.24 @GIC of AIT

Ricoh GR + adapter AS-15 +sync code SC-11
DF-13 connector
by Shoichiro Uchiyama

UAV flight over volcanic crater

Tateyama Murodo Jigokudani

orthorectified photo + contours

hillshade (4 cm)

Leap second• GPS Time: unchanged since Jan 6, 1980
• UT1: the principal form of Universal Time• UTC: Universal Time Coordinated;slow, inconstant drift over decades• occasional intercalary of leap seconds

hillshade (4 cm) + contour 1 m

15
5

Accumulation of leap seconds

GPS Time – UTC =18 seconds
as of late 2017

http://www.oc.nps.edu/oc2902w/gps/timsys.html

assignment of camera positions (1 Hz)

Test case: SfM – GCP RMS <1 cm
t00
t01
t02
t03
averagRMS

by the way,0.0103965
0.1138475-0.0466595-0.0775845
1.81899E-12
0.00532-0.04772625
0.13749075-0.03062325-0.05914125-1.81899E-12
0.00640-0.583418
0.0476117-0.042255-0.005356
00.00136higherresolution

15
6

is higherresolution

is higherresolution
always good?

example of Google Earthprivacy© Google

unexpected points over moving items

noiselarge datasets à time cost for analysis

4 GB / file

1.5億点

0.15 billion points

data sizehigh resolution, but…

one step back, then you will see it.

© Shunji Hayakawa

clarification?

higher-resolutionlowresolutionhigh≠higher-definitionor clarification
clarification?

lowselection
object, purpose

higher-resolution▶more options

resolution

selection
object, purpose

high resolution
highoptimum resolution
what comes after having these data?

17
6• technological development enabled rapid,
accurate acquisition of high-definition
topographic data on site

remarks• now we have (hopefully) more time for:– geomorphological mapping and surveys– archaeological surveys# with support of detailed maps

– visualization and sharing ideas of palaeolandscapes# 3D viewpoints with realistic feelings

other issues

17
7• selection of appropriate resolution/scale ofdata, as well as appropriate methods,
depending on– purposes (area of interest)– restrictions (legal)• more efficiently, more easily…
– for further applications in many field sites– establishment of “protocols”

http://topography.csis.u-tokyo.ac.jp/• including manual distribution and hosting seminars…

Joint Research Assist System

OPEN DATA for researchers!

Jan 26– Abstract –Feb 19