Steps Towards the Virtual Tower:
Remote Airport Traffic Control Center (RAiCe)N. Fürstenau, M. Schmidt, M. Rudolph,
C. Möhlenbrink, A. Papenfuß, S. Kaltenhäuser
German Aerospace Center (DLR), Inst. of Flight GuidanceBraunschweig< ICAS2008 > N. Fürstenau
26.8.08

Overview¾ Introduction¾ Work Analysis & Scenarios¾ Simulation¾ Remote Tower Experimental System¾ Field Testing¾ Outlook and Conclusion
Folie 2< ICAS2008 > N. Fürstenau
26.8.08

Future Tower ControlReduce Cost of Airport Surface
Movement ManagementImprove Situation Awareness
Increase Safety

DEP RWY 28 / 16
LH089

Reduce Weather Dependence

B747

28E68

08:50:00

DEPWind 300° 5 kt
08:47
Photo: DFS

Solution: "Virtual Tower"
Sensor Based (Windowless) Control Centerwith Video Reconstruction of Far View
Concept Study ViTo (2002 – 2004)
Project RapTOr (2005 – 2007)
Project RAiCe (2008 – 2012)
DLR Advanced Control Center Simulator

Folie 3

< ICAS2008 > N. Fürstenau
26.8.08

German
Airports
Motivation for RemoteTower Center(RTC) Research:• meet request of low cost carriers byproviding cost efficient controlled
airspace to small airports• Capacity increase of Hubs byCoordinated central control of RWY
NetworksRAiCe Research Goal:
Methods & Technologies to provide
centralized controlled Airspace forSeveral Airports from a RemoteTower Center (RTC)

Folie 4

< ICAS2008 > N. Fürstenau
26.8.08

Overview¾ Introduction¾ Work Analysis & Scenarios¾ Simulation¾ Remote Tower Experimental System¾ Field Testing¾ Outlook and Conclusion
Folie 5< ICAS2008 > N. Fürstenau
26.8.08

The Role of Visual Surveillance: Present … Future?

Vision based Ground Traffic Control: "Controllers most used Tool" (25 – 70%)
Problems:⇒High Cost of Conventional Tower Control⇒Head Down Times increase with Computer Assistance & with Traffic LoadFolie 6< ICAS2008 > N. Fürstenau
26.8.08

Wind 6 kt 270°
09:38:57

FR8926DFS Tower Layout

RTC – HMI Design Task:• Integrate Controller'sInformation / Interaction
Equipment into High
Resolution Video
Panorama HMI• Reduce Head-Down Time by
Augmented TWR Vision

Folie 7

< ICAS2008 > N. Fürstenau
26.8.08

Work Analysis: rel. no. of Access to Information Sources for All Tasks(Decision & Support Tasks (clearances, communications etc): 29 (PG), 31 (PL))
for Medium Size Int. Airport

100%

97%
97%

ground
tower

80%
60%

55%

52%

55%

45%

40%
20%

30%

14%
13%

17%

21%

21%

21%

24%

28%

35% 34%
32% 31%
28%

35%

35%
35%

16%

13%

0%

adardire
Fa ct
rView
AFTNATAiIDrpEP SorCtI
O
nf
SormID
atioVSTenN
leph avig Syston
ati o emenC
Liontro ght
Msonlito Cen
rfor terStands
Con FID
StroRladSt
Te io ( ripsGleph rouonnde)
AirR
ad portio(Pilot)

0%

R

Percentage of Use for All Identified Tasks

Use of Information Sources for Ground (PG) and Tower (PL) Controller

Folie 8< ICAS2008 > N. Fürstenau
26.8.08

Local large Airport

Scenario 1:
Remote Control of
Small Airport from LocalTower
Remote small Airport :Controlled Airspace

GBit/sDatalink

Design of RTO Controller Workplace
within local Airport TWRorRemote TWR Center for several small Airports

Folie 9< ICAS2008 > N. Fürstenau
26.8.08

Szenario 2
RTC for Surveillance ofseveral Small Airports

Folie 10< ICAS2008 > N. Fürstenau
26.8.08

Overview¾ Introduction¾ Work Analysis¾ RTC - Simulation¾ RTO Experimental System¾ Field Testing¾ Outlook and Conclusion
Folie 11

< ICAS2008 > N. Fürstenau
26.8.08

Airport
Process 1Controller
Model 1

Interaction
Model
Airport
Process 2

TCP/IP

InAP
ProcessTaskxInterAct
ProcessEventxINT
AirportProcess

InterActionModel

AirPortRes

InterActionModel

EventPG
ProcessEvent
HumanResPG

RtoController

AP_ModelPG_ModelOutAP

ActionPG

ProcessEventxInterAct

ProcessTask

A/P Traffic Model

Interaction Model

Human Model

Different Possibilities ofRTC Work Organization

RTC Microworld Simulation with
Colored Petri Nets
Folie 12< ICAS2008 > N. Fürstenau
26.8.08180° RTO Panorama HMI

200° Airport Traffic Simulator•Live Stream•Live Replay•Traffic Simulation•Synthetic Vision with live Data

Airport Towersimulator with RTO-Console
Folie 13

< ICAS2008 > N. Fürstenau
26.8.08

Overview¾ Introduction¾ Work Analysis¾ Simulation¾ Remote Tower Experimental System¾ Field Testing¾ Outlook and Conclusion
Folie 14

< ICAS2008 > N. Fürstenau
26.8.08

Augmented Vision Video Panorama System at Braunschweig Airport
4Video system forpanorama andimage processing
4Gbit/s - Fibre optic0m041600mLAN
4Experimental

Augmented Vision
HMIHeight 20 mCamera
Resolution
ca. 2 arcmin
Folie 15

< ICAS2008 > N. Fürstenau
26.8.08

PT-Zoom camera
with manual orautomatic object
tracking

AugmentedVision:Multilateration
Position and A/CLabel /Transponder CodePen TouchInput Interaction
Display

180° Augmented Vision Videopanorama RTO-HMI:
Visual Resolution 2 arc min or 0.35 m / 588 m
Folie 16

< ICAS2008 > N. Fürstenau
26.8.08

GPS Position & Speed

Image Processing /Movement Detection

Transponder(Multilateration)

Augmented Vision
Folie 17

< ICAS2008 > N. Fürstenau
26.8.08

Overview¾ Introduction¾ Work Analysis & Scenarios¾ Simulations¾ RTO Experimental System¾ Field Testing¾ Outlook and Conclusion
Folie 18

< ICAS2008 > N. Fürstenau
26.8.08

3 series of flight tests with pre-defined
events during 10 rounds of Aerodrome
Circling (12/06 & 5/07)

Validation
Experiments:
Comparing RealView andVideopanorama
Folie 19

< ICAS2008 > N. Fürstenau
26.8.08

GPS−Trajectory D−CODE 21.05.07: 10th Round (all Participants)
600
Altitude/Meter+ A/C Visible
400x A/C Out of Sight

Concrete RWY
2nd RWY

Gear Down
Steep Approach

Takeoff
Touchdown

Real View

200 Video Replay
052.4
52.38

Latitude/Degree

52.36

5 Subjects(2 Controllers, 3Non-Experts)
Time Stamp 11
Events /10 rounds
around Airport
Braunschweig with
DO-228 (D-CODE)
Under Real View
and Video Replay
Conditions

52.34
52.32

4 km

52.3
10.5

ARP
10.55

10.65
10.6
Longitude/Degree

10.7

10.75

D-GPS Trajectories with Event Observation Time Stamps
Evaluation of Real View – Video Replay of Event-Time Observation confirms 2 arcmin
visual Resolution for Panorama, 1 arcmin for PTZ ( Z = 4 )Folie 20

< ICAS2008 > N. Fürstenau
26.8.08

Video Replay: Deviation from Routine Operation

PTZ: Z ≈ 20
Replay: 1 TB High Res. Video / Day
Folie 21

< ICAS2008 > N. Fürstenau
26.8.08

Summary / Conclusion• Structured Work Analysis ⇒ Requirement for Visual Surveillance ⇒Concept: High Resolution Augmented Vision Videopanorama with PTZ & Tracking
Scenarios: RTO Console at large Airport TWR; RTC for several small Airports• Simulation Environment: supporting Work Analysis & Designing RTC WorkOrganisation• Experimental Environment at Braunschweig Research Airport provides facilities
for testing of Vision Based Remote Tower Center Concepts• Augmented Vision HMI: Compact RTC Console & Reduced Head Down Time
• Initial Field Testing: Visual Resolution of Videopanorama System meets Specs• Perspectives (Project RAiCe 2008-2012):
Experimental RTC with 2 Small Airports for Shadow Mode Testing
Folie 22

< ICAS2008 > N. Fürstenau
26.8.08

RAiCe Team (at 2nd RTO Workshop, DLR 04/2008)

C. Möhlenbrink
W. Halle
M. Mittendorf

M. Schmidt

E. Schlüßler

M. Rudolph

N.F.

Folie 23

< ICAS2008 > N. Fürstenau
26.8.08

Folie 24< ICAS2008 > N. Fürstenau
26.8.08

FAirControl Visualization

Microworld
Cognitive + Interaction + Process Model
Human Model:
Visualization active Goal
Interaction Model:
Pilot Message

Scripted
FAirControl
Process

Machine Model:
Taxi LH120

FairControl Model (CPN Tools)

Scenario Simulations for Supporting Work Analysis & System Design
Formal Airport Control Model for simulating interaction between Operator, HMI, andProcess with Airport Microworld of controlled process. Active goal of Human Model:
blue frame (orange arrow). Communication with pilot (white arrow) changes colour ofcall sign (LH120). [Werther, Möhlenbrink et.al. 2007]
Folie 25

< ICAS2008 > N. Fürstenau
26.8.08

RTO Video Panorama
Experimental System:
Simplified Block Diagram
Technical Data
Cameras:4 x (1600x1200), 25 frames/s
PTZ: f = 3.6 – 82.8 mm, 23 fold
Panorama:
5 UXGA Monitors 1600 x 1200
alternatively:
Wide angle tiled projection with
4x2 SXGA (1280 x 1024)
Data Transfer:
GBit Ethernet, Average 100
MBit/s, MJPEG compressed
Storage Capacity: 5x 500 GB

Folie 26

< ICAS2008 > N. Fürstenau
26.8.08

cloudy

Gear down

A/C visible

touchdown
takeoff

Slope β = 0.43 (±0.02):(R2 = 0.994, F = 321, p = 0.003)

Video Resolution / arc min

Δx (eye − video) = (1 − α E /α V )x E

α V = α E (1 − β1 ) = 1.75α E−1Folie 27< ICAS2008 > N. Fürstenau
26.8.08

Mini
Panorama& PTZoom
positioning
PTZ
Camera &Position
Control:
VirtualJoystick
Weather
Communication& RWY / TWY
LightsElectronic Flight Strips

RTO Touch-/Pen Input Interaction Display
Folie 28

< ICAS2008 > N. Fürstenau
26.8.08

RTO Image Processing

Moving Object detection by Static Background Subtraction.
Determination of Traffic Parameters (Position, Speed) and Detection of Debris & DustFolie 29< ICAS2008 > N. Fürstenau
26.8.08