ENRI International Workshop on ATM/CNS. Tokyo, Japan. (EIWAC 2009).

StepsTowardstheVirtualTower:RemoteAirportTrafficControlCenter(RAiCe)
N. Fürstenau, M. Schmidt, M. Rudolph, C. Möhlenbrink, A. Papenfuß, and S. Kaltenhäuser
*Institute of Flight GuidanceGerman Aerospace Center (DLR)
Braunschweig, Germany
norbert.fuerstenau@dlr.de
Abstract: In this paper research is described which aims at the long term goal of a Virtual Airport Tower without theneed of a real tower building, however with improved functionality. Specifically we address the intermediate step of a
Remote Airport Tower Center (RTC) for remote surveillance and control of small airports. Because previous work andtask analyses indicated the importance of the visual information for present days controller's work procedures, anexperimental high resolution video panorama system was developed as main HMI. Field tests of the reconstructed far
view yield the effective visual resolution of a 180°-video panorama in agreement with the theoretical predictions. The
digital video panorama provides the framework for video-see-through augmented vision by integration of additional
information like weather and transponder data, and it allows for panorama replay. An integrated zoom function
provides a "foveal" component by means of a remotely controlled pan-tilt zoom camera, including object tracking
options, with a high resolution exceeding the human eye within an observation angle < 15°. Simulation environments
are under development, based on a two-airport tower-simulator and a Petri-net based computer simulation using asimplified airport microworld. They support design of the new work environment and safety analysis.
Keywords: Virtual tower, remote tower center, panorama display, zoom, augmented vision, resolution, simulation
may be reduced by a high resolution panorama display
in a distance to the operator comparable to radar andflight data displays. Consequently it is assumed that
under the guideline of human centered automation, thereconstruction of the far view from the control tower ofsmall airports will improve the transition process to a
towerless work environment and make it acceptable to
the remote Tower Operation (RTO) controller.

1. INTRODUCTION
Anticipating the future growth of air traffic and theframework defined by programs as SESAR, Vision2020 or NextGen a need for Remote Control Centersalready exists and will become more urgent in the near
future [17[18]]. Furthermore discussions andworkshops with stakeholders during previous Virtualand Remote Tower projects (e.g. [19]) identified theneed of stakeholders for concepts and solutions for
Remote Tower Centers (RTC).

Within the previous DLR project "RapTOr" (Remote
Airport Tower Operation Research) an RTO
experimental system was realized at the Braunschweig
research airport BWE [1][2][14][15]. Research was
accompanied by a structured work and task analysis[5][6] and model based simulations of controller's
decision processes [8]. An experimental 180° video
panorama system was developed as core of the RTO
controller's HMI. For designing a compact RTO work
environment video see-through augmented tower
vision (ATV) was realized by integrating informationfrom real time image processing and electronic
surveillance sensors like multilateration into the digital
videopanorama. ATV was proposed by several authors
before, however aiming at augmenting the real far
view by means of optical see through head mounted[20] and head up displays, e.g.[11][12] which involve
the problem of latency between real world changes andcoherently superimposed information and of visual
instabilities[13]. This is avoided with the present
approach.

Remote Airport traffic Control Center (RAiCe)
describes the goal of providing aerodrome control
service for several small airports from a remotely
located control center (RTC) without direct far view tothe airport surface. Because small airfields usually lackany advanced electronic surveillance system a highresolution augmented vision video panorama as apotential low cost system was proposed to replace thedirect far view out of the tower windows as main
component of the Human Machine Interface (HMI)[1][2]. RTC is realised within the DLR project RAiCe
as intermediate step towards the Virtual Tower as longterm goal.
A number of tower work analyses performed during
the recent years determined the importance of visualsurveillance of the tower and apron controllers forcreating their situational awareness, despite theavailability of electronic surveillance [3][4][6][14]. In
the tower environment of large airports the permanent
refocusing between far view and displays contributes
to workload and increases head-down time [4]. Both

Section 2 introduces the basic RTC HMI concept for
two alternative airport traffic scenarios. Section 3
describes the augmented vision video panorama system

67

N. Fürstenau, M. Schmidt, M. Rudolph, C. Möhlenbrink, A. Papenfuß, S. Kaltenhäuser

as basis of the experimental RTC. Results of field trials
are reported in section 4. In section 5 simulation
environments are described which will be used forsupporting HMI and RTC-work system design.
Development of a two-airport RTC-tower simulator aswell as model based computer simulation are outlined.
Section 6 provides a conclusion and outlook.

resolution video panorama system was set up at
Brauschweig research airport.
It serves as experimental environment forinvestigation of different aspects of the Remote Tower
Center concept and development of a demonstrator[1][2][14].
A block diagram of the augmented vision video
panorama system is depicted in Figure 1. The sensor
component consists of four high resolution (1600 x
1200 pixels) high dynamic range (14 bit/pixel) CCD
cameras (P1, 2, 3, 4) covering the Braunschweig airport
within 180° and a remotely controlled pan-tilt zoom
camera (P5: PTZ).

2. REMOTE TOWER CENTER SCENARIOS
Within RAiCe different RTC configurations andprocesses for work organisation will be designed andvalidated. Therefore a set of scenarios regarding sensor
equipment, traffic management and work organization
is defined. The specific configuration of a center
depends on the specific local traffic situation, forwhich the centre has to cater. Nevertheless the newworking environment and structure has to assure that
air traffic can be managed safely and efficiently.
As a first step two basic scenarios for remote control
center configurations have been chosen. They will be
used for experimentation and validation; therefore they
are implemented as a part task simulation microworld
and as a high fidelity simulation. This is described in
more detail in section 5.
In the first scenario a small local airport is controlledfrom the tower of a larger airport. Because a regional
airport by definition has less IFR than VFR traffic an
improved more efficient kind of work distribution
between (tower, ground, RTO) controllers can be
imagined and will be investigated within high fidelity
simulation environments as well as microworld
computer simulations (see sect. 5). To operationallyrealize this scenario the compact RTO console would
be integrated in the tower of the larger airport.
In the second scenario two small airports will becontrolled out of a remote tower center (RTC). The
center has to be located in a suitable area, taking into
account to be conveniently accessible by thecontrollers and to provide for the high bandwidth datatransmission connections required for the highresolution visual information with 20 – 25 Hz
framerate.

Fig. 1: Schematic block diagram of augmented vision video
panorama system. Wide light arrow indicates visual information forthe controller.

The cameras (photo in Fig.1) are positioned ca. 20 m
above the airport surface, horizontally aligned on topof a building at the southern boundary of the airport
with 100 m distance to Braunschweig tower, 400 m
south of the runway 08/26 (1670 m). The verticalaperture angle of about 20° (half angle with respect tothe horizontal line of sight) allows for a closest
surveillance distance of about 60 m. For each camera
the signals with 25 frames/s are split into two outputs.
One feeds the data compression and encryption(AES256) for transmission to the RTO HMI, while theother drives the simultaneous real time image
processing running on a parallel workstation. Figure 2
shows an aerial view of the Braunschweig research
airport indicating camera position and camera viewing
sectors.

On top of these basic traffic scenarios further scenarios
regarding the sensor equipment, work organization andfurther automation support will be investigated.
3. EXPERIMENTAL VIDEOPANORAMASYSTEM
In this section the setup of the experimental RTO
system at Braunschweig research airport is reviewed[1][2][14].
3.1 Reconstruction of the Far View
Motivated by the above mentioned relevance of visual
information for tower work processes, a highA GBit ethernet switch feeds the images from the fivesensors into a single mode fiber optic data link whichtransfers the typically 100 MBit/s data (night-day

68

ENRI International Workshop on ATM/CNS. Tokyo, Japan. (EIWAC 2009).

average) of the panorama system and PTZ over a
distance of 450 m to the visualisation system. A second
GBit ethernet switch splits the incoming data into five
output channels for decompression with one PC per
camera. The PCs also synchronize the displays of thefour segments. Each camera is remotely controlledwith respect to aperture and J correction. The PTZ
camera is controlled with respect to azimuth, verticalangle and zoom (Z = 1 - 23-fold, focal width 3.6 mm –82.8 mm, corresponding to 54° - 2.5° visual angle). An
overall latency time between image acquisition andpanorama visualization of 230 ms – 270 ms wasmeasured by means of a special shuttered laser
arrangement.

zoom data (roughly 500 GByte of data per day) andprovide the possibility of complete panorama replay.
In order to obtain a compact RTO HMI which should
fit into a typical tower environment of a larger airport,
one of the pen touch-input displays in the console ofFig. 3 is designed to incorporate video panorama
control features as well as traffic information, e.g.
electronic flight strips. A mini-panorama at the top is
updated with 1 Hz and serves for commanding the PTZ
camera orientation via pointing of the touch-pen. The
display also contains buttons for optical PTZparameters and activation of automatic object tracking
via movement detection, a virtual joystic as anadditional option for PTZ orientation, and weather data.
For minimizing the number of additional interaction
systems and displays integration of relevant traffic
information into the videopanorama by using
augmented vision techniques is one of the design goals.

Fig. 2: Braunschweig research airport BWE with 1.67 km runway
08/26 extending E-W, fiber optic data link (thin yellow lines)
connecting sensor containers. Circle with radiating lines indicate
panorama camera position and sectors respectively.(Photo: DLR)

The Augmented Vision Videopanorama (AVP-) HMI
for a single operator / single airport surveillance in its
initial version is based on four 21"-LCD-monitors(UXGA, 1600x1200 Pixels) for displaying thereconstructed panorama and a separate one for displayof the remotely controlled PTZ-camera [14][15].

Fig. 3: RTO HMI for 180° single airport surveillance, integrating
videopanorama. Backprojection displays for cameras no. 1=W – 4= E,
PTZ display below (right), and pen touch-input interaction display(left) integrated in the table.(Photo: DLR)

3.2 Augmented Vision

The latest version (Fig.3) is a backprojection systemusing a selfmade compact construction with beamers
projecting onto 31", with 1400x1050 pixels, 3500
ANSI lm, and contrast 3000:1).

Within the video panorama real-time aircraft position
information is integrated as obtained from themultilateration system at the Braunschweig airport via
the aircraft (a/c) transponder. An example is shown in
Fig. 4 where in display no.4 (E) a yellow transponder
code with multilateration position is shown.

Interaction of the operator with the panorama system(cameras, weather station, microphone) is performed
via pen touch-input display. For PTZ positioning thetarget can be defined manually or by automatic
movement detection. A rectangular contour ispositioned at the respective location of the panorama,
defining the target area to be enlarged. With the
tracking mode turned on the square moves coherently
with the corresponding object. An algorithm for realtime movement detection is running on a separate
parallel processor of the image compression PCs ofeach camera.

It indicates a/c position on the approach glide path.
Under reduced visibility this Augmented Tower Vision(ATV) feature allows for localizing the a/c near thecorrect position because the transponder code, a/c label
and numerical information are integrated near thenominal a/c image location in real time.
Another example of augmented vision data is theintegration of GPS-position information transmitted
via ADS-B. An example is shown in Fig. 4 where DGPS data measured during flight testing (see sect. 4)
are superimposed on the video in the form of flighttrajectories (red)which, after geo-referencing are

The five recording PC's with the compression softwareat the camera position allow for storing panorama and69

N. Fürstenau, M. Schmidt, M. Rudolph, C. Möhlenbrink, A. Papenfuß, S. Kaltenhäuser

transformed
coordinates.

fromgeographicalintodisplay

object size at g = 1 km distance corresponding to 1
Pixel is G / B = 0.6 m / 1 Pixel vertical, or ca. 2 arcmin
angular resolution, and 1 m / 1 Pixel along the line ofsight. The observable resolution at the videopanorama
HMI is reduced due to imperfect optics of the camera,
the dynamic (illumination dependent) image
compression, and resolution of the display system. The
optimistic resolution value of about 2' (two times thediffraction limited value of the human eye) may beapproached with decreasing camera aperture, which isof course possible only under good light conditions
and object – background contrast. For realization of thepanorama only 1424x1066 Pixels of each camera (50°viewing angle) are actually used in order to match the
180° panorama angle.
With the known size and distances of static objects on
the airfield it is possible to evaluate the practicallyachieved effective video panorama resolution ascompared to the theoretical estimate of 2 arcmin. For
verification we used the red-white (1 m squares)
multilateration sensor-containers at the end points ofthe fiber-optic data network as reference objects (seeFig.3, height and width G = 2 m). The nearest
containers as captured by the NE and E-looking
camera P3,4 are located at distances gE = 400.8 m (Ref.Obj. 1) and gNE = 588 m (dark blue circle, Ref. Obj. 2)
respectively. With the above mentioned lens equation
we obtain 7.8 and 5.3 pixels respectively of the camera
chip covered by the container images in the verticaldirection. These values correspond to a measured
resolution of DVexp = 1.7 arcmin for Ref.Obj. 1 andDVexp = 1.4 arcmin for Ref.Obj. 2 [14], which appears
reasonably close to the theoretical estimate.

Fig. 4: Screenshot of camera no. 4 (= E) display showing
augmentation during landing. Superimposed glide path (violett),
GPS-trajectories (red), multilateration position (yellow, fromtransponder) and automatic movement detection (red square).

Contours of the movement areas are superimposed on
the reconstructed panorama for guiding the operators
attention during low visibility to those areas where
moving vehicles are expected. Movement areas arealso the preferred targets of a high resolution(640x512) infrared camera system with PTZ function
operating in the mid-IR range (2-5 Pm) which ispresently integrated in the experimental system for
improving night vision and visibility under CAT I
conditions.
One important advantage of the so called video seethrough augmented vision technique using the digital
video panorama is the easy integration of augmented
vision features. This characteristic avoids the problemof (computational) delay between real scene andaugmented information of the optical see-throughtechnology as realized with the head–up and head
mounted techniques [11][12][20]. Initial laboratory
experiments
and theoretical investigations withsuperimposed information on the far view addressed
the human performance such as response time andhead down time reduction [4] by using transparent
displays for reducing the number of monitors [11][12],and the problem of uncontrolled perceptual switching
due to ambiguous stimuli [13].

The theoretical angular resolution of the PTZ-camera
is given by DZ = pH / Z f0, yielding DZ = 1 arcmin ( Z =4, viewing angle 24 = 15°) and DZ = 0.2 arcmin ( Z =23, viewing angle 24 = 2.5°), with pH = horizontal
pixel size = 4.4 Pm and f0 (Z = 1) = 3.6 mm.
4. FIELD TESTING
In this section we review the results obtained by flighttesting for determining the subjectively experienced
visual resolution of the reconstructed far view. The
main question to be answered refers to thecomparability of the video panorama with the real view
out of the tower windows with regard to the control
tasks of the operators. For validation of thevideopanorama system usability including the zoom
function a flight-test plan was set up for experts andnon-experts to evaluate identical scenarios under realview and video panorama conditions. Here we reporton two experiments performed in 2007 on May 21 withclear sky and 22 with reduced visibility (< 10 km).

3.3 Expected Performance
Performance analysis of the visual system is based on
the assumption that delay and optical resolution are themost important parameters for optimizing system
quality. Other important parameters are visual contrast
and dynamic range which, however, were not in thefocus of the present research.
By using the fundamental relationship G / B = (g/f –1) | g / f , with f = focal length = 12.5 mm, g = object
distance, G = object size, B = image size, and a CCD
pixel size of p = 7.5 Pm (+ 0.5 Pm gap), the vertical

4.1 Experimental Design
Flight tests of two hour duration each, with the DLR
DO-228 (D-CODE) test aircraft were designed with70

ENRI International Workshop on ATM/CNS. Tokyo, Japan. (EIWAC 2009).

successions of approach, touch-and-go (or low
approach) and takeoff. Five subjects (2 controllers of
the Braunschweig Tower (S1,S2) and 3 non-experts (S3,S4, S5, members of the human factors department))
observed the flyby from a position near the panorama
camera system and monitored times of 11
characteristic events e1 – e11: out of sight, low / steep
dept. angle, take-off, touchdown, approach main /grass runway, landing gear down / up, steep approach,first sighting. The measurements were performed withnotebook (touch input) computers by each subject
using a specially designed data input software (GUI).
Pilots received the flight plan for up to 16 approaches.
For the trials a LAN with time synchronized cameraand data acquisition touch-input laptops was used. Oneof the GPS trajectories recorded for each flight withthe onboard Omnistar satellite navigation system isshown in Fig.5, including event observation positionsx(ei) of the corresponding observation times t(ei).

within the approaches. They had to activate thecorresponding field of their input display of the tablet
PC and set a time mark at the time of their observationof one out of the 11 possible events during each of the
D-CODE approaches / flybys (e.g. a/c visible = firstsighting of aircraft under the given (weather)conditions). Also all approaches of additional (non-DCODE) a/c were monitored. Experts and non-experts
were briefed separately before the first experiment,with both groups filling separate questionaires.
4.2 Results and Discussion : Videopanorama
For each trial raw data from all subjects and for all
approaches under real view conditions were collected
into a single data file. Evaluation of the different
approach, touch-and-go, and departure conditions
yields the inter-subject time measurement scatteringwith mean and standard deviation (stdev) of the sample
and standard errors (sterr) of mean for the n = 5
subjects [2][14].

For the present task of determining the perceived video
resolution only the four well defined events with thelowest time variances were used (see Table 1).

Altitude/Meter

800

a/c visible
a/c out of sightgear down
concrete runway

Detailed information on the difference between realview and video panorama are obtained by repeating theexperiments with the video panorama replay after aweek or more in order for the subjects to no longer
remember the different flight conditions. It was
expected that due to lower resolution of thevideopanorama (theoretical estimate DV | 2 arc min,see section 3) as compared to the real view, distantevents of approaching /departing a/c (like first / lastsighting of a/c) should receive an earlier/later markunder real view as compared to video observation.
Correspondingly within-subject evaluations of thedirect viewing and video panorama replay observations
yields time differences t(real view, ei) – t(video, ei) < 0
and > 0 for approaching (app) and departing (dpt) a/c
respectively.

takeoff
touchdown

600
400
200

South−North/KM

0
5
4
3
2
1
0
−4

−2

0

2

4

6

8

10

12

Results of the initial trial #1 were reported in [2][14],showing experimental visual resolution between 1.3
and 2 arc min, already in reasonable agreement withthe theoretical prediction and with the verification
measurements (see sect. 3.3). In Table 1 the results forfour of the 11 possible observation types are shown for
the trials #2, 3 (May 21, sunny day & 22/07, cloudyday), for all subjects and all those flights with pairs ofobservation (time marks) of real view – video, with no.
of observation pairs N, mean 't(real view - video),
standard deviation and std. error of mean.

West−East/KM

Fig. 5: GPS trajectory no. 4 out of 11 test flights of pre-testing at
13/12/06 (clockwise direction). Open / filled symbols represent event
observations under real view / video panorama conditions. Approachdirection 260° with touchdown near ARP at 0 km (52°19'09" N, 10°
33'22" E). Vertical lines = 10 s intervals on flight trajectory.

The distance between the airport reference point ARPand departure and approach turning points was ca. 4
km and 14 km respectively. Flights were performed
under VFR conditions. Each flyby was characterizedby 6 parameters, with parameter values statistically
mixed: 1. approaching main (concrete) or grass
runway; 2. approach angle normal or high; 3. landing
gear out: early, normal, late; 4. low level crossing of
airport or touch and go; 5. touch down point early orlate; 6. departure angle normal, low angle, steep angle.

All displayed events exhibit reproducible andsignificant pos.(dpt.) and neg.(app.) delays between
video panorama and real view conditions. For example
the significant negative delay measured as overallmean for e8 (landing gear visible, -13.0 ± 2.0 s and 13.2± 1.2 s respectively) shows this event to beobservable with video only 0.7 km closer to the airport(a/c speed ca. 100 kn = 185 km/h), as compared to thereal view conditions (e.g. e11(real view): a/c (lights)
recognized at ca. 8 km). If we assume that detection

While pilots had a detailed plan to follow for thesequence of approaches with different parametervalues, the subjects only knew about the different
possibilities (e.g. approach grass or main runway)71

N. Fürstenau, M. Schmidt, M. Rudolph, C. Möhlenbrink, A. Papenfuß, S. Kaltenhäuser

time difference is determined by the difference of
optical resolution between real view (resolution of thehuman eye ca. DE | 1 arcmin = 1/60 °) andvideopanorama system, the measured time difference't(real view-video) = tE – tV from table 2 can be used
for calculating the effective resolution DV of the opticalsystem.

angle before e1 was observable), only e4, e5, e8, e11were used for this evaluation. For applying a
regression procedure the independent variable "eventei" has to be replaced by a quantifiable variable. A
linear model is obtained when considering theobservation distance x as obtained from the GPS
reference trajectory instead of the observation time,yielding the 'x(E – V) = vE(t) 't versus xE dependence
for regression analysis as depicted in Fig. 6 for trial #3(cloudy day).

Table 1: Trial #2(May 21/07, clear view > 10 km) and #3(May22/07,
cloudy, < 10 km). Mean, standard deviation and std. error of event
observation time difference 't = t(real view)–t(replay).

N

Event ei

Mean

S.D.

S.E.

't / s/s/se11: A/C visible

54

-85.1

77.9

10.7

e8: Gear visible

42

-13.0

12.9

2.0

e5: touchdown

22

+1.8

1.0

0.2

e4: Takeoff

17

+2.3

2.5

0.6

N

Mean

S.D.

S.E.

't / s/s/s

Trial #3(cloudy)
Event ei

Observation Diff.: Eye − Video / km2

Trial #2 (clear)

54

-26.5

18.3

2.5

e8: Gear visible

44

-13.2

7.6

1.2

e5: touchdown

25

+2.0

1.0

0.2

e4: Takeoff

23

+2.0

1.4

0.3

1

0.5

0

0

1
2
3
Distance to ARP / km

45Fig. 6: Mean event-observation position differences 'x (real view –video replay) between real far view and video panorama conditions
versus mean GPS-position estimate of xE (= distance from event
position xE to airport reference point ARP) for trial #3. Error bars
represent S.E. of means. Straight line: linear regression.

The scatter plot of the four data points ([xE, 'x = vE't],for e4, e5, e8, e11) is obtained by correlating themeasured time values with the corresponding GPS
position data.

For suitable events with known object size the single't-values allow for calculation of DV via:

 E 1   E v E t/G

1

−0.5
−1

e11: A/C visible

V

Flighttest 22/05/07

1.5

For data fitting the theoretical relationship(1)

x eye  video

where the resolution angle D is given by DE,V = G /xE,V measured in rad, with event observation distance
xE,V under real view (E) and video replay (V)conditions. G is the object size, e.g. aircraft crosssection for e11 or landing gear wheel size for e8. For e8
we obtain in this way DV = 1.4 DE (with G(mainwheel) = 0.65 m, vE = 100 kn). For e11 (using G(cabin)= 1.8 m) trial #3 yields 1.3 DE. Both values are inagreement within the experimental uncertainty,
although smaller (even better) than the theoreticalestimate.

V

1   E / V x E
 E 1  E1

1

(2)
(3)

is employed. With the slope E1 = 'x/xE = 0.429 (r0.02 std.err., R2 = 0.99, significance level F = 321 at p
= 0.003) the corresponding DV estimate of 1.75 arcmin(±0.08) is obtained, exhibiting even better agreementwith the predictions of section 3.4 than the initial
trial#1, reported in [14].
4.3 Results and Discussion: Zoom Function

The extremely large observation time difference ands.d. of e11 in trial #2 is due to real view event
registration under clear view conditions (mostly expert
subjects S1, S2) long before the a/c turned towards
approach at the ILS turning point. In order to obtain astatistically relevant and model based mean value, alinear regression procedure is employed for those
events where the optical resolution (more or lessmodified by image contrast) may be assumed to playthe dominant role for event timing. Because e1 wasunreliable due to observability problems ( the aircraft
quite often vanished from the P1-camera observation

In order to decrease the duration of the replay
experiments for evaluating observations with the PTZ
camera (e11, e8) only the approach sections of thevideos until touchdown (event e5) were used. Because
due to this procedure time synchronization with realview experiments was lost, PTZ experiments wererelated to panorama replay with touchdown time ascommon reference. For data evaluation equation (1)with substitution of DV through DPTZ and DE throughaV was used, yielding72

ENRI International Workshop on ATM/CNS. Tokyo, Japan. (EIWAC 2009).

 PTZ

 V 1   V v't/G

1

states and the airport ressources (three places at theleft).

(4)

The experimental results for the effective PTZ
resolution yield mean values DPTZ = 1.2 – 1.3 arcmin
for zoom factor Z = 3.6 – 4, obtained from 84
observations of events e11 and 24 observations of e8.

The state of the airport process model (Fig.8)
determines the type and content of visual andelectronic surface traffic information (e.g. usage oftaxiways, landing clearance) which can be acquiredand communicated by the controller.

They are reasonably close to the theoretical value DPTZ
| 1' = DE as obtained under the hypothesis ofresolution limited object detection times. These data
were obtained with 20 subjects observing those threerounds around the airport on each of the two days,
which included a touchdown (e5) to be used ascommon PTZ – videopanorama reference with 't(Panorama – PTZ) | 0 s.

The V-shaped sequence of places and (replacement)
transitions represent (from left to right) the
corresponding timed process sequence from approach
via taxi-in, apron/parking (block), pushback, taxi-out,takeoff, airbound.
InAP

EventPG

ProcessTaskxInterAct
ProcessEventxINT

5. RTC-SIMULATION ENVIRONMENTS

AirportProcess

ProcessEvent
InterActionModel

AirPortRes

HumanResInterActionModel

PG

RtoController

AP_ModelThe design and development of the new Remote Tower
Center work environment is supported by tower
simulator based and microworld computer simulations
which are controlled by formal process and decisionmodels.

PG_ModelOutAP

ActionPG

ProcessEventxInterAct

ProcessTask

Fig. 7. Controlled airport system as Human Machine System ismapped on the highest hierarchy level of a Colored Petri Net.
Transitions (rectangles) between places as system states (ellipses)
include subnets for airport process, interaction model, and controller
model (from left to right).

5.1 Airport Microworld Computer Simulation
In previously reported work [6] the formalised resultsof an airport tower work and task analysis provided the
input data for a Formal Airport Control Model(FAirControl) for the simulation of the controller
decision making processes at the tower work positions.
In [8][9] it is shown how the results of a cognitive
work analysis are transferred into an executable humanmachine model, based on Colored Petri Nets (CPN)[10] as graph theoretical tool for simulating thecontrollers work processes in relation to the airport
processes.

The interaction model includes sub networks fordescription of information resources, such as radio
communication and visual perception of the traffic
situation. Consequently the human model(s) andmachine model(s) can work independently from eachother for certain time periods.
1`((cs,ko,#3 event1),in1+1)++
1`((noObject,ok,#3 event2),in2-1)
ProcessEventxINT
1`(event1,in1)++1`(event2,in2)

ProcessInfo
I/O((noObject,ok,cont),0)((cs,va,cont),in1)

update_FarView[in1>0]in1`cont
ObjectxTime

Content

70
dur_induration

INT
du1
@+du1
Inbound

The formal model allows for evaluation of different
alternatives of work organization under simplified but
reproducible laboratory conditions. It also supports thedesign of the new work environment and themonitoring of psychological parameters, e.g.
uncovering of reduced situational awareness. The
model is separated into submodels for the human agent(controller), human system interaction, and the traffic
process [6][8][9]. The corresponding highest hierarchy
CPN level is shown in Fig.7.

input (cs,du1);
actionoutpString(cs,"bounded_in",du1,0);()()
ControlZone

cs
ObjectxTime
ACsIN
In

ACsOutOut
cs

avaiStands(cs,started)cs

ObjectxTime

du1

ObjectxTime

occupStands

inited@+du1

OutboundStandList

UNIT

cs

1`RunwayArea

1`[st1,st2,st3,st4]()

airborne

ObjectxStand15
dur_ou
INT

ProcessStatecs[(#3 event1)=RunwayArea andalso(#3 event2)=TaxiExit2](cs,started)(cs,taxied_out)

Process
State

Approach

TakeOff

Approach

@+du1
du1
75
ObjectxTime

cs
ObjectxTime

landedINT

dur_to()input (cs,du1,in1);
actionoutpString(cs,"started",du1,

taxied(cs,parked)(cs,ready)()TaxiInTaxiOut
TaxiOut
ObjectxTime

ACsBlock
In

res1
UNIT

TaxiInObjectxTime

ObjectxTime

on_block

SystemState0

cs

cs

ready

Block
200

Out
listProcessState

@+du1

du1

input (cs,du1);
actionoutpString(cs,"ready",du1,0);

dur_bl
INT

OutAP
Out
ProcessEventxInterAct1`((cs,messaged,DepOut),(PG,Radio))
InAP
In
ProcessTaskxInterAct1`((cs,requested,StUpEnPBCL),(PG,Radio))Fig. 8. Subnet for the airport process, including lower level subnets
for approach, taxi-in, taxi-out processes.

Rectangles represent substitution transitions between
places (ellipses) as system states representing threesubnets for (from left to right) the airport traffic
process, the human-system interaction, and thedecision making of the cognitive agent (controller) on
the lower CPN-hierarchy levels. Ellipses represent theplaces for tokens which dynamically move througout
the net structure via activated (firing) transitions, andtogether with the transitions they constitute the static
structure of the system. In Fig.7 they represent theprocess tasks, events and the human ressources (threeplaces at the right), which are connected via theinteraction substitution-transition with the input/output

The controller model (human model) is implemented
as a Formal Cognitive Resource (FCR) Model [7][9]and serves for the description of controller behaviourin the tower work environment. In the present version
it is reduced to a completely rule based process. The
task execution process on the 3rd CPN hierarchy
sublevel (as subnet of the controller model on the 2ndlevel, not shown) is depicted in Fig. 9.
Controller's decisions, e.g. "deliver startup clearance"
are substitution transitions (rectangles) representing
decision subnets on the lowest hierarchy level. Places(ellipses) at the left and right of the figure represent73

N. Fürstenau, M. Schmidt, M. Rudolph, C. Möhlenbrink, A. Papenfuß, S. Kaltenhäuser

goal states / process states / human ressources andactions / events respectively.

operators by communication with the airport process
via the interaction module. The interaction of thehuman operators with the CPN controlled traffic
process is monitored and the attention/perception canbe measured via eye-tracking [16].

The CPN-controlled airport human machine system
drives a schematic microworld visualisation via
TCP/IP based communication between the Petri net(Fig. 7) and the visualization , programmed in C++(depicted in Fig. 10). Active communication between
pilot and controller during task execution ishighlighted by different coloring of the labeling of thesymbolized aircraft moving within the microworld
visualization.
deliver_StUpCL[]

DEC1111deliver_EnRCL
DEC1112deliver_PushCL
DEC1113deliver_TaxiCL
DEC1114For simulating the simultaneous control of two airports
from a remote tower center (RTC) the existing
FAirControl microworld is presently being extended
accordingly by adding a second airport process and asecond controller agent. It will provide the possibility
to investigate work distribution and performance ofcooperating controllers in a simplified RTC
microworld environment under different laboratory
conditions of work organisation and for different
traffic scenarios. It also supports the monitoring of
psychological parameters, e.g. uncovering of reduced
situational awareness.

deliver_TaOfCL
DEC1212

listProcessStateI/O PGgoalMem
ProcessTask
I/O

PG

ActionHumanRes
I/O

Out

ProcessEvent[]
ProcessState

Event
I/O

listProcessStatexEval
I/O GoalState

5.2 Airport Tower Center Simulator

ProcessState
deliver_LandCL
DEC2212deliver_TaxiIn
DEC2111deliver_TaxiSt

For investigating possible RTC work system
alternatives with different traffic scenarios, DLR's
Apron and Tower simulator (ATS) was extended by aremote tower operator console as shown in the photoof Fig. 11.

DEC2111A

Fig. 9: Controllers tasks which are implemented in the mental modelof the agents on the third subnet level.

The formal work process model with graphicalrepresentation of the controlled traffic process
improves the communication between domain experts
and system developers by simulating different traffic
situations to establish a basis for a structured interview
of those situations. Interviews of two senior controllers
focused on the visual information from the outside
view. They provided input for the model development[6][8].

Fig. 11: RTC simulator environment. Photo of the 200° vision system
of DLR tower simulator (ATS) with extension by the 180°-RTO
console, showing live Panorama. Photo: DLR.

Fig 10: FairControl Airport microworld: Graphical visualization of
the simulated controlled inbound/outbound sequence of a genericsmall airport.

The main purpose of the RTO console is to provide alive panorama display of the Braunschweig airport
traffic situation including PTZ camera, displayed on
the touch input display which is integrated into theconsole (see also Fig.3). Alternatively the console ispresently prepared for optionally switching into
simulator mode for displaying a second simulated
airport traffic situation within DLR's ATS system, bymeans of a second image generator and coupling to thetraffic simulation engine.

The present FairControl version with one controller
managing the microworld airport traffic [21] (see Fig.7,
in contrast to [8] with two interacting controllers)
serves for validating certain implemented aspects of
the rule based Petri net agents decision making andtask execution, e.g. selection of alternative strategies[16]. For this purpose the agent can be turned off andthe microworld traffic can be controlled by human

There are several reasons for validating the RTC
concept, besides field tests (see sect. 4), also by meansof integration into a real-time simulation environment.
First of all it ensures control and reproducibility overexperimental conditions and constraints. Variation of
traffic mix and load, environmental conditions and thecreation of possibly conflicting situations allows theevaluation of human factors and safety related issues.

74

ENRI International Workshop on ATM/CNS. Tokyo, Japan. (EIWAC 2009).

Furthermore, the ability to vary between different
controller work position (CWP) configurations andoperational procedures in the simulator will enable amore comprehensive analysis of related organisationaland operational constraints for the implementation of
the new RTC concept. The intended simulation setup
supports also the analysis of HMI- and RTC-worksystem design. The real-time simulation capabilities
are intended for validating the two RTC configurations
mentioned in section 2:

Finally, the setup and configuration of field testing
equipment, RTO work positions, and simulation
environments at the Research Airport Braunschweig
also allows for using the implemented RTO CWP
under shadow mode conditions for control of air traffic
at the BWE airport, i.e. with controllers following thereal traffic in parallel to the controller staff in charge,
however without interacting with the pilots.
6. SUMMARY AND CONCLUSION

1) Integrated RTO working position within a real
tower environment (e.g. tower cab of larger airport)
2) RTO working positions located within an RTC for t
2 small airports (including variation of single-airportand multi-airport-CWPs)

Basic scenarios and elements of DLR's experimental
Remote Tower Center (RTC) system at the
Braunschweig Research Airport are described.
Quantitative evaluation of field trials for comparing
real view and video panorama observation verified thetheoretically predicted video panorama and PTZ
resolution of ca. 2 arcmin and ca. 1 arcmin (with zoom
factor Z|4) respectively. The latter one corresponds tothe foveal resolution of the human eye and exceeds itwith increasing Z, approaching the physical diffraction
limit. Presently extensions of the augmented vision
videopanorama system are under way for improving
the low visibility conditions, including high a
resolution infrared sensor and a second PTZ camera
near the runway. Another improvement concerns theaugmented vision function which will be turned into an
active visual assistance system by enhanced usage of
automatic image processing. As most important
extension a second more distant airport will beconnected to the present experimental system forevaluating live video data transmission bandwidth
limitations and delay, and reliability.

Integrated into the simulator, the live traffic source of
the RTO console will be replaced by a virtual
representation of the outside view, based on traffic
movements now generated by a simulation engine.
With switching into simulator mode, not only thecamera view will be replaced by a panorama derived
from an image generator (IG) but also the position data,
flight plan data and weather information, and airfieldlighting will be provided by the simulation. Thus also
the augmented vision capabilities of the RTO CWP areusable and can be examined within the validation setup.
In addition, the setup with two or more RTO consoles(two-airport RTC) connected to t 2 image generatorsand traffic engines leads to a multiple-airport RTC
tower simulation. Both configurations will provide
simulated air traffic at two different airports within a
combined work environment.

Besides the experimental sensor system and video
panorama based HMI a RTC simulator environment isrealized. It consists of a tower simulator including acompact remote tower operator (RTO) console, forsimulated control of two airports and a simplified
microworld computer simulation for reproducible(computer) experiments with different workorganisation alternatives.

Together with the analysis of the simplified
FairControl microworld (lab-type) simulations, thereal-time simulator experiments are an integral part ofthe (cycled) concept development- and validation
process. Due to its characteristics, the experiments
carried out within the simulator will focus on certain
specific issues. The experimental design will cover the
analysis of operational procedures, the dedicated work
environment and the evaluation of its influences oncontroller workload and situational awareness.
Additionally the developed work share within the
combined RTC environment will be observed andanalysed as well as attention- and perception related
factors. Within the experiments variation of different
visual conditions, including reduced visibility
conditions (fog) as well as a variation of available lightsituations (day- and night-time conditions) will be
examined.

7. ACKNOWLEDGMENTS
We are indebted to Monika Mittendorf for help withdata evaluation. Advice during numerous interviews
concerning operational procedures and support ofwork and task analysis by controllers H. Uhlmann, D.
Bensch, and D.Schulz-Rückert of German air
navigation service provider DFS are acknowledged.
REFERENCES

To ensure the validation environment is as realistic as
possible (and necessary), experiences and results fromfield trials at Braunschweig airport (see sect. 4) can beused for calibration and alignment of the simulator
setup based on comparison between recorded traffic
data and video signals and simulated air traffic.

[1] Schmidt, M., Rudolph, M., Werther, B., Fürstenau,

N., "Remote Airport Tower Operation with
Augmented Vision Video Panorama HMI", Proc.
2nd Int Conf. Res. in Air Transportation ICRAT,
Belgrade (2006) p. 221–230[2] Schmidt, M., Rudolph, M., Werther, B.,

Möhlenbrink, C. Fürstenau, N., "Development of75

N. Fürstenau, M. Schmidt, M. Rudolph, C. Möhlenbrink, A. Papenfuß, S. Kaltenhäuser

an Augmented Vision Video Panorama HumanMachine Interface for Remote Airport Tower
Operation", In: M.J. Smith, G. Salvendy (Eds.)
Human Interface II, Lect. Notes Computer Science
vol.4558, Springer-Verlag Berlin Heidelberg(2007), p. 1119-1128,

visual stimuli in augmented vision systems".
Lecture Notes in Artificial Intelligence 4021
(2006) Springer-Verlag, Berlin, New York, 20-31[14] N. Fürstenau, M. Schmidt, M. Rudolph, C.

Möhlenbrink, B. Werther, "Development of an
Augmented Vision Videopanorama HumanMachine Interface for Remote Airport Tower
Operation", in: Proc. 6th Eurocontrol Innovative
Res. Workshop, Eurocontrol Experimental Center,
Bretigny (2007) 125-132[3] Tavanti, M., "Control Tower Operations: A

Literature Review of Task Analysis Studies". EEC
Note 05 (2006)[4] Pinska, E., "An Investigation of the Head-up Time[15] N. Fürstenau, M. Schmidt, M. Rudolph, C.

at Tower and Ground Control Positions". Proc. 5thEurocontrol Innovative Research Workshop(2006) 81-86

Möhlenbrink, W. Halle, "Augmented vision
videopanorama system for remote airport tower
operation", in: Proc. ICAS 2008, 26th Int.
Congress of the Aeronautical Sciences. I. Grant(Ed.), Anchorage, (2008) ISBN 0-9533991-9-2[5] K.J. Vicente, "Cognitive Work Analysis",

Mahwah/NJ: Lawrence Erlbaum Associates, 1999.
[6] Werther, B., Uhlmann, H., "Ansatz zur[16] Möhlenbrink, C, PhD Dissertation, to be publishedmodellbasierten Entwicklung eines
Lotsenarbeitsplatzes". In: Zustandserkennung und
Systemgestaltung, Fortschritt Berichte VDI, 22(2005) 291 -294.

[17] SESAR Definition phase Deliverable 2: "AirTransport Framework: The Performance Target".
Document No. DLM-0607-001-02-00 12-2006[7] Werther, B., "Kognitive Modellierung mit[18] Matas, M., "Future Airport Concept", in: Proc. 4th

farbigen Petrinetzen zur Analyse menschlichen
Verhaltens". PhD Dissertation, DLR-Inst. of Flight
Guidance, (2006).

Eurocontrol Innovative Workshop, Dec. 6-8(2005) 135-150[19] http://www.dlr.de/fl/en/desktopdefault.aspx/tabid-[8] Werther, B., "Colored Petri net based modeling of1149/1737_read-3120/

airport control processes". In: Proc. Int. Conf.
Comput. Intelligence for Modelling, Control &Automation (CIMCA) Sydney (2006) IEEE ISBN
0-7695-2731-0[20] Ellis, S., "Towards determination of visualrequirements for augmented reality displays andvirtual environments for the airport tower". Proc.
NATO workshop on Virtual Media for the
Military, West Point /N.Y., HFM-121/RTG 042
HFM-136, (2006) 31-1-31-9[9] Werther, B., Schnieder, E., "Formal Cognitive

Resource Model: Modeling of human behavior incomplex work environments", in: Proc. Int. Conf.
Computational Intelligence for Modelling, Control& Automation (CIMCA 2005), Wien: 2005, pp.
606 – 611.

[21] Werther, B., Möhlenbrink, C., Rudolph, M.,"Colored petri net based formal airport control
model for simulation and analysis of airport
control processes", In: V.G. Duffy (Ed.) Digital
Human Modeling, Lect. Notes Computer Science
vol.4561, Springer-Verlag Berlin Heidelberg(2007), p. 1027-1036,[10] Jensen, K., "Coloured Petri Nets". Berlin: Springer,

1997.
[11] N. Fürstenau, M. Rudolph, M. Schmidt, B. Lorenz,

T. Albrecht, "On the use of transparent rearprojection screens to reduce head–down time in
the air–traffic control tower", Proc. Human
Performance, Situation Awareness andAutomation Technology (HAPSA II), Daytona
Beach , Lawrence Erlbaum Publishers Inc. (2004),
195–200

COPYRIGHT“Copyright Statement
The authors confirm that they, and/or their company or
institution, hold copyright of all original material
included in their paper. They also confirm they haveobtained permission, from the copyright holder of any
third party material included in their paper, to publish
it as part of their paper. The authors grant full
permission for the publication and distribution of theirpaper as part of the EIWAC 2009 proceedings or as
individual off-prints from the proceedings.”[12] Peterson, S., Pinska, E., "Human Performancewith simulated Collimation in Transparent
Projection Screens". Proc. 2nd Int. Conf. Res. in
Air Transportation, Belgrade (2006) 231-237.
[13] Fürstenau, N., "Modelling and Simulation of

spontaneous perception switching with ambiguous

76