**EVALUATION OF THE USEFULNES AND USABILITY OF**

**COCKPIT SITUATION DISPLAY PERSPECTIVES**

**FOR ROV OPERATIONS IN APPROACH CIVIL AIR SPACE**

+-----------------------------------------------------------------------+
| Arik-Quang V. Dao, Vernol Battiste, and Stacie Granada-Vigil          |
|                                                                       |
| San Jose State University Foundation                                  |
|                                                                       |
| NASA Ames Research Center                                             |
|                                                                       |
| Walt W. Johnson                                                       |
|                                                                       |
| NASA Ames Research Center                                             |
+-----------------------------------------------------------------------+
| A distributed simulation of ROV operations was conducted by NASA Ames |
| Flight Deck Display Research Laboratory (FDDRL) and the Center for    |
| Aeronautic Technologies (CSAAT) at California State University, Long  |
| Beach. The goals of this simulation were 1) to examine ROV            |
| operators\' ability to maintain standard terminal separation from     |
| other aircraft and ROVs, 2) to examine the possibility of operating   |
| ROVs in terminal airspace without major disruptions in the inbound    |
| traffic flows, and 3) to evaluate proposed ROV operator tools. The    |
| current paper focuses on this third goal.  Specifically the paper     |
| describes the motivation behind the development of the manipulatable  |
| 2D/3D Cockpit Situation Display (CSD), and examines its usefulness    |
| and usability. Data from questionnaires, and from observations of how |
| the CSD was used, suggest that both the CSD format, and the ability   |
| to manipulate the CSD viewing angle, were useful and usable. However, |
| workload appeared to play an important role in the perceived          |
| usefulness and usability of the CSD.                                  |
+-----------------------------------------------------------------------+

**INTRODUCTION**

In support of domestic homeland security, ROVs are being considered for
a variety missions such as port, border, and waterway surveillance.
While these homeland security missions may be of vital importance, they
may not be allowed for fear that the ROVs will disrupt and/or pose a
safety hazard to other aircraft. Prime examples of this are ROV missions
proximate to major hub airports which require the ROVs to operate in
terminal airspace. Presently, if these missions are allowed at all, they
are required to operate during off-peak periods when airport traffic is
very light or non-existent.

In response to this challenge, a team composed of researchers at
California State University, Long Beach (CSULB), and at NASA Ames
Research Center (Ames), has initiated a series of joint simulation
studies designed to directly assess the feasibility of this most
difficult ROV operation - safe and non-disrupting ROV flight in a busy
terminal airspace. These are also studies which seek to develop both
tools and procedures that will support safe operations. The immediate
goal of this initial study was to provide a first look at this
operation, and to gather data to support refinement of our operational
concepts and operator tools.

The ROV's specific mission in this initial study was to safely patrol
reservoirs at low altitudes within simulated airspace of the Dallas Fort
Worth (DFW) terminal area, without major disruptions to the inbound
flows to runways. While, as noted previously, assessments of safety,
effectiveness, and feasibility were goals of the study, the adequacy of
one of the operator tools is the focus of this report. This tool, the
Cockpit Situation Display (CSD), was previously developed as an advanced
cockpit display of traffic information (Johnson, Battiste, & Holland,
1999), and is now being considered for inclusion in ROV workstations.
The CSD provides 3D as well as 2D visualization of ownship, traffic,
terrain, weather, as well as a large number of other features. Although,
there are many other relevant issues of interest to the current authors
and the ROV research community, the scope of this paper will be limited
to the usability and usefulness of ROV navigation display -- which in
this case would be the CSD.

# **Overview of ROV Simulation**

The goals of this first study were to examine three issues. The first
goal, related to safety, was to examine how well ROV operators could
maintain standard terminal separation (3NM and altitude of 1000 ft) from
other aircraft and ROVs. The second goal, related to efficiency, was to
examine how well ROVs could be flown in terminal airspace without major
disruptions in the inbound traffic flows. The study's third goal was to
evaluate the operator tools and determine what features need to be
redesigned for more effective ROV operations in terminal airspace. This
report will focus on this third goal, with specific attention given to
the usability of the CSD display perspectives and view manipulation.

## **Motivation for CSD Development**

The initial research and development for the CSD was in response to the
need stated by the RTCA Task Force 3, Free Flight implementation (1995)
to increase situation awareness on the flight deck in order to develop
and progress the notion of free flight. Their definition of free flight
was "safe and efficient flight operating capability under IFR in which
the operators have the freedom to select a path and speed in real time".
The NASA Ames Flight Deck Display Research Laboratory (FDDRL) has
devoted several years of research into developing a CSD that provides
the situation awareness required for free flight on the modern flight
deck (Granada, Dao, Wong, Johnson, and Battiste, 2005; Johnson,
Battiste, Delzell, Holland, Belcher, and Jordan, 1997; Johnson,
Battiste, & Holland, 1999).

The CSD aids situation awareness with a battery of tools available to
the pilot in a single interface. These tools, which presuppose extended
and advanced surveillance capabilities such as ADS-B , include 1) the
ability to view the flight plan and maneuvering intent of other aircraft
(Figure 1), 2) an integrated Route Assessment Tool that allows users to
view, manipulate, and revise their aircraft's flight plan (Figure 2),
and 3) conflict alerting logic that automatically identifies and
visually indicates impending separation violations on current and
proposed flight plans (Figure 3).

![](media/image1.jpeg){width="3.5006944444444446in"
height="3.5006944444444446in"}

Lastly, the CSD expands the user's awareness of the airspace by allowing
pilot selected viewing of traffic relative to ownship in both 2D and 3D
display modes. For more details about the above and many other tools in
the FDDRL 2D/3D CSD see the Ames CSD User Guide
(http://human-factors.arc.nasa.gov/ihh/cdti/download.html).

**Why a 2D/3D Integrated CSD?**

To achieve the situation awareness necessary for free flight pilots need
vertical and horizontal information regarding ownship and traffic
position and movement. Sometimes pilots need to focus on just one or the
other of these two dimensions, while at still other times they need a
combined or integrated view. According to the Proximity Compatibility
Principle (PCP), when tasks require mental integration of two or more
sources of information, the display should present the information in
close proximity. If tasks require focused attention, then the
information should be presented in reduced "display proximity" (Wickens
& Carswell, 1995). Consistent with this, the CSD provides the
flexibility to manipulate the CSD into any view from a vertical profile
view to a god's eye top down view, to an integrated off-axis perspective
view. Additionally, the pilot can pre-set four preferred views which can
be quickly selected to support a variety of cockpit tasks. Thus the CSD
is designed to satisfy display requirements when the task necessitates
mental integration of 3D information, as well as when it requires
focused attention on traditional 2D slices. Furthermore, the

CSD moves smoothly and continuously between its different views, and
does not jump between them, -- effectively supporting visual momentum
(Woods, 1984).

Finally, the CSD also affords viewing weather and terrain information in
3D, providing an ecological representation consistent with user
expectations of how terrain and weather should be depicted.

**ROV Operator Task Requirements**

In the current study, 4 commercial airline pilots were recruited to fly
either one or two ROVs in heavy, medium, or light traffic conditions.
Using the CSD integrated with a MACS desktop flight simulator (Prevot,
2002), the pilots' task was to perform low altitude surveillance flights
over lakes west of DFW. Both traffic and terrain information was
available on the CSD. Figure 4 shows that the
![](media/image2.jpeg){width="2.734027777777778in"
height="3.0027777777777778in"}![](media/image3.jpeg){width="1.3576388888888888in"
height="0.8395833333333333in"}

filed flight plan covers surveillance over three main bodies of water
north and west of the DFW runways. In order of priority, pilots were
required to 1) maintain 3 nm horizontal and 1000 ft vertical separation
from all other aircraft; 2) only change their route by using the CSD to
modify their flight plan; and 3) fly the ROV over these lakes at
altitudes between 1300 ft and 4000 ft. On half of the flights the pilots
were required to manage a single ROV, while on the other half of the
flights the pilots were required to simultaneously manage two ROVs. This
manipulation reflects the desire in the ROV community for systems where
a single pilot can manage multiple ROVs. For more details regarding the
ROV demonstration see Vu, Dion, Chambers, Ngo, Nelson, Kraft, and
Strybel (2006).

**RESULTS**

## **ROV Operator Response to 2D/3D CSD**

The study lasted five days. Participants received four hours of training
on the first day, and then flew experimental scenarios for four hours
each of the remaining four days (totaling 24 trials).

At the end of each day pilots rated the subjective usability and
usefulness of the CSD 3D perspective view and 2D to 3D perspective
manipulation on a 5-point scale (1 = very difficult, 2 = difficult, 3 =
acceptable, 4 = easy, 5 = very easy). Based on average ratings from the
4 pilots recruited for this study, the CSD's 3D perspective view was
determined to be between acceptable and easy to use (M = 3.5).
Similarly, pilots rated the ability to change from 3D to 2D viewing
modes between acceptable and easy to use as well (M = 3.5). Usefulness
was also rated on a 5-point scale (1 = absolutely useless, 2 = useless,
3 = adequate, 4 = useful, 5 = extremely useful). On average, pilots felt
that the CSD's 3D perspective viewing was useful (M = 4) as was the
ability to manipulate the CSD into 2D and 3D views (M = 3.75).

  -------------- ----------- ----------- ----------- ----------- -----------
  *Table 1*.                                                     
  Percent                                                        
  Frequency of                                                   
  CSD                                                            
  Perspectives                                                   
  Used During                                                    
  Periods with                                                   
  No Conflict                                                    
  and Periods                                                    
  with Conflicts                                                 

  Use of CSD                                                     
  Perspectives                                                   

                 2D only     2D/some 3D  Both        3D/some 2D  3D only

  No Conflict    34.8        24.7        24.7        2.2         13.5

  Conflict       30.2        27.9        23.3        4.7         14
  -------------- ----------- ----------- ----------- ----------- -----------

**Observed CSD Perspective Usage**

In addition to the subjective ratings, each pilot was observed by a
single researcher who judged if the pilots viewed the CSD in 2D only,
primarily 2D and some 3D, 3D only, primarily 3D and some 2D, or a
balance of both views. Results were averaged across both 1 and 2 ROV
operating conditions. Table 1 demonstrates that across both periods with
and without conflict alerts, pilots used the 2D perspective most often.
There was however, a tendency to use the 3D perspective a bit more when
pilots were only controlling a single ROV (Table 2), suggesting that
pilots were more willing to switch to the 3D perspectives under a
lighter workload.

  -------------- ----------- ----------- ----------- ----------- -----------
  *Table 2*.                                                     
  Percent                                                        
  Frequency of                                                   
  CSD                                                            
  Perspectives                                                   
  Used Across 1                                                  
  ROV and 2 ROV                                                  
  Conditions                                                     

  Use of CSD                                                     
  Perspectives                                                   

                 2D only     2D/some 3D  Both        3D/some 2D  3D only

  1 ROV          17.2        34.5        25.9        8.6         13.8

  2 ROV          40.2        22.2        23.0        0.8         13.7
  -------------- ----------- ----------- ----------- ----------- -----------

In summary, the pilot questionnaire data shows that most of the pilots
thought that having the ability to switch between various perspectives
on the CSD useful. All of the pilots thought that it would be useful to
have a 3D perspective available. However, comparison of the observer
judgments of CSD use, with the subjective ratings, shows that the
pilot's *perceived* usefulness of the CSD perspectives may have been
influenced by the amount of workload -- in this case, whether they were
operating only 1 or 2 ROVs simultaneously. Under conditions where pilots
were responsible for only 1 ROV they were observed using primarily a 2D
perspective and switching to the 3D perspective some of the time. When
pilots were operating 2 ROVs simultaneously pilots were observed staying
in the 2D display perspective most frequently. This influence of
workload may have been a contributing factor for lower perceived 3D and
2D to 3D perspective manipulation usability ratings.

**DISCUSSION**

In free flight conditions aircraft separation responsibility is
distributed between equipped aircraft and air traffic controllers. The
CSD provides the vertical and horizontal air space information required
to operate in self-separation contexts -- which in turn, also support
greater situational awareness. The operational flexibility of the CSD
was exemplified by the ability of commercial pilots to accomplish
strategic ROV missions. Preliminary data shows that the ability to
manipulate an ego-centric traffic display into both 2D and 3D was
useful. The frequency of observed transitions between 2D and 3D modes on
the CSD and its perceived usability may have been workload dependent.
Future studies will be focused on the effect of 2D to 3D display
manipulation on operator conflict resolution performance and perceived
workload.

**REFERENCES**

Granada, S., Dao, A. Q., Wong, D., Johnson, W. W., & Battiste, V.
(2005). Development and integration of a human-centered volumetric
cockpit situation display for distributed air-ground operations. *Paper
presented at the 2005 International Symposium on Aviation Psychology,*
Oklahoma City, OK.

Johnson, W. W., Battiste, V., Delzell, S., Holland, S., Belcher, S., &
Jordan, K. (1997). Development and demonstration of a prototype free
flight CDTI. *Proceedings of the SAE/AIAA World Aviation Congress*,
Anaheim, CA.

Johnson, W. W., Battiste, V., & Holland, S. (1999). A cockpit display
designed to enable limited flight deck separation responsibility.
*Proceedings of the 1999 World Aviation Conference*, Anaheim, CA.

Prevot, T. (2002). Exploring the many perspectives of distributed air
traffic management: The multi aircraft control system MACS.
*International Conference on Human-Computer Interaction in Aeronautics,
HCI-Aero 2002, 23-35 October*, MIT, Cambridge, MA.

Radio Technical Commission for Aeronautics (RTCA) (1995b). *Free flight
implementation* (RTCA Task-Force 3 report). Washington, D.C.: RTCA Inc.

Vu, K. L., Dion, M., Chambers, V., Ngo, T. K., Nelson, D., Kraft, J., &
Strybel, T. (this volume). *Pilot performance controlling multiple ROVs
in the terminal airspace and strategies for managing them*.

Wickens, C. D. & Carswell, C. M. (1995). The proximity compatibility
principle: Its psychological foundation and its relevance to display
design. *Human Factors*, 37(3), 473-494.

Woods, D. D. (1984). Visual momentum: A concept to improve the cognitive
coupling of person and computer. *International Journal of Man-Machine
Studies*, 21, 229-244.

#