bluuebunny/crossref_metadata_2025 huggingface dataset:
Behind the Scenes of NextGen: Describing the Impact of NextGen Operational Improvements on the Traffic Manager
Authors: Berry Katherine A. , Sawyer Michael W. , Austrian Edward M.
Published: October 2012

In an effort to modernize the National Airspace System (NAS), the Federal Aviation Administration (FAA) has introduced the Next Generation Air Transportation System (NextGen). The introduction of new capabilities and operational improvements (OI) offers the potential for changes to the roles and responsibilities of the traffic management unit (TMU). Each change presents the opportunity to positively or adversely impact the overall safety of the NAS. This paper applies a proactive approach to identifying potential human performance hazards associated with NextGen TMU operations. The TMU performance hazards identified were primarily related to the potential for selecting inadequate airspace and airport configurations. The potential effects of these hazards will directly affect the NextGen controller and thus represent rich opportunities for researchers to mitigate potential issues in the early stages of system design.
How Does Reliance on Automated Tools During Learning Influence Students’ Air Traffic Management Skills When the Tools Fail?
Authors: Vu Kim-Phuong L. , Silva Hector , Ziccardi Jason , Morgan Corey A. , Morales Gregory , Grigoleit Tristan , Lee Samuel , Kiken Ariana , Strybel Thomas Z. , Battiste Vernol
Published: October 2012

Research on potential NextGen technology has shown that advanced conflict detection and resolution tools can increase air traffic controllers’ performance and decrease their workload. However, use of NextGen tools can change the way controllers represent and manage their sector. Kraut et al. (2011) found that when NextGen tools failed in a simulated environment, experienced controllers were able to recover from the error and revert back to manual air traffic management techniques. One question is whether students would be able to recover from failures of technology if they were trained to rely on NextGen tools during acquisition of their air traffic management skills. To answer this question, we performed a simulation in which students were trained over 16 weeks to manage a sector consisting of both NextGen equipped and unequipped aircraft. Reliance on manual skills versus NextGen tools was induced by varying the percentage of equipped aircraft, being mostly NextGen equipped aircraft (75% vs. 25%) or mostly non-equipped aircraft during the first 8 weeks of learning. After the 16 weeks of training, participants were tested under nominal and failure conditions. Results showed that under nominal conditions, the training type was not an important factor. Instead, the percentage of equipped aircraft was the important factor: More equipped aircraft led to better performance and lower workload. However, when comparing the same scenario in which there was a failure of NextGen tools, the group trained to rely more on manual skills early in training performed better than the group trained to rely more on NextGen tools early in training. Implications of these findings are discussed.
Use a Tested Approach for Risk Management and Safety Enhancement: Maintenance Line Operations Safety Assessment (M-LOSA)
Authors: Ma Maggie J.
Published: October 2012

Managing risks has become increasingly important in modern organizations. The aviation industry is maturing in its preference for proactive intervention over post-accident remediation in potentially hazardous circumstances (Jones & Tesmer, 1999). Line Operations Safety Audits (LOSA) collects safety data during normal airline operations through peer observations in strict non-jeopardy conditions. Sponsored by the Federal Aviation Administration (FAA), a multi-year project was launched in October 2008 to capitalize on the successes of flight deck LOSA and extend it to aviation maintenance operations. The project delivered an array of open source tools, including observation data collection forms, procedures, implementation guideline, scenario-based training, and database. The Maintenance LOSA program is expected to positively leverage peer pressure to enforce behavior change and allow subunits of an organization to build in some flexibility to address their key problems and conquer them one at a time. The periodic assessment can help ensure that specific problems identified have been resolved, as well as assure the effectiveness of safety recommendations. The results of the understanding of the benefits and limitations of a threat and error recognition model will also benefit risk mitigation in other high-consequence industries.
Pilot’s Information Use During TCAS Events, and Relationship to Compliance to TCAS Resolution Advisories
Authors: Pritchett Amy R. , Fleming Elizabeth S. , Cleveland William P. , Popescu Vlad M. , Thakkar Dhruv A. , Zoetrum Jonathan J.
Published: October 2012

The Traffic alert and Collision Avoidance System (TCAS) is intended to serve as a backup, redundant system that alerts and advises the pilot when all other methods of aircraft separation assurance fail. A flight simulator experiment examined pilot responses to TCAS advisories in a full air traffic environment. This paper discusses two analyses of the pilots’ use of information within their environment. The first analysis examined air traffic communications manipulated according to four conditions: traffic call-outs, instructions conflicting with the TCAS advised avoidance maneuver, the ability to over-hear relevant party-line information, and no relevant communications. The second analysis used data from an eye tracker to identify when the pilot examined the traffic situation display provided by TCAS. These patterns of information use are then compared with pilot compliance to the Resolution Advisories provided by TCAS.
NextGen Flight Deck Data Comm: Auxiliary Synthetic Speech – Phase I
Authors: Lennertz Tracy , Bürki-Cohen Judith , Sparko Andrea L. , Macchiarella Nickolas , Kring Jason , Coman Mike , Haritos Tom , Alvarado Jeffry
Published: October 2012

Data Comm—a digital, text-based controller-pilot communication system—is critical to many NextGen improvements. With Data Comm, communication becomes a visual task. Although Data Comm brings many advantages, interacting with a visual display may yield an increase in head-down time, particularly for single-pilot operations. This study examined the feasibility of supplementing the visual Data Comm display with an auxiliary synthetic speech presentation. Thirty-two pilots flew two experimental scenarios in a Cessna 172 Flight Training Device. In one scenario, ATC communication was with a text-only Data Comm display; in the other, the text Data Comm display was supplemented with a synthetic speech display annunciating each message (i.e., text+speech). In both scenarios, pilots heard traffic with similar call signs on the party line and received a conditional clearance; however in just one scenario (counter-balanced between communication conditions), pilots received a clearance that was countermanded by a live controller before it was displayed. Results indicated that relative to the text-only display, the text+speech display aided single-pilot performance by reducing head-down time; and it may have prevented participants from acting prematurely on the conditional clearance. Supplementing text Data Comm with speech did not introduce additional complications: participants were neither more likely to erroneously respond to similar call signs, nor to ignore a live ATC voice countermand. The results suggest that the text+speech display did not hinder single-pilot performance and offered some benefits compared to the text-only display.
A Human Factors and Electromyographic Evaluation of Proposed Pointing Devices for Air Traffic Controllers
Authors: Higgins J. Stephen , Willems Ben , Johnson Daniel R. , Zingale Carolina M.
Published: October 2012

To ensure that the Next Generation Air Transportation System is efficient and safe, we conducted a simulation to explore the use of alternative pointing devices at en route air traffic control workstations. The current en route trackball supports limited interactions. However, speed, accuracy, interaction capabilities, and ergonomic issues of different pointing devices must be assessed to determine their effect on performance and musculoskeletal strain. Participants controlled simulated traffic with the current trackball, alternative trackball, hand-shake mouse, and a three button wheel-mouse. We collected surveys and recorded performance, behavior, and muscle activity data. Participants preferred the mouse and current trackball to the other devices. The air traffic measures we evaluated did not differ across any of the devices and there were no physiological indications that any of the devices had the potential to cause injury. The mouse was associated with greater pointing accuracy and speed than other devices. Based on these findings, we recommend using a mouse in future en route air traffic control systems.
UAS Integration into the NAS: An Examination of Baseline Compliance in the Current Airspace System
Authors: Fern Lisa , Kenny Caitlin A. , Shively Robert J. , Johnson Walter
Published: October 2012

As a result of the FAA Modernization and Reform Act of 2012, Unmanned Aerial Systems (UAS) are expected to be integrated into the National Airspace System (NAS) by 2015. Several human factors challenges need to be addressed before UAS can safely and routinely fly in the NAS with manned aircraft. Perhaps the most significant challenge is for the UAS to be non-disruptive to the air traffic management system. Another human factors challenge is how to provide UAS pilots with intuitive traffic information in order to support situation awareness (SA) of their airspace environment as well as a see-and-avoid capability comparable to manned aircraft so that a UAS pilot could safely maneuver the aircraft to maintain separation and collision avoidance if necessary. A simulation experiment was conducted to examine baseline compliance of UAS operations in the current airspace system. Researchers also examined the effects of introducing a Cockpit Situation Display (CSD) into a UAS Ground Control Station (GCS) on UAS pilot performance, workload and situation awareness while flying in a positively controlled sector. Pilots were tasked with conducting a highway patrol police mission with a Medium Altitude Long Endurance (MALE) UAS in L.A. Center airspace with two mission objectives: 1) to reroute the UAS when issued new instructions from their commander, and 2) to communicate with Air Traffic Control (ATC) to negotiate flight plan changes and respond to vectoring and altitude change instructions. Objective aircraft separation data, workload ratings, SA data, and subjective ratings regarding UAS operations in the NAS were collected. Results indicate that UAS pilots were able to comply appropriately with ATC instructions. In addition, the introduction of the CSD improved pilot SA and reduced workload associated with UAS and ATC interactions.
Eye Tracking-Based Target Designation in Simulated Close Range Air Combat
Authors: O’Connell Stephen D. , Castor Martin , Pousette Jerry , Krantz Martin
Published: October 2012

The ability to lock on targets as quickly and easily as possible while maintaining speed and a tactically advantageous position is crucial to success in air combat. The capability of remote eye tracking systems has recently improved significantly and has opened up new possible applications. A modern eye-tracker was integrated in an advanced flight simulator environment and an experiment was conducted. The independent variable was target designation mode that was varied between: Heads-Up Display (HUD), Helmet Mounted Display (HMD) and a solution based on eye tracking. Eleven participants flew in the experiment and mean times to designate targets indicate significant advantages for the eye tracking based solution.
The Design of the UAS Ground Control Station: Challenges and Solutions for Ensuring Safe Flight in Civilian Skies
Authors: Blickensderfer Beth , Buker Timothy J. , Luxion Stephen P. , Lyall Beth , Neville Kelly , Williams Kevin W.
Published: October 2012

Growing civil demands for uninhabited aircraft systems (UASs) are creating pressure to determine minimum design standards and required capabilities for UAS ground control stations (GCSs). This panel will discuss recent and ongoing research initiatives, regulatory guidance, and lessons learned from military UAS operations and mishap reports to shed light on the potential for GCS designs to support safe and non-disruptive UAS operations in the national airspace system (NAS). As part of this discussion, panelists will present GCS design challenges related to the preservation of NAS safety and traffic flow and will suggest strategies for overcoming them.
Human Factors at the Federal Aviation Administration (FAA): From Research to Reality
Authors: Yeh Michelle , Swider Cathy , Abbott Kathy , Donovan Colleen , Neiderman Eric , Piccione Dino
Published: October 2012

The purpose of this discussion panel is to inform human factors researchers and practitioners of the role of human factors throughout the Federal Aviation Administration (FAA) and how the FAA uses human factors research to enhance aviation safety and to support the performance of the national airspace system. Through this panel, the FAA hopes to provide outreach by communicating the Agency’s research needs and requirements to help researchers and practitioners facilitate the transition from research and development to tangible outcomes for the public. This panel will describe the role of the FAA Human Factors Division in coordinating human factors research at the Agency, the process for how research projects are initiated at the FAA, and the challenges faced in transitioning results to implementation in the field or in regulation. Two examples of research programs (Electronic Flight Bag and the Air Traffic Front Line Manager (FLM) Quick Reference Guide) that were successfully implemented are discussed.
Impact of Automation, Task and Context Features on Pilots’ Perception of Human-Automation Interaction
Authors: Mosier Kathleen , Fischer Ute ,   The HART Group*
Published: October 2012

Human-automation interaction (HAI) takes place in virtually every phase of flight under a variety of operational conditions. Pilots must decide when to engage and disengage automation, which automation to use, the extent to which they should monitor and crosscheck the automation, and so on. Currently, little is known about which characteristics of scenarios involving HAI are important in these decisions and precisely how they influence the interaction. The objective of the present research was to examine how systematic variations of automation characteristics, task features and context variables individually and jointly influence pilots’ perception of HAI situations. Commercial pilots received descriptions of aviation situations and were asked to judge the cognitive demands and predict behaviors associated with each HAI event. Results of the study support the notion that HAI comprises a complex interplay between features of the automation and elements of the task and context.
Analysis of the Risks and Benefits of Flight Deck Adaptive Systems
Authors: Dorneich Michael C. , Rogers William , Whitlow Stephen D. , DeMers Robert
Published: October 2012

The objectives of this work were to identify human performance risks and benefits of adaptive systems through a systematic analysis and heuristic evaluation of adaptive system component types and characteristics. As flight deck automated systems have more access to aircraft data, sensor data, stored databases, communicated information, and real time flight crew inputs, as well as more ability to process that information in sophisticated ways to identify situational priorities and context, it is becoming more realistic for those automated systems to adapt their behavior based on context. Automated systems that can make such changes on their own are called adaptive systems. The concern here is with adaptive systems that are perceived by the pilot to behave non-deterministically even though they are technically deterministic. Based on a framework to describe the types and characteristics of adaptive system components, a risk/benefit analysis was preformed to identify potential issues. Based on this analysis, eight representative adaptive system storyboards were developed as the basis of a heuristic evaluation with pilots to validate the analysis and explore more detailed issues and potential risk mitigations. The value of this work is expected to be in suggesting adaptive system issues, risks, and guidelines that need to be considered in making design decisions and approving new adaptive systems on the flight deck.
Method for Evaluating Data Communication Messages in NextGen Flight Deck Scenarios
Authors: Gallimore Jennie J. , Brent Kiss Steven , Munoz Ricardo D. , Oh Chang-Geun , Green Randall , Crory Timothy , Shingledecker Clark , Tsang Pamela S. , Herschler Dan
Published: October 2012

Data communications (DataComm) is one of the Federal Aviation Administration’s (FAA) key technologies supporting the transition to the Next Generation Air Transportation System (NextGen). As communication on the flight deck changes from voice clearances to clearances sent via datalink, human factors experts have anticipated potential implementation challenges, particularly for the data communications that may not be integrated with the flight management system (FMS) and which requires the flight crew to read the DataComm message, interpret it, and then make the appropriate FMS input. Even fully integrated DataComm and FMS flight deck systems could prove challenging for flight crew use in the Trajectory Based Operations context. For example, textual clearances that provide complex 4D trajectory information may be difficult for pilots to interpret in a timely and efficient manner without error. To address this challenge, flight deck displays with graphics, hybrid text and graphics, and other formats integrated with existing navigation displays may enable pilots to more easily identify, understand, and quickly respond to air traffic clearances and instructions. To support FAA Aircraft Certification Service and Flight Standards Service evaluation of new DataComm flight deck technologies and associated procedures for Trajectory Based Operations and other key NextGen applications, this paper describes a method that allows rapid evaluation of new and modified DataComm displays in the cockpit. Our method uses a variety of complex data communications, including the use of concatenated messages to obtain rapid pilot feedback and performance in a part-task, scenario-based simulation. Average data values from an initial study evaluating pilot interpretability measured as percent errors and response time to text-based clearances are presented.
Factors Affecting the Learning of a New Air Traffic Control Sector for Experienced Air Traffic Controllers
Authors: Cho Annie , Histon Jonathan
Published: October 2012

Developing generic airspace is one means of addressing the need for increased staffing flexibility as part of the Federal Aviation Administration’s (FAA’s) NextGen Air Traffic Control modernization efforts. Assessing the similarity of airspace sectors and identifying groups of sectors is a key step in enabling the generic airspace concept. In order to assess similarities of airspace sectors, factors affecting how quickly a controller makes a transition from one sector to another were identified using semi-structured interviews with experienced air traffic controllers. The most important factors appear to reflect familiarity with types of operations and common traffic patterns, providing a basis for classifying groups of sectors for the generic airspace concept.
Prediction of Traffic Complexity and Controller Workload in Mixed Equipage NextGen Environments
Authors: Lee Paul U. , Prevot Thomas
Published: October 2012

Controller workload is a key factor in limiting en route air traffic capacity. Past efforts to quantify and predict workload have resulted in identifying objective metrics that correlate well with subjective workload ratings during current air traffic control operations. Although these metrics provide a reasonable statistical fit to existing data, they do not provide a good mechanism for estimating controller workload for future air traffic concepts and environments that make different assumptions about automation, enabling technologies, and controller tasks. One such future environment is characterized by en route airspace with a mixture of aircraft equipped with and without Data Communications (Data Comm). In this environment, aircraft with Data Comm will impact controller workload less than aircraft requiring voice communication, altering the close correlation between aircraft count and controller workload that exists in current air traffic operations. This paper outlines a new trajectory-based complexity (TBX) calculation that was presented to controllers during a human-in-the-loop simulation. The results showed that TBX accurately estimated the workload in a mixed Data Comm equipage environment and the resulting complexity values were understood and readily interpreted by the controllers. The complexity was represented as a “modified aircraft count” that weighted different complexity factors and summed them in such a way that the controllers could effectively treat them as aircraft count. The factors were also relatively easy to tune without an extensive data set. The results showed that the TBX approach is well suited for presenting traffic complexity in future air traffic environments.
The Role of Mental Computations in Current and Future En Route Air Traffic Control
Authors: Murphy Elizabeth D. , Albert Harold A. , Chen Jennifer M. , Anderson Gregory G.
Published: October 2012

As air traffic control (ATC) becomes increasingly automated, software designers need to know how air traffic controllers process information as they manage operations in today’s system. Extracting knowledge from today’s controller workforce and representing that knowledge in the form of mental computations are essential steps toward needs assessment and development of advanced decision-aiding tools and technologies. A recent task analysis documented information derived by controllers from their cognitive integration of displayed information. This work envisions future, more detailed analyses of the controller’s mental computations as essential to identifying needs for advanced software tools, including predictive displays.
Influence of Team Communication and Coordination on the Performance of Teams at the iCTF Competition
Authors: Jariwala Shree , Champion Michael , Rajivan Prashanth , Cooke Nancy J.
Published: October 2012

Effective team process is critical for the performance of cyber security teams. To examine this, we observed two comparably skilled cyber security teams participating in the International Capture the Flag (iCTF) competition held in December 2011. At the conclusion of the competition, we followed up with a focus group discussion with six members from the two teams. In this paper, we present our findings from the focus group interviews, on the relationship between team level factors and team performance. Findings from the focus group discussion indicate that team level factors such as team communication, coordination, team structure, and leadership play important roles in team performance.
Recurrence Quantification Analysis Used to Assess Team Communication in Simulated Air Battle Management
Authors: Russell Sheldon M. , Funke Gregory J. , Knott Benjamin A. , Strang Adam J.
Published: October 2012

Nonlinear time-series measures provide an opportunity to examine temporal structure in team communications. In this experiment, Recurrence Quantification Analysis (RQA), a nonlinear measure that quantifies visual patterns in recurrence plots, was applied to verbal communication utterances produced in a 5-person air defense simulation. Communication utterances were analyzed with three coding schemes – team position, semantic content, and combined. No differences in RQA percent determinism (%DET) driven by experimental manipulations were detected for communications coded by team position, but differences were detected for semantic content and combined. This implies that a higher resolution of coding was necessary to detect manipulated effects via RQA in this experiment. Specific changes in %DET, when paired with findings from a more traditional communication analysis, i.e., cumulative frequency, indicated positive compensatory adaptations in team communication when faced with training and task constraints. Results are discussed with emphasis on changes in %DET as a result of current experimental manipulations and integration with previous research.
Examining Temporal Regularity in Categorical Team Communication Using Sample Entropy
Authors: Strang Adam J. , Horwood Sharon , Best Christopher , Funke Gregory J. , Knott Benjamin A. , Russell Sheldon M.
Published: October 2012

Communication is inherent to team coordination and performance. Nonlinear time-series measures, such as Sample Entropy (SEn), provide the opportunity to examine the temporal structure of team communication. The aim for this experiment was to develop a method for applying SEn to a set of categorically coded and sequential team verbal communications recorded during a dyadic Air Battle Management (ABM) simulation. Results showed that deterministic temporal regularity was detected in team communication for three categorical coding schemes applied to the data – team role, semantic content, and combined. For data coded for semantic content, SEn was able to detect an increase in communication temporal regularity for teams exposed to high workload. Practical and theoretical implications are considered.
Emotion Expression under Stress in Instant Messaging
Authors: Pirzadeh Afarin , Pfaff Mark S.
Published: October 2012

In spite of rapid growth of text-based instant messaging (IM) in very diverse settings, emotion expression in IM has received limited empirical scrutiny, especially outside of casual settings. The main focus of this study is how stress and personality traits affect verbal and nonverbal cues used in IM to express emotions during a simulated team decision-making task. This exploratory study analyzed a preexisting set of 168 chat logs from an experiment on team decision making under stress. Results of the analysis revealed significant differences in the frequency of several verbal and nonverbal cues as a result of manipulations of mood (happy and sad) and stress (time pressure). In addition, several correlations were found between cue types and three personality traits: extraversion, openness to experience, and locus of control. Understanding how users express emotions in IM helps researchers and designers to support the user’s emotional needs, resulting in the improvement of emotional communication strategies in IM.
Perception of Meta-Information Representation: A Psychophysical Approach
Authors: Fortenbery Nicholas , Jenkins Michael P. , Bisantz Ann M. , D’Arcy Jean-François , Farry Michael , Nagy Allen , Roth Emilie , Pfautz Jonathan , Thomas Gina
Published: October 2012

Previous research has identified many effective methods to visualize different types of meta-information, or information qualifiers; however, these methods are often incorporated without understanding how the graphical codes are perceived and how the encoded information is interpreted by display users. This results in display designers selecting graphical codes to represent meta-information without empirical evidence to determine the appropriateness of these selections. To help address this lack of guidance, this paper presents a systematic study of how people perceive two graphical codes (saturation and opacity) and relate those codes to different types of meta-information. Results were generated using psychophysical scaling methods, and provide visualization designers with a means to more appropriately design meta-information representations.
Evaluating a Multimodal Interface for Firefighting Rescue Tasks
Authors: Streefkerk Jan Willem , Vos Wouter , Smets Nanja
Published: October 2012

Firefighters searching for victims work in hazardous environments with limited visibility, obstacles and uncertain navigation paths. In rescue tasks, extra sensor information from infrared cameras, indoor radar and gas sensors could improve vision, orientation and navigation. A visual and tactile interface concept is proposed that integrates this sensor information and presents it on a head-mounted display and tactile belt. Sixteen trained participants performed a firefighting rescue task with and without the prototype interface, measuring task performance, mental effort, orientation and preference. We found no difference in task performance or orientation, but a significantly higher preference for the prototype compared to baseline. From participants’ remarks, it appears the interface overloaded them with information, reducing the potential benefit for orientation and performance. Implications for the design of the prototype are outlined.
Requirements and Design for Better Cultural Situation Awareness: Delivering the Right Information
Authors: Onal Emrah , Endsley Mica R.
Published: October 2012

Over the past decade, irregular warfare has dominated the focus for U.S. military forces, requiring interaction with civilian populations for extended periods of time. Effective understanding of the impact of cultural factors and maintaining higher Cultural Situation Awareness has become imperative for success in the emerging environment. Based on knowledge elicitation sessions with hundreds of Soldiers across a variety of military occupation codes, and hands-on research and design experience, this paper examines the application of situation awareness theory to help build, improve, and maintain higher Cultural Situation Awareness from requirements to design.
A Comparison of Visualization and Command-Based Decision Aiding in a Simulated Aircraft Departure Sequencing Task
Authors: Riddle Kenyon , Kirlik Alex , Talleur Donald , Carbonari Ronald , Zhang Yijing , Holbrook Jon , Byrne Michael , Bauer David , Beard Bettina
Published: October 2012

Three versions of a prototype ground controller interface were tested for their effect on aircraft departure sequencing in a simulation of Dallas/Forth-Worth International Airport. Two versions featured automated decision aids and a third served as a baseline with no decision aid. The two decision aids performed fundamentally different functions, with a Temporal Constraint Visualization (TCV) providing a visualization of spatiotemporal constraints on the departure sequence and a Timeline display providing release sequences derived by an optimization algorithm. Results indicate that participants in the TCV condition had more efficient departure sequences than the Baseline and Timeline conditions. No significant differences were found between conditions for timeliness of departures or handling of arrival aircraft. These results indicate that, through the use of appropriate decision aids, task performance in complex dynamic environments can be improved with humans retaining full decision making control. Additional research is warranted to investigate situation awareness and failure mode performance using these decision aid types.
Visualizations and Interaction Methods for Resilient Submarine Decision Support
Authors: Stark Robert F. , Woods David D. , Farry Michael , Morison Alex , Thornton Wayne , Wollocko Arthur
Published: October 2012

Submarine commanders must make decisions rapidly to carry out increasingly complex missions. However, the rate of information delivery has outpaced the capacity of the command and control systems that prioritize and filter it. Technology could help commanders filter through data to make decisions, but this decision support must be carefully engineered to support the development of resilient courses of action (COAs). This paper details our experiences applying resilience engineering to submarine decision support. It focuses on designing two features that are essential for a resilient decision support system: (1) user interaction with a decision support system, which blends the user’s operational insights with the technical aspects of the decision support system; and (2) visual representations of trade-offs. The paper ends with a discussion of the lessons learned from this work and a set of recommendations for designing decision support systems.
Social norms and their impact on safety-related rule violations in process control: Does it make a difference if operators are aware that residents will be injured?
Authors: von der Heyde Ananda , Presting Palle , Kluge Annette , Badura Björn
Published: October 2012

As the avoidance of safety-related rule violations is an important concern in high reliability organizations, it is essential to understand the decision-making process that determines rule violating behavior. As a theoretical background, the Integrated Model of Behavior Prediction (IM) is used to design two studies in which the impact of different “goods at stake” (injured residents versus damaged plant) on safety-related rule violations was investigated. An online prestudy was conducted to select two appropriate scenarios out of three with which the “goods at stake” were operationalized followed by an experiment to determine the effect of the “goods at stake” on safety-related rule violations. “Goods at stake” were operationalized in terms of a) a damaged plant versus b) 20 injured residents as a consequence of a deflagration caused by a safety-related rule violation. Although the prestudy showed that subjects are less likely to violate a safety-related rule in the case of injured residents, this result was not replicated by the main experiment. Safety-related rule violations were equally likely in the case of residents injured and plant damage. Post hoc analyses showed that person-related variables also addressed by the IM were significantly related to the decision of a violation, such as skills and abilities.
Operators’ Adaption to Unreliability of Alarm Systems: A Performance and Eye-Tracking Analysis
Authors: Onnasch Linda , Ruff Stefan , Manzey Dietrich
Published: October 2012

Operators in complex environments are supported by alarm-systems that indicate when to shift attention to certain tasks. As alarms are not perfectly reliable, operators have to select appropriate strategies of attention allocation in order to compensate for unreliability and maintain overall performance. This study investigates how humans adapt to differing alarm-reliabilities. Within a multi-tasking flight simulation, participants were randomly assigned to four alarm-reliability conditions (68.75%, 75%, 87.5%, 93.75%), and a manual control group. In experimental conditions, one out of three subtasks was supported by an alarm-system. Compared to manual control, all experimental groups benefited from alarms in the supported task, with best results for the highest reliability condition. However, analyses of performance and eye-tracking data revealed that the benefit of the lowest reliability group was associated with an increased attentional effort, a more demanding attention allocation strategy, and a declined relative performance in a non-supported task. Results are discussed in the context of recent research.
On the Relation Between Reliance and Compliance in an Aided Visual Scanning Task
Authors: Wiczorek Rebecca , Meyer Joachim , Guenzler Torsten
Published: October 2012

Alarms, alerts, and other binary cues affect user behavior in complex ways. One relevant distinction is the suggestion that there are two different responses to alerts – compliance (the tendency to perform an action cued by the alert) and reliance (the tendency to refrain from actions as long as no alert is issued). An experiment tested the dependence of the two behaviors on the Positive and Negative Predictive Values of the alerts (PPV and NPV) to determine whether these are indeed two different behaviors. Results suggest that the compliance is relatively stable and unaffected by irrelevant information (the NPV), while reliance is also affected by the PPV. The results are discussed in terms of multiple-process theories of trust in information sources.
Decision-Making and Risk-Taking Behavior in Lunar Landing
Authors: Wen H. Y. , Johnson A. W. , Duda K. R. , Oman C. M. , Natapoff A.
Published: October 2012

Model-based simulation and human subject experiments can be used to develop quantitative methods for analyzing the human-automation task allocation of a system early in the design process. An integral part of the human-system model is a representation of human decision-making and risk-taking behavior. These behaviors were investigated in a lunar landing human subject experiment. Subjects were asked to select a landing aim point that was near both a point of interest and hazardous region. It was expected that the placement of the landing aim point would vary with the probability of manual versus automatic flight and whether estimated touchdown dispersions were remembered by the subjects from earlier in the experiment or presented graphically on scatter plots. The experiment found that subjects did systematically modify the placement of landing aim points. Further, presenting landing dispersions graphically allowed subjects to compensate for touchdown deviations in both risk-critical and non-risk critical directions.
Perspectives on the Role of Cognition in Cyber Security
Authors: McNeese Michael , Cooke Nancy J. , D’Amico Anita , Endsley Mica R. , Gonzalez Cleotilde , Roth Emilie , Salas Eduardo
Published: October 2012

The cyber security task is an intensely cognitive task that is embedded in a large multi-layered sociotechnical system of analysts, computers, and networks. Effective performance in this world is hampered by enormous size and complexity of the network data, the adaptive nature of intelligent adversaries, the lack of ground truth to assess performance, the high number of false alarms presented by automated alerting systems, by organizational stove pipes thwarting collaboration, and by technology that is thrown at the problem without an adequate understanding of the human needs. Further, the consequences of effective system performance in the cyber security domain are unparalleled because our world is so dependent on its cyber infrastructure. We have assembled a panel of six experts in cognitive engineering to provide perspectives on the cyber security problem and promising solutions.
Designing Collaborative Automated Planners for Agile Adaptation to Dynamic Change
Authors: Truxler Robert , Roth Emilie , Scott Ronald , Smith Stephen , Wampler Jeffrey
Published: October 2012

A common characteristic of domains that require planning for allocation of scarce resources is the need to dynamically revise the plan as new requirements emerge and priorities change. We describe a prototype decision support system for planning and scheduling airlift that we developed for a military transport organization that enables agile plan adaptation as movement requirements, available airlift assets, and priorities change. The collaborative automated scheduler includes visualizations to foster improved situation awareness of available airlift assets versus total demand on those assets; mechanisms to enable users to communicate informal priorities and changes to those priorities; and mechanisms that enable users to explore alternative scheduling options in response to changes in movement requirements, priorities and available assets. A formal user evaluation study that included 12 participants representing three different organizational groups involved in transportation planning provided evidence that the prototype improves the ability to capture and communicate movement priorities, rapidly reallocate airlift assets to accommodate changes, and communicate/collaborate across organizational boundaries in managing airlift demand versus capacity.
Differences in Macrocognition Strategies With Face to Face and Distributed Teams
Authors: Patterson Emily S. , Bernal Fernando , Stephens Robert
Published: October 2012

Deciding is an important macrocognition function for small teams performing overconstrained tasks. It is unclear how macro cognition strategies differ for small teams that work in a traditional face to face configuration as opposed to a physically distributed configuration. In this study, a process tracing analysis was conducted to identify differences in deciding strategies for face to face as compared to distributed three-person teams executing a military logistics planning task. Deciding strategies employed by emergent leaders were found to include conversation dominance, using the identification of a plan for submission as the work stopping rule, and strengthening and weakening commitment to a solution by ruling out options, adding new factors for consideration, and delaying commitment to any solution. Distributed teams used one strategy significantly more often than face to face teams: ruling out options. Implications are discussed.
Black Holes, Keyholes And Brown Worms: Challenges In Sense Making
Authors: Wong B.L. William , Varga Margaret
Published: October 2012

We address the problems faced by analysts who have to sift through large amounts of data quickly and accurately in order to make sense of the information contained within the data or the circumstance represented in the data. We discuss the approach we have taken to visual analytics from the perspective of the Data-Frame Theory of Sense-making and its extension to Causal Reasoning, and how the cognitive strategies that are invoked in these processes need to be supported. We identify 20 problems that designers of visual analytics-type systems need to address in order to support sense-making. In particular, we discuss design issues associated with three exemplar problems: (i) Black holes - the problem of representing missing data; (ii) Keyholes - the problem of being able to access and view only a small part of a large dataset or only part of a problem; and (iii) Brown worms - the problem of dealing with and representing misleading or deceptive data.
Use of an Option Generation Paradigm to Investigate Situation Assessment and Response Selection in Law Enforcement
Authors: Suss Joel , Ward Paul
Published: October 2012

When individuals make decisions in the natural ecology, generation and selection of a course of action is informed by their assessment of the situation. Previous option generation research—largely using complex but static tasks—has examined, separately, the decision strategies employed during the situation assessment and response phases of decision making. This research found that decision makers typically generate a small number of options, and their first option is generally a good one. In dynamic tasks, however, skilled performance involves not only comprehension of the current situation, but the ability to predict impending events. The goal of the current study is to test existing claims about the option generation strategies employed during the situation assessment and response phases of decision making, in the context of a dynamic task.
Origins to Destinations: Looking Forward to the Next 30 Years of Cognitive Systems Engineering
Authors: Pritchett Amy R. , Nemeth Christopher , Feigh Karen , Feary Michael , Pfautz Jonathan D. , Weil Shawn
Published: October 2012

This panel will discuss emerging areas in cognitive engineering from several researchers and practitioners who are defining their own research directions. The panelists will be challenged to discuss what they see as the design challenges that cognitive engineering is uniquely suited to answer. Further, they will be asked what grand challenges cognitive engineering should work toward solving, whether it is in terms of specific design problems or broad capabilities for modeling and understanding complex work domains.
Coherence and Correspondence Competence: Implications for Elicitation and Aggregation of Probabilistic Forecasts of World Events
Authors: Tsai Jennifer , Kirlik Alex
Published: October 2012

One potentially useful concept that arises in the elicitation and aggregation of probabilistic forecasts is Hammond’s (1996) distinction between coherence and correspondence. A study was conducted to test the commonly held assumption that coherence competency, a judge’s ability to reason correctly according to the prescriptions demanded by the problem, directly yields correspondence competency, a judge’s ability to predict the outcome that actually happens in the external world. The role of a visualization aid in terms of moderating these effects was also examined. Participants who were knowledgeable baseball fans predicted the probability with which their favored team would win the 2011 Major League Baseball World Series, giving a prior probability shortly before the start of the Series, and then sequentially updating their answer as the individual games unfolded over time. Results show that for participants using the visualization, their ability to update probabilities according to the dictates of Bayes’ Theorem was correlated with their ability to predict the winner of the 2011 MLB Series – a desirable property that allows for estimation of judges’ outcome performance based on more readily available process information.
Exploring the Relationship Between Topic Area Knowledge and Forecasting Performance
Authors: Miller Sarah M. , Forlines Clifton , Regan John
Published: October 2012

The Intelligence Community (IC) is often asked to make predictions about future world events. One aspect of predicting the quality of forecasts and forecasters is the knowledge that the forecaster has about the question to be forecast. This paper explores the relationship between factual knowledge about a forecast event and eventual performance on a forecast question. The results demonstrated a significant relationship between a forecaster answering a series of factual questions correctly and answering the corresponding forecast question correctly. This relationship is enhanced when controlling for the relative difficulty of the factual question. When controlling for forecaster performance, roughly half of the impact was due to general forecaster performance and half was due to their specific knowledge about a given forecast question. Interestingly, we found that forecasters with more factual knowledge were less calibrated with respect to their probability forecast whereas forecasters who were less knowledgeable were better calibrated in their probability estimates. We discuss the implication of the results related to improving forecast quality.
An Integrated Cognitive Architecture for Cognitive Engineering Applications
Authors: Cao Shi , Liu Yili
Published: October 2012

The increasing complexity of computational cognitive architectures may increase both their modeling capabilities and their difficulty to learn and use as cognitive engineering tools. This paper reports our work dedicated to enhance the usability and the cognitive engineering applicability of a complex computational cognitive architecture called QN-ACTR, which integrates two complementary architectures Queueing Network and Adaptive Control of Thought-Rational. The aim is to provide an easy-to-use interface and intuitive modeling that support both inexperienced and experienced users in using this complex and powerful architecture. The process of model development is greatly simplified with improved visualization and validation methods. The results were examined using heuristic evaluation. The benefits and practice implications are discussed.