FIGURES AND TABLES 

Angles

The Angles test measures the participant’s ability to recognize angles. This test contains 30

multiple-choice questions and allows participants up to 8 minutes to complete them. The

score is based on the number of correct answers (with no penalty for wrong or unanswered

questions). There are two types of questions on the test. The first presents a picture of an

angle and the participant chooses the correct answer of the angle (in degrees) from among 4

response options. The second presents a measure in degrees and the participant chooses the

angle (among 4 response options) that represents that measure.

For each worker requirement listed below, enter the rating best describing the extent to which

this test and/or its subtests measure that particular worker requirement.

5=This test measures this worker requirement to a very great extent

4=This test measures this worker requirement to a considerable extent

3=This test measures this worker requirement to a moderate extent

2=This test measures this worker requirement to a limited extent

1=This test measures this worker requirement to a slight extent

0=This test does not measure this worker requirement

Figure 2.1. Sample Description of an AT-SAT Measure.

Prioritization Self-Awareness Commitment to the Job

Tolerance for High Intensity Rule Application Self-Esteem

Composure Perceptual Speed & Accuracy Translation of Uncertainty

Active Listening Sustained Attention Translating Information

Oral Communication Self-Monitoring/Evaluation Behavioral Consistency

Situational Awareness Creativity Encoding

Planning Working Cooperatively Movement Detection

Execution Recall from Interruption Interpersonal Tolerance

Thinking Ahead Self-Confidence Motivation

Taking Charge Internal Locus of Control Chunking

Reasoning Street Physics Interpreting Information

Time Sharing Task Closure/Thoroughness Mathematical Reasoning

Decisiveness Summarizing Information Written Communication

Short-Term Memory Intermediate-Term Memory Mechanical Reasoning

Scanning Visuospatial Reasoning Angles

Problem Solving Flexibility (IP) Rule Inference

Flexibility (S/A) Dynamic Visual-Spatial Realistic Orientation

Long-Term Memory Professionalism 2-D Mental Rotation

Projection Attention to Detail Numeric Ability (add/sub)

Visualization Verbal Reasoning 3-D Mental Rotation

Concentration Reading Numeric Ability (mult/div)

Confirmation Learning

Figure 2.2. Example of Linkage Rating Scale.

61
FIGURES AND TABLES 

Figure 3.3.1. Plot of PRACCY*PRSPEED. Symbol is value of TRIAL

62
FIGURES AND TABLES 

Table 2.1. SACHA-Generated Worker Requirements

COGNITIVE ABILITIES

REASONING

1. Deductive Reasoning

2. Inductive Reasoning

3. Chunking

4. Mathematical Reasoning

5. Visuospatial Reasoning

6. Mechanical Reasoning

7. Verbal Reasoning

COMPUTATIONAL ABILITY

8. Number Facility

9. Geometry

COMMUNICATION

10. Oral Communication

11. Listening

12. Reading

13. Writing

ATTENTION

14. Selective Attention

15. Time Sharing

16. Vigilance

MEMORY

17. Short-Term Memory

18. Intermediate-Term Memory

19. Long-Term Memory

20. Associative Memory

21. Visual Memory

22. Auditory Memory

23. Numerical Memory

METACOGNITIVE

Local

24. Thinking Ahead

25. Problem Recognition and Definition

26. Deciding on the Steps to solve the Problem

27. Ordering the Steps

28. Creating a Mental Representation for Information

29. Monitoring Performance

30. Evaluating Performance

Global

31. Prioritization

32. Multitasking

(Continued)

63
FIGURES AND TABLES 

Table 2.1. SACHA-Generated Worker Requirements (continued)

INFORMATION PROCESSING

33. Encoding

34. Rule Inference

35. Rule Application

36. Confirmation

37. Sequential Distribution of Processing Steps

38. Information Processing Strategies

39. Learning

40. Flexibility

PERCEPTUAL/SPATIAL ABILITIES

PERCEPTUAL ABILITIES

41. Dynamic Visual-Spatial Ability

42. Movement Detection

43. Verbal and Figural Closure

44. Perceptual Speed and Accuracy

SPATIAL ABILITIES

45. Projection

Flexibility of Closure

47. Scanning Efficiency

48. Two-Dimensional Mental Rotation

49. Three-Dimensional Mental Rotation

50. Spatial Scanning

TEMPERAMENT/INTERPERSONAL

INTERPERSONAL

51. Working Cooperatively

52. Establishing Rapport

WORK & EFFORT

53. Achievement/Self-Confidence

54. General Activity

55. Concentration

56. Taking Charge

57. Adherence to a Work Ethic

58. Thoroughness and Attentiveness to Details

59. Thinking Ahead

60. Decisiveness

61. Realistic Orientation

STABILITY/ADJUSTMENT

62. Stress Tolerance

63. Flexibility

SELF-EFFICACY

64. Internal Locus of Control

65. Self-Sufficiency

PSYCHOMOTOR ABILITIES

66. Multilimb Coordination

67. Control Precision

68. Reaction Time

69. Response Orientation

70. Finger Dexterity

71. Manual Dexterity

72. Eye-Hand Coordination

73. Response Integration

64
FIGURES AND TABLES 

Table 2.2. Worker Requirements Generated by Subject Matter Experts

COGNITIVE

1. Oral Communication

2. Written Communication

3. Reading

4. Active Listening

5. Translating Information

6. Interpreting Information

7. Summarizing Information

8. Numeric Ability (addition/subtraction)

9. Numeric Ability (multiplication/division)

10. “Street” Physics

11. Short-Term Memory

12. Intermediate-Term Memory

13. Long-Term Memory

14. Thinking Ahead

15. Planning

16. Prioritization

17. Execution

18. Problem Solving

19. Reasoning (Thinking)

20. Creativity

21. Self-Monitoring/Evaluating

22. Rule Application

23. Sustained Attention

24. Time Sharing

25. Aviation Science Background

26. Geography

SPATIAL

27. Situational Awareness

28. Visualization

29. Projection

30. Scanning

INTERPERSONAL

31. Professionalism

32. Working Cooperatively

33. Personal Tolerance

WORK/EFFORT

34. Self-Esteem

35. Self-Confidence

36. Aggressiveness

37. Self-Awareness

38. Attention to Detail

39. Task Closure/Thoroughness

40. Decisiveness

41. Consistency

42. Flexibility

43. Concentration

44. Composure

45. Tolerance for High-Intensity Work

46. Motivation

47. Commitment to Job

65
FIGURES AND TABLES 

Table 2.3. Revised Consolidated Worker Requirements List, With Definitions

SME-Generated WORKER REQUIREMENTS

Oral Communication The ability to speak clearly and concisely to individuals so they understand what

is being communicated. Projecting a confident tone of voice is an important

component of this ability.

Written Communication The ability to write legibly and accurately (e.g., strip markings).

Reading The ability to read and understand written information (e.g., ATCS documents,

manuals).

Active Listening The ability to hear and comprehend spoken information. This ability requires an

individual to recognize or pick out pertinent auditory information.

Translating Information The ability to translate symbols or symbolic abbreviations into meaningful

information.

Interpreting Information The ability to put information into meaningful terms. It is the ability to

recognize the implications for a statement or condition (e.g., cold front).

Summarizing Information The ability to summarize and consolidate information most relevant to the

situation.

Numeric Ability (add/sub) The ability to quickly and accurately perform basic math operations (addition

and subtraction).

Numeric Ability (mult/div) The ability to quickly and accurately perform basic math operations

(multiplication and division).

Street Physics The ability to understand the general rules of science as practiced in the ATCS

environment (e.g., aircraft size and turbulence effects, angles of intersect,

overtake speed, headings).

Short-Term Memory The ability to remember pertinent information within a brief period of time (less

than 1 minute). Examples of information include call signs and keywords.

Intermediate-Term Memory The ability to remember pertinent information over a 1-10 minute period.

Long-Term Memory The ability to remember pertinent information over long periods of time.

Examples of information include maps and separation procedures.

Thinking Ahead The ability to anticipate or recognize problems before they occur and to develop

plans to avoid problems. This includes thinking about what might happen.

Planning The ability to determine the appropriate course(s) of action to take in any given

situation.

Prioritization The ability to identify activities that are most critical and require immediate

attention. This involves a constant evaluation of new information followed by

re-prioritization of job activities.

Execution The ability to take timely action in order to avoid problems and to solve existing

problems.

Problem Solving The ability to identify a potential problem or existing problem and to identify

the variables used in solving the problem.

Reasoning The ability to apply available information to make decisions, draw conclusions,

or identify alternative solutions.

Creativity The ability to identify new or novel solutions to potential problems when

existing or established solutions no longer applies.

Self-Monitoring/ Evaluation The ability and willingness to check your own work performance, evaluate the

effectiveness of your decisions, and alter your performance if necessary.

Rule Application The ability to apply learned rules to the real work situation

(Continued)

66
FIGURES AND TABLES 

Table 2.3. Revised Consolidated Worker Requirements List, With Definitions (Continued)

Sustained Attention The ability to stay focused on a task(s) for long periods of time (over 60

minutes).

Time Sharing The ability to perform two or more job activities at the same time.

Situational Awareness Being cognizant of all information within a four-dimensional space (i.e.,

separation standards plus time). This involves the ability to “understand” the

airspace as an integrated whole (e.g., getting the picture).

Visualization The ability to translate material into a visual representation of what is currently

occurring.

Projection The ability to translate material into visual representation of what will occur in

the future.

Scanning The ability to quickly and accurately search for information on a computer

screen, radar scope, or computer printout.

Professionalism The ability to establish respect and confidence in your abilities among other

controllers.

Working Cooperatively The willingness to work with others to achieve a common goal. This includes a

willingness to voluntarily assist another controller if the situation warrants.

Interpersonal Tolerance The ability to accommodate or deal with differences in personalities, criticisms,

and interpersonal conflicts in the work environment.

Self-Esteem Having a positive opinion/image of oneself.

Self-Confidence A belief that you are the person for the job and knowing that your processes and

decisions are correct.

Taking Charge The ability to take control of a situation and reach out and take correct action.

Self-Awareness The ability to maintain an internal awareness of your actions and attitudes. This

includes knowing your limitations.

Attention to Detail The ability to recognize and attend to the details of the job that others might

overlook.

Task Closure/ Thoroughness The ability to continue an activity to completion through the coordination and

inspection of work.

Decisiveness The ability to make effective decisions in a timely manner.

Behavioral Consistency The ability to behave consistently at work (e.g., dealing with coworkers in a

consistent manner; consistently using the correct phraseology).

Flexibility (Stability/

Adjustment)

The ability to adapt to changing situations or conditions.

Concentration The ability to focus on job activities amid distractions for short periods of time.

Composure The ability to think clearly in stressful situations.

Tolerance for High Intensity The ability to perform effectively and think clearly during heavy work flow.

Motivation The desire to motivate oneself through challenges on the job and to progress to a

higher level of skill.

Commitment to the Job The desire to be an ATCS and work hard to be successful.

(Continued)

67
FIGURES AND TABLES 

Table 2.3. Revised Consolidated Worker Requirements List, With Definitions (Continued)

SACHA-Generated WORKER REQUIREMENTS

Confirmation The ability to efficiently select a response option consistent with the application

of inferred rules.

Rule Application The ability to efficiently apply transformational rules inferred from the complete

portions of the stimulus array to the incomplete portion of the array.

Perceptual Speed and

Accuracy

Ability to perceive visual information quickly and accurately and to perform

simple processing tasks with (e.g., comparisons).

Internal Locus of Control Believes that individuals have influence over the outcome of an event; takes

responsibility for outcomes.

Visuospatial Reasoning Ability to perceive and understand principles governing relationships among

several figures.

Flexibility (Information

Processing)

The ability to find new meanings for stimuli, to combine stimulus attributes to

come up with new and different solution protocols, and to employ flexible ways

of relating new information to stored knowledge.

Dynamic Visual-Spatial Ability to deal with dynamic visual movement.

Verbal Reasoning The ability to perceive and understand principles governing the use of verbal

concepts and symbols.

Learning Changes in information-processing strategies over time or trials not due to

maturation or aging.

Encoding Transformation or translation of information; coding; decoding.

Movement Detection Ability to detect physical movement of objects and to judge their direction.

Chunking The ability to organize stimuli into meaningful groups or units.

Mathematical Reasoning Ability to perceive and understand principles governing the use of quantitative

concepts and symbols.

Mechanical Reasoning Ability to perceive and understand the relationship of physical forces and

mechanical elements in a prescribed situation.

Angles The ability to apply the principles of geometry to angles and computations

involving angles. The ability involves both the speed and accuracy of

computation.

Rule Inference The ability to efficiently ascertain the rules governing relations between

stimulus attributes.

Realistic Orientation Prefers dealing with activities which have tangible and measurable

consequences; enjoys activities which require skill, is reinforced by

accomplishing realistic tasks.

2-D Mental Rotation Ability to identify a two-dimensional figure when seen at different angular

orientations.

3-D Mental Rotation Ability to identify a three-dimensional object when seen at different angular

orientations either within the picture plane or about the axis in depth.

OTHER WORKER REQUIREMENTS

Translation of Uncertainty The ability to assign a subjective probability regarding the likelihood of an event

occurring; the ability to use probabilities to identify optimal courses of action

(CTA, 1988).

Recall from Interruption The ability to recall a deferred or interrupted action when priorities permit, and

to be able to resume the action appropriately (Ammerman et al., 1983)

68
FIGURES AND TABLES 

Table 2.4. Mean Worker Requirement Ratings Rank Ordered for all ATCSs

Learning the Job Doing the Job

MEAN SD LABEL MEAN SD LABEL

4.43 0.71 Active Listening 4.60 0.62 Tolerance for High Intensity

4.32 0.79 Tolerance for High Intensity 4.56 0.63 Oral Communication

4.30 0.83 Prioritization 4.53 0.66 Active Listening

4.27 0.80 Composure 4.53 0.70 Prioritization

4.25 0.87 Oral Communication 4.50 0.70 Composure

4.21 0.84 Planning 4.45 0.69 Planning

4.19 0.84 Rule Application 4.40 0.71 Decisiveness

4.18 0.85 Execution 4.37 0.80 Execution

4.16 0.85 Short-Term Memory 4.35 0.81 Thinking Ahead

4.15 0.86 Reading 4.33 0.86 Situational Awareness

4.15 0.82 Decisiveness 4.33 0.79 Scanning

4.13 0.82 Reasoning 4.31 0.74 Reasoning

4.12 0.92 Thinking Ahead 4.30 0.80 Short-Term Memory

4.12 0.90 Scanning 4.30 0.73 Taking Charge

4.11 0.88 Long-Term memory 4.26 0.78 Time Sharing

4.08 0.93 Situational Awareness 4.26 0.85 Visualization

4.06 0.91 Visualization 4.26 0.84 Projection

4.06 0.83 Concentration 4.25 0.79 Rule Application

4.05 0.85 Problem Identification 4.23 0.83 Problem Identification

4.05 0.91 Projection 4.23 0.80 Concentration

4.04 0.83 Flexibility (S/A) 4.22 0.77 Flexibility (S/A)

4.02 0.92 Taking Charge 4.13 0.83 Perceptual Speed and Accuracy

4.01 0.91 Time Sharing 4.11 0.92 Long-Term Memory

3.99 0.87 Perceptual Speed and Accuracy 4.11 0.78 Self-Awareness

3.98 0.92 Commitment to the Job 4.10 0.81 Working Cooperatively

3.91 0.86 Summarizing Information 4.07 0.94 Sustained Attention

3.90 0.90 Self-Awareness 4.06 0.83 Summarizing Information

3.89 0.93 Self-Monitoring/Evaluating 4.04 0.84 Self-Monitoring/Evaluating

3.87 0.92 Translating Information 4.02 0.78 Self-Confidence

3.87 0.87 Intermediate-Term Memory 3.96 0.88 Intermediate-Term Memory

3.87 0.96 Sustained Attention 3.96 0.85 Task Closure/Thoroughness

3.87 0.86 Self-Confidence 3.95 0.89 Reading

3.86 0.87 Task Closure/Thoroughness 3.92 0.90 Creativity

3.82 0.96 Working Cooperatively 3.92 0.90 Recall From Interruption

(Continued)

69
FIGURES AND TABLES 

Table 2.4. Mean Worker Requirement Ratings Rank Ordered for all ATCSs (Continued)

Learning the Job Doing the Job

MEAN SD LABEL MEAN SD LABEL

3.82 0.87 Self-Esteem 3.89 0.86 Verbal Reasoning

3.81 0.86 Verbal Reasoning 3.88 0.98 Translating Information

3.78 0.81 Behavioral Consistency 3.88 0.92 Commitment to the Job

3.77 0.91 Recall From Interruption 3.83 0.92 Internal Locus of Control

3.74 0.95 Visuospatial Reasoning 3.80 0.85 Attention to Detail

3.72 0.98 Interpreting Information 3.80 1.02 Dynamic Visual-Spatial

3.71 0.95 Internal Locus of Control 3.80 1.03 Movement Detection

3.70 1.05 Movement Detection 3.79 0.87 Professionalism

3.69 0.95 Professionalism 3.78 1.00 Interpreting Information

3.69 0.99 Dynamic Visual-Spatial 3.78 0.97 Visuospatial Reasoning

3.68 0.99 Creativity 3.77 0.88 Self-Esteem

3.68 0.90 Interpersonal Tolerance 3.76 0.85 Behavioral Consistency

3.67 0.97 Street Physics 3.74 1.00 Street Physics

3.67 0.93 Motivation 3.72 0.85 Flexibility (IP)

3.59 0.90 Flexibility (IP) 3.68 0.86 Interpersonal Tolerance

3.58 0.92 Attention to Detail 3.67 1.01 Translation of Uncertainty

3.55 0.99 Translation of Uncertainty 3.67 0.91 Aviation Science Background

3.50 0.93 Chunking 3.53 0.92 Chunking

3.48 1.02 Aviation Science Background 3.47 0.93 Motivation

3.39 0.93 Written Communication 3.40 0.96 Written Communication

3.18 1.11 2-D Mental Rotation 3.32 1.13 Geography

3.18 1.17 3-D Mental Rotation 3.24 1.20 3-D Mental Rotation

3.16 1.03 Realistic Orientation 3.20 1.12 2-D Mental Rotation

3.14 1.12 Geography 3.19 1.02 Realistic Orientation

3.13 0.98 Mechanical Reasoning 3.15 1.00 Mechanical Reasoning

3.12 0.99 Mathematical Reasoning 3.08 1.01 Mathematical Reasoning

2.96 0.96 Numeric Ability (add/sub) 2.91 1.02 Numeric Ability (add/sub)

2.83 1.12 Angles 2.79 1.13 Angles

2.64 1.00 Numeric Ability (mult/div) 2.63 1.04 Numeric Ability (mult/div)

70
FIGURES AND TABLES 

Table 2.5. Worker Requirement Ratings for Doing the Job for the Three Options and All ATCSs

WORKER REQUIREMENT ARTCC TERMINAL FLIGHT SERVICE ALL ATCSs

LABEL MEAN SD MEAN SD MEAN SD MEAN SD

Oral Communication 4.56 0.56 4.57 0.66 4.63 0.53 4.56 0.63

Written Communication 3.18 0.94 3.43 0.88 3.49 1.11 3.40 0.96

Reading 3.79 0.98 3.87 0.89 4.26 0.78 3.95 0.89

Active Listening 4.59 0.60 4.55 0.65 4.52 0.65 4.53 0.66

Translating Information 3.68 1.08 3.73 0.98 4.32 0.74 3.88 0.98

Interpreting Information 3.30 1.02 3.59 0.97 4.51 0.64 3.78 1.00

Summarizing Information 3.94 0.87 3.92 0.86 4.46 0.62 4.06 0.83

Numeric Ability (add/sub) 3.00 1.23 2.88 1.01 3.02 0.86 2.91 1.02

Numeric Ability (mult/div) 2.75 1.28 2.58 1.01 2.66 0.95 2.63 1.04

Street Physics 3.97 0.98 3.82 3.38 1.00 1.00 3.74 1.00

Short-Term Memory 4.34 0.77 4.35 0.80 4.17 0.79 4.30 0.80

Intermediate-Term Memory 3.94 0.94 3.93 0.86 4.03 0.80 3.96 0.88

Long-Term Memory 4.29 0.80 4.11 0.80 4.01 0.91 4.11 0.92

Thinking Ahead 4.50 0.64 4.54 0.64 3.81 1.04 4.35 0.81

Planning 4.52 0.59 4.54 0.62 4.18 0.86 4.45 0.69

Prioritization 4.69 0.58 4.61 0.63 4.23 0.84 4.53 0.70

Execution 4.51 0.61 4.50 0.74 3.97 0.94 4.37 0.80

Problem Identification 4.31 0.76 4.33 0.75 3.96 0.97 4.23 0.83

Reasoning 4.42 0.70 4.33 0.69 4.16 0.86 4.31 0.74

Creativity 4.06 0.81 3.97 0.88 3.66 0.95 3.92 0.90

Self-Monitoring/Evaluating 4.09 0.93 4.03 0.83 3.96 0.77 4.04 0.84

Rule Application 4.18 0.87 4.29 0.75 4.20 0.83 4.25 0.79

Sustained Attention 4.13 0.86 4.14 0.92 3.87 1.01 4.07 0.94

Time Sharing 4.41 0.67 4.37 0.74 3.91 0.85 4.26 0.78

Situational Awareness 4.54 0.70 4.46 0.75 3.81 1.02 4.33 0.86

Visualization 4.21 0.90 4.33 0.79 4.12 0.96 4.26 0.85

Projection 4.24 0.90 4.36 0.78 4.04 0.91 4.26 0.84

Scanning 4.32 0.75 4.36 0.80 4.27 0.80 4.33 0.79

Professionalism 3.82 0.92 3.79 0.90 3.76 0.83 3.79 0.87

Working Cooperatively 4.06 0.86 4.12 0.78 4.13 0.85 4.10 0.81

Interpersonal Tolerance 3.51 0.97 3.71 0.85 3.76 0.79 3.68 0.86

Self-Esteem 3.78 0.98 3.75 0.87 3.82 0.85 3.77 0.88

(Continued)

71
FIGURES AND TABLES 

Table 2.5 Worker Requirement Ratings for Doing the Job for the Three Options and All ATCSs (Continued)

WORKER REQUIREMENT ARTCC TERMINAL FLIGHT SERVICE ALL ATCSs

LABEL MEAN SD MEAN SD MEAN SD MEAN SD

Self-Confidence 4.01 0.81 4.00 0.79 4.07 0.77 4.02 0.78

Taking Charge 4.43 0.68 4.38 0.68 4.03 0.81 4.30 0.73

Self-Awareness 4.17 0.82 4.15 0.79 3.94 0.73 4.11 0.78

Attention to Detail 3.82 0.86 3.76 0.86 3.84 0.81 3.80 0.85

Task Closure/Thoroughness 3.97 0.86 3.91 0.84 4.01 0.82 3.96 0.85

Decisiveness 4.40 0.76 4.45 0.69 4.26 0.75 4.40 0.71

Behavioral Consistency 3.65 0.94 3.73 0.85 3.96 0.72 3.76 0.85

Flexibility (S/A) 4.31 0.80 4.21 0.79 4.19 0.72 4.22 0.77

Concentration 4.21 0.80 4.28 0.78 4.11 0.83 4.23 0.80

Composure 4.60 0.65 4.53 0.69 4.38 0.73 4.50 0.70

Tolerance for High Intensity 4.68 0.58 4.66 0.56 4.46 0.68 4.60 0.62

Motivation 3.45 1.00 3.48 0.91 3.47 0.92 3.47 0.93

Commitment to the Job 3.79 0.96 3.90 0.92 3.98 0.88 3.88 0.92

Chunking 3.44 0.90 3.45 0.94 3.79 0.86 3.53 0.92

Mathematical Reasoning 3.27 0.95 2.99 1.01 3.13 1.01 3.08 1.01

Visuospatial Reasoning 3.94 0.87 3.90 0.97 3.39 0.93 3.78 0.97

Mechanical Reasoning 3.18 1.04 3.09 1.00 3.27 0.95 3.15 1.00

Verbal Reasoning 3.81 0.80 3.74 0.89 4.22 0.78 3.89 0.86

Angles 3.16 1.12 2.74 1.08 2.58 1.18 2.79 1.13

Flexibility (IP) 3.83 0.85 3.77 0.84 3.50 0.86 3.72 0.85

Dynamic Visual-Spatial 3.83 0.95 3.99 0.90 3.20 1.12 3.80 1.02

Movement Detection 3.54 1.08 4.03 0.89 3.31 1.10 3.80 1.03

Perceptual Speed & Accuracy 4.18 0.75 4.20 0.79 3.91 0.90 4.13 0.83

2-D Mental Rotation 3.05 1.20 3.35 1.04 2.97 1.11 3.20 1.12

3-D Mental Rotation 2.90 1.26 3.51 1.09 2.691 1.21 3.24 1.20

Realistic Orientation 3.07 1.16 3.24 0.97 3.19 1.03 3.19 1.02

Internal Locus of Control 4.00 0.82 3.81 1.10 3.74 0.84 3.83 0.92

Recall From Interruption 4.03 0.83 3.91 0.95 3.85 0.88 3.92 0.90

Translation of Uncertainty 3.74 0.96 3.79 1.00 3.37 1.00 3.67 1.01

Aviation Science Background 3.61 0.94 3.56 0.90 4.02 0.83 3.67 0.91

Geography 3.11 1.01 2.90 0.94 4.40 0.77 3.32 1.13

72
FIGURES AND TABLES 

Table 2.6. Worker Requirement Ratings for Learning the Job for the Three Options and All ATCSs

WORKER REQUIREMENT ARTCC TERMINAL FLIGHT SERVICE ALL ATCSs

LABEL MEAN SD MEAN SD MEAN SD MEAN SD

Oral Communication 4.26 0.81 4.30 0.87 4.20 0.91 4.25 0.87

Written Communication 3.29 0.98 3.41 0.85 3.44 1.04 3.39 0.93

Reading 4.03 0.97 4.09 0.85 4.34 0.81 4.15 0.86

Active Listening 4.45 0.68 4.51 0.66 4.31 0.74 4.43 0.71

Translating Information 3.82 0.91 3.76 0.91 4.15 0.86 3.87 0.92

Interpreting Information 3.23 1.08 3.59 0.90 4.33 0.81 3.72 0.98

Summarizing Information 3.77 0.87 3.81 0.86 4.24 0.79 3.91 0.86

Numeric Ability (add/sub) 3.14 1.07 2.91 0.96 3.00 0.90 2.96 0.96

Numeric Ability (mult/div) 2.79 1.21 2.62 0.94 2.61 0.97 2.64 1.00

Street Physics 3.79 1.01 3.77 0.90 3.38 0.99 3.67 0.97

Short-Term Memory 4.18 0.85 4.23 0.84 3.98 0.83 4.16 0.85

Intermediate-Term Memory 3.85 0.96 3.89 0.82 3.87 0.86 3.87 0.87

Long-Term Memory 4.37 0.76 4.14 0.86 3.89 0.91 4.11 0.88

Thinking Ahead 4.35 0.79 4.32 0.79 3.54 1.03 4.12 0.92

Planning 4.28 0.75 4.34 0.79 3.89 0.91 4.21 0.84

Prioritization 4.46 0.68 4.41 0.80 3.99 0.92 4.30 0.83

Execution 4.33 0.73 4.32 0.79 3.78 0.98 4.18 0.85

Problem Identification 4.22 0.76 4.16 0.80 3.71 0.94 4.05 0.85

Reasoning 4.21 0.88 4.22 0.76 3.92 0.85 4.13 0.82

Creativity 3.83 0.95 3.82 0.91 3.27 1.08 3.68 0.99

Self-Monitoring/Evaluating 3.91 1.04 3.97 0.90 3.68 0.85 3.89 0.93

Rule Application 4.27 0.75 4.22 0.86 4.03 0.84 4.19 0.84

Sustained Attention 3.95 0.98 3.95 0.92 3.66 0.99 3.87 0.96

Time Sharing 4.17 0.83 4.18 0.84 3.58 0.97 4.01 0.91

Situational Awareness 4.21 0.89 4.21 0.89 3.70 0.98 4.08 0.93

Visualization 4.09 0.85 4.15 0.91 3.86 0.96 4.06 0.91

Projection 4.13 0.84 4.17 0.87 3.76 0.99 4.05 0.91

Scanning 4.16 0.87 4.20 0.89 3.96 0.93 4.12 0.90

Professionalism 3.71 1.04 3.79 0.90 3.47 0.97 3.69 0.95

Working Cooperatively 3.83 1.02 3.94 0.86 3.63 1.10 3.82 0.96

Interpersonal Tolerance 3.71 0.89 3.78 0.90 3.50 0.88 3.68 0.90

Self-Esteem 3.88 0.95 3.88 0.83 3.73 0.88 3.82 0.87

(Continued)

73
FIGURES AND TABLES 

Table 2.6. Worker Requirement Ratings for Learning the Job for the Three Options and All ATCSs (Cont.)

WORKER REQUIREMENT ARTCC TERMINAL FLIGHT SERVICE ALL ATCSs

LABEL MEAN SD MEAN SD MEAN SD MEAN SD

Self-Confidence 3.91 0.92 3.93 0.84 3.74 0.87 3.87 0.86

Taking Charge 4.20 0.92 4.15 0.88 3.64 0.94 4.02 0.92

Self-Awareness 3.92 0.98 4.03 0.87 3.66 0.88 3.90 0.90

Attention to Detail 3.64 0.99 3.64 0.90 3.42 0.91 3.58 0.92

Task Closure/Thoroughness 3.85 0.92 3.88 0.85 3.80 0.86 3.86 0.87

Decisiveness 4.20 0.86 4.23 0.82 3.96 0.79 4.15 0.82

Behavioral Consistency 3.71 0.94 3.83 0.77 3.79 0.75 3.78 0.81

Flexibility (S/A) 4.08 0.81 4.10 0.83 3.86 0.82 4.04 0.83

Concentration 4.00 0.91 4.19 0.78 3.82 0.82 4.06 0.83

Composure 4.35 0.77 4.34 0.78 4.05 0.83 4.27 0.80

Tolerance for High Intensity 4.42 0.80 4.43 0.74 4.09 0.83 4.32 0.79

Motivation 3.69 0.99 3.74 0.89 3.49 1.00 3.67 0.93

Commitment to the Job 3.99 1.01 4.03 0.89 3.90 0.93 3.98 0.92

Chunking 3.43 0.92 3.52 0.93 3.53 0.94 3.50 0.93

Mathematical Reasoning 3.28 0.92 3.09 1.00 3.08 0.99 3.12 0.99

Visuospatial Reasoning 3.91 0.88 3.86 0.93 3.30 0.90 3.74 0.95

Mechanical Reasoning 3.13 1.04 3.10 0.97 3.18 0.93 3.13 0.98

Verbal Reasoning 3.75 0.90 3.73 0.90 3.99 0.74 3.81 0.86

Angles 3.16 1.06 2.82 1.10 2.54 1.13 2.83 1.12

Flexibility (IP) 3.71 0.90 3.70 0.85 3.25 0.92 3.59 0.90

Dynamic Visual-Spatial 3.74 0.97 3.85 0.91 3.17 1.04 3.69 0.99

Movement Detection 3.50 1.10 3.92 0.95 3.20 1.11 3.70 1.05

Perceptual Speed & Accuracy 4.03 0.82 4.09 0.84 3.74 0.93 3.99 0.87

2-D Mental Rotation 3.03 1.14 3.35 1.04 2.89 1.10 3.18 1.11

3-D Mental Rotation 2.89 1.20 3.43 1.06 2.59 1.21 3.18 1.17

Realistic Orientation 3.12 1.14 3.23 0.98 3.06 1.05 3.16 1.03

Internal Locus of Control 3.86 0.92 3.76 0.97 3.47 0.90 3.71 0.95

Recall From Interruption 3.93 0.86 3.81 0.93 3.56 0.93 3.77 0.91

Translation of Uncertainty 3.55 0.98 3.71 0.96 3.20 0.98 3.55 0.99

Aviation Science Background 3.33 1.09 3.45 0.98 3.71 1.00 3.48 1.02

Geography 2.88 1.05 2.85 0.95 4.01 1.00 3.14 1.12

74
FIGURES AND TABLES 

Table 2.7. Survey Subactivities for All ATCSs Ranked by the Mean Criticality Index

MEAN SD SUBACTIVITY LABEL

14.39 1.27 Checking and evaluating separation or traffic movement to ensure separation is maintained

12.96 2.25 Performing aircraft conflict resolution

12.70 2.37 Establishing and maintaining positive aircraft or vehicle identification

12.40 2.40 Establishing arrival sequences

12.34 2.61 Managing airborne departures (e.g., observe takeoff, issue go around)

11.89 2.66 Scanning to maintain awareness of surrounding airspace

11.61 2.61 Issuing arrival and landing information or instructions

11.51 2.59 Observing arrival aircraft

11.43 2.57 Establishing/maintaining/terminating radio communications

11.37 2.67 Managing departure flows

11.37 2.12 Responding to special conditions, contingencies, or emergencies

11.34 2.58 Managing departure traffic

11.32 2.46 Managing personal workload

11.31 2.58 Issuing departure information or instructions

11.26 2.14 Responding to contingencies/emergencies

11.22 2.29 Briefing relieving controllers

11.18 2.43 Assuming position responsibility

11.15 2.68 Recognizing and responding to deviations from ATCS instructions/clearances

10.94 2.79 Establishing/re-establishing/terminating radar identification

10.85 2.78 Disseminating weather information to pilots/other controllers

10.58 2.68 Conducting search and rescue procedures

10.56 2.37 Orienting lost aircraft

10.55 2.81 Prioritizing sector/position tasks

10.54 2.60 Issuing clearances

10.46 2.60 Responding to pointouts based on current or anticipated traffic situations

10.41 2.57 Responding to ground movement requests

10.35 2.62 Initiating pointouts

10.31 2.64 Responding to significant weather information

10.28 2.50 Initiating search and rescue procedures

10.04 2.67 Issuing unsafe condition advisories

10.02 3.05 Performing procedures for non-radar environment

9.92 2.88 Responding to changes in runway or taxiway usage

9.91 2.56 Receiving transfer of control or radar identification

9.90 3.08 Maintaining currency in weather data

(Continued)

75
FIGURES AND TABLES 

Table 2.7 Survey Subactivities for All ATCSs Ranked by the Man Criticality Index (Continued)

MEAN SD SUBACTIVITY LABEL

9.89 2.38 Analyzing initial requests for clearances

9.83 3.03 Managing aborted takeoffs

9.82 2.55 Initiating transfer of control or radar identification

9.81 2.36 Responding to communications failures

9.63 2.46 Executing backup procedures for communications failures

9.61 2.64 Responding to computer failures

9.58 2.68 Processing flight plans or flight plan amendments

9.53 2.55 Executing backup procedures for radar display failures

9.50 2.63 Transferring control responsibilities for departure aircraft

9.49 2.46 Processing pilot requests for clearance

9.41 2.59 Reviewing route of flight

9.40 2.76 Processing weather reports

9.40 2.81 Conducting direct weather observations

9.37 2.74 Responding to traffic management constraints or flow control conflicts

9.32 2.75 Managing automated handoff features

9.30 2.43 Responding to ARTS/BRITE/FDIO failures

9.28 2.78 Performing minimum safe altitude processing

9.13 2.40 Forwarding flight data information

9.12 2.83 Managing staff workload (performing CIC duties)

9.11 2.37 Responding to flow constraints

9.08 2.54 Requesting, processing and entering PIREPS (pilot reports)

9.06 2.73 Broadcasting TIBS/TWEBS/PATWAS/HIWAS

9.00 2.69 Responding to position or sector reconfigurations

8.09 2.85 Providing hazardous area reporting services

8.96 2.52 Executing backup procedures for ARTS/BRITE/FDIO failures

8.86 2.53 Updating flight progress strips

8.85 2.50 Responding to movement area closures or reopenings

8.85 2.66 Responding to imposed airspace restrictions

8.82 3.04 Receiving system status information and automated weather information

8.82 3.04 Receiving system status information and automated weather information

8.81 2.53 Responding to imposed movement area restrictions

8.81 2.88 Planning clearances

8.78 2.42 Responding to requests for transfer of control

8.75 2.36 Operating airport or taxiway lighting systems

(Continued)

76
FIGURES AND TABLES 

Table 2.7. Survey for All ATCSs Ranked by the Mean Criticality Index (Continued)

MEAN SD SUBACTIVITY LABEL

8.74 2.65 Processing emergency locator transmitter signals

8.74 2.61 Monitoring navigational equipment and communication equipment

8.73 2.66 Reporting significant aeronautical data

8.71 2.74 Processing pilot requests for airport information (airport advisories)

8.68 2.57 Performing movement area violation resolution

8.68 2.92 Processing aeronautical data (e.g., NOTAM’s, MOA)

8.51 2.65 Executing backup procedures for loss of flight plan data base

8.50 3.08 Activating flight plans

8.49 2.50 Executing backup procedures for NAVAID failures

8.45 2.56 Responding to airport equipment failures

8.40 2.65 Responding to pilot requests for route planning

8.35 3.02 Maintaining inbound/proposed/suspense lists

8.34 2.63 Responding to special use airspace events

8.28 2.63 Housekeeping or sector/position management

8.24 2.33 Executing backup procedures for sensor or tracking failures

8.23 3.08 Processing flight plan closures

8.12 2.69 Processing departure or en route time information

8.02 2.40 Responding to requests for temporary release of airspace

8.02 2.35 Responding to sensor outages

7.98 3.10 Reviewing proposed/inbound/suspense lists

7.94 2.51 Responding to special operations (e.g., VIP movements, parachutes)

7.93 2.56 Requesting temporary release of airspace

7.92 2.64 Requesting temporary release of movement areas

7.88 2.71 Responding to requests for temporary release of movement areas

7.65 2.76 Handling other facilities starting/closing operations

7.58 2.29 Processing requests for flight following

7.49 2.39 Suppressing or restoring alerts

7.45 2.29 Editing data base messages

7.40 2.70 Monitoring non-controlled objects (e.g., parachutes, balloons, animals)

7.24 2.65 Coordinating with US Customs

77
FIGURES AND TABLES 

Table 2.8. Worker Requirement Definitions Used in the Predictor-WR Linkage Survey

Worker

Requirement Definitions

Prioritization The ability to identify activities that are most critical and require immediate attention. This

involves a constant evaluation of new information followed by re-prioritization of job

activities.

Tolerance for High The ability to perform effectively and think clearly during heavy work flow.

Intensity

Composure The ability to think clearly in stressful situations.

Active Listening The ability to hear and comprehend spoken information. This ability requires an individual

to recognize or pick out pertinent auditory information.

Oral The ability to speak clearly and concisely to individuals so they understand what is being

Communication communicated. Projecting a confident tone of voice is an important component of this

ability.

Situational Being cognizant of all information within a four dimensional space (i.e., separation

Awareness standards plus time). This involves the ability to “understand” the airspace as an integrated

whole (e.g., getting the picture).

Planning The ability to determine the appropriate course(s) of action to take in any given situation.

Execution The ability to take timely action to avoid problems and to solve existing problems.

Thinking Ahead The ability to anticipate or recognize problems before they occur and to develop plans to

avoid problems. This includes thinking about what might happen.

Taking Charge The ability to take control of a situation and reach out and take correct action.

Reasoning The ability to apply available information in order to make decisions, draw conclusions, or

identify alternative solutions.

Time Sharing The ability to perform two or more job activities at the same time.

Decisiveness The ability to make effective decisions in a timely manner.

Short-Term The ability to remember pertinent information within a brief period of time (less than 1

Memory minute). Examples of information include call signs and keywords.

Scanning The ability to quickly and accurately search for information on a computer screen, radar

scope, or computer print-out.

Problem Solving The ability to identify a potential problem or existing problem and to identify the variables

used in solving the problem.

Flexibility The ability to adapt to changing situations or conditions.

(Stability/

Adjustment)

Long-Term The ability to remember pertinent information over long periods of time. Examples of

Memory information include maps and separation procedures.

Projection The ability to translate material into visual representation of what will occur in the future.

Visualization The ability to translate material into a visual representation of what is currently occurring.

Concentration The ability to focus on job activities amid distractions for short periods of time.

Confirmation The ability to efficiently select a response option consistent with the application of

inferred rules.

(Continued)

78
FIGURES AND TABLES 

Table 2.8. Worker Requirement Definitions Used in the Predictor-WR Linkage Survey (Continued)

Worker

Requirement Definitions

Self-Awareness The ability to maintain an internal awareness of your actions and attitudes. This includes

knowing your limitations.

Rule Application The ability to efficiently apply transformational rules inferred from the complete portions

of the stimulus array to the incomplete portion of the array.

Perceptual Speed Ability to perceive visual information quickly and accurately and to perform simple

& Accuracy processing tasks with (e.g., comparisons).

Sustained Attention The ability to stay focused on a task(s) for long periods of time (over 60 minutes).

Self-Monitoring/ The ability and willingness to check your own work performance, evaluate the

Evaluation effectiveness of your decisions, and alter your performance if necessary.

Creativity The ability to identify new or novel solutions to potential problems when existing or

established solutions no longer apply.

Working The willingness to work with others to achieve a common goal. This includes a

Cooperatively willingness to voluntarily assist another controller if the situation warrants.

Recall from

Interruption

The ability to recall a deferred or interrupted action when priorities permit, and to be able

to resume the action appropriately.

Self-Confidence A belief that you are the person for the job and knowing that your processes and decisions

are correct.

Internal Locus of Believes that individuals have influence over the outcome of an event; takes responsibility

Control for outcomes.

Street Physics The ability to understand the general rules of science as practiced in the ATCS

environment (e.g., aircraft size and turbulence effects, angles of intersect, overtake speed,

headings).

Task Closure/

Thoroughness

The ability to continue an activity to completion through the coordination and inspection

of work.

Summarizing

Information

The ability to summarize and consolidate information most relevant to the situation.

Intermediate-Term The ability to remember pertinent information over a 1-10 minute period.

Memory

Visuospatial Ability to perceive and understand principles governing relationships among several

Reasoning figures.

Flexibility The ability to find new meanings for stimuli, to combine stimulus attributes to come up

(Information with new and different solution protocols, and to employ flexible ways of relating new

Processing) information to stored knowledge.

Dynamic Visual- Ability to deal with dynamic visual movement.

Spatial

Professionalism The ability to establish respect and confidence in your abilities among other controllers.

Attention to Detail The ability to recognize and attend to the details of the job that others might overlook.

Verbal Reasoning The ability to perceive and understand principles governing the use of verbal concepts and

symbols.

Reading The ability to read and understand written information (e.g., ATCS documents, manuals).

Learning Changes in information-processing strategies over time or trials not due to maturation or

aging.

(Continued)

79
FIGURES AND TABLES 

Table 2.8. Worker Requirement Definitions Used in the Predictor-WR Linkage Survey (Continued)

Worker

Requirement Definitions

Commitment to the The desire to be an ATCS and work hard to be successful.

Job

Self-Esteem Having a positive opinion/image of oneself.

Translation of The ability to assign a subjective probability regarding the likelihood of an event

Uncertainty occurring; the ability to use probabilities to identify optimal courses of action.

Translating The ability to translate symbols or symbolic abbreviations into meaningful information.

Information

Behavioral The ability to behave consistently at work (e.g., dealing with coworkers in a consistent

Consistency manner; consistently using the correct phraseology).

Encoding Transformation or translation of information; coding; decoding.

Movement Ability to detect physical movement of objects and to judge their direction.

Detection

Interpersonal The ability to accommodate or deal with differences in personalities, criticisms, and

Tolerance interpersonal conflicts in the work environment.

Motivation The desire to motivate oneself through challenges on the job and to progress to a higher

level of skill.

Chunking The ability to organize stimuli into meaningful groups or units.

Interpreting The ability to put information into meaningful terms. It is the ability to recognize the

Information implications for a statement or condition (e.g., cold front).

Mathematical

Reasoning

Ability to perceive and understand principles governing the use of quantitative concepts

and symbols.

Written

Communication

The ability to write legibly and accurately (e.g., strip markings).

Mechanical Ability to perceive and understand the relationship of physical forces and mechanical

Reasoning elements in a prescribed situation.

Angles The ability to apply the principles of geometry to angles and computations involving

angles. The ability involves both the speed and accuracy of computation.

Rule Inference The ability to efficiently ascertain the rules governing relations between stimulus

attributes.

Realistic Prefers dealing with activities that have tangible and measurable consequences; enjoys

Orientation activities which require skill, is reinforced by accomplishing realistic tasks.

2-D Mental Ability to identify a two-dimensional figure when seen at different angular orientations.

Rotation

Numeric Ability The ability to quickly and accurately perform basic math operations (addition and

(add/sub) subtraction).

3-D Mental Ability to identify a three-dimensional object when seen at different angular orientations

Rotation either within the picture plane or about the axis in depth.

Numeric Ability The ability to quickly and accurately perform basic math operations (multiplication and

(mult/div) division).

80
FIGURES AND TABLES 

Table 2.9. Number of Raters and Intra-Class Correlations for Each Scale

Intra-Class

AT-AT-SAT Predictor Number of Raters Correlation

Version 1

Applied Math

Dials

Letter Factory

Scanning

Sound Memory

Time Wall

Version 2

Angles

Air Traffic Scenarios

Analogies

Memory

Planes

EQ-Tolerance for High Intensity

EQ-Composure

EQ-Decisiveness

EQ-Execution

EQ-Taking Charge

EQ-Concentration

EQ-Flexibility

EQ-Self-Awareness

EQ-Working Cooperatively

EQ-Sustained Attention

EQ-Self-Confidence

EQ-Task Closure/Thoroughness

EQ-Interpersonal Tolerance

EQ-Consistency of Work Behavior

12 .95

12 .91

12 .94

12 .92

12 .94

12 .92

14 .89

14 .92

14 .93

13 .88

13 .86

14 .93

14 .95

14 .90

14 .94

14 .89

14 .93

14 .89

14 .95

14 .96

12 .94

12 .96

12 .92

12 .94

12 .87

81
FIGURES AND TABLES 

Table 2.10. AT-SAT Tests Rated as Measuring Each SACHA-Generated Worker Requirement

Worker Requirement a Test Mean b SD

Prioritization Air Traffic Scenarios 4.64 .50

Letter Factory 4.25 .97

Tolerance for High Intensity EQ-Tol. for High Intensity 4.71 .47

Air Traffic Scenarios 4.14 .86

Letter Factory 3.92 1.00

EQ-Composure 3.79 .89

Composure EQ-Composure 4.57 .85

EQ-Tol. for High Intensity 3.43 1.09

Air Traffic Scenarios 3.36 1.01

Active Listening Sound Memory 4.33 .89

Oral Communication

Situational Awareness Air Traffic Scenarios 4.50 1.34

Letter Factory 3.33 1.83

Planning Air Traffic Scenarios 4.14 .77

Letter Factory 4.00 .85

Execution Letter Factory 4.42 .67

Air Traffic Scenarios 4.07 .73

EQ-Execution 3.86 1.79

Thinking Ahead Air Traffic Scenarios 4.14 .66

Letter Factory 3.92 1.08

Taking Charge EQ-Taking Charge 4.43 .94

Reasoning Analogies 3.36 1.65

Air Traffic Scenarios 3.09 1.58

Time Sharing Time Wall/Pattern Recog. 4.08 1.56

Letter Factory 3.50 1.51

Decisiveness EQ-Decisiveness 4.07 1.44

Air Traffic Scenarios 3.57 1.09

Letter Factory 3.50 1.09

EQ-Execution 3.14 1.17

Short-Term Memory Sound Memory 4.92 .29

Letter Factory 3.33 1.23

Memory Test 3.23 2.01

Scanning Scanning 5.00 0

Letter Factory 4.17 .72

Air Traffic Scenarios 3.86 1.23

Time Wall/Pattern Recog. 3.83 1.47

Dials 3.42 1.44

Problem Solving

Flexibility (Stabil./Adjustment) EQ-Flexibility 4.29 1.44

Long-Term Memory

(Continued)

82
FIGURES AND TABLES 

Table 2.10. AT-SAT Tests Rated as Measuring Each SACHA-Generated Worker Requirement (Cont.)

Worker Requirement a Test Mean b SD

Projection Planes 3.77 1.79

Visualization

Concentration EQ-Concentration 4.29 1.44

Letter Factory 3.17 1.70

Self-Awareness EQ-Self Awareness 4.50 .94

Rule Application

(SME-generated)

Perceptual Speed & Accuracy Time Wall/Pattern Recog. 4.58 .67

Scanning 4.58 .67

Dials 4.33 .89

Planes 3.92 1.61

Letter Factory 3.75 .97

Air Traffic Scenarios 3.21 1.63

Sustained Attention EQ-Sustained Attention 4.07 .92

Self-Monitoring/Evaluation EQ-Self Awareness 2.92 2.02

Creativity

Working Cooperatively EQ-Working Cooperatively 4.64 .74

Recall from Interruption

Self Confidence EQ-Self Confidence 4.64 .74

Internal Locus of Control

Street Physics

Task Closure/Thoroughness EQ-Task Closure 4.43 .94

Summarizing Information

Intermediate-Term Memory Memory Test 3.77 1.92

Visual-Spatial Reasoning Analogies 3.57 1.74

Flexibility (Inform. Processing)

(Continued)

83
FIGURES AND TABLES 

Table 2.10. AT-SAT Tests Rated as Measuring Each SACHA-Generated Worker Requirement (Cont.)

Worker Requirement a Test Mean b SD

Dynamic Visual-Spatial Air Traffic Scenarios 4.07 1.44

Planes 3.54 1.66

Time Wall/Pattern Recog. 3.50 1.45

Letter Factory 3.50 1.51

Scanning 3.08 1.73

Professionalism

Attention to Detail Letter Factory 2.50 1.57

Verbal Reasoning

Reading

Commitment to the Job

Self-Esteem EQ-Self Confidence 3.21 1.63

Translation of Uncertainty

Translating Information

Behavioral Consistency EQ-Consist. Of Work Behavior 3.43 1.87

Movement Detection Air Traffic Scenarios 4.29 .91

Time Wall/Pattern Recog. 3.08 1.78

Interpersonal Tolerance EQ-Interpersonal Tolerance 4.29 1.44

EQ-Working Cooperatively 3.00 1.76

Motivation

Chunking

Interpreting Information

Mathematical Reasoning Applied Math 4.83 .39

Written Communication

Mechanical Reasoning

Angles Angles 4.64 .84

Realistic Orientation

(Continued)

84
FIGURES AND TABLES 

Table 2.10. AT-SAT Tests Rated as Measuring Each SACHA-Generated Worker Requirement (Cont.)

Worker Requirement a Test Mean b SD

2-D Mental Rotation

Numeric Ability (add/sub) Applied Math 4.58 .90

3-D Mental Rotation

Numeric Ability (mult./div.) Applied Math 4.25 1.48

Confirmation a Analogies 3.64 1.69

Letter Factory 3.17 1.53

Encoding a Memory Test 3.00 1.47

Rule Inference a Analogies 4.79 .43

Rule Application a Analogies 4.29 1.38

Learning a

a

All but five of the Worker Requirements are rank ordered by mean ratings (of ARTCC controllers from

SACHA job analysis) regarding importance for doing the job. SACHA did not collect data on the last five

worker requirements (Confirmation, Encoding, Rule Inference, Rule Application (SACHA-generated), and

Learning) because it was felt that the complex cognitive nature of these WR definitions would not be fully

graspable by controllers completing the SACHA job analysis surveys.

b A criterion cutoff mean ≥ 3 was established to indicate whether a test was able to measure a particular WR.

85
FIGURES AND TABLES 

Table 2.11. Indicators of the Success of AT-SAT Measures in Measuring Multiple Worker Requirements

Mean of Sum of

Number of Number of WRs Number of WRs Linkage Linkage

WRs With With Ratings > 3 With Ratings > Rating x Rating x

AT-SAT Predictor Ratings > 3 Measured by 1 or 3 Not Measured SACHA SACHA

Fewer Tests by Other Tests Rating Rating

Applied Math

Dials

Letter Factory

Scanning

Sound

Time Wall

Angles

AT Scenarios

Analogies

Memory

Planes

EQ-Tol. for High Int.

EQ-Composure

EQ-Decisiveness

EQ-Execution

EQ-Taking Charge

EQ-Concentration

EQ-Flexibility

EQ-Self-Awareness

EQ-Working Coop.

EQ-Sustained Attent.

EQ-Self-Confidence

EQ-Task Closure

EQ-Interperson. Tol.

EQ-Consis. Of Work

Behavior

3 3 3 13.7 41.2

2 0 0 16.5 32.9

14 7 0 16.2 226.4

3 0 0 17.5 52.5

2 1 1 20.7 41.3

5 2 0 15.6 77.9

1 1 1 14.7 14.7

12 5 0 17.5 209.4

5 5 4 15.6 78.2

3 2 2 13.4 40.1

3 1 1 15.3 46.0

2 0 0 18.4 36.8

2 0 0 19.4 38.8

1 0 0 17.9 17.9

2 0 0 15.6 31.2

1 1 1 19.6 19.6

1 1 0 18.1 18.1

1 1 1 18.5 18.5

1 1 1 18.9 18.9

1 1 1 18.8 18.8

1 1 1 16.8 16.8

2 2 2 15.4 30.7

1 1 1 17.6 17.6

1 1 1 15.1 15.1

1 1 1 12.5 12.5

Table 3.1.1. Regression Coefficients for PTS Pre-Training Screen

St. Err. St. Err.

Variable Beta of Beta B of B T(351) p-Level

Static Vector % Correct .1337 .0507 .2481 .0341 2.636 .0088

Static Vector Correct -.1494 .0601 -.0054 .0022 -2.485 .0134

Reaction Time

Continuous Memory -.0501 .0551 -.0014 .0015 -.909 .3639

Correct Reaction Time

Time Wall Absolute -.0546 .0527 -.0014 .0016 -1.035 .3012

Time Error

Pattern Recognition -.0166 .0506 -.0025 .0075 -.239 .7427

Correct Reaction Time

ATST Safety -.3068 .0536 -.6992 .1221 -5.725 .0000

86
FIGURES AND TABLES 

Table 3.1.2 Regression Table for Pre-Training Screen

Dependent Variable Academy Screen Comprehensive Test Score

Multiple R .4906

Multiple R-Square .2404

Adjusted R-Square .2277

Minimum Pairwise N 358

F (6,351) = 18.54 p < .0000

Standard Error of Estimate 10.59

Intercept 72.07

Standard Error 10.50 t(351) = 6.860 p<.0000

87

FIGURES AND TABLES

Table

3.1.3

Meta-Analysis

of

Prior

ATCS

Validation

Studies

Predictor Measure Grouping 

Criterion Measure Group 

Basic 

Cognitive 

Ability 

Measures 

Experience 

Measures 

Personality 

and Interest 

Inventory 

Measures 

Simulation 

Type 

Measures 

MultiTasking 

Measures 

Visual and 

Spatial 

Measures 

BioMeasures 

(e.g., Age, 

Education) 

Keyboard 

Visual RT 

Measure

Screen Performance 

Number of studies 

Total Sample Size 

Mean sample r Variance of sample r Est Sampling Var(r) 

189 

54,211 

0.22 

0.0174 0.0032 

165 

91,543 

0.17 

0.0084 0.0017 

99 

23,949 

0.12 

0.0055 0.0040 

80 

32,606 

0.25 

0.0191 0.0021 

51 

5,746 

0.08 

0.0053 0.0087 

64 

30,829 

0.24 

0.0154 0.0019 

38 

39,040 

0.11 

0.0042 0.0009 

5,4

0.

0.060.004

Instructor Ratings 

Number of studies 

Total Sample Size 

Mean sample r Variance of sample r Est Sampling Var(r) 

84 

12,045 

0.13 

0.0080 0.0067 

21 

25,398 

0.13 

0.0030 0.0008 

101 

16,802 

0.07 

0.0042 0.0059 

10 

5,973 

0.11 

0.0044 0.0016 

57 

3,417 

0.20 

0.0158 0.0152 

11 

3,680 

0.19 

0.0053 0.0028 

6 

780 

0.13 

0.0074 0.0074 

2

0.

0.060.00

Supervisor or Self Appraisal 

Number of studies 

Total Sample Size 

Mean sample r Variance of sample r Est Sampling Var(r) 

23 

7,205 

0.17 

0.0175 0.0028 

35 

9,302 

0.08 

0.0039 0.0037 

7 

1,616 

0.04 

0.0033 0.0043 

12 

3,677 

0.16 

0.0034 0.0031 

5 

2,244 

0.08 

0.0046 0.0022 

14 

6,472 

0.11 

0.0072 0.0021 

40

0.0140.004

Field Training Duration 

Number of studies 

Total Sample Size 

Mean sample r Variance of sample r Est Sampling Var(r) 

21 

32,667 

0.05 

0.0045 0.0006 

21 

18,070 

0.05 

0.0010 0.0012 

Field Training Process 

Number of studies 

Total Sample Size 

Mean sample r Variance of sample r Est Sampling Var(r) 

4 

1,506 

0.07 

0.0019 0.0026 

9 

7,062 

0.04 

0.0025 0.0013 

4 

1,001 

0.18 

0.0058 0.0038 

12 

8,161 

0.11 

0.0035 0.0014 

2 

379 

0.15 

0.0012 0.0051 

8 

4,179 

0.04 

0.0021 0.0019 

1,1

0.

0.000.00

Attrition 

Number of studies Total Sample Size 

Mean sample r 

Variance of sample r 

Est Sampling Var(r) 

1 

2,992 

0.00 

0.0000 0.0000 

5 

1,912 

0.06 

0.0013 0.0026 

1 

2,992 

0.00 

0.0000 0.0000 

10 

6,252 

0.10 

0.0077 0.0015 

89
FIGURES AND TABLES

Predictor Measure Grouping 

Criterion Measure Group 

Basic 

Cognitive 

Ability 

Measures 

Experience 

Measures 

Personality 

and Interest 

Inventory 

Measures 

Simulation 

Type 

Measures 

MultiTasking 

Measures 

Visual and 

Spatial 

Measures 

BioMeasures 

(e.g., Age, 

Education) 

Keyboard 

Visual RT 

Measure

Simulation Scores 

Number of studies 

Total Sample Size 

Mean sample r 

Variance of sample r 

Est Sampling Var(r) 

9 

1,008 

0.39 

0.0038 

0.00064 

1 

112 

0.42 

0.0000 0.0000 

1 

112 

0.51 

0.0000 0.0000 

3 

336 

0.39 

0.0000 0.0065 

3

0.

0.050.00

Archival Perf Records 

Number of studies 

Total Sample Size 

Mean sample r Variance of sample r Est Sampling Var(r) 

15 

3,165 

0.24 

0.0252 0.0040 

1 

211 

0.22 

0.0000 0.0000 

Controller Skills Test 

Number of studies Total Sample Size 

Mean sample r Variance of sample r Est Sampling Var(r) 

1 

1,225 

0.20 

0.0000 0.0000 

2 

8,228 

0.19 

0.0029 0.0002 

1 

1,255 

0.16 

0.0000 0.0000 

2 

2,510 

0.30 

0.0008 0.0007 

Composite Criteria 

Number of studies Total Sample Size 

Mean sample r Variance of sample r Est Sampling Var(r) 

5 

2,688 

0.26 

0.0029 0.0016 

5 

2,398 

0.16 

0.0011 0.0020 

10 

5,856 

0.23 

0.0006 0.0015 

4 

2,945 

0.17 

0.0001 0.0013 

10 

8,725 

0.09 

0.0132 0.0011 

3,5

0.

0.000.00

Across Criterion Measure Types 

Number of studies 

Total Sample Size 

Mean sample r Variance of sample r Est Sampling Var(r) 

325 

80,395 

0.20 

0.0174 0.0037 

259 

179,880 

0.13 

0.0087 0.0014 

222 

47,790 

0.10 

0.0058 0.0045 

148 

78,702 

0.17 

0.0157 0.0017 

108 

9,163 

0.12 

0.0128 0.0111 

92 

43,134 

0.23 

0.0138 0.0019 

86 

65,448 

0.10 

0.0059 0.0013 

4

11,2

0.

0.040.00

90
FIGURES AND TABLES

Table 3.1.4 Proposed New Measures for the g WR Constructs

CONSTRUCT WORKER REQUIREMENT PRELIMINARY PROPOSED

CATEGORIES CONSTRUCTS TESTS MEASURES

Applied Reasoning Chunking

Translating Information

Interpreting Information

Summarizing Information

Reasoning (Thinking)

Visuospatial Reasoning

Verbal Reasoning

Mechanical Reasoning

Mathematical Reasoning

CreativityRule ApplicationTranslation of Uncertainty

into Probability

Street Physics

Computational Numeric Ability (Addit./Sub.)

Grouping Info. Test

Reasoning Battery

Syllogisms

Analogies

Classifications

Rule Application Test

Revise Available Street

Physics Test

Headings Test

Time Test

Angles

Projection Test Revise Paper Fold. Test

Ships/Planes Test a

Direct. & Distance

Ships/Planes Test

Dial Reading Test

Spatial

Numeric Ability (Mult./Div.)

Angles

VisualizationProjection

Dynamic Visual-Spatial

Perceptual Scanning

Movement DetectionPerceptual Speed & Accuracy

Memory Short-Term (working memory) 2 Short-Term Tests

Direct. & Distance

91
FIGURES AND TABLES

Intermediate-Term Memory Short-Term Test

(Repeated)

Long-Term Memory Map Memory

Communication Oral Communication Consider Revision of

Active Listening Ship Test

Written CommunicationReading Reading Test

a We renamed the Ships test to Planes for face validity purposes.

Table 3.1.5. Proposed New Measures for the Processing Operations WR Constructs

CONSTRUCT WORKER REQUIREMENT PRELIMINARY PROPOSED

CATEGORIES CONSTRUCTS TESTS MEASURES

Perceptual Scanning

Movement Detention

Perceptual Speed and Accuracy

Information EncodingProcessing Rule Inference

Rule Application

Confirmation

LearningFlexibility

Metacognitive Thinking Ahead

Problem Identification

PlanningPrioritizationSelf-Monitoring

Situational Awareness

A Complex Stimuli Test a

Analyze ATST for

Measurement Possibilities b

Dial Reading Test

Reasoning Battery

Info. Process

Measures

STIX Test

NONE

92
FIGURES AND TABLES

Attention Sustained Attention Add scale to PEAQ a

Recall From Interruption Develop TestTimesharing Ships/Planes Test c Parallel Ships Test for

Multitasking aspect ofTimesharing

a Proposed new measure completed.b The SACHA team completed an investigation of the ATST for measurement possibilities. We are currently designing a metacognitive 

analysis.

We renamed the Ships test to Planes for face validity purposes.

93

c
FIGURES AND TABLES

Table 3.1.6. Temperament/Interpersonal Model

Big Five

Temperament/Interpersonal

Construct Categories (Hough’s

Taxonomy)

→ 

→ 

→ 

→ 

→ → → → 

→ 

Temperament/Interpersonal

Constructs

[none]

Professionalism a (I)

Taking Charge b (W)

Self-Esteem b (S)

Self-Esteem b (S)

Self-Confidence b (S)

Execution b (W)

Task Closure/Thoroughness b (W)

Motivation (W)Commitment to the Job (W)

Task Closure/Thoroughness b (W)

Behavioral Consistency (W)Working Cooperatively b (I)

Self-Esteem b (S)

Flexibility (A)

Composure (A)Tolerance for High Intensity

Work Situations (A)

Working Cooperatively b (I)

Interpersonal Tolerance (I)

[none]

Decisiveness (W)

Realistic Orientation (W)

Taking Charge b (W)

Execution (W)

Extraversion → Conscientiousness → 

→ → → 

Affiliation

PotencyAchievement b

Dependability

AdjustmentAgreeablenessIntellectance

Rugged IndividualismLocus of Control

NeuroticismAgreeablenessOpenness

94
FIGURES AND TABLES

Focus c

Internal Locus of Control (S)

Self Awareness (S)Concentration (W)

Attention to Detail (W)

→ 

a The letters included in the parentheses indicate SACHA’s initial worker requirement categorization (W = Work and Effort, I = Interpe

Efficacy, A = Stability/Adjustment).

b Included in more than one higher-order construct.c Not part of Hough’s taxonomy.

95
FIGURES AND TABLES

EXHIBIT 3.2.1

PILOT TEST ADMINISTRATION

TEST BLOCK SEQUENCING

GROUPS SESSION 1 SESSION 2 SESSION 3 SESSION 4 SESSION 5

Group 1

Computers

1 - 10

Block

A

Block

B

Block

C

Block

D

Ending Block 

LFT

Group 2

Computers

11 - 20

Block

B

Block

E

Block

A

Block

C

Ending

Block 2

ATS

Group 3

Computers

21 - 30

Block

C

Block

A

Block

D

Block

E

Ending

Block 3

SVCM, Word

Memory

Group 4

Computers

31 - 40

Block

D

Block

C

Block

E

Block

B

Ending

Block 4

Word Memor

TWPR

Group 5

Computers

41 - 50

Block

E

Block

D

Block

B

Block

A

Ending

Block 1

LFT

EXHIBIT 3.2.2

AIR TRAFFIC SCENARIOS TEST

EXAMPLE OF THE PLANE DESCRIPTORS

96
FIGURES AND TABLES

F BThe first letter The number indicates The second letter

indicates the speed. The level. There are indicates the

F = Fast four levels. “1” is destination. “A”,

M = Medium the lowest (closest to the “B”, “C”, and “D”

S = Slow ground) while “4” is the are exits out of

Highest (furthest from your airspace. “E”

The ground). and “F” are airports.

3



EXHIBIT 3.2.3

SUMMARY OF THE PROPOSED REVISIONS

TO THE AT-SAT PILOT TEST

TEST BLOCK TESTS SUGGESTED REVISIONS

BLOCK A Air Traffic Scenarios � Clarify general test instructions

� Clarify the meaning of and action to take on the new planes� Modify the countdown� Further clarify the purpose and position of the landing heading

indicator

Sound Test � Clarify test instructions

� Clarify the keys to use for the sound level adjustment� Instruct participants to remove the headphones after the test

BLOCK B

Letter Factory Test � Clarify test instructions

� Clarify the demonstration� Clarify the instructions for the mouse exercise� Put a time limit on the mouse exercise

BLOCK C Dials Test � No changes needed

97
FIGURES AND TABLES

EXHIBIT 3.2.3

SUMMARY OF THE PROPOSED REVISIONS

TO THE AT-SAT PILOT TEST

TEST BLOCK TESTS SUGGESTED REVISIONS

Static Vector/

Continuous MemoryTest

� Clarify test instructions� Explain and familiarize the participants with the keys needed to

answer the scenarios

ExperiencesQuestionnaire

� No changes needed

Time Wall/Pattern RecognitionTest

� Relocate the broken wall� Explain and familiarize the participants with the keys needed to

answer the scenarios

Analogy Test � Reduce the required level of vocabulary and reasoning skills

BLOCK D

Classification Test � Reduce the required level of vocabulary and reasoning skills

Word Memory Test � Remove the scoring for the test items

� Remove the extraneous window prompts

Scan Test � Clarify the test instructions

BLOCK E

Planes Test � Reduce the length of the practice session

� Modify the keystrokes for “True” and “False”� Remove the scoring for each test item

Graph continued on next page.

98
FIGURES AND TABLES

EXHIBIT 3.2.3 CONTINUED

SUMMARY OF THE PROPOSED REVISIONS

TO THE AT-SAT PILOT TEST

BLOCK E Angles Test � None

Applied Mathematics

Test

� Clarify the test instructions� Reduce the level of required mathematics skills

ENDING

BLOCK

(Groups 1, 5)

Letter Factory Test � Clarify test instructions

� Clarify the demonstration� Clarify the instructions for the mouse exercise� Put a time limit on the mouse exercise

ENDING

BLOCK

(Group 2)

Air Traffic Scenarios

Test

� Clarify general test instructions� Clarify the meaning of and action to take on the new planes� Modify the countdown� Further clarify the purpose and position of the landing heading

indicator

ENDING

BLOCK

(Group 3)

Static Vector/

Continuous MemoryTest & Word

Memory

� Clarify test instructions (SV)� Explain and familiarize the participants with the keys needed to

answer the scenarios (SV)

� Remove the scoring for the test items (WM)� Remove the extraneous window prompts (WM)

ENDING

BLOCK

(Group 4)

Word Memory &Time Wall/ Pattern

Recognition Test

� Remove the scoring for the test items (WM)� Remove the extraneous window prompts (WM)� Relocate the broken wall (TW)� Explain and familiarize the participants with the keys needed to

answer the scenarios (TW)

99
--- --- ---

--- --- ---

--- --- ---

--- --- ------ --- ------ --- ---

FIGURES AND TABLES

Table 3.3.1. Item Analyses and Scale Reliabilities: Non-Semantic Word Scale on

the Analogy Test (N=439)

Item

Number

Corrected

Item-Total

Correlation

Rerun of Corrected

Item-Total

Correlation

Information-Processing

Corrected Item-Total

Correlationa

Total Total Item

Screen Response Time

Views

1 .082 .164 -.016 .198 .19

11 .1712 .1414 .0016 .0918 .0320 .2222 .0924 .2728 -.01

30 .15

Alpha = .39

15 Items

.17 .36 .46.18 .45 .49.21 .36 .44.18 .44 .56.15 .43 .53.16 .30 .45.23 .37 .42.26 .34 .46.12 .38 .49

Alpha = .44

9 Items

Alpha = .84

9 Items

aStandardized score

100
--- --- ---

--- --- ---

--- --- ---

FIGURES AND TABLES

Table 3.3.2. Item Analyses and Scale Reliabilities: Semantic Word Scale on the Analogy Test (N=439)

Item Corrected Rerun of Corrected Information-Processing

Number Item-Total Item-Total Corrected Item Total

Correlation Correlation Correlationa

Total Screen Total Item

Views Response Time

3 .035 .247 .099 .15

10 .1313 .2015 .1417 .3319 .1221 .1323 .2025 .2326 .2827 .0229 .33

Alpha = .50

15 Items

.24 .31 .38.17 .32 .41

.13 .30 .39.19 .29 .40.14 .40 .54.34 .30 .42.11 .33 .45.13 .42 .52.20 .35 .40.23 .38 .49.31 .34 .45

.33 .39 .42

Alpha = .52 Alpha = .84

12 Items 24 Items

aStandardized score

101
--- --- ---

--- --- ------ --- ---

FIGURES AND TABLES

Table 3.3.3. Item Analyses and Scale Reliabilities: Semantic Visual Scale on the Analogy Test (N=439)

Item Corrected Rerun of Corrected Information-Processing

Number Item-Total Item-Total Corrected Item-Total

Correlation Correlation Correlationa

Total Screen Total Item

Views Response Time

31 .2034 .1935 .1637 .1439 .2441 .1644 .0447 .0650 .2651 .09

Alpha = .40

10 Items

.22 .33 .40.23 .41 .43.13 .40 .42.17 .40 .28.26 .39 .30.15 .47 .42.28 .35 .36

Alpha = .44 Alpha = .77

7 Items 14 Items

aStandardized score

102
FIGURES AND TABLES

Table 3.3.4. Item Analyses and Scale Reliabilities: Non-Semantic Visual Scale on the

Analogy Test (N=439)

Item

Number

Corrected

Item-Total

Correlation

Rerun of Corrected

Item-Total

Correlation

Information-Processing

Corrected Item-Total

Correlationa

Total Total Item

Screen Response Time

Views

32 .25 .25 .56 .5633 .26 .26 .61 .5736 .26 .26 .66 .5138 .26 .26 .52 .4840 .40 .40 .43 .4942 .41 .41 .57 .5243 .37 .37 .49 .5445 .22 .22 .69 .6946 .35 .35 .56 .6048 .30 .30 .62 .6149 .18 .18 .62 .6652 .37 .37 .52 .5253 .18 .18 .52 .57

Alpha = .67 Alpha = .67 Alpha = .93

13 Items 13 Items 26 Items

aStandardized score

103
FIGURES AND TABLES

Table 3.3.5. Distribution of Test Completion Times for the Analogy Test

Percentile Test Completion Time (Minutes)

Minimum

5th

25th50th75th95th

Maximum

Mean

SD

6.1

10.114.918.824.332.1

35.6

19.8

6.59

Table 3.3.6. Estimates of Test Length to Increase Reliability of the Analogy Test

Current Test

No. Items Needed for

Alpha =

Test Part No. Items Alpha .90 .80

Non-Semantic Visual 13 .67 59 26

Non-Semantic Words 9 .44 102 46

Semantic Visual 6 .42 76 34

Semantic Words 12 .52 98 44

Total No. Items 40 335 150

Estimated Test Time 190 minutes 85 minutes

104
FIGURES AND TABLES

Table 3.3.7. Item Analyses and Scale Reliabilities: Non-Semantic

Word Scale on the Classification Test (N=427)

Item Corrected ItemNumber Total Correlationa

145689

131416182122

.03.06.12.07.03.03.06.07.05.00.08.05

Alpha = .10

12 Items

aStandardized score

105
FIGURES AND TABLES

Table 3.3.8. Item Analyses and Scale Reliabilities: Semantic Word Scale on the Classification Test (N=427)

Item Corrected Rerun of Corrected Information-Processing

Number Item-Total Item-Total Corrected Item-Total

Correlation Correlation Correlationa

Total Screen Total Item

Views Response Time

2 .04 --- --- ---3 .21 .23 .48 .367 .20 .21 .40 .45

10 .22 .19 .42 .4011 .10 .13 .32 .5112 .16 .16 .41 .4515 .01 --- --- ---17 .22 .21 .41 .4819 .26 .28 .30 .5120 .28 .28 .20 .3223 .22 .23 .44 .43

Alpha = .46 Alpha = .49

11 Items 9 Items

Alpha = .82

18 Items

aStandardized score

106
FIGURES AND TABLES

Table 3.3.9. Item Analyses and Scale Reliabilities: Non-Semantic Visual Scale on the

Classification Test (N=427)

Item

Number

Corrected

Item-Total

Correlation

Rerun of Corrected

Item-Total

Correlation

Information-Processing

Corrected Item-Total

Correlationa

Total Total Item

Screen Response Time

Views

24272831333637383942434546

.10 .10 .34 .40.09 .10 .48 .60

.04 --- --- ---

.23 .24 .48 .51

.03 --- --- ---.02 --- --- ---

.10 .13 .68 .60.17 .18 .46 .52.15 .14 .29 .24.10 .07 .49 .51.15 .14 .51 .53.15 .17 .41 .46.25 .25 .52 .41

Alpha = .36 Alpha = .40 Alpha = .87

13 Items 10 Items 20 Items

aStandardized score

107
--- --- ---

--- --- ---

--- --- ---

--- --- ---

--- --- ---

--- --- ---

--- --- ---

FIGURES AND TABLES

Table 3.3.10. Item Analyses and Scale Reliabilities: Semantic Visual Scale on the Classification Test (N=427)

Item Corrected Rerun of Corrected Information-Processing

Number Item-Total Item-Total Corrected Item-Total

Correlation Correlation Correlationa

Total Total Item

Screen Response Time

Views

25 .14 .15 .46

26 .0629 .10b

30 .0532 .0634 -.0135 -.0840 .0541 .11 .09 .51

44 .13 .12 .37

Alpha = .18 Alpha = .23 Alpha = .69

10 Items 3 Items 6 Items

.35

.44.41

aStandardized scorebThe alpha was higher excluding this item.

108
FIGURES AND TABLES

Table 3.3.11. Distribution of Test Completion Times for the

Classification Test (N=427)

Test Completion Time

Percentile (Minutes)

Minimum

5th

25th50th75th95th

Maximum

Mean

SD

2.9

6.39.6

12.015.821.7

25.4

12.86

4.69

Table 3.3.12. Estimates of Test Length to Increase Reliability of the Classification Test

Current Test

No. Items Needed for

Alpha =

Test Part No. Items Alpha .90 .80

Non-Semantic Visual 10 .40 138 61

Semantic Visual 3 .23 92 41

Semantic Words 9 .50 83 37

Total No. Items 22 313 139

Estimated Test Time 138 minutes 63 minutes

109
FIGURES AND TABLES

Table 3.3.13. Planning/Thinking Ahead: Distribution of Total Number

Correct on the Letter Factory Test (Form A) (N = 441)

Percentile Number of Attempts

Minimum

5th

25th50th75th95th

Maximum

Mean

SD

0

3556677480

85

63.39

14.64

Table 3.3.14. Distribution of Number of Inappropriate Attempts to Place

a Box in the Loading Area on the Letter Factory Test (Form A) (N = 441)

Percentile Number of Attempts

Minimum

5th

25th50th75th95th

Maximum

Mean

SD

0

27

132599

448

26.78

46.27

110
FIGURES AND TABLES

Table 3.3.15. Recall from Interruption (RI) Score Analyses on the Letter Factory Test (Form A) (N = 405)

Pre-Interruption Sequence Scores Post-Interruption Sequence Scores Differen

No. of Items No. of Items

Within Within

Sequence Sequence Mean SD Composite Mean SD Mea

No.

4 16 14.5 1.49 16 13.7 2.08 -0.86 16 15.2 1.08 16 14.08 1.63 -1.18 20 19.15 1.65 20 17.69 2.49 -1.4

11 11 10.04 1.54 11 8.15 1.54 -1.8

Scale 63 58.89 4.57 63 53.62 5.87 -5.2

Statistics

Alph

Alpha = .79 4 items (sequence scores) Alpha = .73 4 items (sequence scores) (di

Table 3.3.16. Planning/Thinking Ahead: Reliability Analysis on the Letter Factory Test (Form A)

(N = 405)

No. of Items Mean Corrected Alpha if

Sequence No. Within For Each Sequence-Total Sequence

Sequence Sequence Correlation Deleted

234678

11

Alpha = .86

97

1313

8

12

9

7 items (composite scores)

2.81 .60 .853.60 .63 .857.20 .66 .847.80 .66 .844.89 .62 .856.39 .69 .843.37 .61 .85

111
--- ---

--- ------ ------ ------ ---

--- ---

--- ------ ------ ---

--- ---

FIGURES AND TABLES

Table 3.3.17. Situational Awareness (SA) - Reliability Analysis: Three Scales on the Letter

Factory Test (Form A) (N = 405)

Before Item

Deletion After Item Deletion

Sequence No. Corrected Corrected Alpha if Item

(Item No.) Item-Total Item-Total Deleted

Correlation Correlation

SA Level 1 2(1) .14 .14

2(2) .25 .244(1) -.094(2) .17 .216(1) .15 .186(2) .076(3) .18 .206(4) .13 .198(1) .028(2) -.128(3) .068(4) .10 .11

11(1) .17 .1711(2) .09

.41.36

.37.38.38.38.42

.39

Alpha = .31 14 items Alpha = .42 8 items

SA Level 2 1(1) .32 .33 .58

1(2) .30 .30 .581(3) .38 .41 .551(4) .20 .23 .605(1) .31 .35 .575(2) .26 .26 .595(3) .015(4) -.039(1) .27 .28 .599(2) .31 .34 .579(3) .15 .15 .619(4) .04

10(1) -.0110(2) .20 .18 .61

112
--- ---

FIGURES AND TABLES

10(3) .11 .12 .62

10(4) .09

Alpha = .53 16 items Alpha = .61 11 items

(Continued)

Table 3.3.17. Situational Awareness (SA) - Reliability Analysis: Three Scales on the Letter

Factory Test (Form A) (N = 405) (Continued)

Before Item

Deletion After Item Deletion

Sequence No. Corrected Corrected Alpha if Item

(Item No.) Item-Total Item-Total Deleted

Correlation Correlation

SA Level 3 2(3) .21 .11 .49

2(4) .05 --- ---3(1) .20 .24 .423(2) .21 .30 .423(3) .24 .33 .383(4) .18 .24 .424(3) .04 --- ---4(4) .01 --- ---7(1) .07 --- ---7(2) .00 --- ---7(3) .07 --- ---7(4) .04 --- ---

11(3) .11 .13 .4711(4) .17 .26 .42

Alpha = .32 14 items Alpha = .47 7 items

113
FIGURES AND TABLES

Table 3.3.18. Situational Awareness (SA) - Reliability Analysis: One Scale on the Letter Factory Test(Form A) (N = 405)

Corrected Item- Sequence No. Item Difficulty Total Alpha if Item (Item No.) Correlation Deleted

SA Level 1

SA Level 2SA Level 3

2(1) .47 .08 .69

2(2) .33 .29 .674(2) .45 .23 .676(1) .40 .17 .686(3) .47 .27 .676(4) .49 .15 .688(4) .38 .13 .68

11(1) .76 .20 .68

1(1) .67 .27 .671(2) .62 .28 .671(3) .45 .38 .661(4) .46 .17 .685(1) .48 .34 .665(2) .45 .28 .679(1) .45 .29 .679(2) .35 .39 .669(3) .17 .16 .68

10(2) .61 .24 .6710(3) .18 .11 .68

2(3) .45 .23 .673(1) .36 .22 .673(2) .09 .25 .673(3) .62 .34 .663(4) .37 .18 .68

11(3) .14 .11 .6811(4) .21 .10 .68

Note. Scale Statistics: Mean = 10.87 ; SD = 4.05; Alpha = .68 (26 items)

114
FIGURES AND TABLES

Table 3.3.19. Planning/Thinking Ahead: Distribution of Total Number

Correct on the Letter Factory Test (Form B) (N = 217)

Percentile Number of Attempts

Minimum

5th

25th50th75th95th

Maximum

Mean

SD

0

1628333638

40

30.77

7.29

Table 3.3.20. Distribution of Number of Inappropriate Attempts to Place

a Box in the Loading Area on the Letter Factory Test (Form B) (N = 217)

Percentile Number of Attempts

Minimum

5th

25th50th75th95th

Maximum

Mean

SD

0

025

1354

135

13.09

20.73

115
FIGURES AND TABLES

Table 3.3.21. Tests of Performance Differences Between LFT and Retest LFT (N = 184)

LFT Mean Score

Variable Test Retest R t-value

Planning & Thinking Ahead 20.72 22.42 .763 3.25***

Situational Awareness 4.70 4.75 .508 0.30

***p<.001Table 3.3.22. Distribution of Test Completion Times for the

Letter Factory Test (N = 405)

Percentile Test Completion Time

(Minutes)

Minimum 31.0

5th 41.0

25th 46.450th 50.975th 55.895th 64.9

Maximum 107.3

Mean 51.77

SD 7.63

Table 3.3.23. Proposed Sequence Length and Number of Situational Awareness Items for the

Letter Factory Test

Number SA Items

Sequence Length

(Minutes) Number Level 1 Level 2 Level 3

0.5 6 242.5 12 24 2

Note. These recommended sequences would provide a test time of 45 minutes (not including instructions) based on our exp

LFT.

116
FIGURES AND TABLES

Table 3.3.24. Distribution of Number Correct Scores on the Scan Test

(N = 429)

Percentile Number Correct Scores

Minimum

5th

25th50th75th95th

Maximum

Mean

SD

22

77

120128136146

158

123.73

21.78

Table 3.3.25. Scanning: Reliability Analyses on the Scan Test

With Change Itemsa Without Change Itemsa Without Any Part of Change Item

Alpha Alpha

Sequence No. Items Alpha No. Items Actual Expectedc No. Items Actual Expect

T1 61 .88 51 .88 .86 41 .89 .83T2 63 .84 54 .87 .87 45 .85 .79T3 61 .84 53 .84 .82 45 .83 .79T4 54 .88 45 .87 .85 35 .86 .80

aN = 151bN = 429c”Expected” alphas were calculated using the Spearman-Brown Formula to step down reliabilities based on a reduced numb

of “like” items.

117
FIGURES AND TABLES

Table 3.3.26. Distribution of Test Completion Times for the Scan Test (N = 429)

Percentile Test Completion Time (Minutes)

Minimum

5th

25th

50th

75th

95th

Maximum

Mean

SD

13.57

14.45

15.58

16.85

17.93

19.87

25.50

16.92

1.75

Table 3.3.27. Reliability Analyses on the Three Parts of the Planes Test

No. Items With

Test Section No. Items No. Examineesa Item-Total Alpha Correlations

< .100

Part 1 48 297 8 .76

Part 2 48 314 0 .84

Part 3 96 103 24 .80 a

This is the number meeting all inclusion criteria and having data for all items.

Table 3.3.28. Distribution of Test Completion Times for the Planes Test

Percentile Test Completion Time (Minutes)

Minimum

5th

25th

50th

75th

95th

Maximum

Mean

SD

14.1

20.0

25.0

28.3

30.6

34.6

39.5

27.73

4.42

118
FIGURES AND TABLES

Table 3.3.29. Generalizabilty Analyses and Reliability Estimates

error Reliabilities with number of trials =

Variable Trials T(rial) G(roup) T*G SSN(G) SSN*T 1 

Air Traffic Scenarios Test

SEPSKILL P1-2,T1-4 132.43 5.00 2.35 124.66 632.21 0.16 0.28 0.37 0.44 0.50 0.5

P2, T1-4 118.18 3.44 2.86 118.65 612.46 0.16 0.28 0.37 0.44 0.49 0.5

T1-T4 115.88 5.74 4.40 97.60 546.17 0.15 0.26 0.35 0.42 0.47 0.5

T2-T4 136.43 4.10 8.80 106.67 425.51 0.20 0.33 0.43 0.50 0.56 0.6

T3-T4 0.00 10.57 5.85 222.97 207.59 0.52 0.68 0.76 0.81 0.84 0.8

PRCSKILL P1-2,T1-4 12.84 3.93 0.00 267.24 343.24 0.44 0.61 0.70 0.76 0.80 0.8

P2, T1-4 9.45 1.82 0.00 292.95 257.54 0.53 0.69 0.77 0.82 0.85 0.8

T1-T4 8.94 1.61 0.02 312.14 194.76 0.62 0.76 0.83 0.87 0.89 0.9

T2-T4 9.16 0.53 0.00 270.96 171.84 0.61 0.76 0.83 0.86 0.89 0.9

T3-T4 5.69 0.15 0.00 242.15 119.67 0.67 0.80 0.86 0.89 0.91 0.9

EFFNCYA P1-2,T1-4 190.92 0.00 0.58 200.71 264.65 0.43 0.60 0.69 0.75 0.79 0.8

P2, T1-4 93.30 0.00 0.84 215.22 239.49 0.47 0.64 0.73 0.78 0.82 0.8

T1-T4 95.33 0.00 2.20 189.42 156.17 0.55 0.71 0.78 0.83 0.86 0.8

T2-T4 134.88 0.00 3.28 210.56 158.53 0.57 0.73 0.80 0.84 0.87 0.8

T3-T4 50.55 0.00 4.85 261.42 163.73 0.61 0.76 0.83 0.86 0.89 0.9

EFFNCYB P1-2,T1-4 3.96 0.10 0.19 53.49 104.38 0.34 0.51 0.61 0.67 0.72 0.7

P2, T1-4 3.77 0.00 0.25 63.64 87.20 0.42 0.59 0.69 0.74 0.78 0.8

T1-T4 5.13 0.00 0.02 84.13 73.87 0.53 0.69 0.77 0.82 0.85 0.8

T2-T4 6.79 0.00 0.22 100.32 69.45 0.59 0.74 0.81 0.85 0.88 0.9

T3-T4 12.13 0.00 0.00 142.24 57.40 0.71 0.83 0.88 0.91 0.93 0.9

Time Wall/Pattern Recognition Test

PRACCY T1-T3 0.36 2.51 0.51 84.27 39.85 0.68 0.81 0.86 0.89 0.91 0.9

T2-T3 0.19 4.04 0.17 92.34 35.03 0.72 0.84 0.89 0.91 0.93 0.9

PRSPEED T1-T3 10.77 1.76 0.32 65.59 27.05 0.71 0.83 0.88 0.91 0.92 0.9

T2-T3 3.59 2.21 0.46 80.35 22.40 0.78 0.88 0.91 0.93 0.95 0.9

TWACCY T1-T3 2.86 0.33 1.77 52.78 48.20 0.52 0.69 0.77 0.81 0.85 0.8

T2-T3 0.87 1.11 0.00 59.61 41.34 0.59 0.74 0.81 0.85 0.88 0.9

Variance Components:

2 3 4 5 6 

119

FIGURES AND TABLES 

Table 3.3.30. Correlation of Alternative ATST Composites with End-of-Day Retest Measure

Correlation with Retest Measure: Correlation with Retest Measure

Composite SEPSKLL PRCSKLL EFFNCYA EFFNCYB Composite PRACC4 PRSPD4 

SEPSK1-6

SEPSK2-6

SEPSK3-6

SEPSK4-6SEPSK5-6SEPSK6SEPSKWT

PRCSK1-6 0.17

PRCSK2-6 0.18PRCSK3-6 0.18

PRCSK4-6 0.16

PRCSK5-6 0.14

PRCSK6 0.17PRCSKWT 0.18

EFFNA1-6 0.09EFFNA2-6 0.08

EFFNA3-6 0.10

EFFNA4-6 0.09

EFFNA5-6 0.13

EFFNCA6 0.17EFFNAWT 0.10

EFFNB1-6 -0.07

EFFNB2-6 -0.07

EFFNB3-6 -0.09

EFFNB4-6 -0.11

EFFNB5-6 -0.16

EFFNCB6 -0.18

EFFNBWT -0.14

0.080.14

0.23

0.21

0.370.38

0.26

0.370.41

0.47

0.28

0.45 0.16 -0.10 PRACC1-3 -0.03 0.40 0.16 -0.16 PRACC2-3 -0.08 

0.37 0.19 -0.09 PRACC3 -0.13 

0.33 0.14 -0.01 PRACCWT -0.12 

0.19 0.25 0.100.15 0.21 0.08 PRSPD1-3 -0.12 0.58 

0.31 0.21 0.02 PRSPD2-3 -0.19 0.55 

PRSPD3 -0.15 0.51 

0.64 0.20 -0.03 PRSPDWT -0.16 0.67 0.71 0.18 -0.11

0.75 0.21 -0.14 TWACC1-3 0.21 -0.01 0.76 0.18 -0.16 TWACC2-3 0.22 -0.04 0.77 0.10 -0.20 TWACC3 0.08 -0.02 

0.76 0.11 -0.23 TWACCWT 0.18 -0.07 

0.77 0.20 -0.18

0.40 0.59 0.21 Note:

0.34 0.64 0.22 SETSKILL - Avoiding separation errors/crashes

0.27 0.67 0.22 PRCSKILL - Avoiding procedural errors

0.24 0.69 0.22 EFFNCA - % reaching destination

0.19 0.73 0.27 EFFNCB - Speed in reaching destination

0.16 0.71 0.18 PRACCY - % correct pattern classificaitons

0.24 0.73 0.24 PRSPD - Speed in making correct pattern judgmen

TWACCY - Nearness to the Time Wall

-0.14 0.31 0.46 xxxxxxm-n - Mean over trials m through n for xxx

-0.22 0.35 0.54 xxxxxxWT - Weighted mean over trials for xxxxx

-0.23 0.38 0.60-0.18 0.37 0.61-0.23 0.26 0.63

-0.21 0.24 0.61

-0.21 0.33 0.64

121
FIGURES AND TABLES 

Table 3.3.31. Time Distributions for Current Tests

(Based on Cases Completing All Trials)

Instruction Time Total Time

Test

5th Median 95th 5th Median 95th

AT 2.5 5.5 9.5 88.6 91.7 97.0

TW 1.1 2.8 4.8 20.8 24.8 29.9

122