a mean of 70 and maximum of 100, known as the
Transmuted Composite (TMC). About half of all
applicants were expected to score at or above the mean
(Rock, Dailey, Ozur, Boone, & Pickrel, 1984a). Ap-
plicants with 3 years of general experience, 4 years of
college, or any combination of education and experi-
ence equating to 3 years of general experience and
without prior aviation experience, were required to
earn a TMC of at least 75.1 to qualify for employ-
ment. Applicants not meeting these criteria were
ineligible for employment as controllers. TMC was
used in our analyses as the measure of candidate
aptitude, as it provided a measure of ability unad-
justed for previous experience and/or military service.
Descriptive statistics for the TMC scores are pre-
sented in Table 2 for the research sample.
Predictor. The Computer Use Survey (CUS), was
developed to assess prior usage and familiarity with
personal computers and was used as the predictor in
this study. The CUS (see Appendix) was modeled
after a survey used by researchers at the NASA-Ames
Research Center (S. Hart, personal communication,
February 8, 1991), with 3 major blocks of items. First,
respondents indicated the frequency of use of per-
sonal computers and software (8 items) on a 5-point
Likert-type scale: 0 = Never; 1 = Seldom; 2 = Sometimes;
3 = Frequently; 4 = Often. Second, students indicated
ownership of and training on the use of personal
computers (9 items): 0 = No; 1 = Yes. Finally, the study
participants indicated their proficiency on various

kinds of video games (9 items) on the following 5-
point Likert-type scale: 0 = Never play; 1 = Poor, 2 =
Average; 3 = Very good; and 4 = Superior. Only the 9
items relating to video game experience were used in
this analysis. Reliability (a = .82) and principal com-
ponent analyses suggested that the 9 CUS items relat-
ing to video games were related in such a way as to
support combining them into 1 composite score. The
values were summed and averaged to create the pre-
dictor variable, Video Game Average (VIDEO).
Criterion. The criterion in the analysis was perfor-
mance on the Air Traffic Scenario Test. The ATST
(Figure 1), required the subject to control aircraft
within a simplified synthetic airspace, directing them
to their destinations according to a small set of rules.
There were 6 destinations: 4 outbound gates, A, B, C,
and D; and 2 airports, Eand F. The direction of travel,
speed, and altitude of the aircraft, represented by
small arrows next to the quasi - data blocks, were
controlled by mouse. Three alphanumeric characters
comprised the quasi - data blocks: first, aircraft speed
(Slow, Medium, Fast); second, altitude (1 = Lowest,
through 4 = Highest); and third, destination. The
orientation of the aircraft arrow indicated its current
direction of flight. An open circle in an upper corner
of the data block indicated an aircraft waiting to be
activated by (“handed off to”) the subject. The large
arrow in the lower right-hand corner of the screen
indicated the landing direction at airports E and F,
while the bottom horizontal bar icon represented the

TABLE 2
Sample descriptive statistics of predictors and criteria.

Sample (N = 404)
M sD
TMC 93.47 21.67
VIDEO 14.42 6.75
ATSAFE 20.81 21.67
ARVDELAY 32.53 36.37
HNDDELAY 57.81 17.34

eee


seer ew we Se

Kirk, R. E. (1968). Experimental design: Procedures for the
behavioral sciences, Belmont, Calif: Brooks/Cole,

Koson, D., Kitchen, C., Kochen, M., & Stodolosky, D.
(1970). Psychological testing by computer: Effect
of response bias. Educational and Psychological Mea-
surement, 30, 803-10.

Lautenschlager, G. J., & Flaherty, V.L. (1990). Com-
puter administration of questions: More desirable
or more social desirability? Journal of Applied Psy-
chology, 75, 310-14.

Lee, Jo A., (1986). The effects of past computer experi-
ence on computerized aptitude test performance.
Educational and Psychological Measurement. 4G,
727-33.

Lee, J., Moreno, K. E., & Sympson, J. B. (1986). The
effects of mode of test administration on test
performance, Educational and Psychological Mea-
surement, 46, 467-74,

Lohman, D. F. (1979). Spatial ability: A review and re
analysis of the correlational literature (Tech. Rep.
No. 8). Stanford, CA: Stanford University, School
of Education.

Lukin, M. E., Dowd, T., Plake, B. S., & Kraft, R. G.
(1985). Comparing the computerized MMPI, Jour-
nal of ‘Personality Assessment, 13, 407-12.

Lushene, R. E., O'Neil, H. F, & Duna, T. (1974).
Equivalent validity of a completely computerized
MMPI. Journal of Personality Assessment, 38, 353-61.

Manning, C. A. (1991). Procedures for selection of air
traffic control specialists. In H. Wing & C. A.
Manning (Eds.), Selection of air traffic controllers:
Complexity, requirements, and public interest, pp.
13-22. (DOT/FAA/AM-91/9), Washington, DC:
Federal Aviation Administration Office of Avia.
tion Medicine. NTIS #AD A238267,

Martin, C. L., & Nagao, D. H. (1989). Some effects of
computerized interviewing on job applicant re-

sponses. Journal of Applied Psychology, 74, 72-80.

Mazzeo, J., & Harvey, A. L. (1988). The equivalence of
scores from automated and conventional educational
and psychological tests: A review of the literature.
(College Board Report No. 88-8/Educational Test-
ing Service Research Report No. 88-21). New
York: College Entrance Examination Board.

Mead, A. D., & Drasgow, F. (1993). Equivalence of
computerized and paper-and-pencil cognitive abil-
ity tests: A meta-analysis. Psychological Bulletin,
114, 449-59.

Miller, R. G., Jr. (1966). Simultaneous statistical infer-
ence. New York: McGraw-Hill, 1966,

Potosky, D. (1996). Belief about computers and their
subsequent effects on response distortion. Unpub-
lished doctoral dissertation, Rutgers, the State Uni-
versity of New Jersey,

Rabbitt, P.; Banerji, N.; Szymanski, A. (1989), Space
Fortress as an IQ test? Predictions of learning and
of practised performance in a complex interactive
video-game. Special Issue: The Learning Strate-
gies program: An examination of the strategies in

skill acquisition. Acta Psychologica. 71, 243-57,

Rock, D. B., Dailey, J. T., Ozur, H., Boone, J. O., &
Pickrel, E. W. (1984a). Validity and utility of the
ATC experimental test battery. Study of Academy
trainees, 1982. In S. B. Sells, J. T. Dailey, & E. W.
Picktel (Eds.). Selection of air traffic controllers (pp.
459 - 502). (DOT/FAA/AM-84/2), Washington,
DC; Federal Aviation Administration Office of
Aviation Medicine. NTIS#ADA147765,

Rosenfeld, P., Giacalone, R. A., Knouse, S. B., Doherty,
L. M., Mitchell-Vicino, S., Kantor, J., & Greaves,
J. (1991). Impression management, candor, and
microcomputer-based organizational surveys: An
individual differences approach. Computers in
Human Behavior, 7, 23-32.

Sells, S. B., Dailey, J. T., 8¢ Pickrel, E. W. (Eds.) (1984).
Selection of air traffic controllers. (DOT/FAA/AM-
84/2). Washington, DC: Federal Aviation Ad-
ministration Office of Aviation Medicine.
NTIS#ADA147765,

Vansickle, T. R., Kimmel, C., & Kapes, J. T. (1989).
Test-retest equivalency of the computer-based and
paper-and-pencil versions of the Strong-Campbell
Interest Inventory, Measurement and Evaluation
in Counseling and Development, 22, 88-93.

Wilson, F. R., Genco, K. T., & Yager, G. G. (1985).
Assessing the equivalence of Paper-and-pencil vs,
commputerized tests: Demonstration ofa promis-

ing methodology. Computers in Human Behavior,
1, 265-75,


Thursday. The three criterion measures used for this
study; Air Traffic Safety (ATSAFE), Arrival Delay
(ARVDELAY), and Hand-off Delay (HNDDELAY),
were computed using data from the final 6 ATST “for
grade” testing sessions. A lower score represented
better performance on each criterion measure. For
example, higher scores for ATSAFE indicated that
the subject had more crashes or committed more
separation and procedural errors than a subject
with a lower ATSAFE score. Similarly, higher scores
on ARVDELAY and HNDDELAY indicated more
delay time than a lower score.

Procedure

Multiple regression analysis was used to test the
null hypothesis that the predictor variable Video
Game Experience (VIDEO) had no incremental va-
lidity over Cognitive Aptitude scores (TMC) in the
prediction of performance of the 3 criterion measures
of ATST performance. The alternative hypothesis was
that ATST error and delay scores decreased as video
game experience increased, after taking into account
differences in general aptitude for the ATCS occupa-
tion. The amount of variance in ATSAFE accounted
for by TMC was estimated in the first step of the
analysis. Next, the performance measure was regressed
on the predictor variable (VIDEO) in a hierarchical
analysis to assess the incremental validity of the pre-
dictor. These steps were repeated using performance
on ARVDELAY as the criterion and again repeated
with HNDDELAY as the criterion.

As the analyses were performed on the same sample,
the alpha level was adjusted to control the Type] error
rate, using the Bonferroni procedure (Dunn, 1961;
Kirk, 1968; Miller, 1966). Following this adjustment,
only results that were <.014 were considered significant.

RESULTS

Air Traffic Safety (ATSAFE)

The correlations between ATSAFE, TMC, and
VIDEO are presented in Table 3. The correlation
between ATSAFE and the predictor (VIDEO) was
low but significant (r = .12, N = 404, p< .05). The
correlation between ATSAFE and the cognitive

aptitude measure (TMC) was significant (r= .19, N=
404, ps .01). Results from the regression analysis for
ATSAFE are presented in Table 4. The performance
variable (ATSAFE) was regressed on the Transmuted
Composite score (TMC) and the predictor variable
(VIDEO) using a forward hierarchical stepwise vari-
able selection strategy. TMC was entered first into the
regression (R’ = .037, F = 15.60, p< .001), and
accounted for 4% of the variance in ATSAFE. Self-
reported experience on various video games (VIDEO)
accounted for an additional 1.3% of the variance in
ATSAFE (AR? = .013, F = 10.674, ps .001).

Arrival Delay (ARVDELAY)

The correlation between ARVDELAY and
(VIDEO) was r= .20, (N = 404, ps .01). The corre-
lation between ARVDELAY and TMC was lower (r=
.15, N = 404, ps .01). Results from the regression
analysis for ARVDELAY are presented in Table 5.
The criterion (ARVDELAY) was regressed on the
Transmuted Composite score (TMC) and the predictor
variable VIDEO using a forward hierarchical stepwise
variable selection strategy. TMC was entered first into
the equation and accounted for 2% of the variance in
ARVDELAY (A’=.023, F= 8.945, ps.05). Self-reported
experience on various video games (VIDEO) accounted
for an additional 3.6% of the variance in ARVDELAY
(AR’ = .036, F = 12.315, ps 001).

Hand-off Delay (HNDDELAY)

The correlation between HNDDELAY and the
predictor (VIDEO) was r = .30, (N = 404, ps .01).
The correlation between HNDDELAY and TMC was
lower (r = .16, N = 404, ps .01). Results from the
regression analysis for HNDDELAY are presented in
Table 6. The criterion variable (HNDDELAY) was
regressed on the Transmuted Composite score (TMC)
and the predictor variable (VIDEO) using a forward
hierarchical stepwise variable selection strategy. TMC
was entered first into the equation and accounted
for 3% of the variance in HNDDELAY (R? = .026,
F = 10.733, ps .01). Self-reported experience on
various video games (VIDEO) accounted for an
additional 9% of the variance in HNDDELAY
(AR? = .087, F = 25.488, ps .001).

we


i)
ql

iH

I
iW
ul)

FIGURE 1
Air Traffic Scenario Test
Boundary
1A Tf
HEADING
— «— Direction
Airport ~ as F3D Y control icon
LEVEL
L 4 3 <— Altitude
Gate—+ D S4A A B 2 control icon
F «<— Speed
M2F=> F bd control icon
Aircraft —j> Mors <— < Landing
Z. \ ict H direction
Data block Hand of Separation
! distance icon

Figure 1. AIR TRAFFIC SCENARIO TEST (ATST) SCREEN. The boundary encloses a
simplified airspace, with 4 outbound gates, A, B, C, and D, and 2 airports, E and F. The
aircraft and direction of flight are represented by the arrows adjacent to a data block. The
alphanumeric data block indicates aircraft speed (Slow, Medium, or Fast) and altitude (1
= lowest, 4 = highest). Aircraft waiting to be handed off are tagged with a small open circle

in the upper right hand corner of the data block. Aircraft are controlled with a mouse. First,
the subject clicks on an aircraft, and then clicks on the appropriate element of either the
direction control, altitude control, or speed control icons to change that flight parameter.
Subjects are reminded of the required landing direction at airports and minimum horizontal
separation distance by the landing direction and separation distance icons respectively.

minimum lateral separation distance. Aircraft landed
at airports E and F at the lowest altitude and slow
speed in the required direction; aircraft exited gates A,
B, C, and D at the fastest speed and highest altitude.
A difference in altitude between any 2 aircraft was
considered adequate separation; aircraft at the same
altitude had to be separated by at least 5 nautical
miles, as represented by the separation icon. In addi-
tion, all aircraft had to be separated from the airspace
boundary by at least 5 nautical miles. Error counts
associated with separation violations and crashes were
obtained and summed with procedural errors such as
exiting agate at the wrong altitude to create an overall
errorscore (ATSAFE). The system automatically com-
puted the difference between the actual time to reach
destination for each aircraft and the time required for

the optimum flight path, as determined by the system
software. Summed across aircraft, this arrival delay
(ARVDELAY) provided a measure of subject effi-
ciency in routing aircraft and managing the synthetic
airspace. In addition, the system computed the differ-
ence between the time of appearance for an aircraft as
ahand offand the time of acceptance. Averaged across
new aircraft appearing in the scenarios, this hand off
delay (HNDDELAY) provided a measure of subject
efficiency in accepting new aircraft and managing
workload. Performance feedback on these measures
was provided to subjects at the end of each of 20
practice scenarios. The subjects for this study com-
pleted the 20 ATST practice sessions over 2.5 days
(Monday afternoon through Wednesday), followed
by the final 6 ATST “for grade” testing sessions on


DISCUSSION

Self-reported experience on video games was found
to be significantly related to ARVDELAY and
HNDDELAY at the adjusted significance level of ps
-014. These results suggested that those persons with
video game experience were more efficient in han-
dling hand-offs and the routing of aircraft in such a
way as to minimize arrival delays. In short, those with
video game experience were faster.

Atfirst glance, these results seem at odds with those
reported in the initial ASI validation studies. How-
ever, the ASI findings related to the effects of com-
puter and video game experience on composite scores
in predicting screen success. The current study dealt
with video game experience on ATST scores, specifi-
cally. Moreover, delay scores were not considered in
the scoring equation at the time of the original valida-
tion studies. None the less, scores earned on the ATST
were the most heavily weighted components of the
PTS composite, and these results suggest that further
research is needed to investigate any moderating ef-
fects these findings may have on composite scores.
That research might include delay.

Additionally, it should be noted that the criterion
measures were computed from data obtained on the
last day of screening. A considerable amount of learn-
ing had taken place by this time. We suspect that a
much larger effect size might have been noted during
the early learning phases, but our concern here was
with the effect upon composite scores. Future re-
search could investigate the impact of prior video
game experience on learning curves and strategies
used in the work sample test.

REFERENCES

Aerospace Sciences, Inc. (1991). ATCS PTS. Preliminary
Validation: Final teport. (Final report delivered
under contract DTFA01-90-01034). Washing-
ton, DC: Federal Aviation Administration Office
of the Deputy Administrator.

Bliss, J; Kennedy, R. S; Turnage, J; Dunlap, W. (1991).
Communality of videogame performances with track-
ing tasks. Perceptual & Motor Skills, Z3(1), 22-30.

Booth-Kewley, S., Edwards, J. E., & Rosenfeld, P.
(1992). Impression management, social desirabil-
ity, and computer administration of attitude ques-
tionnaires: Does the computer make a difference?

Journal of Applied Psychology, 77, 562-566.

Broach, D., & Aul, J.C. (1993). Analysis of the air traffic
control specialist (ATCS) occupation using the Posi-
tion Analysis Questionnaire (PAQ). Unpublished
manuscript.

Brokaw, L. D. (1984). Early research on controller
selection. In S.B. Sells, J.T. Dailey, & E.W. Pickrel
(Eds.), Selection of air traffic controllers, pp. 39-78.
(DOT/FAA/AM-84/2). Washington, DC: Fed-
eral Aviation Administration Office of Aviation
Medicine. NTIS#ADA147765.

Burke, M. J. (1992). Computerized psychological test-
ing: Impacts on measuring predictor constructs
and future job behavior. In N. Schmitt, W. C.
Borman, & Associates (Eds.), Personnel selection in
organizations. San Francisco: Jossey-Bass.

Collins, W.E., Booone, J. O., & VanDeventer, A. D.
(1984). The selection of air traffic control special-
ist: Contributions by the Civil Aeromedical Insti-
tute. In S.B. Sells, J. T. Dailey, & E. W. Pickrel
(Eds.), Selection of air traffic controllers. pp. 79 -
112. (DOT/FAA/AM-84/2). Washington, DC:
Federal Aviation Administration Office of Avia-
tion Medicine. NTIS#ADA147765.

Dorval, Michel, Pepin, Machel (1986). Effect of playing
a video game on a measure of spatial visualization.

Perceptual & Motor Skills. 62(1), 159-162.

Dunn, O.J. (1961). Multiple comparisons among means.
Journal of the American Statistical Association, 56,
52-64, (1961).

Finegan, J. E., & Allen, N. J. (1 994). Computerized and
written questionnaires: Are they equivalent? Com-
puters in Human Behavior, 10, 483-496.

Human Technology, Inc. (1991). Cognitive task analysis
of en route air traffic control: Model extension and
validation. Volume I: Report. (Final report deliv-
ered under Office of Personnel Management con-
tract OPM-87-9041). Washington, DC: Federal
Aviation Administration Office of Training and
Higher Education.

King, W. C., & Miles, E. W. (In press). A quasi-
experimental assessment of the effect of computer-
izing non cognitive pencil-and-paper measure-
ments: A test of measurement equivalence. Journal

of Applied Psychology,


DOT/FAA/AM-97/4 The Effects of Video Game -

- Experience on Computer-Based
Office of Aviation Medicine : Air Traffic Con ‘oller Specialist,
Washington, D.C.20591. © Air Traffic Scenario Test Scores

Willie C, Young
Dana Broach
William L. Farmer

Civil Aeromedical Institute.
Federal Aviation Administration ©
Oklahoma City, Oklahoma 73125

February 1997 Suh rns 2 Suge

Final Report

This document is available to the public
through the National Technical Information
Service, Springfield, Virginia 22161.

U.S. Department Mog
of Transportation

Federal Aviation: -
Administration



APPENDIX A

COMPUTER USAGE SURVEY

The purpose of this survey is to assess your level of experience on computers and video games. Please read each
question carefully and mark the response that best reflects your experience on the FAA General Answer Sheet.
Your response to each question represents a critical aspect of this research.

For questions 1-8, use the following scale to estimate how many times in the past year you have done the
following:

PNAYRYONS

A = Never
B = Seldom
C = Sometimes

D = Frequently
E = Often

Used a typewriter?

Used a computer?

Used a computer for writing or word processing?

Used a computer for developing budgets?

Used computer programming languages (e.g. BASIC, FORTRAN, or C-Language)?
Played home computer games (software games for a personal computer)?

Played home video games (hook up to a television set, such as Nintendo or Atari)?
Played video arcade games (stand-alone units that usually cost a quarter)?

For questions 9-17, indicate your answer using the following scale:

10.
11.
12.
13.
14,

A= Yes
B=No

Have you ever had a course in typing?

Have you ever had a course on the use of a computer?

Have you ever had a course in word processing on a computer?
Have you ever had a course on computer programming?

Have you ever had a course on computer repair?

Do you own a typewriter?

(continued on next page)

Al


eee _-—hUhT

~~ ee US UE

TABLE 3

Correlations between cognitive aptitude (TMC), predictor (VIDEO),

performance criteria

and the

TC VIDEO ATSAFE ARVDELAY HNDDELAY

TMC

VIDEO 044

ATSAFE 193" = .123*

ARVDELAY 148%" .196* .032

HNDDELAY 161" 301" 215" .295***

TABLE 4

Results of stepwise hierarchical
regression analysis on ATSAFE variable

Model R2 AR2 AF F

TMC .037 15.60***
VIDEO § .050 = .013 5.57* 10.67**

TABLE 5

Results of stepwise hierarchical regression
analysis on ARVDELAY variable

Model R2 AR? AF F

TMC 023 8.95**
VIDEO .058 036 15.37" 12.32***

TABLE 6

Results of stepwise hierarchical regression analysis
on HNDDELAY variable

Model = R2 AR2 AF F
TMC 026 10.73"
VIDEO 113 .087 39,22" 25.49"

*ps 05, “*p<.01, p<

.001


|
I

earned on the work sample test is unknown. The
results of the previous study (ASI, 1991) suggested
that the composite score earned on the PTS was an
equally valid predictor of FAA Academy scores for
persons with little or much prior computer or video
game experience. However, the direct relationship of
prior computer or video game experience to the final
scores earned on the ATST was not investigated by
ASI.

The purpose of this study, therefore, was to extend
and refine the earlier ASI analysis, by evaluating the
incremental validity of prior video game experience
over that of general aptitude as a predictor of work
sample test scores. A finding of no incremental valid-
ity for prior video game experience as a predictor of
ATST scores would be supportive of the earlier ASI
study involving composite PTS scores. However, a
finding that video game experience did have incre-
mental validity would indicate thata careful re-exami-
nation of the ASI-moderated validity analysis, in
terms of the individual scores comprising the PTS
composite, is warranted.

METHOD

Sample

The subjects for this study were 404 air traffic
control students who entered the FAA Academy ATCS
Nonradar Screen in March and April of 1991. Sample
demographic characteristics are presented in Table 1.
The sample consisted primarily of non-minority males.

Measures

Aptitude score. The written ATCS aptitude tests
comprising the initial hurdle in the ATCS selection
process were: (a) the Multiplex Controller Aptitude
Test (MCAT) and (b) the Abstract Reasoning Test
(ABSR). The development, psychometric characteris-
tics, and validity of these written ATCS aptitude tests
have been extensively described elsewhere (Brokaw,
1984; Collins, Boone, & VanDeventer, 1984; Man-
ning, 1991; Sells, Dailey, & Pickrel, 1984). Scoring of
the tests was done by summing the MCAT (weighted
.8) and ABSR (weighted .2) scores. The resulting
weighted scores were then transformed to a score with

TABLE 1
Sample demographic characteristics

SEX
(N= 404)
Male Female

MEAN AGE 26.5 26.3
EDUCATIONAL LEVEL

% High School Only 13.9 10.4
% Some College 44.3 41.3
% College Degree 417 48.3
MINORITY GROUP

% Minority 19.0 15.0
% Non-Minority 81.0 85.0



Tue Errects oF VIDEO GAME EXPERIENCE ON COMPUTER-BASED AIR TRAFFIC
CoNTROLLER SPECIALIST, AIR TRAFFIC SCENARIO TEST SCORES

INTRODUCTION

The FAA undertook a major review of its Air
Traffic Control Specialist (ATCS) selection and train-
ing programs in 1990 to address agency and applicant
costs and other concerns. Three major ATCS selec-
tion policy goals were identified for the project: (1)
reduce the cost of the ATCS selection process, (2)
maintain the validity of the ATCS selection system,
and (3) support agency cultural diversity goals. The
first step toward achieving these goals was to develop
and validate a test battery to replace the 9-week FAA
Academy Nonradar ATCS Screen that serves as the
second major hurdle in the agency’s multiple hurdle
selection process (Broach & Brecht - Clark, 1993).

Development of the new Pre-Training Screen (PTS)
test battery began in late 1990 by reviewing available
information about the cognitive requirements of the
ATCS job. As described in one cognitive task analysis,
controllers attend to multiple information sources,
assess and integrate the data, develop and prioritize
plans of action, and implement those plans under
time pressure while maintaining situational awareness
(Human Technology, Inc., 1991). To assess the cog-
nitive and sensory attributes required to perform
these job functions, the PTS test battery was devel-
oped by Aerospace Sciences, Inc. (ASI). Two com-
puter-administered information processing tests were
designed to dynamically assess cognitive attributes,
such as spatial reasoning, short-term memory, move-
ment detection, pattern recognition, and attention
allocation (ASI, 1991). In addition, the Air Traffic
Scenario Test (ATST), alow-fidelity radar simulation
of air traffic control vectoring and separation tasks,
was developed as a computer administered work
sample.

The ATST (Figure 1), is a computerized work
sample test that was developed by 4 subject matter
experts with more than 30 years of air traffic contro!
experience (ASI, 1991). The task requires the subject
to control aircraft within a simplified synthetic airspace,

directing them to their destinations according to a
small set of rules. The ATST runs on a 80386 class
personal computer (PC) with a standard 640 x 480
16-color VGA display. Because the ATST was a PC -
based application with a strong resemblance to a video
game, concern was raised that prior video game expe-
rience might give some applicants an advantage over
others in obtaining better scores. No data were avail-
able to evaluate this concern, as the relationship of
prior computer or video game experience to perfor-
mance on computerized work samples has not been
investigated. Previous research on computerized tests
has focused on topics such as the equivalence of
computerized to paper-and-pencil formats (Finegan
& Allen, 1994; King & Miles, in press; Lee, Moreno,
& Sympson, 1986; Lukin, Dowd, Plake, & Kraft,
1985; Lushene, O’Neil, & Dunn, 1974; Mazzeo &
Harvey, 1988; Mead & Drasgow, 1993; Rabbit,
Banerji, & Szymanski, 1989; Vansickle, Kimmel, &
Kapes, 1989; Wilson, Genco, & Yager, 1985), faking
on computerized tests of non-cognitive constructs
such as personality and biodata (Booth-Kewley,
Edwards, & Rosenfeld, 1992; Burke, 1992; Koson,
Kitchen, Kochen, & Stodolosky, 1970; Lautenschlager
& Flaherty, 1990; Martin & Nagao, 1989; Potosky,
1996; Rosenfield, Giacalone, Knouse, Doherty,
Mitchell-Vicino, Kantor, & Greaves, 1991), the mod-
erating effects of prior computer experience on scores
earned on computerized versions of traditional achieve-
ment or power tests (Lee, 1986), and the effects of
practice with video games on individual difference
tests for constructs, such as spatial ability (Dorval &
Pepin, 1986; Lohman, 1979). The effects of com-
puter or video game experience on work sample scores
have not been systematically investigated. Aerospace
Sciences, Inc. (ASI, 1991), in the course of validating
the PTS, found that prior computer or video game
experience did not moderate the validity of the com-
posite PTS score as a predictor of performance in
initial ATCS training at the FAA Academy. However,
the relationship of prior computer experience to scores


Technical Documentation Page

1, Report No. 2. Government Accession No. 3. Recipient's Catalog No.
DOT/FAA/AM-97/4

4. Title and Subtitle 5. Report Date

The Effects of Video Game Experience on Computer-Based Air Traffic February 1997

Controller Specialist, Air Traffic Scenario Test Scores

6. Performing Organization Code

7. Author(s) 8, Performing Organization Report No.
Young, W.C., M.Ed.; Broach, D., Ph.D.; & Farmer, W.L., M.S.

9. Performing Organization Name and Address 10, Work Unit No. (TRAIS)

FAA Civil Aeromedical Institute
P.O. Box 25082

Oklahoma City, OK 73125 11. Contract or Grant No.

12. Sponsoring Agency Name and Address 13. Type of Report and Period Covered
Office of Aviation Medicine

Federal Aviation Administration
800 Independence Avenue, S. W.
Washington, DC_20591

15. Supplementary Notes
This research was conducted under task AM-92-B-HRR-509

16. Abstract

The FAA is currently using the Air Traffic Scenario Test (ATST) as a major portion of its selection process. Because the ATST
is a PC - based application with a strong resemblance to a video game, concern has been raised that prior video game experience
might have a moderating effect on scores. Much of the previous research in this atea is associated with topics such as the mod-
erating effects of prior computer experience on scores earned on computerized versions of traditional achievement or power
tests (Lee, 1986), and the effects of practice on video games on individual difference tests for constructs such as spatial ability
(Dorval & Pepin, 1986; Lohman, 1979). The effects of computer or video game experience on work sample scores have not
been systematically investigated. The purpose of this study was to evaluate the incremental validity of prior video game experi-
ence over that of general aptitude as a predictor of work sample test scores.

The Computer Use Survey was administered to 404 air traffic control students who entered the FAA ATCS Nonradar Screen.
The resultant responses from this survey related to video games were summed and averaged to create the predictor (VIDEO).
Three criterion measures derived from the ATST, (ATSAFE, ARVDELAY, HNDDELAY) were regressed on the cognitive
aptitude measure that serves as the initial selection screening test and the predictor (VIDEO). Self-reported experience on video
games was found to be significantly related to ARVDELAY and HNDDELAY, accounting for an additional 3.6% of the vari-
ance in ARVDELAY, and accounting for an additional 9% of the variance in HNDDELAY. The results suggested that those
persons with video game experience were more efficient at hand-offs and routing aircraft. Future research is recommended to
investigate the effect of prior video game experience on learning curves and strategies used in the work sample test.

17. Key Words 18. Distribution Statement
Video games; Selection; Computer; Screen; Hierarchical | Document is available to the public through the
regression; Air Traffic Scenario Test National Technical Information Service,
Springfield, VA 22161
19, Security Classif. {of this report) ] 20. Security Classif. (of this page) | 21. No. of Pages 22. Price
Unclassified Unclassified 13

Form DOT F 1700.7 (8-72)

Reproduction of completed page authorized