Effects of Video Weather Training 

Products, Web-based Preflight Weather 

Briefing, and Local Versus Non-Local 

Pilots on General Aviation Pilot Weather 

Knowledge and Flight Behavior, Phase 1

William Knecht1

Jerry Ball1

Michael Lenz2

1

Civil Aerospace Medical Institute

Federal Aviation Administration 

Oklahoma City, OK 73125

2

FAA Headquarters

Washington, DC 20591

January 2010

Final Report

DOT/FAA/AM-10/1

Office of Aerospace Medicine

Washington, DC 20591

OK-10-0077
NOTICE

This document is disseminated under the sponsorship 

of the U.S. Department of Transportation in the interest 

of information exchange. The United States Government 

assumes no liability for the contents thereof.

___________

This publication and all Office of Aerospace Medicine 

technical reports are available in full-text from the Civil 

Aerospace Medical Institute’s publications Web site: 

www.faa.gov/library/reports/medical/oamtechreports
i 

Technical Report Documentation Page 

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No. 

DOT/FAA/AM-10/1 

4. Title and Subtitle 5. Report Date 

Effects of Video Weather Training Products, Web-Based Preflight Weather January 2010 

Briefing, and Local Vs. Non-Local Pilots on General Aviation Pilot Weather 

Knowledge and Flight Behavior, Phase 1 

6. Performing Organization Code 

 

7. Author(s) 8. Performing Organization Report No. 

Knecht WR,1 Ball J,1 Lenz M2 

9. Performing Organization Name and Address 10. Work Unit No. (TRAIS) 

 

11. Contract or Grant No. 

1

FAA Civil Aerospace Medical Institute 

P.O. Box 25082 

Oklahoma City, OK 73125 

2

FAA Headquarters 

800 Independence Ave., SW 

Washington, DC 20591 

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered 

 

 

 

Office of Aerospace Medicine 

Federal Aviation Administration 

800 Independence Ave., S.W. 

Washington, DC 20591 14. Sponsoring Agency Code 

15. Supplemental Notes 

Work was accomplished under approved task AM-A-07-HRR-521 

16. Abstract 

This research has two main phases. Phase 1 investigated three major questions: 

1) Do video weather training products significantly affect general aviation (GA) pilot weather knowledge and flight 

behavior in marginal meteorological conditions? 

2) How are modern Web-based weather products used during GA preflight briefing? 

3) Do local Oklahoma GA pilots differ appreciably from other pilots in either weather knowledge or weather-related 

flight behavior? 

 Fifty GA pilots took a general weather knowledge pre-test, followed by exposure to either one of two weather training 

videos (the Experimental groups) or to a video having nothing to do with weather (the Control group). They next took a 

post-test to measure knowledge gain induced by the training product. Finally, they planned for, and flew, a simulated flight 

mission through marginal weather from Amarillo, TX, to Albuquerque, NM. 

Question 1: Few highly significant, direct effects were found for the two 90-minute video weather training products all 

by themselves. Follow-up multivariate modeling implied that a combination of higher pilot age, receiving either weather 

training product, and takeoff hesitancy could significantly, correctly predict 86.7% of diversions from deteriorating weather 

and 77.8% of full flight completions. However, we must conservatively conclude that weather knowledge and GA weather 

flying behavior are complex and unlikely to be profoundly changed by a single, brief training product. Phase 2 will address 

this issue. 

Question 2: The data-collecting emulation of www.aviationweather.gov suggested that mere time spent on preflight 

briefing was not a good predictor of either quality of preflight briefing or subsequent flight safety. Nonetheless, these data are 

just an opening look at what should eventually be a far more intensive study of modern weather briefing and its relation to 

flight safety. 

Question 3: No important differences were seen between local and non-local pilots. These findings imply that CAMI 

studies are likely to be generalizable to the national population of U.S. GA pilots. 

 

17. Key Words 18. Distribution Statement 

 

 

Weather, Training, Pre-Flight Briefing, Weather 

Knowledge, Flight Behavior 

Document is available to the public through the 

Defense Technical Information Center, Ft. Belvoir, VA 

22060; and the National Technical Information 

Service, Springfield, VA 22161 

19. Security Classif. (of this report) 20. Security Classif. (of this page) 21. No. of Pages 22. Price 

Unclassified Unclassified 39 

Form DOT F 1700.7 (8-72) Reproduction of completed page authorized 

iii 

Contents

Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Purpose of This Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Research Design, Assignment to Groups, and Order of Treatments . . . . . . . . . . . . . . . . . . . . . . . . 3

Primary Independent Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Secondary Independent Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Dependent Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Apparatus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

Advanced General Aviation Research Simulator (AGARS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Correction of Results for Familywise Error. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Preliminary Examination of Data (Frequency Distribution Normality, Outliers) . . . . . . . . . . . . . . 9

Overall Data Relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Specific Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

Modeling Flight Behavior. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Question 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Question 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Question 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

Implications of This Research for Pilot Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Appendix A: Web Preflight Briefing Screenshots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1

Appendix B: Sample Hypertext Markup Language (HTML) Code Used to Emulate the 

Aviationweather.gov Home Page in Microsoft Visual Studio. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B-1

Appendix C: Code-Behind for a Sample Page Emulation of Aviationweather.gov.. . . . . . . . . . . . . . . C-1

v 

Acknowledgments

Research question 1 was conceived and promoted by Scott Shappell and William Krebs. The authors gratefully 

acknowledge the invaluable assistance of Stanley Roberts, Manager, AFS-630, for providing item difficulties to 

FAA test questions. Janine King, Sally Glasgow, and Suzanne Thomas of Xyant Technology, Inc., and Tammy 

Harris, FAA, provided key scheduling and subject payment support. The authors extend grateful thanks to the 

pilot participants, without whom this study would not have been possible.

1 

Effects of Video Weather Training Products, Web-based

Preflight Weather Briefing, and Local Versus Non-Local Pilots

on G.A. Pilot Weather Knowledge and Flight Behavior, Phase 1

Introduction

Background 

Strategic motivation for this research. The term “adverse 

weather” involves multiple factors such as restricted 

visibility due to low cloud ceilings, fog, rain, snow, 

thunderstorms, or airframe icing. Adverse weather, is a 

perennial concern to general aviation (GA). Analyses of 

GA accidents from the 1970s-2000s show that, despite 

a relatively low incidence rate for weather-related accidents (4-5%, depending on data source and classification 

scheme), their fatality rate is 3-4 times higher than for 

other GA accident causes (Bazargan, 2005; Bud, Mengert, 

Ransom, & Stearns, 1997; NTSB, 1989; NTSB, 2005).

This is largely because weather accidents often involve 

flight into terrain or loss of control, which typically results 

in a high percentage of fatalities.

The role of training. Training is classically cited as a 

way to minimize hazards of flying, including weather.

Yet, the body of actual research concerning measured 

effectiveness of weather-related training in GA is small 

and often involves trying to correlate the implementation of training methods with subsequent reductions in 

accidents or accident rates (Adams & Ericsson, 1992).

Formal logic asserts that correlation is necessary, but 

not sufficient, to demonstrate causation. Hence, we are 

never sure that training increases pilot skill and results 

in safer behavior. We simply assume it. Yet, a large body 

of research in perceptual, behavioral, and educational 

psychology shows that the acquisition and retention of 

learning is often anything but assumable (Ellis, Semb, & 

Cole, 1998; Goldstein, 1999; Mackintosh, 1974; Semb 

& Ellis, 1994).

Characteristically, training is not permanent. Figure 1 

shows a simplified learning decay function. New learning starts at some maximum level (e.g., “100%”), after 

which it decays exponentially with time (assuming it is 

not refreshed), asymptoting at some lower level (here, 

an arbitrary “20%”).

Also, the amount of initial learning, plus the rate of 

decay, are two crucial parameters of knowledge retention.

Another is how well knowledge is transferred from one 

domain to another—for example, from the classroom 

to the real world (Perkins, 1992). Finally, measuring 

“cognition in the wild” is often a very different set of 

circumstances from measuring in a carefully controlled 

laboratory setting (Hutchins, 1995). This makes realworld assessment of training a challenge for researchers.

In the real world, non-instrument-rated pilots are 

supposed to fly by visual flight rules (VFR). VFR pilots 

learn they are supposed to avoid weather. However, they 

sometimes attempt a flight when weather is a factor along 

their route, either as forecast or unknown. Believing 

the weather is safely flyable, the VFR pilot is actually 

ill-prepared to deal with an encounter, since practical 

weather skill training is usually minimal or absent from 

the private pilot syllabus. Similarly, a newly minted 

GA instrument pilot may know intellectually to avoid 

thunderstorms and icing when flying in clouds but has 

little practical knowledge and skill to allow him or her 

to proceed safely.

A final factor involving training has to do with assessment. Following the revelation that pilots can pass 

the FAA certification examinations without answering 

the weather-related questions (Wiegmann, Talleur, 

& Johnson, 2008), the FAA is rethinking the testing 

Figure 1. Simplified learning decay function. 
2 

procedure. Further data on this subject would be useful 

to decision makers.

Purpose of this research

Immediate motivation. At the behest of FAA Human 

Factors Research and Engineering Group program management (AJP-61, see Acknowledgments), Civil Aerospace 

Medical Institute (CAMI) researchers were tasked by the 

Flight Standards division (AFS-810) to explore issue #1 

below. Issues 2-3 were added because they were useful to 

the study, addressing general issues of flight simulation 

methodology critical to both CAMI and the human 

factors flight simulation research community at large.

1. Do video weather training products significantly 

affect pilot weather knowledge and flight behavior 

in the face of potential instrument meteorological 

conditions (IMC)? 

a. If so, what are the immediate effects?

b. Do these effects persist over time?

2. How are modern Web-based weather products used 

during preflight briefing?

3. Do local1

 pilots differ appreciably from non-local 

pilots in either weather knowledge or weather-related 

flight behavior?

Issue 1 is consequential for reasons already stated.

Issue 2 begins human factors study of what promises to 

be the future of preflight weather briefing—self-briefing 

by pilots using Internet-based tools.

Issue 3 addresses the broad question of whether or 

not local (Oklahoma-based) pilots are representative 

of pilots within a larger, continental area. Presumably, 

they are similar but, so far, this has not been directly 

investigated. Since many of the FAA’s general aviation 

studies are conducted by the Oklahoma-based Civil 

Aerospace Medical Institute, this is a statistical validity 

issue calling for study, and one that has bearing on all 

flight simulator studies.

Possible types of training products. Effective training is 

usually based on sound learning theory. Learning theories 

are organized explanations of how we come to know what 

we know and behave as we behave.

Learning theories fall under three major categories: 

behaviorism, cognitivism, and constructivism. To each 

category, three common, fundamental processes apply: 

Knowledge and behavior can either be acquired, maintained, or extinguished.

1

A “local” pilot was defined as one living in Oklahoma at the time 

of the study. For the most part, this meant long-term OK residents 

(n=18). However, there were 4 instances of pilots living in OK whose 

state-of-legal residence was not OK because they were attending local 

flight schools.

Behaviorism explains the acquisition, maintenance, and 

extinction of learning as related to observable behaviors.

Behaviorism asserts that we learn either because instinctive 

behaviors become associated with environmental stimuli 

or because trainable behaviors are rewarded or punished 

(Mackintosh, 1974).

In contrast, cognitivism focus on brain functions—

often not directly observable—particularly memory and 

information processing. Cognitivism suggests that our 

brains may functionally resemble computers, processing 

inputs to produce outputs (Waltz & Feldman, 1988).

Finally, constructivism expands on the computer 

metaphor, but elevates cognition from a relatively 

straightforward “information vector-transformation” role 

to a somewhat richer (yet harder to define) “construction 

of an inner world” (von Glasersfeld, 1995). This “inner 

world” involves an organized set of mental representations of external objects, their relations, and interactions.

Constructivism thus includes the active formation of 

inner abstractions, concepts, rules, and principles that 

can both organize current knowledge and predict future 

outcomes in the environment, based on current behaviors 

of the actor.

At the risk of oversimplifying the best way to train 

pilots, behaviorist training methods arguably apply best 

to well-defined procedural tasks such as stick-and-rudder 

skills—discrete behaviors that can be practiced until they 

become habits. Cognitive training methods arguably work 

best with tasks requiring memorization of information, 

pattern recognition (e.g., recognizing cloud types), and 

other tasks requiring the transformation of information 

from one state to another.

Finally, constructivist training methods involve taking 

what we already know and adding to that to construct a 

richer, more accurate “mental model” of the world and 

how it works. By that logic, since piloting involves understanding a complex machine operating dynamically 

within a complex environment, constructivist methods 

may work best when trying to teach aeronautical decision making.

This sketch of learning theory, albeit brief, leads to 

testable hypotheses. For instance, if a video training 

product embodying a cognitive training approach were 

compared to one embodying a constructivist approach, 

and both were compared to an irrelevant (non-weather 

related video) Control group—would we see significant 

differences between groups in pilot weather knowledge 

and flight behavior, given a simulated flight mission into 

adverse weather?
3 

To set up a formal null hypothesis (H0) and several 

alternatives (H1-3):

• H0

: No significant differences exist between control, 

cognitive, and constructivist training products.

• H1

: Cognitive product will be significantly better than 

the control product (H0

<H1).

• H2

: Constructivist product will be significantly better 

than the control product (H0

<H2).

• H3

: Both cognitive and constructivist products will be 

superior to the Control, plus one will be significantly 

better than the other (H0

<H1<H2 or H0<H2<H1).

Method

The research was conducted in two phases. Phase 

1 examined data collected from January to July, 2008, 

and is the subject of the current report. Phase 2 will be 

similar in approach and will constitute the second half 

of a longitudinal study, to be reported at a later date.

Participants

Forty-eight GA pilot volunteers participated with informed consent. Various exigencies ultimately brought the 

total to 50. The overall group mean age was 41.0 (median 

= 39, SD = 17.5), mean flight hours was 1314 (median 

= 268, SD=2709). A few high-hour pilots elevated the 

flight hours mean, generating a statistical concern that 

will be addressed later.

We recruited local pilots from a list of pilots who had 

participated in previous studies and by advertising in 

local flight schools. Non-local pilots saw an advertisement in Flying magazine, which resulted in more than 

350 responses. The first 18 respondents were selected 

blindly. However, after checking their demographics for 

age and flight hours, a preponderance of higher-hour, 

older pilots was noticed, relative to the local pilots. This 

may not be surprising, given the type of pilots likely to 

read Flying. However, to better equilibrate the groups, 

the remaining six non-local pilots were purposely chosen 

for being relatively younger and having relatively lower 

flight hours (Figure 2).

Research design, assignment to groups, and order of 

treatments

In measuring how training can change a person, changes 

can occur in what we know and/or how we behave.

Therefore, this experiment measured the effects of several 

independent variables on the dependent variables of both 

cognitive weather knowledge and pilot flight behavior.

This was operationalized as follows:

Primary independent variables

1. Weather training video product (Trg Prod)

a. Training product 1 (constructivist)

b. Training product 2 (cognitive)

c. Control (pilots watched a video on aviation 

physiology)

2. Instrument rating

a. Instrument-rated pilots

b. Private pilots (non-instrument-rated)

3. Pilot’s state of residence

a. Local residents

b. Non-local residents

Secondary independent variables

4. Age

5. Flight hours

Dependent variables

1. Weather knowledge test

2. Web-based preflight data

a. Page viewed

b. Pageview duration

3. Flight simulator data

a. Flight duration (in minutes)

Figure 2. Pilot demographics. 
4 

b. Minimum distance to ABQ (statute miles)

c. Cloud clearance

i. Minutes spent in IMC

ii. Minutes spent at < 500' from cloud base (scud 

running)

d. Ground clearance [minutes spent at < 500' Above 

Ground Level (AGL)]

e. Takeoff decision (yes/no)

Each pilot was assigned to one of three primary Training 

product groups (Table 1, “Trg product” column). In turn, 

each Training Product group of 16 pilots was sub-divided 

into two secondary Instrument Rating groups (“Rating” 

column), containing eight instrument-rated pilots (IFR, 

for “instrument flight rules”) and eight non-instrumentrated pilots (VFR, for “visual flight rules”). Finally, each 

Instrument Rating group of eight was sub-divided into 

two tertiary Pilot Residence groups containing four local Oklahoma pilots and four non-local pilots (“Local” 

column, 1=local, 0=non-local). Table 1 embodies the 

resulting experimental structure.

This resulted in a 3x2x2 mixed design with 12 treatment cells. In Phase 1, the variables of training product, 

instrument rating, and residence could be analyzed as 

between-groups comparisons, with group means being 

compared to other group means to look for differences.

Conversely, weather knowledge could be analyzed as a 

within-groups comparison in which each pilot would 

serve as his or her own statistical control over repeated 

administrations of different (but equivalent) test forms.

The 12 cells arguably needed to be equilibrated for 

pilot age and flight hours. Table 1 was therefore set up 

as an ExcelTM spreadsheet, used to allocate pilots to cells 

while maintaining maximum equivalence in cell means 

for age and flight hours.

Two variants of weather direction were used as distractors to help divert attention from the fact that the flight 

scenario would be essentially repeated in Phase 2 (“Flight 

scenario” column). Variant 45 had weather approaching 

from 45° (aeronautical coordinates, 0° = North, increasing 

clockwise), while Variant 135 had weather approaching 

from 135°. However, note that weather direction was 

simply a distractor, not otherwise a variable of interest.

Apparatus 

Weather training products/control materials. Two wellknown video weather training products were selected from 

a list of candidate products. Given the impossibility of 

knowing product quality a priori, two publicly prominent 

products were chosen. The authors of these products 

graciously provided them on condition of confidentiality; therefore, their wishes for confidentiality shall be 

respected in this report.

Training product 1 constituted the “constructivist product.” This focused mainly on the aeronautical 

decision-making aspects of weather flight. It offered 

systematic, mnemonic risk factor checklists applicable to 

specific factors such as the weather in question, internal 

pilot factors affecting performance (e.g., skill, health, 

fatigue), and factors external to the pilot that could affect risk-taking (e.g., passengers needing to arrive at their 

destination by a certain time). After each video lecture 

session, it presented hypothetical flight scenarios and 

asked the student to evaluate these, based on the lecture 

content presented so far.

Training product 2 constituted the “cognitive product.” This focused largely on the recognition of different 

cloud types, visibility conditions, horizon recognition, 

and terrain clearance. For instance, still pictures of common weather types were shown, after which pilots were 

queried as to their appropriateness for VFR flight. This 

technique has been used in research (Wiggins & O’Hare, 

2003). Figure 3 shows a sample picture.

2

Sample exercises showed still pictures of a weather 

situation as seen aloft, asking what recognition factors 

were problematic, and then asking for a go/no-go decision on VFR flight.

A second section presented details of an accident scenario, asking pilots to decide the primary cause. A third 

section began by discussing factors involved in deciding 

whether or not to divert because of weather. It then 

presented a detailed weather flight scenario involving a 

number of possible alternate landing sites, asking which 

was most appropriate. Finally, it presented a list of several 

potential flights, with preflight briefing details of each, 

next asking for a go/divert decision after presentation of 

a still photo of in-flight weather, and finally asked for 

a justified choice of alternate, if diversion was chosen.

The third video group—the Control group—received 

an FAA-produced video on aviation physiology, having 

nothing whatsoever to do with weather.

Timer. A timer utility was written by the experimenters 

to capture the amount of time each pilot spent viewing 

the training product. This timer was activated by each 

pilot at the beginning of training and was turned off 

afterward to capture elapsed viewing time (Figure 4).

Measurement of timing was motivated by the question 

of whether viewing time might influence the subsequent 

weather knowledge posttest score, in which case time 

might be useful as a covariate in statistical analysis.

2

Simulator fidelity can be an issue in weather studies. While visibility 

restrictions such as haze and fog are relatively simple to model, even 

static cloud layers present modeling challenges. Discrete, morphing 

clouds present the greatest challenge because real clouds act as chaotic 

cellular automata evolving within a larger multivariate, non-linear, 

chaotic system. The present study used static, stratiform clouds—

among the easiest to model realistically.
5 

Table 1. Experimental structure (the ellipsis “...” denote data purposely 

omitted to declutter the table). 

Figure 3. Would this type of weather be appropriate for VFR flight? 

Figure 4. Timer used to record 

time spent by each pilot on the 

weather training product. 
6 

Weather knowledge tests. Three parallel forms were 

constructed of a 30-question general weather knowledge 

test and were matched on item difficulty, using questions 

and proportion-correct data provided by FAA’s Airman 

Testing Standards Branch (AFS-630). One-third of the 

questions on each test were taken from private pilot tests; 

two-thirds came from instrument rating tests. This was 

not expected to pose a problem, since pre- minus posttreatment change scores were to be analyzed, which are 

immune from overall test difficulty as long as the tests 

are neither impossibly difficult nor trivially easy (i.e., do 

not suffer from either ceiling or floor effects).

Administration order of the parallel forms was counterbalanced across pilots (see Table 1). This controlled 

for the event that the 3 forms would not be exactly 

equivalent in difficulty.

Each test was administered on a laptop computer using software written by the experimenters in Microsoft 

Visual Studio 2005TM. Figure 5 shows a screenshot of a 

sample question.

The program automatically recorded each question, 

each response, response-correct or incorrect, time spent per 

response, overall percent correct, and total elapsed time.

Preflight weather briefing materials. Briefing materials 

included a verbal description of the flight mission, plus 

standard DFW (Dallas-Fort Worth) and ABQ (Albuquerque) sectional charts.

To simulate Internet weather briefing, we wrote a 

part-task emulation of the NOAA/NWS Web weather 

briefing site www.aviationweather.gov, also in Microsoft 

Visual Studio 2005. This emulation automatically recorded 

which pages were viewed and the view duration of each 

page. Figure 6 shows a sample page. Appendix A shows 

screenshots of all pages. Appendix B illustrates sample 

hypertext markup language code used to create the home 

page. The core html was taken from www.aviationweather.

gov and modified for use here. Appendix C illustrates 

code-behind for a sample page. In Visual Studio, codebehind can be written to create dynamic Web pages that 

respond to events such as mouseovers and button clicks.

Specifically, pages were written to convey information on:

1. SIGMET3

/AIRMET4 (Java tool; graphical) 

3 Significant Meteorological Information

4 Airman’s Meteorological Information

Figure 5. Sample screenshot from the Weather Question Tester program. 
7 

2. Convection

a. Convective SIGMETs (graphical) 

b. CCFP5

 (graphical; looping .gif movie)

c. Convective outlook (graphical) 

3. Turbulence (graphical) 

4. Icing (graphical) 

5. Winds/Temps (text)

6. Prog charts (graphical) 

7. TAFs6

/FAs7

a. TAF Java tool (graphical, with popup text)

b. TAF Station model (graphical) 

c. Area Forecast (text)

8. PIREPs8

 (text; not used in this experiment)

9. METARs9

 (Java tool; graphical, with popup text)

10. Radar (graphical; looping NEXRAD10 .gif)

5 Collaborative Convective Forecast Product

6 Terminal Aerodrome Forecast

7

 FA=Aviation area 18-h forecast

8

 Pilot Report

9

 Meteorological Aerodrome Report

10 Next Generation Radar

11. Satellite (graphical; looping cloud cover .gif)

Advanced General Aviation Research Simulator (AGARS)

The AGARS is a real-time, fixed-based flight simulator 

that can represent complex interactions of environment, 

hardware, communications, crew resource management, 

and situational and risk variables for simulated general 

aviation flight protocols. It provides dynamic, controlloaded responses. Based on current dual-core server class 

AthlonTM CPUs running the Linux OS (Fedora core 7), 

it was configured as a Piper Malibu for the purposes of 

this experiment.

Equipped with a high-resolution visual system with a 

150 degree field of view, AGARS allows meteorological 

conditions to be precisely controlled. It continuously captures up to 150 variables at 30Hz for a four-hour mission 

and includes up to 85 programmable non-routine events.

Panels and consoles are reconfigurable to allow testing of innovative display concepts created using rapid 

prototyping software (GLStudioTM). It is equipped with 

Figure 6. Sample screenshot from the Web-based emulation of 

www.aviationweather.gov. 

Figure 6. Sample screenshot from the Web-based emulation of www.aviationweather.gov.
8 

an experimenter operating station (EOS), an ATC workstation, and a workstation for controlling pseudo-vehicles.

During the course of a flight scenario, the EOS allows 

the experimenter to visually monitor the cockpit and 

simulation environment. In addition to digital recordings of the flight data, a digital camera is used to record 

a global view of the cockpit and pilot onto a stand-alone 

DVD player hard drive and later recorded to DVDs. All 

cockpit, ATC and experimenter communications are also 

recorded onto these DVDs (Figure 7).

In-flight weather updates are an option available to the 

modern GA pilot through automated recorded systems 

such as the Automated Weather Observation System 

(AWOS). To emulate AWOS, we wrote a control panel 

capable of triggering prerecorded METAR information 

(Figure 8).

Pilots could tune the cockpit radio to one of a set 

of given frequencies, alerting the experimenter to click 

the corresponding button on the AWOS control panel, 

triggering the corresponding recorded METAR, which 

played back through the pilot’s headphones.

The Flight Service Station (FSS) is also an option to 

GA pilots to receive air traffic control services such as 

flight following, vectors-to-destination, and weather.

To emulate this option, one of the experimenters (Ball) 

served as a pseudo-FSS briefer during the flight phase.

Procedure 

Upon arriving at the simulator lab, we asked pilots 

to plan an east-to-west, VFR flight from Amarillo, TX 

(AMA) to Albuquerque, NM (ABQ). This route takes 

approximately 90 minutes to fly in the Malibu with a 

high speed cruise setting. We instructed pilots to plan the 

route utilizing the following equipment in the cockpit: 

2 VORs (VHF OmniRange Navigation System) and an 

ADF (Automatic Direction Finder). Additionally, pilots 

had access to the Web-based weather emulation on a 

stand-alone PC during preflight planning. Upon finishing 

Figure 7. The CAMI Advanced General Aviation Research Simulator 

(AGARS). Photos used by permission of participant. 

Figure 8. AWOS emulator. 
9 

their flight plan, pilots took the post-weather knowledge 

test. Next, we offered a 15-minute convenience break 

to each pilot. Following the break, each pilot had a 30 

to 40-minute training and familiarization session with 

AGARS. Specific training was provided on the usage of 

the autopilot, the horizontal situation indicator (HSI), 

and the flight parameters and characteristics of the Malibu 

aircraft (e.g., maximum/stall speeds, associated power 

settings). The time between finishing the preflight planning and the actual flight was approximately 60 minutes.

Due to pilot unfamiliarity with the simulator and the 

complexity of the Piper Malibu, pilots were allowed to 

ask for assistance with flight settings at anytime during 

the course of the flight scenario.

The route consisted of gradually rising terrain during 

the first two-thirds of the flight, followed by a dramatic 

elevation change during the last one-third of the flight.

During the course of the flight, pilots were exposed to 

deteriorating VFR weather conditions. Initially, the visibility was set at 8 nautical miles and gradually decreased 

to 5 miles visibility by the time the pilots had flown 

approximately 2/3 of the route. Concomitantly, cloud 

ceilings were lowered from 4500 feet AGL to 3500 AGL 

across the same stretch of terrain, gradually squeezing 

the pilots closer to the ground.

These terrain issues, coupled with marginal visual 

meteorological conditions (VMC) and potential rapidly 

changing barometric pressure, resulted in a potentially 

dangerous flying situation with hazardous encounters 

throughout the course of the flight. Shortly into the 

flight, the barometric pressure dropped from the preflight planning level of 30.10 to 29.98. This afforded a 

potential error between actual and intended altitude for 

pilots not receiving a barometric update (either from 

AFSS or AWOS) after departure. Specifically, pilots 

failing to update their Kollsman setting would fly an 

actual altitude approximately 120' below their intended 

altitude. The authors acknowledge that, given the aircraft’s 

blind transponder encoder plus the departure airspace, 

in real life an air traffic controller would have normally 

detected the altitude discrepancy and issued a correction 

to the pilot. However, since the study’s specific purpose 

was specifically to study both errors of commission and 

omission, we purposely skipped this correction to study 

the consequences.

Results

Correction of results for familywise error

In statistical analysis, when a large number of tests 

are conducted, it is likely that a small percentage may 

falsely indicate “significant” results of treatment effect or 

correlation. Given, say, 100 tests at a significance level 

of p<.05, by definition we expect 5 of those, on average, 

to be falsely positive because that is precisely how “p = 

.05” is defined.

In statistical practice, a mathematical correction for 

experiment-wide (“familywise”) error can be applied to 

each individual test result to account for the effect of multiple comparisons. However, while this makes the analysis 

less prone to Type 1 error (a “false alarm” = declaration of 

effect where there is none), it also elevates Type 2 error (a 

“miss” = failure to declare true effect where, indeed, one 

exists).

In preliminary studies (such as this), it is commonplace 

to omit the familywise error correction because misses are 

considered equally as important as false alarms. Such will 

be the case in the results that follow. The practical cost is 

lower reliability than it might seem, merely looking at pvalues and effect sizes. The reader is hereby alerted to that 

possibility. The benefit, of course, is increased sensitivity

to effects and relations within the data. Good practice 

simply demands that the situation be clearly stated.

Preliminary examination of data (frequency distribution 

normality, outliers)

It is also true that parametric statistics are usually more 

powerful—meaning more likely to detect treatment effects 

when they truly exist. However, to justify their use, score 

frequency distributions need to be approximately normal.

Otherwise, distributions should either be mathematically 

normalized, or distribution-free (nonparametric) statistics 

should be used. Following convention, the first task was 

to check distributional normality, particularly to rule out 

floor and ceiling effects, which indicate that tasks were 

either too easy or too difficult.

Normality of pilot age and flight hours. Despite the 

dynamic spreadsheet used to assign pilots by age and 

flight hours, the flight hours data showed serious nonnormalities for the collapsed score distribution, evidenced 

both by probability-of-z test for skewness and kurtosis 
10 

(p(z(skew)) < .001 , p(z(kurt)) < .001) and by failure of the 

Kolmogorov-Smirnov test (p < .001). This was primarily 

due to the presence of a small number of very high-hour 

pilots. Figure 9 shows the frequency histograms.

Normality of weather knowledge test scores. As a main 

dependent variable, weather knowledge pre- and post-test 

scores did appear to be normal, as Figure 10 shows. This 

was supported by 2-tailed Shapiro-Wilk tests of normality 

(ppre-test =.297, ppost-test =.786, both non-significant (NS).

Grouped by instrument rating, knowledge scores 

did not significantly deviate from normality (2-tailed 

Kolmogorov-Smirnov pnon-IR =.143, pIR =.200, both NS).

Figure 11 shows the average-score11 histograms.

Normality of Web-based weather information data. 

Because the Web-emulation data presented considerable problems with normality, we used medians and 

percentiles to describe those data. During their pre-flight 

planning, we instructed pilots to close out the Web-based 

weather briefing tool after finishing with it. This was 

necessary to capture the page view time duration of the 

final page viewed. Nonetheless, on the basis of observed 

pilot behavior, plus analysis of page view durations, we 

11 This was based on average combined pre-test and post-test scores 

((pre-test score + post-test score)/2).

concluded that a considerable number of pilots forgot to 

close out the final page. Inspection of durations for final 

pages revealed that at least 15 of 53 sessions appeared 

inordinately long (e.g., > 5 SD above the mean), versus 

30 of 1024 non-final pages (p(C2

)= 5.04E10-15).

To complicate the matter, some pilots moved back and 

forth between some pages and the sectional, even before the 

final page was opened. These individuals were obviously 

not forgetting anything. They were merely comparing the 

20 30 40 50 60 70

Age

0

5

10

15

Count

Age x Frequency

2500 5000 7500 10000 12500

Flight hours

10

20

30

40

Count

Flight hr x frequency

Figure 9. Frequency histograms for numbers of pilots (y-axis) 

by age and flight hours (x-axis). 

0.400 0.500 0.600 0.700 0.800 0.900

K Pretest

2

4

6

8

10

Count

0.400 0.500 0.600 0.700 0.800 0.900

K Postest

0

4

8

12

Count

Figure 10. Frequency histograms for numbers of pilots (y-axis) 

by weather knowledge pre- and post-tests (x-axis). 

Figure 11. Frequency histograms for 

numbers of pilots (y-axis) by average 

weather knowledge ((pre + post-test)/2) 

scores (x-axis). 
11 

Figure 12. Whole-group (N=50) page view durations for the part-task 

emulation of www.aviationweather.gov. Partial box plots show 25th, 50th, and 

75th percentiles. Note that the y-axis is logarithmic. 

Web information with the more detailed information on 

the sectional—a perfectly legitimate real-world strategy.

However, this strategy resulted in normality problems for 

page view duration. In total, some 45 page views were 

suspected of being outliers, which seriously inflated those 

page category means and standard deviations, making 

parametric analysis unjustified.

Figure 12 shows the page view duration data.

Normality of flight simulator data. The normality check 

for flight simulator data involved:

• Flight duration

• Minimum final distance to Albuquerque (ABQ)

» Cloud clearance

» Time spent in IMC

• Scud running (time spent at < 500' from cloud base)

• Ground clearance (time spent at < 500' AGL).

Figure 13 visually illustrates the fact that only flight 

duration met the Kolmogorov-Smirnov test of normality.

In fact, the “poor” normality of most of these data 

was actually a predictable artifact of good scenario 

design. A good flight scenario is challenging but not 

impossible. Twenty of 50 pilots (40%) made it all the 

way to Albuquerque. The remaining 30 (60%) diverted 

to alternates. Therefore, the scenario was challenging, 

but not impossible. In fact, to obtain normal-looking 

data for all variables would have required a scenario so 

difficult that few pilots would have made it to ABQ—a 

certain way to overtax pilots and discourage them from 

returning for Phase 2 of the study.

Training product study time. We instructed pilots to 

use the timer utility (Figure 4) to monitor how long 

they studied their training product. Unfortunately, due 

to scheduling constraints, most pilots had to study the 

Figure 12. Whole-group (N=50) page view durations for the part-task emulation of www.

aviationweather.gov. Partial box plots show 25th, 50th, and 75th percentiles. Note that the y-axis is 

logarithmic.
12 

training product the night before their simulator session.

Compliance with session timing proved to be low, with 

useful data collected on just 29 pilots. Table 2 shows the 

number of pilots per group (n), means, and standard 

deviations (SD) for compliant pilots.

Despite a significant Kruskal-Wallis group difference 

(p = .002), the 31 missing data points made this result 

suspect. Therefore, study time was not used as a covariate 

in subsequent analysis.

Conclusion. Most of these data were simply too extreme to be corrected by any standard mathematical 

transform such as a log or square root. Therefore, most 

analysis needed to be done using distribution-free (nonparametric) statistics.

Overall data relations 

Table 3 shows correlations between key variables.

Statistically significant correlations are highlighted in 

gray. P-values approaching .05 significance are also 

included for the sake of interest. Spearman correlations 

(rs

) are nonparametric, being based on rank order. Pointbiserial correlations (rpb) are used when one variable is 

dichotomous, the other continuous. However, rpb is 

still a mean-based statistic, therefore not purely nonparametric.As such, rpb may be subject to higher Type I 

error when the continuous distribution is non-normal.

Some caution is appropriate.

25.00 50.00 75.00 100.00

Flight duration, minutes

2

4

6

8

10

Count

100.00 200.00 300.00

Min dist to ABQ

0

5

10

15

Count

0.00 25.00 50.00 75.00

Minutes < 500' cloud clearance

0

10

20

30

Count

0.00 25.00 50.00 75.00

Minutes in IMC

10

20

30

40

Count

0.00 5.00 10.00 15.00

Minutes < 500' ground clearance

10

20

30

40

Count

Figure 13. Frequency histograms for numbers of pilots (yaxis) by flight duration, minimum distance to Albuquerque, 

minutes below 500’ cloud clearance (scud running), minutes 

in IMC, and minutes below 500’ ground clearance (x-axis). 
13 

Table 2. Minutes spent viewing each 

training product (instruction-compliant 

pilots only). 

 n Mean SD 

Trg prod 1 7 93 34 

Trg prod 2 9 58 17 

Control 13 109 52 

ntotal 29 89 

Recognizing trivial relations. Five of the 8 largest correlations are statistically significant but trivial. These cells 

are marked in the lighter shade of gray. To summarize:

1. Instrument rating x Pilot age (rpb = .523) merely 

means that instrument-rated pilots tend to be older.

All pilots start out non-instrument-rated, and at least 

some time elapses before a fraction (about half) go 

on to get their instrument rating, making this correlation anticipated.

2. Instrument rating x Flight hours (rpb = .401) simply 

means that instrument-rated pilots tend to have more 

flight hours. This follows logic similar to Point 1.

3. Pilot age x Flight hours (rs

 = .757) merely means that 

as pilots get older, they tend to accumulate flying time.

4. Flight duration x Minimum distance to ABQ (rs

 = 

-.936) only means that the longer pilots flew, the 

closer they tended to get to the destination (ABQ).

5. Scud running x IMC penetration (rs

 = .676) was partly 

a side effect of the way scud running was defined (as 

time spent with < 500' cloud clearance). Figure 14 

demonstrates.

Oddly, visual inspection of the flight profiles revealed 

a substantial number of pilots who flagrantly penetrated 

IMC immediately after takeoff. The issue, then, was 

that the clearance threshold used to determine “scud 

running” was exceeded as pilots simply flew level out of 

the steadily rising cloud layer. This was logically different 

from intentional scud running. Therefore, in the future, 

it may be necessary to consider some method of filtering 

out such unintentional scud running to distinguish it 

from the intentional variety.

Marginal relations. Given the 50 correlations computed, 

two-three can be expected to appear “significant” near 

p = .05 merely due to chance. These marginal cells are 

highlighted in a darker gray, but the text is not boldfaced.

They should be considered tentative.

Figure 14. Flight profile of a pilot immediately 

climbing into IMC and maintaining level flight 

thereafter. Similar profiles occurred for 20% 

of pilots. 

Table 3. Correlations between key variables (see Fig. 18 for a graphical representation of these data). 

Variable 2 

Variable 1 

Instrument 

rating 

(1=instrument

-rated)1

Locality of 

residence 

(1=Local)1

Pilot age2

Pilot flight 

hours2

Ave. wx 

Knowledge2

Web pre-flight 

duration2

Flight 

duration2

Minimum dist 

to ABQ2

Minutes scud 

running2

Minutes in 

IMC2

Instrument Rating 1.0 

State of Residence 3 1.0 

Pilot Age .5231 (.0001) 3 1.0 

Pilot Flight Hours .4011 (.004) 3 .757 (<.001) 1.0 

Ave. Wx Knowledge .233 (.057) .271 -.035 .086 1.0 

Web Preflight Duration .020 -.348 (.013) .417 (.003) .227 .204 1.0 

Flight Duration -.039 .042 -.423 (.002) -.270 -.010 -.222 1.0 

Minimum Dist to ABQ .013 .013 .422 (.002) .303 (.032) .029 .242 -.936 (<.001) 1.0 

Minutes scud running -.013 -.012 .051 .107 -.054 .027 .013 -.042 1.0 

Minutes in IMC -.020 -.005 -.089 -.084 -.123 -.124 .028 -.035 .676 (<.001) 1.0 

Minutes < 500’ AGL -.281 (.048) .144 -.167 -.289 (.041) -.229 -.039 .379 (.007) -.384 (.006) -.095 -.174 1

rpb = Point-biserial correlation; 2rs = Spearman rho correlation; Low p-values are in parentheses (all others are non-significant (NS)); 3 No 

correlation run because sample had been partitioned for these factors. 
14 

1. Instrument rating x Minutes < 500' AGL (rpb = -.281)

2. Pilot flight hours x Minimum distance to ABQ 

(rs

 = .303)

3. Pilot flight hours x Minutes < 500' AGL (rs

 = -.289)

The first correlation implies that instrument-rated 

pilots had slightly less tendency to spend time too close to 

the ground. However, the effect size was small, accounting 

for r2

pb = (.281)2

 = only 8% of the measurement variance.

The second correlation (.303) implies that pilots with 

higher flight hours tended to stay slightly farther away 

from ABQ. This effect size was about 9%.

The third correlation (-.289) implies that pilots 

with higher flight hours also tended to spend slightly 

less time too close to the ground (effect size 8%). All 3 

correlations coincide with common sense, if we assume 

that more training and experience tend to produce more 

cautious pilots.

Non-trivial relations. Beyond trivial and marginal 

relations, a few others emerged. In Table 3, these cells 

are highlighted in a darker shade of gray, with boldfaced 

text. They are:

1. Locality of residence x Web preflight duration (rpb

= -.348)

2. Pilot age x Web preflight duration (rs

 = .417)

3. Pilot age x Flight duration/ Minimum distance to 

ABQ (rs

 = -.423/.422)

4. Flight duration/minimum distance to ABQ x Minutes 

< 500' AGL (rs

 = .379/-.384)

Correlation 1 (-.348) implies that local Oklahoma 

pilots tended to spend slightly less time using the Web 

preflight briefing tool than non-Oklahoma pilots did 

(x̄= 20.31 v. 13.66 min). The effect size was modest, 

accounting for r2

pb = 12% of the variance.

Correlation 2 (.417) implies that older pilots tended 

to spend somewhat more time using the Web tool than 

younger pilots did. Effect size was 18%.

Correlations 3 (-.423/.422) imply that older pilots 

tended to have somewhat shorter flights (and, hence, to 

end up farther away from ABQ). Effect size was about 

18%.

Finally, Correlations 4 (.379/-.384) represent the flight 

scenario’s tendency to “squeeze” pilots between clouds and 

terrain near ABQ. The farther one flew, the more one 

got squeezed. Figure 15 illustrates this with two sample 

flight profiles illustrating a) IMC penetration and scud 

running, and b) violation of ground clearance. Profile B 

clearly shows the mountains near ABQ, with the “squeeze” 

this could pose. Another way to view this is that the 

majority of scenario danger tended to be concentrated 

near the destination.

Specific effects

Effect of the weather training products on GA pilot weather 

knowledge. Did viewing a weather training product significantly improve pilots’ weather knowledge test scores? 

Seemingly not. Repeated measures analysis of variance 

(ANOVA) for posttest-pretest score gain x training 

product interaction yielded a non-significant pF

 = .734.

Relation between pilot weather knowledge and subsequent 

flight safety. Were pilots with higher weather knowledge 

safer pilots? Seemingly not. As Table 3 showed, average 

weather knowledge ((pre+posttest score)/2) did not correlate significantly with any flight behavior variables.

Spearman correlations between weather knowledge scores 

and flight duration, minimum distance to ABQ, minutes 

scud running, minutes in IMC, and minutes < 500' AGL 

all ranged from -.229 < rs < .029, all NS.

At least as measured by these test questions, weather 

knowledge did not seem strongly influenced by age, flight 

Figure 15. Flight profiles for two pilots. Since the flight was basically east-towest, the x-axis is drawn as degrees longitude by feet altitude-above-MSL on 

the y-axis. 
15 

hours, or instrument rating (rs = -.035, .051, rpb =.233 

[respectively], all NS). Although instrument-rated 

pilots showed a slightly higher average knowledge 

score (67.6%) than did non-instrument-rated pilots 

(60.8%), this just missed the statistical criterion for 

reliability (1-tailed pt

 = .057, NS).

12

Effect of Web preflight briefing time on subsequent 

flight safety. Were pilots who spent more time on their 

Web-based weather briefing safer pilots? Not significantly.

Spearman correlations of Web preflight duration with 

flight duration, minimum distance to ABQ, minutes 

scud running, minutes in IMC, and minutes < 500' 

AGL ranged from -.222 < rs < .242 respectively, all NS.

Takeoff hesitancy. We told pilots that the best way to 

give good flight data was to treat this mission as if it were 

a real flight. Given those instructions, 12 of the 50 pilots 

initially stated that having to fly this mission VFR, they 

would choose not to even take off. This was perhaps 

predictable, given the weather and being scrutinized by 

FAA officials at an FAA facility.

Therefore, to overcome any reservations they might 

understandably have about being scrutinized, pilots who 

declined to take off were explicitly asked to take off and 

fly at least briefly. All complied.

What kind of pilot tended to hesitate? Locality of 

residence had no reliable statistical effect—18% of local 

(Oklahoma) pilots hesitated versus 32% of non-local 

(non-Oklahoma) pilots (2-tailed pC2 = .251, NS). If we 

had predicted an effect, it would have trended the way it 

did, since locals would be more likely to know the terrain 

and be skilled in handling the high winds typical of the 

Midwestern U.S.

Instrument rating did not demonstrably matter 

(15% hesitancy for instrument-rated v. 33% for noninstrument-rated, 2-tailed pC2 = .138, NS). This, too, 

trended in the anticipated direction, since one would 

expect somewhat greater confidence from instrumentrated pilots.

Despite the confidence-building tendencies often 

associated with experience, neither age nor flight hours 

seemed to affect hesitancy (2-tailed Mann-Whitney U, 

pU = .146, .625 respectively, NS).

So, overall, the cause of this takeoff hesitancy initially 

appeared mysterious.

Effect of takeoff hesitancy on subsequent flight safety. Did 

the 12 hesitators end up flying safer than the remaining 

38 pilots? Not remarkably. There were no significant 

differences between hesitators and non-hesitators for 

minutes spent in IMC, minutes scud running, or minutes 

< 500' AGL (2-tailed Mann-Whitney pU = .102, .147, 

.498 respectively, all NS). However, hesitators did seem 

12 This was based on average combined pre-test and post-test scores 

([pre-test score + post-test score]/2).

to continue their conservatism into their flight, making 

significantly briefer flights (pU = .002), with consequently 

less penetration into the marginal weather close to ABQ 

(pU < .001).

Effect of the weather training products on takeoff hesitancy. So, what caused takeoff hesitancy? It could have 

been the weather training products. Table 4 shows the 

number of pilots who initially hesitated versus the values 

expected by chance (in parentheses). The exact form of 

pC2 is .035, implying that the training groups differed.

Pairwise tests of odds-ratios implied that the unusual 

group was the Control, where 17 of 18 pilots showed 

no hesitancy to take off.

In other words, studying a weather training product 

may have made pilots more hesitant to take off into 

deteriorating weather. However, cognitive priming is an 

alternate hypothesis we will consider in the Discussion 

section.

Effect of the weather training products on subsequent 

flight safety. Did viewing a weather training product affect flight safety? Not significantly. Limited signs point 

to yes, but the overall effect seems to be no.

The Control group showed significantly less takeoff 

hesitancy, as we have seen. It also displayed greater flight 

duration and, consequently, lower minimum distance 

to ABQ (Kruskal-Wallis pKW = .007, .005, respectively).

Follow-up pairwise Mann-Whitney U tests implied that 

the Control group was significantly different from both 

weather training products (pU-TRG1 x CONTROL = .011, .004 

respectively and pU-TRG2 x CONTROL = .004, .005, respectively), 

although the 2 weather products themselves did not differ 

significantly (pU = .867, 1.0 respectively, NS).

Now—because the maximum hazard of this flight lay 

near the destination—we might be tempted to conclude 

that the longer flights of the Control group should predict 

greater risk exposure. As Table 3 showed, this was supported by a moderate correlation (.379, p=.007) between 

flight duration and minutes < 500' AGL.

However, we found no significant overall differences 

between the 3 training groups for subsequent minutes 

spent in IMC, minutes scud running, or minutes < 500' 

AGL (pKW = .245, .158, .812 [respectively], all NS). Even 

though the Control group showed less hesitancy and longer 

flight duration, and even though longer flight duration 

Table 4. Takeoff hesitancy. 

Trg Prod 1 Trg Prod 2 Control 

Yes 12 (12.2) 9 (12.2) 17 (13.7) Initial takeoff decision No 4 (3.8) 7 (3.8) 1 (4.3) 

.152 

Pairwise odds-ratios, 1-tailed p .004 

.037 
16 

correlated significantly with minutes < 500' AGL, the 

net effect of the weather training videos on subsequent 

flight safety seemed nonsignificant.

So, how can there be no significant differences in flight 

safety among the 3 training groups? If seeing the weather 

training video related to takeoff hesitancy, and takeoff 

hesitancy related to flight duration, and flight duration 

related to minutes spent < 500' AGL—how could weather 

video not relate to minutes spent < 500' AGL? 

The answer lies in the nature of causation versus correlation. If each factor perfectly caused the next factor in 

the chain, then the first factor would perfectly predict 

the final factor. In symbolic logic, A ⇒ B (A implies 

B), and so on, so A ⇒ B ⇒ C ⇒ D, therefore A ⇒ D.

This is easy to see in a Venn diagram (Figure 16a). But, 

if each factor only partially predicts the next factor, then 

the overall relational strength between the first and last 

factors can theoretically be zero (Figure 16b).

IMC penetration. IMC penetration ranged in duration 

from 0.02-86.6 minutes. The majority of pilots avoided 

IMC altogether. However, 16 pilots spent more than 1 

minute in IMC. Ten spent more than 4 minutes. Given 

the instruction to fly VFR, why any pilot should spend 

this much time in IMC was curious.

Instrument rating did not demonstrably influence 

long-duration IMC penetration. Instrument-rated pilots 

made 6 “long-duration” penetrations (>4 minutes), compared to 4 made by non-instrument-rated pilots (pbinomial, 

2-tailed = .754, NS). Furthermore, as Table 2 indicates, 

neither did age, flight hours, or location of residence affect 

long-duration IMC penetration. In short, factors that did

influence it remain unclear at this point in the analysis.

Individual differences. Pilots’ approaches to preflight 

briefing and flight behavior were quite varied. Extensive 

notes were taken while each pilot flew the mission to 

capture these nuances. We also graphed out all 50 flights 

pictorially, similar to Figures 14 and 15. We will study 

and elaborate upon these notes and graphs in the Phase 

2 technical report. However, the following anecdotes 

are noteworthy:

1. Many IFR pilots stated that, if this were a real mission, they would simply file an IFR flight plan 

and fly. Therefore, the VFR mission felt somewhat 

artificial to them.

2. As stated earlier, 16 pilots spent more than 1 minute 

in IMC, and 10 spent more than 4 minutes.

3. At least one pilot admitted to becoming lost near the 

end of the flight.

4. Over half the pilots volunteered concerns that the 

weather questions on their FAA certification tests were 

often only marginally relevant to real-world flying.

5. One pilot experienced controlled flight into terrain 

(CFIT).

The CFIT was a perfect illustration of how a simple 

lapse of terrain proximity (AGL) awareness can become 

fatal. Contrary to expectations, the pilot was actually 

instrument-rated, a Certified Flight Instructor, and older 

and more experienced than most (in the 90th percentile for 

age, the 94th percentile for flight hours). On approaching 

the mountains east of Albuquerque, while busy studying 

the sectional, head-down, the pilot simply flew into a peak.

Similar incidents have been documented; for example, 

a CFIT involving a Piper PA-28-181 on 25 Mar. 2008 

near Bridger, MT (NTSB, 2008). The pilot survived this 

incident and was able to verify lapse-of-attention during 

the NTSB interview.

Rule violations. Despite this being stressed as a VFR-only 

flight, some pilots appeared to disregard that instruction.

Figure 17 (a) shows the numbers of the 50 pilots who 

spent 0.5, 1, 2, 3, and >4 minutes in IMC. Figure 17 

(b) shows the same pilots, but over the entire time range 

from 0.5-82.6 minutes in IMC.

Figure 16. a) Venn diagram embodying causation A B C D; b) 

Venn diagram embodying correlation A rAB B rBC C rCD D. Figure 16a. Venn diagram embodying causation A ⇒ B ⇒ C ⇒ D; 

b) Venn diagram embodying correlation A rAB B rBC C rCD D.
17 

We can argue that those who spent less than a minute 

in IMC probably did so inadvertently. But, strangely, 10 

pilots (20%) spent significant time (>4 minutes) in IMC.

Nine spent more than 10 minutes. The characteristic pattern of these violations involved nearly equal numbers13

of instrument-rated and non-instrument-rated pilots 

ascending into IMC right after takeoff, and then flying 

fairly level, accidentally emerging from the clouds downrange because the cloud layer itself rose slowly. Figure 14 

shows an example. It was as if these pilots were practicing 

terrain avoidance planning—setting their base altitude 

above all known obstacles—but in complete disregard 

of visual flight rules.

This behavior may have been due to willful rule violation. An alternate hypothesis is that perhaps IMC was 

merely difficult to see in AGARS. Simulators sometimes 

13Slightly more IFR pilots did this than VFR pilots, but the differences 

were not statistically significant.

suffer from such difficulties. The mathematics of trying to 

accurately model clouds, fog, haze, and mist are actually 

quite challenging (Hill, 1990).

Several factors discount this lack-of-saliency hypothesis, however. First, AGARS represents a capital investment of over $1M U.S., and great care was devoted to 

ensuring visual fidelity. More objectively, 80% of pilots 

managed to avoid significant IMC, meaning they spent 

< 4 minutes above the cloud base. The average time-inIMC for this 80% was only 24 s, with 28 pilots (56%) 

spending no time at all in IMC. This suggests that most 

pilots who entered IMC did so very briefly, realized it, 

and self-corrected, implying that they did indeed perceive 

the physical stimulus. In contrast, the flight profiles of 

the high-time-in-IMC pilots typically involved climbing 

straight into the clouds and staying there in level flight—

implying a conscious decision to maintain altitude despite 

being able to see the ground.

A third plausible set of related hypotheses is that 

many, if not all, of these pilots simply either forgot the 

instructions, failed to take a simulated flight as seriously 

as they would a real one, succumbed to old habit patterns 

(the IFR pilots), or may have simply been overwhelmed 

trying to deal with an unfamiliar aircraft over unfamiliar 

terrain. Otherwise, we are left with the unpalatable and 

implausible notion that some 20% of GA pilots willfully 

fly VFR-into-IMC.

This issue is serious enough to be revisited in a Phase 

3 report, to see if those data shed any light on the situation. Should this behavior greatly decrease, then a case 

might be made for these violations being inadvertent, 

rather than willful.

Modeling flight behavior.

Cluster analysis. One of the most interesting questions 

we wanted to answer is: “What differentiated pilots who 

chose to complete the flight through deteriorating weather 

from pilots who chose not to complete the flight?” To 

investigate this question meant constructing models—

simplifications of the situation that still captured its 

major, essential features.

Cluster analysis is one approach to modeling. Cluster 

analysis starts with a set of measurements (“variables”) 

taken on individuals (“cases”—here, individual pilots). It 

then explores the relations between variables by combining 

individual cases into groups (“clusters”). The end goal is to 

group cases so that those within the same cluster are more 

similar to each other than they are to cases from different 

clusters. This similarity is operationalized by calculating 

a “mathematical distance” between cases. Once done, it 

becomes the job of the researcher to interpret what each 

cluster means in logical and practical terms.

3

4

5

6

7

8

9

0.5 1.5 2.5 3.5

Time in IMC (minutes)

Number of pilots

VFR IFR

≥4

Figure 17a. This shows number of pilots (y-axis) by 

time-spent-in-IMC (x-axis) for VFR and IFR pilots with 

at least 0.5 min in IMC. Figure 17b. More-detailed 

frequency counts spanning the time range 0.5 – 82.6 

min, for VFR and IFR combined. 

Figure 17 a. (top). This shows number of pilots 

(y-axis) by time-spent-in-IMC (x-axis) for VFR and 

IFR pilots with at least 0.5 min in IMC.

Figure 17b. More detailed frequency counts 

spanning the time range 0.5–82.6 min, for VFR and 

IFR combined.
18 

Here, we used a “TwoStep” cluster analysis procedure 

(SPSS, 2001) to classify the pilots based on demographic 

characteristics as well as behavioral responses to the simulated flight scenario. Schwarz’s Bayesian Criterion (BIC) 

was used as the clustering criterion and the log-likelihood 

was used as the distance measure. The number of clusters 

was determined automatically within the two-step process 

(maximum set to 15). In the first step, sequential clustering 

calculated the BIC for each cluster within a specified range 

and used that to estimate the initial number of clusters.

In the second step, the estimate of clusters was reduced 

by finding the largest increase in “mathematical distance” 

between the two closest clusters using an agglomerative 

hierarchical clustering method (SPSS, 2001).

SPSS’s “TwoStep” cluster analysis works with both 

categorical and continuous variables when using the 

log-likelihood method. Assumptions of normality often 

tend to be relaxed in cluster analysis, so nonparametric 

follow-up tests were used to examine individual relations 

(see below).

We selected candidate variables based upon logic and 

prior results of correlational analysis (e.g., Table 3). The 

initial categorical candidate variables were: 

1. Weather training product (1, 2, or Control [C])

2. Pilot’s rating (VFR or IFR)

3. Local pilot or not (Locality of residence)

4. Go/NoGo takeoff decision (TO Decision)

5. Did they make a preflight weather call prior to takeoff?

6. Number of en route weather updates

7. If they inadvertently flew into Instrument Meteorological Conditions (IMC) during the simulation

8. Final flight decision (discussed below)

Final flight decision was initially divided into 4 

categories, a) Return to AMA, b) Divert to alternative 

airport, c) Go direct to ABQ, or d) Go indirectly to ABQ 

by flying north or south around the nearby mountain 

range. Logically, a and b represented diversions, whereas 

c and d represented completed flights to ABQ. Therefore, after excluding the single CFIT from analysis, final 

flight decision was simplified by collapsing the initial 4 

categories into a binary (2-category) categorical variable 

To ABQ (“Did a given pilot complete the entire flight to 

ABQ, yes or no?”).

The following continuous candidate variables were also 

considered for cluster analysis:

1. Age

2. Total flight hours

3. Minimum final statute miles from ABQ (MDABQ)

4. Minutes spent in IMC

5. Minutes spent less than 500' below the cloud ceiling 

(scud running)

6. Minutes spent at less than 500' AGL

7. Number of weather rechecks after the 30-45 min 

preflight ground delay (Recheck)

Note that weather knowledge scores were excluded 

for non-significance (see Table 3).

Summary of the cluster analysis

The analysis excluded 2 additional cases for incomplete 

data. The remaining cases sorted into just 2 clusters.

Individual variable results were then significance-tested, 

both by SPSS confidence intervals and cross-checked by 

alternate tests.

Table 5 shows all significant (p <.05) Cluster 1-2 

differences. To summarize Table 5, compared to the 32 

Cluster 1 pilots, the 16 Cluster 2 pilots tended to be

1. younger, (Age)

2. lower flight hours, (FH)

3. closer final minimum distance to ABQ, (MDABQ)

4. more minutes flying less than 500' AGL, (M<500AGL)

5. usually did not receive a weather training product, 

(Trg Product)

6. greater % “Go” responses for takeoff (100%), (Takeoff 

[TO] Decision)

7. less likely to recheck weather just before takeoff,

(Recheck, Y=Yes)

8. greater % flew all the way to ABQ (100%), (To ABQ)

Interestingly, 100% of the 16 Cluster 2 pilots flew 

direct to ABQ through the nearby mountain pass, whereas 

only 1 pilot in Cluster 1 did so. In stark contrast, over 

84% of Cluster 1 pilots decided to divert or return to 

the departure airport (AMA). Of the 4 Cluster 1s who 

did fly to ABQ, 3 did so by flying completely around the 

troublesome mountain range.

Together, these results support the notion of Cluster 

2 pilots as greater risk-takers.

Table 5. Cluster analysis, significant results (p < .05). Trg Product TO Decision Recheck? To ABQ? 

Min<500AGL 1 2 C NoGo Go N Y N Y 

Cluster n Age1 FH1 MDABQ1 Mean Median n n n n n n n n n 

1 32 48.5 482 197.6 .25 0.0 12 13 7 12 20 7 25 27 4 

2 16 23.0 132 4.8 1.96 0.0 2 3 11 0 16 9 7 0 16 

pdifference <.0012 .0042 <.0012 .0062 .0073 .0053 0173 1.0E-83

1

Median. 2Mann-Whitney U test. 3Chi-square test. n=number (frequency count). 
19 

In contrast, there were no significant differences 

between Cluster 1 and Cluster 2 for instrument rating, 

locality of residence, number of en route weather updates, 

minutes spent in IMC, or minutes scud running.

Binary logistic regression analysis. From Table 3, we 

recall that some of the candidate variables in the cluster 

analysis were significantly correlated (e.g., rs_age-flight h = .757).

Therefore, to further simplify the model, we used stepwise 

forward likelihood-ratio binary logistic regression to cull 

out redundant (highly correlated) 

variables. Binary logistic regression makes no assumptions about 

the distributions of the predictor 

variables (Tabachnick & Fidell, 

2001). It can take a candidate set of 

variables, categorical or continuous, 

and select only those demonstrating 

significant orthogonal (uncorrelated) ability to help predict a binary outcome.

Specifically, we wanted to predict which pilots would or 

would not risk flying completely through the deteriorating 

weather (To ABQ = 1/Yes or 0/No). Table 6 summarizes 

the smallest set of variables capable of doing that reliably.

Note that TO Decision reflects “takeoff hesitancy” as 

discussed earlier, and that training product is broken 

out into its 3 groups. Negative B-weights mean that a 

positive value for the independent variable subsequently 

related to a reduced groupwise tendency to fly all the way 

to ABQ.

14 For example, pilots hesitant to take off (TO 

Decision = 1) subsequently showed a reduced tendency 

to fly all the way to ABQ. Similarly, pilots receiving 

either weather training product subsequently showed 

reduced tendency to fly all the way to ABQ, compared 

to the Control group.

In practical terms, this is a moderately strong model, 

accounting for 64.0% of the explainable (Nagelkerke) 

variance in the data. It implies that pilot experience (flight 

hours) may work in combination with an instinctive reaction to a weather situation to affect ultimate continuation 

into adverse weather. Impulsivity may be further reduced 

by the presentation of a training product. This contrasts 

somewhat with the null conclusion reached earlier about 

training product, so we will revisit that theme in the 

Discussion section.

Table 7 compares the prediction success rate for completed flight to ABQ made by logistic regression (boldface) 

14 B-weight functions similarly to a correlation coefficient, except that 

it is not restricted to -1≤ r ≤1. The Control group has no B-weight 

because SPSS essentially used it as an “invisible baseline,” in relation 

to which the B-weights Trg Prod1 and Trg Prod2 could then be 

compared. For example, BTP1 = -3.08, which is less than 0, means 

that TrgProd1 pilots were less likely to complete the flight to ABQ 

than were Control group pilots.

versus cluster analysis (italics, in parentheses). Grey cells 

represent successful predictions.

This shows that a simplified logistic model containing 

only pilot age, initial takeoff decision, and training product 

correctly predicted 83.3% of these pilots’ overall decisions 

whether or not to fly through the deteriorating weather all 

the way to ABQ. The model was slightly better at predicting 

those who did not make it to ABQ (86.7% correct) than 

it was at those who did (77.8% correct).

Overall, this 3-variable model produced a gain of about 

21% from the base rate predicted by a constant only (62.5%, 

p=.000004). Compare this to the 8-variable cluster model’s 

correct predictions of 91.5%, versus a “complete” 15-variable 

logistic regression (not shown) where 100% of all cases 

were predicted correctly. However, note that the “complete” 

model was vastly overfitted, meaning it contained too many 

predictors given the number of cases. A case/predictor ratio 

of > 10/1 is typically a rule of thumb in regression analysis, 

implying that our models should arguably be limited to 

48/10 = 4-5 predictors. This shows that modeling involves 

a tradeoff. Simpler models, while offering somewhat less 

accurate predictions, compensate with greater reliability.

Finally, recall our earlier statements that completing 

the entire flight to ABQ did not always reflect dangerous 

behavior (as measured, for example, by time spent < 500' 

AGL). In fact, we cross-checked the logistic regression model 

against regular linear regression15 by substituting the original 

binary outcome variable To ABQ with the continuous flight 

risk outcome variable Minutes < 500' AGL. This showed 

no effect of the “FH+TO Decision+Training Product” 

model on actual flight risk (p = .612, NS). So, again, we 

15 However, bear in mind that this was not technically a reliable 

analysis because flight hours and minutes < 500' AGL were both 

severely non-normal distributions, which violates the assumptions 

of linear regression.

Table 6. Binary logistic regression for To ABQ 

 B pif term removed

Age - 0.081 .002 

TO decision -21.20 .016 

Control14

 Trg Prod 1 - 3.08 

Trg Prod 2 - 2.53 

.006 

Constant 4.64 

Nagelkerke R2 = .640 

Table 7. Success rate for binary logistic regression versus (cluster 

analysis)

Predicted To ABQ Observed To ABQ Did not make it to ABQ Made it to ABQ % correct 

Did not make it to ABQ 26 (27) 4 (4) 86.7 (87.1) 

Made it to ABQ 4 (0) 14 (16) 77.8 (81.3) 

Overall % correct Base logistic prediction rate = 62.5% 83.3 (91.5) 
20 

need to consider the distinction between mere flying and 

the subsequent hazard of flying.

Cognitive versus constructivist training product. In assessing the training products, a goal was to ascertain outcome 

differences due to a cognitive training product versus a 

constructivist training product. Cognitive learning theory 

relies heavily on notions of memory and information processing. In contrast, constructivist learning theory assumes 

that people construct an inner representational world based 

on symbols and their relations. We therefore hypothesized 

that perhaps one or the other training product might show 

greater effect on weather knowledge versus subsequent 

weather-flight behavior.

However, the collected results above show no significant 

support for either paradigm. The results perhaps do favor the 

conclusion that both training products were reliably distinct 

from the Control group, but otherwise not from each other.

Nonetheless, training products by themselves did not seem 

to influence flight behavior. Instead, training products may 

act in concert with other factors (flight hours and initial 

takeoff decision) to predict whether or not a pilot tends to 

press on to the destination in the face of adverse weather.

Discussion

The purpose of Phase 1 of this research was to investigate 3 major questions:

1. Do video weather training products significantly affect 

general aviation (GA) pilot weather knowledge and 

flight behavior in the face of potential instrument 

meteorological conditions? 

2. How are modern Web-based weather products used 

during preflight briefing?

3. Do local Oklahoma GA pilots differ appreciably from 

others in either weather knowledge or weather-related 

flight behavior?

Question 1

Summary. No highly significant effects were found 

for two 90-minute video weather training products all 

by themselves on weather knowledge or subsequent 

flight safety on a simulated flight involving deteriorating weather. Effects could not be measured because the 

“signal” of the training product was small compared to 

the “noise” of individual variability between pilots.

The training products did demonstrably affect some 

aspects of flight behavior to a degree. Viewing either 

training product suppressed both initial willingness to 

take off and subsequent flight duration. However, that 

may have been due to pilots being cognitively primed 

to act conservatively, given the nature of the study. With 

encouragement from an authority figure, all the hesitators 

did fly. Nonetheless, their subsequent safety records were 

not discernibly different from the Control group, which 

received no weather training product.

We conclude that weather knowledge and GA weather 

flying behavior are likely too complex to be profoundly 

changed by any single, brief training product. However, 

it is similarly absurd to conclude that weather training 

products have “no effect.” An apt analogy is building a 

house. Just as it takes many bricks to build a house, it 

takes many study sessions to master the complexities of 

weather and to greatly affect subsequent weather flying 

behavior.

Method. In Phase 1 of this project, 50 GA pilots 

participated in a study that was designed to test pilot 

Figure 18. Graphical summary of major relations between single variables. 
21 

weather knowledge and flight behavior. Pilots took a 

general weather knowledge pre-test, followed by exposure 

either to 1 of 2 weather training videos (the experimental 

groups), or to a video having nothing to do with weather 

(the Control group). They then took a knowledge posttest to measure knowledge gain induced by the training 

product. Next, they planned for, and flew, a simulated 

flight mission through marginal weather from Amarillo, 

TX (AMA) to Albuquerque, NM (ABQ). Numerical flight 

data were collected and flight behaviors noted.

Results. In Figure 18, arrows graphically depict significant relations between variables, with correlation values 

overlaid. Single-headed arrows imply directional causation 

(e.g. instrument rating could conceivably cause minutes 

< 500' AGL to vary, but the reverse would not be true).

Double-headed arrows make no assumption about what 

might cause what.

First, viewing a single 90-minute weather training 

product did not all by itself demonstrably improve pilots’ 

general weather knowledge test scores. This is unsurprising, given the briefness of the training product compared 

to the great complexity of weather.

Second, as Figure 18 shows, relations between variables were complex. By itself, viewing a training product 

showed no direct effect on flight safety. Instead, compared 

to the Control group, both training products seemed to 

induce a mediating effect of takeoff hesitancy in the face of 

deteriorating weather. “Hesitators” flew only after direct 

encouragement by the experimenter. In contrast, 17 of 18 

Control group pilots took off without any encouragement.

Subsequently, hesitators continued their conservatism, tending to make significantly shorter flights than 

non-hesitators. Since the bulk of the scenario danger lay 

near the flight’s destination, we might imagine a chain 

of events—that the training product induced takeoff 

hesitancy, which induced shorter flights, which led to 

lower groupwise risk exposure. However, these separate 

events, though individually related, did not result in a 

statistically significant event chain from beginning to 

end. Figure 16 depicted that graphically—the beginningstate, independent variable of training product by itself 

did not significantly correlate with any of the end-state, 

dependent flight-risk variables (time scud running, time 

in IMC, or time at < 500' AGL).

Otherwise, there were no single distinguishing pilot 

characteristics (such as age, instrument rating, locality of 

residence, or flight hours) that seemed to explain takeoff 

hesitancy—other than having viewed either training 

product. If anything, the two separate factors most likely 

to produce hazardous behavior (as measured by minutes 

spent < 500' AGL) were simply lack of an instrument 

rating and low flight hours.

Single-variable correlational analysis was, therefore, 

followed up by multivariate modeling. In modeling flight 

behavior, we saw that a combination of higher pilot age, 

receiving either weather training product, and takeoff 

hesitancy correctly predicted 26 of 30 diversions (86.7%) 

from deteriorating weather (Tables 6-7, and Figure 18).

Successful flight completions (full penetration into the 

weather) were harder to predict with the same model (14 

of 18 correct = 77.8%). This suggests that the reasons 

why pilots divert may be slightly more homogeneous than 

the reasons why they press on into deteriorating weather.

An alternative explanation for these results. Objectively, 

some behavioral results for the weather training products 

might be explained by an alternate hypothesis unrelated 

to what we otherwise might expect. In a cognitive priming 

hypothesis, exposure to a particular stimulus “primes” the 

participant’s subsequent sensitivity to similar members 

of that category (James, 1890). For instance, presenting 

the word “nurse” tends to lower subsequent recognition 

response time for the word “doctor” more than it does 

for the word “lawyer.” Applying the priming concept 

here, exposure to the training product could, in effect, 

“tip off the participants” that the following study was 

to be about weather. Given the context of FAA officials 

conducting an experiment within an FAA facility, one 

could argue that some subsequent behavioral effects might 

owe more to the experimental groups (and, particularly, 

the older pilots within them) being primed to think 

about weather and risk than to a strict training effect of 

the video products themselves.

This issue of true learning versus priming will be revisited in Phase 2 of this study. If the hesitancy effect and/

or the shorter-flight effect persist over time, a stronger 

argument may be made for true learning to have taken 

place. This remains to be seen.

In the meantime, it should not be construed that 

weather training products “have no effect.” The proper 

conclusion is simply that weather and pilots are both 

complicated subjects. For a pilot to understand weather 

takes time and effort. Just as we do not build an entire 

house from one brick, we do not arrive at a deep understanding of weather from one 90-minute training 

module. It takes many.

Other findings. There were a few statistically significantbut-logically trivial relations.

• Instrument-rated pilots tended to be older and have more 

flight hours. Older pilots tended to have more flight hours.

Longer flights tended to bring one closer to the destination.

• Beyond the trivial, there was a slight-but-significant tendency for older pilots to spend a bit more time on their 

Web-based preflight briefing. This is consistent with the 

plausible assumption that older pilots may be less familiar 

with Web-based preflight briefings.
22 

• Instrument-rated pilots spent slightly less time too close 

to the ground (< 500' AGL). So did higher- flight hour 

pilots. However, instrument-rated pilots also tended to be 

older, with higher flight hours. So, it is subtle to pinpoint 

whether rating, age, or flight hours was most related to 

ground clearance.

• There was also a slight tendency for older pilots and higher 

flight hour pilots to fly shorter flights (meaning they penetrated the weather slightly less). Age and experience may 

engender some risk aversion (Hunter, 2002). Alternatively, 

younger pilots might have been merely “gaming the system,” 

treating the flight more like a game than a real flight. It is 

difficult to say, because either or both effects could operate, 

yet produce the same net result on measurable behavior.

• Finally, the one instance of actual CFIT seen in this study 

underscores the rather humbling methodological point that 

genuine accidents rarely follow the exact pattern implied by 

group statistics. This incident happened because of nothing more elaborate than momentary in-flight attentional 

lapse while the pilot was studying the sectional. This was 

a Control group pilot, but not younger or lower-flight 

hour, as our models would lead us to believe. So, once 

again, Nature reminds us that correlation does not imply 

causation.

Question 2

The construction of a data-collecting emulation of www.

aviationweather.gov was a significant, reusable achievement.

However, its use as a research tool proved far from simple, as 

Figure 12 shows.

Table 3 suggests that mere time spent on preflight briefing 

was not a good predictor of either quality of preflight briefing nor subsequent flight safety. Nonetheless, these data are 

just the opening salvos in what will eventually have to be a 

far more intensive study of modern weather briefing and its 

relation to flight safety.

Question 3

We did not see important differences between local pilots 

and non-local pilots. The only significant finding was that the 

locals took slightly less time to brief for this relatively local flight.

But, Oklahoma pilots are arguably more familiar with their 

own local terrain and weather patterns and need less briefing 

time for a flight such as this, so the issue is probably trivial.

More importantly, these findings directly address the issue 

of whether CAMI studies are generalizable to the national 

population of U.S. GA pilots. The fact is that U.S. pilots study 

a fairly uniform curriculum (largely driven by the licensing 

exams). This guarantees a measure of pilot uniformity. What 

is certainly far more important to research planning is the 

individual variation in knowledge and skill present between 

one pilot and another—not where a particular pilot happens 

to live. Yes, there are specific regions where certain flying 

skills are more called-upon than others. The high winds in 

the Midwestern U.S. are a good example. But—unless the 

task to which a given group of pilots is put depends critically 

on some small, specific set of skills—geographical region-ofresidence probably will not matter a great deal.

What this means is that researchers simply need to adhere 

to standard practices in selecting pilots and assigning them to 

treatment conditions. As long as designs are counterbalanced, 

and pilots are reasonably well-matched for age, flight hours, 

and instrument rating over treatment cells, there is probably 

only an occasional need to recruit non-locally. Our final 

cost figures put the human effort and dollar cost of testing a 

non-local pilot at approximately 5-10 times the expense of 

recruiting a local pilot. Therefore, what non-local pilots are 

best used for is precisely when an elite sample is required but 

not locally available. For instance, if high-hour, young, VFR 

pilots were needed for some reason, then we would probably 

want to consider recruiting non-locally.

Implications of this research for pilot training

The first training implication of this research has to do with 

the value of specificity. For commercial reasons, many flight 

training products are general, seeking to appeal to as broad an 

audience as possible. While a degree of generality can be good, 

when it comes to weather training, there is much to be said 

for specificity. Just as flight instructors teach pilots specifically 

how to recover from stalls and spins, there is arguable merit 

in teaching specific recovery strategies to fit specific weather 

encounters. For instance, the Aircraft Owners and Pilots Association offers an array of interactive on-line courses, many 

of which deal with weather recognition and decision making.

This leads to a second implication of this research, namely 

that certain kinds of learning are best done with real-world 

experience. However, for reasons of common sense and liability, VFR flight training purposely excludes bad weather 

encounters. Even IFR training typically avoids known extreme 

weather hazards. This leaves pilots to experience these hazards 

ab tempestas—by “encounter with storm,” that is, by accident, 

the hard way.

Finally, during this experiment, many pilots informally 

expressed the opinion that the future of weather briefing looks 

increasingly Internet-based, as opposed to coming solely from the 

Flight Service Station. If the future is, as we suspect, Web-based 

and graphical, rather than simply text-based, then pilot training 

will need to address these new technologies and trends. Pilots 

will have to become skilled at self-briefing through products such 

as www.aviationweather.gov, and these products themselves will 

become the subject of human factors research scrutiny through 

such methods as content analysis and usability testing.

All in all, the future looks hopeful. We have new technologies 

that can place up-to-date weather information within reach of 

any pilot with access to an Internet-enabled computing device.

We also have video instruction technologies capable of helping 
23 

pilots learn how to cope with the complexities of weather. The 

twin tricks will be to enable universal access to this weather 

information and to teach pilots how best to use it. In summary, 

what the current study has taught us is that, no matter how 

well we provide information and how well we teach pilots how 

to use it, weather is, and will remain, a formidable opponent.

References

Adams, R.J., & Ericsson, K.A. (1992). Introduction to 

cognitive processes of expert pilots. (Technical Report 

DOT/FAA/RD-92/12). Washington, DC: Federal 

Aviation Administration, Research and Development Service.

Bazargan, M. (2005). Analysis of national top 10 first 

causes of fatal accidents of general aviation between 

1983 and 2002. Part 1-national-all aircraft types. 

(Technical Report 61022). Daytona Beach, FL: 

Embry-Riddle Aeronautical University, Center for 

General Aviation Research.

Bud, M.J., Mengert, P., Ransom, S., & Stearns, M.D.

(1997). General aviation accidents, 1983-1994: 

Identification of factors related to controlled flight 

into terrain (CFIT) accidents. (Technical Report 

DOT/FAA/AAR 100-97-2). Washington, DC: 

Federal Aviation Administration, Office of Aviation Research.

Ellis, J.A., Semb, G.B., & Cole, B. (1998). Very longterm memory for information taught in school.

Contemporary Educational Psychology, 23, 419-33.

FAA (2008). 2008 airmen knowledge tests. Downloaded 1 

Jun, 2009 from http://www. faa.gov/data_research/

aviation_data_statistics/test_statistics/media/2008/

annual/2008_Annual Statistics for Airmen 

Knowledge Testing.pdf

Goldstein, E.B. (1999). Sensation & perception (5th Ed.).

New York: Brooks/Cole.

Hill Jr., F.S. (1990). Computer graphics. New York: 

Macmillan.

Hunter, D.R. (2002). Risk perception and risk tolerance 

in aircraft pilots. (Technical Report DOT/FAA/

AM-02-17). Washington, DC: Federal Aviation 

Administration, Office of Aerospace Medicine.

Hutchins, E. (1995). Cognition in the wild. Cambridge, 

MA: M.I.T. Press.

James, W. (1890). The principles of psychology (Vol. 2).

New York: Holt.

Mackintosh, N.J. (1974). The psychology of animal learning. New York: Academic Press.

NTSB (2008). Report SEA08LA095. Downloaded 

30 Jan, 2009 from NTSB Aviation Database & 

Synopses site www.ntsb.gov/ntsb/brief.asp?ev_

id=20080331X00400&key=1.

NTSB (1989). Safety report: General aviation accidents 

involving visual flight rules flight into instrument 

meteorological conditions (Technical Report NTSB/

SR-89/01). Washington, DC: National Transportation Safety Board.

NTSB (2005). Risk factors associated with weather-related 

general aviation accidents (Technical Report NTSB/

SS-05/01). Washington, DC: National Transportation Safety Board.

Perkins, D.M. (1992). Transfer of learning. Downloaded 

25 Nov, 2008 from http://learnweb.harvard.edu/

alps/thinking/docs/traencyn.htm

Semb, G.B., & Ellis, J.A. (1994). Knowledge taught in 

schools: What is remembered? Review of Educational 

Research, 64, 253-86.

Statistical Package for the Social Sciences. (2001). The 

SPSS two-step cluster component. SPSS Technical 

Report TSCWP-0101.

Tabachnick, B.G. & Fidell, L.S. (2001). Using multivariate statistics (4th Ed.). Boston: Allyn & Bacon.

von Glasersfeld, E. (1995). Radical constructivism: A way 

of knowing and learning. London: Falmer Press.

Waltz, D. & Feldman, J.A. (1988). Connectionist models 

and their implications. Readings from cognitive science. Norwood, NJ: Ablex.

Wiegmann, D.A., Talleur, D.A., & Johnson, C.M.

(2008). Redesigning weather-related training and 

testing of general aviation pilots: Applying traditional 

curriculum evaluation and advanced simulation-based 

methods. (Technical Report FAA-08-1). Downloaded 19 Nov, 2008 from www.humanfactors.uiuc.

edu/ Reports&PapersPDFs/TechReport/08-01.pdf.

Wiggins, M. & O’Hare, D. (2003). Expert and novice pilot perceptions of static in-flight images of 

weather. International Journal of Aviation Psychology, 13, 173-87.

A1 

APPENDIX A 

Web preflight briefing screenshots 

B1 

APPENDIX B

Sample hypertext markup language (html) code used to emulate the aviationweather.gov home page in Microsoft 

Visual Studio. The core html was taken from www.aviationweather.gov and modified for use here.

<!DOCTYPE HTML PUBLIC “-//W3C//DTD HTML 4.01 Transitional//EN” “http://www.w3c.org/TR/1999/REC-html401-19991224/loose.dtd”>

<!-- saved from url=(0032)http://adds.aviationweather.gov/ -->

<HTML><HEAD><TITLE>ADDS - Aviation Digital Data Service</TITLE>

<META http-equiv=Content-Type content=”text/html; charset=iso-8859-1”>

<META 

content=”Aviation Digital Data Service (ADDS)
 provides comprehensive user-friendly aviation weather graphics including
 icing, 

turbulence, and convection.” 

name=description>

<META 

content=”aviation, weather, icing, turbulence,
 convection, pirep, metar, taf, airmet, sigmet, satellite, radar, surface,
 wind, temperature, aloft, airplane, NEXRAD, GOES, WSR-88D, precipitation,
 rain, snow, sleet, thunderstorm, en-route, prognosis, chart” 

name=keywords>

<META content=”ADDS - Aviation Digital Data Service” name=DC.title>

<META content=”ADDS - Aviation Digital Data Service” name=DC.description>

<META content=”NOAA's National Weather Service - Aviation Digital Data Service” 

name=DC.creator>

<META scheme=ISO8601 content=2003-07-21 name=DC.date.created>

<META scheme=ISO8601 content=2005-10-19 name=DC.date.reviewed>

<META scheme=DCTERMS.RFC1766 content=EN-US name=DC.language><LINK 

href=”ADDS - Aviation Digital Data Service_files/mystyle.css” type=text/css 

rel=StyleSheet><LINK href=”ADDS - Aviation Digital Data Service_files/menu.css” 

type=text/css rel=StyleSheet><LINK 

href=”ADDS - Aviation Digital Data Service_files/flyout.css” type=text/css 

rel=StyleSheet><LINK href=”ADDS - Aviation Digital Data Service_files/main.css” 

type=text/css rel=STYLESHEET><LINK href=”/layout/ops/favicon.ico” 

rel=”shortcut icon”>

<STYLE type=text/css>BODY {

FONT-SIZE: 10pt; MARGIN: 0px; COLOR: #000000; FONT-FAMILY: Verdana, Arial, Helvetica, sans-serif

}

.bc {

FONT-WEIGHT: normal; FONT-SIZE: smaller; FONT-STYLE: normal; FONT-FAMILY: sans-serif

}

</STYLE>

<SCRIPT language=JavaScript type=text/javascript>

<!--

function setNavChildren(node) {

 if (node.nodeName == “LI”) {

 node.onmouseover=function() {

 this.className=”over-” + this.className;

 }

 node.onmouseout=function() {

 this.className=this.className.replace(“over-”, “”);

 }

 }

 for(var child = node.firstChild; child != null; child = child.nextSibling) {

 setNavChildren(child);

 }

}

var startList = function() {

 if (document.all && document.getElementById) {

 navRoot = document.getElementById(“nav”);

 if (navRoot) {

 for (i=0; i < navRoot.childNodes.length; i++) {

 setNavChildren(navRoot.childNodes[i]);

 }

 }

 }

}

// It has to be in a function or the prevLoad variable will cause infinite recursion

function addStartList() {

 var prevLoad = window.onload;

 window.onload = function() {

 if (prevLoad) {

 prevLoad();

 }

 startList();

 }

}
B2 

addStartList();

// -->

</SCRIPT>

<META content=”MSHTML 6.00.2900.3086” name=GENERATOR></HEAD>

<BODY bottomMargin=0 leftMargin=0 

background=”ADDS - Aviation Digital Data Service_files/background.gif” 

topMargin=0 rightMargin=0 margin=”0” marginwidth=”0” marginheight=”0”>

<TABLE cellSpacing=0 cellPadding=0 width=”100%” 

summary=”This table is used for formatting purposes only” 

background=”ADDS - Aviation Digital Data Service_files/topbanner.jpg” 

 border=0><TBODY>

 <TR>

 <TD align=right height=19><A 

 href=”http://adds.aviationweather.gov/#contents”><IMG height=1 

 alt=”Skip Navigation Links” 

 src=”ADDS - Aviation Digital Data Service_files/skipgraphic.gif” width=1 

 border=0></A> <A href=”http://weather.gov/”><SPAN 

 class=nwslink>weather.gov</SPAN></A> </TD></TR></TBODY></TABLE>

<TABLE cellSpacing=0 cellPadding=0 width=”100%” 

summary=”This table is used for formatting purposes only” border=0>

 <TBODY>

 <TR>

 <TD><A href=”http://www.noaa.gov/”><IMG height=78 

 alt=”NOAA logo - Click to go to the NOAA homepage” 

 src=”ADDS - Aviation Digital Data Service_files/noaaleft.jpg” width=85 

 border=0></A></TD>

 <TD align=middle>

 <TABLE cellSpacing=0 cellPadding=0 width=500 

 summary=”This table is used for formatting purposes only” border=0>

 <TBODY>

 <TR align=middle>

 <TD 

 background=”ADDS - Aviation Digital Data Service_files/blank_title.jpg” 

 height=20><A href=”http://www.weather.gov/”><SPAN 

 class=regionwhite>NOAA’s National Weather Service</SPAN></A> </TD></TR>

 <TR align=middle>

 <TD noWrap align=middle 

 background=”ADDS - Aviation Digital Data Service_files/blank_name.jpg” 

 height=58><A href=”http://aviationweather.gov/”><SPAN 

 class=officewhite>Aviation Weather Center</SPAN></A><BR><A 

 href=”http://adds.aviationweather.gov/”><SPAN 

 class=regionwhite>Aviation Digital Data Service 

 (ADDS)</SPAN></A></TD></TR></TBODY></TABLE>

 <TD width=”100%” 

 background=”ADDS - Aviation Digital Data Service_files/ncep_bkgrnd.jpg”> </TD>

 <TD align=right><A href=”http://www.nws.noaa.gov/”><IMG height=78 

 alt=”NWS logo - Click to go to the NWS homepage” 

 src=”ADDS - Aviation Digital Data Service_files/nwsright.jpg” width=85 

 border=0></A></TD></TR></TBODY></TABLE>

<FORM id=sform action=http://www.usasearch.gov/search method=get>

<TABLE cellSpacing=0 cellPadding=0 width=”100%” 

summary=”This table is used for formatting purposes only” 

background=”ADDS - Aviation Digital Data Service_files/navbkgrnd.gif” 

 border=0><TBODY>

 <TR>

 <TD vAlign=top align=left width=94><IMG height=23 alt=”” 

 src=”ADDS - Aviation Digital Data Service_files/navbarleft.jpg” width=94 

 border=0></TD>

 <TD class=nav id=menuitem align=middle width=”15%”><A 

 href=”http://adds.aviationweather.gov/”>Home</A></TD>

 <TD class=nav id=menuitem align=right width=”15%”><A 

 href=”http://www.weather.gov/pa/”>News</A></TD>

 <TD class=nav id=menuitem align=right width=”20%”><A 

 href=”http://www.weather.gov/organization.php”>Organization</A></TD>

 <TD class=yellow align=right width=”20%”><LABEL for=sform>Search</LABEL> 

  </TD>

 <TD class=searchinput noWrap align=left width=”20%”><INPUT type=hidden 

 value=nws.noaa.gov name=affiliate> <INPUT type=hidden value=firstgov 

 name=v:project> <INPUT maxLength=256 name=query>   <INPUT type=submit value=Go> </TD>

 <TD width=”10%”> </TD>

 <TD vAlign=bottom align=right width=24><IMG height=23 alt=”” 

 src=”ADDS - Aviation Digital Data Service_files/navbarendcap.jpg” width=24 

 border=0></TD></TR></TBODY></TABLE></FORM>

<TABLE cellSpacing=0 cellPadding=0 width=810 

summary=”This table is used for formatting purposes only” border=0>
B3 

 <TBODY>

 <TR vAlign=top>

 <TD>

 <TABLE cellSpacing=0 cellPadding=0 width=130 

 summary=”This table is used for formatting purposes only” border=0>

 <TBODY>

 <TR vAlign=top align=left>

 <TD><!-- <form style=”margin:0 0 8px 0” method=”POST” action=”http://www.srh.noaa.gov/zipcity.php”> -->

 <FORM id=weather style=”MARGIN: 0px 0px 8px” 

 action=/phputils/airport_weather.php method=post><LABEL class=yellow 

 for=location>Local forecast by<BR>”City, St” or Zip 

 Code</LABEL><BR><INPUT class=searchinput id=location size=8 

 value=”City, St” name=inputstring>  <INPUT type=submit value=Go name=Go2></FORM></TD></TR>

 <TR vAlign=top align=left>

 <TD>

 <UL id=nav>

 <LI><SPAN class=yellow>Advisories</SPAN> 

 <UL>

 <LI><A 

 href=”http://adds.aviationweather.gov/airmets/”>SIGMET/AIRMET 

 <SPAN class=indicator>»</SPAN></A> 

 <UL>

 <LI><A 

 href=”http://adds.aviationweather.gov/airmets/java/”>Java 

 Tool</A> </LI>

 <LI><A href=”http://adds.aviationweather.gov/airmets/”>US 

 (CONUS)</A> </LI>

 <LI><A 

 href=”http://aviationweather.gov/products/sigmets/intl/”>International</A> 

 </LI></UL></LI>

 <LI><A href=”http://aviationweather.gov/products/cwsu/”>Center 

 Weather</A> </LI></UL>

 <LI><SPAN class=yellow>Forecasts</SPAN> 

 <UL>

 <LI><A 

 href=”http://adds.aviationweather.gov/convection/”>Convection 

 <SPAN class=indicator>»</SPAN></A> 

 <UL>

 <LI><A 

 href=”http://adds.aviationweather.gov/airmets/”>Convective 

 SIGMETs</A> </LI>

 <LI><A 

 href=”http://aviationweather.gov/products/ccfp/”>CCFP</A> 

</LI>

 <LI><A 

 href=”http://adds.aviationweather.gov/convection/java/”>NCWF 

 Java Tool</A> </LI>

 <LI><A href=”http://aviationweather.gov/products/ncwf”>NCWF 

 Loop</A> </LI>

 <LI><A 

 href=”http://www.spc.noaa.gov/products/outlook/day1otlk.html”>Convective 

 Outlook</A> </LI>

 <LI><A 

 href=”http://www.spc.noaa.gov/products/watch”>Convective 

 Watches</A> </LI>

 <LI><A href=”http://www.spc.noaa.gov/products/md”>Mesoscale 

 Discussion</A> </LI></UL></LI>

 <LI><A 

 href=”http://adds.aviationweather.gov/turbulence/”>Turbulence</A> 

 </LI>

 <LI><A href=”http://adds.aviationweather.gov/icing/”>Icing</A> 

 </LI>

 <LI><A href=”http://adds.aviationweather.gov/winds/”>Winds/Temps 

 <SPAN class=indicator>»</SPAN></A> 

 <UL>

 <LI><A 

 href=”http://aviationweather.gov/products/nws/winds/”>Winds/Temps 

 Text</A> </LI>

 <LI><A 

 href=”http://adds.aviationweather.gov/winds/”>Winds/Temps 

 Images</A> </LI></UL></LI>

 <LI><A href=”http://adds.aviationweather.gov/progs/”>Prog Charts 

 <SPAN class=indicator>»</SPAN></A> 

 <UL>

 <LI><A 

 href=”http://aviationweather.gov/products/swh/”>High-Level 
B4 

 (250-630)</A> </LI>

 <LI><A 

 href=”http://aviationweather.gov/products/swm/”>Mid-Level 

  (100-450)</A> </LI>

 <LI><A 

 href=”http://aviationweather.gov/products/swl/”>Low-Level 

 (SFC-240)</A> </LI>

 <LI><A 

 href=”http://adds.aviationweather.gov/progs/”>Surface</A> 

 </LI></UL></LI>

 <LI><A href=”http://adds.aviationweather.gov/tafs/”>TAF / FA 

 <SPAN class=indicator>»</SPAN></A> 

 <UL>

 <LI><A href=”http://aviationweather.gov/products/fa/”>Area 

 Forecasts</A> </LI>

 <LI><A href=”http://adds.aviationweather.gov/tafs/java/”>TAF 

 Java Tool</A> </LI>

 <LI><A href=”http://adds.aviationweather.gov/tafs/”>TAF 

 Request Forms</A> </LI></UL></LI></UL>

 <LI><SPAN class=yellow>Observations</SPAN> 

 <UL>

 <LI><A href=”http://adds.aviationweather.gov/pireps/”>PIREPs 

 <SPAN class=indicator>»</SPAN></A> 

 <UL>

 <LI><A 

 href=”http://adds.aviationweather.gov/pireps/java/”>PIREP Java 

 Tool</A> </LI>

 <LI><A href=”http://adds.aviationweather.gov/pireps/”>PIREP 

 Request Forms</A> </LI></UL></LI>

 <LI><A href=”http://adds.aviationweather.gov/metars/”>METARs 

 <SPAN class=indicator>»</SPAN></A> 

 <UL>

 <LI><A 

 href=”http://adds.aviationweather.gov/metars/java/”>METAR Java 

 Tool</A> </LI>

 <LI><A href=”http://adds.aviationweather.gov/metars/”>METAR 

 Request Form</A> </LI></UL></LI>

 <LI><A href=”http://adds.aviationweather.gov/radar/”>Radar <SPAN 

 class=indicator>»</SPAN></A> 

 <UL>

 <LI><A href=”http://adds.aviationweather.gov/radar/”>Site 

 Radar</A> </LI>

 <LI><A href=”http://aviationweather.gov/obs/radar/”>United 

 States Radar</A> </LI></UL></LI>

 <LI><A 

 href=”http://adds.aviationweather.gov/satellite/”>Satellite 

 <SPAN class=indicator>»</SPAN></A> 

 <UL>

 <LI><A 

 href=”http://adds.aviationweather.gov/satellite/”>United 

 States</A> </LI>

 <LI><A 

 href=”http://aviationweather.gov/obs/sat/intl/”>International</A> 

 </LI></UL></LI></UL></LI>

 <LI class=flyout><A 

 href=”http://adds.aviationweather.gov/java/”>Java Tools <SPAN 

 class=indicator>»</SPAN></A> 

 <UL class=flyout>

 <LI><A 

 href=”http://adds.aviationweather.gov/fpt_application/”>FPT 

 Application</A> </LI>

 <LI><A 

 href=”http://adds.aviationweather.gov/flight_path/”>Flight Path 

 Tool</A> </LI>

 <LI><A 

 href=”http://adds.aviationweather.gov/airmets/java/”>AIR/SIGMETs</A> 

 </LI>

 <LI><A 

 href=”http://adds.aviationweather.gov/convection/java/”>Convection</A> 

 </LI>

 <LI><A 

 href=”http://adds.aviationweather.gov/tafs/java/”>TAFs</A> </LI>

 <LI><A 

 href=”http://adds.aviationweather.gov/pireps/java/”>PIREPs</A> 

 </LI>

 <LI><A 
B5 

 href=”http://adds.aviationweather.gov/metars/java/”>METARs</A> 

 </LI></UL>

 <LI><SPAN class=yellow>Related Information</SPAN> 

 <UL>

 <LI><A href=”http://adds.aviationweather.gov/”>Home <SPAN 

 class=indicator>»</SPAN></A> 

 <UL>

 <LI><A href=”http://adds.aviationweather.gov/”>ADDS Home</A> 

 </LI>

 <LI><A href=”http://aviationweather.gov/”>AWC Home</A> 

 </LI></UL></LI>

 <LI><A href=”http://aviationweather.gov/iffdp/”>Flight 

 Folder</A> </LI>

 <LI><A href=”http://aviationweather.gov/std_brief/”>Standard 

 Briefing</A> </LI>

 <LI><A href=”http://aviationweather.gov/testbed/”>Aviation 

 Testbed</A> </LI>

 <LI><A href=”http://aviationweather.gov/static/links/”>Aviation 

 Links</A> </LI></UL>

 <LI><SPAN class=yellow>Contact Us</SPAN> 

 <UL>

 <LI><A href=”http://aviationweather.gov/static/faq/”>FAQ</A> 

 </LI>

 <LI><A 

 href=”mailto:ncep.awc.adds@noaa.gov?subject=ADDS - Aviation Digital Data Service -- Feedback”>Feedback 

 <SPAN class=indicator>»</SPAN></A> 

 <UL>

 <LI><A 

 href=”mailto:ncep.awc.adds@noaa.gov?subject=ADDS - Aviation Digital Data Service -- Feedback”>ADDS 

 Feedback</A> </LI>

 <LI><A 

 href=”mailto:ncep.awc.avwx@noaa.gov?subject=AWC WEB Page -- Feedback”>AWC 

 Feedback</A> </LI></UL></LI>

 <LI><A href=”http://adds.aviationweather.gov/info/”>Site 

 Information <SPAN class=indicator>»</SPAN></A> 

 <UL>

 <LI><A href=”http://adds.aviationweather.gov/info/”>ADDS Site 

 Info</A> </LI>

 <LI><A href=”http://aviationweather.gov/static/info/”>AWC Site 

 Info</A> </LI></UL></LI></UL></LI></UL><BR><BR clear=all></TD></TR>

 <TR>

 <TD align=middle><A 

 href=”http://www.cfigroup.net/nwsAviWxsurvey3”><IMG 

 style=”MARGIN: 0px 7px 0px 0px” height=43 

 alt=”NWS Aviation Services Customer Satisfaction Survey 2007” 

 src=”ADDS - Aviation Digital Data Service_files/NOAA_Survey.jpg” 

 width=104 border=0></A></TD></TR>

 <TR>

 <TD align=middle><A href=”http://celebrating200years.noaa.gov/”><IMG 

 style=”MARGIN: 0px 7px 0px 0px” height=43 

 alt=”NOAA is celebrating 200 years of science, service, and stewardship. Visit the NOAA 200th celebration Web site to learn more.” 

 src=”ADDS - Aviation Digital Data Service_files/noaa200insignia100.jpg” 

 width=100 border=0></A></TD></TR>

 <TR>

 <TD align=middle><A href=”http://www.usa.gov/”><IMG 

 style=”MARGIN: 0px 7px 0px 0px” height=30 

 alt=”USA.gov is the U.S. Government’s official Web portal to all Federal, state and local government Web resources and services.” 

 src=”ADDS - Aviation Digital Data Service_files/usagov_logo.gif” 

 width=110 border=0></A></TD></TR></TBODY></TABLE></TD>

 <TD vAlign=top align=left>

 <TABLE cellSpacing=0 cellPadding=0 width=680 align=left 

 summary=”This table is used for formatting purposes only” border=0>

 <TBODY>

 <TR vAlign=top>

 <TD vAlign=center align=right rowSpan=2><A 

 href=”http://adds.aviationweather.gov/”><IMG height=34 

 alt=”ADDS Home Page” hspace=0 

 src=”ADDS - Aviation Digital Data Service_files/home_w.gif” width=65 

 align=left border=0></A></TD>

 <TD width=680><A 

 href=”http://adds.aviationweather.gov/turbulence/”><IMG height=17 

 alt=Turbulence hspace=0 

 src=”ADDS - Aviation Digital Data Service_files/turb1.gif” width=90 

 align=left border=0></A><A 

 href=”http://adds.aviationweather.gov/icing/”><IMG height=17 

 alt=Icing hspace=0 
B6 

 src=”ADDS - Aviation Digital Data Service_files/icg1.gif” width=65 

 align=left border=0></A><A 

 href=”http://adds.aviationweather.gov/convection/”><IMG height=17 

 alt=Convection hspace=0 

 src=”ADDS - Aviation Digital Data Service_files/convect1.gif” 

 width=87 align=left border=0></A><A 

 href=”http://adds.aviationweather.gov/winds/”><IMG height=17 

 alt=Winds/Temps hspace=0 

 src=”ADDS - Aviation Digital Data Service_files/wind1.gif” width=103 

 align=left border=0></A><A 

 href=”http://adds.aviationweather.gov/progs/”><IMG height=17 

 alt=”Prog Charts” hspace=0 

 src=”ADDS - Aviation Digital Data Service_files/progs1.gif” width=90 

 align=left border=0></A><A 

 href=”http://adds.aviationweather.gov/java/”><IMG height=17 

 alt=”Java Tools” hspace=0 

 src=”ADDS - Aviation Digital Data Service_files/java1.gif” width=85 

 align=left border=0></A></TD>

 <TD vAlign=center align=right rowSpan=2><A 

 href=”http://adds.aviationweather.gov/info/”><IMG 

 title=”ADDS General Help Page” height=32 

 alt=”ADDS General Help Page” hspace=0 

 src=”ADDS - Aviation Digital Data Service_files/fyi_help.gif” 

 width=32 align=right border=0></A></TD></TR>

 <TR vAlign=top>

 <TD width=680><A href=”http://adds.aviationweather.gov/metars/”><IMG 

 height=17 alt=METARs hspace=0 

 src=”ADDS - Aviation Digital Data Service_files/metars1.gif” 

 width=78 align=left border=0></A><A 

 href=”http://adds.aviationweather.gov/tafs/”><IMG height=17 alt=TAFs 

 hspace=0 src=”ADDS - Aviation Digital Data Service_files/tafs1.gif” 

 width=64 align=left border=0></A><A 

 href=”http://adds.aviationweather.gov/pireps/”><IMG height=17 

 alt=PIREPs hspace=0 

 src=”ADDS - Aviation Digital Data Service_files/pireps1.gif” 

 width=76 align=left border=0></A><A 

 href=”http://adds.aviationweather.gov/airmets/”><IMG height=17 

 alt=AIRMETs hspace=0 

 src=”ADDS - Aviation Digital Data Service_files/airmets1.gif” 

 width=113 align=left border=0></A><A 

 href=”http://adds.aviationweather.gov/satellite/”><IMG height=17 

 alt=Satellite hspace=0 

 src=”ADDS - Aviation Digital Data Service_files/satellite1.gif” 

 width=91 align=left border=0></A><A 

 href=”http://adds.aviationweather.gov/radar/”><IMG height=17 

 alt=Radar hspace=0 

 src=”ADDS - Aviation Digital Data Service_files/radar1.gif” width=98 

 align=left border=0></A></TD></TR></TBODY></TABLE><BR clear=all>

 <TABLE cellSpacing=0 cellPadding=0 width=”100%” 

 summary=”This table is used for formatting purposes only” border=0>

 <TBODY>

 <TR vAlign=top align=middle>

 <TD id=contents vAlign=top align=middle>

 <TABLE cellSpacing=0 cellPadding=10 width=680 bgColor=#ffffff 

 summary=”This table is used for formatting purposes only” 

 border=0><TBODY>

 <TR vAlign=top>

 <TD colSpan=2>

 <TABLE cellSpacing=0 cellPadding=0 width=”100%” 

 bgColor=#ffffff 

 summary=”This table is used for formatting purposes only” 

 border=0>

 <TBODY>

 <TR>

 <TD>

 <TABLE cellSpacing=0 cellPadding=0 width=”100%” 

 bgColor=#1b28b4 

 summary=”This table is used for formatting purposes only” 

 border=0>

 <TBODY>

 <TR>

 <TD align=left><IMG alt=”Top News of the Day” 

 src=”ADDS - Aviation Digital Data Service_files/newsleft.gif” 

 border=0> </TD>

 <TD align=right><IMG alt=”Top News of the Day” 

 src=”ADDS - Aviation Digital Data Service_files/newsright.gif” 
B7 

 border=0> </TD></TR></TBODY></TABLE></TD></TR>

 <TR vAlign=top>

 <TD align=left>

 <TABLE 

 style=”BORDER-RIGHT: #1b28b4 2px solid; BORDER-TOP: #1b28b4 2px solid; BORDER-LEFT: #1b28b4 2px solid; BORDERBOTTOM: #1b28b4 2px solid” 

 cellSpacing=0 cellPadding=4 width=”100%” 

 summary=”This table is used for formatting purposes only”>

 <TBODY>

 <TR>

 <TD vAlign=center align=middle width=”5%”><IMG 

 alt=”Top News of the Day” 

 src=”ADDS - Aviation Digital Data Service_files/noaabullet.gif” 

 border=0> </TD>

 <TD width=”95%”>December 6, 2006: A new icing 

 product (CIP-SEV) has been made operational, see 

 the <A 

 href=”http://adds.aviationweather.gov/icing”>icing 

 page</A> and its <A 

 href=”http://adds.aviationweather.gov/icing/description3.php”>FYI/Help</A> 

 for more details. In addition, a site update has 

 been released. Please see the <A 

 href=”http://adds.aviationweather.gov/info/ops_whats_new.php”>information 

 page</A> for more details. 

 </TD></TR></TBODY></TABLE></TD></TR></TBODY></TABLE></TD></TR>

 <TR vAlign=top>

 <TD align=left width=250>The <STRONG>Aviation Digital Data 

 Service (ADDS)</STRONG> makes available to the aviation 

 community text, digital and graphical forecasts, analyses, and 

 observations of aviation-related weather variables. ADDS is a 

 joint effort of NCAR Research Applications Program (<A 

 href=”http://www.rap.ucar.edu/”>RAP</A>), Global Systems 

 Division (<A href=”http://www.esrl.noaa.gov/gsd/”>GSD</A>) of 

 NOAA’s Earth System Research Laboratory (<A 

 href=”http://www.esrl.noaa.gov/”>ESRL</A>), and the National 

 Centers for Environmental Prediction (<A 

 href=”http://www.ncep.noaa.gov/”>NCEP</A>) Aviation Weather 

 Center (<A href=”http://aviationweather.gov/”>AWC</A>). <BR 

 clear=all> <BR><B>Current AIR/SIGMETs:</B><BR><A 

 href=”http://adds.aviationweather.gov/data/airmets/airmets_ALL.gif”><IMG 

 height=160 alt=”Current AIR/SIGMETs” hspace=0 

 src=”ADDS - Aviation Digital Data Service_files/airmets_ALL_sm.gif” 

 width=250 border=0></A> </TD>

 <TD align=left width=380><STRONG>The <A 

 href=”http://www.nws.noaa.gov/”>National Weather Service</A> 

 operationally supports this site as well as the following 

 operational products:</STRONG><BR><BR>

 <TABLE cellSpacing=0 cellPadding=3 width=380 align=left 

 summary=”This table is used for formatting purposes only” 

 border=1>

 <TBODY>

 <TR vAlign=top>

 <TD align=left> ?<STRONG> 

 METARs</STRONG><BR> ?<STRONG> 

 TAFs</STRONG><BR> ?<STRONG> 

 PIREPS</STRONG><BR> ?<STRONG> 

 AIR/SIGMETs</STRONG><BR> ?<STRONG> 

 Satellite</STRONG><BR> ?<STRONG> Radar</STRONG> </TD>

 <TD align=left> ?<STRONG> Analysis & Prognostic 

 Charts</STRONG><BR> ?<STRONG> Graphical wind & 

 temperature charts</STRONG><BR> ?<STRONG> National 

 Convective Weather Forecast</STRONG><BR> ?<STRONG> 

 Current & Forecast Icing 

 Potential</STRONG><BR> ?<STRONG> Graphical 

 Turbulence Guidance</STRONG> </TD></TR></TBODY></TABLE><BR 

 clear=all><BR>

 <TABLE cellSpacing=0 cellPadding=3 width=380 align=left 

 bgColor=#ccffff 

 summary=”This table is used for formatting purposes only” 

 border=1>

 <TBODY>

 <TR vAlign=top>

 <TD align=left><FONT color=black><STRONG>The <A 

 href=”http://www.faa.gov/”>Federal Aviation 

 Administration</A> funds and directs the continuing 

 development of ADDS as well as other experimental 
B8 

 products being developed by the <A 

 href=”http://www.faa.gov/aua/awr”>FAA Aviation Weather 

 Research Program</A> (AWRP).<BR><BR>The results of the 

 latest ADDS development efforts along with new 

 experimental AWRP algorithm results can be viewed on the 

 <A 

 onmouseover=”self.status=’http://weather.aero’; return true” 

 onmouseout=”self.status=’’; return true” 

 href=”http://adds.aviationweather.gov/layout/ops/jump_exp.php”>experimental 

 ADDS site</A>. 

</STRONG></FONT></TD></TR></TBODY></TABLE></TD></TR>

 <TR vAlign=top align=left>

 <TD colSpan=2>

 <TABLE cellSpacing=1 cellPadding=2 width=”100%” align=left 

 bgColor=#ffffff 

 summary=”This table is used for formatting purposes only” 

 border=0>

 <TBODY>

 <TR>

 <TD bgColor=#003399 colSpan=2><FONT 

 color=#ffffff><B>  Other 

 News</B>  (Updated: December 7, 2006)</FONT> 

 </TD></TR>

 <TR>

 <TD vAlign=center align=middle width=28><IMG 

 alt=”News Bullet” 

 src=”ADDS - Aviation Digital Data Service_files/noaabullet.gif” 

 border=0> </TD>

 <TD>07 December 2006: <A 

 href=”http://www.weather.gov/os/notification/tin06-59aaa_graph_turbulence.txt”>TIN 

 announcement for the GTG product update.</A> The update 

 scheduled for January 8 2006 has been postponed until 

 further notice. </TD></TR>

 <TR>

 <TD vAlign=center align=middle width=28><IMG 

 alt=”News Bullet” 

 src=”ADDS - Aviation Digital Data Service_files/secbullet.gif” 

 border=0> </TD>

 <TD>20 April 2006 : Link provided to Space Weather 

 Prediction Center?s (SWPC) experimental <A 

 href=”http://www.sec.noaa.gov/aviation/”>Space Weather 

 for Aviation Service Providers</A> on ?<A 

 href=”http://aviationweather.gov/testbed/”>Aviation 

 Testbed</A>? page. </TD></TR>

 <TR>

 <TD vAlign=center align=middle width=28><IMG 

 alt=”News Bullet” 

 src=”ADDS - Aviation Digital Data Service_files/noaabullet.gif” 

 border=0> </TD>

 <TD>COSPAS-SARSAT will cease satellite monitoring of the 

 121.5 MHz frequency on February 1, 2009. If you fly an 

 aircraft with an ELT, please visit <A 

 href=”http://www.sarsat.noaa.gov/”>http://www.sarsat.noaa.gov/</A> 

 for further information. 

 </TD></TR></TBODY></TABLE></TD></TR></TBODY></TABLE></TD></TR></TBODY></TABLE>

 <TABLE cellSpacing=0 cellPadding=0 width=”100%” align=center 

 summary=”This table is used for formatting purposes only” border=0>

 <TBODY>

 <TR vAlign=center align=middle>

 <TD bgColor=#ffffff>

 <TABLE cellSpacing=0 cellPadding=0 width=”96%” align=center 

 summary=”This table is used for formatting purposes only” 

 border=0><TBODY>

 <TR vAlign=center>

 <TD bgColor=#ffffff>

 <HR>

 </TD></TR></TBODY></TABLE></TD></TR>

 <TR vAlign=center>

 <TD align=middle width=”100%” bgColor=#ffffff><A class=linkWhite 

 href=”http://adds.aviationweather.gov/turbulence/”>Turbulence</A>     

 <A class=linkWhite 

 href=”http://adds.aviationweather.gov/icing/”>Icing</A>     

 <A class=linkWhite 

 href=”http://adds.aviationweather.gov/convection/”>Convection</A>     

 <A class=linkWhite 

 href=”http://adds.aviationweather.gov/winds/”>Winds/Temps</A>     
B9 

 <A class=linkWhite 

 href=”http://adds.aviationweather.gov/progs/”>Progs</A>     

 <A class=linkWhite 

 href=”http://adds.aviationweather.gov/java/”>Java Tools</A><BR><A 

 class=linkWhite 

 href=”http://adds.aviationweather.gov/metars/”>METARs</A>     

 <A class=linkWhite 

 href=”http://adds.aviationweather.gov/tafs/”>TAFs</A>     

 <A class=linkWhite 

 href=”http://adds.aviationweather.gov/pireps/”>PIREPs</A>     

 <A class=linkWhite 

 href=”http://adds.aviationweather.gov/airmets/”>SIGMET/AIRMET</A>     

 <A class=linkWhite 

 href=”http://adds.aviationweather.gov/satellite/”>Satellite</A>     

 <A class=linkWhite 

 href=”http://adds.aviationweather.gov/radar/”>Radar</A><BR><FONT 

 style=”COLOR: #ffffff”>0</FONT><BR>Page loaded:   <A 

 class=linkWhite href=”http://www.time.gov/”>22:20 UTC</A><BR>03:20 

 PM Pacific  |   04:20 PM 

 Mountain  |   05:20 PM 

 Central  |   06:20 PM Eastern 

 </TD></TR></TBODY></TABLE>

 <TABLE cellSpacing=0 cellPadding=0 width=”100%” align=center 

 summary=”This table is used for formatting purposes only” border=0>

 <TBODY>

 <TR vAlign=center align=middle>

 <TD bgColor=#ffffff>

 <TABLE cellSpacing=0 cellPadding=0 width=”96%” align=center 

 summary=”This table is used for formatting purposes only” 

 border=0><TBODY>

 <TR vAlign=center>

 <TD bgColor=#ffffff colSpan=4>

 <HR>

 </TD></TR>

 <TR vAlign=top>

 <TD class=gray align=left bgColor=#ffffff><A 

 href=”http://www.noaa.gov/”><SPAN class=gray>NOAA’s</SPAN></A> 

 <A href=”http://www.nws.noaa.gov/”><SPAN class=gray>National 

 Weather Service</SPAN></A><BR><A 

 href=”http://www.ncep.noaa.gov/”><SPAN class=gray>National 

 Centers for Environmental Prediction</SPAN></A><BR><A 

 href=”http://adds.aviationweather.gov/”><SPAN 

 class=gray>Aviation Weather Center</SPAN></A><BR><SPAN 

 class=gray>7220 NW 101st Terr., Room 118</SPAN><BR><SPAN 

 class=gray>Kansas City, MO 64153-2371</SPAN><BR><A 

 href=”mailto:ncep.awc.adds@noaa.gov”><SPAN class=gray>Contact 

 ADDS Internet Services</SPAN></A><BR><SPAN class=gray>Page 

 last modified: February 07, 2007 </SPAN></TD>

 <TD align=left bgColor=#ffffff><A 

 href=”http://www.weather.gov/disclaimer.php”><SPAN 

 class=gray>Disclaimer</SPAN></A> <BR><BR><A 

 href=”http://www.weather.gov/credits.php”><SPAN 

 class=gray>Credits</SPAN></A> <BR><BR><A 

 href=”http://www.weather.gov/glossary”><SPAN 

 class=gray>Glossary</SPAN></A> </TD>

 <TD align=right bgColor=#ffffff><A 

 href=”http://www.weather.gov/privacy.php”><SPAN 

 class=gray>Privacy Policy</SPAN></A> <BR><BR><A 

 href=”http://www.weather.gov/admin.php”><SPAN class=gray>About 

 Us</SPAN></A> <BR><BR><A 

 href=”http://www.weather.gov/careers.php”><SPAN 

 class=gray>Career Opportunities</SPAN></A> 

 </TD></TR></TBODY></TABLE></TD></TR></TBODY></TABLE></TD></TR></TBODY></TABLE></BODY></HTML>

C1 

APPENDIX C

Code-behind for a sample page emulation of aviationweather.gov. In Visual Studio, dynamic Web pages use codebehind to respond to events such as mouseovers and button clicks.

Imports System

Public Class _Default

 Inherits System.Web.UI.Page

 Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Load

 pnl_msgbox.Visible = False

 btn_Overwrite.Visible = False ‘Comment this out to enable the Overwrite

 End Sub

 Protected Sub btn_Begin_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles btn_Begin.Click

 ‘Below, we check to make sure a valid data file name has been typed in

 If TxtBox_EnterID.Text = “” Then

 Label1.Text = “ENTER AN ID#”

 Panel1.BackColor = Drawing.Color.Red ‘NOTE: Not all colors will work

 Else

 Panel1.BackColor = Drawing.Color.White

 myGlobals.currentURL = “ADDS pages/ADDS Homepage.aspx”

 myGlobals.dataFilename = “Pilot data files/Wx06 “ & TxtBox_EnterID.Text & “.txt” ‘Name the path/filename

 If My.Computer.FileSystem.FileExists(myGlobals.dataFilename) Then

 ‘ We have to avoid erasing the datafile if a S wants to initiate a second session

 pnl_msgbox.Visible = True

 Else

 Do_Stuff()

 End If

 End If

 lbl_BriefingEnded.Visible = False

 End Sub

 Protected Sub lnkbtn_End_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles lnkbtn_End.Click

 Session.Abandon()

 lbl_BriefingEnded.Visible = True

 End Sub

 Private Sub Do_Stuff()

 ‘Call up the ADDS home page in a separate window

 Response.Write(“<script language=” & Chr(34) & “JavaScript” & Chr(34) & “ type=” & Chr(34) & “text/javascript” _

 & Chr(34) & “>window.open(“ & Chr(34) & myGlobals.currentURL & Chr(34) & “,” _

 & Chr(34) & “_blank” & Chr(34) & “)</script>”)

 ‘ Record start of session

 Dim sr2 As StreamWriter, currentTime As Date, temp As String

 currentTime = Date.Now

 temp = currentTime.ToString & ControlChars.Tab & “Session start”

 sr2 = File.AppendText(myGlobals.dataFilename)

 sr2.WriteLine(temp)

 sr2.Close()

 End Sub

 Protected Sub btn_Append_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles btn_Append.Click

 Do_Stuff()

 End Sub

 Protected Sub btn_Overwrite_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles btn_Overwrite.Click

 ‘ Erase the old file

 Dim sr1 As New StreamWriter(myGlobals.dataFilename)

 sr1.Close()

 Do_Stuff()

 End Sub

 Protected Sub btn_Cancel_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles btn_Cancel.Click

 ‘ Do nothing

 End Sub

End Class