DOT/FAA/AM-96/2 

Office of Aviation Medicine 

Washington, DC 20591 

• 

-~- ~. 

Human Factors in Aviation 

Maintenance - Phase Five 

Progress Report . 

Willitam T. Shepherd 

Office of Aviation Medicine· 

· Federal Aviation Administration 

Washington, DC 20591 

Galaxy Scientific Corporation 

Atlanta, GA 30345 

January 1996 

Final Report 

• 

This document is available to the public 

through the National Technical Information 

Service, Springfield, Virginia 22161. 

U.S. Department 

ofTransportation 

Federal Aviation 

Administration 

' ~. 

I 

' . -1 

J 
l 

Technical Report Documentation Page 

1. Report No. 2. Government Accession No. 3. Recipient's Catalog No. 

DOT/FAA/AM-96/2 

4. Title and Subtrtle 5. Report Date 

Human Factors in Aviation Maintenance January 1996 

Phase V Progress Report 

6. Pertorming Organization !Ado 

7. Authorial 8. Performing Organization Report No. 

William T. Shepherd and Galaxy Scientific Corporation 

9. Performing Organization Name and Address 10. Work Unit No. ITRAIS) 

Galaxy Scientific Corporation 

2310 Parklake Drive, Suite 325 

Atlanta, GA 30345 11. Contract or Gn1nt No. 

DTFA01-94C01013 

12. Sponsoring Agency name and Address 13. Type of Report and Period Covered 

Federal Aviation Administration Final 

Office of Aviation Medicine Aprill994 - March 1995 

800 Independence Avenue 

Washington, DC 20591 14. Sponsoring Agency Code 

15. Supplemental Notes 

16. Abrnact 

The fifth phase of research on human factors in aviation maintenance continued to look at the human's role in the 

aviation maintenance system via investigations, demonstrations, and evaluations of the research program outputs. 

This report describes the following areas: (Ch. 2) PENS mobile computing software for FAA inspectors; (Ch. 3) 

STAR computer-based training for aviation regulations; (Ch. 4) HIS digital documentation systems, a hypertext mu1-

timedia software system; (Ch. 5) software/hardware distribution on the Internet; (Ch. 6) human factors program reviewing human performance issues associated with inspection; (Ch. 7) human factors audit program providing a valid 

tool for evaluating human factors in maintenance tasks; (Ch. 8) a study of how the design of workcards affects their 

use and the subsequent potential for error; (Ch. 9) the process of visual inspection and evaluation measuring visual 

inspection performance; (Ch. 1 0) a battery of mechanical aptitude tests, a simu1ated NDI task, and the ability of the 

tests to predict performance; (Ch. 11) the resu1ts of a report on an evaluation of a teamwork training program in a 

FAR 147 school; and (Ch. 12) ARAC ru1e changes and impeoding ru1e changes. 

17. Key Word a 18. Distribution Statement 

Human factors, Aviation maintenance, Computer-based This document is available to the public through the 

job aids, Computer-based instruction, Digital documenta- National Technical Information Service, Springfield, 

tion, Human factors guide, Ergonomics audit, Checklist, Virginia 22161. 

Team training, On-line electronic information, Visual inspection, NDI performance, Training and certification 

19. Security Classif. (of this report) 20. Security Classif, {of this page) 21. No. of Pages 22. Price 

Unclassified Unclassified 290 

Form DDT F 1700.7 CB-721 Reproduction of completed page authonzed 
Acknowledgments 

ACKNOWLEDGMENTS 

This program was sponsored by the Federal Aviation Administration. Technical 

program management was provided by Dr. William T. Shepherd, Program 

Manager, Office of Aviation Medicine. This program was conducted under contract DTFA01~94-Y01013. 

The authors thank Jean Watson, Office of Aviation Medicine, for her assistance and 

support during this program. Thanks also to Sheldon Kohn, Dan Lyle, and Suzanne 

Morgan for editing and compiling this report. 

The authors also recognize the many government and industry personnel who 

continue to participate with the research team. As the work continues, the number of 

contributors (FAA entities, air carriers, and consortiums of industry grou_ps) has 

grown beyond a reasonable size to list all the individuals who have proVIded 

guidance and cooperation. 

lll 
I 

• 

j 

Table of Contents 

TABLE OF CONTENTS 

Cll \PI FR 0'\E- 0\ Un II\\ 

1.0 INTRODUCTION ....................................................................................................................................................... 1 

1.1 JOB AIDING FOR AVIATION SAFETY lNSPECfORS (CHAPTER 2) ••••••••••••••••••••.••••••••••••••••••••••••••••••••••••••••••••••••••••. 2 

__ 1.2 COMPUTER-BASED TRAINING FOR REGULATORY DocuMENTS (CHAPTER 3) ••..••..•...••••....••••...••.•••••••••••........... 2 

/~~ 1.3 DIGITAL DOCUMENTATION SYS1'EMS (CHAPTER4) ............................................................................................. 2 

1.4 ON-RAMP TO lNFORMA TION SUPERHIGHWAY (CHAPTER 5) ................................................................................ 3 

1.5 DEVELOPMENT OF AN AIRLINE HUMAN FACfORS PROGRAM (CHAPTER 6) ••••••••••••••••••••••••••••••••••••••••••••••••••.•••• 3 

1.6 AN AUDIT SYSTEM FOR MAINTENANCE HUMAN FACfORS (CHAPTER 7) ••••.•••••••••.••••••••••••••••••••••••••••••••••••••••••••• 3 

1.7 CHECKLIST RELIABILITY (CHAPTER 8) ................................................................................................................ 3 

1.8 COOPERATIVE WORK WITH AGING AIRCRAFT INSPECTION VALIDATION CENTER (CHAPTER 9) •••••••••••••••••••• 3 

1.91NDIVIDUAL DIFFERENCES IN INSPECTION PERFORMANCE (CHAPTER 10) .••••••••••.•••.•••••••••••••••••••••••••••••••.•••••••• .4 

1.10 STUDY OF TEAMWORK IN MAINTENANCE (CHAPTER 11) •••••••••••••••••••••••••••••••.•••••••••••••••••••••••••••••••••••••••••••••••• 4 

1.11 ADVANCED CERTIFICATION INITIATIVES (CHAPTER 12) ••••••••••••••••••••••.••••••••••••••••••••••••••••••••••••••••••••••••••••••••• 4 

1.12 HUMAN FACfORS WORKSHOP-APPENDICES •••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• 4 

REFERENCES ................................................................................................................................................................. 4 

l'll \I' II· R I \\0- .JOB \IDI'\(,: PI·.RH>R\1 \'\l'E E'\11 \'\( 1·.\11.'\ r S\ STE\1 iPF'\Si 

2.0 INTRODUCTION ................................................................................................................................................. S 

2.1 BACKGROUND .................................................................................................................................................... S 

2.2 SUMMARY OF FIELD STUDY RESULTS ....................................................................................................... 6 

2.2.1 INSPECTOR CHARACTERISTICS .............................................................................................................................. 6 

2.2.2 MATERIALS ........................................................................................................................................................... 6 

2.2.3 REsULTS--COMPUTER PLATFORMS ....................................................................................................................... 7 

2.2.4 ADDffiONAL ISSUES .............................................................................................................................................. 7 

2.3 TRAINING ............................................................................................................................................................. 8 

2.4 VERSION 2 OF THE PERFORMANCE ENHANCEMENT SYSTEM SOFTWARE .................................. 8 

2.4.1 SOFIWARECONVERSION TO VISUAL BASIC .......................................................................................................... 8 

2.4.2 EXPANDED SOFIWARE CAPABH..ITIES .................................................................................................................. 10 

2.4.3 PTRS .................................................................................................................................................................. 13 

2.4.4 DIGITAL REGULATORY GUIDANCE DocUMENTS ................................................................................................. 13 

2.5 ON-GOING COMPUTER EVALUATIONS .................................................................................................... 13 

CHAPTER TWO· APPENDIX 

INTRODUCTION ..................................................................................................................................................... 14 

AIRUNEPARTNER'S NEEDS ......................................................................................................................................... 14 

SOFIW ARE PROTOTYPE ................................................................................................................................................ 14 

EVALUATION ................................................................................................................ : ............................................... 16 
Table of Contents 

CHAPTER TIIRH.- S r\R 

3.0 INTRODUCTION ....... _.. .. _.. .............................................. _.. .............................................................................. 17 

3.1 PHASE V O¥ERVIEW ..................................................................................................................................... 17 

3.2 USER-CENTEREJ) DESIGN ............................................................................................................................ 17 

3.3 DESIGN OVERVIEW ....................................................................................................................................... 18 

3.4 COOPERATION WITH DIGITAL DOCUMENT PROVIDERS ................................................................. 20 

3.5 THE STAR PROTOTYPE ................................................................................................................................ 21 

3.6 USER ACCEPTANCE AND TRAINING EFFECTIVENESS ....................................................................... 22 

3.7 FUTURE RESEARCH PHASES ...................................................................................................................... 23 

3.8 SUMMARY ......................................................................................................................................................... 23 

3.9 REFERENCES ................................................................................................................................................... 24 

Cll \PTER FO!'R- !JI(;ri \L IJO( l \II "'I\ 110"1 SYSH.\IS 

4.0 INTRODUCTION ............................................................................................................................................... 25 

4.1 DIGITAL DOCUMENTATION PROCESS ..................................................................................................... 25 

4.1.1 CONVERT TO DIGITAL FORM ............................................................................................................................... 26 

4.1.2 ADDMARKUPS .................................................................................................................................................... 26 

4.1.2.1 Use HIS Author Mode ................................................................................................................................. 27 

4.1.2.2 Write and Use a Macro ............................................................................................................................... 27 

4./.2.3 Write and Use a Filter Program .................................................................................................................. 27 

4.1.3 INDEX .................................................................................................................................................................. 28 

4.1.4 STRUCTURE TOPICS ............................................................................................................................................. 29 

4.1.5 DISCUSSION ......................................................................................................................................................... 29 

4.2 THE ELECTRONIC HUMAN FACTORS GUIDE FOR AVIATION MAINTENANCE ........................... 30 

4.2.1 DESIGNING THE ELECTRONIC GUIDE ................................................................................................................... 30 

4.2.1.1 Achieving the Accessibility Goal ................................................................................................................. 30 

4.2.1.2 Achieving the Maintenance Goal ................................................................................................................ 30 

4.2.1.3 Achieving the Ease of Use Goal .................................................................................................................. 31 

4.2.2 THE INTERFACE FEATURES ................................................................................................................................. 32 

4.2.2.1 The Introduction .......................................................................................................................................... 32 

4.2.2.2 The Table ofContents .................................................................................................................................. 32 

4.2.2.3 The Information Viewer ............................................................................................................................... 32 

4.2.2.4 Section Buttons ............................................................................................................................................ 33 

4.2.2.5 Text Window ................................................................................................................................................ 33 

4.2.2.6 Media Window ............................................................................................................................................. 34 

4.2.2.7 Electronic Guide Control Buttons ............................................................................................................... 34 

4.2.3 USER FEEDBACK AND INTERFACE MODIFICATIONS ............................................................................................. 35 

4.3 FAAJAAM CD-ROM #3 ..................................................................................................................................... 37 

4.3.1 HYPERMEDIA INFORMATION SYSTEM(HIS) ......................................................................................................... 37 

4.3.2 ELECTRONIC HUMAN FACTORS GUIDE ................................................................................................................ 37 

4.3.3 ERGONOMICS AUDIT PROGRAM-ERNAP ........................................................................................................... 38 

vi 
I 

I 

I 

I 

I 

Table of Contents 

4.3.4 COORDINATING AGENCY FOR SUPPLIER EVALUATION (CASE) ........................................................................... 38 

4.3.5 OFFICEOFAVIATIONMEDICINEVIDEOBROCHURE ............................................................................................ 38 

4.3.6 PENS VIDEOBROCHURE ......................................................................................................................... : .......... 38 

4.4 FUTURE PLANS FOR DIGITAL DOCUMENTATION RESEARCH AND DEVELOPMENT .~ ............. 38 

4.5 REFERENCES .................................................................................................................................................... 39 

l'H \1''1 ER ~ 1\ E ·TilEr\\ I"FOR\1 \TIO'I SK' \\ \ Y 

5.0 INTRODUCTION ............................................................................................................................................... 41 

5.1 USER NEEDS SURVEY ..................................................................................................................................... 42 

5.2 POTENTIAL SKYWAY SERVICES ................................................................................................................ 42 

5.2.1 INTERNET SERVICES ............................................................................................................................................ 43 

5.2.1.1 Electronic Mai/... ......................................................................................................................................... 43 

5.2.1.2 The World Wide Web ................................................................................................................................... 43 

5.2.1.3 FTP .............................................................................................................................................................. 44 

5.2.1.4 Gopher ......................................................................................................................................................... 44 

5.2.1.5 UstServers ................................................................................................................................................... 44 

5.2.1.6 Other Services ............................................................................................................................................. 44 

5.3 THE SKYWAY INTERNET- WWW IMPLEMENTATION ...................................................................... 45 

5.3. I INTERNET SERVICE PROVIDERS ........................................................................................................................... 45 

5.3.2 THE SKYWAY, WWW, HTML, ANDHTML AU1HORING ................................................................................... 45 

5.4 EXISTING AVIATION AND HUMAN FACTORS ON-LINE RESOURCES ............................................. 49 

5.5 SUMMARY AND CONCLUSIONS .................................................................................................................. 50 

5.6 FUTURE PLANS ................................................................................................................................................. 50 

5.7 REFERENCES .............................................................................................................................. ., .................... 51 

CHAPTER FIVE- APPENDIX , 

DRAFT OF USER NEEDS SURVEY ...................................................................................................................... 52 

( II \I' II R Sl\ · Ill \1 \" I \( I OI{S PRO(, R \\1 Ill·.\ 1.1.01'\11· '\I \'\ll 1\II'U.\11· '\I \riO'\ 

6.0 STUDY BACKGROUND ................................................................................................................................... 51 

6.1 HUMAN FACTORS TASK FORCE FORMATION ....................................................................................... 57 

6.1. I TASKFORCE 0BJECfiVES AND GUIDELINES ........................................................................................................ 51 

6.1.2 PROGRAM DEVELOPMENT ......................................................................................................................... , ......... 58 

6.1.3 REDIRECTION OF TilE ERGONOMICS TASK FORCE ............................................................................................... 59 

6.2 COMMUNICATIONS AT THE ATLANTA MAINTENANCE BASE ......................................................... 60 

6.2. I TYPICAL AIRLINE INDUSTRY COMMUNICATION PROBLEMS (ASRS REPORTS) .................................................... 60 

6.2.1.1 Summary of Communication Failures ......................................................................................................... 62 

6.2.2 IDENTIFICATION OF COMMUNICATION PROBLEMS WITHIN THE INSPECTION DEPARTMENT ................................ 62 

6.2.3 RESULTS FROM TilE COMMUNICATION USER NEEDS ANALYSIS .......................................................................... 63 

6.2.4 REsULTS FROM PERSONAL INTERVIEWS WITH INSPECTORS ................................................................................ 64 

6.2.5 REsULTS FROM CONVERSATIONS WITH MANAGEMENT ....................................................................................... 66 

vii 

• 
Table of Contents 

6.3 POSSIBLE SOLUTIONS TO COMMUNICATION PROBLEMS ................................................................ 67 

6.3.1 COMMUNICATION TOOLS .................................................................................................................................... 67 

6.3.1.1 Available Communication Tools ................................................................................................................ 67 

6.3.2 PROPOSED SHIFT TURNOVER LOG ....................................................................................................................... 69 

6.3.2.1 First Draft Genera/Information: Proposed Shift Turnover Log ................................................................ 70 

6.3.2.2 Evaluation of First Draft ............................................................................................................................. 73 

6.3.3 VERSIONTwOOFTHESHIFTCHANGELOG ......................................................................................................... 74 

6.3.3.1 Design of Second Version of Shift Change Log ........................................................................................... 74 

6.3.3.2 Evaluation of Version Two of the Shift Change Log ................................................................................... 77 

6.3.4 OTHER COMMUNICATION SOLUTIONS ................................................................................................................. 78 

6.4 GUIDE TO AIRLINES ON ESTABLISHING HUMAN FACTORS PROGRAM ....................................... 80 

6.5 CONCLUSIONS .................................................................................................................................................. 80 

6.6 REFERENCES .................................................................................................................................................... 81 

CHAPTER SIX· APPENDIX .................................................................................................................................. 82 

Cll.\1'1 ER S~.\ EN -llli\1 \N ~ \(' r<JRS \liD I r l'ROGR UI FOR \L\INTENANC~. 

7.0 PROJECT OBJECTIVE AND CONTEXT ...................................................................................................... 93 

7.1 STRUCTURE OF THE AUDIT ......................................................................................................................... 95 

7.1.1 THEAUDITPROGRAM ......................................................................................................................................... 96 

7.T.2 AUDIT PROGRAM EVALUATION ......................................................................................................................... 102 

7.2 RELIABILITY EV ALUA TION ....................................................................................................................... 103 

7.3 VALIDITY OF ERGONOMIC AUDIT FOR AIRCRAFT MAINTENANCE ............................................ 106 

7.4 FINAL MODIFICATIONS TO THE MAINTENANCE AUDIT ................................................................. 107 

7.5 REFERENCES .................................................................................................................................................. 107 

CHAPTER SEVEN • APPENDIX 

AUDIT CHECKLIST .............................................................................................................................................. 109 

Cll \I'TER EI<;(IT · 1\Jl'RO\ 1:\(, 1111·. RELI \IIII.ITY 0~ \I \IN'I I·. I' \NCE CHECKLISTS 

8.1 INTRODUCTION ............................................................................................................................................. 127 

8 .1.1 CHECKLIST OBJECTIVES .................................................................................................................................... 127 

8.1.2 WORKCARD DESIGN ISSUES .............................................................................................................................. 127 

8.1.3 PuRPOSEOFPROJECT ........................................................................................................................................ 128 

8.2 STUDY OF WORKCARD USAGE ................................................................................................................. 128 

8.2.1 TASK ANALYSIS ..... ,_, ....................................................................................................................................... 128 

8.2.1.1 Mechanics' Attitudes Towards the Workcards ........................................................................................... l29 

8.2.1.2 Content of the Check ................................................................................................................................. 129 

8.2.1.3 Task and Environmental Factors .............................................................................................................. 129 

8.2.1.4 Sequence ofTasks ...................................................................................................................................... J29 

8.2.2 NON-COMPLIANCE IN USING WORKCARDS ....................................................................................................... 130 

8.2.3 RELATIONSHIP BE1WEEN WORKCARD AND CHECKLIST OBJECTIVES ................................................................ 130 

8.3 NATIONAL DATA ON THE EFFECTS OF NOT MEETING WORK CARD GOALS ............................ 131 

viii 

• 
Table of Contents 

8.3.1 APPLICABLEHUMANERRORREsEARCH ........................................................................................................... I32 

8.3.2 POTENTIAL ERRORS REl.ATIIDTOWORKCARDS ................................................................................................ 133 

8.3.2.1 Omissions Related to Interruptions ........................................................................................................... 133 

8.3.2.2 Omissiops Related to Workcard Sequence ................................................................................................ /34 

8.3.2.3 Omissions Related to Workcard Non-Compliance .................................................................................... 134 

8.3.2.4 Rule-Based Errors ..................................................................................................................................... 134 

8.3.3 THE CHALLENGE OF DEVELOPING A JOB AID .................................................................................................... 135 

8.4 mE JOB AID ................................................................................................................................................... 135 

8.4.11l1EDEVELOPMENTOFTIIEJOB AID ................................................................................................................. I35 

8.4.2 DOES THE JOB AID MEET CHEcKLIST OBJECTIVES? ......................................................................................... 140 

8.5 EVALUATION OF THE JOB AID ................................................................................................................. 141 

8.5.1 DIRECT OBSERVATION ...................................................................................................................................... 141 

8.5.2 FIRSTWORKCARDEVALUATION ....................................................................................................................... 141 

8.5.3 SECOND WORKCARD EVALUATION ................................................................................................................... 141 

8.5.4 0VERALLREsULTS ............................................................................................................................................ I42 

8.6 CONCLUSION ·····························································-··················································································142 

8.7 REFERENCES .................................................................................................................................................. 143 

CHAPTER EIGHT ·APPENDIX 

APPENDIX 8-A RESULTS OF INQUIRY ABOUT PRESENT WORK CARDS ............................................. 144 

APPENDIX 8-B RATINGS OF PROBABILITY AND DISCREPENCY .......................................................... 147 

APPENDIX 8-C SEQUENCE OF TASKS FOR LOWER LEVEL - CHECK 2 ON B-737 ............................. 150 

APPENDIX 8-D TASKS OCCURING SEQUENTIALLY •••••••••••.....•.••••••..•••..••••••••••....................••......•..•.•••..... 155 

APPENDIX 8-E FIRST INQUIRY FEEDBACK ON THE PROPOSED JOB AID ......................................... 160 

APPENDIX 8-F INQUIRY FEEDBACK ON REVISED JOB AID •••••.......•..••••••••••••••••••••...••.•••.••.•.....•.••.••....••• 163 

CII\Prl R "\1"\1" -Sl PPORI"O~ !"Ill: I\\/\\"\(\ ISl \1.1"\SPH"IIO"\ RI.S~.\RCII PRO(;I{\\1 

9 .0 OBJECTIVE •...•.••.......•••••••••......•••••••..•..••.••.••.••••.•.•..••••.•.••••••••••••••••..•..........••..•••.•...••..•••.•••••....•..•.......••.......••• 167 

9.1 BACKGROUND AND NEED ••............•.•.••••••••••••••••••.•••••••.•.•...................•••••••••••••.........................••.......•....... 167 

9.2 DEFINITIONS .••••••........••••.••..•••....••.••••••••••••••.•••••••••••••••••••••••••.•...•...•••••••..•.••....•.••••••••••.••••.•...........••.....•••......• 167 

9.3 DESIGN OF THE VIRP EXPERIMENTS ......................................................................................................... 169 

9.3.1 BENCHMARK STUDY ............................................................................................................ ; ........................... 170 

9.3.2 FOLLOW-ON STUDIES ........................................................................................................................................ 174 

9.4 CONCLUSIONS ••••..•.....••••.•••••••....•...••••••••••••••••••••••••••••••••••••••••••••••.•••••••••.....••••.........•.•.............•.•....••••.••......•• 174 

9.5 REFERENCES .••••.•.......•••••••...••....••••••••••••..••..•..••••.•..•••.....•...•••••••••••.•....••.....••..•............................•.••.......•.....• 175 

ix 
Table of Contents 

Cll \PTER IF:\- ;\;0:\DI-XI Rl C II\ F 1:\SPI..C 110;\; I'LIUOR\1 \:>10, 

10.0 INTRODUCTION .......................................................................................................................................... 177 

10.1 METHODOLOGY .......................................................................................................................................... 178 

I 0.1.1 SUBJECTS ........................................................................................................................................................ 178 

10.1.2 APPARATUS ....................... : ............................................................................................................................. I78 

IO.I.2.I Inspection Window ................................................................................................................................. I78 

IO.I.2.2 Macro-View and Directionals ................................................................................................................ I79 

IO.I.2.3 Eddy-Current Meter ................................................................................................................................ I79 

IO.I.2.4 Lower Right Window .............................................................................................................................. I79 

I 0.1.3 CRACK AND METER CHARACTERISTICS ........................................................................................................... 179 

10.1.4 PREDICTORS AND/OR TASKCORRELATES ........................................................................................................ I80 

IO.I.4.I Subjective Rating Scale (SRS) ................................................................................................................. I80 

IO.I.4.2 Bennett Mechanical Comprehension Test ............................................................................................... I80 

IO.I.4.3 Typical Experiences Inventory ................................................................................................................ I80 

JO.I.4.4 Arithmetic and Digit Span Tests of the Wechsler Adult Intelligence Scale (WAIS) ................................. I80 

IO.I.4.5 Eysenck Personality Inventory (EP1) ....................................................................................................... I8I 

IO.I.4.6 Matching Familiar Figures Test (MFFT) ............................................................................................... I8I 

IO.I.4.7 Jackson Personality Research Form (PRF) ........................................................................................... I8I 

IO.I.4.8 Figure Preference Test ............................................................................................................................ I8I 

10.1.5 PROCEDURE ........................................................................................................................................... , ....... 182 

10.2 RESULTS AND DISCUSSION ...................................................................................................................... 182 

10.2.1 TASK PERFORMANCE ...................................................................................................................................... 182 

I0.2.I.I Peiformance Measures: Reliability, ....................................................................................................... I82 

Intercorrelations, and General Observations ........................................................................................................ I82 

I0.2.I.2 Peiformance Change Across Sessions .................................................................................................... I83 

10.2.2 RATING SCALE VARIABLES ............................................................................................................................. 183 

I0.2.2.I Pre- to Post-Task Changes ...................................................................................................................... I83 

10.2.3 PREDICTOR VARIABLES AND PERFORMANCE .................................................................................................. 184 

I 0.2.4 GENDER, LIKING FOR INSPECTION, AND SELF ESTIMATES OFT ASK PERFORMANCE ........................................ 185 

10.3 SUMMARY AND CONCLUSIONS .............................................................................................................. 186 

10.4 REFERENCES ......................................................................................... : ..................................................... 186 

CIL\PTER ELE\ EN- TK\\IS \:\ll'l ~- \\1\\ ORK 

11.0 INTRODUCTION ........................................................................................................................................... 189 

11.1 BACKGROUND AND LITERATURE REVIEW ........................................................................................ 189 

I 1.1.1 INTRODUCTION ................................................................................................................................................ I89 

11.1.2 LITERATURE ON TEAMS .................................................................................................................................. 190 

11.1.3 TEAM AND TEAMWORK DEFINED ...................................................................................................................... 191 

11.1.4 TEAM EVOLUTION ........................................................................................................................................... 191 

11.2 FRAMEWORK FOR TEAMWORK IN THE AIRCRAFT MAINTENANCE ENVIRONMENT ........ 192 

11.3 TEAM TRAINING .......................................................................................................................................... 194 

11.3.1 LECTURE ......................................................................................................................................................... I94 

11.3.2 TEAM MEETINGS ............................................................................................................................................. I94 

11.3.3 RoLE-PLAYING ................................................................................................................................................ I94 

11.3.4 TASK DEMONSTRATION ................................................................................................................................... I94 

X 
Table of Contents 

11.3.5 FEEDFORWARDTRAINING ............................................................................................................................... 195 

11.3.6 TEAM DECISION-MAKING ........•...........•......•••••...••••....................••••....•.•••..•..................................................... 195 

11.3.7 FEEDBACK TRAINING ...................................................................................................................................... 195 

11.4 TEAM TRMNING STUDY ........................................................................................................................... 195 

11.4.1 SUBJECTS •••.•.......•••••••••••••••••••.••.•••••••••••••••••••••...•••••••••.•••••••••••••••••••••••••••..••••••••••••••••••••••••............••••••••.••••.•••• 196 

11.4.2 TASK ............................................................................................................................................................... 196 

11.4.3 PROCEDURE····················································································'······························································· 196 

11.5 MEASURING TEAMWORK SKILLS, TEAM ATTITUDE, AND TASK PERFORMANCE ............... 196 

11.5.1 TEAMwORKSKIU.S ..........................•••.•.............•.....•...................................................................................... 196 

11.5.2 TEAM ATIITUDE .••....•.••.........•••••........•.••...•••••....•••...................••••....•..••..•...................................................... 197 

11.5 .3 TASK PERFORMANCE ....................•......•••.................................•...................................................................... 197 

11.6 RESULTS AND DlSCUSSION ...................................................................................................................... 197 

11.7 CONCLUSIONS .............................................................................................................................................. 198 

11.8 FUTURE APPLICATIONS OF TEAM TRAINING WITIIIN A & P SCHOOL CURRICULUM ........ 198 

11.8.1 COMPUTER-BASED TOOL FOR TEAM TRAINING .............................................................................................. 199 

11.8.2 FUNcnoNALDESCRIPTION ....••........•••••••............••........................•.....•.•.......................................................... 199 

1 1.8.2.1 The Trainee's Module .............................................................................................................................. /99 

11.8.2.2 The Instructors Module ........................................................................................................................... 200 

11.9 REFERENCES ................................................................................................................................................ 200 

CHAPTER ELEVEN ·APPENDIX 

TEAM TRAINING .................................................................................................................................................. 205 

Cll \I' II R f\\ 1.1 \I - TR \1"\1"\(, \ "\ll li.Rllll( \ 110"\ 

12.0 INTRODUCTION ................................. - ...................................................................................................... 229 

12.1 BACKGROUND .............................................................................................................................................. 229 

12.2 AVIATION INDUSTRY DYNAMICS AND REGULATORY CHANGE ................................................. 230 

12.3 THE AVIATION RULE MAKING ADVISORY COMMITTEE (ARAC) PROCESS ............................ 231 

12.4 ARAC - 65 ACTION REGARDING ADVANCED OR SPECIAL CERTIFlCA TION ............................ 231 

12.5 SUGGESTED SKILL AREAS FOR ADVANCED OR SPECIAL CERTIFICATION (ARS-1) ......... ; .... 232 

12.5.1 AIRCRAFT ELECTRONICS ......•••..............•.......•.......................•••....................•.................................................. 233 

12.5.2 COMPOSI1E STRUCfURAL REPAIR ................................................................................................................... 233 

12.5.3 NoN-DESTRUCTIVE INSPEcnON ··············-~·-···································································································· 233 

12.5.4 METAL STRUCTURES REPAIR ....................................................•.•................................................................... 234 

12.5 .5 BALLOON AND GUDER REP AIR .............•.........•......................•........................................................................ 235 

12.5.6 OriiER.l'OTENTIAL SKILL AREAs ..................................................................................................................... 235 

12.6 INFORMAL FAR 145 REPAIR STATION TECHNICIAN SURVEY ...................................................... 235 

12.7 ESTABLlSHED TRAINING AND CERTIFlCATION STANDARDS ...................................................... 236 

12.8 THE CANADIAN AIRCRAFT MAINTENANCE SPECIALIST CERTIFICATION SYSTEM ............ 240 

12.9 JOINT AVIATION REGULATIONS (JAR) 65 REVISION STATUS ...................................................... 241 

xi 
Table of Contents 

12.10 ORGANIZATIONS THAT ARE POTENTIAL CERTIFICATION STANDARD DEVELOPERS AND 

"KEEPERS OF THE FLAME" ............................................................................................................................. 242 

12.11 OTHER REGULATORY IMPROVEMENT ELEMENTS TO CONSIDER .......................................... 243 

12.12 CONCLUSIONS ............................................................................................................................................ 243 

12.13 REFERENCES .............................................................................................................................................. 243 

PHASE V RH'OR r- APPENDIX 

MODIFIED PROCEEDINGS FROM MEETING 9 ................................................................................................. 245 

AGENDA .................................................................................................................................................................. 246 

KEYNOTE ADDRESS, DR. JON L. JORDAN ....................................................................................................... 251 

HUMAN FACTORS IN MAINTENANCE AND ENGINEERING, EDWARD ROGAN ...................................... 255 

OVERVIEW OF FAA NO! RESEARCH AT SANDIA LABS, PATRICK WALTER ............................................ 261 

LIST OF ATTENDEES ............................................................................................................................................ 269 

"'' 

"'' 

xii 
list ofTabks 

LIST OF TABLES 

TABLE 2.1 CHAAACI'ERisncs oF TilE FouR CoMPUTERS UsED IN FIELD STUDY .......................................................... 7 

TABLE 2.2 COMMON FEA TURF.') OF TilE FoUR COMPUTI!RS ...........••...................................•••........••.............................. 7 

TABLE 3.1 SOURCES OF INFORMATION FOR NEEDS AssESSMENT .......................................•..........•••••......................... 17 

TABLE 3.2 SUMMARY LI!ARNING ISSUES AND WHERE CBT COULD SUPPORT INSTRUCTION .......••............................. 18 

TABLE 3.3 REsEARCH QUESTIONS .........•.•...................•......................•.............................•.....•.••••..•..•......................... 18 

TABLE 3.4 MEDIA INFORMATION TYPES ............................•..••.••............................................•....••............................... 19 

TABLE 3.5 TASKS FOR i'HAsES VI AND VII .............•........••.•.•..••............................................•.....•.............................. 23 

TABLE 5.1 COST ESTIMATES FOR EsTABUSHING ON-SITE SKYWAY .......................................••••.•............................... 45 

TABLE 5.2 FAA SUPPORTED Pulluc ACCESS ON-LINE BBSs ................................................•••..•............................... 49 

TABLE 5.3 AVIATION/HUMAN FACTORS INTERNET-BASED SERVICES ...................................•..................................... 50 

TABLE 5.4 AVIATION/HUMAN FACTORS CD-ROM DATABASES ....................... , ......................................................... 50 

TABLE 6.1 COMMUNICATION FAILURES ........................•...••..•......•..............................................•.•••............................ 62 

TABLE 6.2 SUMMARY OF USER NEEDS ANALYSIS RESULTS .....................................................•.................................. 64 

TABLE 6.3 COMMUNICATION TOOLS MATRIX ........................................................................•.••.................................. 69 

TABLE 6.4 EvALUATION OF PROPOSED SHIFT TuRNOVER LoG .................................................................................... 73 

TABLE 6.5 EVALUATION OF PROPOSED SHIFT TuRNOVER LoG .................................................................................... 78 

TABLE 6.6 COMPARISON OF FIRST DRAFT AND SECOND DRAFT ......................................................•........................... 79 

TABLE 7.1 GENERIC TASK DESCRIPI10N OF INSPECTION ..••.......................................................•.•............................... 94 

TABLE 7.2 GENERIC FuNCTIONS IN AIRCRAFT REPAIR ................................................................................................ 94 

TABLE 7.3 CLASSIFICATIONS OF MODULES INEAAM ....................................................................•............................ 96 

TABLE 7.4 RELIABIUTYDATA ON MAINTENANCE AUDIT FOR FOUR TASKS ..................................••..•....................... 103 

TABLE 7.5 RESULTS OF Q TEST ON MAINTENANCE AUDIT RESULTS ......................................................................... I 03 

TABLE 7.6 CLASSIFICATION OF DIFFERENCES BY ERROR TYPE ..................................................•..••.......................... I 04 

TABLE 7.7 REuABILITYDATA ON MAINTENANCE AUDIT VERSION 2.0 ········'···························································· 105 

TABLE 7.8 RESULTS OF Q TEST ON MAINTENANCE AUDIT VERSION 2.0 ................................................................... I 05 

TABLE 7.9 ISSUES IDENTIFIED BY CHECKUST ............................................................................................................ I 06 

TABLE 7.10 ERGONOMIC ISSUES IDENTIFIED BY EXPERTS AND CHECKLIST .......... ' .................................................... 107 

TABLE 9.1 CLASSIFICATION OF INDICATION & DEFECT TYPE .................................................................................... 168 

TABLE 9.2 CLASSIFICATION OF STRUCTURE FOR FUSELAGE INSPECTION ............................•...................................... 169 

TABLE 10.1 MEAN VALUES FOR TilE PERFORMANCE VARIABLES ............................................................................. 183 

TABLE 10.2 MEAN PRE- AND PosT-SESSION RATINGS .••.....••..............................................•...........•......................... 183 

TABLE 10.3 LoADINGS OF EACH PREDICTOR VARIABLE ON TilE THREE FACTORS ..............•..................................... 184 

TABLE 10.4 DISUKING OF INSPECTION BASED ON GENDER .....•................................................................................. I 85 

TABLE 11.1 TEAM SKill.S ..............................................................................................................•........................... 209 

TABLE 11.2 TASK DI!COMPOSmON ...................•...••.................................................................................................. 210 

TABLE 11.3 DEMOGRAPHICS QUESTIONNAIRE .•..•.•••••.............................................................••...•............................. 211 

TABLE 11.4 TOOL DESCRIPI10N ....••.......................................................................................................................... 212 

TABLE 11.5 TEAM EXERCISE ON LoST ON TilE MOON .•..........................................•.................................................. 214 

TABLE 11.6 TEAM TRAINING PROGRAM .............•.••..•..•••.........................................................•................................. 217 

TABLE 11.7 INSTRUCTORS' QUESTIONNAIRE TEAM PERFORMANCE MEASUREMENTS ............................................ 219 

TABLE 11.8 POST SESSION QUESTIONNAIRE .............•.....•....•.•................................................................................... 220 

xiii 
List ofT abies 

TABLE 11.9 PRE-TRAINING QUESTIONNAIRE ............................................................................................................. 221 

TABLE 11.10 POST TRAINING QUESTIONNIARE .......................................................................................................... 222 

TABLE 11.11 DATA COLLECfiON INSTRUMENT ON TEAM PERFORMANCE ................................................................. 223 

TABLE 11.12 SUMMARY OF TASK PERFORMANCE ..................................................................................................... 224 

TABLE 11.13 AMP SCHOOL CURRICULUM················································································································ 225 

TABLE 11.14 TEAMPROJECTS ................................................................................................................................... 226 

IC 

''" 

"'' 

xiv 
r 

' 

List of Figures 

LIST OF FIGURES 

FIGURE 1.1 U.S .• AIRLINEACCIDENTRATEPER 100,000 DEPARTURES (1957-1993) ....................•............................... ! 

FIGURE 2.1 PERFORMANCE ENHANCEMENT SYSTEM VERSION] .................................................................................. 9 

FIGURE 2.2 PERFORMANCE ENHANCEMENT SYSTEM VERSION 2 .................................................................................. 9 

FIGURE 2.3 WORK PROGRAM MANAGEMENT .............................................................................................................. II 

FIGURE 2.4 BUBBLE HELP EXAMPLE ........................................................................................................................... l2 

FIGURE 2.5 MICRO HELP EXAMPLE ............................................................................................................................. 12 

FIGURE 2A.1 EXAMPLE CASE FORM .......................................................................................................................... 15 

FIGURE 2A.2 EXAMPLE COMMENT .............................................................................................................................. 15 

FIGURE 2A.3 EXAMPLE OF ON-LINE STANDARD .......................................................................................................... 16 

FIGURE 3.1 LEARNING ENVIRONMENTS IDENTIFIED FOR STAR .................................................................................. 20 

FIGURE 3.2 COLORED ITEMS CAN BE SELECTED FOR DETAILED EXPLANATIONS ....................................................... 22 

FIGURE 4.1 HIS DIGITAL DocUMENTATION PROCESS: FOUR BASIC STEPS ................................................................ 25 

FIGURE 4.2 EXAMPLES OF COMMON MARKUPS (GML SYNTAX) ................................................................................ 27 

FIGURE 4.3 ADDING A HOTWORD IN HIS AUTHORING MODE ...................................................................................... 28 

FIGURE 4.4 HIS TABLE OF CONTENTS: UNSTRUCTURED VS. STRUCTURED TOPICS ................................................... 29 

FIGURE 4.5 THE CYCUC DESIGN MODEL .................................................................................................................... 31 

FIGURE 4.6 THE ELECTRONIC GUIDE TABLE OF CONTENTS ........................................................................................ 32 

FIGURE 4.7 THE ELECTRONIC GUIDE INFORMATION VIEWER ...................................................................................... 33 

FIGURE 4.8 ELECTRONIC GUIDE "Go To" DIALOG BOX .............................................................................................. 35 

FIGURE 4.9 THE SEARCH DIALOG Box ........................................................................................................................ 36 

FIGURE4.10 FAA AAM CD-ROM #3 MAINMENU ................................................................................................... 37 

FIGURE 5.1 AAM USEOFTHEINTERNET(ADAPTEDFROM NEJMEH, 1994) ................................................................ 41 

FIGURE 5.2 EXAMPLE OF AN INTERNET GOPHER ........................•................................................................................ 44 

FIGURE 5.3 CURRENT SKYWAY IMPLEMENTATION ...................................................................................................... 46 

FIGURE5.4 SKYWAYWWW PAGE AS VIEWED FROM MOSAIC VIEWER ................................................................... 47 

FIGURE 5.5 HUMAN FACTORS GUIDE WWW PAGE ..................................•................................................................. 48 

FIGURE 5.6 GRAPHIC FROM CHAPTER I OF TilE HUMAN FACTORS GUIDE .................................................................. .48 

FIGURE 6.1 INSPECTION SHIFT TuRNOVER LOG (FIRST DRAFT) .................................................................................. 71 

FIGURE 6.2 INSPECTIN LOG: BLUE BAY (FIRST DRAFT) ....................•......................................................................... 72 

FIGURE 6.3 LEAD INSPECTOR SHIFT TuRNOVER LOG (SECOND DRAFT) ..................................................................... 76 

FIGURE 6.4 INSPECTOR SHIFT TuRNOVER LOG (SECOND DRAFT) ............................................................................... 77 

FIGURE7.1 TITLESCREEN .................................................................•......................................................................... 97 

FIGURE 7.2 HEADING !NFORMA TION SCREEN .............................................................................................................. 98 

FIGURE 7.3 MAIN PROGRAM SCREEN .................................................•...•............................................•....................... 98 

FIGURE 7.4 MAINTENANCE PREPARATION SCREEN ..................................................................................................... 99 

FIGURE 7.5 HELP SCREEN ........................................................................................................................................... 99 

FIGURE 7.6 COMMENT SCREEN ................................................................................................................................ I 00 

FIGURE 7.7 EAAM SCREEN ..................................................................................................................................... 100 

FIGURE 8.1 INSPECTION SHIFT TuRNOVER LOG (FIRST DRAFT) ................................................................................ 136 

FIGURE 8.21NSPECTION LOG: BLUE BAY (FIRST DRAFT) ......................................................................................... 138 

FIGURE 8.3 LEAD INSPECTOR SHIFT TuRNOVER LOG (SECOND DRAFT) .................................................................... 139 

FIGURE 8.4 INSPECTOR SHIFT TuRNOVER LOG (SECOND DRAFT) ............................................................................. 140 

FIGURE 9.1 SUBJECT INSPECTING B-737 STRUCTURE ............................................................................................... 172 

XV 
List of Figures 

FIGURE 9.21NSPECfOR INSPECTING TEST PANELS FOR CRACKS ............................................................................... 173 

FIGURE 10.1 NO! TASK SIMULATION (DRURY ET AL., 1992) .................................................................................... 179 

FIGURE 11.1 A MODIAED TEAM EFFECTIVENESS MODEL ....................................................................................... 205 

FIGURE 11.2 EvALUATION 0 FI'EAM PERFORMANCE MEASURES BY INSTRUCTOR· ENGINE REMOVAL .................... 206 

FIGURE 11.3 EVALUATION OF TEAM PERFORMANCE MEASURES BY INSTRUCTOR- ENGINE INSTALLATION ............. 206 

FIGURE 11.5 SELF EVALUATION - ENGINE INSTALLATION ......................................................................................... 2fY7 

FIGURE 11.4SELFEVALUATION- ENGINEREMOVAL ................................................................................................ 2fY7 

FIGURE 11.6 PRE AND POST TRAINING DATA .................................................................................................•.......... 208 

APPENDIX 

FIGURE A.1 B737 TESTBED AIRCRAFT ...................................................................................................................... 264 

FIGURE A.2 FAA HANGAR ................................................................................................................................................ 265 

FIGURE A.3 EDDY CURRENT INSPECTION EXPERIMENT .................................................................................................... 266 

FIGURE A.4 SCANNER ASSESMENT .................................................................................................................................... 267 

FIGURE A.S HU25A GUARDIAN AIRCRAFT ....................................................................................................................... 268 

t< 

XVI 
A&P 

AAM 

AANC 

ACS 

AFS 

AME 

AMOs 

AMT-T 

AMTs 

ANSI 

AOL 

ARAC 

ARC 

ARS 

ASA 

AS Is 

ASNT 

ASRS 

ATA 

ATEC 

ATP 

AWS 

BBS 

CACRC 

CAMC 

CASE 

CBT 

CD-ROM 

CFR 

CMAQ 

CPS 

CRM 

CSD 

DOS 

DOT 

EAAM 

EMT 

EPA 

EPI 

ERNAP 

FAA 

FARs 

FBO 

FCC 

FSAS 

FSDO 

LIST OF AD BREVIA TIONS 

Ail:frame and Powerplant 

Office of Aviation Medicine 

Aging Aircraft NDI Development and Demonstration Center 

Aviation Compliance Services 

Aviation Flight Standards Service 

Aircraft Maintenance Engineers 

Approved Maintenance Organization 

Aviation Maintenance Technician - Transport 

Aviation Maintenance Technicians 

American National Standards Institute 

America On Line 

Aviation Rulemaking Advisory Committee 

American Red Cross 

Aviation Repair Specialists 

American Sailing Association 

Aviation Safety Inspectors 

American Society for Non-destructive Testing 

Aviation Safety Reporting System 

Air Transport Association 

Aviation Technician Education Council 

Aircraft Technical Publishers 

American Welding Society 

Bulletin Board System 

Commercial Aircraft Composite Repair Committee 

Canadian Aviation Maintenance Council 

Coordinating Agency for Supplier Evaluation 

Computer-Based Training 

Compact Disc, Read Only Memory 

Code of Federal Regulations 

Cockpit Management Attitudes Questionnaire 

Comprehensive Professional Services 

Crew Resource Management 

Constant Speed Drive 

Disk Operating System 

Department of Transportation 

Ergonomic Audit for Aircraft Maintenance 

Emergency Medical Technicians 

Environmental Protection Agency 

Eysenck Personality Inventory 

Ergonomics Audit Program 

Federal Aviation Administration 

Federal Aviation Regulations 

Fixed Base Operators 

Federal Communications Commission 

Flight Standards Automation System 

Flight Standards District Office 

xvii 

Ust of Abbreviations 
List of Abbreviations 

FfP 

GML 

GSC 

HCI 

ms 

HfML 

lAM 

ICAO 

lFR 

ISO 

ISP 

ISSA 

JAA 

KB 

LAN 

MAINAUD 

MB 

MESH 

MFFf 

MRM 

MSG 

MUDs 

NAFfA 

NASA 

NDI 

NDT 

NRW/C 

NRR 

NSI 

NTSB 

OCR 

OJis 

PADI 

PAMA 

PC 

PENS 

POD 

PRF 

PTRS 

RAM 

RB 

SAE 

SB 

SFARS 

SGML 

SHEL 

SLIP 

SRC 

STAR 

File Transfer Protocol 

Galaxy Markup Language 

Galaxy Scientific CotpOration 

Human-Computer Interaction 

Human Factors Information Systems ? or Hypermedia 

Hypertext Markup Language 

International Association of Machinists 

International Civil Aviation Organization 

Instrument Flight Rules 

International Standards Organization 

Internet Service Provider 

International Sailing School Association 

Joint Aviation Authority 

Knowledge-Based 

Local Area Network 

Audit Program for Maintenance 

MegaBytes 

Managing Engineering Safety Health system 

Matching Familiar Figures 

Maintenance Resource Management 

Maintenance Steering Group 

Multiple User Domains 

North American Free Trade Agreement 

National Aeronautic and Space Athninistration 

Non-Destructive Inspection 

Non-Destructive Testing 

Non-Routine Work Card 

Non-Routine Repair 

National Standards Institute 

National Transportation Safety Board 

Optical Character Recognition 

On-The-Job Injuries 

Professional Association of Diving Instruction 

Professional Aviation Maintenance Association 

Personal Computer 

Performance Enhancement System 

Probability of Detection 

(Jackson) Personality Research Form 

Program Tracking and Reporting Subsystem 

Random Access Memory 

Rule-Based 

Society of Automotive Engineering 

Skill-Based 

Supplemental Federal Aviation Regulations 

Standard General Markup Language 

Software, Hardware, Environment, Liveware 

Serial Line Internet Protocol 

Structure Repairs Committee 

System for Training Aviation Regulations 

xviii 
SUNY 

TC 

TCP/IP 

TEAM 

URL 

VIRP 

WAD 

WAIS 

www 

State University of New York 

Transport Canada 

Transmission Control Protocol/Internet Protocol 

Team Evolution And Maturation 

Universal Reference Locator 

Visual Inspection Research Program 

World Aviation Directory 

Wechsler Adult Intelligence Scale 

World-Wide Web 

xix 

List of Abbreviations 
CJL\PTER l 

PHASEV 

OVERVIEW 

William Jo/mson, VICe President 

Galaxy Scientific Corporation - lnlonnation Division 

1.0 1'\TROIH C riO'\ 

Aviation safety is most commonly measured by accident rate vs. I 00,00 departures. Trends, 

depicted in Figure 1.1, show that aviation safety 

benefits from continuous improvement, meaning that 

this earth's safest transportation is becoming even 

safer. Hardware is the primary reason that aviation 

safety is improving. Modern power plants and 

aircraft systems have increasing reliability. Aircraft, 

air traffic control, and airport navigation, landing, 

., 

and communications digital systems have also 

contributed to the safety factor. Some suggest that the 

extent to which hardware can increase safety has 

reachcul an assemtote; it is not likely to make much 

more improvement However, attention to the human 

as operator and maintainer of the aviation safety 

system, has the highest potential for additional safety 

enhancement. In fact, human error is the #1 cause of 

aviation incidents and accidents (NTSB}. 

!!! 

::J2 

t 

g_ 

U.S. Airline Accidents Rate 

(1957-1993) 

~ 

§ 

8' 

..... 

()) 

Q.l 

-E 

()) 

"0 

~ J2 '57 'tR '61 '63 '65 '61 '(/1 '71 '73 75 '77 ?9 '11 - '16 '87 '10 "91 '93 

~ 

Year 

Figure 1.1 U S A~rlrne Accrdent Rate per 100 000 Deoarlu res I 1 957 1 993 I 

NTSB US A1rCarners Operat1ng Unclc·1~ CFR 121 Arl Sciledulecl Servrcc IA~rlrnesl 1994 
a 

" 

Phase V Overview 

Since 1989 !he FAA Office of Aviation Medicine has 

conducted research related to human factors in 

aviation maintenance. The research program is !he 

world's largest such study of human performance in 

maintenance. Invt>lving universities, government 

laboratories and private industry, !he research 

addresses many aspects of human performance in 

.,maintenance. The research ranges from basic 

scientific experimentation to applied studies in airline 

work environments. The applied studies represent !he 

largest part of !he program. 

The human factors in aviation maintenance research 

program uses airline and industry maintenance 

facilities as !he primary laboratories. FAA inspectors 

working on airline air worlhiness have also helped to 

define, develop, and evaluate products of the human 

factors research. 

In !he six years of !he research, the Office of 

Aviation Medicine has conducted and published 

proceedings of nine workshops on Human Factors in 

Maintenance and Inspection. The research team has 

published over 200 technical papers. Three CDR OMs have been published and distributed to over 

3,000 recipients. 

This report documents the primary research and 

development efforts conducted in !he fif!h year of !he 

research program. As in previous years, !he report 

represents a broad spectrum of human performance 

research and development, each shall be described 

briefly in !he remainder of this introductory chapter. 

1.1 Job Aiding for Aviation Safety Inspectors 

(Chapter 2) 

The ferformance Enhancement System (PENS) is an 

ongoing research and development effort to empower 

FAA Aviation Safety Inspectors (ASis) with mobile 

computing software and hardware. The chapter 

describes two mobile computing applications, one for 

government (PENS) and !he other for industry 

(CASE). 

PENS provides AS Is wi!h a mobile computer to 

collect and analyze data in the field. The system, 

described in the chapter, also permits ASis electronic 

access to critical data like !he Federal Aviation 

Regulations and !he FAA Inspectors Handbooks. The 

chapter also describes an extensive field test of PENS 

and ongoing evaluations of emerging mobile 

computing hardware and software technology. 

2 

Chapter I 

The airlines share a system to audit providers of 

goods and services. The system is named 

Coordinating Agency for Supplier Evaluation 

(CASE). The CASE system is comprised of paper 

forms and a hard copy instruction guide book to 

complete !he forms. The CASE mobile computing 

software has integrated all information into a 

complete digital system. The chapter 2 appendix 

describes !he CASE software. 

1.2 Computer-based Training for Regulatory 

Documents (Chapter 3) 

The System for Iraining Aviation Regulations 

(STAR) combines multimedia training software and 

!he FAA Human Factors Information System (HIS) 

to provide a mix of training and digital 

documentation. The training system is being 

designed to present cases, or scenarios, to learn about 

!he Federal Aviation Regulations and o!her 

regulatory documents for maintenance. The chapter 

describes how STAR instructional design and 

training system analysis were conducted. 

Descriptions of STAR functionality are also 

included. 

1.3 Digital Documentation Systems (Chapter 4) 

The research program has a rich history applied to 

digital documentation systems. The Human Factors 

Information System (HIS) is a hypertext multimedia 

software system that was developed for FAA COROMs 1-3. This special purpose system was 

designed to meet specific FAA hypertext 

requirements and to minimize costs associated with 

mass production and distribution of certain FAA 

databases. This chapter describes !he design and 

evolution of HIS. It also shows interface examples of 

how HIS is applied to the CO-ROMs and to the 

digital Human Factors Guide. 

1.4 On-Ramp to Information Snperhighway 

(ChapterS) 

The Office of Aviation Medicine has distributed 

research results via !hree CD-ROMs, as previously 

described. This media has worked well as !he number 

of installed CD-ROM computers has increased in 

government and throughout the aviation industry. 

The research related to the "FAA Information 

Skyway" is developing the hardware/software 

infrastructure to, eventually, distribute research 

results via the Internet. 
l 

j 

l I 

Chapter I 

The chapter describes a user assessment of the online information needs of the aviation maintenance 

community. The chapter describes the kinds of 

services that are needed and likely to be provided by 

an "Information' Skyway." The initial World-Wide 

Web has been established and is operational. The 

chapter describes the services/reports that are 

currently available. It also describes future directions. 

1.5 Development of an Airline Human Factors 

Program (Chapter 6) 

This project was done in cooperation with Northwest 

Airlines, at the DC-9 base in Atlanta. The goal was to 

establish a human factors task force to review a 

variety of human performance issues associated with 

the inspection department. 

The chapter describes how the task force was formed 

and the composition of worker and management 

participants. Also described are a variety of 

opportunities for improvement in decision making 

and communication in the maintenance process. 

1.6 An Audit System for Maintenance Human 

Factors (Chapter 7) 

The purpose of this task was to provide a valid, 

reliable, and usable tool for evaluating human factors 

in maintenance tasks. A software tool was designed 

and developed as a product of this research. As 

reported in the chapter the majority of the work went 

towards the ergonomics audit information with the 

software development task being secondary. The 

chapter includes hard copies of most of the forms 

contained in the software program. The final version 

of the ergonomics software package shall be included 

with the digital publication of the Human Factors 

Guide. 

1.7 Checklist Reliability (Chapter 8) 

Maintenance workcards are the technician's 

equivalent of the pilot's checklist. The workcard is 

meant to ensure that maintenance is performed in the 

correct order and that no step is omitted. The chapter 

reports on a study of how the design of workcards 

affects their use and the subsequent potential for 

error. 

The chapter describes a task analysis of workcard 

usage conducted in an airline maintenance 

environment. The research analyzed maintenance 

data from the Aviation Safety Reporting System to 

determine if workcard usage or non-usage 

3 

Phase V Overview 

contributed to safety infractions. Also reviewed is 

application literature on human error with respect to 

checklists. The chapter ends with a description of the 

creation and evaluation of a workcard for shift 

turnover. 

1.8 Cooperative Work with Aging Aircraft 

Inspection Validation Center (Chapter 9) 

The Office of Aviation Medicine has engaged in 

cooperative research with the FAA Technical Center 

via the Aging Aircraft Inspection Validation Center 

(AANC). The research supports the Visual Inspection 

Research Program at Sandia National Laboratories in 

Albuquerque, NM. The chapter describes the process 

of visual inspection and describes an evaluation 

measuring visual inspection performance. 

1.9 Individual Differences in Inspection 

Performance (Chapter 10) 

Numerous research studies have shown a wide range 

of individual performance differences among 

inspection personnel. This basic scientific study 

measures relationships between NDI task 

performance and psychometric measures of 

mechanical ability and attention-concentration. The 

chapter describes a battery of mechanical aptitude 

tests, a simulated NDI task, and the ability of the tests 

to predict performance. The exciting answer to these 

predictive questions can be found in the chapter! 

1.10 Study of Teamwork in Maintenance 

(Chapter 11) 

Most maintenance activities are conducted by teams 

of aviation maintenance technicians (AMTs). 

Therefore, teanl planning, coordination, and 

communication are critical to safe and efficient 

completion of all maintenance tasks. This chapter 

reports on a study of teamwork in maintenance and 

outlines a training program focusing on teamwork. 

The chapter reports the results of an evaluation of a 

teamwork training program conducted in a FAR 147 

school. The chapter ends with a technical 

specification for a computer-based training system 

for team training. 

1.11 Advanced Certification Initiatives 

(Chapter 12) 

FAR 65 addresses the certification of aviation 

personnel other than flight crew members. Over the 

past few years the FAA, in cooperation with an 

Aviation Rulemaking Advisory Committee (ARAC), 

has been revising Part 65 to address competencies 
Phnse V Overview 

and requirements for Aviation Maintenance 

Technicians. This chapter reports on the ARAC 

activities and impending rule changes. This chapter 

also considers methods to create an "advanced 

certification" system 'that could be administered by 

private industry instead ofF AA. 

1.12 Human Factors Workshop-Appendices 

The Office of Aviation Medicine has conducted nine 

workshops on Human Factors in Maintenance and 

Inspection. The proceedings from eight of these 

workshops are published in hard copy and on the 

FAA CD-ROMs. The ninth conference was held in 

November, 1994, and focused on review of the 

Human Factors Guide for Avr'ation Maintenance. 

Few speakers at the ninth meeting spoke on topics 

other than specific chapters of the Guide. Therefore, 

a dedicated 9th Meeting Proceedings shall not be 

published. 

The appendices of this report contain papers from the 

9th meeting that are not directly related to the Human 

Factors Guide. The first speaker was Dr. Jon L. 

Jordan, Federal Air Surgeon. Dr. Jordan's paper 

reviewed the five year progress of the research 

4 

Chapter I 

program. He highlights major program products and 

looks to the future of the research program. 

Dr. Patrick Walter is the Director of the Aging 

Aircraft Inspection Validation Center at Sandia 

National Laboratory. His paper describes the research 

program at Sandia. The appendix also contains a 

paper from Mr. Eddie Rogan, Human Factors 

Engineer- British Airways. Mr. Rogan describes the 

human factors research at British Airways with 

specific reference to the Managing Engineering 

Safety Health (MESH) system. MESH is a method 

for reporting, analyzing, and mitigating human error 

in maintenance. 

Also included in the appendices is a list of attendees 

who participated in the Agenda 9th Workshop. 

NTSB, Broad Cause/Factor Assignments, 14 CFR 

121 Operations, 1992 
I 

l 

CH.\PTER 2 

JOB AIDING: 

PERFORMANCE ENHANCEMENT SYSTEM 

Charles Layton, Ph.D. 

Galaxy Scientific Corporation 

2.0 1'\TROill C riO'\ 

One of the tasks in the Human Factors in Aviation 

Maintenance and Inspection Research Program 

involves investigating advanced technologies and 

how these technologies might he applied to aviation 

maintenance tasks. We have been investigating pen 

computing technology and have developed a 

prototype application, called the Performance 

Enhancement System (PENS), for the FAA Flight 

Standards Service. We have also been working on a 

transition of our experiences from this project to 

industry. The bulk of this chapter describes the Flight 

Standards work, while Chapter 2 - Appendix 

addresses the work we have done with an industry 

partner. 

We had several milestones with PENS in the last 

year. The first field study was completed in April 

1994, and the results of that study were published last 

fall. Fall1994 also saw the initiation of FAA training 

of Aviation Safety Inspectors on PENS concepts. 

Version 2 of the system software was completed in 

preparation for a second field study in Winter 

199411995. Finally, a number of computers have 

been evaluated in-house, and several units have been 

selected for in the study to evaluate. 

2.1 B \( h.<;ROl '\I> 

The Performance Enhancement System represents 

a series of investigation and implementation 

phases supporting the goal of matching the needs and 

responsibilities of Flight Standards Service (AFS) 

Aviation Safety Inspectors (ASis) with automation 

capabilities. This project is a direct result of the AFS 

Training and Automation Committee's Information 

Systems Strategy, which recommended that all future 

automation systems he developed in conjunction with 

the work force so that systems are designed to meet 

workers' needs and desires. The Training and 

Automation Committee has been instrumental in 

supporting PENS and in providing project oversight. 

5 

Field data collection is one characteristic of ASI 

activities. The data are collected on paper forms, and 

data entry clerks transcribe these forms into computer 

databases. These data are then recorded in a national 

database and are used to monitor the aviation 

industry's safety. Another characteristic of field 

inspectors' activities is that they must authoritatively 

answer questions as they arise. This requires ASis to 

carry voluminous, cumbersome field copies of 

regulations and guidance. 

Four primary concerns provided the impetus for 

development of PENS. First, data entry clerks are a 

significant annual expense for AFS. If it were easy 

for inspectors to enter data into the computer 

databases themselves, AFS would save the money it 

now spends on data entry. Second, there is a 

significant time delay of up to two weeks in form 

transcription. By decreasing that time delay, AFS 

could he more effective at monitoring and ensuring 

compliance in the aviation industry. Third, many data 

transcription errors occur in the current process, so 

many that the Government Accounting Office has 

repeatedly criticized the FAA for the poor quality of 

its data. Fourth, paper regulations and guidance 

materials are not flsed effectively because they are 

bulky and difficult to maintain. The combination of 

all these factors points toward automation as a 

potential solution. Field automation, at a minimum, 

would allow AS Is: I) to store data directly in the 

proper database format; 2) to verify the validity of 

data at the time of an inspection; 3) to eliminate the 

time delay associated with transcription; and 4) to use 

on-line guidance materials quickly, easily, and with 

minimal maintenance of the documents. Other 

benefits would accrue as more tools were added to 

field computers. 

The project began as an investigation, sponsored by 

the Office of Aviation Medicine (AAM), into the 

utility of pen computers for aviation industry 

inspectors and maintenance technicians. This phase 
Job Aiding: PENS 

of the project continued from approximately January 

until August 1992. During this time, FAA 

Administrator Thomas Richards learned about pen 

computers and thought that they might be a good tool 

for Aviation Safety Inspectors. To this end, he 

requested briefings from the Right Standards 

Service. The Right Standards Service learned of the 

AAM research and requested information in August 

1992. After a series of briefings to FAA personnel, 

including Clyde Jones, AFS Director Thomas 

Accardi, and Associate Administrator for Regulation 

and Certification Anthony Broderick, we briefed 

Administrator Richards in November 1992, and 

Acting Administrator Joseph Del Balzo in January 

19~. . 

Between January and August 1993, PENS received a 

lot of publicity within Right Standards Services, both 

in AFS Headquarters and in the field. The project 

continued with a low level of funding from the Office 

of Aviation Medicine. From August 1992 through 

August 1993, a series of task analyses and prototypes 

were carried out to determine the basic content of a 

field computer tool. The Fort Lauderdale Right 

Standards District Office (FSDO) was fundamental to 

the success of these initial analyses and prototypes. 

Funding for a national field human factors study of 

PENS concepts was provided in August and October 

of 1993. Because of all of the publicity the project 

had received over the previous year, AFS 

Headquarters felt considerable pressure to start the 

field study quickly once funding was available. After 

some very rapid prototyping and testing with Atlanta 

FSDO inspectors, the national field study began on 

November 15, 1993, continuing until March I, 1994. 

2.2 Sl \1\L\RY OF FIELD STl'll\ 

RESl LTS 

The following is a summary of Performance 

Enhancement System concepts that were 

evaluated, the nature of the field study, the important 

results, and considerations for full implementation. 

The full results and discussion can be found in The 

Petformance Enhancement System Field Evaluation 

Report. 

2.2.1 Inspector Characteristics 

Four airworthiness (maintenance) aviation safety 

inspectors at each of nine sites, a total of 36 

inspectors, participated in the study. The inspectors 

averaged 49 years in age, had been inspectors for five 

6 

Chapter2 

and a half years (most airworthiness inspectors are 

former aircraft mechanics), and had five and a half 

years of computer experience. Sixty-five percent of 

the inspectors use the current data entry system, and 

sixty percent own computers. 

Note that inspectors' computer experience correlates 

with their experience as ASis. The current computer 

systems installed at the field evaluation sites run a 

very limited set of DOS applications, not Microsoft 

Windows applications. PENS runs in Microsoft 

Windows for Pen Computing. 

Training was given according to time, rather than to 

criterion. Inspectors were trained for two days. The 

first day consisted of an explanation of file storage 

conventions, DOS, Windows, and handwriting 

recognition, including training the computer to 

recognize the inspectors' handwriting. The second 

day consisted of training on PENS software. 

We spent much more time covering basics in 

Windows than we thought would be necessary. Even 

though each office had Windows installed on its 

workstations, inspectors were generally 

inexperienced Windows users. The most likely 

explanation for their inexperience was that few 

inspectors had any need to run Windows software. 

The extra Windows training did not significantly 

affect the amount of training devoted to PENS; there 

was time left at the end of the second training day. 

2.2.2 Materials 

Three different models of pen computers and one 

standard notebook computer were fielded at each 

office. Thus, 36 co~puters were put into the field. 

Computers were selected based on their particular 

combination of features and their differentiating 

characteristics. That is, the computers were selected 

because they had certain features in common, but 

each also had a particular feature that made it unique. 

These computers allowed inspectors to evaluate the 

tradeoffs between weight, versatility, and speed. The 

computers' features are summarized in Table 2.1 

(next page). The features listed in Table 2.2 (next 

page) are common to all four computers. 

2.2.3 Results--Computer Platforms 

The inspectors were asked to rate a number of 

usability characteristics of each computer. The 

characteristics included weight, ease of use, screen 

characteristics, environments in which the computer 

was used, and the like. With regard to particular 
1 

I 

I 

I 

Chapter 2 Job Aiding: PENS 

Table 2.1 Charactenst1cs of the Four Computers Used 1n F1eld Study 

Q!l.iQ !:;Qov~rtibl~ NEC Y!:!:s~ill! TelePOO SL IQsbiba Sal~lli~ I12Q!! 

486/25 MHz CPU 486/25 MHz CPU 386/25 MHz CPU 486/25 MHz CPU 

200 Mb Hard Drive 80Mb Hard Drive 200Mb Hard Drive 120 Mb Hard Drive 

Built-in K~yboard Separate Keyboard Separate Keyboard Built-in Keyboard 

Pen St Jus Pen St Ius Pen St Ius Trackball 

Table 2.2 Common Features of the Four 

Computers 

8Mb RAM 

Backlit LCD Monochrome display 

PCMCIA Data Storage Card 

DOS6.0 

Windows 

Microsoft Word 2.0 (except the NEC VersaPad) 

PENS Proto! Software 

characteristics of pen computers, the only significant 

result was that the GRiD Convertible was judged 

more comfortable than the NEC VersaPad. This 

result is consistent with inspectors' comments that its 

case made the VersaPad difficult and cumbersome; 

the Convertible was much more compact and easy to 

use. 

When ratings for pen computers are compared with 

the notebook computer (Toshiba Satellite Tl900), 

both the GRiD Convertible and the TelePad SL were 

judged to be faster. Inspectors generally disliked the 

VersaPOO, and that may have biased the inspectors' 

evaluations. We originally thought that the VersaPad 

was a good computer to use to examine trOOeoffs 

between computer characteristics because it had a 

smaller hard disk and was also much lighter. 

Finally, inspectors &!dressed the tradeoff between 

weight and capability. Many inspectors complained 

that the VersaPad did not have enough hard disk 

capacity because it was too small to contain on-line 

versions of both the FARs and the Airworthiness 

Inspectors' Handbook. 

Perhaps the most telling data on the computers were 

collected in response to the question, "Would you use 

this computer in the field as part of your job?" 

Inspectors generally preferred the GRiD Convertible 

and the TelePOO SLover the NEC VersaPad and the 

Toshiba Satellite. However, none of these computers 

are currently in production: the GRiD Convertible 

and the NEC VersaPad have been removed from the 

market; the TelePad SL is due to be replaced this Fall 

with the TelePad 3; and the Toshiba Satellite Tl900 

has been replaced with another model. 

7 

Because the notebook computer was comparatively 

heavy and cumbersome, it was extremely difficult for 

inspectors to use it while they performed an 

inspection. While they could easily operate a pen 

computer with two hands, the notebook computer 

really needed to lie on a flat surface. Inspectors 

indicated that they definitely would not be able to use 

a standard notebook computer as part of their daily 

routine, although a pen computer was feasible. 

Inspectors were unanimous in requesting smaller, 

lighter computers. They were particularly interested 

in devices that would fit in their coat pockets such as 

personal digital assistants, e.g., Apple Newton, 

Tandy/Casio Zoomer, etc. However, such devices 

currently do not have either the storage or the 

processing resources to run applications necessary for 

AS!s. Inspectors were also intrigued by the 

possibility of using speech recognition for data 

collection, as this would keep their hands free. 

2.2.4 Additional Issues 

Interviews with inspectors revealed that, although 

immediate recording of field data may not always be 

required, immediate access to previous data or 

regulatory materials is required. For inspectors, a 

computer is more useful as an information 

management and retrieval tool than as a data 

collection vehicle for inspection activities. 

Inspectors raised a number of additional concerns 

during the study. Many inspectors were concerned 

about liability for the equipment should it be stolen, 

dropped, or left on an airplane. Some inspectors were 

concerned with perceptions of people they were 
Job Aiding: PENS 

inspecting, i.e., they were worried that they appeared 

inept or incompetent when using a computer. Other 

inspectors were concerned that a computer lent an air: 

of permanence to notes they made, and, as a result, 

operators would be less cooperative, even though 

notes on paper have the same degree of permanence. 

While there are practical solutions to all these issues, 

the issues themselves go well beyond the questions of 

which computer is better or if a field computer can be 

used for one-time data capture. 

With regard to environmental considerations, 

inspectors noted that the computers stopped working 

when the temperature approached freezing. Cold 

temperatures also make it more difficult to use a 

computer because of the inspector's need to wear 

gloves, bulky coats, etc. Finally, as one might expect, 

inspectors were reluctant to use computers in snow or 

rain for fear of damaging the machines. 

2.3 TH \1:\1'1(; 

The Regulatory Standards and Compliance 

Division, AMA-200, has begun training new 

ASls on the concepts embodied in the Performance 

Enhancement System. Although the system is not 

ready for full implementation, inspectors should be 

initiated into future system capabilities as they 

receive their first training. In this way, inspectors will 

see the system as a tool in their compliance arsenal 

and as an integral part of their jobs. 

Version 2 of the software was only recently 

completed, so the training group has provided only a 

brief system introduction during the training courses. 

However, the training group has indicated that they 

will gladly incorporate more training as soon as the 

system is ready for full implementation. 

2.-1 \ EHSIO"i 2 OF THic 

I'EHFOH\1 \\l'E E\H.\:\CE\IE\T 

S\ S"l E\1 SOl· I\\ \RE 

Version 2 of the Performance Enhancement System software has been completed and is 

ready for the next field study. This software 

8 

Chapter 2 

incorporates changes and improvements over the last 

version in four major areas: 

I. the code was converted from CIC++ to 

Microsoft Visual Basic to allow significant 

improvements in the software's design and 

maintainability 

2. the software has greatly expanded its 

functionality to address all three AS! 

specialties: Operations, Airworthiness, and 

Avionics 

3. the Program Tracking and Reporting 

Subsystem (PTRS) data collected have been 

subjected to the same validation procedures 

used on data entered through the Flight 

Standards Automation System (FSAS) 

4. the three leading FAA digital regulatory 

guidance document systems will be 

compared in the field study. 

The following sections address each of these areas. 

2-4.1 Software Conversion to Visual Basic 

One of the biggest changes in Version 2 is that it has 

been converted from CIC++ to Visual Basic, which is 

rapidly becoming the standard development 

environment for Microsoft Windows software. This 

switch has improved the "look and feel'' of the . 

software, has made development easier, has increased 

maintainability, has improved our ability to add 

functionality, and has improved database capabilities. 

The enhancements in Version 2 improve usability 

and user acceptance. As shown in Figures 2.1 and 2.2 

(next page), the scroll bar has been removed from the 

PTRS form and has been replaced with tabs. This 

change makes navigation between sections of the 

form easier and more direct. Forms generally have 

more visual depth, appearing three dimensional. This 

new appearance facilitates functional grouping and 

makes buttons distinct from fields. Version 2 gives 

users the impression that it is a professional product, 

rather than a research and development tool. 
9 
" 

Job Aiding: PENS 

Because many development tasks are handled by 

Visual Basic, rather than by a programmer, software 

development has become much easier. Since the 

programmer does not have to worry about low level 

Windows routines necessary to make buttons work, 

he or she can focus on greater design issues of layout, 

error prevention, database support, and the like. 

Furthermore, Visual Basic improves Version 2's 

maintainability because it is now much easier to 

follow the software's flow of control and structure. 

Since Visual Basic uses the Basic programming 

language (which is frequently the first computer 

language one learns) the odds that the FAA will be 

able to maintain the software are greatly improvedespecially when Visual Basic is compared with an 

esoteric language like C or C++. 

Visual Basic supports myriad control features 

allowing one to add features supporting specific 

requirements of an application. These controls are 

called VBXs, and many are supplied by Microsoft 

with Visual Basic. Thousands more are available 

from third parties. Had the project been continued in 

CIC++, these types of controls would have been 

developed in-house, requiring significant time and 

effort. In Version I of PENS, virtually any desired 

control outside the very limited set supported by the 

CIC++ compiler would have to have been developed 

from scratch. 

Finally, Visual Basic includes database support for a 

variety of databases, including Microsoft Access and 

Paradox 3.5. This support allows us easily to migrate 

the software to support future databases as AFS 

systems evolve. The current AFS standard database 

format is Paradox 3.5, but it appears that in the near 

future Microsoft Access and SQL formats will be 

10 

Chapter2 

used. Visual Basic has built-in support for each of 

these formats. 

2.4.2 Expanded Software Capabilities 

Version I of PENS consisted of three primary 

modules: the data collection and on-line policy 

module; the data transfer module; and the 

supervisory review module. Each module and its 

improved version is discussed in turn. 

The data collection and on-line policy module 

consisted of the PTRS form for data collection, the 

Federal Aviation Regulations (FARs), and the 

Airworthiness Inspector's Handbook (FAA Order 

8300.10). Version 2 of this module has been split into 

its constituent parts. The data collection portion has 

been expanded to include the ten forms most 

commonly used in the field (not in the office), 

including the PTRS form. These ten forms address 

the operations and avionics specialties, in addition to 

airworthiness. 

New data management capabilities have been 

designed into Version 2. Work has been divided into 

three general categories: work yet to be begun resides 

in the "In Box"; work started, but incomplete, resides 

in "Work in Progress"; the "Out Box" contains 

completed activities before they are transferred to the 

office databases. A fourth data repository, the 

"Archive," maintains a backup set of all data that 

have ever resided on the portable computer. With this 

structure, inspectors quickly determine what 

activities are currently open, what activities are 

completed, and what activities remain to be 

accomplished. This capability is illustrated in Figure 

2.3 (next page). • 
Extensive error prevention mechanisms have been 

built into these fonns. The philosophy of the PENS 

design process is to guide users so that they enter 

correct data, not to correct errors after-the-fact. 

Wherever possible, databases have been incorporated 

to allow the user to select from a set of possible 

entries, rather than to generate his or her own entries. 

Data that can be inferred from previous entries are 

automatically entered into the forms. For example, 

values for the Callup, Start, and Completion Dates 

are constrained by the inspection's status. As shown 

in Figure 2.3, the "Start Date" field is grayed because 

the Status is "P" for planned. Once the Status is "0" 

for open, the "Start Date" field is immediately 

available. Finally, data that are redundant across 

fonns are automatically shared so that an inspector 

need record those data only once. 

The on-line help system has been expanded to 

include Version 2's new functional capabilities. Help 

now addresses how to use the software, rather than 

how to complete a given activity. However, steps to 

complete an activity will be included in Version 3 of 

the software because Job Task Analyses arc to be 

incorporated. Two additional help features have also 

been incorporated in Version 2: Bubble Help and 

Micro Help. Bubble Help is familiar to most 

Microsoft software product users; it is the text 

II 

description appearing when the pointer rests on an 

icon. Bubble Help ensures that toolbar functionality 

is clear. Bubble Help is illustrated in Figure 2.4 (next 

page). Micro Help is a text description of the 

function currently in use appearing at the bottom of 

the screen. For example, when a user clicks on the 

"Make-Model-Series" field in the PTRS form, Micro 

Help indicates that the code may be selected from a 

list. Micro Help is shown in Figure 2.5 (next page). 

The on-line FARs and Handbooks in Version I were 

very difficult Jo maintain and keep current. Because 

some commercial vendors specialize in such 

documents, it was deemed appropriate that inspectors 

compare the most promising of commercial 

alternatives. The in-house versions of these 

documents are not incorporated in Version 2. This 

topic is discussed in more detail below. 

The data transfer module has been divided into two 

separate utilities in Version 2. One of these utilities 

transfers FSAS data to the field computer; the other 

transfers data from the field computer to FSAS. The 

former utility will be used rarely, for example when a 

field computer is initially loaded with the inspector's 

work program. The inspector will use the latter utility 

whenever he or she returns from the field and is 

ready to transfer field data to the office file server. 
Job Aiding: PENS Chapter 2 

12 
Chapter 2 

The supervisory review module has been dropped 

from Version 2 because inspectors rarely used it in 

the first field evaluation. 

2.4.3 PTRS' 

Data Validation, the Regulatory Support Division, 

AFS-600, and the Operational Systems Branch, AFS620, in particular, have been instrumental in allowing 

us to test the PTRS data collection software. The 

Operational Systems Branch initiated a procedure 

that allows us to send PTRS data collected with our 

software through the same upload procedure utilized 

in FSDOs, including data validation. This allows us 

to ensure that all data are consistent with the current 

FSAS data entry system. With Version I, we had 

difficulties with some hidden database fields our 

software did not fill and we were unaware of these 

difficulties until we started field-testing the software. 

Version 2's data validation capability allows us to 

work out such kinks before we get the software into 

the field. 

2.4.4 Digital Regulatory Guidance Documents 

As noted above, one of the critical needs inspectors 

cited in the first field study is an ability to research 

policy and regulatory guidance while they are in the 

field. Version I of the software supported a prototype 

of this capability. At the time, it was necessary for us 

to develop this prototype in-house because the 

products were not available commercially. However, 

three commercial providers now have released 

extensive Windows-based systems: Aviation 

Compliance Services (ACS) released the FAR 

Library; Aircraft Technical Publishers (A TP) 

released the United States National Aviation 

Regulatory Library; and Summit Aviation released 

the Computerized Aviation Publications Library. 

Each system contains the Federal Aviation 

Regulations, some Advisory Circulars, some FAA 

Orders, and additional publications. Each package is 

unique, and each publisher releases updates on its 

own schedule. 

The ACS and Summit systems have a simple 

document viewer with simple searching techniques. 

The A TP system is a powerful research tool, 

containing significant cross referencing of documents 

and aircraft information. There are significant cost 

differences among the products. Our current plan is 

to compare all three products in a small field study 

and then to let inspectors determine which product 

best meets their needs. ACS and A TP have agreed to 

13 

Job Aiding: PENS 

supply their product at cost; negotiations with 

Summit are underway. 

2.5 ON-GOJM; COl\II'UTER 

EVALll \TIONS 

We are continuing to evaluate portable computers 

to stay abreast of the latest developments in 

portable computing technology. Portable computers 

are becoming smaller and lighter, with more 

processing power, and a longer battery life. New 

developments in pen computer technology have 

allowed manufacturers to reduce their size and 

weight while simultaneously increasing their 

capabilities and battery life. These units have 

improved so much recently that they deserve a fresh 

look from inspectors, particularly from airworthiness 

inspectors. 

Subnotebook computers offer a compromise between 

the capabilities of full notebook computers and their 

weight. Subnotebooks typically have somewhat 

smaller hard disk drives of around 120MB (although 

this is increasing) and use external floppy drives; 

they are much smaller than notebook computers and 

weigh approximately half as much. A subnotebook 

computer will fit in a large overcoat pocket, which 

approaches inspectors' requests for a unit that would 

fit in a pocket. 

While subnotebook computers may fit a majority of 

inspectors' needs, inspectors may also wish to do 

research on policy guidance in the field. In the last 

year several notebook computers with internal CD 

ROM drives have been introduced. These CD ROM 

notebooks ha,ve full multimedia capabilities, as well. 

These machines come in two configurations. One 

design has a CD ROM drive underneath its keyboard; 

the other uses a separate CD ROM docking station 

attached beneath a standard notebook computer. The 

first design has CD ROM available always; its 

drawback is that the user must always carry 

additional weight. The second design has the merit of 

allowing an inspector to leave the CD ROM drive 

(and its weight) behind when it is not needed; its 

drawback is that an inspector has to keep track of a 

second piece of equipment. 

We envision providing samples of these computers to 

inspectors at the Atlanta FSDO prior to the actual 

field study. These inspectors will give us a first pass 

evaluation of the options; in turn, we can determine 

which computers offer the most promise for the field 

study. 
Chapter 2 - Appendix 

Job Aiding: Transition of Performance Enhancement System 

Concepts to Industry 

I:\ I'RODl Cl 10\ 

The Performance Enhancement System's success 

has brought the aviation industry's attention to 

the possibilities of supporting mobile maintenance 

technicians and auditors with portable computing 

technology. This is somewhat ironic, given that we 

started the research with these applications in mind 

but were unable to interest industry. During the last 

year, we have been working with a partner airline to 

transition PENS job aiding concepts to industry 

personnel. The following is a brief description of that 

work. 

Airline Partner's Needs 

Our partner airline has two groups of maintenance 

auditors within the Technical Standards office: 

Compliance Auditors and Vendor Surveillance 

Analysts. Both groups use a variety of forms to 

document the results of their audits. Both groups also 

have standards which they apply to the organizations 

that they audit, including Federal regulations (Federal 

Aviation Regulations, Airworthiness Directives, etc.) 

and internal standards. Our partner airline wanted to 

support both groups of auditors. 

The Vendor Surveillance group is responsible for 

auditing companies supplying materials and services 

to the airline to ensure that those companies are in 

compliance with Federal guidelines and with industry 

standards. Our partner airline is a member of the 

Coordinating Agency for Supplier Evaluations 

(CASE). The CASE organization is a consortium of 

airlines that pool their resources and auditing results. 

If a CASE member, e.g., our partner airline, 

evaluates a supplier and certifies that the supplier is 

in compliance with Federal regulations and CASE 

14 

standards, then other CASE members know that they 

can use the supplier without having to perform their 

own audit. CASE provides both auditing forms and 

standards to its members. There are currently six 

CASE forms, although this number changes as new 

forms are added and old forms are retired. 

The Compliance Auditor group is responsible for 

ensuring that our partner airline's maintenance 

operations are in compliance with Federal guidelines 

and with its own standards. The Compliance Auditors 

use approximately 32 forms. 

Software Prototype 

We have developed prototype software to support 

both Compliance Auditors and Vendor Surveillance 

Analysts. Both prototypes were developed for use on 

pen computers because the auditors wanted 

capability similar to the clipboards they currently use. 

The collected data are stored in databases and can be 

printed out in standard report formats or exported to 

Microsoft Word. This is a vast improvement over the 

current method of manual transcription of 

handwritten paper forms. 

We developed an application that contains four of the 

forms Vendor Surveillance Analysts use most 

frequently. Each form is saved separately because a 

vendor normally provides only one supply or service. 

An example is shown in Figure 2a.l (next page). The 

application allows an inspector to identify whether a 

vendor is in compliance and to make a comment for 

each item on the form, as shown in Figure 2a.2 (next 

page). 
r 

Chapter 2 

The_ -(~ +=~rs: c:Lr~ f\of 

\ q___, lli?_J 11.(::._\o_m IV\ CD b f~'~ 

15 

Job Aiding: PENS- Appendix 
Job Aiding: PENS- Appendix 

The application also contains links to the CASE 

standards appropriate to the questions on the auditing 

forms. This allows an auditor quickly to access the 

standards for reference while performing an audit. As 

shown in Figure 2a.l, there is a button next to a 

surveillance item ("Does ROV hold an FAA repair 

station certificate?") that identifies the standard. 

When an auditor pushes the button, the standard 

appears in Windows Help, as shown in the figure. 

Auditors like ·this capability because they can read 

the standard and because they can copy and paste it 

into their reports. Whereas their reports previously 

contained the auditor's recollection of the standard, 

they now contain the standard's exact wording. 

We developed a similar application for the 

Compliance Auditors. Unlike the Vendor 

Surveillance application, forms are saved in 

"sessions"; all forms used in a given audit are saved 

together. This difference in design results from the 

fact that a given maintenance facility of our partner 

airline normally performs several different types of 

Chapter2 

maintenance and requires multiple forms. Because 

the content of the forms is proprietary to our partner 

airline, we cannot publish examples. However, the 

format and content are very similar to the Vendor 

Surveillance forms. Because our partner airline has 

proprietary standards for evaluating their practices, 

its managers have been unwilling to share them with 

us so we could put them on-line. 

Evaluation 

Both prototypes are currently under evaluation at the 

airline. We provided both groups of auditors with a 

number of pen computers and copies of the prototype 

software. Auditors are also using the software on 

their desktop computers. We expect the evaluation to 

run sixty to ninety days. Upon successful completion 

of the evaluation, we plan to work with the airline 

and the CASE organization to determine how these 

concepts can be applied within the broader aviation 

community. 

Regulations of ony individual, equipment, or fuility sheD be kept 

cuzrent and sheD be available forinepection. (145.39, 145.103, 

143.17] 

K. Vendors that deal in non-aircraft puts. materials. or maintenance 

activities shell segregate the aircraft function from other f'unctions 

to preclude getting unapproved parts or mat.erials on an aircraft 

unit. (145 _JjJ 

L. The vendor shatt display his repair station certificate and 

operation specifications at a place in the repair station that is 

normally accessible to the public and is not obscured. 1145.19] 

M. The U.S. domestic repair stations shall have an active, FAA 

approved anti-drug testing plan that complies with FAR 121, 

Appendix I. The plan may be the vendor's plan, a consortium plan 

to which the vendor subscn"bes, or an air carrier CUJtomet's plan. 

The vendor shall provide proof of membership in the plan and 

FAA acceptance of the plan. Letters of acceptance or exemption 

16 
CHAPTER3 

SYSTEM FOR TRAINING OF 

AVIATION REGULATIONS 

Terry Chandler, Ph.D. 

Galaxy Scientific Corporation 

J.O 1'\ IIWill t 110'\ 

The ability to use FAA regulatory documents is a 

requirement for all who are associated with operations, maintenance, and surveillance of aircraft and 

associated air transportation systems and services. 

Schools, airlines, manufacturers, and the government 

require thorough knowledge, as well as reasonable 

appreciation, of the Federal Aviation Regulations 

(F ARs) and the host of associated documents. 

Studying FAA regulatory documents is difficult. Instructors are given the arduous task of conveying the 

meaning of subtle and seemingly ambiguous material 

to a student body who do not always recognize the 

importance of what they are learning. The two most 

difficult aspects of learning the regulations are a) 

learning how to navigate through the FARs and other 

related documents and b) comprehending the meaning 

of particular statements within the FARs. FARs are 

legal documents written precisely to define the regulations pertaining to aviation. Unfortunately, it is not 

easy for most people to extract the intent of each 

statement from this style of writing. In addition, it is 

not always obvious where one needs to look to get a 

complete sense of the regulations' intent. Often, information relevant to a task is distributed across many 

parts of the FARs. For example, knowing one's eligibility to perform an IFR inspection may not be obvious when specifications for how to do the 

inspection are outlined in Part 43, Appendices E and 

F, but the privileges and limitations for who can perform the inspection are stated in 91.4llb and 

91.413c. 

The purpose of the System for Training in Aviation 

Regulations (STAR) project is to aid instructors in 

teaching about the FARs (and other related documents) by providing a system that motivates the student to understand why learning the FARs is both 

relevant and necessary, develops students' study and 

cognitive skills in document research and understanding, and c) makes the content of the FARs more interesting and therefore more memorable. 

17 

Our approach to designing and developing STAR is 

to incorporate multimedia presentations and storytelling techniques within several different types of 

learning environments. The goal is to provide a comprehensive curriculum for acquiring the skills and 

content necessary for efficient document research and 

comprehension. 

J.l I'll \SI· \' 0\fo R\ II·\\ 

The project began in earnest on October 3, 1994. 

In the six months ending April!, 1995, the project team will have conducted a needs analysis, developed a research approach guiding the design of 

STAR, and built the initial prototype. A preliminary 

evaluation of the prototype will be conducted prior to 

April I. A great deal of time has also been spent assessing the best way to integrate digital document 

products with government-owned multimedia training 

systems. A detailed discussion of each of these areas 

is presented below. 

J.2 l Sl R-l'l '\II IU ll 1>1 SH,'\ 

We are employing a user-centered approach to 

technical design (Chandler, 1994; Rasmussen, 

1992; Greenb~um & Kyng, 1991; Norman, 1986). 

Instructors from the FAA Academy in Oklahoma 

City, three Part 147 schools, and one flight training 

academy were interviewed regarding current instructional practices. Table 3.1 shows the sources of information for our needs assessment. 

Table 3.1 Sources of lnformatron for Needs 

Assessment 

• Mike Monroney Aeronautical Center 

• Embry-Riddle Aeronautical University 

• Clayton State College - Aviation Dept. 

• Atlanta Area Technical School 
STAR 

Instructors were asked to identify the major issues 

preventing students from learning aviation regulations 

and to try to envision how a CBT system could address some of these difficult instructional issues. The 

responses to our inq~ires were as varied as the people 

in attendance, but a pattern did emerge. Table 3.2 

summarizes the learning issues instructors identified 

and areas where CBT could support instruction. 

As a result of these interviews, several general research questions emerged to guide the development 

of STAR and its evaluation. Table 3.3 lists theresearch questions. Our answer to the question "How 

do we induce students to think deeply about the subject?" will embody our philosophical approach to instruction. This will become more apparent during the 

discussion below of the design overview. "Which 

learning situations are most effective for what types 

of learning?" is the question that will guide the experiments for evaluating STAR's success as an instructional system. The other three questions identify 

technical issues pertinent to user interface design and 

system functionality that we will need to address 

throughout the project. 

Chapter 3 

We decided to focus our attention on the training of 

Aviation Maintenance Technicians (AMTs) for the 

first two phases of this project and, then to incorporate training for pilots later. We sought the assistance 

of Jack Moore, Dean of Clayton State College -

Aviation Department, as our domain expert for this 

phase of the project. He and other instructors of Part 

147 schools in Atlanta have provided stories, examples, strategies, technical information and documentation to be used as a basis for developing the 

curriculum. We will expand this information base to 

other Part 147 schools around the country during the 

second phase of the project. 

3.3 DI·S((,'\ 0\ Ill\ 11-\\ 

When teaching subtle information such as aviation regulations, there are advantages to providing students with many vantage points to the same 

body of information. Experiencing complex material 

repeatedly under different circumstances provides the 

Ieamer with multiple opportunities to gain a deep understanding of the subject. Each vantage point not 

only covers different aspects of the same material, but 

also reinforces different kinds of study skills. In addition, information conveyed through one learning enTable 3.2 Summary Learnmg Issues and Where CBT.Could Support lnstructron 

Students need help in 

• knowing who the players are (e.g., owner, AMT, pilot, FAA maintenance inspector), what their responsibilities are to each other, and for what regulations each must be responsible 

• understanding the objectives of the FARs and when and how to apply them 

• understanding the codependency of regulations to each other 

• learning to extract the root meaning from the FARs' legalese 

• performing document research procedures 

• recognizing when appropriate (or optimal) procedures. are applicable 

• integrating the individual pieces of their job tasks into a total picture 

CBT could support instruction with 

• a system that supports mu~imedia presentations during class lectures 

• a series of scenarios that elucidate the subtle applications of the regulation 

• drill and practice sessions that show each student where his or her weak points are 

• a mechanism that allows instructors to monitor how the students are doing 

• technical aids that support students while they go through the learning process 

Table 3.3 Research Ouestrons 

• How do we induce the students to think deeply about the subject? 

• Which learning situations are most effective for what kinds of learning? 

• When is it more effective to use what kinds of presentation types to convey the salient points in the 

learning environment? 

• What kinds of information retrieval mechanisms are the most valuable to students? to instructors? 

• How can we translate digitized material meant for a personal computer into a medium suitable for 

distance learning broadcasting? 

18 
r Chapter3 

vironment may be more salient to a learner than another approach. Students with different learning styles 

are more likely to benefit when different vantage 

points are provided. In this way, we provide students 

not only with ~ultiple ways of viewing the information, but also with multiple opportunities to learn. 

The core of the system is a document browser that has 

full text searching capabilities both within and among 

documents. This allows students to search and view 

the documents in their entirety. It also gives students 

practice in manipulating the documents on-line, a 

practice that we anticipate will be the norm in the future. 

Several instructors identified a desire to have multimedia clips punctuate important points they make 

during lectures about the regulations. They see this as 

a means for making their instruction more interesting 

and motivational for the students. Instructors at the 

FAA Academy in Oklahoma are particularly interested in this since they are developing a center for 

distance learning. 

The document browser is designed to support efficient review of media clips to augment class presentations. Associated with each document are all the 

multimedia information clips presented in the other 

learning environments. For example, a video about 

instrument inspection will be indexed with the document section that discusses instrument inspection. 

The browser becomes an archive for the documents 

and all the media clips. Each media clip is further indexed by one of nine information types listed in Table 

3.4. A "Very Important Point" information type, for 

example, may warn students of a regulation that is 

often violated and why or how it gets violated. A "For 

Your Information" information type may point out the 

subtle difference between when an inspection must be 

STAR 

completed every 2 years vs. every 24 months. A "For 

Example" may show a student what a correct log entry looks like. By using the documents themselves as 

indexes, augmented with classifying the media clips 

into information types, we have developed a simple 

system for organizing what is often a very difficult 

body of information to catalog. We see this as a natural way for instructors to review media clips relevant 

to the material they will be covering in class. 

Surrounding the document browser (Figure 3.1, next 

page) are four categories of learning environments: 

overviews, scenarios, brain teasers, and technical 

support. Overviews show students how FARs are organized, how different parts are related to each other, 

and who is responsible for what aspects of those 

regulations.- Scenarios are interactive stories that set 

each student into a true-to-life situation where the 

regulations are often subtle. The scenarios present 

students with choices they need to make within the 

context of a given situation and show the students the 

consequences of those actions. It is important to note 

that there is often more than one right or wrong answer and that understanding why one action is wrong 

in a particular context is just as important as understanding why another action is right. 

Brain teasers present challenges to the student. They 

require students to exercise certain skills they will 

need to develop in order to efficiently search the 

regulations and understand what they find. Brain 

teasers can vary in complexity. They can be ofthe 

"FAR Jeopardy" variety where students can practice 

quick responses to specific facts. Brain teasers can 

also be of the "project" variety where solving a challenge entails a deep understanding of both the search 

process and the regulations themselves. We see this 

area as a space where instructors can develop their 

own challenges for their own students. 

Table 3.4 Medra lnformatron Types 

• General Procedures 

• Strategies for W~hin Document Search 

• Strategies for Between Document Search 

• For Your Information (FYI) 

• Very Important Point (VIP) 

• For Example 

• Personal Experience 

• System Information 

• Terminology 

19 
STAR 

Technical supports are comprehension aids such as a 

technical dictionary. Another example is an interactive time line showing the progression of ownership of 

a particular type certificate by different manufacturers. These aids provide "as needed information" that 

can be explored in their own right or use in conjunction with other, more formal learning environments. 

Each learning environment could be a stand-alone 

application. Together they provide multiple vantage 

points for the student to explore aviation regulations. 

Part of our assessment of the total project will be to 

identify which learning environments are most effective for what types of learning. By focusing on the 

evaluation in this manner, we not only will assess the 

effectiveness of the application, but gain a better understanding of what types of learning is occurring (or 

needs to occur) and how we should tailor our training 

systems to achieve specific learning objectives. 

Our long-term goal is to develop authoring tools for 

the most successful learning environments so that the 

domain expert, i.e., the instructor, can contribute directly to the system rather than remain dependent on 

application engineers for knowledge acquisition and 

implementation. In this way, the system can take on a 

life of its own becoming a repository of pedagogical 

expertise in aviation training. 

20 

3.4 COOI'ERATIO~ \\ ITII J)I(;JT \L 

Don \lE:-.11 I'RO\ mFRS 

Chapter 3 

Digital documentation is a critical component of 

STAR and other document-oriented training 

systems such as The Human Factors Guide (see 

chapter 4) and The Inspector Handbook (see chapter 

4 ), currently under development at Galaxy Scientific -

Atlanta. Over the last four months, the digital documents group has identified what functionality such a 

system must support, who the key commercial publishers are, and the feasibility for a commercial vendor's product to be integrated into a governmentowned multimedia training system. 

The details of this evaluation are presented in chapter 

4. To summarize our findings, it became apparent that 

what is needed are functions that give each system 

designer the power to do full text search of documents and, the flexibility to display the retrieved 

document in a manner consistent with the training 

system's interface. Though the group continues to 

evaluate the commercial market, the FAA Hypermedia Information System (HIS) seems to be best suited 

for providing that flexibility. We have begun the 

process of extracting the functional components from 

HIS so that they can be used by the different training 

systems. 
r 

Chapter 3 

For the ftrst phase of system development, we began building a prototype for the document 

browser and the scenario learning environment. Scenarios lend themselves to capturing the instructional 

information. When a Part 147 instructor tells of a 

typical situation where interpreting the regulations is 

subtle, personal experiences, examples, "By the Way" 

information, warnings, document search strategies, 

and g~neral procedures naturally flow from the telling 

of the scenario. This information is not found in textbooks or the regulations themselves, but is crucial to 

an in-depth understanding of the regulations. The interchange of stories is not only the most common way 

that we exchange information, but is considered the 

optimal form for retention of the information received 

(Bruner, 1990; Shank, 1990). The document browser 

serves primarily to organize the information that is 

being collected. 

Scenarios are essentially interactive stories. Through 

a slide show presentation, students are told of an unclear situation where several actions are possible. 

They are asked a question about what they should do 

given the situation and are presented with several actions that they could tal<e. Following is the textual 

passage presented to the user for the opening scene of 

the special inspections scenario. 

You are a technician with both A and P ratings. During a I 00 br inspection on an IFR equipped C-172, 

you notice that the altimeter and transponder have not 

been tested and inspected in the last 24 months. When 

you inform the owner that these tests and inspections 

are due, he asks: "If these tests and inspections are 

due, why didn't you do them as part of the 100 hour 

inspection?" How do you respond to this question? 

Once a student chooses an answer, a new scene in the 

scenario is presented. The new scene shows the con21 

STAR 

sequences of the action and the rationale for why the 

student should or should not have made that choice. 

Imbedded in each explanation are references to relevant FAR passages and other supporting documents 

and examples. For example, a student might be shown 

a sample of a correct log entry for the type of maintenance work he or she did or a comparison between 

two passages from the FARs where a distinction 

needs to be made. 

Although for each scenario there is the "best" path to 

take, our objective is not to train students to take that 

path. Rather, to get the most out of the scenario, they 

should explore all the paths. By doing so, they acquire a deep understanding of the situation and an appreciation for the subtle distinctions they need to 

make with respect to fully comprehending the intent 

of the regulations. In this sense, there is no right answer, only deeper understanding. How we entice students to explore all of the scenario paths rather than 

just to ftnd the "right" answer is part of the larger research question about inducing students to think 

deeply about the subject. 

While each scene in the scenario has a multimedia 

presentation that "tells the story", students also have 

access to other relevant material that has bearing on 

the situation. In the gray scale background graphic 

used to set the scene seen in Figure 3.2 (next page), 

there are colored items in the picture. When a user 

clicks on one of the colored items, a video or detailed 

graphic or explanation of the item is presented. In our 

instrument flight scenario, for instance, clicking on 

the altimeter will bring up a video that explains the 

functionality of an altimeter in the aircraft. Also, 

along the bottom of the screen are buttons that access 

other related jnformation categorized by information 

type, e.g., FYI, Personal Experience, General Procedures, etc. Students may navigate through the scenario but also can explore tlie details of each scene in 

its own right. 
r 

STAR 

As stated previously, the most important research 

question that we will be addressing in this project is, 

"How do we induce the students to think deeply about 

the subject?" The cognitive and educational literature 

claims that to achieve this goal the student needs to 

be actively involved in the learning task (Brown, 

1992; Scardarnalia & Bereiter, 1992; Resnick, 1991; 

Bransford et. al., 1990; Papert, 1980). They need to 

be asking the hard questions and trying to answer 

them. There is always a risk of losing the students by 

challenging them with something that is beyond their 

technical knowledge, skill level, imagination, or, on 

the opposite end of the scale, boring them to death. 

While scenarios in their present "canned" state do not 

necessarily induce the students to think for themselves, they may serve as a stepping stone to the more 

open-ended challenges presented in the brain teaser 

learning environment. Scenarios do show the students 

the kind of thinking process they need to employ in 

order to make sophisticated decisions about illspecified problems. By mimicking the reasoning pre22 

Chapter 3 

sented in the scenarios, students should be able to 

solve the brain teaser challenges. It will be important, 

when developing the b{ain teaser learning environment in the next phase of research, that some of the 

brain teasers are similar in structure to those in the 

scenarios so that students can practice transferring 

reasoning skills to new situations. 

3.6 l'si·Il. Acn.l' r \'\CF \'\D TR \I'\I'\C 

EI •I·TTI\ I· '\Iss 

The culminating event for this phase of the project 

is to present the STAR prototype at the 34th Annual Conference of ATEC in Aprill995. The conference will provide a wide audience of aviation instructors from across the nation. We will use this forum as 

a vehicle to give us feedback on the STAR concept 

and design, and also an opportunity to tap conference 

attendees expertise. We will set-up several vehicles 

(including a video camera) for capturing their stories 

-
Chapter 3 

and experiences for further development of the system. 

In preparation for the conference, the project team 

will first cond,uct an in-house technical evaluation at 

Galaxy Scientific. That session will focus primarily 

on compatibility issues in the user interface design 

(Maddox & Johnson, 1986). The instructors and a 

select group of students at Clayton State College will 

also have an opportunity to evaluate the STAR prototype. We will ask them to focus on system understandability, content accuracy, information 

presentation and ease of use (Maddox & Johnson, 

1986). Formal evaluations of the system in a classroom setting will begin in Phase VI. 

3.7 Ft I l IU RI SF \RCII I'll \SI·S 

Phase V will draw to a close in April 1995. 

Table 3.5 outlines the tasks for Phases VI and 

VII. System Evaluation will be an important part of 

Phase VI. We will be analyzing what the students 

learn from the system in both a non-directed and a directed setting. First, we will evaluate the robustness 

of the system and how students explore the system 

when it is not tied to a formal class activity. A history 

trace will be kept of each student's activity on the 

system. The second part of the evaluation will be in a 

more formal classroom setting where students will be 

asked to use the system in the context of one or more 

classroom tasks. The focus here will be on what the 

students learn. Pre- and post-testing will be one instrument for this analysis. Another instrument will be 

based on the pedagogical dimensions developed by 

Reeves (1994) for evaluating interactive learning environments. Analysis of students' history trace will 

also be made to see if patterns emerge between 

learning success and application use. These results 

will be the bases for making decisions with regard to 

incorporating intelligent tutoring agents into STAR. 

In preparation for the extensive evaluation of the 

Table 3.5 Tasks for Phases VI and VII 

Phase VI 

STAR 

system, the scenario and document browser will be 

developed into fully functional learning environments. The major task to fulfill this goal is producing 

the curriculum and multimedia materials to build at 

least one complete instructional unit. An example unit 

could be a series of scenarios about AMT's privileges 

and limitations. To show the extent of the instructional possibilities, we will also create several different types of scenarios that are not part of the core 

unit. In tandem with these other efforts, prototypes for 

the "overview", "technical support" and "brain 

teaser" learning environments will be developed and 

initial evaluations of their interface design, robustness, and content accuracy will be conducted during 

Phase VI. 

A comparative study between traditional instruction 

and instruction incorporating STAR as an integral 

part of the curriculum will be made during Phase VII. 

In preparation for this study, the overview, technical 

support, and brain teaser prototypes will be developed into full learning environments. The content of 

the training system will be expanded to training pilots 

and the potential for converting the training systems 

into authoring systems will be assessed. 

3.S St \1\I \R\ 

The STAR project gives us an opportunity to bring 

out the complexity, subtlety, and interesting aspects of what is normally thought to be a dry subject. 

It provides a vehicle for practicing skills in document 

research and complex decision-making. It gives students practice with computerized tasks that they will 

be expected to use with facility in the near future. It 

provides a vehicle for interacting with the subject 

matter from s~veral different vantage points, increasing the chances of each student acquiring an in-depth 

understanding of the material. And, as researchers, it 

gives us the opportunity to evaluate what instructional 

vehicles are best suited to achieve the learning objec-

• Convert the scenario and document browser into fully functioning Learning Environments. 

• System evaluation - non-directed setting. 

• System evaluation - formal classroom setting. 

• Develop prototypes of the overview, technical support and brain teaser learning environments. 

Phase Vll 

• Convert the overview, technical support and brain teaser into fully functional Learning Environments. 

• Conduct comparative study between traditional instruction and instruction incorporating STAR. 

• Expand content of system to include curriculum for Aviation Flight Schools. 

• Assess potential for converting training systems into authoring systems. 

23 
STAR 

tives we have set for our students. This indeed is an 

opportunity. 

.~.9 lhHRI'\(I·S 

Bransford, J.D., Sherwood, R., Hasselbring, T., Kinzer, C.,&. Williams, S. (1990). Anchored instruction: Why we need it and how technology 

can help. In D. Nix & R. Spiro (Eds), Cognition, 

education and multimedia: Explorations in high 

technology. (pp. ll5-l42). Hillsdale, NJ: Erlbaum. 

Brown, A.L. (1992). Design experiments: Theoretical 

and methodological challenges in creating 

complex interventions in classroom settings. 

Journal ofLeaming Sciences, 2(2) 141-178. 

Bruner, J. (1990). Acts of Meaning. Cambridge: Harvard University Press. 

Chandler, T.N., (1994). The Science Education Advisor: Applying a User Centered Design Approach 

to the Development of an Interactive Case-Based 

Advising System. Journal of Artificial Intelligence in Education. 5(3) 283-318. 

Greenbaum, J., & Kyng, M. (!WI). Design at work: 

Cooperative design of Computer Systems. 

Hillsdale, NJ: Lawrence Erlbaum. 

Maddox, M.E., & Johnson, W.B. (1986). Can you see 

it? Can you understand it, does it work? An 

evaluation plan for computer based instruction. 

Proceedings of the International Topical Meeting 

on Advances in Human Factors in Nuclear Power 

24 

Chapter3 

Systems (pp. 380 389). LaGrange, IL: American 

Nuclear Society. 

Norman, D.A., & Draper, S.W. (Eds). (1986) User 

centered system design: New perspectives on 

human-computer interaction. Hillsdale, NJ: 

Lawrence Erlbaurn Associates. 

Paper! S. (1980) Mindstorms: Children, Computers, 

and Powerful Ideas. NY: Basic Books. 

Rasmussen, J., (1992). The Ecology of Work and Interface Design. In A. Monk, D. Diaper and M.D. 

Harrison (Eds), People and Computers VII: Proceedings of the HCI '92 Conference. 

Reeves, T.C., (1994). Multi-dimensional Evaluation 

of Interactive Learning Systems. Paper presented 

at the Association for Educational Communications and Technology. 

Resnick, M., (1991). Beyond the Centralized Mindset. Proceedings of the International Conference 

on the Learning Sciences. 

Scardamalia, M., & Bereiter, C. (1992). An architecture for collaborative knowledge-building. In E 

De Corte, M.Lynn, H. Mandl, & L. Verschaffel 

(Eds.), Computer-based learning environments 

and problem solving (NATO-AS! Series F: 

Computer System Sciences). Berlin: SpringerVerlag. 

Schank, R., (1990). Tell me a story: A new look at 

real and artificial memory. New York, Charles 

Scribner's Sons. 

-
r 

CHAPTER4 

DIGITAL DOCUMENTATION 

SYSTEMS 

Julie Jones, T. Kiki Widjaja, Donia Williams 

Galaxy Scientific Corporation 

4.0 INTRODl!CTIOl\ 

Digital documentation systems are a key 

component of the Human Factors in Aviation 

Maintenance research program. This study of digital 

documentation systems was undertaken in an effort to 

address problems associated with the publication, 

distribution, and use of large quantities of printed 

information in the aviation industry. Digital 

documentation systems have an advantage over paper 

or microfiche documents in terms of compactness of 

information. For example, a bookshelf of manuals 

and reference materials can be stored electronically 

on a single CD-ROM. Other advantages of electronic 

documents include the potential cost savings and 

faster, more effective a~cess to needed information. 

With a paper/microfiche system, a maintenance 

technician could spend considerable time researching 

information for a given maintenance task on an 

aircraft. With a properly developed digital 

documentation system, the time can be substantially 

reduced, perhaps to only a few hours. Air carriers will 

save money from quicker tum-around times on 

maintenance tasks. General Aviation will benefit from 

reduced paper-based research associated with Annual 

Inspections. 

The conversion from printed to electronic 

information, however, is not without costs, and the 

research program is investigating ways of efficiently 

creating, accessing, and maintaining digital 

documentation with a focus on ensuring an interface 

that is compatible with the aviation users. The 

Hypermedia Information System (HIS) has been 

developed to investigate digital documentation 

storage and retrieval issues. Hypermedia is a 

computer-based technology that allows non-linear 

access to information. The information may be in the 

form of text, graphics, audio, video, or animation. For 

more information on the HIS system, see Chapter 6 of 

the Phase IV report (FAAIAAM & GSC, 1994). 

This chapter describes research and development 

activities related to digital documentation completed 

in the past year. Section 4.1 details the process for 

converting documentation from paper to electronic 

form. Section 4.2 describes how the initial prototype 

of the digital Human Factors Guide was designed 

and developed. Section 4.3 describes the contents of 

CD-ROM #3. Finally, Section 4.4 discusses future 

plans for digital documentation research. 

4.1 DIGITAL DOCL:l\IENT.\TIOl\ 

PROCESS 

The process of converting a document into digital 

form requires several steps. Figure 4.1 illustrates 

the basic digital documentation process. This section 

describes basic steps used to process a paper 

document for the HIS: convert it to digital form, add 

markups, index the text, and structure the topics. 

~ ~ ~ 

Convert to Digital Add Markups Index Structure 

Figure 4.1 HIS D1g1tal Documentation Process. Four Bas1c Steps 

25 
Digital Documentation Systems 

4.1.1 Convert to Digital Form 

If no electronic version of the document is available, 

the first step is to convert printed text to digital form. 

For small documents, it may be feasible to type the 

document using a word processor; for larger 

documents, typing may be too labor-intensive. 

Fortunately, commercially available hardware an~ 

software semi-automates this process. A scanner ts 

similar to a photocopier; it is attached to a personal 

computer. Optical Character Recognition ~OCR) 

software converts a scanned image of text mto an 

ASCII text file, i.e., OCR software "recognizes" 

bitmap characters and "types" the corresponding 

ASCII character into a text file. OCR software does 

not preserve formatting such as balding or italic~. For 

more information on the OCR process and a revtew 

of commercial OCR software products, see 

Mantelman, 1994. 

Since neither typing nor OCR conversion is errorflee, a major part of this step is to verify the output 

for accuracy. V eriflcation can also be timeconsuming and tedious, although standard word 

processing tools like spell checkers can assist. Some 

other techniques have been developed to locate errors 

quickly. For example, the same document may be 

processed by two typists, or by two OCR packages. 

Resulting flies are compared using a software uUhty 

program that locates any differences between the two 

flies. Since differences often correspond to errors, 

this technique helps automate the verification process. 

Since many documents contain figures and images, as 

well as text, the conversion to digital form is not 

complete until non-text portions of the document are 

processed. Scanners can also assist in this process. 

Depending on the !!Oality of the original paper . 

document and the capabilities of the scanner, varymg 

amounts of post-scanning cleanup may be necessary 

to obtain good quality graphics. In instances where 

the item does not scan well, it may be necessary to 

recreate the graphic or figure using a software 

drawing package. 

It is difficult to offer a general rule for how long it 

takes to complete this first step. The necessary time 

26 

Chapter4 

depends on several factors, including: the document's 

quality and length, the number and complexity of 

graphics, and speed and capabilities of personnel, 

tools, and techniques. A simple document with few 

graphics can be processed relatively quickly, but a 

large document with special layout can take 

substantial time. For example, the Air Transportation 

Operations Inspector's Handbook is approximately 

five hundred pages long, laid-out in columns. The 

conversion took over three person-weeks to complete. 

Given the labor intensive natore of conversion, it is 

extremely beneficial to omit this step. This is possible 

only when an electronic version of the original . 

document exists. However, even when an electromc 

copy exists, some processing may be needed to have 

electronic data in a format compatible with HIS tools 

running on mM PC-compatible computers. For 

example, if the digital document exists on a 

mainframe, the data would need to be converted to an 

ffiM PC-compatible text file format. 

4.1.2 Add Markups 

As soon as an electronic version of a document is 

available, the next step is to add special markups to 

the me. Markups are standardized sequences of 

characters used to "mark" portions of the text with 

formatting and hypermedia information. Figure 4.2 

(next page) shows Galaxy Markup Language (GML) 

syntax for some common markups. GML was 

developed a few years ago for the HIS system and is 

similar to standard markup languages like SGML 

(Standard General Markup Language) and HTML 

(Hypertext Markup Language). 

HIS allows for thre<> methods of completing the 

markup step: use the point and click authoring mode 

in the HIS viewer, write and use a macro, or write 

and use a filter program. Each method is described 

below. The markup method chosen depends on the 

size of the document, the number of markups to be 

made, the format of the electronic me, and the 

programming capabilities of the person doing the 

processing. 

-
r 

I 

I 

Chapter4 Digital Documentation Systems 

Commo11 Fonnattiltg Markups 

Bold: <B> ... text ... <IE> 

lt~ics: <I> .. . text ... <II> 

Underline: <U> .. . text ... <JU> 

Font: 

Indent: 

<F"(font),(fontsize)"> ... text.. 

<indent"(level),(pre-indent string)''> ... text ... </indent> 

Center: <center> ... text ... </center> 

Flush Right 

Margins 

<right> ... text ... </right> 

<margin"(left margin),(right margin)> .. . text ... 

Tag: 

ContJUII Hypermedia Markups 

<I'G"(tag reference),(caption)"> 

Hot Table: <HT"(file),(caption)"> .. . text ... <IHT> 

Hot Graphics: <HG"(file),(caption)"> ... text ... <IHG> 

Hot Media: <HM"( file),( capt ion), (start),( end)"> ... text.. . <IHM > 

Hot Executable: <HE''( command line)"> ... text ... <IHE> 

Hot Link: <HL''(tag reference)"> .. . text ... <IHG> 

Figure 4.2 Examples of Common Markups (GML Syntax) 

4.1.2.1 Use HIS Autho{Mode 

A person with no programming skills can use the HIS 

viewer's authoring mode for adding markups to a 

document. Author mode allows a text file to be 

loaded into the viewer and marked up manually. 

Manual markups are accomplished by a user selecting 

portions of the text and then choosing the type of 

markup desired, e.g., bold, topic, or hotword. 

For example, if a user wants to create a hotword 

linking to a graphics file, he or she would select the 

portion of the text he or she wants to be the hotword, 

and then select the menu option to create the link. As 

shown in Figure 4.3 (next page), a dialog box is then 

displayed that allows the user to specify the type of 

link to be created. The authoring system interprets the 

user's point and click actions as instructions to add 

the proper markup to the text file. At the end of each 

authoring session, the user must save changes to save 

markups that were added. While this method is 

feasible for small documents with few markups, it is 

too tedious and time-consuming for large documents 

with a substantial number of markups. 

4.1.2.2 Write and Use a Macro 

The process of adding markups can be automated 

with the help of macro facilities in some word 

processing packages. For example, Microsoft Word 

27 

contains a macro facility which records a series of 

mouse and keyboard actions in a Word Basic 

program. A user needs only minimal programming 

skills to edit these macro programs. Such commercial 

tools can be used to convert formatting information in 

Word files to corresponding GML markups and to 

add other GML markups such as topic tags and 

hotword links. 

One of the greatest benefits of such automation is that 

an unlimited number of files can be processed once 

the macro is written and tested. If the contents of a 

document change over time, a filter automating the 

markup process saves time and money by keeping the 

on-line system current with changes. If the documents 

to be processed are Word files (or a format easily 

converted to Word), this method is the obvious 

choice for adding markups. 

4.1.2.3 Write and Use a Filter Program 

Writing a filter program to add markups to a file 

requires the most programming skill. Before the 

program can be written, one must analyze the 

document to see how it is organized, i.e., Volumes, 

Parts, Chapters, Sections, etc. A user can then write a 

filter that uses lexical tools automatically to place 

markups in the appropriate places. Once the filter is 

written, it can be tested on a representative file to 
Digital Documentation Systems 

1. GENERAL {New-93-3 

to 

1f Pz:ogi:8li.S 

• Documents 

* Pz:oceduz:es 

w Systems 

B. This genei:ic 

pz:oceas can :result 

a pz:oposal. It is 

section is not all 

in conducting day to 

locate and fix any mistakes. If the document is fairly 

uniform, writing and debugging a filter does not take 

very long. However, the filter for FARS took 

approximately a week to write because PARS are not 

uniform, i.e., SFARS and appendices are 

intermingled with Parts. 

After the filter is debugged, a user can write a batch 

file to run the filter on all of the document's files. 

Depending on the document's size and the number of 

markups to be added, run-time may take from 3 to 20 

minutes per document. Although filter programs are 

useful for automating the bulk of the mark-up 

process, it is likely that some markups will need to be 

added manually. A user can add these additional 

markups directly to the GML file with a text editor; 

the HIS Authoring mode can also be used to add a 

small number of mark-ups. 

4.1.3 Index 

The third step in the process is to index marked-up 

files. Indexing is a technical term for building a 

28 

Chapter 4 

database to support full-text searching and 

hypermedia linking. For full-text searching, the 

database stores every word in the document and its 

location in the document. Certain words are not 

indexed because no one would want to search for 

them; these "stop" words include articles (e.g., a, an, 

the) prepositions (e.g., of, at, in) and pronouns (e.g., 

she, he, it, you). 

For hypermedia linking, the database stores 

information for two primary types of markups: tags 

and hotwords. The tag markup designates topics for 

the Table of Contents. The database stores the 

location of each tag markup so the user can jump 

directly from the topic in the Table of Contents to the 

associated text. The hotword markup designates 

words or phrases in the document which link to other 

information. The database stores the location of each 

hotword and the location of its associated text, 

graphic, video, or audio. 
Chapter 4 

HIS tools include an indexing program that processes 

GML files. For a single small document, indexing 

may take only a few minutes; for large documents, it 

can take several hours. The HIS indexing tool allows 

a developer to, index a group of files as a batch job. 

The developer can set up the job and allow it to run 

unmonitored overnight. This feature minimizes the 

impact of a slow indexing process. This process can 

be repeated over several nights to index very large 

documents. For example, it took about eighteen hours 

to index the FAR text into an HIS database. 

4.1.4 Structure Topics 

In the HIS system, topics correspond to items listed in 

the Table of Contents, such as the chapter, section, 

and subsection headings. In the markup step, all 

topics are identified with the tag markup. The indexer 

stores each topic's location in the database, so a user 

can jump from the Table of Contents to any topic's 

beginning. The final step in the conversion process is 

to structure topics into an outline so the HIS Table of 

Contents viewer displays the topics hierarchically. 

To illustrate the effect of the structuring process, 

Figure 4.4 shows HIS displaying the Table of 

Contents for an examplo document, both before and 

after structuring. For the unstructured document, 

notice that all topics are listed without any indenting. 

After the topics are structured, HIS displays only 

topics at the highest level of the outline, such as the 

chapter titles. When the user clicks a page icon, the 

next outline level appears. 

The structuring process does not require a lot of time, 

compared with the time required for other steps in the 

process. This step is partially automated, so a small 

C'l CHAPTER ONE D001s 

C'l SECTION ONE Wings 

C'l VOLUME TWO AIRCRAFT INTERIORS 

C'l CHAPTER TWO Seats and armrests 

C'l SECTION TWO Cockpit 

VOLUME THREE AIRCRAFT ENGINES 

CHAPTER THREE Fluids 

SECTION THREE Wiring 

Digital Documentation Systems 

program must be wrillen to add level information to 

topics in the HIS database. A structuring program is 

customized to the syntax. of the topics in a document; 

therefore, it will only be valid for documents with the 

same syntax. For small documents, run-time can take 

less than an hour; for larger documents such as the 

FARS, run-time may take several hours. 

4.1.5 Discussion 

The digital documentation process obviously requires 

some investment of time. The actual time required 

depends on several factors, including the size and 

state of the original document. To illustrate all the 

steps for the HIS system, in this section, we discussed 

the four basic steps necessary if a large document 

does not exist in digital form. There are substantial 

time savings to be gained if the process can start with 

an electronic, rather than a paper, document. 

We did not discuss additional steps required if audio, 

video and/or animation are to be included in the 

digital documentation. Additional time and effort is 

required to locate and/or create such media, as well as 

to process it into a form the HIS system can use. If 

the additional media already exists, and is easily 

located, costs are lower than if original media must be 

created. Appropriate footage may not exist, or may 

take a long time to locate. When appropriate footage 

is located, copyright permissions must be obtained 

before it can be used in the project. 

The benefits of digital documentation, with or without 

additional media, must be weighed against the costs 

for converting and maintaining on-line 

documentation. Informal evaluations of the HIS 

Figure 4.4 HIS Table of Contents Unstructured vs Structured Toprcs 

29 
" 

" 

Digital Documentation Systems 

system have been conducted, with positive results. 

The benefits of quicker and more accurate access to 

information, as well as portability of electronic data, 

provide sufficient benefits to warrant conversion of a 

variety of aviation ll!aintenance data to digital form. 

.u 1"111-: tLLCTRO\IC Ill \1 \\ 

I· \C I"ORS CllllE Hm \\I\ 110\ 

\1 \1\TE\ \\CE One of the major digital documentation projects 

completed during the past year was the design 

and development of a prototype Electronic Human 

Factors Guide. This Electronic Guide (E-Guide) is 

the digital counterpart of the paper-based Human 

Factors Guide for Aviation Maintenance (the Guide). 

The Guide describes fundamental human factors 

concepts and guidelines for aviation maintenance 

supervisors and technicians. Its goal is to provide 

practical, usable guidance to supervisors and planners 

in the aviation maintenance industry. 

The &Guide utilizes the IDS functionality to improve 

access to the Guide's content. It provides the HIS 

full-text search capability, as well as hypertext linking 

between chapters. The &Guide expands on the 

Guide's content by incorporating video that 

supplements the paper-based Guide's text and still 

images. 

The IDS authoring tools were selected for 

development of the E-Guide over commercially 

available tools for three primary reasons. First, the 

IDS technology met the functional requirements that 

were desired. Second, most commercially available 

tools that meet the functional requirements do not 

meet the cost requirements. That is, substantial fees 

are required for distributing the commercial software 

used to view the electronic information, typically 

around $50/copy. Documents developed with HIS 

authoring tools do not incur any "per copy" costs. 

Finally, customization is possible using the internally 

developed HIS software. If a new feature is needed or 

a change in an existing function is required, the HIS 

authoring tools can be modified. Such control is not 

possible with commercial software tools. 

In this section, we describe design issues and 

interface features of the prototype system. We 

conclude with a summary of initial user feedback 

about the E-Guide and the modifications we are 

implementing. 

30 

Chapter4 

4.2.1 Designing the Electronic Gnide 

The E-Guide was designed in coordination with the 

paper-based Guide. As with the paper-based Guide, 

there were three design goals for the &Guide: 

• it should be readily accessible to the 

aviation community 

• it should be easy to maintain 

• it should be easy to use. 

In this section, we discuss how we achieved these 

goals during the design and development of the initial 

E-Guide prototype. 

4 2 1.1 Achjeyjng !he Accessjbjljty Goal 

One goal of the Human Factors Guide research 

program is to provide wide and easy access to the 

information written for the Guide. The E-Guide will 

be accessible in two ways: CD-ROM and Internet. A 

CD-ROM disc holds approximately 650 megabytes of 

data; this is sufficient space for the Guide's text and 

media, as well as relevant documentation such as the 

F AAI AAM meeting proceedings and phase reports. 

Because such a large quantity of information can be 

stored on one CD-ROM disc, theE-Guide can easily 

be distributed to the aviation community at a 

reasonable cost. The cost to replicate each disk, 

including packaging materials, is approximately 

$1.65. 

The research team is investigating the lnternet as an 

alternative means for information distribution (see 

Chapter 5, Skyway). The Guide's complete text will 

be on the Internet to ensure wide distribution of the 

information, especially to those without a CD-ROM 

player. To date, one draft chapter of the &Guide has 

been successfully cqnverted to HTML and placed on 

the Internet. 

4.2.1.2 Achieving !be Maintenance Goa! 

The Guide is intended to provide practical guidance 

to aviation maintenance supervisors and planners. 

Since issues and problems of maintenance constantly 

change, the Guide needs periodic updating to address 

new problems. The challenge is to keep the 

information in the Guide current at minimal cost. 

The paper version solves this problem by providing 

the Guide in a three-ring binder, instead of in book 

form. A chapter can be added, eliminated, or 

upgraded without discarding the whole book. This 

keeps the cost to upgrade and distribute information 

at a minimum. 

• 
Chapter4 

The cost to upgrade the system includes the cost of 

modifying both digital documentation and interface 

software, as well as the cost to redistribute the 

software. Redistribution costs are minimized by using 

CD-ROM and the Internet. The cost of modifying 

software depends on the effort involved in 

reprocessing portions of the digital documentation. 

We streamlined the HIS digital documentation 

process in the following ways to minimize this cost: 

• The Guide is being developed in Word to 

eliminate the need to convert from paper to 

digital form. 

• We created a customized Word macro to 

automate markup. The macro automatically 

deletes unnecessary formatting information from 

the Word files, adds the required hypermedia 

commands, and saves the file in the proper HIS 

text format. 

• We created a separate HIS database for each 

chapter. This modularizing of the databases 

allows a chapter to be added, deleted, or 

modified without reprocessing the contents of 

other chapters. 

4.2.1.3 Achieving the Ease of Use Goal 

Both the paper and electr'onic versions of the Guide 

are designed to be easy to use. The E-Guide retains 

ease-of-use features of the paper Guide, including its 

organizational structure of the sections and the 

chapter icons. There are other factors to be 

considered in designing and implementing a useable 

software interface that go beyond the features 

inherited from the paper version. 

User interface design is a critical project element 

because it plays such a major role in users' 

Digital Documentation Systems 

acceptance of the electronic version. A user, 

especially a computer novice, is more likely to use the 

E-Guide if the interface allows him or her to focus on 

finding and using the Guide's information, rather than 

focusing on navigating and using the software. The 

research team developed a customized interface for 

theE-Guide which exploits the Human Factors 

Guide's specific structure, rather than simply using 

the Hypermedia Information System's (HIS's) 

generic interface. 

To ensure an intuitive, user-friendly program for the 

custom interface, we are using the cyclic design 

model to design and develop the E-Guide. Figure 4.5 

shows the four iterative steps involved in the process: 

analyze, design, implement, test. We have completed 

one cycle to date. 

To further ensure a usable, commercial appearance 

for the E-Guide, the researchers evaluated interface 

features of twelve commercial CD-ROM applications. 

Each application was evaluated for its ease of 

navigation, overall ease of use, screen layout, and 

media integration. For details of this evaluation, see 

Hartzell, 1994. TheE-Guide prototype design was 

based on this evaluation, as well as human interface 

design research findings and guidelines. 

4.2.2 The Interface Features 

In this section, we describe interface features of the 

E-Guide' s initial prototype. We follow this section 

with a summary of initial user evaluation feedback 

and a description of the resulting modifications we 

will make to the initial prototype software. 

4.2,2.1 The lni!Odyction 

TheE-Guide's introduction is a real "attentionCyclic Process 

;PDes~n~ 

Analyze Implement 

~ Test I:fl ----------- --------- -

Figure 4.5 The Cycl1c Des1an Model 

31 
Digital Documentation Systems 

getter." It starts animation of the title: Human Factors 

Guide for Aviation Maintenance. A video clip 

introducing the FAAJAAM research program follows 

the animation. This introduction plays until a user 

presses any key or clicks a mouse button: the system 

proceeds to display the Table of Contents. 

4 2 2.2 The Table of Contents 

The Table of Contents in the paper Guide is in the form 

of a conventi~nal text outline of chapter titles. The EGuide presents the Table of Contents as a unified scene 

(Figure 4.6. Since the Guide is intended for members 

of the aviation community, we chose a hangar for the 

scene. Each graphical image in the hangar represents a 

chapter in the Guide. We chose each image to illustrate 

the chapter it represents, while always maintaining the 

aviation maintenance theme. For example, a time clock 

with punch cards represents the chapter on Shiftwork 

Scheduling. This pictorial Table of Contents serves as 

an overview map from which the user can access any 

chapter. Pointing at an image with the cursor displays a 

pop-up displaying the chapter's title; selecting the 

image displays the chapter's Introduction in the 

Information Viewer. 

32 

Chapter4 

4 2 2 3 The ln!orma!ion Viewer 

The Information Viewer displays the Guide's content 

(Figure 4.7, next page). The Information Viewer's 

design is critical for meeting the ease-of-use goal; this 

is the primary screen for accessing information in the 

HumtJn Factors Guide. We conducted an analysis of 

user needs to identify displays and controls to include 

in the viewer. We desigued the Information Viewer to 

use dedicated locations for all display areas and 

controls: all information and program functionality is 

visible on the screen. In this section, we describe key 

features of the Information Viewer: the Section buttons, 

the Text Window, the Media Window, and theEGuide Control Buttons. 

4.2.2.4 Section Buttons 

Each of the Guide's chapters is divided into twelve 

sections: Introduction, Background, Issues and 

Problems, Regulatory Requirements, Concepts, 

Methods, Reader Tasks, Guidelines, Related Issues, 

Where to Get Help, References, and Further Reading. 

In the E-Guide, sections are represented by twelve 

section buttons grouped together just above the Text 

• 
Chapter 4 Digital Documentation Systems 

I Text Window I 

Media Wirnd<ow·-

Media Oescoription._ 

Media Control But~onrs ~-

Window (which displays the section's text). Each of 

the twelve section buttons has a distinct icon. The 

icons are metaphors for familiar objects; this allows 

users to have quicker recognition of each section 

button. If a user is unsure what an icon represents, the 

section's name is displayed in a help balloon near the 

button whenever the user places the cursor on top of 

the button. 

A user selects a section button to view a different 

section of the current chapter. When the user selects a 

section button, the button is inverted, and the mouse 

cursor changes to an hourglass until the Information 

Viewer has retrieved the section text. This design 

gives users immediate access to information in any 

section and allows them quickly to identify what text is 

currently displayed by noting which section button is 

currently inverted. 

4.2.2.5 Text Window 

As mentioned above, the Text Window is located 

below the section buttons. This window displays the 

selected section's text in the same format as the 

paper-based Guide. The text's size is slightly larger 

33 

than the paper version's to make it easier to read the 

computer screen. 

Within the text, some words are displayed in a 

different color; such words are called hotwords. A 

hotword indicates that there is associated text or 

media related to that word. The association is called a 

hyper/ink; it wovides a software connection between 

the hotword and another document, graphic image, or 

definition. Hotwords give users rapid access to 

information; selecting the hotword displays its 

associated text or media. Section text is displayed on 

the left side of the screen; graphics and other media are 

displayed in the Media Window on the right. A user 

can view text and its corresponding media 

simultaneously. 

4.2,2.6 Media Window 

The Media Window displays tables, figures, video, and 

animation associated with the current chapter's content. 

In the Information Viewer, the Media Window is 

located to the right of the Text Window. Below the 

Media Window, the Media Description box contains a 

short description of the image currently displayed in the 

Media Window. Until a user selects a figure or media 
Digital Documentation Systems 

file, the Media Window displays the FAA AAM logo. 

The logo serves as a filler, blending the Media Window 

into the background and preventing the user from being 

distracted by an empty window. 

The Media Control Buttons are directly beneath the 

description. The graphic on each button illustrates the 

media the button controls, e.g., a video camera for the 

video control, a camera for the photo/figure control, 

and a chart for the table and charts control. When the 

current chapter has no media of a given type, the 

corresponding control button is disabled. A user may 

select an ef!abled media button to display a list of 

associated media for the current chapter. For example, 

when a user clicks on the video control button, a list of 

video clips relevant to the current chapter is displayed. 

A user can select any item in the list to. view the 

associated video. When a user selects a figure, table, or 

other media file, the Media Window replaces its 

previous contents with the newly selected file. The 

transition effect draws the user's focus to the Media 

Window. 

The Media Window's default size is a relatively small 

ISO x 130 pixels. This size is appropriate for video 

clips or animation playback; however, a table or a 

figure is typically much larger. The Media Window 

displays a scaled-down version of tables and figores in 

overview. To see the image's details, the user can 

enlarge the table or figure to its original size. The 

enlarged table or figure is displayed in a separate 

window with the caption as the window's title. The 

main Information Viewer window is deactivated while 

this enlarged window is displayed, preventing the user 

34 

Chapter4 

from getting lost or confused by there being too many 

windows on the screen. 

Tables and figures in the E-Guide are taken directly 

from the paper Guide. The graphics are stored as image 

files, preserving their format and color. The audio, 

video, and animation media, which are not part of the 

paper Guide, had to be collected and processed for the 

E-Guide. The current design of the Information Viewer 

allows the following file formats: WA V files for audio, 

A VI files for video, FU and FLC files for animation, 

GIF and BMP for still images. 

4.2.2.7 Electronic Guide Control Buttons 

E-Guide control buttons access navigational and 

system functions. These control buttons are located at 

the bottom of the Information Viewer screen. The basic 

functions of the buttons are as follows: 

• Next and Previous chapter buttons 

display the next or previous chapter in 

the Text Window 

• Table of Contents button displays the 

Table of Contents overview map 

• Go To button allows a user to go directly 

to any section of any chapter 

• Search button allows a user to search the 

Guide for specific words or phrases 

• Print button allows a user to print 

selected text or graphics from the Guide 

• Help button displays the on-line E-Guide 

Help window 

• Exit button exits the E-Guide. 

• 
I 

I 

Chapter 4 

Many of these functions are straightforward. A user 

simply clicks the appropriate control button and its 

corresponding action occurs. Some functions require 

additional input, typically supplied in a dialog box. For 

example, Figure 4.8 shows the "Go To" dialog box in 

which a user must give the desired chapter and section. 

"Search" is one of theE-Guide's most useful functions; 

it requires additional user input. This function is used 

much as one might use a combination of the Table of 

Contents and the Index in the paper-based Guide. If a 

user wants information on a specific topic in the paperbased Guide, he or she might scan headings in the 

Table of Contents or look up the specific topic in the 

Index. 

In theE-Guide, a user selects "Search" to locate 

relevant material. A dialog box helps a user provide 

information necessary for the search (see Figure 4.9, 

next page) with options to search the current section, 

the current chapter, or the whole book. A user must 

specify one or more words or phrases. To search for a 

single word or a phrase, a user types the desired term or 

phase in the "Find" box and selects the Search button. 

When a user has supplied necessary information, he or 

she executes the search by clicking on the Search 

button. The hourglass cursor is displayed until the 

search is complete. A dialog box then displays a list of 

chapter numbers and section names in which the term is 

35 

Digital Documentation Systems 

found. As shown in Figure 4.9 (next page), the system 

automatically highlights a search term contained in the 

currently displayed section. 

TheE-Guide is also capable of complex searches with 

wildcards. A wildcard search means that a user can use 

wildcard characters to search for variations of a word. 

The E-Guide suppotts two standard wildcard 

characters: "?"represents any single character, and"*'' 

represents one or more characters. For example, a 

search for "eire*" would find terms such as "circa," 

"circadian," "circular," and "circumstances." A search 

for "eire?" would yield only "circa" from the above list. 

4.2.3 User Feedback and Interface Modifications 

We demonstrated the first prototype of the E-Guide at 

the Ninth FAA AAM Meeting on Human Factors in 

Aircraft Maintenance and Inspection. In addition, 

several attendees used the prototype in a workgroup 

setting, identifying several interface and usability 

issues. The issues, notes, and "wish-list features" are 

summarized below, along with the modifications we 

will make to the E-Guide: 

• Text Display: An attendee suggested 

implementing an option to display the text in a 

full-screen window. Although while in the fullscreen mode, the user cannot view the supporting 

media simultaneously. There may be times when 
Digital Documentation Systems Chapter 4 

The reality of the fatigue-induced performance problems 

as·sociated with shiftwork has been evident for many years. 

However, only recently have the underlying mechanisms that 

cause these effects been identified and explained. The key 

concepts related to shiftwork·related performance problems are 

described below. 

:mrg Ntt!t RhVthms 

The term circadian refers to the fact that the types of cycles 

we described in previous sections have a period of nearly I but 

not exactly I a 24-hour day. The word circa is Latin for about or 

approximately. The physiological and psychological variables 

that follow our internal clock are said to be circadian. Body 

temperature is an I of a circadian cycle. Figure 4·1 

shows the temperature I for an 

the user is only interested in reading text. We will 

implement this option. 

• Table of Contents: The Table of Contents 

represents each chapter in the Guide with a 

graphic image. Although this approach provides a 

unifying theme and lends a commercial look to the 

prototype, some users may be more comfortable 

with a traditional Table of Contents. Participants 

suggested that the E-Guide include an option to 

switch between the two Tables of Contents. We 

will implement this option. 

• lconized-Section Buttons: Due to users' 

unfamiliarity with icons and contents of the 

Human Factors Guide, they did not utilize the 

section buttons very much. Participants 

recommended adding a menu list of all sections as 

an option to the section buttons. We will 

implement a menu that allows a user to make a 

selection with the mouse or the keyboard. 

• Tables and Figures: Since current tables and 

figures are image files, users cannot perform 

searches on their information. Users identified 

expanding the search capability to include this 

information as a necessary modificalion: important 

information resides in tables and figures. We will 

investigate the feasibility of adding such a feature. 

36 

• Hyperlinks: At the time of the conference, we had 

not implemented linking from one portion of the 

text to another. Participants indicated their desire 

to have footnotes linked to the associated 

reference. They were also interested in links 

among the E-Guide and other FAA and DOT 

documents referenced in the text. We will 

implement hyJ2"rlinks to references; we will 

implement linking to additional documents as time 

and money permit. 

Other general feedback participants gave us on both the 

paper-based and electronic versions of the Human 

Factors Guide included the following: 

• Glossary: Attendees commented that many 

aviation maintenance managers may not be 

familiar with the technical meaning of terms (e.g., 

fatigue) we use in the Guide. Some attendees 

suggested including definitions from an aviation 

dictionary. We plan to add a glossary to both 

versions of the Guide. 

• Examples: The attendees recommended adding a 

section in the Guide of "Examples of Best/Current 

Practices" from the airline industry. We will 

include two new sections in both versions of the 
Chapter 4 

Guide: Example Scenarios and 

Acknowledgments. 

4.3 FAA/AAl\1 Cl>-R0\1 #3 

For the third consecutive year, one of the digital 

documentation task's major deliverables is a CDROM. As in the past, the current CD-ROM contains 

several software programs produced as part of the 

FAA AAM Human Factors in Aviation Maintenance 

research program (Figure 4.1 0). In this section, we 

briefly describe the contents of CD-ROM #3. Readers 

may find additional details on a particular application 

by referring to the corresponding chapter in this 

report. 

4.3.1 Hypermedia Information System(HIS) 

The Hypermedia Information System (HIS) project 

provided the impetus for developing the first CDROM. During the past year, we have improved and 

expanded the HIS' features and contents. The 1995 

version of HIS provides over 5,000 pages of 

information related to aviation maintenance and 

inspection, including the following: Human Factors in 

Aviation Maintenance Phase Reports and Meeting 

Proceedings, Federal Avlation Regulations (Parts 1-

37 

Digital Documentation Systems 

200), the Airworthiness Inspector's Handbook (Order 

8300.1 0), and the Air Transportation Operations 

Inspector's Handbook (Order 8400.10). 

The HIS program contains a graphical user interface 

that makes it easy for a user to browse through these 

documents, and hypermedia technology affords rapid 

access to specific information. The full-text search 

function allows searching within and across all 

documents in the system. Storing digital 

documentation electronically on CD-ROM is one 

feasible method for improving distribution and access 

to information. 

4.3.2 Electronic Human Factors Guide 

Since the paper-based Human Factors Guide will not 

be published until later this year, CD-ROM #3 

contains only a demonstration version of the 

Electronic Human Factors Guide that is similar to the 

initial prototype described in this chapter. However, 

since the text for all chapters is under revision, only 

two revised Chapters are included in the 

demonstration program: Chapter I (Human Factors) 

and Chapter 4 (Shiftwork and Scheduling). 
'"' 

Digital Documentation Systems 

4.3.3 Ergonomics Audit Program-ERNAP 

The ERgoNomic Audit Program (ERNAP) is a 

computerized job aid that helps managers evalua_te or 

design ergonomically efficient procedures and 

systems for maintenance or inspection. ERNAP is 

simple to use; it evaluates existing and proposed tasks 

and setops by applying ergonomic principles. If an 

evaluation is unfavorable, ERNAP suggests 

ergonomic interventions. 

The complete ERNAP system contains twenty-three 

modules spanning Pre-Maintenance, Maintenance, 

and Post-Maintenance. The CD contains an initial 

prototype of the software. A complete version is to be 

published on the Electronic Human Factors Guide 

CD-ROM in June 1995. 

4.3.4 Coordinating Agency for Supplier 

Evaluation (CASE) 

The vendor audit program for the Coordinating 

Agency for Supplier Evaluation (CASE) Air Carrier 

Section is an adaptation of the Aviation Safety 

Inspectorjob-aiding software. Auditors from each 

participating airline perform insPe<:tions of their 

respective vendors and contribute their findings to 

CASE resources. The software is designed to help 

auditors collect required data during on-site 

inspections of vendors. 

The fully functional CASE program is designed to 

operate on a pen cotnputer running Microsoft 

Windows for Pen Computing. The CD-ROM contains 

a demonstration program illustrating the main 

features without requiring the special operating 

system. 

4.3.5 Office of Aviation Medicine Video Brochure 

The Office of Aviation Medicine Video Brochure 

describes the FAA's Office of Aviation Medicine 

(AAM) goals, organization, and work in a series of 

short video clips. The software is designed to be used 

either on a "public access" computer (video kiosk) or 

on a personal computer. The AAM Video Brochure 

uses the Microsoft Video for Windows system, which 

displays digital video on a computer without 

requiring special hardware. 

4.3.6 PENS Video Brochure 

The PENS Video Brochure describes the 

Performance Enhancement System (PENS) research 

program in a series of short video clips. The Video 

Brochure software is designed to be used either on a 

38 

Cbapter4 

"public access" computer (video kiosk) or on a 

personal computer. The PENS Video Brochure 

displays digital video on the computer without 

requiring special computer hardware. 

PENS is an electronic performance support system 

designed for Aviation Safety Inspectors. It provides 

data entry and validation support, as well as on-line 

access to policy guidance such as Federal Aviation 

Regulations, Airworthiness Directives, and 

Inspector's Handbooks. The system is currently used 

by the FAA Flight Standards Service. 

.JA Fl 'lllH" l'L \"\S I OR IJI(;II \L 

llOCl \II\"\ r \riO"\ IH.Sl·. \IH II \"\D 

Dl•.\ EI.OI'\11· "\T 

S orne current digital documentation research and 

development efforts continue through the next 

year. We will continue work on the Electronic Human 

Factors Guide. The first complete E-Guide will be 

published on CD-ROM in June 1995. As we revise 

the paper-based· Hu1111Jn Factors Guide, theE-Guide 

will also be updated. 

Work on the HIS system continues. As our work tbe 

E-Guide demonstrated, there are specialized needs for 

digital documentation, i.e., a generic interface like the 

HIS may not always be desirable. However, a custom 

interface may well want pieces of the HIS' 

functionality. We now have the idea of carving 

modules out of the HIS software for use in other 

programs. We used this process for the Search 

function used in the E-Guide. We are likely to 

continue modularization of the HIS during the 

coming year. We will publish a new HIS on CDROM #4 in Marcil 1996. This CD will also contain 

software developed for other projects within the 

overall research program. 

We have new research and development avenues to 

address in the coming year. Current systems have 

demonstrated the feasibility of digital documentation 

for the aviation industry, but technological and 

organizational changes have occurred since we began 

our research. New hypermedia and multimedia 

development tools are available. Commercial systems 

providing large-scale imaging tools for document 

management have been developed. New digital 

documentation standards are evolving as commercial 

companies enter the market with products providing 

aviation-specific digital documentation libraries. Our 

research and development work should not replicate 

services now available commercially. 
r 

' 

Chapter4 

Our future research will adapt to the aviation 

maintenance industry's current needs. We have to 

pose questions as to what needs commercial suppliers 

are already meeting (or will be meeting in the near 

future) and what needs remain for further research 

and development. In conjunction with this type of 

needs analysis, we need to review new tools, 

standards, and techniques formally. We can then 

define further investigations to match technology and 

needs. 

39 

Digital Documentation Systems 

FAAJAAM & GSC. (1994). Phase IV Report 

(Chapter 6: The Hyperrnedia.Inforrnation 

System) 

Hartzell, Karin. (1994). "CD-ROM Interface 

Evaluation" unpublished report. 

Maddox, Michael. (1994). "Introducing a practical 

human factors guide into the aviation 

maintenance environment." 

Mantelman, Lee. (1994). "Windows OCR packages 

put a typist in your tank". Imaging Magazine. 

Vol. 3, Number 12, December, 1994. pp. 8-26. 
CHAPTERS 

THE FAA 

INFORMATION SKYWAY 

Thomas Coonan 

Galaxy Scientific Corporation 

5.0 1:\TRODl'CTIO~ 

The Office of Aviation Medicine (AAM) Human 

Factors in Aviation Maintenance research team 

has been exploring alternative methods for disseminating the products from the research program. Examples include publication of project results on CDROM, the Human Factors Guide for Aviation Maintenance, and annual meetings and reports. The program has included efforts to involve the research and 

user communities in its decision-making processes. 

Another avenue for disseminating information is 

through an on-line electronic information source. 

This new distribution channel has been termed the 

FAA Information Skyway. 

This report presents our vision of what the Skyway 

is, of our progress with our User Needs Survey, a 

survey of existing services, and a snapshot of the 

World-Wide Web (WWW)-based Skyway to date. 

As shown in Figure 5.1, the AAM will use the lnforFAA site fa document 

updates and new 

doCument/ sollwore 

re·releoses 

E-mail. 

fax. vldeoconferenclng 

with users and 

researchers 

Users place 

Inquiry 01 order 

for Information 

~. 

Program 

newsletter or 

· 1ourno1 distribution 

' Users requests 

for new Information 

Gopher. FTP and www site 

fa new produCt 

odverllsement and Info. 

41 
Information Skyway 

mation Skyway to: 

I. Disseminate information from the Human 

Factors Research Program, Office of Aviation Medicine, and the FAA to all Internet 

users 

2. Maintain and update official aviation-related 

documents and standards generated by the 

Office of Aviation Medicine for immediate 

world-wide use 

3. Provide additional Maintenance Human 

Factors-oriented Internet services, such as 

notification bulletins, information archiving 

and retrieval, and conducting world-wide 

discussion groups. 

A substantial portion of the FAA Information Skyway will be based on the WWW, a Standard General 

Markup Language (SGML)-based hypermedia information layer available through the Internet. The 

WWW allows hypertext access across all WWW 

hosts and documentation. Most WWW hosts are government- sponsored research organizations or commercial publishers. 

Internet and the WWW are explosively growing mediums for information access (Stefanac, 1994). Previously restricted to government research and 

educational frrms, Internet fl:Cently opened access to 

general business organizations. Seven thousand businesses and organizations now. have 15 million Internet users-there are one million more users each 

month. Over a recent 12 month period, WWW traffic 

increased 341,634%; and a new network is joining 

the Internet every 10 minutes. Twenty-one large 

Bulletin Board~Systems (BBSs) have also connected 

to the Internet, at least for e-mail transfer. More than 

half of all registered networks are now commercial. 

Surveys have also been done on existing WWW users (Pitkow, 1994). 

Immediate benefits for the AAM of the FAA Skyway 

include publicity and immediate distribution of the 

Office's public information, research results, and official notifications. Previous AAM experiences with 

electronic distribution of research information, by 

way of CD and SGML, technically position the AAM 

to pursue this form of publication. 

Long-term benefits of the FAA Skyway are based on 

current research and development activity among 

commercial aviation manufacturers and FAA AAM. 

Commercial aviation manufacturers are beginning to 

distribute documentation electronically in an SGML 

42 

Chapter 5 

format. (Remember that WWW is SGML-based, 

too.) Current AAM and FAA research projects are 

evaluating how to use portable computers to support 

maintenance and inspection activities. The merging 

of portability, world-wide access, and a plethora of 

electronic aviation-related documentation will serve 

to bring timely information to our maintenance and 

inspection users. 

5.ll SER :\EEllS SL R\ E\ 

The Information Skyway User Needs Survey has been created empirically to determine needs in 

the community. The survey's intent is to establish 

what members of the Aviation/Human Factors community have, need, and want from existing or potential on-line electronic information services. 

Specifically, the survey includes questions on what 

classes of FAA information and services community 

members desire, what computer resources users have 

access to, and individual affiliations and job functions. The survey will be distributed to people across 

the airline, academic, and government sectors. The 

survey is included in this report as Chapter 5 - Appendix. 

The question arises as to how innovative an approach 

the Skyway should take. An innovative strategy attempts to identify, refine, and specialize emerging 

technologies and prepare users for the new and hopefully ubiquitous technology. Alternatively, a more 

conservative and applied strategy minimizes risk by 

employing only the most widely available tools, if 

not innovative tools. 

The Skyway occupjes the more innovative position 

on this scale. The Internet is a major information 

~technology and, while not yet on every desktop, is 

here to stay. We predict that the Internet will be a 

primary source for electronic information - including 

Aviation and Human Factors information. 

5.2 l'OTE:\Tl.\L SK\ \\ \\ SER\ ICES 

The User Needs Survey will help us determine 

what the Skyway should do, what information it 

should include, and how it should be accessed. There 

are two immediately apparent ways for members of 

the public to access computerized on-line information: the Bulletin Board System (BBS) and the Internet. 
Chapter 5 

BBSs are typically accessed with low-speed modems 

over standard telephone lines. A BBS is often hosted 

on a PC with many modem ports. One advantage of 

BBSs is that they require modest equipment: a PC 

with a low-speed modem and modest graphics, and 

no pre-established account. BBS services typically 

include E-Mail (amongst users of the BBS), real-time 

CHAT conversations, and uploading and downloading files .. Usually, these systems do not offer advanced services such as document searching, 

hypertext, or multimedia. 

The Internet is a computer network pioneered in the 

1960s. Today, many millions of users in the public, 

academic and governmental sectors share in this 

global fabric. Internet services are typically more advanced than a BBS's and include E-Mail, file 

up/down loading, hypertext, multimedia, video conferencing, etc. Until recently, it was difficult to connect to the Internet. Only university researchers or 

government officials could afford the specialized 

communications connections or could use the UNIX 

environment. However, access is now much easier. 

New protocols (such as Serial Line Internet Protocol 

or SLIP), modems, public domain software and 

commercial Internet Service Provider (ISP) companies make access feasible for many people. This 

trend continues; in fact, reports are that the upcoming 

Windows 95 will come bundled with Internet software and that the Internet will reside on most desktops. 

The Internet, specifically the World Wide Web, is 

our first experiment in the Information Skyway. We 

do not see the Skyway necessarily as a single medium or service, so our initial foray into an Internetbased Skyway does not preclude future work with 

BBSs or any other means of effectively delivering information electronically. 

5.2.1 Internet Services 

Before discussing Internet services, we will briefly 

discuss methods of access. Until recently, Internet 

connectivity required high-speed digital communications found only in sophisticated labs and large offices. With the introduction of SLIP protocol and 

high-speed modems, a typical PC can cost-effectively 

establish a true Internet connection. ISPs offer a SLIP 

dial-up bridge into the Internet for a few dollars per 

month. In fact, Internet access is now as easy as dialing up a bulletin board. 

43 

lnformtJtion Skyway 

We made a survey of Internet Services, seeking out 

both mainstream and emerging Internet technology. 

Services we investigated included E-Mail, Gopher, 

video conferencing, Lotus Notes, WWW, File Transfer Protocol (FTP), ListServers, and Multiple User 

Domains (MUDs). We gave most attention to WWW 

and FTP as potential services due to their widespread 

use, high growth, and appropriateness for digital 

documentation. 

5.2.1.1 Electronic Mail 

E-Mail is a core Internet service and is available in 

many environments other than the Internet. Different 

E-Mail systems typically communicate via Gateways. 

For example, E-Mail is routinely exchanged between 

CompuServe, America On-Line (AOL), and the Internet users, as well as many localized proprietary 

LAN-based E-Mail systems such as ccMail, PROFS, 

and Microsoft Mail. Text-based E-Mail can be enhanced with multimedia attachments, as well as with 

groupware-oriented enhancements such as ListServers (see Section 5.2.1.5, next page). 

5,2,1 2 The World Wide Web 

The WWW, commonly referred to as "the Web", is 

one of the fastest growing Internet services. A user 

views WWW documents called "pages" by using a 

WWW viewer or browser. Many browser programs 

are available for most platforms, including NCSA 

Mosaic, CELLO, NetCruiser, and NETSCAPE. Web 

pages may include text, graphics, or multimedia. 

Links within the text allow the user to branch off to 

other WWW pages or other Web sites anywhere in 

the world. The ability to move between documents 

and/or host COIIli>Uters by using links embedded in 

the text is called "hypertext". WWW pages may also 

be searched for key words or phrases. 

WWW documents use the HyperText Markup Language (HTML) format for providing text and graphical hypertext. The HTML format is standardized and 

extensible. Web servers may provide back-end programs triggered by the reader's manipulation of the 

page. For example, a WWW page may present an 

interactive form or provide a front-end to a large database system. 

WWW pages may include references or links to the 

other Internet services. For example, the user may 

click on a link that triggers an FTP download of a 

particular file or that makes a link to a Gopher menu. 

In this way, WWW subsumes many other Internet 

services. 
Information Skyway 

5213FTP 

File Transfer Protocol (FI'P) is perhaps the oldest 

Internet-based service. Simply put, FI'P allows users 

to retrieve files from sites on the network. FI'P archives are maintained throughout the Internet. FI'P 

users access files organized in hierarchical directories 

on specific hosts. There are many topic-specific FI'P 

archives. For example, Microsoft maintains an archive for Visual Basic software and there are FI'P 

general archives dedicated to electronic versions of 

popular manuals. 

5.2.1.4 Gopher 

Gopher is a precursor to WWW and presents information in a hierarchical menu. Users view a linear 

list of items which lead to other Gopher menus or to 

text. Gopher's simplicity allows it to easily run on 

almost any client interface, including text-based 

terminals. Like the WWW, Gopher items link easily 

to other Gopher items on other distant nodes. Figure 

5.2 shows one example series of Gopher menus. 

5.2.1.5 ListSeryers 

One popular service is the ListServer (also known as 

a mail reflector). ListServers are an extension to EMail. ListServers are established for particular topics 

44 

Chapter 5 

(similar to UseNet groups). Users send specific EMail "command" messages to the server to subscribe 

and unsubscribe from the list and to request lists of 

current subscribers. Once subscribed, users send 

messages to the group and, likewise, receive messages from the entire group. Since a ListServer is 

based on simple E-Mail mechanisms, any E-Mail 

user, on the Internet or not, may utilize the service. A 

potential Skyway service is one or more ListServers 

for topics such as "Human Factors in Aviation." 

5.2.1.6 O!her Seryjces 

Other, more exotic Internet services include MUDs 

and Video Conferencing. MUDs are text-based 

groupware programs originally intended for multiplayer role-playing games. MUDs have been suggested as a new vehicle for real-time conferences 

where participants interact with each other in 'rooms' 

based on a particular sessions, topics, etc. 

While seamless video requires higher bandwidth 

links, several real-time video conferencing systems 

exist on the Internet. The CU_SeeMe video conferencing system is a simple, low-bandwidth video system which has been employed in K-12 schools. The 

DRUMS system from Sprint integrates Silicon 

-
r 

I 

I 

I Chapter 5 

Graphics Indy systems, video cameras, and highspeed TCP/IP (Transmission Control Protocol/Internet Protocol) links to bring together professional studio video producers and their clients. 

There are other important network-based services 

which are not necessarily Internet-based at all, but 

still may be accessed by the Internet. For example, 

Lotus Notes is a groupware product running on 

LANs (Local Area Networks) such as Novell. Corporations are using Lotus Notes for E-Mail, group 

scheduling, group coordination, etc. 

5.3 THE SKY\\' A Y II'VJ'ER"iETWWW ll\IPLE!\IENT.\TION 

The present accessible Skyway is a collection of 

WWW documents. This implementation will be 

added to and changed as the results of the User Needs 

Survey are analyzed. The following sections of this 

report detail the status of this WWW effort. The first 

section considers how we access the Internet, and the 

following sections consider the actual WWW implementation. 

5.3.1 Internet Service Providers 

When discussing services, it is often important to 

distinguish between providing the service and consuming the service. Computer terminology for this is 

client vs. server. It is typically easier to be the client 

of an Internet service than to be the server. For example, there are now many popular and inexpensive 

packages in any bookstore that allow a user to access 

the Internet (and become a client). For instance, it is 

relatively easy to setup an IBM PC (or a Mac) to access the many FTP and WWW information sources 

now on the Internet. The Skyway must be a server 

publishing WWW information. 

Several alternatives exist for the Skyway server. The 

server is where the Skyway information resides and 

is where the WWW and FTP protocols are impleSparcServer5 $15,351 

ISDN Setup $250 

ISDN $95 

Dedicated TCP/IP link $375 

45 

Information Skyway 

mented. One approach is to employ an Internet 

Service Provider's (ISP) UNIX machine and a SLIP 

connection. The ISP's machine maintaining the actual data storage is continuously connected to the Internet. Galaxy Scientific corporation connects to the 

ISP's machine as needed over a low-speed modem 

and uploads our information. This method is the most 

cost-effective for small scale prototyping, but offers 

the least control and poor cost-per-bit for larger scale 

data storage. Another approach is to establish an onpremises host which provides all data storage and 

server implementation. This approach-requires more 

extensive set-up and hardware. 

We are now using an off-site ISP host. Specifically, 

an Atlanta-based ISP named MindSpring, Inc., provides us with disk storage, FfP, and WWW server 

access, and a SLIP account for approximately 

$50/month plus $1/Mbytes/Month storage fee. 

We have investigated establishing an on-site host. 

Some cost estimates for doing so are shown in Table 

5.1. 

With our off-site ISP, our responsibility included 

authoring and uploading our HTML documents. With 

an on-site host, we would be responsible also for installing and maintaining the service, specifically for 

managing a WWW and FTP server. 

5.3.2 Tbe Skyway, WWW, HTML, and HTML 

Authoring 

Initially, we implemented parts of the Human Factors Guide on the World-Wide Web. WWW provides 

adequate sup}'ort for the text and graphics in this 

document. Future FARs, reports, etc., may also be 

published in WWW format. 

Internet users work with Universal Resources Locator (URLs) when navigating on the net. URLs function as precise addresses by which Internet resources 

are located. It has become increasingly common for 

onetime includes storage and 

software 

onetime high-speed communications 

monthly dedicated line cost 

monthly link to the Internet 
Information Skyway 

organizations to include a central WWW URL along 

with their standard business address. The current 

Skyway URL is: 

I http://www.mindspring.com/-galaxy/skyway.btml I 

One significant advantage of the WWW is its widespread availability. Web browsers are available for 

most common platforms. The popular MOSAIC 

viewer, for example, is available forMS-Windows, 

for the Macintosh, and for UNIX platforms. 

Authoring the HFG WWW version (and WWW information in general) requires utilizing the H1ML 

Ch!'Pter 5 

format. HTML is a dialect of SGML; a much larger 

specification. HTML is a simple text-based markup 

language like LaTeX or TROFF. Much HTML 

markup work is done manually. While this method 

works fine for typical 'pages,' larger document databases, such as the Skyway, require a more sophisticated and scaleable approach. Since Galaxy primarily 

utilizes Microsoft Word 6.0 for desktop publishing, 

we investigated tools that directly convert Word to 

HTML. CU_H1ML is one such tool; it meshes well 

with Word 6.0. CU_HTML uses Word 6.0 templates 

and macros to transform Word 6.0 documents automatically into the HTML format. This approach is 

depicted in Figure S. 3. 

WORD 6.0 t---r-~--

WORDI.ODOC 

c 

....... 

---!I ....., 

CU_HTML 

HTMLADD-ON 

Figure 5.3 Current Skyway Implementation 

46 

WORDI.ODOC 

HTIILDOC 

TCP/IPa 

SUP~• 

• 
Chapter 5 

Currently, the Skyway consists of an introductory 

Skyway VvWW page which can be reached from any 

Internet Web browser using the URL: 

I http://www.nlindspring.com/-gaiaxy/skyway.html I 

Figure 5.4 shows this page viewed from MOSAIC 

running on MS-Windows. 

There are two hypertext links. One link takes the user 

to the Galaxy Scientific homepage; the other, to the 

Human Factors Guide. Figure 5.5 (next page) shows 

47 

Information SKyway 

the MOSAIC page introducing the Human Factors 

Guide. 

Only Chapter I is present now. The text of Chapter I 

is broken into several subpages for general hypertext 

organization and to minimize the amount of time a 

user must wait while information is being downloaded. In addition to the text, chapter figures and 

tables can be found. For example, Figure 5.6 (next 

page) shows the MOSAIC page containing one particular graphic. 
• 

Information Skyway Chapter 5 

48 
Chapter 5 

Encoding Chapter I of the Human Factors Guide has 

shown that the WWW is a viable medium for disseminating information. While many existing WWW 

pages are quite small, our effort explores issues associated with largtr documents. Advantages of 

WWW publishing include world-wide immediate access, multi-platform support, and instantaneous updates. 

Information Skyway 

SA EXIST!:\(;.\ VL\TIO:\ .\:\D 

HL\1.\:\ F \CTORS 0:\-LI:\E 

RESOLRCI<S 

We surveyed existing public aviation- and human 

factors-related sources. While this survey is incomplete, if for no other reason than that these 

sources change continuously, the results provide a 

glimpse of the existing electronic landscape and indicate the existing demand in this area. The first area 

we explored was dial-up Bulletin Board Systems 

(BBSs), as shown in Table 5.2. We then surveyed 

existing Aviation/Human Factors Internet-based 

services, as shown in Table 5.3 (next page). Finally, 

we surveyed Aviation/Human Factors CD-ROM databases, as shown in Table 5.4 (next page). 

Table 5.2 FAA Supported Publ1c Access On-L1ne BBSs - ~ ~ "'ry'- "~n=m,'~'$'Y{ 

; . . Airports BBS (202)267 -5205 

Air Traffic Operations Service BBS (202)267-5331 

(800)446-2777 

Air Transport Division BBS (202)267-5231 

Pilot Examiner BBS ( 405)954-4530 

(800)954-4530 

FAA Headquarters BBS (202)267-5697 

Office of Environment & Energy BBS (202)267-9647 

Navigation and Landing BBS (202)267-6547 

Aviation Rulemaking Advisory BBS (202)267 -5948 

Orlando FSDO BBS ( 407)648-6963 

( 407)648-6309 

(800)645-3736 

(800)645-FSDO 

Portland MMEL BBS (207) 780-3297 

Safety Data Exchange BBS (800)426-3814 

Aeromedical Forum BBS (202)366-7920 

Conte! Dual User Access Terminal System (DUATS) (800)767 -9989 

CompuServe Information Service (CIS) GO AVIATION 

49 
Information Skyway 

www 

www 

www 

www 

www 

www 

www 

www 

www 

Usenet 

Use net 

Usenet 

http://www.dtic.dla.mil/iac/cseriac/iac.html 

<unable to locate at this time> 

http://www. virtual-airli ne.co.uklvirtual/ OR 

http://www.demon.co.uk/virtuall 

http://www.sonic.net/aso/ 

http://www.CdnAir.CAI 

http://www.iconz.co.nz/airnz/ 

airnz.html 

http://www. wintemet.comi-tela/ 

nwa-info.html 

http://www.winternet.com/-tela/ 

nwa.html 

http://wwwiseanet.com/Bazar/ 

Aeroflotf Aeroflot. html 

rec.aviation ..... 

sci. aeronautics 

sci.aeronautics.airliners 

airline 

Chapter 5 

FAA Home 

CSERIAC Home Page 

Embry-Riddle Avion WWW Site 

"The Virtual Airline" 

"Knowledge and Resources for the Airline 

rnn11r~·rrv" 

Aircraft Shopper On-line 

Airline: Canadian Air 

Airline: Air New Zealand 

Airline: Northwest 

Northwest (travel survey) 

Airline: Aeroflot 

(hierarchy of many subgroups) 

Discussion for Aeronautics 

Airliner Technology 

Discussion Group for Airline Industry 

Table 5.4 Avratron/Human Factors CD-ROM Databases 

CD-ROM Summit Aviation 

CD-ROM ATP 

CD-ROM ACS 

CD-ROM CounterPoint Publishing 

5.5 SUI\1'\IARY AND CONCLCSIO'IS 

We need more analysis to determine Skyway requirements accurately. This is proceeding. 

Meanwhile, the WWW is proving to be a promising 

delivery vehicle for digital documentation. Purely as 

a hypermedia delivery system, it works well. Advanced WWW features and other Internet services 

promise innovative new ways to integrate and engage 

the Aviation and Human Factors community. 

50 

Database of FARs, ACs,ADs 

Database of FARS,JARs, SBs 

Database-of FARS,JARs, SBs 

Database of CFR and FR 

5.6 FUTl RE PLANS 

Once we have received and evaluated more User 

Needs Surveys and obtained a clearer picture of 

our user, we will discuss with the FAA how the Skyway should fit into the overall FAA information plan. 

Also, we are in the process of implementing the nextgeneration Skyway node, which will be much more 

powerful and flexible. 

Finally, we are planning the next set of Skyway 

services including archives, newsletters and more experimental services. 
Chapter 5 

5.7 REI· EIH"\t FS 

Nejmeh, B.A. (1994). Internet: A strategic tool for 

the software enterprise. Communications of the 

ACM, Vol. 37;No. II, pp. 23-27. 

Pitkow, J. and Recker, M. (1994). Results from the 

First World-Wide Web user survey. Computer 

Netw~ and ISDN Systems, 27, pp. 243-254. 

Stefanac, S. (1994). Multimedia Meets the Internet. 

NewMedia, Nov., pp. 56-63. 

Information Skyway 

51 
Chapter 5 - Appendix 

Draft of User Needs Suney 

The "Information Skyway" will be an electronic system for disseminating safety-related information from the 

Federal Aviation Administration (FAA). This system may also be used to distribute other types of FAA-produced 

information, such as regulations concerning commercial and general aviation. As the first step in producing this 

system, Galaxy Scientific Corporation is conducting a survey and designing a proof-of-concept prototype for the 

FAA. The survey and prototype will be used to determine the feasibility of hosting and maintaining an on-ramp to 

the Information Superhighway. 

Please help design the Information Skyway by filling out this survey. The data from this survey will be used to 

determine the form and content of an electronic information system being built by the FAA Office of Aviation 

Medicine. The information obtained from this survey is confidential, and you do not need to identify yourself. 

This survey is designed to be easy to fill out electronically; for multiple choice questions, replace the '_' character 

with an 'X'. For questions that require text, just type your answer after the question. 

After you have filled out this survey, please return it to Galaxy Scientific. E-Mail is preferred, but you can also 

return it via fax or regular mail. 

A TIN: Electronic Information Survey 

Galaxy Scientific Corp. 

2310 Parklake Drive NE, Suite 325 

Atlanta, GA 30345 

phone:404-491-1100 

fax: 404-491-0739 

email: galaxy@mindspring.com 

---------- Notice ------------

This information collection conforms to legal and administrative standards established by the Federal Government 

to assure confidential treatment of statistical information. The information you provide will be used only for 

statistical purposes and will not be published or released in any form that would reveal specific information reported 

by an individually identifiable respondent. This questionnaire has been approved by the Office of Management and 

Budget, and has been given OMB Approval Number 2120-0587. 

AGENCY DISPLAY OF ESTIMATED BURDEN: 

The public reporting burden for this collection of information is estimated to average five minutes per response. If 

you wish to comment on the accuracy of the estimate or make suggestions for reducing this burden, please direct 

your comments to OMB and the FAA at the following addresses: 

Office of Management and Budget 

Paperwork Reduction Project 

MS 2120-0587 

W asbington, DC 20503 

US Department of Transportation 

Federal Aviation Administration 

Offtce of Aviation Medicine AAM-240 

Washington, DC 20503 

52 

l 
Chapter5 Appendix - Draft of User Needs Survey 

------------------ Electronic Infonnation Survey --------------

A. INFORMATION NEEDS 

I. What types of FAA-produced aviation information do you currently use? (choose 

all that apply) 

FARs 

_ Airworthiness Directives 

_ Guidance materials (Advisory Circulars, etc.) 

_ Technical publications 

General Aviation Airworthiness Alerts 

_Other (please describe below) 

2 What FAA-produced information WOULD you use if given easy access? 

_Regulations (FARs, Airworthiness Directives, etc.) 

_Guidance materials (Advisory Circulars, etc.) 

_ Technical publications 

General Aviation Airworthiness Alerts 

Human factors information 

_Other (please describe below) 

3. What non-FAA safety-related aviation information do you currently use? 

(choose all that apply) 

_ Service Bulletins 

_ Government and Commercial Standards (please describe) 

_ Conference proceedings and magazines 

_ Informal discussions 

_Other (please describe below) 

4. Wbat non-FAA safery-related aviation information WOULD you use if given easy 

access? (choose all that apply) 

Service Bulletins 

_ Government and Commercial Standards (please describe) 

_ Conference proceedings and magazines 

Informal discussions 

_Other (please describe below) 

5. What computer data transfer and communications hardware do you have access 

to? 

_CD-ROM 

Modem 

-Internet 

6. Wbat type of computer(s) do you use? 

DOS without Windows 

DOS with Windows 

Macintosh 

UNIX 

_Mainframe 

-Other (please describe) 

7. What aviation-related electronic resources do you currently use? 

_ FAA bulletin boards 

53 
Appendix- Draft of User Needs Survey 

_Commercial on-line services (America On-line, CompuServe, etc.) 

_CD ROM-based Commercial Services (Aircraft Technical Publications, 

Aviation Compliance Services, Summit Aviation, etc.) 

_ Internet newsgro11ps and mailing lists 

_Other (please describe below) 

8. Do you take part in any electronic discussion groups related to aviation? 

_Yes 

No 

9. If (8) is No, would you take part in any aviation-related electronic 

discussion groups if you had access? 

_Yes 

_No 

10. If you are involved in General Aviation, what electronic information 

resources would you use? 

_ Flight training material 

_ Maintenance information 

_Aviation medicine 

_ Accident/incident reports 

_Other (please list below) 

I I. Would you use a computer to submit safety-related information if you had a 

computer and appropriate software? 

_Yes 

_No 

12. Do you use any of the following PC-based flight simulation software? 

_ Microsoft Flight Simulator 

_IFf-PRO 

_ AssureSoft 

_ FS-1 00 Desktop Cockpit 

_Other 

B. OTHER COMMENTS 

I. Describe what you would like to see in the Information Skyway. 

2. What do you like/dislike about existing aviation-related electronic information sources? 

ChapterS 

3. How would an electronic repository of safety-related aviation information affect your decision making? 

54 

.. 
ChapterS 

C. ABOUT YOURSELF (OPTIONAL) 

I. Your main job responsibility: 

Aviation maintenance 

Researcher 

Student 

Pilot 

_Document management 

- Regulatory 

_Management 

_Other (please describe below) 

2. Sector of your worl<: 

Part 121 airline 

Part 135 airline 

General aviation 

_Military 

_ Govemrnent (other than military) 

Academic 

_Other (please describe below) 

3. What is your most advanced pilot certificate? 

Student 

Recreational 

Private 

Commercial 

_ Airline Transport 

_ Certified Flight Instructor 

None 

4. Do you have an instrument rating? 

Yes 

No 

5. About bow many TOTAL flying hours do you have? 

Appendix- Draft of User Needs Survey 

6. Contact information (may be used to gather more information, but will not be 

disclosed or distriboted) 

Name: 

Address: 

City/State/Zip: 

Phone: 

Email: 

------------------------------------------------------------------

Thank you for your cooperation, please return this survey to Galaxy Scientific 

Corp. via fax or E-Mail. 

fax: 404-491-0739 

emaH: plaxy@mindspring.com 

55 
CHAPTER 6 

HUMAN FACTORS PROGRAM 

DEVELOPMENT AND IMPLEMENTATION 

Colin G. Drury. Ph.D., Caren Levine and Jacqueline L Reynolds 

State University of New York at Buffalo 

6.0 STl I>\ B \Ch:CROl \I> 

This project was initiated to provide a practical 

demonstration of human factors/ergonomics 

implementation in an airline maintenance 

organization and, hence, to give airlines guidance on 

implementing their own programs. Ergonomics, and 

its American synonym Human Factors, is "the science 

that facilitates maximum human productivity, 

consistent quality, and long-term worker health and 

safety" (Burke, 1992). Human factors measures the 

job demands imposed by the workplace, environment, 

and schedule. It then compares these with the 

workforce's capabilities to meet these demands 

consistently. Where task demands exceed human 

capabilities, performanc.; will break down, leading to 

human errors, which can' manifest as safetycompromising incidents and/or on-the-job injuries. A 

better (safer) match between task demands and human 

capabilities can be achieved by changing the task 

demands (workplace, environment, organization 

design), by changing human capabilities (training, 

placement), or by both. Whether the organization's 

initial motivation for the human factors program is 

public safety, improved productivity, or reduced 

injuries, the analysis is the same. Indeed, the same 

analysis can be used to specify system interventions, 

e.g., workplace changes, or personnel interventions, 

e.g., training. 

The motivation behind the current project arose 

specif•cally from human factors analyses conducted 

in 1993 on restrictive spaces in aircraft inspection 

tasks (see Reynolds and Drury, 1993). As part of that 

project, on-the-job injuries (OTis) analyzed were 

found to be space-related. Hence, when we sought a 

site for demonstrating human factors/ergonomics 

intervention, it was natural to choose inspectors and 

to consider on reduction, as well as performance 

improvements, i.e., error reduction. 

57 

6.1 Hl \I\\ I· H T<)){S "I \Sh: I·ORCt 

FOR\1 \'110\ 

The human factors program at Northwest Airlines 

was created with the mission "to redesign work 

environments to prevent on-the-job injuries." The 

program was initiated by the formation of the Human 

Factors Task Force made up of members of both 

management and the hangar workforce. The job titles 

of task force members included Safety Manager, lAM 

Safety Representative, Inspector, Lead Inspector, and 

Northwest Airlines Process Specialist (Training 

Department). Representatives from the University at 

Buffalo were assigned to act as task force advisors. 

The initial focus of the program was the inspection 

department at the Atlanta Maintenance Base. 

The inclusion of inspectors on this task force was 

critical to its potential for success. Inspectors have 

unparalleled expertise in their jobs and domain 

knowledge that leads to an understanding of what 

changes are most necessary and to what solutions may 

or may not work. Inspectors on the task force were 

encouraged to communicate with other inspectors and 

to act as spokespeople for their entire crew. 

Typically, inclusion of work force representatives in 

analysis and reilesign of their own jobs makes them 

more inclined to accept ergonomic solutions task 

force implements. This is because they actively 

contributed to the solution-development process. 

6.1.1 Task Force Objectives and Guidelines 

Burke (1992) emphasizes the benefits that can be 

obtained when a human factors task force addresses 

human factors issues within an organization. A team 

approach gives the organization maximum input from 

various people who will be affected by any changes. 

For a task group to be successful, its members must 

be comfortable working together and must fully 

understand the importance of their commitment and 

contribution to the task force. In recent years, 

Northwest Airlines has emphasized team activities. 

There are well-established procedures for teams to 
Human Factors Development and Implementation 

form, gain confidence, organize their activitiest and 

implement their findings. 

The initial objectives of the Northwest Airlines 

Human Factors Task Force were as follows: 

I. Develop a process for identifying and addressing 

ergonomics issues within the inspection 

department that could later be expanded to all 

Northwest Airlines departments 

2. Involve employees in the ergonomics process 

3. Reduce the number of Olls 

4. Develop ergonomic solutions that could be 

implemented, with results that could be measured 

5. Teach employees about ergonomics, so they 

could help widen the task force's focus 

6. Commit to transfer the technology and the 

processes this task force used to other areas at 

Northwest Airlines. 

The task force's guidelines were as follows: 

I. Focus on inspection jobs and tasks in the hangar 

area 

2. Identify the jobs and tasks to analyze 

3. Establish an action plan to effect short- and longterm improvements 

4. Members should commit to a one-year 

participation in the task force 

5. The group leader to be elected by the entire task 

force 

6. A task force member may work on this project up 

to I 00% of his or her time 

7. After its initial start-up meetings, the task force 

will establish its own agenda 

8. The group leader will communicate a weekly 

report to all task force members. 

6.1.2 Program Development 

The steps that the Northwest Airlines Human Factors 

Task Force took closely followed the seven general 

steps in an ergonomic process, as described by Burke 

(1992). 

I. Determine the measurement criteria and target 

the jobs to be studied. 

• Determine which areas should be 

targeted for analysis and intervention. 

• Choose the specific criteria which will 

help determine target areas, e.g., injury 

rate. 

2. Gather job background information. 

• Document the job to be analyzed, 

including the job description, the tools 

58 

Chapter6 

necessary to perform the job, physical 

dimensions of the workspace, etc. 

3. Identify ergonomic risk factors. 

• Identify conditions likely to act as 

barriers to optimal productivity and 

consistent quality and/or that have been 

associated with a high incidence of 

injuries. 

4. Discover ergonomic interventions. 

• Brainstorm about all possible 

interventions to each risk factor, 

considering the following: 

- changing inputs/materials 

- changing output/product 

-changing 

machine/environment 

- changing procedures dealing 

with workers, e.g., 

training. 

5. Screen interventions. 

• Choose interventions to implement 

based on decision criteria such as cost, 

benefits, utility, consequences of no 

action, injury rate, etc. 

6. hnplement interventions. 

• Orient those affected about why the 

intervention was chosen, what its 

expected impact is, and who to contact 

with questions/comments/concerns. 

7. Track the effectiveness of the interventions. 

• Assess each intervention's effectiveness 

and decide whether to expand, amend, 

alter, or abandon the particular 

intervention. 

Once the human "ractors task force was selected, it 

was necessary to educate its members about what 

human factors is and how human factors can be used 

to improve the workplace. The University at Buffalo 

conducted a one-day training seminar, using materials 

developed from previous FAN AAM projects and 

ICAO's SHEL model of human factors. The training 

specifically built on the University at Buffalo's 

previous involvement with Northwest Airlines' 

Atlanta Maintenance Base and its inspection 

activities. 

The task force selected jobs to be analyzed in the first 

phase of the human factors program. The following 

jobs were identified by inspectors as five of the 

longest, most-difficult inspection tasks: Electrical and 

Equipment Compartment Inspection (E&E 

Compartment), Keel Inspection, Fuel Tank 

Inspection, Combustion Chamber Inspection (PS4 
~ 

i 

Chapter6 

Drain Box), and Nose and Forward Accessory 

Compartment Inspection (Forward Access 

Compartment). 

Four of the jobs were analyzed using the electronic 

inspection audit program the University at Buffalo 

developed (see Koli and Drury, 1995}. Inspectors on 

the task force conducted the audits. The audit results 

for the keel inspection are provided in Appendix 6-A 

as an example of their work. To progress from 

analysis to redesign for each of these audits, a list of 

ergonomic risk factors was identified. A few risk 

factors from all four tasks were combined into 

problems with workcards and problems with lighting. 

The list of ergonomic risk factors for each area is 

included in Appendix 6-B. The nominal group 

technique was utilized to rank each risk factor for 

each of the six main areas, for four specific tasks 

(E&E compartment, forward accessory compartment, 

keel, and PS4 Drain Box), and for two general areas 

( workcards and lighting). 

As follows, the three risk factors with the highest 

rankings were chosen for closer study in each of the 

six areas: 

• Workcards 

• Card content inaccurate 

• Breaks between cards inappropriate 

• Card contrast varied 

• Lighting 

• Fixtures dirty 

• Lighting inadequate at the back of the 

hangar 

• No preventive maintenance program for 

lighting 

• Keel Inspection 

• Body positioning 

• Cleaning 

• Lighting 

• PS4 Drain Box Inspection 

• Body positioning 

• NOT equipment 

• Cleaning 

• E&E Compartment Inspection 

• Lighting 

• Temperature 

• Equipment 

• Forward Access Compartment Inspection 

• Ladder design 

• Ladder control 

• Work planning 

59 

Human Factors Development and Implementation 

From this list, task force members took responsibility 

for pursuing specific potential solutions in the 

following areas: 

• Improved cleaning 

• Ladder purchase and control 

• Workcard design 

• Improved task lighting. 

An action plan provided a time line for these 

activities that ensured analysis, implementation, and 

measurement of results within the time frame of this 

FAA/AAM project. 

6.1.3 Redirection of the Ergonomics Task Force 

Initially, the Task Force followed-up on members' 

assignments to track progress according to the action 

plan. However, it became apparent that the Task 

Force as a whole was not progressing on developing 

solutions, as agreed. The r~searchers met with the 

task force and management to learn the reasons for 

the lack of progress and to help develop alternative 

strategies. 

A number of factors that had not prevented progress 

in team formation, job analysis, and solution 

generation surfaced when it was time for 

implementation. 

I. The workforce members of the task force felt that 

they had no mandate to pursue their assignments 

as part of their busy schedules. 

2. Some of the solutions had, or appeared to have 

had, implications beyond the Task Force's 

control. For example, workcard design is a 

headquarters function, not easily controlled or 

changed at a remote base. 

3. Other solutions required expenditure, e.g., task 

lighting, which was not immediately seen as 

available in the current fiscal climate. 

4. Perhaps most importantly, although task force 

members were opinion leaders within their 

groups, and a seuior management person acted as 

"champion" of the effort, neither management 

nor the workforce felt a groundswell of support 

for the Task Force's activities. 

For these reasons, the task force was disbanded, and 

the ergonomics efforts were refocused on a different 

problem that could have broad-based support and be 

entirely under control at the maintenance base. 
Human Factors Development and Implementation 

Specifically, many task force members recognized 

communication between shifts as one area in need of 

improvement. Also, communication between shifts 

needed no 'outside' assistance to implement a 

solution. Instead of having task force members 

implement the ergonomics audit program, which 

worked very well to identify human factors problems, 

a broad-based instrument was designed to obtain 

input about Communication issues from inspectors on 

all three shifts. We reasoned that such input would 

produce buy-in to potential solutions, thus easing 

implementation. Of course, broader participation 

meant that expectations would be raised for more 

people, forcing at least some implementation if 

management/workforce trust was to be preserved. 

Fortunately, improving communication of technical 

information between participants has a good history 

in human factors generally, and in aircraft 

maintenance specifically (see Taylor, 1992). 

6.2 C0\1\Il 1\IC \TIO:'I:S \T TilE 

\TL \:\T\ \I \1:-.iTE"\ \'\CI'. B \SE 

As in any industry, effective communication 

within an organizational unit, and among 

organizational units, is critical for maintaining 

productivity in airline inspection and maintenance. 

Taylor (1992) writes, "Effective communication is no 

longer limited to merely acquiring the information 

that an individual needs to make decisions. 

Communication is increasingly a systems issue-it is 

inextricably bound to cooperation, coordination, and 

otherwise working together in a joint task or job for 

which individuals cannot succeed by working 

separately." Airline inspectors often help drive the 

heavy maintenance of aircraft. They are the first to 

look over an aircraft and have the task of identifying 

all the problems with it. Inspectors decide which 

problems maintenance must fix before an aircraft can 

leave the hangar, as well as which problems can be 

delayed until the next maintenance check. After the 

maintenance work is performed, inspectors must 

ensure that it was done properly. An aircraft cannot 

leave the hangar until all work is signed off by the 

appropriate authority, usually the inspectors. An 

inspector must be able to share information with 

management and other employees so that everyone 

understands an aircraft's current status. At Northwest 

Airlines' Atlanta Maintenance Base, for example, an 

inspector may find it necessary to communicate with 

the following people: 

• other inspectors on the same shift 

• inspectors on the two other shifts 

60 

Chapter6 

• mechanics 

• the lead inspector 

• the inspection manager 

• the maintenance manager 

• engineers 

• other management 

• the flight crew. 

The inspector must have the communication tools and 

skills to share information with other members of the 

organization, as necessary. Although communication 

is an important aspect of aircraft maintenance, it fails 

at times. To understand possible failure modes, a 

national source of error data (Aviation Safety 

Reporting System, or ASRS) was analyzed 

specifically to identify communication errors in the 

maintenance environment. 

6.2.1 Typical Airline Industry Communication 

Problems (ASRS Reports) 

Fortunately, human errors in aircraft maintenance are 

rare. Since errors are unlikely to be observed during a 

study such as ours, possible errors must be inferred 

from other sources. A review of NASA's ASRS 

mechanic reports identified that serious consequences 

can occur when inspectors and mechanics are unable 

to communicate efficiently with their co-workers. It is 

important to remember that ASRS reports are 

reported by individuals on a voluntary basis. In many 

cases, the reports have not been corroborated by the 

FAA or NTSB, and the data cannot be used to infer 

the prevalence of a particular problem within the 

national aviation system. The incidents discussed here 

occurred over many years (January, 1987-February, 

1994) at many airlines. They are not Northwest 

Airlines incidents. 

Some common communication problems present 

themselves upon a close review of the ASRS reports. 

First, many incidents are caused by mechanics 

becoming distracted in the middle of performing a 

task." Mechanics often do not write down what they 

have accomplished, or what parts of a task need to be 

completed. At times, a mechanic may have to allow 

someone elsdo ftnish·'a task. This J!!!IY lead to 

dif'ficulties when the second mechruiic does not 

clearly understand the situation or does not realize 

specifically what remains to be do11e. Other times, a 

mechanic intends to come back and finish a task but 

forgets that the task was not completed. This could 

lead to serious problems if the uncompleted task is 

not detected before the plane takes off. This type of 

problem may also occur at shift changes, when 

mechanics cannot finish a task before their shift ends. 

' 
Chapter6 

The next shift assigned to finish the work may not ,. 

clearly understand where the previous shift left off. " 

This may result in duplication of effort on some tasl<s 

or, more seriou~ly, the omission of some tasks .· 

completely, e.g., the second shift assumes that the , 

previouS shift has performed a ce~ task and d0$0S 

not verify this to be the COjle. ·• 

• I was assigned to aircraft work 

release items ... .! was in the process 

of reinstalling the plug and covers 

for the turbine section wben another 

mechanic asked if I needed any help. 

I asked him if be would install the 

ignitors. I saw him install the 

outboard ignitor. Then he went 

under the engine to what I thought 

was to install the inboard ignitor. 

While be was under the engine, I 

saw him install the screen back on 

the starter, but I did not go back and 

check his work, because I trust the 

work he does. I am the one who 

signed off the block on the 

paperwork .... lbe inboard ignitor was 

never installed. (ACN #250135) 

Another mechanic was assigned the 

open and close of the engine. He 

opened all plug panels and ignitors. I 

stopped to help him close the engine. 

I installed the outboard ignitor and 

installed the starter air deflector, 

only per maintenance manual 72. 

The inboard ignitor was never 

installed. I did not know [if) the 

inboard ignitor was left out, or [was 

even) out at all. (ACN #250330) 

Another problem, somewhat related to the problem 

described above, is that generally one mechanic must 

sign off on the completion of a task, although more 

than one person may have actually worked on the 

task. Thus, it is difficult to pinpoint who actually 

completed the work when a problem arises. The 

mechanics who assisted may later forget, or deny, that 

they participated in completing the task in question. 

Oil was serviced to full by another 

mechanic. However, he was 

reassigned to another aircraft before 

completing the log entry. At 

departure time, I completed [the) 

maintenance sign off in [the) 

logbook. The oil tank cap apparently 

61 

Human Factors Development and Implementation 

was not latched in the closed 

position. (ACN #245568) 

Maintenance inspected [the) aircraft 

and found all six securing screws 

missing from left-hand most 

outboard wing trailing edge 

panel....Further investigation shows 

several individuals were involved in 

the close-up of the aircraft at 

completion of the check, but no one 

person assumes responsibility or full 

knowledge of this one particular 

panel. However, there is a signature 

of a supervisor who specifically 

signed stating. "All panels were 

secured." (ACN #101899) 

A third problem occurs when mechanics are given 

incorrect verbal descriptions of discrepancies or 

descriptions varying from the written description in 

the log book. Similarly, a mechanic can be assigned 

to perform a task without receiving all the correct 

paperwork which accompanies the task. This can lead 

to the mechanic making an incorrect diagnosis of the 

problem and, consequently, taking incorrect action to 

correct the problem. In some cases, inaccurate 

diagnosis led a mechanic incorrectly to defer 

maintenance that should have been completed 

immediately. 

I was told [verbally) that the roll 

spoiler outboard ground caution 

light was illuminated. I sent an A&P 

down to check [it] out and defer the 

systelJl. He was unable to duplicate 

any problem, but we, by phone 

conversation, decided to defer the 

system in case the pilots had a 

problem on the morning departure. 

[Later,] wben reviewing the 

logbook, I discovered I had been 

given wrong information from 

maintenance control about which 

light had illuminated. The roll 

spoiler outboard hydraulic light was 

the light that actually was written up, 

and this would not be something you 

would defer. (ACN #243444) 

These problems emphasize the importance of written 

communication in the airline industry. Verbal 

communicatioQp although often more convenient, is 

mo~-prone, especially when information must 

be remembered for long periods of time or must be 
Chapter6 

The next shift assigned to finish the work may not , 

clearly understand where the previous shift left off. .,. 

This may result in duplication of effort on some tasks 

or, more seriously, the omission of some tasks ~· 

completely, e.g., the second shift assumes that the , 

previous shift has performed a cert;!in task and d~s 

not verify this to be the ~· 

l was assigned to aircraft work 

release items ... .I was in the process 

of reinstalling the plug and covers 

for the turbine section when another 

mechanic asked ifl needed any help. 

I asked him if be would install the 

ignitors. I saw him install the 

outhoard ignitor. Then be went 

under the engine to what I thought 

was to install the inboard ignitor. 

While be was under the engine, I 

saw him install the screen back on 

the starter, but I did not go back and 

check his work, because I trust the 

work be does. I am the one who 

signed off the block on the 

paperwork. ... The inboard ignitor was 

never installed. (ACN #250135) 

Another mechanic was assigned the 

open and close of the engine. He 

opened all plug panels and ignitors. I 

stopped to help him close the engine. 

I installed the outhoard ignitor and 

installed the starter air deflector, 

only per maintenance manual 72. 

The inboard ignitor was never 

installed. I did not know [if] the 

inboard ignitor was left out, or [was 

even] out at all. (ACN #250330) 

Another problem, somewhat related to the problem 

described above, is that generally one mechanic must 

sign off on the completion of a task, although more 

than one person may have actually worked on the 

task. Thus, it is difficult to pinpoint who actually 

completed the work when a problem arises. The 

mechanics who assisted may later forget, or deny, that 

they participated in completing the task in question. 

Oil was serviced to full by another 

mechanic. However, he was 

reassigned to another aircraft before 

completing the log entry. At 

departure time, I completed [the] 

maintenance sign off in [the] 

logbook. The oil tank cap apparently 

61 

HumJlll Factors Development and Implementation 

was not latched in the closed 

position. (ACN #245568) 

Maintenance inspected [the] aircraft 

and found all six securing screws 

missing from left-band most 

outhoard wing trailing edge 

panei ... .Furtber investigation shows 

several individuals were involved in 

the close-up of the aircraft at 

completion of the check, but no one 

person assumes responsibility or full 

knowledge of this one particular 

panel. However, there is a signature 

of a supervisor wbo specifically 

signed stating, "All panels were 

secured." (ACN #101899) 

A third problem occurs when mechanics are given 

incorrect verbal descriptions of discrepancies or 

descriptions varying from the written description in 

the log book. Similarly, a mechanic can be assigned 

to perform a task without receiving all the correct 

paperwork which accompanies the task. This can lead 

to the mechanic making an incorrect diagnosis of the 

problem and, consequently, taking incorrect action to 

correct the problem. In some cases, inaccurate 

diagnosis led a mechanic incorrectly to defer 

maintenance that should have been completed 

immediately. 

I was told [velbally] that the roll 

spoiler outhoard ground caution 

light was illuminated. I sent an A&P 

down to check [it] out and defer the 

system., He was unable to duplicate 

any problem, but we, by phone 

conversation, decided to defer the 

system in case the pilots had a 

problem on the morning departure. 

[Later,] wben reviewing the 

logbook, I discovered I had been 

given wrong information from 

maintenance control about which 

light had illuminated. The roll 

spoiler outhoard hydraulic light was 

the light that actoally was written up, 

and this would not be something you 

would defer. (ACN #243444) 

These problems emphasize the importance of written 

communication in the airline industry. Verbal 

communicatioq, although often more convenient, is 

motYetibi•prone, especially when information must 

be remembered for long periods of time or must be 
Human Factors Development and Implementation 

passed sequentially through a number of people. The 

"telephone" game provides a good example of this 

problem: as information is passed from one person to 

another, the message tends to become increasingly 

confused. Written communication can serve as a 

permanent record of events and is less subject to the 

frailties of human memory. However, since written 

records may be used as an investigative tool to prove 

the actions a maintenance crew took, workers may 

feel, "It gives them something to hang you with!" 

There is understandable reluctance in all branches of 

the airline industry to write anything not specifically 

required to be committed to paper. 

6,2.1.1 Summary of Communication Failures 

Table 6.1 presents the types of communication 

failures contributing to the incidents reported in the 

ASRS database. The data in this table are 

representative only of the twenty-eight ASRS reports 

we analyzed. 

Type of Failure: 

• F = failure to communicate 

• V = verbal communication wrong/inadequate 

• W = written communication wrong/inadequate 

• M = memory failure (forgot to do something) 

(See Table 6.1) 

6.2.2 Identification of Communication Problems 

Within The Inspection Department 

Table 6.1 shows that certain failure types are 

associated with different communication needs. 

While ASRS data is not a statistically valid random 

sample of errors, it can be used to identify forms of 

failure. 

Obviously, a mechanic communicating with himself 

or herself at a later time can have a memory failure 

(M). When this happens, the mechanic usually relied 

on memory rather than a written note or a job aid, 

62 

Chapter 6 

such as a checklist, that would have prevented 

memory failure. Mechanics communicating with 

flight crew are subject to failures of both written (W) 

and verbal communication (V). Communication 

problems in the opposite direction, i.e., from flight 

crew to maintenance, are either failures to 

communicate at all (F) or a breakdown of the written 

process (W). Perhaps this results from the widely 

different background training of Flight Operations 

and Technical Operations and the lack of 

opportunities for verbal communication between 

these groups. Clearly, methods of improving 

communications between these groups are needed, 

e.g., extensions of CRM and MRM to joint training. 

Communication problems between mechanics, and 

between mechanics and supervisors, are all either 

failure to communicate at all (F) or a failure of verbal 

communication (V). This also includes shift change 

communication in the final column of Table 6.1. 

Clearly, written communication does not fail; if 

people use written communication, then this is 

adequate. The main emphasis for addressing these 

problems should be ensuring that mechanics and 

supervisors use written communication. Thus, the 

new focus of this project became redesigning 

communication forms so mechanics and supervisors 

can use them more easily. 

Since communication is critical to the successful 

performance of airline inspectors, we decided to 

examine the communication system for inspectors 

currently in place at the Atlanta Maintenance Base to 

see if improvements could be made. We expected that 

an inspector's (or a mechanic's, or a supervisor's) 

effectiveness can b<; improved by providing better 

communication tools that make it easier to collect 

necessary information and to pass that information to 

other supervisors and mechanics. 

After interviewing many inspectors, it was obvious 

that each inspector ~e_\Vs the job (and the larger 
r 

Chapter6 

system) differently. The shift on which the inspector 

usually works (and thus the inspector's lead 

inspector), as well as years experience as an 

inspector, are just two factors that appear to affect 

each inspector's perceptions. Due to such wide 

variations among inspectors, we decided to question 

all inspectors to gain a broad view of the actual 

communication system in the inspection department 

The user needs analysis was designed to identify tools 

currently supporting communication within the 

inspection department and between inspectors and 

other departments. The user needs analysis we used is 

included as Appendix 6-C. As a follow-up to the 

communication user needs analysis, we conducted 

further personal interviews with many inspectors. 

These interviews did not follow any pre-defined 

format; their purpose was simply to allow inspectors 

to talk about communication issues at Northwest 

Airlines and to provide background information to 

help interpret the user needs analysis responses. 

A particular focus of the communication user needs 

analysis was the shift turnover log>'Curtently, the shift 

turnover log is a bound book with numbered pages. 

Entries are made in the log each day, usually by the 

lead inspectors. Information in the log includes 

personnel issues, e.g., who called in sick, who left 

early, who is working overtime, etc., and aircraft 

issues, which are usually only a quick summary of 

each aircraft's status, e.g., in buy-back, shakedown, 

etc. An entry occasionally includes a description of a 

problem an inspector encountered during the shift. It 

is difficult to identify who made an entry in the log, 

and few entries are ever followed-up with another 

entry describing how the problem was resolved. The 

existing shift turnover log does not serve as a 

communication tool, showing the tasks with multishift implications, nor does it provide the information 

necessary for subsequent shifts to "pick up" where a 

previous shift left off. Thus, our communication user 

needs analysis was designed to identify whether 

inspectors use the existing shift turnover log as a 

helpful source of information and/or whether a 

different type of log would better serve inspectors' 

needs. 

6.2.3 Results from the Communication User Needs 

Analysis 

We received 17 responses to our user needs analysis 

from the approximately 30 inspectors at the 

Maintenance Base. User needs analysis responses are 

summarized in Table 6.2 (next page). 

63 

Human Factors Development and Implementation 

User needs analysis responses identified a general 

problem with inspectors' job satisfaction. Many 

inspectors report having difficulty obtaining 

information they need to perform the job. They are 

unwilling to share information with others, unless it is 

absolutely necessary. This reluctance to communicate 

is a serious problem and must be addresoed if 

inspection productivity is to be improved. The 

inspectors also identified shortcomings in the 

communication system at Northwest Inspectors do 

not use the shift turnover log regularly, almost always 

need to search for more information after being 

assigned a job, have experienced on-the-job problems 

caused by miscommunication, and deal with each 

other almost always verbally. The shift turnover log is 

seen as a managerial tool, not as a way to 

communicate. 

It is important to note that the average years of 

experience of inspectors responding to the user needs 

analysis is 6.6, with a standard deviation of 3.6. 

Previous studies have indicated that it is common in 

the aircraft industry to have mechanics with long 

service and with very short service, with very few in 

the middle (Taylor, 1990). At the Atlanta 

Maintenance Base, the less-experienced inspectors 

tended to return completed user needs analysis (over 

half had only 3-5 years experience); our results reflect 

their particular dissatisfaction with the current 

communication system. This result is not altogether 

unexpected. Experience_ as an inspector often means 

increased knowledge, information, and familiarity. 

Less-experienced inspectors may require more 

external information to perform a task (they cannot so 

easily rely on internal knowledge) than more 

experienced inspe~tors. Less-experienced inspectors 

also may be less able to respond to verbal instructions 

and information. Therefore, they may be less satisfied 

with, and more able to recognize problems in, current 

modes of communication. Experienced inspectors are 

accustomed to the way things are done and may be 

reluctant change. Our results may reflect a 

communication system designed to meet the needs of 

experienced inspectors, and of those with managerial 

responsibilities, while de-emphasizing the increased 

information demands of those with less experience. 

User needs analysis responses also indicate that many 

inspectors perceive a lack of what is termed 

situational awareness in human factors; they do not 

understand how their specific tasks lit into the larger 

picture of airline maintenance. Inspectors may be 

unaware of what is happening beyond their own work 

assignments and of how their assignments affect (and 

are affected by) other departments. For example, jobs 
r I 

I 

v 

Human Factors Development and Implementation 

Destinations of Information 

Do You Read Shift Turnover Log? How Often? 

Purpose of Shift Turnover Log? 

Attendance at Regular Crew Meetings? 

are often assigned to inspectors in what they perceive 

as a random manner, e.g., large jobs may be assigned 

only early in a shift, more difficult jobs may be 

delayed until easier ones are completed, etc. Many 

times, there seems to be little consideration of how 

job scheduling affects the maintenance department. 

Chapter 6 

(inspection, 

No- 9 (53%) 

Yes, When Acting Lead- 5 (29%) 

lead turnover information, personnel notes, status 

of aircraft, communication between shifts, written 

I 

No Time Needed- 4 "'"'-"~•• 

Time Is 

No- 7 (41%) 

Yes- 8 (47%) 

Used- 1 

6.2.4 Results From Personal Interviews with 

Inspectors 

64 

During site visits to the Atlanta Maintenance Base, 

we spoke personally with many inspectors about 

communication at Northwest Airlines. These 

conversations generally support the results from the 
Chapter6 

user needs analysis, although they provide more 

insight into inspectors' specific communication needs. 

Some points inspectors made in these conversations 

include the following. 

~~Inspectors acknowledge that they almost always 

communicate verbally with their lead inspector and 

with other Northwest employees. Most inspectors had 

never really considered the consequences if, at some 

later time', there was a problem with an inspection 

they conducted. Although workcards and non-routine 

cards provide a written account of the completed 

tasks, there is important, not legally required 

information that is never permanently recorded. 

Without written records, it is impossible to remember 

exactly what occurred and what steps had been taken. 

Even if an inspector did everything correctly, there 

would be no way to prove this in an investigation. 

The following incident is taken from the ASRS 

database: this is not data collected at Northwest 

Airlines. It illustrates the potential danger in failing to 

maintain accurate written records of all maintenance 

activities. 

A 'visiting' mechanic was assigned 

to repair an' engine. While 

performing the work, he accidentally 

dropped a rag into the gearbox 

cavity. After searching, 

unsuccessfully, for the rag, the 

mechanic notified (verbally) the lead 

mechanic of the problem. The lead 

mechanic ordered a horoscope of the 

engine, which did not show that the 

rag was inside. Although the 

mechanic continued to say that the 

rag was still inside the engine, the 

lead mechanic ordered that the repair 

be completed so that the plane could 

be released for a flight. The 

mechanic was sent home before the 

leak check on the engine was 

completed. On its initial flight, the 

plane was forced to tum back to the 

originating aiiport due to a low oil 

pressure warning. The engine was 

removed for further repair. During 

the investigation, the rag was found 

to have clogged the scavenge pump 

filter screen. The mechanic was 

interviewed twice by airline quality 

assurance, and the incident was 

written up in a report submitted to 

the FAA. (ACN #233249) 

65 

Human Factors Development and Implementation 

From an analysis of this incident it is clear that if the 

mechanic had made a written entry in the maintenance 

log concerning this incident, there would have been 

little question that his actions were totally appropriate. 

He could have recorded that he dropped the rag inside 

the engine and was unable to locate it. The lead 

mechanic was informed of the incident and eventually 

decided on his own that the rag was no longer inside 

the engine because his search had not located the rag. 

Without the written log, it is difficult to determine the 

actual events surrounding this incident. The lead 

mechanic could insist that the mechanic was unsure if 

the rag actually was inside the engine or that he was 

never informed of the problem, especially since the 

mechanic signed off on the repair. Alternatively, if the 

problem had not manifest immediately, the mechanics 

involved in this incident may then )lave been unable to 

provide accurate information to the quality assurance 

people investigating the incident. 

~e weekday day shift and early part of the 

weekday afternoon shift currently have far better 

information resources available. During weekdays 

(Monday thrnugh Friday, 8:00a.m.- 5:00p.m.), each 

department in the organization is fully staffed. 

Management, engineers, planners, and the most 

experienced inspectors are all readily available for 

consultation. During the second half of the afternoon 

shift, on the night shift, and on weekends, it is 

difficult and time-consuming to get information from 

these resources. For example, an inspector on the 

weekend shift must call an engineer at home for 

consultation on a technical problem. The engineer, if 

he or she happens to be at home, generally first tries 

to solve the problem over the telephone or, if 

appropriate, to Ifostpone addressing the problem until 

the next weekday shift. The engineer may be required 

to come into the hangar in an emergency, but this is 

generally the last resort. 

,~: Inspectors receive most of their information, 

including work assignments and any important items 

from the previous shift, from their shift lead. 

Therefore, they receive only information that the shift 

lead chooses or remembers to pass along. For 

example, an entry in the ASRS Database (ACN 

#196273) describes the following incident, which 

illustrates potential danger in filtering critical 

information thrnugh the lead inspectors. 

Several mechanics noted [that the) 

#I engine [was) making a loud 

unfamiliar noise. This information 

was passed on to the lead and 
Human Factors Development and Implementation Chapter6 

supervisory personnel by second 

shift mechanics so as to alert third 

shift mechanics who were to work 

the aircraft that night and early 

morning. I, the third shift mechanic, 

was assigned to work this particular 

aircraft. However, I received no 

information concerning this 

particular loud engine noise until 

about ? am that morning, and then it 

was passed on to me by another 

mechanic, not [by] the lead man who 

assigned me to work on the aircraft. 

Based upon the information that was 

made available to me, a pilot writeup [of an] indication problem, [I] 

replaced [the] #I engine tac 

indicator .... Had I been informed 

about the true condition of the 

engine, I would have treated the 

write-up quite differently. 

user needs analysis results, we made the following 

recommendations. 

I. It is important to train inspectors how to 

communicate. Inspectors must learn what is 

expected, so they understand what information 

must be communicated and why it is important. 

Inspectors should be trained in both verbal and 

written communication skills. Training also helps 

standardize communication so every inspector is 

able to pass and receive useful information. 

2. Inspectors must be challenged to understand the 

importance of good communication. They must 

understand benefits that are to be gained by 

improving communication. Any new 

communication procedures must not add to 

inspectors' workload or be at all difficult for 

them to use. 

4lupdates to maintenance manuals usually have a cover letter 

that each inspector must sign off. These documents are 

maintained in a notebook kept in the inspection office. 

Inspectors are expected to check the book daily and to read 

and sign off any new entries. This is easy when the workload is 

light and when there are few updates. However, when 

inspectors are busy or when there are a lot of updates, many 

inspectors fall behind. No supervisor or lead inspector ever 

seems to question inspectors about failing to keep current with 

the updates. An inspector may learn about updates only when 

they happen to relate to a particular problem he or she is 

addressing. 

S. Inspectors receive much information, from updates 

and elsewhere, that they see as irrelevant to their 

current responsibilities. For example, they often 

receive service alerts for DC-lOs and Boeing 727s; 

only DC-9 maintenance is performed at the Atlanta 

Maintenance Base. Inspectors feel overloaded with 

information and are concerned that they are not 

always able to filter out relevant DC-9 information. 

6.2.5 Results from Conversations with 

Management 

We also met with managers connected with the 

inspection department to discuss their perceptions of 

the communication system at Northwest Airlines. 

Many managers had never recognized that 

communication problems existed, although our user 

needs analysis results helped convince them that there 

was room for improvement. From discussion of the 

66 

3. Communication tools must be developed for shift 

turnovers, for passing general information such 

as management memos and aircraft alerts, for 

recording detailed problems and follow-ups, as 

necessary, etc. The medium of communication, 

e.g., logbook, verbal, blackboard, etc., must be 

chosen that best meets different communication 

needs. It is important to provide only the 

information inspectors need and not to overload 

them with unnecessary information. Information 

should be presented in a form that is easy to use 

and that allows inspectors easily to elicit specific 

details, as necessary. 

4. New communication tools must meet the needs 

and the expectations of all involved with the 

inspection department, including managers, 

leads, and inspectors. These individuals need to 

have input into redesigning the communication 

system. 

5. At the Atlanta Maintenance Base, there are three 

distinct inspection groups: support shops, engine 

shops, and major maintenance. The 

communication system, especially the shift 

turnover log, should be standardized for all these 

groups. Such standardization would make it 

easier for inspectors to move among groups, 

effectively obtaining necessary information, and 

allowing better, more-effective cross-utilization 

of personnel. 

6. The maintenance department holds a daily 8:00 

a.m. production meeting; the inspection 

department is invited to attend this meeting. The 
Chapter6 

infonnation from this meeting should be used to · 

help schedule tasks for the afternoon and night 

shifts. Tiie day shift attendee at this meeting must 

relay infonnation through a shift turnover log to 

the other shifts. It should become standard 

practice to use the shift turnover log to 

communicate such infonnation. 

6 . .3 POSSII\I.E SOU TIO\S TO 

C0\1\ll \IC.\TIO\ PROBI.E\IS 

After we completed the broad-based user needs 

analysis of workers and management, we 

considered possible solutions for improved 

communication at the maintenance base. 

6.3.1 Communication Tools 

As discussed above, communication could be 

facilitated by implementing a new communication 

system. However, in choosing the most appropriate 

tool for improving communication, it is necessary to 

consider who is trying to communicate with whom 

and what is being communicated. The human factors 

principle of fitting the tools to the user applies here 

no less than in designing hand tools. It may be 

necessary to use different cbmmunication tools to 

satisfy different types of communication 

requirements; in fact, it is improbable that one 

communication tool could address all communication 

needs. 

6.3,1,1 Available Communication Tools 

A formal written log, e.g., the shift turnover log, is a 

pennanent written record of activities within the 

inspection department. The document can serve 

legally as evidence for scheduling/staffing 

considerations and job control, and as a written 

account of problems inspectors encountered. A 

fonnal written log is usually hound so that pages and 

the information on them cannot be removed. 

Informal written notes can substitute for the current 

reliance on memory and verbal communication. 

Inspectors may forget to pass on infonnation to the 

lead inspector or to inspectors on the next shift. 

Writing down information relieves the inspector of 

relying on memory for the transfer of information. 

Infonnal notes can be addressed to an individual or to 

an entire crew. 

Tape recorders can replace infonnal written notes 

(discussed above). Many inspectors do not like to 

write down infonnation because the process of doing 

67 

Human Factors Development and Implementation 

so is cumbersome and time-consuming. Allowing 

each inspector to make personal notes and notes to 

others on a tape recorder eliminates the need for 

written notes. The tape can then be transcribed into a 

written log and/or passed to the oncoming shift for 

the next inspector. This allows an inspector to replay 

verbal infonnation during a shift. Tape recorders are 

best suited for recording infonnation for selfreminding or for another individual in a closely 

related occupation. 

Computer software tools can be developed to meet 

inspectors' communication needs. Tools such as 

electronic mail, electronic bulletin boards, electronic 

turnover logs, electronic databases, etc., can transfer 

infonnation among people. A computer tool allows 

more than one person to access infonnation 

simultaneously; this is not feasible with a formal 

written log since there is only one copy. Electronic 

tools provide flexibility in the presentation of 

infonnation. For example, each inspector may request 

only infonnation directly pertaining to the task at 

hand, and the inspector wiU not have to read 

irrelevant infonnation (see comments in #5 of Section 

6.2.4). 

Blackboards/Wbiteboards are quite useful for 

recording infonnation that only needs to be used for a 

short time. Blackhoards/Whiteboards should be 

utilized for communicating infonnation to an entire 

crew since the information becomes general 

knowledge. Infonnation could be left on the board for 

each of the three shifts to see and then be erased. It is 

important not to erase information that might be 

needed later, unless it is transcribed into a pennanent 

written log. For example, inspector work assignments 

are generally writlen on a whiteboard during every 

shift. This board is erased at the end of every shift, 

and work assignments are not recorded. It is therefore 

difficult quickly to trace previous work assignments; 

one must research completed workcards to do so. 

Formal crew meetings are useful for presenting 

infonnation to all inspectors. Meetings pennit twoway discussions about the .infonnation, as well as the 

opportunities for questions. Since the same 

infonnation can be presented to all three shifts, this 

ensures that all inspectors receive the same 

information. However, crew meetings are often 

ineffective in meeting inspectors' communication 

needs. Inspectors often ask questions at these 

meetings that are never answered, and the meetings 

can tum into gripe sessions. 
Human Factors Development and Implementation 

Although informal verbal communication is used in 

many information exchanges, it is not well-suited for 

many tasks. Verbal communication is short-lived. If 

the person receiving verbal information forgets 

something, it is very difficult for his or her memory to 

be refreshed. An inspector could be in the position of 

having to call an off-duty inspector at home to have 

information repeated. On the other hand, an inspector 

may refer to a written record of information as many 

times as necessary. Thus, written communication is 

less demanding on an inspector's memory. In 

addition, relying on memory for recording 

information is ineffective if the information needs to 

be kept for a long time. For example, an inspector 

who discovers and resolves a particular problem on 

an aircraft may not recall details of what occurred 

five months later, when the FAA is questioning him 

or her about a critical incident with that aircraft. 

Generally, verbal communication to more than one 

individual is difficult because it is nearly impossible 

to relay verbally exactly the same information, in 

exactly the same manner, more than once. 

Inspectors use non-routine work cards (NR W /Cs) 

to identify areas on an aircraft that require 

maintenance. The workcards are a formal recording 

procedure that allows inspectors to communicate their 

findings to the mechanics who will perform the 

needed repairs. Each non-routine workcard is then 

bought back to the inspector, who rereads the original 

68 

Chapter6 

write-up to ensure that the work is completed as 

specified. 

Table 6.3 (next page) illustrates how various tools 

can be used to meet communication needs between 

various inspection and maintenance personnel. 

As ASRS report analysis indicates, the issue in 

choosing an appropriate communication tool is one of 

ensuring ease of use so that necessary communication 

occurs. Table 6.3 (next page) shows a matrix of 

which tools can be useful for which tasks. For 

example, a small tape recorder, such as a microcassette dictating machine, provides easy and rapid 

memory augmentation. In some organizations, 

inspectors have such a device taped to their flashlight 

so as to have it instantly accessible. This is an 

example of improving ease of use and, hence, of 

decreasing the probability of missed communication. 

Another example is a board which can be used for 

rapid communication with many people. Although 

Table 6.3 (next page) indicates that a board can be 

used by leads and managers, it can also serve as a 

source of situational awareness when it carries notes 

from inspectors or mechanics. Again, the primary 

function of this tool is to promote ease of use. 

• 
j 

t 

Chapter6 

Table 6.3 CommuniCatiOn Tools r.tatnx -

inspector to seH * * * 

inspector to inspector * * * 

I (same shif!l. 

inspector to inspector * * 

t other shift}_ 

inspector to mechanic * * 

' (same shift}_ 

inspector to mechanic 

I (other shift) 

* 

inspectorto lead * * * 

in~ector (same shift) 

inspector to lead * * * 

insJJector j_other shift}_ 

in~ector to manaoer * 

lead inspector to lead * * * 

in~ector (other shift) 

lead inspector to * * * 

in~ector:J.same shift) 

lead inspector to * * * 

in~ector.J!>ther shift}_ 

lead inspector. to crew * * 

I (same shif!l. 

lead inspector to crew * * 

I (other shif!l. 

lead inspector to * * 

man~r 

manager to lead * * 

inspector 

manager to inspector * * 

manager to crew 

liall shift~ 

* * 

mechanic to lead * * 

in~ctor 

mechanic to inspector * * 

As Table 6.3 shows, computer systems are available 

to facilitate almost any activity, but their ease of use 

is not always appropriate for the demands of 

communication. If people need to be trained and then 

must later remember how to access the tool, or how to 

direct a notice, then the tool's frequency of use will 

drop. Fortunately; advances in human-computer 

interaction (HCI) have improved interface destgn, 

particularly for infrequent users. 

The other major cluster of tool use is in handwritten 

logs. The shift turnover log is the basis for human 

factors intervention in this project. 

69 

Human Factors Development and Implementation 

- ~ ~"''~~ ~~~~~ -,.,~ 

j j f j1~ .. ~ , .. ... l N ,,;; ';" ~~' 

* 

* * * * 

* * * 

* * * * 

* * * * 

* * * 

* * 

* * * 

* * 

* * 

* * 

* * * 

* * * 

* * * 

* * * 

* * 

* * * 

* • * 

* * 

6-3.2 Proposed Shift Turnover Log 

The proposed shift turnover log was designed to 

improve communication among inspectors from 

different shifts. The present shift turnover log is used 

mainly by the lead inspectors and does not contain 

much information that inspectors can utilize. It does 

not tecord activities that took place during a shift or 

help the next shift know what they need to 

accomplish. 

The proposed shift turnover log is intended for use by 

all inspectors. It allows an inspector to record 

activities during a shift, leaving a written account of 

what needs to be accomplished and helping prevent 
Human Factors Development and Implementation 

rework. Rework in inspection, i.e., more than one 

inspection of the same area, is often caused by 

miscommunication between two inspectors. This is 

especially true when an inspection is carried over 

from one shift to the next, and the second inspector 

does not understand where to start and stop the 

inspection. In this situation, an inspector typically 

does "a bit more" so there is no doubt the workcard 

was covered. 

6.3,2.1 Rrst Draft Genera! Information: Prooosed SMt 

Turnover l.Qg 

This proposed shift turnover Jog (Figure 6.1, next 

page) will allow inspectors easily to obtain necessary 

information about an aircraft to which they are 

assigned. This Jog is organized into five separate, 

bound books. Each book has sequentially numbered 

pages to prevent any pages from being removed. 

The first book is the general shift turnover Jog. It can 

be used, as the current log is used, to pass information 

between shift leads. 

70 

Chapter6 

Information included in this Jog includes any 

personnel infonnation such as assigned overtime, 

call-ins, and field-trips, as well as any general 

problems. The shift lead inspector should complete 

this log for the following shift. 

The other four Jogs correspond to the hangar bays 

(Figure 6.2, page 72). Each book, including the 

pages, is color-coded to match the bay color. The 

book should contain enough pages for it to be used 

during the estimated duration of the aircraft's stay in 

the hangar: three pages for each day, plus a few extra. 

A new book can be started for each new aircraft; 

therefore, each book contains the complete inspection 

history for one aircraft. The Jog can be filed when the 

aircraft leaves the hangar. Inspectors assigned to a 

particular aircraft should complete this Jog. 

The specifications and instructions for the proposed 

shift turnover Jog are included as Appendix 6-D. 
Chapter6 Human Factors Development and Implementation 

Frgure 6.1 lr·spec•ron Shrlt Turnover Log !Frrsl Dralll 

General Shift Information 

Data: To Be-By: Morning All8moon Night Shift 

IMd Inspector: 1....._, 

Filled In Bv: 

Personnel Information 

C&ll-lns 

Name Reeeon Time 

~ 

Name Reeeon Number of Hours 

Field Tripe 

Departure Return 

Name Destination Time Time 

Speciallnstructions!General Problems 

Problem NeecledAcdonlAiert Resolution Data Time 

• 

. , 

71 
Human Factors Development and Implementation Chapter6 

Aircraft, Status (Please Circle): Line Initial Shakedown Inspection 

72 
Chapter 6 

6.3.2.2 Evaluation o1 First Draft 

A sample of the inspectors was asked to evaluate the 

proposed shift turnover log. Responses of the 

seventeen inspectors are summarized in Table 6.4. 

These results indicate that the proposed shift turnover 

log offers many improvements over the current 

version. A One-Sample Wilcoxon test was performed 

to determine whether the median response for each 

question was significantly different from the 0, midpoint(4), or end-point of the rating scale(8). After 

performing this analysis, we find that the inspectors 

felt that the use of a separate log for recording 

personnel issues and general problems was 

significantly better than. useful (median= 5.65, 

73 

Human Factors Development and Implementation 

p=.038). They also indicated that they would read the 

turnover log for the aircraft to which they were 

assigned more than three times per week (median = 

7 .0, p=.009). Inspectors also felt that the proposed 

turnover log was more useful than the current 

turnover log (median = 5.225, p=.002) and that they 

would use the proposed turnover log more often than 

they use the current turnover log (median = 4.5, 

p=.037). 

Other trends in the data, although not statistically 

significant, are that the inspectors generally found the 

proposed log easy to understand and that both the 

general and the aircraft sections contain the right 

amount of information. Unfortunately, inspectors 

3.88 2.5 

4.53 2.18 

2.14 

4.63 2.8 

6.33 2.54 

4.09 1.85 

4.29 1.99 

3.81 1.78 

3.83 1.85 

5.38 1.51 
Human Factors Development and Implementation 

indicated that they would he likely to make an entry 

in the log only three times per week, not every day as 

the log would require. Comments from the user needs 

analysis indicated that many inspectors feel that 

maintaining the log is the lead inspector's duty. There 

are clear issues of culture, expectations, and training 

surrounding any change in the shift.turnover log. 

The inspectors indicated that the proposed shift 

tumo'ver log does not meet their needs for 

information, as indicated by the less-than-useful 

ratings given to the type of information the log 

contains. They do not find the proposed shift turnover 

log's layout particularly easy to use. Finally, 

inspectors rated the usefulness of the proposed shift 

turnover log (Questions 17 and 18: mean 4.64 

compared to 4.35) as only slightly higher than the 

usefulness of the current shift turnover log; a MannWhitney analysis indicates that this difference is not 

statistically significant. 

6.3.3 Version 2 of the Shift Change Log 

6.3,3 1 Design of Second Version of Shift Change Log 

From these results, it appears that inspectors approve 

of the idea of developing a new format for the shift 

turnover log and will utilize an improved log, 

especially its sections pertaining to their specific 

work assignments. However, more work is necessary 

to find a layout that will meet inspectors' information 

needs. 

After analyzing the results, we concluded that 

inspectors supported the idea of maintaining a 

separate log for each hangar bay; however, they were 

not satisfied with the information on or the format of 

the proposed log. More work was needed to design a 

log better meeting the inspectors' information needs. 

We decided to use a team approach for the next phase 

of shift turnover log design. We held meetings with 

each inspection shift to discuss how the log should be 

designed. Inspectors were encouraged to contribute to 

the process by indicating the information they would 

like to see included in the turnover log. 

Unfortunately, of the 10 to 15 inspectors in each 

meeting, only a few provided input for redesigning 

the shift turnover log. Their overall suggestions were 

to simplify the proposed shift turnover log and to 

reduce the writing required to complete it. One 

inspector suggested that the log should include only a 

simple heading (aircraft number, date, shift) and a 

blank space for inspectors to write; this is basically 

74 

Chapter 6 

the same as the current turnover log (it is not being 

utilized effectively). 

Although user needs analysis results had indicated 

otherwise, most inspectors reacted negatively to the 

idea of a redesigned turnover log. Some of their 

opinions were the following: I) inspectors would not 

use a redesigned log unless it was mandated by upper 

management; 2) separating the log by hangar bay 

would make the log too difficult for leads to use; 3) 

leads are the only ones who need a shift turnover log; 

4) inspectors depend on leads to pass along 

information; and 5) it is not the inspectors' 

responsibility to pass information during a shift 

turnover. These comments weye symptomatic of 

inspectors' general attitudes, implying that 

communication between shifts is not the most serious 

problem within the inspection department. 

In addition, the shift schedule (7:00 a.m.-3:00p.m., 

3:00 p.m.-11:00 p.m., and 11:00 p.m.-7:00a.m.) 

does not allow for overlap of oncoming and outgoing 

shifts. Many inspectors felt that a shift turnover log 

(either verbally or written) would require too much 

time and would place too many additional 

requirements on the inspectors. What the inspectors 

fail to realize is that this is the exact reason an 

effective shift turnover log is essential. 

Inspectors also indicated that it is the lead inspector's 

responsibility to perform a shift turnover. The lead 

should extract the important information from each 

crew member and pass this information to the next 

shift. The oncoming lead is responsible for reading 

the information in the log and distributing it, as 

necessary. Although many inspectors indicated that 

they require information passed between shifts, they 

believe that someone else is responsible for providing 

this information. 

Many inspectors indicated that they would find a log 

for the particular aircraft to which they were assigned 

helpful. This would allow them quickly to 'get a feel' 

for the aircraft's status. These inspectors also stated 

that it is most important for leads to understand what 

is happening, and the proposed shift turnover log 

should he designed for leads, not for other crew 

members. This is troubling; as one sees in the ASRS 

reports, it is critical for inspectors working on an 

aircraft to have a good understanding of the problems 

previous shifts encountered. 

In addition, many inspectors have regular 

opportunities to serve as the lead for a shift, e.g., 

when the permanent lead takes a day off, and many 

inspectors eventually become permanent leads. 
Chapter6 

Although inspectors do not feel responsible for 

knowing information in the turnover log, they are 

expected to have a full understanding of it when they 

act as lead for a shift. An effective turnover log could 

ensure that an acting lead inspector is quickly able to 

extract necessary information. If all inspectors 

regularly read the redesigned log, there will be less 

information to absorb when he or she becomes a 

temporary lead inspector. 

There also seems to be a large mismatch between the 

inspectors' need for information and the effort they 

are willing to make to obtain it. On the original 

communications user needs analysis, inspectors 

indicated that they rarely if ever have enough 

information, that they often must search for 

information to perform their jobs, and that they would 

like information to be readily available. However, 

when inspectors were asked to provide more 

information about events occurring during their shift 

through the shift turnover log, most were extremely 

reluctant to do so. They felt that completing a written 

log at the end of each shift would be too timeconsuming and difficult. Inspectors seem to want to 

receive information from the previous shifts, but not 

to provide information to the next shift. 

Inspectors are reluctant to write down any 

information not specifically required. They feel that 

their signatures on workcards fulfill their legal record 

keeping requirements. They do not want to record 

additional information in a log which could be used 

against them in an investigation; they do not realize 

that information in a written log could protect them in 

an investigation. This is also part of a current national 

debate: can maintenance and inspection personnel be 

75 

Human Factors Development and Implementation 

disciplined merely for providing information which 

could help the system? 

Many inspectors seem unwilling to make an effort to 

improve the communication process. They are 

unhappy with how management treats them and, thus, 

have little motivation to improve the situation. Most 

simply want to perform their jobs and to take on as 

little responsibility as possible. Inspectors are 

distrustful of management and do not believe that 

management wants to aid the inspectors by trying to 

improve communication. During small group (or oneon-one) discussions, inspectors offered suggestions 

for improving internal communication in the 

inspection department. During the shift meetings few 

people were willing to discuss a need for improved 

communication. Even individual inspectors who want 

to improve their jobs do not want to appear 

sympathetic to management's needs or wants. Some 

inspectors bad a hard time believing that management 

had not sent us. Sociotechnical problems between 

management and inspectors must be resolved before 

any proposed shift turnover log can meet information 

needs of both groups. As is true of many human 

factors issues in aircraft maintenance and inspection, 

searching for a consensus solution to a technical 

problem reveals broad social issues when it is time 

for implementation. 

Based on input we received in evaluation meetings, 

we simplified the shift change log for its final version. 

We did this to address inspectors' (other than leads') 

unwillingness to provide shift information, although 

the changes somewhat reduce the information's utility 

to the reader. Fignres 6.3 and 6.4 (following pages) 

show the second draft of the shift change log . • 
l 

Human Factors Development and Implementation Chapter6 

Figure 6.3 Lead Inspector Shift Turnover (Second Draft) -- - --------

General Shift Information 

Date: 1 To Be Read By: Morning Afternoon Night Shift 

Leed lnsp8ctor: Manager: 

Filled In By: on the Morning Afternoon Night Shift 

Personnel Information 

can-ms 

Name Reason Time 

OVertime 

Name Reason Number of Hours 

Field TriDS 

Departure Time Return 

Name Deatlnatlon Time 

~I 

Special Instructions/General Problems 

Problem Neadad Action/Alert . 

76 
Chapter6 

6 3.3.2 Evaluation of Version 2 of the Shj!t Change 

LQ!I 

We used the same evaluation form as in Section 

6.3.2.2 to obtain feedback on Version 2 of the new 

shift change log. Nineteen inspectors evaluated the 

log shown in Figures 6.3 (previous page) and 6.4. 

Table 6.5 (next page) summarizes these results in the 

same way Table 6.4 summarized those for the first 

version. 

A One-Sample Wilcoxon test showed that inspectors 

still appreciated the idea of separating personnel 

information from aircraft information (median= 

5.025, p = .011), that they found information in the 

proposed log more than useful (median = 4.95), p = 

.003), that they would read all sections of the log 

77 

HUITUJn Factors Development and Implementation 

more than three times per week (median= 5.300, p = 

.036), that they would read the section of the log for 

the aircraft to which they were assigned almost every 

shift (median= 7.375, p = .001), and that they would 

make entries into the log more than three times per 

week (median= 6.00, p = .023). 

Inspectors also thought that information in the log's 

general section is more than useful (median = 4.562, 

p = .015), and that information in the aircraft section 

is more than useful (median= 4.600, p = .012). They 

preferted the proposed to the current turnover log 

(median= 5.450, p = .001) and would use the 

proposed log more than they use the current log 

(median= 5.150, p = .005). Inspectors found the 

new format of both general and aircraft sections 

better than easy to use (median = 4.650, 4.738, p = 
Human Factors Development and Implementation 

.015, .016). Finally, they indicated that the proposed 

log is more than useful (median= 5.200, p = .002). 

It is possible to use data in Tables 6.4 (previous page) 

and 6.5 directly to compare the two versions of the 

shift change log. A two-sample turnover test was 

performed to compare results from the evaluations of 

the first and second drafts. Table 6.6 (next page) 

presents the results of this analysis. 

These results indicate that inspectors rated the second 

draft significantly higher in both information content 

and format (at the p < .01 significance level). Since 

these were the first draft's main weaknesses, the 

second draft appears better able to meet inspectors' 

communication needs. 

78 

Chapter 6 

4.09 2.10 

4.70 1.69 

.34 

5.35 2.57 

5.96 2.19 

turnover 4. 

4.14 1.02 

4.83 1.29 

5.48 1.42 

4.86 1.49 

4.93 1.52 

4. 

5.26 1.43 

Although the result was not significant, inspectors felt 

that the second draft was more useful (mean= 5.26 

versus 4.64 in first draft) and that they would be more 

likely to make frequent entries in the second draft 

(mean= 5.96 versus 4.21). These data support the 

findings that the second draft is better suited to 

inspectors' communication needs. We therefore 

proposed that this version become the base's standard 

shift change log. 

6.3.4 Other Communication Solutions 

During 1995, Northwest Airlines management will 

implement two programs to improve communication 

with its workforce. First. they will introduce a bulletin 

board for posting company news and announcements. 
I 

I 

1 

Chapter6 

Each shift will have its own copy of each 

announcement, and each inspector will sign off 

after reading each posting. This system is designed to 

ensure that aU inspectors are aware of important 

company business. 

Management will also schedule meetings with 

inspectors, and inspectors will determine the 

frequency of these meetings. These meetings will help 

management better understand each inspector's needs 

and concerns. Inspectors issues and concerns wil1 be 

79 

Human Factors Development and Implementation 

4.53 4.70 0.79 

4.63 5.35 0.43 

4.21 5.96 0.11 

4.29 4.14 0.79 

3.81 4.77 0.081 

log? 5.38 5.48 0.85 

4.85 5.20 0.51 

turnover 

3.64 4.93 0.011 

0.68 

4.64 5.26 0.19 

recorded on a form that includes to whom the issue is 

assigned and an expected resolution date. The form 

will be posted on the bulletin board so that everyone 

is aware of progress made toward resolving the 

issues. 

Other possible solutions inspectors suggested include 

the following. 

I. Allow each inspector to carry a small tape 

recorder throughout the day so that an inspector 
Human Factors Development and Implementation 

can record information, notes, and messages as 

events happen. The tapes can he passed to the 

inspector taking over on the next shift. This 

second inspector can listen to the previous 

inspector's notes as often as necessary. The tapes 

can he transcribed into the written log of daily 

activities for permanent record keeping. 

2. DevelOp a shift turnover log in the form of a 

simple checklist, allowing inspectors quickly to 

complete the log with minimal writing. 

Eventually, a bar code system could allow even 

simpler completion. 

3. Use one-on-one shift turnovers in which 

incoming inspectors walk around the hangar with 

outgoing inspectors to ensure that all necessary 

information is relayed. 

4. Use a blackboard/whiteboard temporarily to 

record information that may be useful for all 

inspectors. Information often passes to inspectors 

through informal, impromptu meetings, often 

over a particular problem one inspector 

encountered. When absent, a particular inspector 

may never know that he or she missed hearing 

important information. When this problem is 

again encountered, it may be completely new to 

some inspectors, although others previously 

discussed and resolved it. Inspectors would find 

it helpful for this type of information to he 

written down so that they all may review it. 

6.4 (;L IDE TO .\JI{LI:\ES Ol\ 

EST \BLISHI:\G HL l\1 \:"\ F.\CTORS 

1'1W<;R.\l\1 

One of the outcomes of this study was to be a 

guide for airlines on how to establish and 

implement their own human factors/ergonomics 

programs. The information on task force formation, 

training, and procedures was written as a guide in 

Chapter 2 of the FAA's Human Factors Guide for 

Aviation Maintenance. 

That chapter presents the following seven-step 

process: 

Establish mission and structure 

Form human factors task force 

Train task force 

Analyze jobs 

Design solutions 

Reanalyze changes 

Transfer technology. 

80 

Chapter6 

This material was presented and used as the basis for 

a workshop at the FAA/AAM Annual Human Factors 

in Maintenance meeting in Albuquerque, New 

Mexico, during November 1994. C. G. Drury 

summarized progress of the current project in a 

presentation entitled "Integrating Human Factors into 

Maintenance Program." Project results since that time 

(Sections 3 and 4 of this report) provide additional 

feasible structures for human factors implementation. 

A broader program with limited objectives, but wide 

involvement, may serve as a viable first project to 

gain visibility for human factors in a maintenance 

organization. Lessons learned from the 

communications/shift log study reported in Sections 3 

and 4 are being incorporated into Chapter 2 of the 

Guide and will form the basis of a proposed new 

Guide chapter covering communications processes. 

6.5 COl\CLliS10l\S 

This project demonstrates that a human factors 

program in an airline maintenance environment 

succeeds only when it adapts to the mainten:lnce 

base's specific environment. Our initial methodology 

of using a workforce/management team to target 

specific jobs did not produce successful 

implementations, despite its success in many other 

industries. Our airline partner's specific needs 

required a different approach based on involving the 

maximum number of people, instead of a small task 

force, and limiting the scope to one issue, i.e., 

communication, rather than searching broadly for 

ergonomic mismatches. 

Focusing on communication brought potential 

solutions under diject control of employees at the 

site, while still demonstrating potential for improved 

human error rates. The use of outside data, in this 

case the ASRS reports, provided specific instances of 

human factors needs which could be related to local 

conditions and suggested practical improvements. 

The specific choice of the shift turnover log showed 

how involvement of both human factors professionals 

and the inspection workforce can produce a practical 

refined job aid. The new log meets more 

communication needs than its predecessor and has 

good acceptance in the user community. 

Jl 
Chapter6 

6.6 IU.I· ERE"\CES 

Burke, M. (1992). Applied Ergonomics Handbook, 

Boca Raton: Lewis Publishers. 

Human Factors Guide for Aviation Maintenance 

Koli, S. and Drury, C. G., (1995). Ergonomic Audit 

for Visual Inspection of Aircraft, Human Factors 

in Aviation Maintenance-Phase Four, Progress 

Report, DOT/FAAIAM-93, National Technical 

Information Service, Springfield, VA. 

NASA (1994), Mechanics Reports, Search Request 

NO. 3601, Aviation Safety Reporting System, 

CA. 

81 

Human Factors Developmenl and lmp/ementlltion 

Reynolds, J. and Drury, C. G. (1993). An evaluation 

of the visual environment in aircraft inspection. 

Proceedings of the Human Factors and 

Ergonomics Society 37th Annual Meeting, 

Seattle, WA, 34-38. 

Taylor, J. C. (1992). Communication Guidelines for 

Maintenance. Interim Report for the FAA Office 

of Aviation Medicine. 

Taylor, J.C. (1990). Organizational context for 

aircraft maintenance and inspection. In 

Proceedings of the Human Factors Society 34th 

Annual Meeting, Volume 2, 1176-1180. 

• 
Appendix 6-A 

Ergonomic Audits of Inspection Tasks 

TO: 

fROM: 

Task Description: 

Date: 

Tune: 

Station: 

Hangar Bay: 

Aircraft No. 

MIENo. 

Q/ANo. 

:John Lane 

:John W. Ditty 

:Keel Inspection 

:4f27/94 

:!O:OOa.m. 

:Atlanta 

:RED 

:9153 

HUMAN FACTORS MISMA TCHESIRECOMMENDATIONS IN PRE-INSPECTION/ DOCUMENTATION 

A. Information Readability 

I. Dot matrix printers with a 5X7 matrix of dot characters is minimally acceptable for reading purposes. If 

used, check for character specifications: 

Minimum Character Height = 3.1 mm to 4.2 mm 

Maximum Character Height = 4.5 mm 

Width/Height ratio = 3:4-4:5 

IMP: Do not use lower case letters, since features can get easily confused. 

2. Standards not prescribed. State "TIME" & "QUALITY" standards to ensure consistent print quality. 

B. Information Content 

Text 

3. Feedforward information not provided to the inspector. Present inforn'lation on 

a: previous faults detected 

b: locations of prior faults 

c: likely fault-prone areas for the specific task & current aircraft under inspection. 

C. Information Organization 

4. Incorrect sequencing of tasks in the workcard. Tasks need to be sequenced in the natural order in which 

the task would be carried out by MOST inspectors. 

5. Avoid carryover of tasks across pages at ILLOGICAL points. Tasks should begin and end on the same 

page. For longer tasks, break into several subtasks with multiple sign-offs. Each subtask should begin and 

end on the same page. 

6. Excessive number of tasks per action statement. More than 3 actions/step increases the probability of 

action slips. 

82 
Chapter 6 Appendix- Hunum Factors Program 

HUMAN FACfORS MISMATCHES/RECOMMENDATIONS IN PRE-INSPECTION/COMMUNICATION 

I. No ongoing program to maintain adequacy of connnunication channels. 

HUMANFACfORSMISMATCHES/RECOMMENDATIONSINPRE-INSPECI10N/VISUAL 

CHARAcmRISTICS 

I. Fluorescent bulbs: "Fair" to "Good" color rendition properties. Color rendition is the ability to distinguish 

true colors correctly. This is especially useful in detecting corrosion faults. For best results, consider 

incandescent bulbs. 

2. Flicker exists. Consider: 

a. appropriate shielding of ends of fluorescent lamps 

b. regular replacement of fluorescent lamps. 

3. lighting fixtures dirty. Keep lighting fixtures free/clean from dirt/paint. 

4. No "Shades/shields" on illumination source. This may cause "direct" or "disability" glare. 

5. lllumination sources not working. Consider regular replacement of light sources. 

HUMAN FACfORS MISMATCHES/RECOMMENDATIONS IN PRE-INSPECTION/ ACCESS 

ACCESS-STEP LADDERS 

ACCESS -TALL STEP LADDERS 

HUMAN FACfORS MISMATCHES/RECOMMENDATIONS IN INSPECI10N/DOCUMENTATIONPHYSICAL HANDUNG AND ENVIRONMENTAL FACfORS 

I. Corrent light conditions inadequate for quick and easy reading of workcard. 

2. The inspector does not sign-off workcard after each subtask. This may lead to errors of omission. 

HUMAN FACfORS MISMATCHES/RECOMMENDATIONS IN INSPECI10N/TASK LIGHTING 

l. The average task illumination is 152.50 fc. and the variance is 2318.75. The recommended task 

illumination should be 200.00 fc. The variance is exceptionally higb. 

2. Handlamps deliver a max. of 85 fc. of light. This illumination level is inadequate for "Fine Inspection. • 

Handlamps also lack aiming control. Consider using of Standing Lamping (Halogen 500 watts-1200 fc.). 

3. Consider headlamp for hands-free illumination: except in explosive environments, e.g., fuel tank 

inspection. 

4. The portable/personal lighting equipment interferes with the inspection task. 

5. The operator felt difficulty in handling with respect to the size of the lighting equipment. 

6. The operator felt difficulty in handling with respect to the weight of the lighting equipment. 

7. The operator experienced glare from the task surface. Consider: 

a. reducing glossiness of material 

b. screening of sunlight penetrations 

c. repositioning the light source 

d. use diffusing light sources, e.g., fluorescent lamps 

83 
Appendix- Human Factors Program 

HUMAN FACTORS MISMATCHES/RECOMMENDATIONS IN INSPECTION/THERMAL 

CHARAC1ERISTICS 

Chapter6 

1. The current DBT is 0.00 deg. cent. The recommended temperature is between 20-26 degrees centigrade. 

2. The current task has been identified as having illGH physical workload. The DBT is 0.00 cent. and the 

.clo value for clothing is 0.79 clo. The recommended DBT values for mGH workload and clo values 

between 0.75-1.0 are 14-20 deg. cent. Consider change in clothing. 

HUMAN FACTORS MISMATCHES/RECOMMENDATIONS IN INSPECTION/OPERA TOR PERCEPTION OF 

THERMAL ENVIRONMENT. 

1. The operator found the summer temp. at the workplace to be slightly warm. 

2. Operator wanted the summer temp. at the workplace to be cooler than the current temp. 

3. Operator is generally not satisfied with the temp. at workplace during summer. 

4. The operator found the winter temp. at the workplace to be slightly cool. 

5. Operator wanted the winter temp. at the workplace to be warmer than the current temp. 

6. Operator is generally not satisfied with the temp. at workplace during winter. 

HUMAN FACTORS MISMATCHES/RECOMMENDATIONS IN INSPECTION/AUDITORY 

CHARACTERISTICS 

1. The maximum sound level at this task is 105 db A. Noise levels above 90 db A indicate the need for 

management intervention and control. 

2. This task involves verbal communication. The average noise level is 95.60 dbA. The distance of 

communication is 4.00 feet. The noise level for communication at a distance of 3.5-6.0 feet should not 

exceed 60 dbA. • 

HUMAN FACTORS MISMATCHES/RECOMMENDATIONS IN INSPECTION/NON-DESTRUCTIVE 

TESTING 

1. NOT equipment was not easily maneuverable during inspection. 

Displays, Controls, and Knobs 

2. The inspector experiences division of attention. Consider using two inspectors for the NOT inspection. 

3. Visual checks are not highlighted by aural signals. Auditory signals help by providing redundancy gain. 

HUMAN FACTORS MISMATCHES/RECOMMENDATIONS IN INSPECTION/ACCESS-ACTIVITY 

1. Inspection affected by parallel work, e.g., opening or closing of panels, cleaning other inspections, or 

repair. Also check for obstruction due to equipment, e.g., tool boxes, lighting equipment, access 

equipment, etc. 

2. The operator felt that access was difficult. 

84 

1 
I 

I 

I 

I 

Chapter6 Appendix- Hllltlllll Factors Program 

3. The operator felt that access was dangerous. 

4. Access equipment was repositioned too frequently. This consumes a lot of OPerator effort. Consider using 

multiple access equipment. 

HUMAN FACI'ORS MISMATCHES/RECOMMENDATIONS IN INSPECTION/POSTURE 

I. . The operator felt that the workspace was constrained. 

The following extreme postures were observed during the current inspection task: Urgent intervention is 

requested. 

2. Arms in air, back bent, and loading on leg(s). 

3. Arms in air, back bent and kneeling, or laying or crawling. 

4. Arms in air, back twisted, and loading on leg(s). 

5. Arms in air, back twisted, and kneeling or laying or crawling. 

6. Back bent and twisted and loading on leg(s). 

7. Back bent and twisted and kneeling, laying. or crawling. 

HUMAN FACI'ORS MISMATCHES/RECOMMENDATIONS IN PRE-INSPECTION/ SAFETY 

I. The inspection area is not adequately cleaned for inspection. Consider appraisal of pre-inspection 

processes like "open-up" and "cleaning". 

HUMAN FACI'ORS MISMATCHES/RECOMMENDATIONS IN POST-INSPECTION/FEEDBACK 

l. Consider inclusion of standard information like ATA codes, station#, Sup.#, employee#, etc., in the 

workcard. This considerably reduces the cognitive load on the inspector. 

85 
Appendix- Human Factors Program 

I) W orkcards 

• Card contrast changes 

Appendix 6-B 

Ergonomic Risk Factors 

• Ribbon changing-establish preventive maintenance program 

• Graphics-<Onfusion using graphics/time to get graphics 

• Graphics on cards--could one get too reliant on cards and not use the manual? 

• Card content inaccurate? 

• Graphics attached to card until buy-<~ff 

• Breaks between cards is not good 

• Use of if/then statements 

2) Lighting 

• Fixtures are dirty 

• Need a preventive maintenance program for lighting 

• Lighting at the back of the hangar is inadequate 

• Color of hangar bays-to ensure good reflectance, need a light color floor 

• Repairs must be performed by facilities department 

3) Keel Inspection 

• Check task lighting-cannot read workcard 

• Fuselage stand lighting 

• Handling lighting equipment cords and small lights 

• Temperature in the summer is too hot 

• Task performed in very noisy environment 

• Sheet metal work often interferes with task access 

• Task performed in a restricted space 

• Difficult to get back on to the ladder 

• Task requires less-than-optimal posture 

Chapter6 

• Task must often be recleaned-cleaners do not understand necessary level of cleanliness required for 

this task 

• Cleaners' work of is not inspected before task begins 

• Time pressure 

4) PS4 Drain Box Inspection 

• NDT equipment design-probe is difficult to place/equipment is not easy to maneuver 

• Scaffolds/ladders can be slippery/task is difficult to access 

• Sign-offslbuy-backs on shift change 

• Task light cords in the way 

• Check lighting levels on task 

• Task too bot when the engine is still warm 

• Qeaning is often inadequate-not enough time to clean on an overnight inspection 

86 
Cbaprer6 Appendix- HwriiJII Factors Program 

5) E&E Compartment Inspection 

• Check task lighting 

• Cannot read workcard 

• Need fixed task lighting for a number of tasks-need to design an appropriate lighting fixture 

• Temperature high, due to equipment, in the summer 

• Task requires less-than-optimal postures 

6) Forward Alxes!wry. Compartment Inspection 

• Task requires a high ladder-{)ften difficult to find appropriate ladder 

• Requires a different type of ladder than those available 

• Check task lighting-use of headlamps 

• Task is performed in a restricted spactHiifficult to access 

• Task requires less-than-optimal postures 

I 

I 

87 
Jl i 

II 

Appendix- Human Factors Program Chapter6 

Appendix 6-C 

General Communication 

User Needs ADlllysis 

Your help is needed to assess the quality of internal and external communications in the Hangar Inspection 

Department. Here is an excellent opportunity for you to help us make improvements in the Inspection Department 

Communications System which will give you clear information on your work assignments and make the 

workplace less stressful. 

Please complete the questionnaire below and return to the Atlanta Safety Department by October 20, 1994. 

Remember, if you do not complete and return a questionnaire, you miss an opportunity to make a difference. 

I. 

2. 

3. 

4. 

5. 

6. 

7. 

8. 

9. 

10. 

II. 

12. 

13. 

14. 

How many years experience do you have as an inspector? 

Where (or from whom) do you get necessary information? 

Is information given to you verbally or in written form? 

Whom do you regularly pass information to? 

How do you pass information (verbally or in written form)? 

Do you regularly have all necessary information when working on a task, or are you constantly going 

back for more information? 

Do you ever read the shift-turnover log? If so, how often do you do so? 

Do you ever write information in the shift-turnover log? If so, how often, and under what circumstances? 

What do you see as the purpose of the turnover log? 

If you could design a shift-turnover log, what type of information would you include? 

Should the turnover log be a SEPTRE program similar to Hangar Daily Stat, or book, or both? 

Do you attend regular crew meetings? If so, who is in attendance at these meetings? 

Do you feel that regular crew meetings are infonnative and beneficial, or are they a waste of your time? 

Have you ever had a problem caused by miscommunication, either between you and another inspector, 

you and the lead inspector, you and a manager, between you and mechanics, or you and engineering in 

the work area? If so, please describe. 

15. How much turnover time do you have between shifts? Is it sufficient? If not, how much time is needed? 

If additional space is needed, please write your response on the back of the page, referencing the question number. 

Thank you for your time and input. 

John Lane 

Safety Manager 

88 
Cbapter6 Appendix- HUitiQII Factors Program 

Appendix (;.D 

Specifications for Proposed Shift Turnover Log 

A) General Shift 1\amover Log 

I) The first section of this Jog records general shift infonnation: 

Date: Enter the date on which the shift begins. 

To Be Read By: Circle the shift for which this page has beeo written: morning (1st shift), afternoon (2nd shift), or 

night (3rd shift). Each lead inspector should complete this Jog for the following shift. 

Lead IDspector: Enter the name of the acting lead inspector on the shift for which this page is intended. 

Manager: Enter the name of the inspection manager on duty during the shift. 

Filled In By: Enter the name of the lead inspector who completed this page and circle his or her shift. 

Example: The day shift lead inspector should begin this log for the afternoon shift. In the first section of the log, 

the "to be read by" shift is the afternoon shift. The lead inspector is the afternoon lead inspector's name. The 

manager is the afternoon manager's name. The day shift lead should enter his or her name and circle "morning 

shift" in the "filled in by" box. 

2) The second section of this log records personnel information. Information should be recorded as it is received. 

The lead inspector should enter information in the log that is to be read by the shift this personnel information 

affects. ' 

Call-ins should be entered on the log for the shift the inspector was supposed to work. 

Name: Enter the name of the inspector who called in. 

Reason: Enter the reason the inspector called in, e.g., sick, family emergency, etc. 

Time: Enter the time the call was received. 

Overtime should be entered on the log for the shift on which the inspector is going to work the overtime hours. 

Name: Enter the name of the inspector who is working the overtime. 

Reason: Enter the reason the inspector is working overtime. 

Time: Enter the number of overtime hours the inspector is expected to work. 

Field Trips should be entered on the log for the shift on which the field trip begins. 

Name: Enter the name of the inspector assigned to a field trip. 

DestiDation: Enter the destination of the field trip. 

Departure Time: Enter the time the inspector departed. 

Return Time: Enter the time the inspector is expected to return. 

89 
Appendix- Human Factors Program Chapter6 

Example: If Inspector A is supposed to work the midnight shift and calls in sick at 6:00p.m., the afternoon shift 

lead inspector should record this information on the log the night shift lead inspector is to read. Similarly, if day 

shift Inspector B is asked to work late (overtime), this information should be recorded on the log the afternoon 

shift lead inspector is to read. 

3) The third section of this log records special instructions and general problems. This information, recorded by 

the lead inspector, is to be read by the lead inspector on the following shift. Information intended for both 

following shifts should be recorded on both log sheets. The "resolution," "date," and "time" should be completed 

by the shift resolving the problem or completing the project. 

Needed Action 

/Alert: 

Resolution: 

Date: 

Time: 

B) Aircraft Log 

Describe the problem or situation. Each problem on a given day should be numbered 

sequentially. 

Enter the action the oncoming shift must complete or describe the alert/warning the shift needs 

to be aware of. Number the actions with numbers of the problem to which they refer. 

Describe the resolution determined or implemented for the problem and include any further 

developments of a situation. Number the actions with numbers of the problem to which they 

refer. 

Enter the date the problernlsitnation is resolved. 

Enter the time the problernlsitnation is resolved. 

I) The frrst section of this log records general information about the aircraft: 

Aircraft Number: 

Day: 

Shift: 

Inspectors Assigned: 

Aircraft Status: 

General Information 

/Notes: 

Enter the number of the aircraft. 

Enter the number of days the aircraft has been in the hangar. 

Circle the shift (morning, afternoon, night) compleoog this log. 

Enter names of all inspectors assigned to this aircraft on this shift. 

Circle the status of this aircraft: Line (not yet in the hangar), Initial Shakedown (initial 

inspection in the hangar), Inspection (performing scheduled inspections), Buy-back (the 

buy-back of non-routine workcards). 

Enter any information about this aircraft important for the next shift to know and/or 

understand. Some of this information may also be reported to the oncoming lead 

inspector and recorded in general shift turnover log. 

2) The second section of this log describes ongoing long-term projects: 

Project: Describe the project being worked on, including the location on the aircraft, if relevant. 

Number projects sequentially. If more space is needed, continue on the back of the 

page. 

90 

' 
i 

I 

I 

I 

Chapter 6 Appendix- Human Factors Program 

Status: Describe the project's status, e.g., project is 30% complete or project is waiting for a 

specific part. etc. 

Needed Action/ Alert: Describe any actions the next shift must perfonn or describe any warnings/alerts the 

next shift should be aware of concerning this project. 

Inspector: Enter the name of the inspector who entered this project into the log. 

3) The third section of this log describes other ongoing projects/problems: 

Inspector: 

Project/Problem: 

Enter the name of the inspector who entered this project/problem into the log. 

Describe the project, e.g., bag-bin inspection not completed, or the problem, e.g., tail 

section not clean enough to inspect at 2:30 p.m., that the next shift must be aware of. 

Number each project/problem consecutively. 

Needed Action/ Alert: Describe actions the oncoming shift should take concerning the projects or problems. 

Resolution: Describe the resolution to the project/problem that was developed and implemented. 

Date: Enter the date the project was completed or the problem was resolved. 

Time: Enter the time the project was completed or the problem was resolved. 

91 
CHAPTER 7 

HUMAN FACTORS AUDIT PROGRAM 

FOR MAINTENANCE 

Steven G. Chervak and Colin G. Drury, Ph.D. 

State University of New York at Buffalo 

7.0 I'IW.IU ·1 OB.II· (' 1'1\ I. \\f) 

('()\ 1'1· \'1 

This project's objective was to provide a valid, 

reliable, and usable tool for evaluatmg human 

factors in maintenance tasks. The project was part of a 

broader initiative to apply human factors to reduce 

human error potential in aircraft inspection and 

maintenance. As Drury (1994) pointed out, there is a 

need to move from project-level interventions, such as 

better lighting, workcards and training, to higher-level 

process interventions. Two high-level interventions in 

this phase of the FANAAM project were (a) to 

provide a tool for assessing the current state of human 

factors/ergonontics in the hangar (this project) and (b) 

demonstrating a team apptoach to ergonontic 

interventions. 

The need for an ergonomics evaluation system has 

been apparent for some time, and manufacturing audit 

programs have been developed (e.g., Drury, 1990) to 

provide a rapid overview of factors likely to impact 

human/system ntismatches at each workplace. In the 

aircraft inspection context, there is no fixed workplace, 

so any andit program has to start with the workcard, 

rather than the workplace, as the basic unit Such an 

auditing system was produced in conjunction with two 

airline partners (Lofgren & Drury, 1994) and tested for 

both large airliners and helicopters. The system was 

tested for reliability, and modified where needed, 

before being validated against human factors expert 

judgments. Significant agreement was fouud between 

the two cases. The system can he used from either a 

paper data collection form (with later data entry) or 

directly from a portable computer. The computer is 

used to compare the data collected against appropriate 

standards and to print a report suitable for use in an 

existing airline audit environment. The report allows 

the airline to direct ergonomic changes to major 

ntismatches. 

93 

The scope of this report was to use the Ergonontic 

Audit for Aircraft Visual Inspection as a starting point 

for improvement and refinement to produce an 

Ergonontic Audit for Aircraft Maintenance (EAAM). 

This report details the differences and similarities 

between the two programs and the process used to 

develop the new program/user interface. The EAAM 

was designed to give an overall, generalized 

assessment of ergonomic factors applicable to 

performing a maintenance task. Program input and 

output were formatted in a way a person unfamiliar 

with details of the science of ergonontics could 

understand. This meant the program bad to he easy to 

use, had to help guide the person doing the audit 

through the steps with relative ease, had to describe 

less-familiar ergonomic principles, and had to allow 

the user to access on-line help when questions arose. 

The results had to he printed in an easily usable form 

appropriate to the organization's needs and free from 

unnecessary technical terminology. As with the 

inspection ergonontics audit, the project's overall aim 

was to discover human/system mismatches, not to 

provide prescriptive solutions to problems. Prescriptive 

solutions still require the depth of ergonomic 

knowledge, which is best provided by a trained 

ergonontist. • 

A task description of a generic maintenance task must 

he developed and compared to that of an inspection 

task in order to deterntine both differences and 

similarities between the two. Once these differences 

and similarities have been identified, the inspection 

audit can he modified to accommodate differences and 

to provide an accurate tool with which to begin the 

ergonomic audit and, eventually, the correction 

process. 

From detailed task descriptions and task analyses of 

inspection activities, Drury, Prabhu and Gramopadhye 

(1990) developed a generic function description of 

inspection (Table 7 .I, next page). These descriptions 

have been used throughout the FANAAM project to 
r 

I 

. I >I 

m ' 

11 

Human Factors Audit Program for Maintenance Chapter 7 

structure inspection interventions (Drury, 1994). Now 

that these descriptions are to be extended to 

maintenance tasks, a series of tasks were observed at 

the airline partner's maintenance facility. From these 

observations, we developed the equivalent set of 

generic functions for maintenance shown in Table 7.2. 

ln~iate Read and understand workcard. 

Select equipment. 

Calibrate equipment. 

Tables 7.1 and 7.2 clearly show the many areas of 

overlap between the two activities. Initiate (workcards, 

preparation), parts of Access (getting to the worksite 

with appropriate equipment), Buy-Back and Respond 

(final paperwork) have close parallels in these 

activities. Other major functions are different, but have 

the same ergonomic concerns. For example, the Search 

function of inspection depends on good lighting (at 

least for visual inspection) as do the Diagnosis and 

Replace/Repair functions of maintenance. Still, other 

functions are different between inspection and 

maintenance. For example, Opening/Closing access 

can require hand or power tools, while Replace/Repair 

can involve high levels of force exertion or manual 

lifting: none of these are typically part of inspection. 

Access 

Search 

Decision 

Respond 

Buy-Back 

Locate area on aircraft. 

Move to worksite. 

Position self and equipment. 

Move eyes (or probe) across area to 

be searched. Stop H any indication. 

Re-examine area of indication. 

Evaluate indication against standards. 

Decide whether indication is defect. 

Mark defect indication. 

Wr~e up non-routine repair (NRR). 

Return to search. 

Examine repair against standards. 

Sign off if repair meets standards. 

ln~iate 

Site Access 

Part Access 

Diagnosis 

Replace/Repair 

Reset Systems 

Close Access 

Respond 

Read and understand workcard. 

Prepare tools, equipment. 

Collect parts, supplies. 

Inspect parts, supplies . 

Move to worksite with tools, equipment, parts, supplies. 

Remove ~ems to access parts. 

Inspect/store removed ~ems. 

Follow diagnostic procedures. 

Determine parts to replace/repair. 

• 

Collect and inspect more parts and supplies. 

Remove parts to be replaced/repaired. 

Repair parts, H needed. 

Replace parts. 

Add fluids supplies. 

Adjust systems to specHication. 

Inspect adjustments. 

Buy-back, H needed. 

Refit ~ems removed for access. 

Adjust items refitted. 

Remove tools, equipment, parts, unused supplies. 

Document repair. 

94 
J 

Chapter? 

The implication of these differences was that the audit 

system for aircraft inspection had to be changed, 

primarily by adding modules to cover maintenance 

tasks. While this change was being introduced, the 

opportunity was taken to reconfigure the user interface 

of the whole data collection and analysis program, 

using a more modem Windows-based programming 

language. 

7.1 STRl CTl RE 01· TilE \l DIT 

An audit program consists of d~ta collection, data analysis, and results presenlat!on. Data collecnon 

involves a series of structured job observations and 

recording these observations. Data analysis has a data 

input step, and a step where data are compared with 

human factoiS slat!dards and good practice. Finally, 

results presentation takes conclusions drawn from the 

data analysis and provides them to the user in a useful 

format. Each step can be either a pencil-and-paper 

activity or a computer-based activity. The audit 

program previously developed for aircraft inspection 

and the one developed here for maintenance tasks have 

only specified computer-allocation for the analysis and 

results presentation steps. Data collection can either 

use hard-copy forms or a portable computer, 

whichever best fits with the organization's needs. In 

practice, many organizations prefer to use a form for 

initial data collection so as to have a permanent record 

in a highly reliable medium. Data entry then consists of 

transferring data from the paper form to its mimic on 

the computer's data input module. 

The audit program for maintenance inspection was 

developed for an mM peiSOnal computer as an 

integrated program called EAAM. As with the 

inspection audit program (ERGO), a number of 

featores were required to ensure that the system gave 

maximum benefit to the user population, typically, 

maintenance superviSOIS or quality auditors. Any audit 

program (Koli & Drury, 1995) must 

• be modular, so as to include maximum coverage 

without unnecessary length; inserting new 

modules to modify the checklist and program for a 

particular industry is straightforward 

• be self-explanatory, so as to minimize training 

time for auditors 

• be based on standards from ergonomics/human 

factors 

95 

Human Factors AudiiProgramfor Maintenance 

• have standards bnilt into the analysis program, 

rather than into the checklist, to reduce any 

tendency to "bend" data in borderline cases 

• rely on measurements and easily observable 

conditions to reduce judgment errors 

• be usable in different aviation environments, e.g., 

large fixed-wing aircraft, general aviation aircraft, 

or rotary wing aircraft, and in different 

maintenance situations, whether line maintenance 

or hangar maintenance. 

In addition, a structore was required to group audit 

modules by the human factors principle involved, 

rather than by generic function. The functions listed in 

Table 7.2 (previous page) ensure that coverage is 

achieved, i.e., all issues which should be raised are 

indeed part of the audit system. Structure in the 

program should group together the relevant issues. For 

example, the visual environment is important in a 

number of functions of Table 7.2 (previous page), e.g., 

Part Access, Diagnosis, ReplaceiRepair, Close Access, 

but the issues are constant, i.e., the amount aud quality 

of lighting. However, the visual environment is only 

one type of environment; there are thermal and 

auditory environments, .as well. Thns modules are 

grouped in a classification scheme using the following 

four major groupings, following Prabhu and Drury 

(1992) and Latorellaand Drury (1992): 

• Information Requirements - documents, 

communication 

• Environment- visual, auditory, thermal 

• 

• Equipment/Job Aids - design issues, availability, 

standards 

• Physical Activity/Workspace- access, postore, 

safety. 

This classification formed the basis of the ERGO 

program and was retained for EAAM. 

A second classification scheme was used to reflect the 

audit program's actual employment Some factors do 

not change during the job and can be conveniently 

evaluated before the job begins. e.g., workcards' 

quality. Other factors need the job to be in progress 

before they can be measured, e.g., forces, noise levels, 

or task lighting. The only module which has to wait for 

job completion is the evaluation of feedback to the 
Human Factors Audit Program for Maintenance 

mechanic. Thus, the audit is divided for convenience 

into three phases: 

Pre-Maintenance 

Maintenance 

Post-Maintenance. 

Table 7.3 shows how various modules are classified by 

ergonomics gi-ouping and phase of audit. Clearly, there 

are far more physical activity modules in this system 

than were necessary in the inspection audit program. 

7.1.1 The Audit Program 

The audit program for maintenance (MAINAUD) 

will produce a printed form for data entry, referred to 

as an Audit Checklist (see Chapter 7- Appendix). 

The data entry/data analysis/results presentation 

program (EAAM) reused some of the inspection 

audit's background data and calculations, e.g., in the 

enVironment modules. However, we took the 

opportunity to reprogram the whole audit system in 

Visual Basic 3.0, instead of Turbo Pascal6.0. Turbo 

Pascal is a structured, high-level language with 

multiple overlapping windows, mouse support, a 

multi-file editor, and an enhanced debugging facility. 

Visual Basic includes these factors and has greater 

mouse support abilities, is more user-friendly, and 

can more easily be expanded to incorporate the 

changes that may occur in the future. The advantage 

of Visual Basic is that it allows a programmer to 

create a program that a person with very little 

Chapter? 

computer experience can use with relative ease. 

Visual Basic also allows the flexibility of having the 

final program run on a conventional computer with 

keyboard and mouse as input or on a pen-based 

computer system with stylus input. Visual Basic 

objects, once defined and coded, can be reused in 

other programs, saving coding effort and reducing 

coding errors. We chose Visual Basic because of the 

similarity of its user interface to other Windowsbased programs. It uses many of the same symbols 

for execution as the popular Microsoft programs such 

as Word, Excel, or Office. A person familiar with any 

of these programs should have no problem 

recognizing similarities in Visual Basic and adapting 

to the Maintenance Audit program, EAAM. 

The Title Screen (Figure 7.1, next page) has an 

attached HELP system to provide assistance in using 

the program. At this level, the HELP screen offers a 

program overview and explanation. Next, heading 

information is required, e.g., the name of the job, the 

date, the analyst's name, etc. (Figure 7 .2, page 98). 

The files for input and report document are specified 

here. 

The main program screen lists the modules available 

and asks the analyst to choose those relevant to the 

current job audit. Once the analyst chooses a set of 

modules, each module is presented (Figure 7.3, page 

98), in turn, from the Pre-Maintenance phase through 

the Post-Maintenance Phase. Each module (e.g., 

Information 

Requirements 

1 . Documentation 

2. Communication 

6. Documentation 23. Buy-Back 

7. Communication . 

Environment 3. Visual 

Characteristics 

B. Task Lighting 

Equipment/ 

Job Aids 

Physical Activtty 

Workspace 

4. Equipment Design 

5. Access Equipment 

9. Thermal Characteristics 

10. Thermal Perception 

11. Audttorv Characteristics 

12. Equipment Availability 

13. Access Availability 

14. Hand Tools 

15. Force Exertion 

16. Manual Materials Handling 

17. Vibration 

1 B. Repet~ive Motion 

19. Physical Access 

20. Posture 

21. Safety 

22. Hazardous Materials 

96 
J 

Chapter 7 

Figure 7 .4, page 99) requires a series of 

measurements or classifications. A context-sensitive 

HELP screen is available for each module; it gives 

detailed explanations of terms used and of 

meaSurement procedures (Figure 7 .5, page 99). This 

practice follows the recommendations of Patel, Drury 

and Lofgren (1994) for workcards in that it supports 

different kinds of users, from novice to expert. Each 

module.also provides a comment screen (Figure 7.6, 

page I 00) to allow the analyst to record comments or 

notes. 

97 

Human Factors Audit Program for Maintenance 

As each module is run, its data are stored in the file 

the user specified in the heading information screen. 

When all modules have been run, the final report 

doc~ment is produced, with instructions on how to 

obtain a hard copy through Windows software 

(Figure 7.7, page 100). 
Human Factors Audit Program for Maintenance 

Matn Screen lable of Contents 

Click on Modules in which you would like to report. 

I. Premaintenance Phase 

~.~, ... l. .. _ .. P.®.~.~~-~- ····- t.tod. 2 Com11t00ication 

Nod. 3 Visual Characteristics 

Mod. 4 Electricei/Pneumatic Equipment Issues 

Mod. 5 Access 

II. Maintenance Phase 

Mod. 6 DocW~enletion 

Nod. 7 Comraunicalion 

Mod. 8 Task lighting 

Mod. 9 Thermal Characteristics 

Mod. 10 Operator Perception of TheriHI EnviOIIIIM!ftl 

Mod. 11 Auditory Char ecteristics 

Nod. 12 Electricai/Pneuraatic Equipment Usage 

Nod. 13 Access Equipment 

Nod. 14 Hand Tools 

Mod. 1 5 Force Exertion 

• 

I Select All 

Ill. Postmaintenance Phase . 

I LM-od ___ 23 __ B•_•·_B~--k-----------------------~-- I ~ 

Figure 7.3 Ma1n Program Screen 

'IX 

Chapter 7 
Chapter 7 Human Factors Audit Program for Maintenance 

I 

I 

Celcius to Fahl"enheit Conversion • 

F 

To do conversion dick on the arrowl 

99 
Human Factors Audit Program for Maintenance Chapter 7 

~~ Comments DDI 

T Jlpe ,our commenls lor Module 1 - DOCUMENTATJ:OJC heJe. 

l 

Return 

Figure 7.6 Comment Screen 

!00 
Chapter?' 

This program is designed to be run on any ffiM 

Personal Computer with at least an INTEL 386 

processor, 4MB of RAM, DOS 5.0, and WINDOWS 

3.1. The program itself occupies 2 MB of hard disk 

space in its stand-alone form. If a user desires to 

input data directly from the job into the program, a 

portable computer is necessary; otherwise, a desktop 

machine is fine. The program can also be run on penbased computers with WINDOWS compatibility. 

[Incidentally, the inspection audit ERGO can also run 

on pen-based systems.) 

The modules available in EAAM are as follows: 

Pre-Majntenance J>bag 

MODULE IDOCUMENTATION 

Information Readability; Information 

Content, i.e., Text & Graphics, and 

Information Organization. 

MODULE2-

COMMUNICATION 

Between-shift cohununication, availability 

of lead mechanics and supervisor for 

mechanics' questions and con<:ems. 

MODULEJ.. 

VISUAL CHARACI"EIIISTICS 

Overall lighting characteristics of the 

hanger, i.e., overhead lighting, condition of 

overhead lighting, and glare from daylight. 

MODULE4-

ELECTRIC/PNEUMATIC 

EQUIPMENT DESIGN ISSUES 

Evaluation of the equipment which uses 

controls, i.e., ease of control, intuitiveness 

of controls, labeling of controls for 

consistency and readability. 

MODULESACCESS EQUIPMENT 

Evaluation ofladders and scaffold for 

safety, availability, and reliability. 

Maintenance Phase 

MODULE6-

DOCUMENTATION 

Physical handling of documents and the 

environmental conditions effecting the 

101 

Human Factors Audit Program for Maintenance 

documents' readability, i.e., weather and 

light. 

MODULE7-

COMMUNICATION 

Communication issues between co-workers 

and supervisors, and whether or not 

suggestions are considered. 

MODULESTASK UGHTING 

The overall lighting available to the 

mechanic for completing the task. Evaluates 

points such as light levels, whether personal 

or portable lighting is used, and whether 

lighting equipment causes interference with 

the work task. 

MODULE !ITHERMAL CHARACTERISTICS 

The current thermal conditions the task is 

being performed in. 

MODULE tOOPERATOR PERCEPTION OF 

THERMAL ENVIRONMENT 

Operator perceptions of the work 

environment at present, during the summer, 

and during the winter. 

MODULEllAUDITORY CHARACTERISTICS 

Determine if sound levels in the current 

work environment will cause hearing loss or 

interfere with tasks or speech . • 

MODULE12-

ELECTRICAUPNEUMATJC EQUIPMENT 

Availability of any electrical/pneumatic 

equipment, whether the equipment is 

working or not, and ease of using the 

equipment in the work environment. 

MODULE13-

ACCESS EQUIPMENT 

Availability of ladders and scaffolds, 

whether the equipment is working or not, 

and ease of using the equipment in the work 

environment. 

MODULE14-

HANDTOOLS 

Evaluates the use of hand tools, whether 

hand tools designed properly to prevent 
Hunum Factors Audit Program for Maintenance 

fatigue and injury, and usability by both 

left· and right-handed people. 

MODULE ISFORCE EXERTION 

Forces exerted by the mechanic while 

completing a maintenance task. Posture, 

hand positioning, and time duration are all 

accounted for. 

MODULE16-

MANUAL MATERIAL HANDUNG 

Uses NIOSH 1991 equation to determine if 

the mechanic is handling loads over the 

recommended lifting weight. 

MODULE17· 

VIBRATION 

Amount of vibration a mechanic encounters 

for the duration of the task. Determines if 

there are possible detrimental effects to the 

mechanic because of the exposure. 

MODULE18-

REPETJT/VE MOTION 

The number and frequency limb angles 

deviating from neutral while performing the 

task. Takes into consideration arm, wrist, 

shoulder, neck, and back positioning. 

MODULE19-

ACCESS 

Access to the work environment. Whether it 

is difficult or dangerous, if there is conflict 

with other work being performed at the 

same time. 

MODULEZOPOSTURE 

Evaluates different whole-body postures the 

mechanic must assume in order to perform 

the given task. 

MODULE21· 

SAFE7Y 

Examines safety of the work environment 

and what the mechanic is doing to make it 

safer, e.g., personal protective devices. 

MODULE22-

HAZARDOUS MATERIAL 

Lists types of chemicals involved in the 

maintenance process, whether they are being 

used properly, if workers are following 

102 

Chapter? 

disposal guidelines, if the company is 

following current EPA requirements for 

hazardous material safety equipment. 

Post-Maintenance fbase 

MODVLE23-

BUY-BACK 

Usefulness of feedback information to the 

mechanic and whether buy-back is from the 

same individual who assigned the work. 

7.1.2 Audit Program Evaluation 

The EAAM program is only part of an audit system. 

Suitable jobs must still be chosen for auditing, using 

some sampling plan. The output from the audit must 

be incorporated into a management structure which 

will use it effectively to improve job design. None of 

these issues are essentially different from tbe 

equivalent issues for inspection, so they will not be 

repeated here. Koli and Drury (1995) give details of 

these procedures. More detail and a discussion of 

their relationship to the broader field of human 

factors can be found in Koli (1994). 

Any tool designed for human use should be evaluated 

for its fit to human capabilities and limitations; this is 

a basic principle of ergonomics. The audit program 

for maintenance tasks is such a tool, and, like its 

predecessor for inspection, had to be evaluated. Koli 

and Drury ( 1995) tested the inspection audit program 

ERGO for reliability, i.e., whether different analysts 

auditing the same job obtain the same results. That 

reliability stu~ used three jobs, two on a DC-9 

inspection and one on a Sikorski S-58T inspection. 

There were significant differences between the two 

auditors tested. On further analysis, these differences 

were shown to be due mainly to inputs requiring 

auditor judgment. These inputs were modified to 

reduce the need for judgment. The program was 

retested on another DC-9 task, showing no significant 

differences this time between auditors. 

Validity of a tool measures whether the tool gives the 

same output as another trusted tool. Koli ( 1994) 

tested the validity of ERGO by compating its outputs 

to those of six ergonomics experts viewing a video 

tape of a DC-8 power plant inspection. The audit 

program always found at least as many ergonomic 

issues as any expert, and no issues found by the 

experts were missed by ERGO. 
' 

Chapter? 

The current program was tested for both reliability 

and validity in the same way. In addition, its interface 

was tested for usability, using standard human factors 

usability testing techniques (McClelland, 1990). 

Initially, a single user was observed and questioned 

while using the audit program, partly to assess its 

usability and partly to develop more detailed 

measures of interaction between the user and the 

program., The particular user was a member of the 

quality assurance department who regularly 

performed safety audits and occasional ergonomics 

audits. Following this analysis, a more detailed 

observation protocol was developed for usability 

testing on four other members of the user population. 

7.2 RFLI \BILI'I) E\ \Ll \TIO'\ 

Two analysts observed four different maintenance 

tasks on DC-9 aircraft at the airline partner's 

maintenance base. The tasks were the following: 

I. Replace overhead passenger service unit 

2. Close keel box 

3. Close forwartl cargo compartment access 

4. Replace escape window 

For each task, analysts used the paper data collection 

Human Factors Audit Program for Maintenance 

form as a more severe test of the audit. Direct 

computer entry of data would have given access to 

HELP screens. However, since at least some users 

will want to use paper data entry, this form was used 

as a worst case. Each analyst recorded answers for 

each question in each module independently for later 

comparison. The number of questions differed 

between the four tasks, as different modules applied 

for each task. Note that lillY. difference in results 

between the analysts was counted, whether it affected 

the audit outcome, or not. 

The total number of differences between the two 

analysts' data sheets were tallied; the results are 

shown in Table 7.4. Also shown in Table 7.4 is a X' 

test of the hypothesis that the number of errors is 

equal to zero. This is a very stringent test: for 125 

questions only four differences would be needed to 

conclude that the number of errors was significantly 

different from zero. 

As with the initial reliability study of the Inspection 

Audit, the audit for maintenance was not reliable 

enough, averaging 85%. The Cochran Q test, a robust 

and strong test of the differences between auditors 

used to evaluate the reliability of the Inspection 

Audit, was performed on each task to determine the 

Table 7.4 Rel1ab1l1ty Data on Ma1ntenance Aud1t for,;our Tasks . 

1. Replace overhead 118 12 90% 12.6 <.001 

passenger service unit. 

2. Close keel box. 163 22 87,% 23.6 <.001 

3. Close forward cargo 159 27 84% 24.5 <.001 

compartment access. 

4. Replace escape window. 134 24 83% 26.4 <.001 

Table 7.5 Results of 0 Test on Ma1ntenance Aud1t Results ~- '~ ~ 'i '~**"\"'*~*~ 

' j " ' 

1. Replace overhead 10 1.60 >0.25 (ns) 

passenger service unit. 

2. Close keel box. 14 7.14 <0.01 

3. Close forward cargo 10 0.40 >0.25 (ns) 

compartment access. 

4. Replace escape window. 12 0.33 >0.25 (ns) 

103 
'' 

il ' 

I 

Ci 1 

I' 

Human Factors Audit Program for Maintenance 

agreement between auditors in terms of output 

results. For example, if the percent of time the 

mechanic spent in a particular posture is estimated as 

10% by one analyst and 20% by the other, but both 

results lead to the same outcome, a difference was 

not scored. Table 7.5 (previous page) shows the 

results of this test. 

The statistic values show significant differences 

between the two analysts for one of the tasks, with a 

magnitude similar to those reported for the same test 

of the Inspection Audit. However, the non-significant 

findings on three of the four tasks showed that even 

the first version of this maintenance audit had been 

based on lessons learned in the inspection audit. Note 

that the number of outcome differences was 

considerably smaller than the number of recording 

differences. Defined on outcomes, reliability was in 

fact 92%. 

These reliability results can be analyzed in more 

1 Documentation 

2 Communication 

3 Visual Characteristics 

4 Electric/Pneumatic Equipment 

Issues 

5 Access Equipment 

6 Documentation 

7 Communication 

8 Task Lighting 

9 Thermal Characteristics 

10 Operator Perception of Thermal 

Environment 

11 Auditory Characteristics 

12 Electrical/Pneumatic Equipment 

13 Access Equipment 

14 Hand Tools 

15 Force Exertion 

16 Manual Material Handling 

17 Vibration 

18 Repetitive Motion 

19 Access 

20 Posture 

21 Safety 

22 Hazardous Material 

Chapter? 

detail to determine the cause of each difference and, 

hence, be used directly to modify the EAAM audit 

program. Each difference was classified as one of the 

following: 

.Iud&ment Error W- A magnitude had to 

be judged by the analyst, e.g., Was handling 

the workcard difficult? 

Definition Error IDl- A lack of definition 

of terms resulting in different assumptions 

by different analysts, e.g., Does the working 

day include lunch break (8 hrs) or no lunch 

break (7 hours)? 

No Help on Form <Hl- Errors where help is 

available on the program but not on the 

form, e.g., What is ulnar deviation of the 

wrist? 

Non-Observation !Nl- Where one analyst 

observed an activity, but the other did not, 

e.g., Is shift change work documented? 

Other Errors !Ol- All other errors, e.g., 

2 1 3 

3 3 

4 4 2 10 

Design 0 

6 6 

3 .- 2 5 

0 

2 2 4 

3 3 

0 

2 3 5 

3 2 1 6 

1 1 

4 5 1 10 

1 2 3 

0 

1 1 

7 7 

2 1 3 

3 3 

1 1 2 

0 

104 
Chapter 7 

where one analyst states that the hand tool 

requires a power grip, while the other 

analyst records nothing. 

Table 7.6 (previous page) shows the number of each 

type of difference counted for each module of the 

audit. As can be seen, 70% of all differences were 

either judgment or definition related. Changes to 

improve the reliability of these questions are 

relatively simple, either by replacing judgment with 

measurement or by adding/refining definitions. A 

further 12% of the differences were due to no help 

facility on the data collection form. Specific helpful 

expansions can be provided on the form to improve 

reliability here, too. Non-observation errors and other 

errors perhaps represent a minimum of errors (less 

than 2% of responses) which are not simple to 

correct. 

105 

HuTMn Factors Audit Program for Maintenance 

Overall reliability was in the same range as the initial 

version of the Inspection Audit. Specific changes 

were made to the program and to the data collection 

form to secure the improvements required. 

Version 2.0 of the Audit Program for Maintenance 

was developed and retested on a single job with the 

same two analysts. The rewording of questions 

involved 9 of the 228 questions in EAAM. -The retest 

was performed on the task "Replace first class seats" 

on a DC-9. Results of the X' test and Cochran's Q test 

are shown in Tables 7.7 and 7.8, respectively. 

The reliability is now much higher at 93% when 

calculated on number of differences and the same at 

93% when calculated on number of different 

outcomes. At this point the reliability was considered 

to be established. 
r 

II! 

1

1 I 

i' 

Human Factors Audit Program for Maintenance 

7.3 VALIDITY OF EIU;()NOI\IIC 

'AUDIT FOR AIRCR\FT 

\L\INTE!\ \NCE 

The ergonomic audit program was developed as a 

rapid screening tool to identify ergonomic 

mismatches in aircraft maintenance tasks. The 

majority of people using this audit program will 

have little training and expertise in ergonomics. In 

order to evaluate the effectiveness of the program in 

finding ergonomic mismatches, we compared the 

results of the audit program to those of four 

practitioners in the field of ergonomics. The task 

chosen was a Aileron Removal on the left wing of a 

DC-9 aircraft. This task was audited using the EAAM 

program and simultaneously videotaped for later 

analysis by the ergonomic practitioners. 

The EAAM program found 55 ergonomic issues 

which needed to be addressed. The issues were 

classified into I 0 different categories listed in Table 

7.9. 

Method: A group of four ergonomic practitioners, 

all professors actively involved in conducting 

ergonomic assessments, were provided with the 

necessary documentation required to complete an 

aileron removal. They were each asked to view the 

video tape made of the aileron removal and evaluate 

all aspects of the task, operator, equipment, 

documentation, and environment that they would 

address in evaluating the system for possible human 

factor mismatch (Koli, 1994). 

106 

Chapter? 

Results: The results of the four subjects and that of 

the checklist are listed in Table 7.10 (next page). 

Note that in some cases, for example 

"Communication", the practitioners raised more 

issues than the checklist. These "extras" were false 

alarms, where the maintenance task met the standards 

even though the practitioners thought it did not. 

To determine whether the checklist produced more or 

less overall ergonomic issues than the practitioners, 

the differences between the checklist and the mean 

number of issues found by practitioners were 

analyzed using at-test. The value of the !-statistic 

was t = 4.57, which was significant at p < 0.01. This 

indicates that there is considerable difference 

between the evaluation of the checklist and that of 

the practitioners, and that the checklist found more 

issues. 

The relatively poor performance of the practitioners 

when compared to that of the checklist arises from 

various sources. First, there is a trade-off between 

direct observation and videotape. Doing analyses by 

direct observation allows the analyst to move around 

for the best view and to use three dimensional cues. 

This inflexibility of movement and unconscious 

editing by the cameraman performing the video 

taping could have resulted in loss of certain 

information. One advantage of videotape analysis is 

the analyst can play a segment over or freeze action 

in order to analyze a situation more closely, but only 

one practitioner used this facility. A second reason 

why the checklist outperformed the practitioners is 

because it had been evolved by studying the task 

domain over an &tended period of time. All aspects 

of the maintenance task were thoroughly investigated 

before the development of the exhaustive checklist. 

In other words, the checklist was developed 

specifically for aircraft maintenance tasks. The 

practitioners, on the other hand, had to rely on 

memory to identify the issues. 

Overall, the checklist fared as well as, indeed better 

than, ergonomic practitioners at identifying 

ergonomic mismatches. However, one issue 

involving safety was brought up by practitioners 

which was not identified directly by the EAAM 

audit: Safety aspects of the mechanics movements. 
Several of the auditors made reference to one of the 

mechanics' "jumping" back and forth between two 

ladders in order to complete the aileron removal. The 

ergonomic audit program does not djrectly address 

the issues of safety in personnel movement, but does 

however ask general safety questions of maintenance 

personnel. For example, "bo you feel access to the 

work area is dangerous'?" or " Do you feel access to 

the work area is difficult?". This audit was designed 

so that such general questions would raise awareness 

of a broader degree of personal safety issues, which 

could then be further investigated by ergonomic 

practitioners. 

7.-t IT\ \L \IODIFIC.\TIO:\S TO THE 

\I \1:\Tl·.:\ \:\CE \l DIT 

On the basis of the high reliability and validity 

demonstrated by the Maintenance Audit 

system, no further modifications were made in 

structure or content. Some interface changes have 

been made by Galaxy Scientific Personnel, but these 

changes do not affect reliability or validity. For 

1995/96, it is expected that the Inspection Audit 

(ERGO) and the Maintenance Audit (EAAM) will be 

combined with earlier audits into a single audit 

program. 

7.5 HUFHE:\CtS 

Drury, C. G. (1994). Ergonomics on the hangar floor. 

In Proceedings of the Human Factors and 

Ergonomics Society 38th Annual Conference, 

Seattle, WA, 106-110. 

107 

Human Factors Audit Program for Maintenance 

Drury, C. G. ( 1990). The information environment in 

aircraft inspection. In Final Report - Second 

Federal Aviation Administration Meeting on 

Human Factors Issues in Aircraft Maintenance 

and Inspection, Information Exchange and 

Communications, 98-109. 

Drury, C. G., Prabhu, P. and Gramopadhye, A. 

(1990). Task Analysis of Aircraft Inspection 

Activities: Methods and Findings, In 

Proceedings of the Human Factors Society 34th 

Annual Conference, Santa Monica, California, 

1181-1185. 

Koli, S. (1994). Ergonomic Audit for a NonRepetitive Task. Unpublished Masters Thesis. 

State Unive:.Sity of New York at Buffalo. 

Koli, S. and Drury, C. G. (1995) in press. Ergonomic 

Audit for Visual Inspection of Aircraft, Human 

Factors in Aviation Maintenance- Phase Four, 

Progress Report, DOT/FAA/AM-93, National 

Technical Information Service, Springfield, VA. 

Latorella, K. A. and Drury, C. G. (1992). A 

framework for human reliability in aircraft 

inspection, In Meeting Proceedings of the 

Seventh Federal Aviation Administration 

Meeting on Human Factors Issues in Aircraft 

Maintenance and Inspection, Atlanta, GA, 71-

82. 
I ' I 

I' 

Human Factors Audit Program for Maintenance 

Lofgren, J. and Drury, C. G. (1994). Human Factors 

Advances at Continental Airlines. In 

Proceedings of the 8th FAAIAAM Meeting on 

Human Factors in Aviation Maintenance and 

Inspection "Trends and Advances in Aviation 

Maintenance Operations", Alexandria, VA, 

November 16-17, 1993, 117-138. 

McClelland, I. ( 1990). Product assessment and user 

trials. In Wilson, J. R. and Corlett, E. N. (Eds. ), 

Evaluation of Human Work, London: Taylor & 

Francis, 218-247. 

Patel, S., Drury, C. G. and Lofgren, J. (1994). Design 

of workcards for aircraft inspection. Applied 

Ergonomics 1994, 25(5), 283-293. 

Prabhu, P. and Drury, C. G. (1992). A framework for 

the design of the aircraft inspection information 

environment. In Meeting Proceedings of the 

Seventh Federal Aviation Administration 

Meeting on Human Factors Issues in Aircraft 

Maintenance and Inspection, Atlanta, GA, 83-

92. 

Chapter7 

• 

108 
Chapter 7 - Appendix 

Audit Checklist 

109 
r 

Appendix - Audit Checklist Chapter? 

~TENANCEPREPARATION 

A. Information Requirements 

MODULE 1. DOCUMENTATiON (Work Cards) 

a. Information Readability 

1.1s the text layout of this workcard consistent with the other workcards? (YIN) __ 

2. Is the text materia/justified to the left margin? (YIN) __ 

3. Aie typographic cues used for segregating important text material in the workcard? (YIN) __ 

4. Has a simple block font style been used to print this workcard? (YIN) __ 

5. Aie dot-matrix printers used for printing workcard? (YIN) __ 

6. If yes, its resolution matrix is: 

(alb/c)_ 

a.5X5 

b.5X7 

c. 7 X 9 or higher 

7. Aie the graphics/attachments legible with reference to print quality? (YIN)_ 

8. Aie there time & quality standords for changing printer ribbons & toner cartridges? (YIN) __ 

~ If yes, are the standards .obeyed? 

10. Have acronyms/abbreviations been used in the workcard? 

11. If yes, how many for the entire task? a. less than five? 

b. greater than five? 

b. Graphics 

• 

12. Is spatial information of body station positions presented in pictorial form? 

13. How are figures represented? a. Perspective(3-Dimensional) 

b. mode in which the user sees it 

14. Do figures have back references to workcard? 

15. Aie figures/graphics for mirror-image tasks separately drawn? 

16. In figures/graphics, are close-up views distinguished from distant views? 

110 

(YIN)_ 

(YIN)_ 

(alb) -

(YIN)_ 

(alb) -

(YIN)_ 

(YIN)_ 

(YIN)_ 
I 

I 

I 

t 

l 

Chapter7 

MAINTENANCE PREPARATION 

c. Information Organization 

17. Is there a definite ordering/sequencing of tasks? 

18. Does task information carry-over to the next page? 

19. What is the maximum number of tasks per action statement? 

MODULE 2. COMMUNICATION 

a. Shift Changes 

I. Is there an overlap of personnel to conununicate prior shift work? 

b. Work in Progress 

2. Is shift change work documented? 

3. If yes, are the written documents conununicating shift change, legible? 

4. Are the conununication channels evaluated for effectiveness? 

a. 2 

b.3 

Appendix - Audit Ch£cklist 

(YIN)_ 

(YIN)_ 

c. more than 3 (albic) __ 

(YIN)_ 

5. Is there an on-going program to maintain adequacy of conununication channels? 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(alb) -

.(YIN)_ 

6. Would the mechanic be considered A) Novice or B) Expert 

7. Is the Leadman available for questions by the mechanic? 

8. Is the Supervisor available for questions by the mechanic? (YIN)_ 

• 

Ill 
I J[ ,, ' " I~ I 

Appendix - Audit Checklist Chapter7 

MAINTENANCE PREPARATION 

MODULE 3. VISUAL CHARACTERISTICS 

I. What is the type of light source used for general illumination? 

a. incandescent 

b. fluorescent 

c. mercury-vapor 

d. high pressure sodium vapor 

e. low pressure sodium vapor (alb/c/d/e) __ 

2. If fluorescent bulbs are used, does flicker exist? (YIN) __ 

3. If fluorescent bulbs are used, are they installed in pairs? (YIN) __ 

4. Are lighting fixtures free/clean from dirt/paint? (YIN) __ 

5. Are illumination sources provided with shades or glare shields? (YIN)_._ 

6. Are all the illumination sources working? (YIN) __ 

7. Is there indirect glare from the source? (YIN) __ 

8. Is the general lighting source within the line of sight? (YIN) __ 

MODULE 4. ELECFRICAUPNEUMATIC EQUIPMENT DESIGN ISSUES 

I. Are controls requiring precision performed manually? 

2. Do selector switches have fixed scales and moving pointers? 

3. Are toggle switches used in sequence, mounted in a horizontal array? 

• 

4. Are controls labeled with all "words" or "symbols"? 

5. Are labels typographically consistent? 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

6. Do push buttons prevent slipping of fingers (eg., surface texture, shape of knob etc.)? (YIN) __ 

7. Do push buttons have an audible click or snap feel to indicate control action? 

8. Are edges of knobs, dials, switches or instrument rounded? 

9. Are labels readable in all weather conditions? 

10. Have abbreviations been avoided on labels wherever possible? 

II. Are emergency controls clearly distinguished from normal controls? 

12. If the control function is RAISE, is the movement of the control UP? 

112 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

., 
Chapter7 Appendbc - Audit Checklist 

MAINTENANCE PREPARATION 

13. If control function is ON, is movement RIGHT, CWCKWISE, FORWARD or PUSH?(YIN) __ 

14. If control function is INCREASED, is movement RIGHT, CWCKWISE or FORWARD?(YIN) __ 

15. If control function is RIGHT, is the movement RIGHT or CWCKWISE? 

. 16. If the control is RETRACT, is the movement UP, REARWARD or PUlL? 

MODULE 5. ACCESS EQUIPMENT- LADDERS, SCAFFOWS 

(YIN)_ 

(YIN)_ 

L Do ladderslscaffolds have non-skid suifaces on landings? (YIN) __ 

2. Do ladders/scaffolds have safety screens behind open stairs and at landings? (YIN) __ 

3. Do ladders have luurd rails? (YIN) __ 

4. What is tbe cross section of tbe hand rails? a. circular 

b. rectangular 

c. otiiCI" (albic) __ 

5. What is the angle of inclination of the ladder with the horizontal? A= 0 

6. What is the riser height? R= __ incbes 

7. What is tbe tread length? X= __ incbes 

8. If non-tread ladders are used: what is the distance between vertical rails? Y= inches 

9. If non-tread ladders are used: What is tbe cross section of the rungs? a. circular 

b. rectangular 

c. otber (albic)_ 

10. If non-tread ladders are used: What's the cross section of the vlrtical rails? a. circular 

b. rectangular 

c. other (alb/c) __ 

113 
Appendix - Audit Checklist 

MAINTENANCE PREPARATION 

ACCESS EQUIPMENT- PORTABLE LADDERS (Step Ladders & Tall Step Ladders) 

Step lsvttfm 

II. What is the height of the step ladder? 

12. Does the step ladder have non-slip treads? 

13. Does the step ladder have rubber feet? 

Tall Steo I •riders 

14. Does the tall step ladder have braces on the lower steps? 

15. Do the folding braces of the ladder have locking detents? 

114 

Chapter? 

H= __ inches 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 
Chapter? Appendix - Audit Checlclist 

~ANCESTAGE 

A. Information Requirements 

MODULE 6. DOCUMENTATION (Physical Handling&: Environmental Factors) 

1. When did the mechanic last peiform this task? a. a day ago 

b. a week ago 

c. a month or more 

2. Does the Mechaoic read the wOtkcatd? 

3. Do you feel the information content of the wotkcatd complete with respect to 

the scope of the task? 

4. Do you feel a novice mechanic can understand this current wOtkcatd? 

5. Do you feel there is any handling difficulty with respect to the size of the 

workcardlgraphic attachments while conducting maintenance? 

6. Do you feel there is adequate readability in the current light conditions? 

7. Is maintenance being conducted in conditions of: a. wind 

b. rain 

c. snow 

8. Does the mechanic sign-off the wotkcatd after each subtask? 

9. Do writing tools facilitate writing in all positions? 

MODULE 7. COMMUNICATION 

(Maintenance person to be asked the following questions) 

I. How easy is communication (work-related) with co-worker? a. very easy 

b. adequate 

(alb/c)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_· -

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

c. very difficult (alb/c) __ 

2. Did you get explicit verbal instructions from the supervisor? (YIN)_ 

3. How easy is communication with supervisor? a. very easy 

b. adequate 

c. very difficult (alb/c) __ 

4. Are you given feedback when you are not peiforming up to the standard? (YIN)_ 

5. Are you encouraged to help identify error likely situations in: a. existing design(YIN) __ 

b. maint. proc. (YIN) __ 

6. Are the suggestions reviewed? (YIN) __ 

115 
',,!'1 

~· ,! ' 

,:'· i j)llj 

Appendix - Audit Checklist 

Chapter7 

MAINTENANCE PHASE 

MODULE 8. TASK UGH17NG 

I. What type of worl: is being audited? 

a. otdinary maintenance 

b. detailed maintenance 

c. fine maintenance (albic) __ 

2. Does mechanic look from bright to datk places routinely? (YIN)_ 

3. Indicate the light levels taken from 4 zones during the task. Zone 1 =· ___ fc 

Zone 2- fc 

Zone3= fc 

Zone4- fc 

4. What type of light source is used as portable lighting equipment? a. hand lamp (YIN) __ 

b. standing lamp (YIN)_ 

5. What type of light source is used as persona/lighting equipment? a. 2D cell flashlight 

b. 3D cell flashlight 

c. 4D cell flashlight 

d.Headlamp 

e. Other (albic/die) __ 

6. Does the portable or personal lighting equipment interfere with 

the maintenance tuk? (YIN) __ 

7. Do you feel any difficulty in handling with respect to the size ol 

the lighting equipment? (YIN) __ 

8. Do you feel any difficulty in handling with respect to the weight 

of the lighting equipment? 

9. Do you experience discomfort glare from the task surface ? 

10. Do you experience discomfort glare from workcard surface? • 

II. Are there excessive contrasts between different colors in the task area? 

116 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 
Chapter7 Appendix - Audit Checklist 

NUUNTENANCEPHASE 

MODULE 9. THERMAL CHARACTERISTICS 

Measurement tools: Dry and Wet bulb thermometer and an anemometer to measure the wind speed. 

1. Describe the physical worlcloadlmuscular effort? a low 

b. moderate 

c. high (albic) _ 

2. What is the wind speed? _ Mph 

3. The air temperature is approximately? •p 

4. What is the Humidity of the hangar? 

MODULE 10. OPERATOR PERCEPTION OF THERMAL ENVIRONMENT 

This module evaluates the perceptions of the ooerators to climate changes. All the questions in this module are to 

be addressed to the inspector performing the task. 

I. How do you feel now? 

I 2 

I I 

hot warm 

3 

I 

slightly warm 

2. Indicate how you would like to be now? 

SUMMER 

4 

I 

neutral 

5 

I 

slightly cool 

a. warmer 

b. cooler 

c. no change 

Scale reading __ 

6 7 

I I 

cool cold 

(alb/c)_ 

3. How do you feel during summer? Scale reading __ 

I 2 3 4 5 6 7 

I ________ JI ________ ~I ________ JI _________ ,I __ • ______ ~I I 

hot warm slightly warm neutral slightly cool cool cold 

4. Indicate how you would like to be during summer? a. warmer 

b. cooler 

c. no change (albic)_ 

WINfER 

6. How do you feel during winter? Scale reading __ 

I 2 3 4 5 6 7 

I I 

hot warm slightly warm neutral slightly cool cool cold 

7. Indicate how would you like to be during winter? a. wanner 

b. cooler 

c. no change (alb/c)_ 

117 
Appendix • Audit Checklist 

MAINTENANCE PHASE 

MODULE II. AUDITORY CHARACTERISTICS 

Measurement Tools: Sound-level metet that measures sound in decibels. 

I. The noise levels recorded over the entire inspection task duration are: 

2. At each reading, the main source of noise from: 

a) pneumatic tools 

b)music 

C) COnVCINation 

d) engines 

e) passing aircraft 

f)othet 

3. What is the approximate exposure time to the existing noise levels? 

4. Does the maintenance person wear earplugs? 

5. Does the maintenance person wear earmuffs? 

Chapter7 

Reading# I __ dBA 

Reading# 2 __ dBA 

Reading# 3 __ dBA 

Reading# 4 __ dBA 

Reading# 5 __ dBA 

answer (a,b,c,d,e,f) 

Reading# I __ 

Reading# 2 __ 

Reading# 3 __ 

Reading# 4 __ 

Reading# 5 __ 

__ hours/day 

(YIN)_ 

(YIN}_ 

6. The mmcimum distance which the maintenance person needs to communicate verbally is? __ feet 

7. Is thete a high pitch noise component? (e.g.,. over 2000Hz) 

8. Is the main source of noise from other workstations? 

MODULE 12. ELECTRICAUPNEUMATIC EQUIPMENT 

A vailabilitv 

I. Is equipment available? 

2. Is the equipment working at all times? 

3. If no, are there any satisfactory substitute arrangements? 

4. Is electrical/pneumatic equipment easily maneuverable during maintenance? 

118 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 
Cbapter7 

NUUNTENANCEPHASE 

Disp!avs. Conlrols. Knobs 

5. Can you easily understand all the labels/display menus? 

6. Are conlrol elements easily differentiated by touch? 

7. Are conlrol movements as short as possible? 

8. Is there division of attention? 

MODULE 13. ACCESS EQUIPMENT 

Availabilitv 

I. Is correct access equipment available? 

2. If no, is satisfactory substitute equipment available? 

3. The access equipment is: a. fixed 

b. mavable 

c. both of the above 

4. If movable, is it easily maneuverable? 

MODULE 14. HAND TOOLS 

I. Is there shoulder adduction during tool operation? 

2. Is forearm fully extended during tool operation? 

3. Does tool operation involve noticeable: a) Wrist ulnar deviation? 

Appendix - Audit Checklist 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

albic)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

b) Wrist radial deviation? 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

c) Wrist flexion? 

d) Wrist extension? 

• 

4. Does the tool vibrate perceptibly? 

5. Can the tool be used by both left and right handed people? 

6. Does the tool handle end in the palm? 

7. For power tool, does the tool handle provide electrical insulation? 

8. Does the tool handle provide heat insulation? 

9. Does the tool handle have sharp edges or corners? 

10. Is the tool handle compressible? 

II. Is the tool handle hard enough to resist embedding of particles? 

12. Is the tool grip non-absorbent to sweat, oil, grease, etc.? 

119 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 
Appendix - Audit Checklist 

MAINTENANCE PHASE 

13. Is a heavy grip needed to avoid slippage? 

14. Arc there any unguarded pinch points on the tools? 

15. Arc there stops to prevent the handles from fully closing? 

·16. The type of activating trigger is : a. single finger? 

b. multiple finger strip? 

c. thumb? 

17. If a thumb operated trigger is used, is the thumb hyperextendcd? 

18. Is the trigger very frequently used? 

19. The grip on the tool is: a. pulp pinch 

b. lateral pinch 

c. power grip 

20. If the tool is heavy is it supported or counter balanced? 

120 

Chapter7 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(albic)_ 

(YIN)_ 
I 

Chapter7 

MAINTENANCE PHASE 

MODULE 15. FORCEEXEimON 

I. Does the task involve: 

2. Does the task involve use of 

3. Is the type of grip: 

4. V enicallevel of first force application : 

5. Muscle groups involved in the task: 

Horizontal pushing? 

Horizontal pulling? 

Venical pushing? 

V enical puUing? 

One arm? 

Both arms? 

a. power grip? 

b. hook grip? 

c. finger pinch grip? 

a. Above head height 

b. Head height 

c. Shoulder height 

d. Elbow height 

a. whole body 

Ap~ndbc - Audit Checklist 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(albic)_ 

(albic/d)_ 

b. primarily arm and shoulders (alb)_ 

(YIN) _ 

_ (Kg.) 

6.1s the person's arm moving while the force is being applied? 

7. What is the force being applied? 

121 
Appendix - Audit Checklist 

MAINTENANCE PHASE 

MODULE 16. MANUALMATER1ALHANDUNG 

1. Do loads have proper handles? 

2. Can these handles be used by the whole hand? 

3. If protective clothing is indicated, is it provided? 

4. Is the task area clear of obstructions? 

5. Is the floor clean, dry and non-slip? 

6. Is the area for setting down the load clear? 

NIOSH EQUATION 

I. What is the objects weight? 

2. Frequency of Task? 

3. Hand distance away from body at start? 

4. Hand height at start? 

5. Hand distance away from body at conclusion? 

6. Hand height at conclusion? 

7. Width of Object? 

8. Back Rotation angle? 

9. Task Duration? 

5. Is the floor clean, dry and non-slip? 

6. Is the area for setting down the load clear? 

MODULE 17. VIBRATION 

I. Is hand-arm vibration present? 

2. Are anti-vibration tools being used? 

3. Are anti-vibration gloves being used? 

4. Are workbreaks provided to avoid constant vibration exposure? 

5. Do hands remain warm while working? 

6. Can the tool be supported or rested while working? 

122 

Chapter7 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(kg)_ 

(Uft/Min) __ 

(em) __ 

(em) __ 

(em)_ 

(em) __ 

(em) __ 

(Deg.)_ 

(Hrs.)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 
Chaptcr7 

MAINTENANCE PHASE 

7. Does worker experience: a. tingling of the digits (finger) ? 

b. numbness of the digits? 

c. blanching of digits? 

8. What is the vibration frequency? 

.9. What is the duration of maximum continuous vibration exposure? 

10. What is the total duration of vibration exposure on this shift? 

II. What is the vibration acceleration? 

MODULE 18. REPEI7TJVE MOTION 

I. Does the task require the following to be performed? 

a. Reach with anns above shoulder level 

b. Work with anns above shoulder level 

c. Reach behind the body 

d. Inward rotation of forearm with bent wrist 

e. Outward rotation of forearm with beJtt wrist 

f. Ulnar deviation of wrist combined with supination 

g. Radial deviation of wrist combined with pronation 

h. Flexion of wrist 

i. Extei1Sion of wrist 

j. "Ootbes wringing" motion with hands 

k. Hand/wrist contacting sharp edges 

I. Flexion of the back 

m. Extension of the back 

n. Flexion of the shoulders 

o. Extension of shoulders 

p. Flexion of neck 

q. Extension of neck 

2. If a tool is being used: 

a. Can the location of the tool be adjusted? 

b. Is the tool suspended? 

c. Is the tool handle made of non-metallic material? 

123 

Appendix - Audit Checlclist 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(HZ)_ 

(Min)_ 

(MiJt)_ 

(mls"2)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 

(YIN)_ 
'''1'1:1 

Appendix - Audit Checklist Chapter? 

MAINTENANCE PHASE 

MODULE /9_ ACCESS 

I. Is there any conflict due to parallel wort? 

2. Do you think access is: 

3. How often was access equipment repositioned? 

a. difficult? 

b. dangerous? 

(YIN)_ 

(YIN)_ 

(YIN)_ 

a. I or 2 times in the entire task 

b. 3 or more times (AlB) __ 

MODULE 20. POSTURE 

# 

1 

2 

3 

4 

5 

6 

I. Do you feel that the workspace is constrained? (YIN)_ 

2. How often were the following postures adopted by Mechanic during the task? 

10% 25% 25% 

leg(s) bent 

kneeling/crawling/laying 

back twisted bent 

back twisted kneeling/crawling/laying 

leg(s) bent 

kneeling/crawling/laying 

0% -never observed 10% -25% -occasionally observed 

0%- 10%- seldomly observed above 25%- frequently observed 

MODULE 21. SAFETY 

I. Is the work area free of clutter, dirt, oils, etc? (YIN) __ 

2. Are safety attachments used when the mechanic performs maintenance at heights? (YIN) __ 

3. Is the maintenance person wearing safety shoes? (YIN) __ 

4. If task requires, is the maintenance person wearing eye protection? (YIN) __ 

124 

1 
Chapter7 Appendix - Audit Checklist 

MAINTENANCE PHASE 

MODULE 22. HAZARDOUS MATERIAL 

I. Is training provided for proper handling and clean up of hazardous materials? 

2. Are all hazardous materials properly labeled with type and caution information? 

3 .. Are eyewash stations available for emergency use? 

4. Are shower stations provided for emergency use? 

5. Are all hazardous materials properly labeled with type and cautions? 

6. Were hazardous material signed out and weighed? 

7. Were hazardous material signed in and weighed? 

8. If unused material was discarded, was it done properly? 

9. Does Work Card give proper Hazardous material Identification#? 

I 0. Hazardous Material being used is in the form of: a) Paint 

b)Epoxy 

(YIN}_ 

(YIN}_ 

(YIN}_ 

(YIN}_ 

(YIN)_ 

(YIN)_ 

(YIN}_ 

.(YIN)_ 

(YIN)_ 

c) Cleaning Agent 

d) Lubricant 

e) More than one 

f) Others (a/blcldlelf) _ 

II. Is safety equipment (corresponding to the type of hazardous material) being used? (YIN) __ 

12. Is the recommended safety equipment readily available? (YIN) __ 

13. Does the safety equipment cause restriction in movement? (YIN) __ 

14. Is the General Maintenance Manual available for review of Hazardous Material use? (YIN) __ 

15. What% of total task time are the hazardous materials being used? a) 10%- 24% 

b) 25%-49% 

c)50%-74% 

d)75%-99% 

e) 100% (alb/cldle) _ 

16. Does the use of a hazardous material intrude on ·other workers? (i.e., fumes, aerosol) (YIN) __ 

125 
Appendix • Audit Checklist Chapter? 

POST MAINTENANCE 

MODULE 23. BUY-BACK FOR ROUTINE MAINTENANCE 

I. Was the maintenance task required to be bought back by: 

a) the initial inspector? 

b) any Inspector (besides initial inspector)? 

c) maintenance foreman? 

d) maintenance person himself? (albic/d) __ 

2. Did the task pass buy · back on the first try? (YIN)_ 

3. If No to question 2, was the same inspector used for the latter attempts at buy-back? (YIN) __ 

4. Was the maintenance person present when the buy back was done? (YIN) __ 

5. If "Yes" to #4, was feedback information given to the maintenance person? (YIN) __ 

6. If "No" to #4, was maintenance person informed of discrepancies by written notice? (YIN) __ 

7. Does the maintenance person feel feedback information is informative and useful? (YIN) __ 

8. Is the supervisor available for questions by the maintenance person? (YIN) __ 

• 

126 
, . 

CHAPTER 8 

IMPROVING THE RELIABILITY OF 

MAINTENANCE CHECKLISTS 

Amy Pearl and Colin G. Drury, Ph.D. 

State University of New York at Buffalo 

X. I 1\TIWIH (' 110\ 

Patel, Prabhu, and Druty (1993) describe a workcard as "the prime source of on-line directive and 

feedforward information in aircraft inspection. It is 

the primary document that starts the inspection and 

serves as a major influencing factor on inspection 

performance"(p.l). The workcard can also be viewed 

as a checklist that aids the mechanic in recalling all 

the numerous tasks to be performed in a check. Once 

a task or group of tasks is finished, the mechanic or 

inspector is required to sign it as being satisfactorily 

completed. As the workers perform these tasks repeatedly, there is a tendency to perform them at least 

partially from memory, w~ a block of sign-oft's 

made at a convenient time. This is not how workcards 

are intended to be used; and such use can result in errors. Since the safety of civil aircraft is highly dependent on reliable inspection, we undertook an 

analysis of how workcards are presently used and 

how workcards design affects their use and the subsequent potential for error. 

8.1.1 Checklist Objectives 

Workcards and other forms of checklists are common 

throughout the aviation industry. In addition to workcards being used for all inspection and maintenance 

tasks, flight crews use checklists to prepare the aircraft for each new stage of a flight Degani and Wiener ( 1990; 1993) reviewed the role of checklists in 

the cockpit, the potential effects of their design, and 

sociotechnical factors affecting their use. Although 

the content of flight deck checklists differs substantially from those for maintenance and inspection, the 

checklists' objectives (as Degani and Wiener describe 

them), as well as many of their design concepts and 

performance factors, are similar. 

Degani and Wiener defined checklist objectives that 

are pertinent to aircraft maintenance: to assist the user 

in recalling proeedures, to outline a convenient sequence for motor movements and eye fixations, to a!-

127 

low mutual supervision within crews, to distribute tasks 

among crew members, and to act as a quality control 

tool for management and government regulators 

(Degani and Wiener, 1990, p.7). The first objective of 

a workcard is to remind mechanics or inspectors of 

items to be checked; any type of job aid shares this 

goal. By providing information externally, a job aid reduces the information a person must store and process 

(Swezey, 1987). Listing tasks in an order providing a 

convenient sequence of motor movements should reduce the time spent accessing the task areas. Workcards also provide written records of tasks to be 

performed and ease the supervision and distribution of 

tasks. Finally, sign-offs of tasks on a workcard verify 

that the work is complete, as dictated by the airline and 

by FAA regulations. Workcards used in aircraft maintenance and inspection tasks should meet these checklist objectives. For this project, we analyzed methods 

maintenance technicians use to perform different levels 

of checks to determine if their workcards met these 

goals. More-detailed 8-, C- and D-checks have fewer, 

larger tasks on each workcard. Lower-level checks (Achecks and below) were the main focus of this study 

because they typically consist of]arger lists (2()....100 

items) of relativ<j,ly short tasks. These are what is called 

"checklists." Although people performing these checks 

are classified as mechanics, these tasks' functions are 

associated with inspection, i.e., checking whether specific aircraft features meet pre-defined criteria for safe 

flight Our earlier work on inspection is directly relevant to the present study: Patel, et a!. ( 1993) investigated specific design issues relevant to inspection using 

workcards. 

8.1.2 Workcard Design Issues 

Patel, eta!. (1993) found that usable documentation 

must embrace the following factors: information readability, information content, information organization, 

and physical handling and environmental factors. Information readability issues are concerned with the 

documentation's typographic layout, as well as conventions concerning sentences, words, and letters. In-
Improving the Reliability of Maintenance Checklists 

fonnation content involves what infonnation to give, 

bow to give it, and in what order. Docwnentation must 

be appropriate, accurate, complete, and easily comprehensible. Infonnation organization deals with the classification and differentiation of directive information 

and other information such as notes and warnings. The 

structure of directive infonnation should be broken 

down into the command verb, the action qualifier, and 

the object of the action (Inaba, 1991 ). Patel, et al. 

(1993), in their study of A- and C-checks, pointed out 

that tasks should be listed in the natural sequence most 

inspectors use during a check. Finally, the workcard 

must be physically suitable for the tasks and the environment. Inspectors should be able to carry workcards 

with them while they perform tasks, without the workcards hindering task performance. Workcards should 

be resilient to all types of weather and to dirt and oil 

because inspections are performed under a variety of 

adverse conditions. 

Patel, et al. (1993, p. 13-16) developed a set of guidelines for designing documentation for aircraft inspection tasks. Using these guidelines to redesign 

workcards, they found significant improvements in inspectors' and mechanics' ratings of redesigned workcards when compared with old workcards. These 

researchers also observed that, for A-check workcards, 

the sequence of tasks did not match the sequence mechanics typically follow to perform checks. There is 

some variability in the ways mechanics and inspectors 

sequence their tasks throughout a check, and the number of sign-offs varies across tasks. These findings 

demonstrate the need for investigation of issues related 

to workcard task sequence and the optimal number of 

sigu-offs. 

8.1.3 Purpose of Project 

This project's original aim was to undertake an experimental evaluation of checklist reliability. The factors of interest were the grouping of tasks and the 

number of sign-offs required. Different workcard formats were to be designed for less-detailed, frequently 

performed checks such as low-level and A-checks. 

Possible formats would have included workcards with 

sign-offs after each step, with sign-offs only after the 

most salient items, and two-level checklists providing 

more-detailed infonnation for less-experienced mechanics. The methodology of this project changed from 

an off-line experiment to a field study at the request of 

our airline partner and after our observation of mechanics perfonning these checks. 

The task analysis described in the next section shows 

that present workcards do not provide mechanics and 

128 

Chapter 8 

inspectors with the most useful infonnation. Although 

mechanics and inspectors do read workcards for 

changes, they do not continually use workcards as they 

perform the checks. They are highly practiced in their 

tasks, and the fact that checks are repetitive makes it 

difficult to ensure that all tasks are performed to the 

same level each time. Job aids or redesign of workcards 

may help achieve the reliability required in aircraft inspection. This is why we changed the project's aims to 

detennining how mechanics use workcards, why mechanics do not use workcards continually during some 

checks, the possible effects of mechanics not using 

workcards, and how to make workcards meet checklist 

objectives Degani and Wiener (1990; 1993) defined. 

S.2 STl I>Y OF\\ ORKC \IW l S \CE 

The project's frrst objective was to detennine how 

mechanics actually use workcards during frequently performed checks. We needed to study workcard usage on the hangar floor to establish the degree 

that workcards meet Degani and Wiener's checklist 

objectives. A task analysis of a system is the foundation 

of any human factors investigation (Drury, Prabbu, and 

Grarnopadhye, I 990). 

8.2.1 Task Analysis 

Our study of mechanics' current use of workcards during checks consisted of videotaping and observing mechanics perfonning three levels of checks, as well as 

interviews and workcard evaluations. We made no 

videotape without the mechanics' pennission. Videotaping is an unintrusive way to gain accurate information on how a mechanic normally performs a check. 

The specific checks we studied were A-checks and two 

less-detailed ohecks: lower-level check I (least comprehensive) and lower-level check 2 (more comprehensive, but less than an A-check). Our activity during 

our first two trips to a hangar consisted of following 

mechanics as they performed the check. An observer 

asked questions to gain a basic understanding of each 

check for various types of equipment. The primary data 

we gathered from videotapes were the seqpence of 

tasks a mechanic performed, the number of times a mechanic referred to the workcard, and the approximate 

number of times a mechanic was interrupted. After mechanics finished a check, we interviewed them, often 

while they viewed the videotape of their inspection activity. We also questioned supervisors and lead mechanics about the workcards' usefulness and asked for 

their suggestions for change. In order to gain opinions 

from an adequate number of mechanics, we distributed 

evaluations on both the workcards and the subsequently 

developed job aids at one maintenance base. We pres-
ChapterS 

ent results of videotaping, interviews, and workcard 

evaluations so that readers may develop an understanding of workcards' usefulness for frequently perfonned, 

repetitive checks. 

8.2,1.1 Mechanics' Altitudes Towards !he Workcan!s 

Responses to interviews and workcard evaluations we 

distributed to mechanics provided many interesting insights. Perhaps tile most important finding is that mechanics use individual methods and skills to complete 

checks. Lock and Strutt ( 1985), in their study of the reliability of inspections in British aviation, had similar 

findings. The implication of this finding is that it is difficult to establish reliability of checks because mechanics do not value the standard workcard. 

Workcard evaluation results are presented in Appendix 

8-A. Question 5 in Section II showed that some mechanics do not usually refer to a workcard during a 

check. About half responded that they perform a particular check in the same sequence each time they perform the check. Most indicated that they sequence tasks 

based on locations on the airplane; they start with the 

nose and work around the aircraft to check for discrepancies. If a check is assigned to two people, tasks are 

typically divided logically I e.g., into exterior and interior tasks. The exterior is usually checked before the 

interior. Some mechanics sequence tasks by difficulty 

and/or the probability of finding a discrepancy that 

must be fixed. If they need assistance, they request a 

''floater'' to help them. Appendix 8-B shows mechanics' 

ratings of task difficulty and the probability of finding a 

discrepancy for B-737 Iower-Ievel2 checks. Tires and 

brakes generate the most concern because of the time 

required to change them when a discrepancy is identified. 

Although workcard evaluation results indicate that mechanics find workcards useful, interviews with and observations of mechanics performing checks indicate 

that workcards are not always used as intended. Many 

mechanics view workcards as guides only for inexperienced workers who may refer to it during a check: 

checks become routine and easily memorized. Also, 

mechanics typically check more items than the workcard requires because of their conscientious natures. 

Most mechanics feel that they only need to refer to a 

workcard for interim changes before performing a 

check. When mechanics find a discrepancy during a 

check, most state that they make a note to fix the discrepancy after they finish the check. However, the observer rarely saw notetaking, with the exception of one 

mechanic. This could be because some mechanics do 

not carry workcards continuously while performing a 

129 

Improving the Reliability of Maintenance Checklists 

check. After completing a check, mechanics return to 

the workcard to sign-off the tasks. The question remaining is, if mechanics do not use the workcard to sequence tasks for a check, what are the reasons for this 

and how do they sequence the required tasks? 

8.2.1 ,2 Content of !he Chec!s 

One reason mechanics rarely use the workcard while 

performing these checks is that the lower-level and Achecks are repetitive and frequent. Most of these mechanics perform fifteen lower-level2 checks and five 

A-checks every month. They have done these checks at 

this maintenance base for an average of9 years (this result carne from the workcard evaluations shown as Appendix 8-F). Furthermore, checks for various kinds of 

equipment are similar, with only a few, possibly important, differences. Mechanics easily memorize the 

checks and believe they do not need workcards as portable job aids. 

8,2,1.3 Task and Environmental factors 

Lower-level and A-checks are mobile: their tasks are 

located throughout an airplane's exterior and interior. 

Mechanics walk around a plane to check for defects, 

bending, kneeling, or reaching into an access panel. 

These movements are not conducive for carrying an 8.5 

X II inch workcard that a mechanic can refer to, make 

notes on, and sign-off tasks. In addition, many line 

checks are perfonned outside in a variety of weather 

conditions such as wind, cold, rain, and/or snow. Carrying a paper workcard and writing on it is even less 

practical in these circumstances. 

8.2,1.4 Seauence of Tasks 

Patel, et al. (1993) found that mechanics' ordering of 

tasks for an A-check did not match the workcard's order. In the current study, mechanics also rarely perConned tasks in the order listed on the workcard. In a 

second workcard evaluation, mechanics were asked to 

order tasks of a B-737 Iower-Ievel2 check in the sequence they normally complete the check. Appendix 8-

C presents results of this workcard evaluation. No mechanic provided the sequence given in the workcard. 

Subjects I and 2 have an additioual column in their tables since they were videotaped. In addition to sequence data from workcard evaluations, transcript 

analyses from videotapes of subjects performing 

checks show that mechanics do not use workcards to 

sequence their tasks. Tasks that are difficult to observe 

directly are indicated by asterisks in Appendix 8-C. 

This does not indicate that tasks were not performed, 

only that the observer could not see them on the 

videotape. 
I , I 

,I 

''I'''. 111:1 ,, ,,, 

• • I 

Improving the Reliability of Maintenance Checklists 

Workcard evaluations and videotapes indicate that mechanics tend to sequence tasks by spatial cues on the 

airplane, associating a specific area on the aircraft with 

all checks for that area. For example, at the right main 

landing gear, a mechanic checks tires for serviceability, 

checks the tire pressure, checks the tie bolts, cleans the 

strut piston, cleans the downlock viewer and indicator, 

and checks the brakes. All these tasks are performed at 

the righl main landing gear before the mechanic moves 

to another area. The workcard's functional organization, however, asks a mechanic to check all tires for 

serviceability before moving to another sign-off task. 

This would require a mechanic to walk around the nose 

landing gear, the right main landing gear, and the left 

main landing gear and then to revisit the same locations 

to check the tire pressures. The workcard sequence 

does not reflect the way most people work. Tasks such 

as "Check fuselage, empennage, and wings for obvious 

damage or irregularities as viewed from the ground" 

demonstrate this point even more dramatically. A mechanic does not check the entire fuselage for discrepancies at once; instead, he or she checks the fuselage 

while working around the aircraft perfonning other 

checks. This is demonstrated by the numerous times 

mechanics being videotaped checked the fuselage; they 

often cover the same~ more than once and re-visit 

the same task numerous times (see Appendix 8-C). 

Mechanics organize tasks by spatial cues, not by workcards' functional order, hecause areas to be inspected 

are very large. Humans optimize their use of time by 

minimizing the distance to be traveled. By checking 

everything in a particular aircraft area before moving to 

an adjacent area, a mechanic saves significant time and 

energy compared with that necessary to walk around 

the airplane as many times as would be necessary to 

check everything by functions. Using spatial cues, instead of functional locations, reduces the number of 

things a mechanic must remember, hence reducing his 

or her mental workload. 

There is a mismatch between the tool provided for the 

job (workcard) and mechanics' natural way of working. 

Such a mismatch can be addressed either by altering 

the tool or by altering the way of working. The alteration chosen depends ultimately upon what system reliability is obtainable. 

8.2.2 Non-Compliance in Using Workcards 

Our observations from other airlines during previous 

projects confirm this project's findings. For rarely performed tasks, such as most C- and D-checks, inspectors 

use workcards to perform the check. Mechanics do not 

use workcards for frequently performed checks, i.e., A130 

Chapter 8 

checks and below. They have memorized these checks, 

"gaining a feel for items to check" through frequent 

repetition. One of the problems with this is that mechanics may not receive feedback on the accuracy of 

their judgments since problems rarely occur. Also, 

since workcards are not physically compatible with the 

environment and the tasks, even inexperienced mechanics who want to use workcards have difficulty doing so. 

Finally, the functional sequence of tasks on workcards 

does not match the way people sequence tasks distributed over large areas. Tasks with only one sign-off for 

a particular function are often distributed over large areas of an aircraft, e.g., check the tire pressure of the 

main landing gear tires, and are performed as a mechanic reaches the area. Since mechanics tend to signoff all tasks when the entire check is complete, tasks 

that are not completed sequentially should have separate sign-offs. We conclude that present workcards do 

not provide useful information for mechanics and, consequently, do not meet the checklist objectives Degani 

and Wiener (1990; 1993) defined. 

8.2.3 Relationship Between Workcard and Checklist Objectives 

To review, the objectives of a checklist are to aid the 

user in recalling procedures, to outline a convenient sequence for motor movements and eye fixations, to allow mutual supervision within a crew, to distribute 

tasks among crew members, and to function as a quality control tool for management and government regulators (Degani and Wiener, 1990;1993). Since present 

workcards do not provide a convenient sequence for 

motor movements and eye fixations, they are not used 

continuously during checks. The workcards do not aid 

the user to recall procedures. The present workcards 

cannot be used <:onveniently to distribute tasks among 

mechanics hecause many sign-offs are not separated. 

The practice of signing off tasks at the end of the 

checks diminishes the workcards' ability to serve as a 

quality control tool. A job aid needs to be designed that 

meets checklist objectives listed above and that accommodates mechanics' different work methods. Mechanics working for many different airlines would use 

such a job aid. 

S.3 :\ATIO:\AL D \T \ 0:\ TilE 

EFFECTS OF '\OT :\IEETII\C 

\\ORKCARD CO.\LS 

That the present system appears to be working is 

demonstrated by high reliability, i.e., accidents are 

extremely rare. However, mechanics' workcard use is 

reduced because the job aids do not match their needs 

and individual work methods. The danger of not using 
' 

Chapter 8 

workcards during a check is that a mechanic must then 

rely solely on his or her memory. If a mechanic were to 

become distracted, he or she could forget to perform a 

check, yet automatically sign it off because he or she 

has performed the check so many times correctly. A 

.mechanic's confusion with similar checks and other 

aircraft may result in him or her substituting a required 

task with a task appropriate for another check or aircraft. 

Our observations from other airlines indicate that 

similar patterns in workcard usage exist throughout the 

industry. It is worthwhile to place our findings in a 

broader context by analyzing similar errors reported 

elsewhere. The following examples of errors relating to 

these issues are taken from NASA's Aviation Safety 

Reporting System (ASRS). These voluntary reports are 

subject to reporting biases, and no airline is named in 

these reports. 

The following excerpts from ASRS' reports illustrate 

the importance of workcards meeting checklist goals. 

They also illustrate other problems, such as the speedaccuracy tradeoff and poor training, but all have a 

common contributing cause of mechanics' not following procedures specified '?n the workcard. 

* I had just completed an outside 

service inspection ... when an FAA 

inspector pointed out that I had 

failed to check for water in the fuel 

tanks and had missed a couple of unreadable placards but had signed off 

blocks saying I had checked these 

items. Both were inadvertent oversights, were not deliberate, and did 

not cause any significant unsafe 

conditions. The problem arose because I was in a hurry to get the job 

done. Also, in the 2 years that I have 

worked on these aircraft, I have 

never heard of any mechanics finding water in the fuel tanks. I have 

corrected the situation by slowing 

down and paying attention to the 

checklist and my actions. 

* While performing an A 

check, ... one of my coworkers, Y, 

pencil-whipped the aircraft landing 

gear and flap lube. I had been 

working the engines all night and 

know that the flaps had not been extended for tubing. 

131 

Improving the Reliability of Maintenance Checklists 

• I did not perform a pilot static leak 

check on the altimeter system after 

altimeter replacement. .. .I was at fault 

because I was unaware that the 

maintenance manual had been revised to reflect this change. 

• Due to an oversight, not having the 

sign-off document immediately 

available, I did not document the 

company form that I had complied 

with XXXX, a visual inspection of 

the cargo door prior to takeoff. 

* I feel my actions may well be the 

cause of the gear failure due to improper reassembly of the uplock activator, and failure to follow proper 

procedures. In addition, I made several mistakes in following the proper 

procedures, as called for by company maintenance manuals. I failed 

to enter a discrepancy on a mechanic's discrepancy list. I did not 

use proper maintenance manual reviews. I did not perform a gear retraction following reassembly of the 

activator. 

These reports all illustrate errors that could be attributed to not using or not complying with workcards or 

maintenance manuals. The first two reports provide examples of workers signing-off tasks they did not perfonn The example of a mechanic not performing a fuel 

tank sump check demonstrates one of the effects of experience. Since the mechanic does not expect to find a 

problem, the ctt.:k is not taken seriously. The report of 

an inspector or mechanic being unaware of a maintenance manual revision is an example of a failure to read 

interim changes. The fourth account states that the reporter did not have the workcard immediately available, probably because the-workcard was incompatible 

with the task and environment. The last report provides 

another example of a mechanic not complying with 

proper procedures. This could be attributed to numerous factors such as training, the mechanic's attitudes, 

time constraints, and environmental factors that make 

using the maintenance manual either difficult or inconvenient. 

• After servtcmg #I engine and 

while servicing #3 I was distracted 

by another crew member standing 

below my servicing buggy. He 

wanted me to check something else 
Ojo~.l 1 J 

.,_.,,.1 I 

~~ ,I ,, 

~ 

• • 

Improving the Reliability of Maintenance Checklists 

on the aircraft and after doing so I 

returned to my servicing buggy, still 

thinking that I had finished #3 engine. I moved on to another aircraft. 

This aircraft took off and during the 

first part of the flight the crew noted 

the #3 engine oil level falling and 

then stabilizing at an acceptable 

level. Upon landing the crew called 

maintenance, who found the #3 engine oil service door missing, along 

with the oil cap. 

• During the reassembly procedure 

the screws were not installed in the 

panel. I was called away by a coworker and foreman to help on another problem on the aircraft. Then a 

push to get the aircraft on line occurred .... The aircraft was stopped at 

its next destination; the panel was 

found missing. 

* On the aircraft's right wing tail 

light assembly, I removed the light 

assembly to change the top 

bulb .... Note: On removal of the unit, 

I had laid the 8 securing screws on 

top of the wing. Before I secured the 

unit into the wing tip, I wanted to be 

sure it worked. I went into the 

cockpit and activated the lights. I 

went out to the wing tip to find them 

working properly and returned to 

cockpit to shut them off, as the lights 

would be blinding while securing the 

unit. After shutting lights off from 

cockpit, I stopped for 3-4 minutes to 

talk to a mechanic who was doing 

aircraft interior work ... After leaving 

the interior of the aircraft, I was 

thinking I wantOd to finish all exterior work quickly, as it was 18 degrees F with the wind chill factor. A 

ladder I had out on the left engine 

caught my eye as I was coming 

down the stairs. I was running 

through my mind items I had to 

complete to get inside out of the 

weather. With the wing tip light 

fixed, all I had to do was put the ladder away [without securing the 

screws]. 

132 

Chapter 8 

These errors demonstrate potential negative effects of 

inattention and distractions. Although the mechanics 

we interviewed all strongly stated that if they were distracted they would not need to make a note to remember which tasks to complete, most research in human 

error suggests otherwise. Reason (1990) developed a 

human error model that particularly considers the effects of inattention. 

8.3.1 Applicable Human Error Researcb 

Rasmussen (1982) models human performance and its 

interactions with a possibly unaccommodating environment, categorizing it on the basis of human information processing. At the skill-based (SB) performance 

level, people perform familiar, routine tasks requiring 

little attention. Rule-based (RB) activities involve using 

established rules to make familiar decisions or to solve 

common problems. Knowledge-based (KB) performance is employed when no known rules are available 

for the situation and a person must resort to reasoning, 

to mental models, and to high-order cognitive processes to appraise the available information, to assign 

goals, and to develop methods for achieving them. 

Reason (1990) describes two cognitive modes for differentiating between the sequential reasoning used for 

KB tasks and the automatic control used for SB and 

RB tasks. The attentional mode for knowledge-based 

activities requires high cognitive effort and is characteristic of the decision-maker's low level of experience 

with the problem or situation. During SB and RB perfonnance, the schematic mode involves semi-automatic 

actions with few or no attentional checks. A person's 

intentions or matching conditions in the environment 

activate strongly associated groups of actions called 

"schemata." " 

Reason writes, "When cognitive operations are underspecified, they tend to default to contextually appropriate, high-frequency responses ... or ... the more often a 

cognitive routine achieves a successful outcome in relation to a particular context, the more likely it is toreappear in conditions of incomplete specification" 

(1990, p. 97). In other words, when a person cannot 

define all aspects of a situation, he or she resorts to 

habitual actions. Incomplete specification of a situation 

can be attributed to a combination of situational factors 

and/or a person's lack of attention. Errors result from 

activation of the wrong schemata or from activating the 

right schemata either in the wrong order or at the wrong 

time. As a person becomes practiced with a habitual 

task, the chances of activating a common, yet inappropriate, schemata increase. 
ChapterS 

Error.; often occur in "strong-but-wrong" form, i.e., behavior is appropriate to past circumstances because of 

lack of attention to changed circumstances. Skill-based 

performance errors occur because actions at this level 

are directed by schemata most active wben an attentional cbeck is omitted or mistimed. Rule-based perfonnance errors are usnally attributed to inappropriate 

associations between contextual cues and previously 

applicable rules. Knowledge-based performance errors 

are unpredictable since the person does not have the 

knowledge to deal with the unfamiliar situation. These 

errors are due to "bounded rationality" and incomplete 

or inaccurate mental models (Reason, 1990). 

The potential skill-based errors is particularly important for repetitive lower-level and A-checks. Experienced mechanics quite familiar with the tasks operate at 

the skill-based level wben they move between tasks 

within a check. When an attentional check is omitted, 

the mechanic does not specifically note where he or she 

is in the task sequence. The mechanic then can easily 

be "captured" by a schema or another task that be or 

sbe frequently would perform in that situation, even if 

the mechanic's intentions call for a different action. For 

example, an attentional check can be omitted because 

of an external interruption such as another crew member asking the mechanic tci check something. The distraction could be internal, e.g., the mechanic worrying 

about other tasks, the weather, even time pressure. 

Mechanics may use rules to determine if an indication 

is a discrepancy. One objective of workcards and 

maintenance manoals is to externalize rules so the mechanic does not need to remember them. For example, 

the workcard gives the acceptable range of tire pressure. If the mechanic does not use the workcard, the 

potential for rul<>-based errors rises since the mechanic 

is forced to rely on memory. Rules often differ among 

tasks which are otherwise similar, e.g., different tire 

pressures are acceptable for different aircraft. 

Knowledg<>-based errors are not relevant to the checks 

under study in this projecL As mentioned, lower-level 

and A-checks are repetitive and familiar for these mechanics. Knowledg<>-based reasoning rarely occurs; 

wben it does, a workcard is likely to be of little assistance. In knowledg<>-based situations, maintenance 

manuals and a mechanic's experience and knowledge 

are the best resources. The goals of checklists are to 

assist skill-based and rul<>-based performance and to 

compel mechanics to make more attentional checks 

while they work in the scbernatic mode. The errors 

listed in the next section are associated with workcards' 

failure to meet objectives for checklists. 

133 

Improving the Reliability of Maintenance Checklists 

8.3.2 Potential Errors Related to Workcards 

We derived the following potential errors after considering Reason's theories of human error and from our 

study of workcard usage. We made our predictions of 

potential types of errors related to workcards knowing 

that mechanics rarely use workcards, that they sign-off 

all tasks at the end of the check, and that the potential 

for distractions and interruptions is high as they perform these checks. The first three kinds of errors are 

omissions related to skill-based performance. The last 

category is related to rul<>-based errors. There are other 

kinds of potential errors, but the following are most 

relevant to findings of our study of workcard usage. 

8.3.2.1 OmjAAjons Related to Interruptions 

Reason's (1990) theories predict that distractions and 

interruptions occurring while workers perform highly 

skilled, familiar tasks, such as lower-level and Achecks, are particularly critical. When the mechanic directs attention back to the check, he or she may not 

finish a task or fail to perform a task. Since checks are 

performed in the schematic mode, task completion 

within a check is fairly automatic. A mechanic recovers 

from most interruptions by making a conscious effort to 

ensure continuity. Unless the mechanic makes an effort 

to recall what be or sbe was doing wben interrupted or 

distracted, the mechanic can continue the check after 

being interrupted as in the most frequently occurring 

circumstances. Since the mechanic has previously 

completed the task numerous times, be or sbe may honestly believe the task to have been completed. As the 

ASRS' examples illustrate, the mechanic may never direct attention back to the task, particularly if there is 

time pressure to complete the check. After an interruption, the mechanic may start on a new set of tasks and 

never return to lois or her original mental task IisL Possible remedies for these types of errors include the following: 

a) Workcards should be designed to be 

easy for workers to make notes on or 

to sign off complete tasks 

b) Mechanics should be informed of effects of interruptions and distractions, as well as the importance of 

malting notes about incomplete 

tasks. 

We need to consider ways to combat all errors frequent 

enough to be captured by ASRS. 
I 

,I 

"' 

•l•r::l 1111 

\,.I :11 

Improving the Reliability of Maintenance Checklists 

8 3 2 2 Omissions Belated to Workcard Seauence 

Workcards sequence tasks by functions. If mechanics 

actually followed workcards' sequences, the probability 

of distraction would increase as they constantly moved 

around the aircraft to complete functional checks. In 

turn, this would increase the likelihood of an omission 

associated with an interruption or distraction. Sign-offs 

for some tasks are not separated, although the tasks are 

spatially separated. For example, there is a single signoff for serviceability of both right and left main landing 

tires. However, tires are checked separately. This 

workcard sequence of tasks may increase the probability of a mechanic signing-off the task after checking 

one side of the main landing tires, but before checking 

both sides. 

8,3 2 3 Omissions Belated to Worlscard Non-Compliance 

Task analysis of mechanics performing checks revealed 

that workcards' functional sequence of tasks rarely 

matches the spatial sequence mechanics use. The task 

analysis also predicted and revealed that mechanics 

rarely use workcards, partly because they do not match 

work habits and partly because they are physically incompatible with the tasks and environment. Mechanics 

disregarding the task sequence on a workcard rely on 

memory and are thus more likely to omit a task, particularly one they perceive as unlikely to reveal a discrepancy. Since mechanics assigned to frequent checks 

generally perform them on a number of different aircraft, they may unknowingly confuse checks, e.g., 

substitute a task from a different check or aircraft. 

Workcards help them recall tasks to be performed. 

Most mechanics decrease the chances of this type of error by performing substantially more checks than the 

workcard requires. For example, a mechanic may treat 

part of a lower-level check as the equivalent part of an 

A-check. 

A lack of a rigidly performed sequence is likely to induce omission(s) when the task sequence is not habitual 

and requires more attention. A number of mechanics 

indicated that they do not follow the same task sequence each time they perform a check. Also, mechanics' practice of signing-off all tasks at a convenient 

break, even at the end of a check, instead of immediately after completing a task, increases the likelihood of 

an omission when a mechanic frequently performs the 

checks. If an omission is possible due to a distraction, 

time pressure, or some other reason, the mechanic 

signing-off tasks must pay careful attention to each one 

he or she signs-off, and must actually recall performing 

that task at that time. Since sign-offs are highly repetitive and require very little attention, a mechanic could 

134 

Chapter 8 

easily assume that a task was completed because it 

previously was always completed. 

8 3.2.4 Rule-Based Errors 

One of the objectives of a checklist is to aid users to recall procedures (Degani and Wiener, 1990; 1993). 

Workcards mainly outline tasks to be performed; they 

also remind mechanics of some specification limits, 

such as those for tire pressures. Other specification 

limits are not given on the workcards, so one recommendation for improvement is to include all limits on 

the workcard. If a mechanic does not regularly use a 

workcard throughout a check, he or she may confuse 

specification limits among airplanes. 

More likely causes of rule-based errors relate to the 

nature of a check and the high experience levels of mechanics performing them. Because mechanics are familiar with the checks, they may not readily recognize 

unusual circumstances, as Reason predicts. Although 

experience normally assists mechanics by directing 

their attention to likely locations of defects, it may hinder them when circumstances substantially differ from 

their expectations. As Lock and Strutt write, "There is a 

danger that too much familiarity with a particular item 

could lead an experienced inspector to miss a significant defect, if it does not conform to the expected pattern (condition) or expected locations which are fixed 

in the inspector's mental model of the aircraft and its 

pattern of deterioration" (1985, p. 6.5). Paradoxically, 

mechanics' high level of experience and expertise is 

one of the greatest challenges we face in developing a 

job aid for the checks. 

8.3.3 The Challenge of Developing a Job Aid 

Task analyses f>erformed with existing workcards revealed potential causes of error as checks are currently 

performed. A job aid needs to be designed that reduces 

the potential for errors associated with workcards incompatible with mechanics' work habits and for errors 

related to mechanics' failure to use workcards throughout a check. These errors all stem from the fact that the 

present workcard is frankly not useful for mechanics. 

The design difficulty is compounded by the fact that 

highly skilled, well-trained, and experienced mechanics 

view workcards as guides for inexperienced mechanics 

and as quality control tools. 

This project's challenge was to help increase the reliability of an already reliable system. Mechanics' work 

is extremely reliable without workcards. Even when 

mechanics make an error, they rarely receive feedback. 

Due to the redundancy and frequency of checks, airplanes normally fly without incident. However, there 

1 
Chapter 8 

remains a slight possibility that not using workcards 

during lite check, or using workcards that do not match 

work melltods, could result in an error willt adverse 

consequences. Adding to 1he challenge is lite fact lltat 

as mechanics' experience increases, lite probability lltey 

use a workcard as intended decreases. It is worlhwhile 

to explore developing a job aid that reduces lite small 

probability of error because it is compatible willt mechanics' work habits and meets Degani and Wiener's 

checklist ~bjectives. Any increase in reliability is worlh 

lite effort in an industry affecting public safety as directly as airlines. 

SA HI!· .I 0 II \Ill 

The proposed job aid must meet individual mechanic's 

work melhods, must be physically compatible willt 

llteir environment and tasks, and must meet guidelines 

for workcard design Patel, Prabhu, and Drury (1992) 

developed. Mechanics are more likely to use a job aid 

willt lltese characteristics. 

SA. I The Development of the Job Aid 

Observations and videotapes of checks revealed that 

1he task sequence differs among mechanics. Even lite 

same mechanic performs ~ for 1he same check in a 

different sequence on different nights. These findings 

suggest that lite job aid must be flexible in task sequencing and adaptable to different circumstances. 

Most mechanics order tasks by using spatial locations 

on an airplane. Appendix 8-D lists grouped tasks of a 

B-7371ower-level2 check commonly occurring sequentially wilhin a check. We developed this list after 

analyzing 1he videotaped checks. We organized tasks in 

a FROM/fO chart that showed 1he number of times 

two tasks were performed sequentially. We follow each 

task in Appendix 8-D willt a list of tasks performed sequentially to 1he first task for a group. Groups largely 

mirror tbe spatial layout of tasks on tbe aircraft. Workcard tasks could be divided into tbe spatial areas in 

which mechanics perform a group of checks, as revealed by sequential analysis. 

The proposed job aid organizes tasks spatially by listing all tasks for a particular area of 1he aircraft on one 

135 

Improving the Reliability of Maintenance Checklists 

pocket-sized card. The cards are laminated and placed 

on a ring so that a mechanic easily can change lite order 

of cards. Fignre 8.1 (next page) shows tbe front page of 

1he cards. Dividing tasks by area into small cards allows a mechanic to sequence areas according to his or 

her individual work habits. Tasks are organized willt 

lite spatial layout most mechanics prefer. A mechanic 

can use a grease pencil to note discrepancies, interrupted tasks, or sign-off tasks completed. Notes can 

llten be copied onto reports or wiped off tbe job aid 

when lite check is complete. The job aid cards are designed to have a bar code on each card so that a future 

scanning system could check which cards had been 

completed or to match cards willt bar codes located on 

lite aircraft. This feature was removed after initial design and is not used in lite current evaluation. 

Job aids were designed for bollt lower-level checks and 

for A-<:hecks on 1hree fleets of aircraft. The workcards' 

design follows Patel, et al.'s (1992) guidelines for information readability, information content, information 

organization, and physical handling and environmental 

factors. Some guidelines were particularly important 

for this job aid. 

The guidelines for information content recommend that 

·~nformation provided should be supportive of lite inspector's personal goal to read qnickly and also understand tbe information, to ensure its usage and eliminate 

personal biases" (Patel, et al., 1992, p.14). We accomplished this in tbe job aid's design by meeting ollter 

guidelines such as tbe following: 

Resort to use of primary typographic spatial 

cues like vertical spacing, lateral positioning, 

paragraphing and heading positioning as far 

as possible; if space usage is premium, llten 

resort to use of secondary cueings, e.g., boldfacing, italics, underlining. color coding and 

capital cueing in a decreasing order of preference 
·' '1111::1 !111' 

::i :Ill 

,,. 

,, 

Improving the Reliability of Maintenance Checklists Chapter 8 

136 
Chapter 8 

Distinguish between directive information, reference information, 

warnings, cautions, notes, procedures and methods 

Directive information should be broken into the command verb (e.g., 

check), the objects (e.g., valves, hydraulic lines) and the action qualifiers (e.g., for wear, frays). Use a 

consistent typographic layout 

throughout the document 

[1be content] should have certain 

consistent and common elements to 

foster generalizations across contexts 

(Patel, et al., 1992, pp. 13-15). 

Each workcard's heading refers to a spatial location on 

the aircraft combined with a functional description, 

e.g., right main landing tires, right forward fuselage, 

flight deck, right CSD oil. We capitalized the headings 

and centered them on the top of each workcard. Each 

heading's color indicates where the group of tasks 

listed on the workcard is located on the aircraft, e.g., 

green indicates radome and forward fuselage. Colorcoding makes sorting earth. by aircraft areas easier: 

mechanics can arrange cards in their preferred sequence quickly. Tasks to be performed are leftjustified. Cautions are indented and bold. Notes are indented from the cautions and presented in a smaller 

font (see Figure 8.2, next page). Each task is numbered 

on the workcard and separated from other tasks with 

blank lines. This arrangement makes it easier for mechanics to distinguish among tasks and to mark completed tasks with a grease pencil. Tbe command verb 

immediately follows the number; it is followed by the 

object and the action qualifiers, as in the following example: 

1) Check: forward lavatory for general appearance 

and condition. 

The command verb and the object are bold because 

mechanics already know the action qualifier and simply 

need a reminder of the task to be performed. Some mechanics suggested listiag only the object to be checked 

on the workcards. We could not investigate this idea in 

this project because regulations do not allow workcards' content to be changed. Tbe typographic layout 

and general content is consistent throughout workcards 

for all checks, ensuring consistency for mechanics. 

The following are the organizational issues and physical handling/environment factors we considered pertinent to the design of the job aid: 

137 

Improving the Reliability of Maintenance Checklists 

Task information should be ordered/sequenced in the natural order 

most inspectors would perform the 

tasks 

Tbe page should act as a naturally 

occurring information module 

Tbe workcard's pages should be a 

handy size 

H use of a workcard demands exposure to environmental agents like 

wind, rain, snow or even harsh and 

oily floor conditions, we should take 

adequate precautions to avoid excessive degradation" (Patel, et al., 

1992, p. 16). 

One of the primary goals of our job aid is to meet the 

guideline concerning the order of task information. 

Patel, et al. (1992) ordered tasks in an A-<:heck by 

finding the most common sequence among mechanics 

they surveyed. For our study, we took an approach 

based upon groups of tasks that mechanics perform sequentially. We then listed each group of tasks on one 

card (for an example, see Figure 8.3, page 139) so that 

workcards act as naturally occurring information modules. Since mechanics can arrange the groups of tasks 

in any order they choose; our job aid provides a natural 

sequence to all mechanics, not to IOOSt mechanics. 

Further, the pocket-sized cards leave mechanics' hands 

free, wben necessary. Tbe cards are laminated to protect them against environmental agents and to provide a 

better writing surface than paper (see Figure 8.4, page 

140). 

Although we encourage mechanics to make notes on 

the job aids and to check tasks completed, the job aid 

does not replace workcards' sign-qff sheets. Tbe first 

card of the job aid explains what the job aid is and instructs the mechanic to read interim changes included 

in the workcard and to sign-off tasks on the workcard. 

Tbe second card shows the headings' colors and associates colors with areas of the aircraft. These features 

belp meet the checklist objectives and, consequently, 

reduce the potential for error. 
Improving the Reliability of Maintenance Checklists Chapter 8 

I 

II 

138 
Chapter 8 Improving the Reliability of Maintenance Checklists 

139 
~-~-- --~----------. 

Improving the Reliability of Maintenance Checklists Chapter 8 

140 
I 

I 

Chapter 8 

8.4.2 Does The Job Aid Meet Checklist Objee· 

lives? 

To review, the objectives of a checklist are to aid the 

user in recalling procedures, to outline a convenient sequence for motor movements and eye fixations, to aJ. 

low mutual supervision in a crew, to distribute tasks 

among crew members, and to function as a quality 

control tool for management and government regulators (Degani and Wiener, 1990; 1993). Dividing tasks 

spatially in small cards affords a mechanic the flexibility to sequence areas according to his or her individual 

work habits while also organizing the tasks spatially. 

The job aid provides a convenient sequence for motor 

movements within an area while allowing a mechanic 

to determine the most convenient sequence between 

areas. In addition, dividing tasks into cards that can be 

separated allows for easier task distribution among 

crew members, allowing mutual supervision in a crew. 

Features of our job aid such as allowing mechanics to 

sequence and distribute tasks, the convenient size and 

surface of the cards, and, possibly, increased ease of 

reading the workcards (in compliance with the Patel, et 

al.'s (1992) guidelines) should promote mechanics' use 

of the job aid, in turn aiding users in recalling procedures. Although our job aid will not replace a sign-off 

sheet as a quality control tQOl, it should reduce sign-off 

errors since mechanics no longer have to rely on memory to know which tasks are complete. Since tasks are 

separated logically into cards, mechanics can check 

cards as they complete the tasks. 

Since our job aid meets these objectives, it should reduce errors associated with workcards, as the task 

analysis predicts. Omissions related to workcards not 

matching inechanics' individual work habits should be 

reduced since the job aid allows flexibility in the sequence of task areas. Omissions related to interruptions 

should also decrease. Tasks are separated into small, 

logical groups so that a mechanic can quickly scan the 

card he or sbe was working with before being interrupted. The workcards' easier writing surface should 

encourage mechanics to take notes about tasks intermpted, tasks completed, and of discrepancies found. 

Omissions and rule-based errors arising from mechanics not using the workcard should be reduced since the 

job aid was designed in a way that encourages its use. 

To determine whether these predictions are valid, we 

obtained feedback from mechanics and observed them 

using our job aid while performing checks. 

S.5 E\ \Ll \TIO\ or rm: .JOB \ID 

Our evaluation of the job aid consisted of the same 

methodology we used for task analysis. We ob141 

Improving the Reliability of Maintenance Checklists 

served mechanics performing the check using the job 

aid, had interviews with selected mechanics, and distributed workcard evaluations to evaluate and further 

refine the job aid. 

8.5.1 Direet Observation 

We videotaped a mechanic perfornting a lower-level 

2 check while using the job aid. He rearranged the 

cards to reflect his preferred sequence for the check 

and followed the cards almost exactly during the 

check. The mechanic frequently referred to the cards 

to ensure he had completed all tasks in sequence. After he thought he had completed the exterior checks 

and referred to the cards, he found that be did not 

check the fuel tank sump. In the aircraft's interior, the 

mechanic noted blown lights on a piece of paper because the job aid he used was a prototype made of 

cardstock and not laminated. The mechanic's sequencing of tasks demonstrated the expected spatial 

sequence; he performed tasks while walking clockwise around the aircraft. General observation indicated that this mechanic followed our job aid's task 

sequence significantly more than the workcard' s task 

sequence. 

8.5.2 First Workeard Evalnation 

Appendix 8-C shows results of a preliminary workcard 

evaluation we used for feedback after developing our 

first job aid. Tbe placemarker page received a ''useful" 

rating. This page is a colored instruction card intended 

be placed on top of the card stack. As a mechanic 

turned each card over, the placemarker page separated 

completed cards from those yet to be performed. Our 

observations and interviews revealed that mechanics 

were reluctant to move the placemarker page after they 

completed tasks tm a card. We removed the placemarker feature since it might be more confusing than 

helpful. Mechanics, instead, can use a grease pencil to 

track completed tasks. 

General results from the first workcard evaluation and 

those from subsequent interviews with mechanics and 

an inspector suggested .that they found the division of 

tasks into small cards useful, that they would rearrange 

the cards into their own preferred order, and that they 

would find a grease pencil useful. In addition to preferring the job aid to the workcard, they indicated that 

they would be more likely to perform tasks in the job 

aid's order they arranged than with the workcard's 

dictated order. They generally liked the card system 

and found it useful. Two suggestions we used to design 

the revised job aid were to make the cards smaller and 

to color-code cards by spatial areas of the aircraft so 

that it would be easier to order the cards. Due to time 
' 

' ,, 

ill' 

Jl·l' 

Improving the Reliability of Maintenance Checklists 

constraints, only three mechanics filled out the preliminary workcard evaluation. After revising job aid, we 

distributed another workcard evaluation. 

8.5.3 Second Workcard Evaluation 

Seventeen mechanics completed the second workcard 

evaluation after they viewed a demonstration of the job 

aid. The results, presented in Appendix 8-F, reveal little difference between the present workcard and the 

proposed job aid. The only factor revealing a difference between the workcard and the job aid was the mechanics' opinion that they would perform the check in 

the order given. They indicated that they seldom perform tasks in the workcard's order but wouldsometimes to usually-perform tasks in the order they 

arranged while using the job aid. This result is encouraging given that the job aid's main goal is to provide a 

task order mechanics will follow so they use the workcard and do not rely on memory. Mechanics found 

color coding of cards (3.65), division of tasks into the 

smaller cards (3.82), and the grense pencil (3.88) 

slightly less than useful (which would be a 4.0 rating). 

These findings are somewhat surprising since many 

mechanics make notes and a mechanic recommended 

color-coding. One mechanic suggested that the entire 

card be color-coded. Our question regarding the usefulness of dividing tasks into smaller cards was probably inappropriate since tasks were divided so that 

mechanics could arrange the sequence (which received 

a favorable response). 

One potential reason for the "neutral to slightly above" 

evaluation of the job aid versus the workcard is that 

many respondents did not use the job aid to perform a 

check, but only saw a demonstration. Had they used the 

job aid, many mechanics may have been more convinced about its usability. Also, mechanics who had 

been trained to use workcards were reluctant to accept 

a change. They seemed concerned about issues of 

tracking interim changes and the ease of updating cards 

for new information. If lamination becomes too costly, 

there is an alternate possibility of printing cards on card 

stock, which is more resilient to environmental factors 

than ordinary paper. Such cards could be used once and 

be updated as easily as the workcards. The job aids 

printed on card stock that were used for the DC-9 

lower-level 2 check we videotaped and reported in 

8.5.1 appeared to work well. 

Another possible reason for mechanics' neutral responses reflects their belief about the reliability of their 

work. As we previously discussed, these mechanics are 

experienced and extremely familiar with tasks performed in a check. They typically receive little, if any, 

feedback about the danger of interruptions and of fail142 

Chapter 8 

ing to use the workcard of to follow its task sequence. 

Since relationships between human error and using the 

workcard are not obvious, any possibility of increasing 

these checks' reliability is worth investigating. 

8.5.4 Overall Results 

Observations we made of mechanics using the job aid 

while performing a check generally revealed closer 

compliance with the task sequence the mechanics arranged while using the job aid than observations we 

made of mechanics using traditional workcards. Interviews and informal discussions revealed that mechanics 

had generally favorable responses to the job aid. The 

first workcard evaluation's results reflects this finding. 

In contrast, the second workcard evaluation's results 

revealed mostly neutral responses to the job aid. Most 

mechanics completing the second workcard evaluation 

were unfamiliar with the goals of this project. Hence, 

they were skeptical about the project and logistics of 

implementing the job aid. In contrast, the first workcard 

evaluation and direct observation involved a small 

numbers of people who understood the project's goal 

of increasing workcard compliance. After other mechanics begin using the job aid, we expect initial neutral reactions to be followed by acceptance with 

increased use. 

S.6 CO:\CLl'Sin'< 

I

n this study, we examined issues in developing a 

job aid for frequently performed, long, sequential 

tasks to increase reliability of task performance. Our 

most important recommendation from this project is 

to design flexible job aids meet individual work 

methods. To do so, it is important to identify factors 

influencing individual work methods. Our task 

• analysis found that mechanics performing low-level 

checks and A-checks use the spatial locations of tasks 

and, sometimes, perceived task difficulty for sequencing the tasks. Other factors may be more important 

for sequencing less frequently performed checks. 

Separating tasks allows for a natural division of work 

and, more importantly, makes it easier for mechanics 

to track completed tasks. The job aid should allow 

mechanics quickly to see what tasks are completed. 

Further, sign-offs for tasks located on different aircraft sections should be separated since generally they 

are not performed sequentially. 

Another potential method for helping mechanics to 

track completed tasks is a bar code reader. A bar code 

could be printed on each card of a check. After a mechanic completes all tasks on a card, he or she could 
Chapter 8 

scan the bar code, using a small, lightweight computer attached to his or her belt. After the check is 

complete, the computer could identify any tasks mechanics missed. After mechanics are sure that all tasks 

are completed, they can do their "sign-offs" either 

manually or with the computer (when computer recognition of signatures becomes common). Either approach would significantly reduce mechanics' current 

reliance on memory. As bar code readers are relatively inexpensive, airlines should further investigate 

this option. 

The job aid must be resilient to environmental factors 

and compatible with task factors. Task analysis 

should identify conditions under which mechanics 

will use the job aid. The job aid must not physically 

hinder users performing their tasks. 

Mechanics must understand the importance of using 

workcards, especially the ways interruptions and distractions can lead mechanics to omit tasks. Factors 

such as weather, absences by co-workers, reassignment, and time pressure all contribute to the potential 

for distractions. 

Finally, workcards, as a form of checklists, must meet 

objectives of checklists (Oegani and Wiener, 1990; 

1993). Workcards should aid users to recall procedures by outlining a convenient sequence for motor 

movements and eye fixations. Workcards should 

permit mutual supervision within a crew, as well as 

helping a crew distribute tasks among themselves. 

Taken together, these factors should increase a workcard's ability to function as a control tool for management and government regulators, thereby 

increasing the checks' reliability. 

S.7 REFI-.IH '\lTS 

Degani, A. and Wiener, E. L. (1990). The Human 

factors of flight deck checklists: The normal 

checklist. (Contractor Report 177549). 

143 

Improving the Reliability of Mainte111J1Jce Checklists 

Moffett Field, CA: NASA Ames Research 

Center. 

Degani, A., and Wiener, E. L. (1993). Cockpit 

checklists: concepts, design, and use. Human 

Factors, 35(2), 345-359. 

Drury, C. G., Prabhu, P. and Gramopadhye, A. 

(1990). Task analysis of aircraft inspection 

activities: Methods and findings. In Proceedings of the Human Factors Society 34th Annual Meeting, Santa Monica, CA, 1181-

1185. 

Inaba, K. (1991). Converting technical publications 

into maintenance performance aids. In Proceedings of the Second International Conference on Human Factors in Aging 

Aircraft, Biotechnology Inc., VA, 167-193. 

Lock, M. W. B., and Strutt, J. E. (1985). Reliability 

of in-service inspection oftranspon structures. CAA Paper 85013, London. 

Patel, S., Prabhu, P. and Drury, C. G. (1993). Design 

of workcards. Human Factors in Aviation 

Maintenance-Phase Three, Volume I Progress Repon, DOT/FAA/AM-93115, Washington, DC: Office of Aviation Medicine, 

Chapter 7, 113-132. 

Rassmussen, J. (1982). Human errors. A taxonomy 

for describing human malfunctions in industrial installations. Journal of Occupational 

Accidents.~. 311-333. 

Reason, J. (1990). Human E"or. New York: Cambridge University Press. 

Swezey, R.W. (1987). Design of job aids and procedure writing. In Salvendy, G. (ed.), Handbook of Human Factors, 1039-1957. 
APPENDIX 8-A 

Results of Present Workcard Evalua~ons 

L Summary Statistks 

Number of respondents = 8 

Age of respondents: Mean=40.38 sd=7.73 

Years worked as a mechanic: Mean=l7.4 sd=9.80 

Average number of lower-level checks performed per month: 

Mean=l4.25 sd=8.25 

II. Open-Ended Questions 

1. Do you normally perform the tasks on a lower-level 2 check in the same order every time you 

do the check? 

Yes: 3 

No: 4 

Depending on aircraft type: 1 

2. Normally, how do you sequence the tasks you must perform to complete a lower-level 2 

check? 

Subject 1: 

Subject 2: 

Subject 3: 

Subject4: 

Subject 5: 

Subject 6: 

Subject 7: 

Subject 8: 

Starting at the nose of aircraft, I wrap around wings and empennage finishing 

at the nose again. • 

Nose to left side of aircraft to nose. 

Sometime start on the outside, sometimes start inside. 

Start at nose, work way around. 

Outside, inside, work release items. 

Inside right to left, inside back to front. 

Outside, inside, pilot items. 

Habit. · 

3. If you are doing the check with another person, how does this change your strategy for 

performing the check? 

Subject 1: 

Subject 2: 

Subject 3: 

Assistant on check would service tires, APU oil, engine oil and CSD oil and 

hydraulic fluid. 

None. 

One person will do the outside, the other one will do the inside. 

144 
Chapter 8 

Subiect4: 

Subject 5: 

Subject 6: 

Subject 7: 

Subiect 8: 

Appendix - Improving the Reliability of Maintenance Checklists 

Usually split inside and outside. 

Depends on level of experience. 

None. 

None. 

One man assigned to inside, One man outside. 

4. What do you do when you fmd a discrepancy, e.g., do you make a note to fix it after you are 

finished with the check, or do you fix it as soon as you find it? 

Subject 1: 

Subject 2: 

Subject 3: 

subject4: 

Subject 5: 

Subject 6: 

Subject 7: 

Subiect 8: 

Make notes. 

Make note of discrepancy. 

Made a note and fix it after the check is done. 

Make a note usually unless able to fix on spot 

Fix after. 

Make a note. 

Fix after the check. 

Make a note. 

5. Could you please comment on the usefulness of the workcard, e.g., do you need to refer to the 

workcard while perfonning the check? 

Subject 1: 

Subject2: 

Subject 3: 

Subject4: 

Subject 5: 

Subject 6: 

Subject 7: 

Subject 8: 

No, unless there is a new revision. 

No. 

Sometimes. 

Used as guide since things checked are usually more than required 

No. 

No. 

Sometimes. 

The first 4 to 5 times you do the check on any specific ale after that no. 

ill. General Questions on the Usefulness of the Present workcards 

I. How useful do you find the workcard? 

Mean=4 sd=0.535 

[0= of no use 2= not very useful 4= useful 

6= considerably useful 8= extremely useful] 

2. How often do you refer to the workcard? 

Mean=4:125 sd=l.727 

[0= always 2= usually 4= sometimes 6= seldom 8= never] 

3. Would you prefer a workcard that is: 

Mean=4.688 sd=l.945 

[0= more concise 4= about the same 8= more detailed 

145 
Appendix - Improving the Reliability of Maintenance Checklists 

4. How would you rate the ease of understanding of the workcard? 

Mean=5.125 sd=L959 

[0= very difficult 4= moderately easy 8= very easy] 

5. Do you have any problems handling the workcard? 

Mean=6.625 sd=L408 

[0= always 4= sometimes 8= never] 

6. Do you perform the tasks in the order given by the workcard? 

Mean=2.750 sd=L389 

[0= never 4= sometimes 8= always] 

7. When do you sign off complete items on the workcard? 

Five mechanics responded at end of workcard. 

One mechanic responded between intermittently and end of workcard. 

One responded after every section. 

One responded after every task. 

146 

Chapter 8 
J 

' 

Chapter 8 Appendix - Improving the Reliability of Maintenance Checklists 

APPENDIX 8-8 

Mechanics' Ratings of Probability of Discrepancy and 

Difficulty ofB-737 Lower-Level2 Check Tasks 

The approximate likelihood of finding a discrepancy was rated: 

0= never 4= sometimes 8= always 

The diffictilty of perfonning the task was rated: 

0= very easy 4= moderately easy 8= very difficult 

Task 

Check left engine inlet and reverser area. 

Check right engine inlet and reverser area. 

Check brakes for wear with pressure applied. 

Check main landing tires for serviceability. 

Check nose landing tires for serviceability. 

' Check nose tire pressure. 

Check main landing tire pressure. 

Accomplish a visual check of MLG wheels for 

broken or missing tie bolts. 

Clean MLG strut piston with solvent. Clean MLG 

downlock viewers/indicators. 

Clean NLG strut piston with solvent. Clean NLG 

downlock viewers/indicators. 

Check fuselage for obvious damage as viewed 

from the ground. 

Check empennage for obvious damage as 

viewed from the ground. 

Check wings for obvious damage as viewed 

from the ground. 

Check tail-skid (737 -400 only) 

Check engine fire bottle pressure. 

Check APU fire bottle disc and thermal relief 

indicator. 

147 

Prob. of Difficulty of 

Discrepancy: Task: mean (sd) 

mean(sd) 

1.2 (2.0) 22 (2.4) 

2.5 (1.3) 1.6 (1.6) 

2.9 (1.7) 4.0 (2.3) 

3.5 (1.3) 2.6 (1.4) 

2.8 (1.5) 1.8(1.1) 

4.2 (1.8) 3.3(2.8) 

3.7 (2.0) 1.8 (1.6) 

2.2 (2.2) 2.9 (2.1) 

3.3 (1.8) 1.6 (1.6) 

3.3 (1.2) • 2.3 (1.6) 

3.1 (2.0) 1.8 (2.2) 

2.6 (1.1) 2.6 (2.4) 

2.3 (1.1) 1.7 (1.4) 

1.8 (2.5) 0.5 (0.6) 

1.1 (0.9) 0.9 (0.7) 

0.6 (0.7) 1.1 (2.4) 
"r ,,,I 

,, l!k· ,

1 

Jl'l : ~ ' 

I''" I ~1111 1 ~ ,,,,,,, 

:,:1:' 

;1111' 

,1111111 

m' 

,, 

Appendix - Improving the Reliability of Maintenance Checklists 

Task 

indicator. 

Check exterior lights for proper operation. 

.. 

Check fuel tank sumps. 

Service hydraulic fluid for standby system. 

Service hydraulic fluid for system B. 

Service hydraulic fluid for system A. 

Service auxiliary power unit oil to NON RON 

aircraft. 

Service engine oil for engine #1. 

Service engine oil for engine #2. 

Service constant speed drive engine #1. 

Service constant speed drive engine #2. 

Service oxygen-crew, portable. 

Check attendants' seats for proper operation 

and condition. 

Ensure outboard seat in the emergency exit row 

has a non-standard thinner seat bottom cushion 

installed. 

Check that a yellow lifevest is installed under 

each seat. 

Check LH overhead stowage bin row (10) for 8 

spare yellow passenger life vests. 

Check forward LH closet for 2 each yellow 

demo lifevests. 

Check LH emergency equipment bin for 2 demo 

lifevests. 

Check protective breathing equipment for 

serviceability. 

Check lavatory flush pumps/timers. 

Check emergency lighting system. 

148 

Chapter 8 

Prob. of Difficulty of 

Discrepancy: Task: mean (sd) 

mean(sd) 

4.1 (0.9) 1.9 (1.8) 

3.3 (1.2) 2.1 (1.1) 

3.3 (1.4) 2.5 (1.4) 

2.7 (1.7) 1.9(1.8) 

2.7 (1.5) 1.7(1.6) 

3.9 (1.8) 2.7 (1.9) 

4.9 (2.3) 1.1 (1.4) 

4.4 (2.6) 0.9 (0.6) 

3.2 (1.6) 1.6 (1.4) 

2.6 (1.4) 0.8 (0.8) 

2.4 (1.4) 2.4 (2.0) 

2.3 (1.0) 2.4 (1.9) 

2.5 (1.9) 0.6 (0.6) 

5.7 (2.3). 1.9 (1.8) 

3.0 (2.1) 1.2 (1.4) 

2.4 (1.8) 0.6 (0.6) 

3.0 (2.3) 1.1 (1.7) 

0.8 (1.0) 1.2 (1.4) 

2.7 (2.3) 1.8 (1.8) 

2.7 (1.9) 2.8 (1.9) 
Chapter 8 Appendix - Improving the Reliability of Maintenance Checklists 

Prob.of Difficulty of 

Task Discrepancy: Task: mea~ (sd) 

mean(sd) 

Check and repair the entrance area for 3.3 (1.2) 2.1 (2.3) 

appearance and condition. 

Check cabin area for appearance and condition. 4.3 (1.8) 2.5 (1.7) 

Check galley area for general appearance and 2.4 (1.8) 2.2 (1.6) 

condition. 

Check forward lavatories for general 2.4 (1.5) 2.0 (1.2) 

appearance and condition. 

Check rear lavatories for general appearance 2.8 (1.3) 1.9 (1.4) 

and condition. 

• 

149 
" 'I ' 

Appendix - Improving the Reliability of Maintenance Checklists 

APPENDIX 8-C 

Sequence of Tasks for Lower Level-Check 2 on B-737 

Five mechanics completed this evalul!tiOrl- The first two respondents were also videotaped 

performing this check 

Chapter 8 

In Table A2, the order in which each task was performed is indicated by its task number. Mechanics 

ml -m5 completed the evaluation and are denoted by mlq-m5q. Mechanics ml and m2 were also 

videotaped and are denoted by ml -v and m2-v. Note that mechanic 2 split the check with another 

mechanic, so many tasks were not observed. For mechanic ml, some tasks could not be seen due to 

the video camem's position. 

!50 
I 

J 

! 

Chapter 8 Appendix - Improving the Reliability of Maintenance Checklists 

Table Al. Workcard Order for Tasks 1-27 

Task# Description 

1 Check engines inlet and reverser area. 

2 Check brakes for wear with pressure applied. 

3 Check tires for serviceability. 

4 Check tire pressure. 

5 Accomplish a visual check of the main landing gear for broken or missing tie 

bolts. 

6 Clean MLG & NLG strut piston with solvent. Clean MLG & NLG downlock 

viewers/indicators. 

7 

Check fuselage, empennage, and wings for obvious damage or irregularities 

as viewed from the ground. 

8 Check tail skid. 

9 Check engine fire bottle pressure. 

10 Check APU fire bottle discharge disc (yellow) and thermal relief disc (red). 

11 Check exterior lights for proper operation. 

12 Fuel tan,k sumps. 

13 Hydraulic fluid (System A, B, and Standby). 

14 Auxiliary Power Unit Oil. 

15 Engine oil. 

16 Constant speed drive #1 , #2. 

17 Oxygen-Crew, portable. 

18 Attendants' seats for proper operation and condition. 

19 Ensure outboard seat in the emergency exit row has a non-standard thinner 

seat bottom cushion installed. 

20 Check passenger life vest, for aircraft that are equipped for over water 

operation. 

21 Protective breathing equipment (PBE) for serviceability. 

22 Lavatory flush pumps/timers. 

23 Emergency lighting system. 

24 Entrance area for appearance and condition. 

25 Galley area for general·appearance and condition. 

26 Cabin area for general appearance and condition. 

27 Lavatories for general appearance and condition. 

151 
Appendix -Improving the Reliability of Maintenance Checklists ChapterS 

Table A2: Order of Performing T~ on B· 737 Lower-Level Check 2. 

Order I m..q I m1·v j m2..q 1 m2.v'l m3-q m4-q I ms.q 

1 11 7 3 6 2 7 15 

2 4 6 6 7 3 6 16 

3 6 4 7 1 3 6 15 

4 3 6 7 1 5 7 16 

5 6 3 7 16 4 12 2 

6 7 4 1 7 4 13 3 

7 16 6 16 2 1 13 3 

8 4 7 2 3 1 13 4 

9 3 1 3 4 7 15 4 

10 5 16 5 6 7 15 5 

11 2 7 6 9 7 16 1 

12 1 1 9 13 8 16 1 

13 9 7 10 7 9 1 6 

14 7 3 13 2 10 1 6 

• 

15 7 2 13 3 11 2 7 

16 10 5 13 6 6 3 7 

17 8 4 1 2 6 3 8 

18 16 6 11 3 12 4 9 

19 1 13 16 4 14 4 10 

20 12 6 18 6 15 5 11 

21 14 9 20 9 15 6 13 

22 13 10 23 13 16 6 13 

!52 
ChapterS Appendix - Improving the Reliability of Maintenance Checklists 

Orderl m-q I m1-v I m2-q I m2-V1 I m3-q I m4-q I m5-q 

23 13 7 25 2 16 7 13 

24 13 7 26 3 13 8 7 

25 23 14 27 4 13 9 14 

26 20 7 20 6 13 10 17 

27 18 7 21 1 18 14 18 

28 20 7 22 7 17 11 19 

29 21 3 27 1 21 27 20 

30 27 6 23 9 19 23 20 

31 27 2 20 13 20 22 20 

32 19 5 20 7 20 18 20 

33 25 4 19 24 20 21 21 

34 20 6 8 21 20 20 22 

35 22 13 14 20 27 20 23 

36 17 1 17 20 22 24 24 

37 24 7 6 22 24 25 25 

• 

38 26 1 6 27 25 26 26 

39 20 7 4 22 26 27 12 

40 15* 17 4 27 27 20 27 

41 22 12* 17 23 20 27 

42 27 15* 8* 19 

43 23 10* 17 

44 21 11* 

45 20 12* 

153 
Appendix - Improving the Reliability of Maintenance Checklists Chapter 8 

Order I m-q I m1-v I m2-q I m2-V1 I m3-q I m4-q I m5-q 

46 19 14* 

47 18 15* 

48 20 18* 

49 24 19* 

50 21 23* 

51 20 25* 

52 26 26* 

53 8 

54 11* 

55 12* 

56 15* 

57 25* 

* Asterisks represent tasks perfonned by other mechanics or not observed due to video 

restrictions. 

• 

!54 
ChapterS Appendix - Improving the Reliability of Maintenance Checklists 

APPENDIX 8-D 

Tasks Occurring Sequentially 

Tasks which follow each heading task are listed. 

Check left engine inlet and reverser area. 

Check main landing tire pressure. 

Accomplish a visual check of MLG wheels for broken or missing tie bolts. 

Check fuselage for obvious damage as viewed from the ground. 

Check wings for obvious damage as viewed from the ground. 

Service constant speed drive engine # l. 

Service constant speed drive engine #2. 

Check right engine inlet and reverser area. 

Check main landing tires for serviceability. 

Check main landing tire pressure. 

Oean MLG strut piston with solvent. Clean MLG downlock viewers/indicators. 

Check fuselage for obvious damage as viewed from the ground. 

Check wings for obvious damage as viewed from the ground. 

Check fuel tank sumps. 

Check brakes for wear with pressure applied. 

Check main landing tire pressure. 

Accomplish a visual check of MLG wheels for broken or missing tie bolts. 

Check empennage for obvious damage as viewed from the ground. 

Check wings for obvious damage as viewed from the ground. 

Check main landing tires for serviceability. 

Check main landing tire pressure. 

Accomplish a visual check of MLG wheels for broken or missing tie bolts. 

Oean MLG strut piston with solvent. Oean MLG downlock viewersfmdicators. 

Check fuselage for obvions damage as viewed from the ground. 

Check wings for obvious damage as viewed from the ground. 

Service constant speed drive engine # l. 

155 
Appendix - Improving the Reliability of Maintenance Checklists 

Check nose landing tires for serviceability. 

Check nose tire pressure. 

Oean NLG strut piston with solvent. Oean NLG downlock viewers/indicators. 

Check fuselage for obvious damage as viewed from the ground. 

Check nose tire pressure. 

Oean NLG strut piston with solvent. Oean NLG downlock viewers/indicators. 

Check fuselage for obvious damage as viewed from the ground. 

Check main landing tire pressure. 

Accomplish a visual check of MLG wheels for broken or missing tie bolts. 

Oean MLG strut piston with solvent. Oean MLG downlock viewers/indicators. 

Accomplish a visual check of MLG wheels for broken or missing tie bolts. 

Clean MLG strut piston with solvent. Oean MLG downlock viewers/indicators. 

Check fuselage for obvious damage as viewed from the ground. 

Clean MLG strut piston with solvent. Clean MLG downlock viewersl'mdicators. 

Check empennage for obvious damage as viewed from the ground. 

Check wings for obvious damage as viewed from the ground. 

Service constant speed drive engine #2. 

Clean NLG strut piston with solvent. Clean NLG downlock viewersl'mdicators. 

• 

Check fuselage for obvious damage as viewed from the ground. 

Check exterior lights for proper operation. 

Check fuselage for obvious damage as viewed from the ground. 

Check empennage for obvious damage as viewed from the ground 

Check wings for obvious damage as viewed from the ground. 

Check exterior lights for proper operation. 

Check fuel tank sumps. 

Service APU unit oil to NON RON aircraft. 

Service constant speed drive engine #I. 

Check and repair the entrance area for appearance and condition. 

!56 

Chapter 8 
Chapter 8 Appendix - Improving the Reliability of Maintenance Checklists 

Check empeDIIIIgt! for obvious damage as viewed from the ground. 

Service APU unit oil to NON RON aircraft 

Check wings for obvious damage as viewed from the ground. 

Check fuel tank sumps. 

Service hydraulic fluid for standby system. 

Service constant speed drive engine # 1. 

Service constant speed drive engine # 2. 

Service oxygen-crew, portable. 

Service hydraulic Duid for standby system. 

Service hydraulic fluid for system B. 

Service hydraulic Duid for system B. 

Service hydraulic fluid for system A. 

Service oxygen-crew portable. 

Check Ul emergency equipment bin for 2 demo lifevests. 

Oleck protective breathing equipment for serviceability. 

Oleck lavatory flush pumps/timers. 

Check forward lavatories for general appearance and condition. 

Check attendants' seats for proper operation and condition. 

Oleck that a yellow lifevest is installed under each seat. 

Check Ul emergency equipment bin for 2 demo lifevests. 

Check lavatory flush pmnpsliimers. 

Oleck emergency lighting system. 

Oleck and repair the entrance area for appearance and condition. 

Check forward lavatories for general appearance and condition. 

Ensure outboard seat in the emergency exit row has a non-standard thinner seat bottom 

cushiou installed. 

Check that a yellow lifevest is installed under each seat. 

Oleck emergency lighting system. 

!57 
Appendix - Improving the Reliability of Maintenance Checklists 

Check cabin area for appearance and condition. 

Check that a yellow lifevest is installed under each seat 

Check LH overhead stowage bin row ( 1 0) for 8 spare yellow passenger life vests. 

Check LH emergency equipment bin for 2 demo lifevests. 

Check protective breathing equipment for serviceability. 

Check lavatory flush pumpsftimers. 

Check cabin area for appearance and condition. 

Check LH overhead stowage bin row (10) for 8 spare yellow passenger life vests. 

Check cabin area for appearance and condition. 

Check forward LH closet for 2 each yellow demo lifevests. 

Check and repair the entrance area for appearance and condition. 

Check cabin area for appearance and condition. 

Check LH emergency equipment bin for 2 demo lifevests. 

Check and repair the entrance area for appearance and condition. 

Check cabin area for appearance and condition. 

Check protective breathing equipment for serviceability. 

Check emergency lighting system. 

Check and repair the entrance area for appearance and condition. 

Check cabin area for appearance and condition. 

Check lavatory flush pumps/timers. 

Check rear lavatories for general appearance and condition. 

Check emergency lighting system. 

Check forward lavatories for general appearance and condition. 

Check and repair the entrance area for appearance and condition. 

Check forward lavatories for general appearance and condition. 

Check cabin area for appearance and condition. 

Check rear lavatories for general appearance and condition. 

!58 

Chapter 8 
Chapter 8 Appendix - Improving the Reliability of Mainteru:mce Checldists 

APPENDIX 8-E 

First Evaluation Feedback on the Proposed Job Aid 

L Mechanics' Ratings of Job Aid 

Three mechanics (Ml-M3) responded. 

Question 

How useful would you find the placemarker page? 

O=of no use 4=useful 8=extremely useful 

How useful do you think the dMsion of tasks into 

small cards would be? 

O=of no use 4=useful 8=extremely useful 

Would you rearrange the cards to suit your individual 

work habits? 

O=never 4=sometimes 8=always 

Would you read the interim page at the end of the 

"official" w/c before starting the check? 

<>=never 4=sometimes 8=always 

Would you use the grease pencil to make notes while 

completing the check? 

O=never 4=sometimes 8=always 

How would you rate the size of the cards? 

O=too small 4=about right 8=too big 

How useful do you find the present w/c system? 

O=af no use 4=useful 8=extremely useful 

How useful do you think the proposed job aid would 

be? 

O=of no use 4=useful 8=extremely useful 

Do you perform the tasks in the order given by the 

present w/c? 

O=never 4=sometimes 8=always 

Would you perform the tasks in the order you 

arranged using the job aid? 

O=never 4=SOmetimes 8=always 

159 

M1 M2 

5 

5 6 

7 8 

4 

7 8 

5 6 

4 1 

6 6 

0 

6 8 

M3 Mean 

3 4 

5 5-3 

5 6.7 

3 3.5 

1 5.3 

1 4 

5 3.3 

5 5.7 

0 

5 6.3 
Appendix - Improving the Reliability of Maintenance Checklists Chapter 8 

Question M1 M2 M3 Mean 

How often do you refer to the present workcard as 4 0 5 3 

you perform a lower-level 2 check? 

O=never 4=sometimes B=always 

How often would you refer to the job aid as you 6 6 3 5 

perform a lower-level 2 check? 

· O=never 4=Sometimes B=always 

How often do you refer to the present workcard as 5 5 7 5.7 

you perform an A-check? 

O=never 4=Sometimes B=always 

How often would you refer to the job aid as you 6 6 7 6.3 

perform an A-check? 

O=never 4=sometimes B=always 

II. Open-Ended Questions 

1. Comments and suggestions on the design of the cards: 

a Size of the cards 

Subject 1: Could be a little smaller to stow in pockets when both hands are 

needed. 

Subject 2: 

Subject 3: 

Shirt pocket with a grommet to allow the cards to fan open, or some 

firm type of clip. • 

Good size for information that is on each card. 

b. Groupings of the tasks 

Subject 1: 

Subject 2: 

Subject 3: 

OK-after rearranging to preference. 

From aircraft access (fwd med) toward nose and around to right buy 

areas (normal course). 

Good idea. I think it's easier to start at the nose gear and continue 

around the aircraft in one complete circle. 

c. Placemarker/instructions page 

Subject 1: OK 

160 
ChapterS 

Subject 2: 

Subject3: 

Appendix - Improving the Reliability of Maintenance Checklists 

Instructions on front as a cover. Check boxes at item number with 

back page having colored stripes-"Check off area" to recall page with 

check. 

Once I got used to doing a check on an aitoraft, I don't think I would 

use the placemarker/instruction card and just use the sign-off sheet 

d. Wording of the cardsfmstructions 

Subiect 1: 

Subject 2: 

Suruect 3: 

Wouldn't hurt to go into more detail. 

Revision date in large print to match sign-off sheet date. Common 

abbreviation naming component only. Include limits. Leave out 

procedure (manuals dictate procedure). 

Simplified and easy to understand. 

e. Ease of understanding the instructions 

Subject 1: 

Subject 2: 

Subject 3: 

Good. 

Very brief-reference changes only-new or limited experience 

personnel should consult MIM until they are confident in their 

procedure. 

The cards are very easy to understand. 

f. Ease of rearranging the order of the cards 

Subiect 1: 

SuQject 2: 

SuQject 3: 

OK. 

Not necessary if color-code by geograpJ;iic areas of aircraft. 

Rearrange the cards in order of doing the check. 

2. How well do you think this idea can be extended to other checks? 

SuQject 1: 

Subject2: 

Subject 3: 

The more involved the check, the more useful the cards. 

Very well. 

Very easily. 

161 
Appendix - Improving the Reliability of Maintenance Checklists Chapter 8 

3. General comments 

Subject 1: I like the card system better. 

SuQiect 2: Its nice to see that people are interested in approaching these tasks in a realworld manner. 

• 

162 
ChapterS Appendix - Improving the Reliability of Maintenance Checklists 

Appendix 8-F 

Evaluation Feedback on Revised Job Aid 

Statistical Data on Respondents 

N= 17 

Age= 36.47(8.15) years 

Number of years in civil aviation = 14.35(7 .58) 

Number of years as a mechanic= 12.94(6.95) 

Number of years as an inspector = 0.29(0.99) 

Number of years performing lower-level 2 checks= 9.59(6.76) 

· Approximate number of lower level 2 checks performed in a month= 15.85(9.07) 

Number of years performing A-checks= 9.59(6.76) 

Approximate number of A-checks performed in a month= 4.65(4.00) 

163 
Appendix -Improving the Reliability of Maintenance Checklists Chapter 8 

Present Job Aid 

Question Workcard Mean (sd) 

Mean (sd) 

How would you rate the ease of readability of the 5.47(1.42) 6.12(1.27) 

text? 

"O=terrible 2=poor 4=fair 6=good 8=excellent 

In general, how easy is the information to 6.06(2.19) 6.12(1.65) 

understand? 

O=very difficult 4=moderately easy 8=very easy 

How would you rate the effort required in 5.35(2.42) 5.59(1.77) 

locating a particular task? 

O=very difficult 4=moderately easy 8=very easy 

What would be the chance of you missing a sign- 5.94(1.84) 6.35(1.27) 

off or a task? 

O=always 2=usually 4=sometimes 6=seldom 

8=never 

How would you rate the ease of physically using 5.47(2.10) 6.06(1.92) 

the workcard/job aid? 

O=very difficult 4=moderately easy 8=very easy 

Would you perform the tasks in the order given 2.47(2.40) 5.18(2.40) 

by the workcard/job aid? 

O=never 2=seldom 4=sometimes 6=usually 

8=always 

• How often do/would you refer to the 5.12(2.42) 5.41(2.09) 

workcard/job aid as you perform a lower-level 2 

check? 

O=never 2=seldom 4=sometimes 6=usually 

8=always 

How often do/would you refer to the 6.18(1.85) 6.47(1.59) 

workcard/job aid as you perform an A-check? 

O=never 2=seldom 4=sometimes 6=usually 

8=always 

How useful do you find the workcard/job aid? 4.06(2.19) 5.12(2.12) 

O=of no use 4=useful 8=extremely useful 

How useful would you find the color-coding of 3.65(1.90) 

the tasks into areas? 

O=of no use 4=useful 8=extremely useful 

164 
ChapterS Appendix - Improving the Reliability of Maintenance Checklists 

How useful do you think the division of tasks 3.82(L98) 

into small cards would be? 

O=of no use 4=useful 8=extremely useful 

Would you rearrange the cards to suit your 5.76(2.44) 

individual work habits? 

O=never 2=seldom 4=sometimes 6=usually 

8=always 

Would you read the interim page at the end of the 6.00(2.21) 

workcard before starting the check? 

O=never 2=seldom 4=sometimes 6=usually 

8=always 

Would you use the grease pencil to make notes 3.88(2.34) 

while completing the check? 

O=never 2=seldom 4=sometimes 6=usually 

8=always 

I 

165 
. . 

Cll \I'TER 9 

SUPPORT OF THE FAA/AANC 

VISUAL INSPECTION RESEARCH PROGRAM (VIRP) 

Colin G. Droty, Ph.D. 

State University of New York at Buffalo 

'I .II 011.1 L l'"ll\ 1: 

This project's objective is to provide human factors 

inspection expertise to support the V tsual 

Inspection Research Program (VIRP). Note: The 

material in this chapter is the result of a collaborative 

effort among many organizations and is not solely the 

work of C. G. Drury, SUNY at Buffalo, or of Galaxy 

Scientific Corporation. 

'1.1 II \Ch:CIWl \ll \\ll \H·:I> 

Over the past two decades there have been several 

studies of human reliability in aircraft structural 

inspection (Rummel, Hardy, & Cooper, 1989; Spencer 

& Schurman, 1994; and Murgatroyd, Worrall, & 

Waites, 1994). All of these studies to date have 

examined the reliability of Non-Destructive Inspection 

(NDI) techniques, such as eddy-current or ultrasonic 

technologies. However, over 80% of civil aircraft 

inspection does not use NDI and is classified as Visual 

Inspection (Goranson & Rogers, 1983). Both the FAA 

(Natiofllll Aging Aircraft Research Program Plan, 

1993, p. 26, p. 35) and the ATA have recognized the 

need for equivalent studies of the reliability of visual 

inspection as a research priority. 

Flight safety is dependent upon airframe integrity; for 

the civil airline fleet, this includes the detection and 

repair of structural defects as they appear. Data on 

airframe structural forces, material characteristics, and 

models of crack growth are used in the Maintenance 

Steering Group-3 (MSG-3) process to determine safe 

inspection schedules. This assumes that there are 

multiple inspection opportunities between the time a 

crack becomes detectable and the time it compromises 

safety. This process is, thus, very sensitive to 

assumptions about crack delectability. For example, 

overestimation of inspection reliability would lead to 

longer inspection intervals, compromising safety. 

Conversely, underestimation of inspection reliability 

would lead to shorter interVals, increasing costs 

because of unnecessary inspection. 

167 

While there is a need to obtain accurate measures of inservice visual inspection reliability, there is also a 

parallel need to understand the process of aircraft 

visual inspection to improve it. There is a large body of 

literature on visual inspection in the manufacturing 

industry (e.g., Drury, 1992), and an increasing number 

of papers applying this to aircraft inspection (e.g., 

Drury, 1995). However, there are still no on-aircraft 

studies which quantify the effects of the many 

variables affecting human factors in visual inspection. 

Thus, a second major goal of the VIRP is to provide 

quantitative evaluations of the effectiveness of visual 

inspection enhancements. 

'1.2 DEFI\ITIO\S 

Quantifying visual inspection is inherently more 

complex than quantifying NDI. Visual inspection 

uses many senses and is expected to detect many 

indications beyond cracks. It may be applied to many 

different structures and surface trea1ments. 

Bobo and Puckett (1994), in the FAA's latest Advisory 

Circular on Visual Inspection for Aircraft, use the 

following definition: 

• 

Visual Inspection is the process of using the 

eye, alone or in conjunction with various 

aids, as the sensing mechanism from which 

judgments may be made about the condition 

of a unit to be inspected. 

Visual inspection involves using the "eye, alone or 

with various aids," and also shaking, listening, feeling, 

and sometimes smelling, the aircraft and its 

components. Additionally, the process of any 

inspection can be analyzed as a combination of various 

functions, the two most important functions are search 

and decision-making (e.g., Latorella & Drury, 1992) . 

In visual inspection, a search process uses most of the 

human body's senses to detect and locate an indication. 

There is then a secondary process of combining 

relevant knowledge, sensory input, and pertinent logic 
Support of VIRP 

to detennine if the indication represents a flaw. The 

inspector must then make a decision whether or not 

this flaw is sufficiently sensitive to pose a risk to the 

continued safe operation of the aircraft or aircraft part. 

The Visual Inspection Research Program uses the 

following definition of "Visual Inspection": 

. Visual inspection is the process of 

examination and evaluation of systems and 

components by use of human sensory systems, 

aided only by mechanical enhancements to 

sensory input, such as magnifiers, dental 

picks, stethoscopes, and the like. The visual 

input to the inspection process may be 

accompanied by such behaviors as listening, 

feeling, smelling, shaking, twisting, etc. 

In addition to defining the process of visual inspection, 

definitions of both the types of indications, i.e., 

potential defects detectable with visual inspection and 

the structure on which this inspection is practiced, need 

to be addressed. 

The types of indication possible in aircraft structures 

were derived from findings at The Aging Aircraft NonDestructive Inspection Center (AANC) and on other 

20 Wear and Tear 

21 

22 

23 

24 

25 

26 

27 

28 

30 Corrosion 

31 

32 

33 

34 

40 Broken 

41 

42 

43 

44 

168 

Chapter9 

documents relating to inspection. A two-level 

classification scheme was developed; each major 

heading was given a two-digit number ending in zero. 

Below this level, individual indication types shared the 

same first digit with the appropriate major heading. 

Table 9.1 shows the current version of this scheme, 

which can be expanded or modified as needed. 

To fully characterize an indication on an aircraft, it is 

necessary to know the type of indication (Table 9.1) 

and the structure on which it is found. As results of the 

baseline inspection of the fuselage area of the AANC's 

Boeing-737 test bed became available, the findings 

were classified into the two-level scheme shown in 

Table 9.2, next page. This table only includes 

structural items needed in the current research; there 

are obviously many more structural elements on an 

aircraft. As with Table 9.1, this classification scheme 

gives sufficient detail for the test bed used in VIRP, but 

should be expanded and modified as necessary to 

better characterize visual inspection tasks. 

From the definitions given in this section, the VIRP 

was able to design representative experimental 

evaluations. 

Loose 

Pulled 

Bent 

Dent 

Scratch 

Frayed 

Leaking 

Lighting Hole 

Pillowing 

Exfoliation 

lntergranular 

Material Missing 

Crack 

Disbond 

Delamination 

Part Missing 
Chapter9 

10 

11 

12 

13 

14 

15 

20 

21 

22 

23 

30 

31 

32 

33 

34 

35 

36 

37 

40 

41 

. 42 

50 

51 

52 

'U m:sH;'\ <H THE\ IRI' 

L:\PERI\IJ:'\TS 

Skin 

Fasteners 

Other Structure 

Other Material 

The research team responsible for designing, 

conducting, and analyzing the VIRP experiments 

includes personnel from Sandia National 

Laboratories/AANC, SAIC, AEA (UK) as well as 

State University of New York (SUNY) at Buffalo. To 

design the experiments, we beld working sessions 

which included airline inspection representatives 

(through the ATA) and FAA Technical Center 

representatives. This group met formally on two 

occasions during 1994 at AANC facility in 

Albuquerque; the research team performed its detailed 

design work outside these meetings. 

Reliability of NDI for crack detection is typically 

reported as one or more Probability of Detection (POD) 

curves, plotted against crack length. As the design 

progressed, it became obvious to the research team that 

visual inspection was a multifaceted activity; unlike 

NDI of cracks, it could not be cbaracterized by a series 

of performance curves plotted against a single 

Support of VIRP 

Doubler 

Extension Skin 

Interior Skin 

Bulkhead 

Panel 

Rivet 

Screw 

Bott 

Support Structure 

Frame 

Stringer 

Track 

Bracket 

Web 

Mount 

Clip 

Rod 

Strap 

Seal 

Paint 

characteristic. While an equivalent curve can be 

generated for visual inspection for the single defect type 

of crack, as Table 9.1 (previous page) shows, it would 

only give a partial description of inspection 

performance. Thus the goals of VIRP were defined as 

follows: • 

A. To establish probabilities of detection for a range 

of different types of visual inspection (cracks, 

corrosion, wear and tear, and mechanical) for a 

"typical" aircraft visual inspection. 

B. To provide quontified "best practice" guidance on 

improving visual inspection reliability. 

A research program was developed based on these 

goals. This process has been described fully in the 

research team's 1994 White Paper on VIRP and is only 

summarized here. 

The VIRP experiments are designed to achieve Goals A 

and B (above) in a series of experiments. Because of 

the large number of factors potentially affecting 

169 
Support of VIRP 

perfonnance, a single experiment cannot economically 

provide a measure of overall perfonnance and 

simultaneously quantify the effects of important 

parameters. Thus, the program was developed as a 

Benchmark Experiment (Goal A), followed by a series 

of Follow-On Studies giving parametric measures of 

various factors of interest (Goal B). 

The detailed protocols for the Benchmark Experiment 

were partly based upon AANC's 1992-94 study of 

human reliability in eddy current inspection (Spencer, 

et al., 1994 ). Because the main vehicle for testing was 

AANC's high-cycle Boeing-737, that aircraft had to he 

subjected to a thorough inspection to determine 

potential indications/defects. This was performed in a 

Baseline study during 1994, using qualified commercial 

inspection personnel to perform a D-check package on 

the fuselage structure. This study's findings were placed 

into a database that could he accessed either by the job 

card ( workcard) on which the defect was found or by 

the defect type. This database was used to develop a 

new set of job cards specific to VIRP, each containing 

known defects. These job cards were often designed as 

subsets of the original job cards so as to include specific 

areas and specific defects of most interest. 

To determine the factors to he included in the 

experimental program's design, the working group 

(ATA, FAA, and research team) listed factors known or 

suspected to affect inspection perfonnance under four 

headings (see Czaja, Drury, & Shealy, 1981): 

• Task: The actions the inspector performs, for 

example: which defects are inspected for, the level 

of inspection, the time constraints, etc. 

• Operator: Individual characteristics of the 

inspector, such as visual ability, training, 

motivation, familiarity with the task. 

• Machine: Details of the structure inspected and of 

the tools used, from mirrors and flashlights to 

layout of the job card. 

• Environment: The surroundings of the inspection 

task. This obviously includes visual, thermal, and 

auditory environments, but can also include 

restrictiveness of access and even managerial 

climate. 

Based on these considerations, the working group 

decided that the Benchmark experiment would he 

conoerned primarily with using the factors to ensure that 

170 

Chapter9 

results would he representative of industry practice. The 

Follow-On experiments would then examine specific 

factors one or two at a time. In Ibis way, any data 

obtained in the Follow-On experiments, e.g., new 

flashlight designs or better training, could he compared 

directly against the Benchmark study to measure the 

effectiveness of any changes in inspection "best 

practice." 

9.3.1 Benchmark Study 

During the benchmark study, a group of inspectors, 

who have not seen the test aircraft previously, will he 

asked to make a visual inspection of specific areas 

defined by the VIRP jobcards. The benchmark will he 

set up as a "typical" scenario by controlling key 

variables. Each inspector will inspect a number of areas 

of the aircraft in order to assess that inter-inspector 

reliability. Videotapes of inspectors performing 

inspection tasks will he made. Following the actual 

aircraft inspection, each inspector will he interviewed 

using a structured interview schedule to elicit his or her 

expert judgments about the factors influencing 

successful performance. Analysis of the results will 

include consideration of the types of errors inspectors 

may make. The outputs of the benchmark study will he 

as follows: 

Quantitative Results 

I. probabilities of detection for different 

flaw/defect types and sizes 

2. inter-inspector reliability 

3. estimate of the effects of inspector 

characteristics included in the design 

(see below) 

• 

Use of videotape as a recording medium will allow a 

classification of whether an unreported defect was due 

to an inspector not reacting to the defect (search 

failure), or reacting, but deciding not to report it 

(decision failure). After Ibis experiment, it will he 

possible to measure the reliabilities of the search 

process and of the decision process so that detailed 

guidance can he given on suitable improvement 

interventions. 

Both factors to he varied in Ibis experiment concern 

difficulty of the task. Job cards were developed to 

provide inspection tasks with either high or low 

physical access difficulty and with high or low visual 

complexity. Twelve experienced airline inspectors, 

recruited through the ATA members, will inspect each 

area of the B-737 test bed over a two-and-a-half-day 
j 

Chapter9 

period (Figure 9.1, next page). They will also inspect a 

sample of the crack test panels developed for the NDI 

eddy-current reliability experiment (Figure 9.2, page 

173) to detennine bow reliable inspectors are on a 

higbly-controlled, but realistic, task of the aircraft. 

Factors to be fixed were chosen so that they would be at 

the ''best practice" level. Thus, only experienced 

inspectors will be nsed. Each will use a good standard 

tool kit (mirror, tlashligbt, etc.), and the jobcards will 

tx; well-designed {Pale!, Drury & Lofgren, 1994). The 

hangar environment is low-noise with minimum 

distractions, and the support stands are sturdy and of the 

correct beigbl. 

In addition to the primaty data of whether or oot each 

inspector detected each defect, secondaiy data will be 

available from a video debriefing procedure. This 

Support of VIRP 

procedure prompts inspectors to describe what they 

were doing, and why they were doing it, during various 

inspection procedures. The procedure we will use is 

called a Retrospective Verbal Protocol (e.g., Ohnemus 

& Biers, 1993). It provides valuable insight into the 

cognitive mechanisms of inspection (e.g., Kleiner, 

Drury, Sharit, & Czaja, 1989). To improve the 

precision of the experiment and to obtain a greater 

understanding of individual factors in ain:raft visual 

inspection, a small battery of tests will be given to each 

subject. These tests, which provide co-variates for later 

analysis, include visual performance, mechanical 

comprehension, and field dependence (e.g., Thackray, 

1992; Drury, & Wang, 1986). 

At. of March 1995, a pilot subject has been tested, and 

the lessons learned were iiiCOipOOiled into the 

Benchmark: Study. Ten test subjects have now been run. 

• 
Support of VIRP Chapter 9 

172 
Chapter 9 Support of VIRP 

173 
Suppon of VIRP 

9.3.2 FoUow-On Studies 

While a large variety of studies are possible following 

the benchmark study, only those of most direct benefit 

to the user community, e.g., to FAA and AT A, will be 

performed as part of the VIRP. The developed 

protocols and the characterized B-737 test bed could be 

used as the basis for specific commercial studies in a 

manner similar to AANCs continuing work in NDI. No 

follow-on studies will be finalized until the results of 

the benchmark study are available; indeed, the design 

of the follow-on studies is likely to be an ongoing 

activity of the group as industry and FAA needs are 

better defined. 

In the White Paper produced before the Benchmark 

Study began, we identified four potential follow-up 

studies: 

1. Effects of fatigue and rest pauses on the detection of 

flaws 

Objective: To assist in providing guidance on 

the effective use of rest pauses or other work 

changes to enhance inspection and to combat 

the effects of fatigue. 

Background: Studies of hlllll,aD reliability in other 

domains have shown that, with fatigneltime on shift, the 

performance of experts tends to deteriorate; in extreme 

instances performance reverts to that of relatively 

untrained personnel. Studies have also clearly related 

the ability to detect signals to levels of attentiveness. 

The negative effects of both of these factors may be 

controlled with rest pauses. Data from this study could 

be compared with that from the benchmark study. 

2. Perceptual factors 

Objective: To form a basis for guidance on 

suitable lighting levels, color enhancements, 

etc., needed to design an appropriate physical 

environment for visual inspection tasks. 

Background: Visual detection will be 

influenced by pertinent factors in the physical 

environment such as contrast, color 

enhancement, light levels, etc. Job aids such 

as flashlights, mirrors, etc., will interact with 

such factors. Aspects such as the color of the 

174 

Chapter9 

inspection surface may affect ease of 

detection. 

3. Search criteria 

Objective: To study the effects of search 

criteria on the probability of detecting flaws 

and to assist in the development of guidance 

on suitable search criteria 

Background: The ability to detect signals has 

been shown to be dependent on the search 

criteria provided, e.g., general.Ym!§ detailed 

inspection. Factors such as the number of type 

of flaws to be searched for may influence the 

probability of detection of both these and 

other types of flaws. 

4. Decision criteria 

Objective: To study the effects of decision 

criteria on the probability of detection of flaws 

and to provide guidance on suitable decision 

criteria 

Background: The criteria provided to or 

assumed by inspectors will influence both the 

hit/miss and false alarm rates. Criteria may 

also be affected by the actual or perceived 

consequences of calling or failing to call a 

flaw. 

'l.-1 CO\CLl SIO\S 

Tbe VIRP is designed to respond directly to industry needs, as expressed through the AT A, and to FAA 

concerns. Over the first year a test bed has been 

characterized, protocols developed, and job cards 

produced so that subsequent studies will benefit in 

terms of reduced d8sign time and effort. As the 

Benchmark study is completed and analyzed (Spring, 

1995), benefits in data handling and analysis for 

subsequent studies will also be available. The whole 

VIRP effort has been unique in the way it has combined 

knowledge of human inspection behavior, experience of 

aircraft inspe«tion, and statistical design of experiments. 

Future experiments will extend the VIRP effort to 

investigate the effects of inspector fatigue, the visual 

environment, and for the criteria used by the inspector. 
Cbapter9 

'!5 Rl I I Rl '\t I·" 

Bobo, S. N. and Puckett, C. R (1994). Visual 

Inspection for Aircraft, Draft Advisocy Circular 

AC 43-<?>, FAA Aging Aircraft Program, Federal 

Aviatioo Administration, Atlantic City 

Intematiooal Airport, New Jersey. 

Czlga, S., Drury, C. G. and Shealy, J. (1981). Human 

Factors in Manufacturing. Unpublished training 

guide. Buffalo, NY: Applied Ergonomics Group, 

Inc. 

Ilruiy, C. G. (1992). Inspectioo perfonnance. In G. 

Salvendy (Ed)., Handbook of Industrial 

Engineering, Second Edition, 88, New York: John 

Wliey and Sons, 2282-2314. 

Ilruiy, C. G. (in press). Human Factors in Aviation 

Maintenance. In Garland, D. J., Wise, J. A and 

Hopkin, V. D. (Eds.), Aviation Human Factors, 

Cbapter 25, New Y ode:: L. Erlbamn Associates Inc. 

Ilruiy, C. G. and Wang, M-J. (1986). Are Research 

Results in Inspectioo Tasks Specific? Proceedings 

of the Human Factors Society 30th Annual 

Meeting 1986, 476480. 

Goomson, U. F. and Rogeis, J. T. (1983). Elements of 

Damage Tolerance Verification, 12th Symposium 

of lntemational Commercial Aeronautical Fatigue, 

Toulouse, France. 

Kleiner, B. M., Drury, C. G., Sbarit, J. and Czaja, SJ. 

"Evaluating the Effects of Automatioo on the 

Human Operator'', Proceedings of the Human 

Factors Society 33m Annual Meeting, 1989, 

733-736. 

Latorella, K. A and Drury, C G. (1992). A framework 

for human reliability in ain:raft inspectioo. 

Proceedings of the Seventh Federol Aviation 

Administration Meeting on Human Factors Issues 

in Aircroft Maintenance and Inspection. 

Wasbingtoo, DC: Office of Aviation Medicine, 

71-82. 

175 

Support of VIRP 

Murgatroyd, R. A., Worrall, G. M. and Waites, C. 

(1994). A Study of the Human Factors Influencing 

the Reliability of Aircroft Inspection, 

AEAifSDIU173, U.K.: AEA Technology. 

National Aging Aircroft Research Program Plan 

(1993). Atlanta City, NY: Federal Aviation 

Administration Technical Center. 

Ohnemus, K. R. and Biers, D. W. (1993). 

"Retrospective versus Concurrent Thinking Out 

Loud in Usability Testing," 37th Annual Meeting 

of the Human Factors & Ergonomics Society, 

Seattle, Washington, October, 1993. 

Patel, S., Ilruiy, C. G. and Lofgren, J. (1994). Design of 

workcanls for ain:raft inspection, Applied 

Ergonomics 1994, 25(5), 283-293 

Runnnel, W. D., Hardy, G. L. and Cooper, T. D. 

(1989). Applications ofNDE reliability to systems. 

Metals Handbook, 9th Edition, Volume 17,674-

688. 

Spencer, F. and Sclmrman, D. (1994). Reliability 

Assessment at Airline Inspection Facilities, Volume 

IJJ: Results of an Eddy Current Inspection 

Reliability Experiment. DOT/FAA/Cf-92/12,ffi, 

Atlanta City, NJ: FAA Technical Center. 

Spencer, F. and Schunnan, D. (1994). Human factors 

effects in the FAA eddy current inspection 

reliability experiment In Meeting Proceedings of 

the 8th FAAIOAM Meeting on Human Factors in 

Aviation Mpintenance and Inspection, 'Trends and 

Advances in Aviation Maintenance Operations", 

16-17 November, Alexandria, VA, 63-74. 

Thackray, R. (1992). Human Factors Evaluation of the 

Work Environment of Operators Engaged in the 

Inspection and Repair of Aging Airr:roft. Report 

No.DOT/FAA/AM-92/3. Washington, D.C.: 

Federal Aviatioo Administration. 
CHAPTER lO 

CORRELATES OF INDIVIDUAL DIFFERENCES IN 

NONDESTRUCTIVE INSPECTION PERFORMANCE: 

A FOLWW-UP STUDY 

Richatd I. Thackray, Ph.D. 

Galaxy Scientific Corporation 

Federal Aviation Administration 

Human Resources Research DMsion 

Human Factors Research Laboratory 

CMI Aeromedical Institute 

Oklahoma City, OK 

111.0 1\TIWIH CTIO\ 

I

n an earlier review of studies and programs dealing 

with nondestructive inspection (NDI) reliability, a 

repeated finding was the existence oflarge individual 

differences among inspectors in their inspection proficiency (FANAAM & GSC, 1993). The few studies 

cited in this review that attempted to determine possible 

reasons for these differences in NDI proficiency were 

generally unsuccessful. 

While the above review was confined largely to NDI 

reliability in the Air Force and the nuclear power industry, a recent study of commercial aviation inspection/repair facilities confirmed that inspector-toinspector differences were a major source of variation 

in the commercial field as well (Spencer & Schurman, 

1994). While differences among facilities in the procedures used (or in the training inspectors received) undoubtedly accounted for some of the differences found 

in this study, it seems unlikely that these factors accounted for all of the variation among inspectors. 

In the review report noted above, research studies of 

individual differences in inspection and vigilance, interviews with NDI training supervisors and inspectors, 

and opinions of experts in the NDI field suggested a 

nornber of skills, aptitudes, and traits, measures of 

which might be relevant to NDI selection and/or proficiency. To explore these possibilities, a study was conducted to examine relationships among many of these 

aptitndes, traits and performance on a simulated eddycurrent inspection task. More specifically, the study 

sought (a) to determine the relationships of various 

predictor measures derived from these skills, aptitudes 

and NDI performance and (b) to examine evidence of 

fatigue changes, if any, over a simulated day-shift pe177 

riod (Shepherd & GSC, in press).1 In addition to these 

primary purposes of the study, a number of other relationships were also examined. A summary of the major 

findings follows: 

• Accuracy of inspection (low nornbers of 

missed faults and false alarms) was found to 

be positively related to test measures of mechanical ability and attention-concentration. 

• Speed of inspection was positively related to 

test measures of such traits as extroversion, 

impulsivity, and lack of meticulousness. 

• Accuracy and speed of inspection were found 

to be unrelated. 

• There were increases in the percentage of 

faults missed and in the percentage of good 

rivets called "faulty" (false alarms) both 

within and between performance sessions 

over the simulated day-shift period. Although 

statisticajly significant, these percentage increases were relatively small, ranging from 

0.8 to 4.5 percent. 

• Expressed liking for inspection was unrelated 

to performance (missed faults, false alarms, or 

speed) on the NDI task. 

• There were no differences between males and 

females in either task performance or in liking 

for inspection. 

1 A more extensive background and rationale for the 

predictor measures employed, as well as the need for 

further information on possible fatigue-related performance changes, were provided in the earlier study 

and are not reviewed here. 
Individual Differences in NDI Performance 

The present study was conducted to follow-up on the 

findings of this previous study .

2 Of particular concern 

was the question of whether the relationships between 

NDI task perfonnance and psychometric measures of 

mechanical ability and attention-concentration would 

hold for a different group of subjects drawn from a 

somewhat different population. A secondary purpose of 

this follow-up study was to re-examine a number of the 

relationships noted above. 

The task employed in this study was a slightly modified 

version of the computer-simulated NDI eddy-current 

task used in the previous study. This task was developed by Drury and his colleagues at the State University of New York (SUNY) at Buffalo and was 

described in detail in the previous study and in studies 

by Drury, Prabhu, Gramopadhye, and Latorella (1991), 

and Latorella, Gramopadhye, Prabhu, Drury, Smith, 

and Shanahan (1992).1t utilized a SUN SPARC workstation and incorporated a standard keyboard and optical three-button mouse as input devices. As Latorella et 

a!. (1992) have emphasized, this task was not developed to devise a simulator that could be used for training on actual NDI tasks, nor was the aim to develop a 

task that could be used to measure absolute values of 

the probability of detecting particular types and sizes of 

faults. The aim was to devise a task that closely approximated the characteristics and requirements of 

eddy-current inspection tasks to enable laboratory investigation of factors that may influence NDI performance. 

The task modification referred to above involved necessary software changes that did not change the essential nature of the NDI simulation but did change some 

of its response characteristics. A software problem 

during the previous study would cause the system to 

malfunction at times, with resulting loss of data. Correeling this problem resulted in a simulation with 

somewhat faster response characteristics. The effects of 

these changed characteristics on task performance will 

be described in subsequent sections. 

10.1 :\11 IIIOilOI.O<;' 

10.1.1 Subjects 

A total of 37 subjects, 18 males and 19 females, 

participated in the study. Subjects ranged in age 

from 18 to 29 years, had normal visual acuity (as detenrtined from an Orthorater screening test), and were. 

paid $10.00 an hour for their participation through an 

2 Unless otherwise noted, the words "previous study" 

henceforth refer always to this earlier study. 

178 

Chapter 10 

existing Federal Aviation Administration (FAA) contract. Most subjects were currently employed and attending a junior college, a vocational institute, a 

military training program, or a local university on a 

part-time basis. Educational levels ranged from high 

school graduate to college graduate. Approximately 

one-third of the subjects were Air Force enlisted personnel assigned to Tinker Air Force Base. 

None of the subjects was an aircraft mechanic or inspector and none had prior training or experience in 

aircraft maintenance or inspection. As in the previous 

study, this ensured a more heterogeneous sample, 

thereby maximizing differences among individuals. 

The inclusion of college students appeared justifiable 

on the basis of several recent studies of inspection perfonnance that used both students and inspectors 

(Gallway, 1982; Gallway & Drury, 1986). The former 

study was reasonably similar to the present one in that 

it involved selection tests and inspection performance. 

Neither study found any significant differences between 

students and inspectors in the. comparisons made. Finally, educational levels in the present study were comparable to those of inspectors in the recent field study 

of NDI reliability conducted by Sandia (Spencer & 

Schurman, 1994). 

10.1.2 Apparatus 

The basic apparatus consisted of a SUN SP ARC Model 

4/50GX-16-P43 workstation, a 19-inch color monitor, 

and a 3-button optical mouse. Although the nature of 

the task and its physical characteristics have been described in the previous study and elsewhere (Drury et 

al., 1991; FAA/AAM & GSC, 1994; Latorella et al., 

1992), task elements are briefly reviewed here. 

• 

The display consisted of four basic task elements 

(windows). These are shown in Figure 10.1 (next page) 

and described in the following sections. 

10.1.2.1 Inspection Window 

The lower left portion of the screen displayed the inspection window and contained the actual rivets to be 

inspected. Although it was possible to present more 

than one six-rivet row of rivets to the subject, only a 

single row was used in this study. Each subject used 

an optical mouse to move the cursor around the circumference of each simulated rivet. The subject was 

free to examine the rivet until he or she decided 

whether or not a crack was present. If the subject decided that a rivet was defective, he or she pressed the 

right mouse button, causing a red cross to appear over 

the "defective" rivet; the words "rivet marked bad" 

appeared on the screen. If the subject decided that a 
Chapter 10 

rivet was nondefective, he or she pressed the middle 

button, causing the words "rivet marked good" to ap-

. pear on the screen. If a subject realized that he or she 

made an incorrect response, it could be corrected by 

pressing the appropriate button. 

When all of the six rivets had been inspected, the subject clicked the left mouse button on the directional 

block labeled "right." This caused a black marker ring 

to circle the last rivet inspected, and the next six rivets 

in the row appeared in the inspection window. 

10.1 2 2 Macro-V"!eW and Directionals 

A macro-view in the upper left portion of the screen 

displayed a side view of the aircraft fuselage and the 

row of rivets being inspected. Since only a small portion of this row was being inspected at any given time 

during the task, the subject could move the cursor over 

the words "Where am I?" in this area and a momentary 

circle would appear over the portion of the rivet row 

currently being examined. 

10,1 2 3 EddY-Current Meter 

The upper right portion of the display contained a 

simulated analog meter that served as the eddy-current 

output indicator. Deflections beyond a set point on the 

meter produeed an audible signal. Meter deflections 

could be caused by: 

U. B. Airhnes 

B •l•t••• 

Individual Differences in NDI Peiformance 

• touching a rivet edge with the cursor or moving the cursor over the head of a rivet 

• the cursor passing over a crack, all of which 

were "subsurface" and invisible 

• the cursor passing over or near simulated corrosion, scratches, or paint chips. (These were 

simulated by 2 mm jagged lines at random locations adjacent to a rivet.) Not all rivets 

contained such "noise," and no rivet contained 

more than one such noise spot. 

10 1 2 4 Lower Bight Window 

The lower right portion of the display could be used by 

the subject to exercise a number of options (e.g., to 

"zoom" to take a closer look at a rivet being inspected, 

to stop the task in order to take a break, or to display 

elapsed time). The only feature used in the present 

study caused a number to appear on each rivet and was 

used only by the experimenter during training feedback 

sessions to enable location and rechecking of rivets incorrectly classified. 

10.1.3 Crack and Meter Characteristics 

As was noted earlier, the developers of this task never 

intended it to be used as a simulator for NDI training or 

to measure absolute values of the probability of detecting particular types and sizes of faults. Their aim was to 

develop a task that, by approximating the characteris0 EJju;l~~ zoom~~~ 

eeeeeBEJ~~ 

0 

0 

Figure 10.1 ~DI Task Srmula:ron (Drury et al 1992) 

179 
Individual Differences in NDI Peiformance 

tics and requirements of eddy-current inspection tasks, 

could be used in the laboratory to investigate factors 

that may influence NDI performance. Nevertheless, to 

provide as much realism as possible, the range (14 to 

350 mils) and mean (approximately 100 mils) of fault 

sizes employed were designed to correspond with those 

that might be encountered in the field and approximated those derived from data reported in the recent 

Sandia eddy-current reliability study (Schurman & 

Spencer, 1994 ). Meter deflection was proportional to 

crack size, with the simulated needle showing a similar 

rapid, abrupt deflection when the cursor passed over or 

was in close proximity to either cracks or noise elements. 

10.1.4 Predictors and/or Task Correlates 

The previous study identified a number of variables, 

measures of which showed significant relationships to 

performance on the NDI task or appeared to warrant reexamination. A few of the tests and measures used in 

the earlier study failed to correlate with any of the performance criteria and were discarded. The variables 

retained included measures of the following: 

• Mechanical Aptitude 

• Attentiveness/Distractibility 

• Extroversion/Impulsivity 

• Motivation/Perseverance 

• Decision Time/ Accuracy 

The tests and measures used for each of these were discussed in detail in the previous study. For purposes of 

review, however, those employed in this study are 

briefly described in the following sections. 

1 0.1.4,1 Subjective Bating Scale ISBS) 

This is a simple self-rating scale that the author developed and has used in numerous studies (e.g., Thackray, 

Bailey, & Touchstone, 1977; Thackray & Touchstone, 

1991) to assess current feeling levels, with measures 

generally taken before and after periods of task performance. The basic instrument consists of five 9-point 

scales measuring the dimensions of attentiveness, tiredness, strain, interest, and annoyance. One additional 

scale measuring effort required to remain attentive 

during task performance was also included. Although 

the previous study failed to show significant relationships of these measures to task performance, this scale 

was retained so as to allow comparisons of feeling 

states of subjects used in the two studies. 

1 0.1.4.2 Bennett Mechanical Comprehension Test 

One of the recommendations of the Southwest Research Institute study of ways to improve NDI techniISO 

Chapter 10 

cian proficiency was to select individuals who scored 

high on mechanical/electronics aptitude (Schroeder, 

Dunavant, & Godwin, 1988). This recommendation 

was also echoed in interviews with NDI instructors; 

they believe that individuals who are above average in 

mechanical aptitude make better inspectors (Shepherd 

& GSC, in press). The previous study found that the 

Bennett Mechanical Comprehension Test, a measure of 

the ability to perceive and understand relationships of 

physical forces and mechanical elements in practical 

situations, shows a significant relationship to performance; individuals scoring higher on the test were more 

accurate in their performance on the NDI task. This 

was the most promising test result found in the previous 

study, and there was a definite need to re-examine this 

finding in the follow-up study. 

1 0.1.4,3 Iypjcal Experiences Inventory 

The ability to resist distraction, if it can be measured, 

would appear to have at least face validity in selecting 

inspectors (Wiener, 1975). The Typical Experiences 

Inventory is a scale developed for use in several previous studies (Pearson & Thackray, 1970; Thackray, 

Jones, & Touchstone, 1973).It consists of a series of 

statements designed to measure ability to work under 

conditions of(a) time stress, (b) threat offailure, (c) 

distraction, (d) social stress, and (e) physical stress. In 

the previous study, the subscale measure of distraction 

susceptibility showed a significant relationship to attitudes towards inspection, i.e., individuals expressing 

dislike of inspection tasks scored higher in distraction 

susceptibility. Because of this finding, it was decided to 

include this scale in the follow-up study. 

10.1 4.4 Aritbmetic and Djgij Span Tests of the Wechsler 

Adult Intelligence Scale IWAI$) • Scores on three subtests of theW AlS (the Arithmetic, 

Digit Span, and Digit Symbol sub tests) have been 

shown in numerous factor analytic studies to measure a 

factor that has been variously named "Freedom from 

Distractibility," "Attention-Concentration," or 

"Concentration-Speed" (e.g., Goodenough & Karp, 

1961; Karp, 1963 ). In the previous study, a factor 

analysis found that the Arithmetic and Digit Span, but 

not the Digit Symbol, loaded highly on the same factor 

that included the Bennett Mechanical Comprehension 

Test. Consequently, the Arithmetic and Digit Span 

subtests were retained in the present study to verify the 

earlier findings. 

10,1 4 5 Eysenck Personality lnventorv IEPn 

The Eysenck Personality Inventory is a short inventory 

that measures extroversion and neuroticism. As indicated in the previous study, extroversion has been 
J 

I 

Chapter 10 

studied extensively in the context of vigilance research 

because of the hypothesis, originally formulated by Eysenck (1967), that extroverts should have more frequent lapses of attention and hence more omission 

enors than introverts. Reviews of the use of this personality dimension in vigilance research (Berch & 

Kantor, 1984; Wiener, 1975) have lent some support to 

the belief that extroverts generally do not perform as 

weD on vigilance tasks as introverts. Much less research has been conducted on personality variables in 

the area of inspection, and no studies of extroversion 

and inspection performance had been conducted at the 

time ofWiener's 1975 review. 

In the factor analysis of the previous stndy, extroversion failed to load on the factor correlated with performance errors, but did load positively on Factor I, 

which was the factor correlating significantly with 

speed of inspection. These findings led to the decision 

to include the Eysenck Test in order to re-examine relationship of extroversion to performance. 

10 1 4.6 Matching Familiar Figures Test IMEFD 

The MFFT is a test developed by Kagan and his associates (Kagan, Rosman, Day, Albert, & Phillips, 1964) 

and consists of a series of 12 "stimulus" pictores, each 

of which is associated with 8 ''response" pictores. Except for the one correct picture in each of the response 

sets, aU differ from the stimulus pictore in some minute 

detail. Subjects point to the pictore they believe to be 

the correct one in each set and continue to point until 

the correct one is identified. Both time to first response 

and number of errors are scored. According to the 

authors, the test measures a cognitive style known as 

reflection-impulsivity. Those who make quick, inaccurate decisions on the test are said to have an impulsive 

cognitive style; those who make slow, accurate decisions are said to have a reflective cognitive style. 

The previous study found a significant inverse relationship between MFFT error scores and scores on the 

W AlS Arithmetic scale, i.e., high scores on the latter 

scale were associated with few errors on the MFFT. 

Because the Arithmetic scale loaded on the same factor 

as the Bennett Mechanical Comprehension Test, it 

seemed desirable to re-examine these relationships in 

the foUow-up study. 

10.1 4J Jackson PersonalitY Research Eonn (PRE! 

The Jackson Personality Research Form (Jackson, 

1974) is a widely used test designed to yield a set of 

scores for personality traits broadly relevant to the 

functioning of individuals in a wide variety of situations. It is a personality test that focuses primarily upon 

181 

Individual Differences in NDI Performance 

areas of normal functioning, rather than psychopathology. 

The Form E used in this study consists of sixteen 

scales, of which four were re-examined in the followup study. The included scales were (a) Endurance, (b) 

Cognitive Structore, (c) Change, and (d) Impulsivity. A 

brief description of each and the reason(s) for its inclusion are as follows: 

• 

• 

• 

• 

Endurance A measure of the willingness to 

work long hours and to be patient and unrelenting in work habits. This was included as a 

possible measure of intrinsic motivation or 

perseverance in task performance. 

Cognitive Structure A measure of the need to 

make meticulous decisions based upon definite knowledge with a dislike of ambiguity 

and uncertainty. It was felt that this trait might 

be positively related to search time, i.e., the 

time spent in searching each rivet for possible 

faults. 

Clwnge A liking for new and different experiences, with a dislike and avoidance of routine activities. Inclusion of this trait is selfevident, since NDI tasks are so often referred 

to as boring and monotonous. 

Impulsivity A measure of the tendency to act 

on the "spur of the moment" and without deliberation. This was included as an additional 

measure of impulsivity to be compared with 

the impulsivity measure derived from the 

MFFT. 

Three of the above scales (Endurance, Cognitive 

Structore, and Impulsivity) were retained in the followup study beca~ they showed high loadings on the 

factor (Factor I) of the previous study that was correlated with speed of inspection. The "Change" scale 

failed to correlate significantly with any of the criterion 

measures of the previous study, bot was included to reexamine its possible relationship to expressed dislike of 

inspection tasks. 

1 0.1.4.8 FIQUre Jlrelerence Jest 

This test is a paired comparison version of the 

Munsinger and Kessen (1964) test of preference for 

complex versus simple perceptual stimuli. Subjects 

choose which pair, of a set of 66 pairs of figure drawings that differ in complexity, they prefer. A recent 

study of industrial workers determined that preference 

for simple stimuli on this test was related to preference 

for repetitive, unchanging work requiring a constant focus of attention (Rzepa, 1984). Although this test failed 

to correlate significantly with any of the criterion 
/ 

Individual Differences in NDI Performance 

measures of the previous study, it did show a significant relationship to measures of distraction susceptibility and was retained as a further possible measure of 

attitude toward inspection. 

1 0.1.5 Procedure 

Upon arrival, subjects were given a brief description of 

the purpose of the research and signed an informed 

consent form. The various tests and measures fanning 

the predictor battery were then administered. Following 

completion of this phase, subjects received practice 

sessions in the use of the mouse, were required to read 

and be tested on a document describing eddy-current 

testing and the need for it, and then began performance 

training. 

The initial phase of training began with practice in use 

of the computer mouse. This was accomplished with a 

display program consisting of a single simulated rivet 

head with a training circle surrounding it. Subjects 

practiced using the mouse and cursor to circle the rivet 

while staying within the circle. After each pre-selected 

block of training trials, feedback was provided consisting of average times required to circle the rivet, and averages of the number of times the cursor head touched 

the rivet or went outside the circle. Training continued 

until the subject reached a consistent level of performance. This usually required I 0 to 20 minutes of practice. 

Training on the inspection task consisted of three separate training sessions, each 60 rivets long. Thirty percent of the rivets in each training session contained 

faults (cracks). In addition, the second and third sessions also contained small, but visible (2 mm) ''noise" 

spots at various locations at or near a rivet. Frequency 

of "noisy rivets" was also thirty percent. Location of 

faults and noise was randomly assigned for each task 

session (both training and subsequent test tasks). Performance feedback was automatically provided after 

each block of 10 rivets. In the first session, training 

circles were provided around each rivet to assist the 

subject in keeping the cursor in the appropriate region 

while circling the rivets; no training circles were used 

in the second and third sessions. 

Following a noon lunch break, subjects performed two 

300-rivet task sessions. These sessions were self-paced, 

and task duration for each subject varied from a minimum of about 60 minutes to the maximum allowable 

duration of 90 minutes. There was a scheduled 15 minute rest break between each session, although subjects 

were told they could take short (10-20 second) 

"stretch" breaks as needed during any session. No 

182 

Chapter 10 

feedback was provided following the task sessions, and 

the frequency of both faults and noise was held at 30 

percent each. 

Subjective rating scales were administered at the beginning and end of each task session. At the end of the 

second session, subjects were debriefed and asked several questions about their performance. These included 

questions about how well they thought they had performed, and whether they felt that inspection was a type 

of work that they could see themselves doing or would 

choose to do on an everyday basis. 

10.2 1{1. Sl I IS \'-IJ DIS( l SSIO'-

10.2.1 Task Performance 

10 2 1.1 Per!ormance Measures· Reliability, 

lntercorrelatjons and General Observations 

Three performance measures were derived from the 

NDI inspection task: (a) percentage of faults 

missed, (b) percentage of good rivets marked faulty 

(false alarms), and (c) mean time per rivet. Of the two 

types of error (failing to detect a faulty rivet or calling a 

good rivet bad), missed faults were more common. On 

the average, approximately 7.8% of faulty rivets were 

missed, while only about 1.2% of good rivets were 

marked faulty. The percentage of false alarms was 

comparable to the 2% obtained in the previous study 

and to false alarm rates found in the recent Sandia/FAA 

study (Schurman, 1993). The percent faulty rivets 

missed, however, was considerably less than the 23% 

missed in the previous study. The most reasonable explanation for this difference between the two studies 

involves the software modifications to the NDI simulation that were mentioned earlier. These changes, by 

eliminating mos~ of the previous slight lag in meter response, apparently increased the likelihood that faults 

would be detected. Test trials conducted by the author 

following the software modifications confirmed that the 

change in meter characteristics did, indeed, increase the 

probability of fault detection. 

The two measures of performance error (percent 

missed faults and percent false alarms) were found to 

be positively correlated (r =.50, p <.OJ), but neither 

was significantly related to speed of inspection (p > 

.OJ). The lack of a relationship between speed of inspection and measures of performance error was consistent with findings of the previous study. However, 

the significant correlation between missed faults and 

false alarms was not anticipated, since the previous 

study found them to be unrelated. Examination of the 

score distributions for these two variables revealed that 

they appeared generally unrelated, except for three in-
1 

. 

J 

Chapter 10 

dividuals who had exceptionally high false alarm rates 

and who were also above average in missed faults. Inclusion of these individuals may have biased the relationship, resulting in a correlation that was spuriously 

high. A nonparametric measure (the Spearman rank order correlation) computed for these two variables failed 

to reach significance (1!>.01 ), suggesting that this 

measure may better approximate the true relationship 

between missed faults and false alarms for this particular set of data. 

1 0.2.1.2 Performance Change Across Sessions 

One of the purposes of the previous study was to examine the data for any evidence of fatigue changes during 

the morning and afternoon sessions. While examination 

of possible fatigue effects was not a principal concern 

of this follow-up study, the earlier study had shown 

some evidence of fatigue-related perfonnance changes, 

and it was decided to compare performance change 

over the two test sessions. Mean values for each perfonnance variable are shown in Table 10.1. 

Analyses of variance revealed a significant increase in 

percent missed faults (f(l/35)=70.7, J!<.OI) and a significant decrease in mean time per rivet (f(l/35)=42.5, 

J!<.OI). Percent false alarms ~hawed no significant 

change <E<I.OO). · 

The changes, although statistically significant for 2 of 

the 3 measures, were relatively small and generally in 

accord with the findings of the previous study. Also 

consistent with the earlier study was the finding of no 

gender differences in perfonnance levels or change 

across sessions. Consequently, gender is not shown as a 

variable in the table. 

183 

Individual Differences in NDJ Peiformance 

10.2.2 Rating Scale Variables 

10 2 2.1 Pre- to Post-Tasl< Changes 

Measures of attentiveness, tiredness, strain, interest, 

and annoyance were obtained for each subject at the 

beginning and end of the two performance sessions. An 

additional item administered only at the end of the performance sessions required subjects to rate the effort 

required to ntaintain alertness when the sessions began 

and when they ended. Mean pre- and post-task values 

for each rating variable are shown in Table 10.2. Separate analyses of variance revealed significant pre- to 

post-task decreases in attentiveness (f(l/36)=36.6, 

J!<.OI) and interest (f(l/36)=64.4, J!<.OI), along with 

significant increases in tiredness (f(l/36)=27 .2, J!<.OI ), 

annoyance (f( 1/36)=9 .I, J!<.O 1), and effort 

(f(l/36)=30.5, J!<.OI). The increase in strain shown in 

Table 10.2 was not significant (f(l/36)=3.8, J2>.01). 

Pre-session ratings indicated that subjects began each 

session feeling moderately attentive, somewhat above 

their normal energy level, moderately relaxed, moderately interested, and not annoyed. Since all variables 

were rated on 9-point scales, with 5 representing the 

midpoint or average value for each feeling state, it is 

apparent that post-session levels for all variables were 

near or below this midpoint value. Thus, subjects could 

not be characterized as inattentive tired strained 

bored or annoyed following the ~rforn:...ce ses~ions. 

Ratings of perceived effort indicated that slight effort 

was required to maintain involvement in the task initially, with moderate effort required towards the end of 

a task session. 

Initial levels of all the rating variables, as well as the 

magnitude and difl'Ction of changes, were remarkably 
Individual Differences in NDI Peiformance 

similar to those obtained in the previous study. This 

clearly indicates that the samples used in both studies 

were comparable in terms of their initial feelings and 

attitudes, as well as in changes that occurred resulting 

from task performance. 

10.2.3 Predictor Variables and Performance 

A number of exploratory analyses were conducted using factor analysis solved for 3 to 5 factors. The clearest relationships were found using a principal 

components analysis with varimax rotation and solved 

for 3 factors. Loadings of each predictor variable on 

the 3 factors are shown in Table I 0.3. A cut-off criterion of .60 was again used to select those variables 

contributing to factor interpretation. This means that a 

variable would have to explain at least 36% of a factor's variance in order for it to be included in a factor's 

interpretation. The factors were identified with labels as 

follows: 

• Factor I -Mechanical Aptitude This factor 

appears to stand alone as an ability factor, in 

contrast to the other factors which represent 

personality dimensions. Three tests loaded· 

substantially on this factor: The Bennett Mechanical Comprehension Test and the W AIS 

Arithmetic subtest showed high positive 

loadings, while the MFFT error score showed 

a high negative loading. The Bennett Test 

would seem to define the factor, while the 

other two suggest important attentional components associated with it. 

• Factor 2- Tirelessness/Patience Scales 

loading positively on this factor (PRF Cognitive Structure and PRF Endurance) suggest a 

meticulous, unfaltering personality style, 

while the negative loading on the PRF Impulsivity scale suggests deliberation and patience. 

184 

Chapter 10 

• Factor 3- Extroversion/Experience Seeking 

This factor is characterized by high loadings 

on the EPI Extroversion Scale and the PRF 

Change Scale. Taken together, these two 

scales would appear to identify an outgoing 

personality dimension with a dislike and 

avoidance of routine activities. 

Pearson product moment correlations between each 

factor score and the various performance criterion 

measures showed only one of the factors to be significantly related to performance. Factor I, which had 

substantial positive loadings on both the Bennett Mechanical Comprehension Test and the W AlS Arithmetic subtest, and a negative loading on the Matching 

Familiar Figures Test error score, was negatively corre· 

lated with missed faults (I=-.62, 1!<.01) and with false 

alarms (I=-.53, 1!<.01). Unlike the previous study, the 

present study found speed of inspection (mean 

time/rivet) to be unrelated to any of the factors. 

Both the present and previous studies find a significant 

relationship between a measure of mechanical comprehension (the Bennett Mechanical Comprehension Test) 

and perfonnance accuracy. This is interesting for several reasons. One reason is that it is consistent with one 

of the recommendations of the Southwest Research 

Institute study of ways to improve NDI technician proficiency. That recommendation, based mostly on 

speculation, was to select individuals for NDI who 

scored high on mechanicaVe1ectronics aptitude 

(Schroeder, Dunavant, & Godwin, 1988). NDI instructors also believe that individuals who are above average in mechanical aptitude make better inspectors 

(FAA/AAM & GSC, 1993). The Bennett Mechanical 

Comprehension Test, as indicated in the manual for this 

test, has been va1idated on various groups of aircraft 

employees, with validity coefficients ranging from .52 
Chapter 10 

to .62. These groups have included shop trainees and 

aircraft factory workers in mechanical jobs (Bennett, 

1969). The fiudings of both the present and previous 

study suggest that the Bennett test may be a useful 

predictor ofNDI performance, as well. This would 

support the above-noted reconunendation of the 

Southwest Research Institute, as well as the opinions 

expressed by NDI instructors, of the relationship between mechanical ability and NDI performance. 

The other two tests loading on Factor I were the 

Arithmetic sub test of the W AIS and the error score of 

the Matching Familiar Figures Test. With regard to the 

first of these, several factor analytic studies have shown 

the W AIS Arithmetic and Digit Span subtests and, less 

frequently, the W AIS Digit Symbol sub test to load on a 

factor that has been variously named "Freedom from 

Distractibility" or "Attention-Concentration" 

(Goodenough & Karp, 1961; Karp, 1963). In the previous study, the Digit Span subtestloaded on the factor 

containing the Bennett, while in the present study the 

Arithmetic subtest showed the highest loadings on this 

factor. Both studies, then, found evidence of an additional dimension (attention-concentration) that was related to NDI task performance. As mentioned in an 

earlier section of this paper, studies by Gallwey ( 1982) 

and Wang and Drury (1989) have also found a relationship of these attention-concentration subtests to inspection performance. Wang and Drury, however, 

noted that while a measure such as theW AIS Digit 

Span correlated with performance errors in some of the 

inspection tasks studied, it failed to correlate in others. 

The authors concluded that the relationships of W AIS 

subtest measures of attention-concentration to inspection performance may have to be empirically determined for different inspection tasks. 

The other variable with a high loading on Factor I was 

the MFFT error score, which loaded negatively on this 

factor. The Matching Familiar Figures Test is, according to its developers, a measure of the cognitive style 

known as reflection-impulsivity (Kagan et al., 1964); 

those making quick, inaccurate decisions on this test 

are said to have an impulsive cognitive style, while 

those who are more deliberate and accurate are said to 

have a reflective style. The high negative loading of the 

MFFT error measure shown in Table I 0.3 (previous 

page), taken in conjunction with the lower, but positive 

loading on the MFFT time measure, suggests that individuals who were slow and act'llf3.te in their performance on the MFFT also tended to be more accurate in 

their performance on the simulated NDI ta•k. However, 

since the MFFT did not show significant loadings on 

the mechanical comprehension factor in the previous 

185 

Individual Differences in NDI Peiformance 

study, the validity of this apparent relationship to NDI 

task performance is questionable. 

10.2.4 Gender, Liking for Inspection, and SeifEstimates of Task Performance 

During the debriefing period, subjects were asked 

whether they thought they might like inspection work 

or could visualize themselves as an inspector. They 

were told that the NDI task they just completed represented only one type of inspection activity and that they 

should try to base their answer on inspection jobs in 

general. The answers were coded "I" if inspection appealed to them and "2" if it did not. This variable was 

then correlated with the predictor measures and with 

performance. Like the findings of the previous study, 

the variable "liking" was not significantly related to 

any of the factor scores or with any measure of performance (p>.O I). The lack of a relationship between 

liking for inspection and actual task performance is 

consistent with findings of Summers (1984) in his follow-up study of the early Air Force "Have Cracks, Will 

Travel" study (Lewis et al., 1978). Summers found no 

relationship between expressed liking for (or dislike of) 

inspection among Air Force technicians and actual NDI 

performance. 

As with the previous study, there was an apparent gender difference in attitudes toward inspection, with 

males showing a greater liking for inspection and females a greater dislike. These data are shown in Table 

I 0.4. A chi-square test, however, revealed the obtained 

gender differences to be nonsignificant (p > .01). AIthough not related to liking for inspection and, as noted 

above, not related to any performance measures, gender was significantly correlated (r = -.62, p < .0 I) with 

scores on the Bennett Mechanical Comprehension Test. 

As with the previous study, males tended to score 

higher than females. This finding is entirely consistent 

with normative data published for the test (Bennett. 

1969) and was expected. However, because of the 

substantial loadings of this test on the factor (Factor I) 

which was signilicantly correlated with performance 

accuracy, an indirect relationship of gender to performance is suggested. 
Individual Differences in NDI Performance 

During debriefing, subjects were also asked to evaluate 

how well they thought they performed relative to others 

perfonning the same inspection task. Twenty-seven of 

the 37 subjects felt their performance was about the 

same as most, nine felt that it was better, and only one 

subject believed his performance to be worse than 

most. Separate 1-tests were conducted to compare the 

performance (missed faults and false alarms) of subjects believing their performance was better than most 

with those who thought it was about the same. None of 

the comparisons yielded significant (p > .0 I) I values, 

showing that perceptions of performance were unrelated to actual performance. The lack of a relationship 

between self-ratings of inspection performance and 

actual NDI performance is in accord with similar findings of the earlier Air Force NDI study (Summers, 

1984) noted above. 

ln .. ~ St \1\1 \In \'\ll CO'\('Jl SIO'\S 

A previous study examined the relationships among 

a number of predictor tests and measures and performance on a simulated eddy-current inspection task 

(Shepherd & GSC, in press). The tests and measures 

employed were intended to tap various skills, aptitudes, 

and traits that research studies of inspection, interviews 

with NDI training superviSors and inspectors, and 

opinions of experts in the NDI field had suggested 

might be relevant to NDI proficiency (Shepherd & 

GSC, in press). While the obtained relationships between a number of the predictor measures and task performance were encouraging, findings were considered 

to be tentative until validated in a subsequent study using a different group of subjects. 

The study reported here was conducted to follow-up 

the earlier results. The basic approaches of the two 

studies, including the procedures followed and task 

employed, were essentially the same. Except for the 

fact that a different group of subjects was used, the 

major differences between this study and the previous 

one were that (a) fewer predictor measures were employed, since those showing no promise in the previous 

study were eliminated and (b) the task sessions were 

shorter, as examination of possible fatigue effects was 

not a principal concern of the follow-up study. A summary and comparison of the principal common findings 

of the two studies follows: 

• Both studies were consistent in finding a significant relationship between scores on the 

Bennett Mechanical Comprehension Test and 

performance accuracy on the simulated NDI 

task, i.e., higher scores on the Bennett Test 

were associated with more accurate NDI task 

186 

Chapter 10 

performance. This finding was the single most 

important of the two studies and supports the 

beliefs and opinions of NDI experts that mechanical aptitude may be a good predictor of 

NDI proficiency. 

• Both studies were consistent in finding a significant relationship between NDI task performance accuracy and scores on W AIS 

measures of attention-concentration. In the 

previous study, the W AIS Digit Span subtest 

showed the greater relationship, while in the 

follow-up study it was the W AIS Arithmetic 

subtest. 

• The follow-up study, but not the earlier one, 

found an apparent relationship between 

MFFT error scores and performance accuracy. Because of this lack of consistency between studies, the validity of this relationship 

is uncertain. 

• There were statistically significant increases 

in the percentage of faults missed during the 

task sessions in both studies. This increase 

occurred over the simulated day shift of the 

earlier study and during the shorter afternoon 

sessions of the follow-up study. The increase 

in percentage of faults missed, however, was 

relatively small in both studies and may not be 

of practical significance. 

• The two studies agreed in finding no relationship between gender and either liking for inspection or performance on the simulated 

NDI task. 

• Liking for inspection was found to be unrelated to task performance in both studies. 

• No relationship existed between speed of inspection and performance in either study. • 

InA RFI LIU '\( ~ s 

Bennett, G. K. Bennett Mechanical Comprehension 

Test· Manual Forms SandT. New York: The 

Psychological Corporation, 1969. 

Berch, D. B. & Kanter, D. R. (1984). Individual differences. In J. S. Warm (Ed.). Sustained anention in 

human performance. New York: Wiley. 

Drury, C. G., Prabhu, P., Gramopadhye, A., & Latorella, K. (1991). Nondestructive testing in aircraft inspection. Report of a pilot study prepared 

under subcontract 89-1014-SC-3 to Galaxy Scientific Corporation, Mays Landing, New Jersey. 

Eysenck, H. J. (1967). The biological basis of personality. Springfield, Illinois: Thomas. 
Chapter!O 

Federal Aviation Administration Office of Aviation 

Medicine and Galaxy Scientific Corporation 

(1993). Human factors in aviation maintenancephase three, volume I progress repon. 

DOT/FAA/AM-93/15, Office of Aviation Medicine, Washington, D. C. 

Federal Aviation Administration Office of Aviation 

Medicine and Galaxy Scientific Corporation 

(1995, in press). Human factors in aviation maintenance- phase Jour, volume I progress repon. 

DOT/FAA/AM-, Office of Aviation Medicine, 

Washington, D. C. 

Gallwey, T. J. (1982). Selection of tests for visual inspection on a multiple fault type task. Ergonomics, 

25, 1077-1092. 

Gallwey, T. J. & Drury, C. G. (1986). Task complexity 

in visual inspection. Human Factors, 28, 585-606. 

Goodenough, D. R. & Karp, S. A. (1961). Field dependence and intellectual functioning. Journal of 

Abnormal and Social Psychology, 63, 241-246. 

Jackson, D. M. Personality Research Form Manual. 

New York: Goshen, '1974. 

Kagan, J., Rosman, B., Day, D., Albert, J., & Phillips, 

W. (1964). Information processing in the child: 

Significance of analytic and reflective attitudes. 

Psychological Monographs, 78, (I, Whole No. 

578). 

Karp, S. A. (1963). Field dependence and overcoming 

embeddedness. Journal of Consulting Psychology, 

27, 294-302. 

Koelega, H. S. (1992). Extraversion and vigilance performance: 30 years of inconsistencies. Psychological Bulletin, 112, 239-258. 

Latorella. K. A., Gramopadhye, A. K., Prahhu, P. V., 

Drury, C. C., Smith, M. A., & Shanahan, D. E. 

(1992, October). Computer-simulated aircraft 

tasks for off-line experimentation. Paper presented 

at the Annual Meeting of the Human Factors Society, Atlanta, Georgia. 

Lewis, W. H., Pless, W, M. & Sproat, W. H. (1978). 

Reliability of nondestructive inspections- Final 

repon. Report No. SA-ALCJMME 76-6-38-1, 

Lockheed-Georgia Company, Marietta, Georgia. 

187 

Individual Differences in NDI Peiformance 

Munsinger, H. & Kessen, W. (1964). Uncertainty, 

structure and preference. Psychological Monographs: General and Applied, 78, Whole No. 9. 

Pearson, D. W. & Thackray, R.I. (1970). Consistency 

of performance change and autonomic response as 

a function of expressed attitude toward a specific 

stress situation. Psychophysiology, 6, 561-568. 

Rzepa, T. (1984). Typological determinants of operator 

functioning in monotonous work conditions. Polish Psychological Bulletin, /5, 135-141. 

Schroeder, J. E., Dunavant, D. W., & Godwin, J. G. 

( 1988). Recommendations for improving Air 

Force nondestructive inspection technician proficiency. SwRI Project No. 17-7958-845, San Antonio Air Logistics Center, Air Force Logistics 

Command, Kelly Air Force Base, Texas. 

Schurman, D. L. Personal communication, September, 

1993. 

Schurman, D. L. & Spencer, F. W. Human factors effects in the FAA eddy-current inspection reliability 

experiment In FAAIAAM 8th Meeting on Human 

Factors Issues in Aircraft Maintenance and Inspection: Trends and Advances in Aviation 

Maintenance Operations. Fmal report of a meeting 

heldNovemher 16-17,1993,Alexandria, VA. 

Spencer, F. W. & Shurman, D. L. Reliability Assessment at Airline Inspection Facilities, Volume Ill: 

Results of an Eddy Current Inspection Reliability 

Experiment. FAA Technical Center, 

DOT/FAAICT-92112, ill, Final Draft, March 

1994. • 

Summers, R. H. (1984). Nondestructive inspection: 

Improved capabilities of technicians: Final Repon. AFHRL-TP-83-63, Training Systems Division, Air Force Human Resources Laboratory, 

Lowry Air Force Base, Colorado. 

Thackray, R. I., Bailey, J.P., & Touchstone, R. M. 

(1977). Physiological, subjective, and performance 

correlates of reported boredom and monotony 

while performing a simulated radar control task. In 

R. R. Mackie (Ed.). Vigilance: 1'h£ory, Operational Peiformance, and Physiological Correlates. 

New York: Plenum. 

Thackray, R. I., Jones, K. N., & Touchstone, R. M. 

(1973). Self-estimates of distractibility as related 
Individual Differences in NDI Peiformance 

to performance decrement on a task requiring sustained attention. Ergonomics, 16, 141-152. 

Thackray, R.I. & Touchstone, R. M. (1991). Effects of 

monitoring under high and low tasldoad on detection of flashing and coloured radar targets. Ergonomics, 34, 1065-1081. 

188 

Chapter 10 

Wang, M. J. & Drury, C. G. (1989). A method of 

evaluating inspector's performance differences and 

job requirements. Applied Ergonomics, 20.3, 181-

190. 

Wiener, E. L. (1975). Individual and group differences 

in inspection. In C. G. Drury & J. G. Fox (Eds.). 

Human reliability and quality control. New York: 

Taylor & Francis. 
I 

J 

1 

' 

/ 

CH.\PTER 11 

TEAMS AND TEAMWORK: IMPLICATIONS FOR TEAM 

TRAINING WITHIN THE AIRCRAFT INSPECTION AND 

MAINTENANCE ENVIRONMENT 

Anand K. Gramopadhye, Ph.D., Subbarao lvaturi, Robert Blackmon, David Krause 

Deparbnent of Industrial Engineering 

Clemson University 

11.0 1\ I ROIH C 110\ 

This report is divided into four sections. In the first 

section, Background and Literature Review, we 

review state-of-the-art literature on team training. In 

the next section, we outline a general framework for 

considering/evaluating tasks' potential for team 

training, also identifying team training strategies for 

improving different team competencies. In the 

section on Team Training for Aircraft Inspection 

Maintenance, we outline implications of team 

training for aircraft/inspection tasks and report results 

of a study evaluating effectiveness of team training 

for ari aircraft maintenance \ask. In the final section, 

Team Training for A & P Schools, we describe how 

team training could be incorporated in an A & P 

school curriculum and provide a functional 

description of a computer-based team training tool. 

We performed this project in close cooperation with a 

major maintenance repair facility and an A & P 

school so that results address the aviation 

community's concerns. 

II. I II \("1\.(;JWl \D \\D 

L1 1"1 R \ I l IU R I \ IE\\ 

11.1.1 lntrodoction 

Previous FAA reports on human factors in aviation 

maintenance (Shepherd, 1991; FAA, 1993) have 

recognized the importance of training. To this point, 

training for aircraft maintenance and inspection 

systems, essentially, bas aimed at improving 

individual skills (Shepherd and Parker, 1990), 

ranging from improving diagnostic skills through 

aircraft maintenance training (Johnson, 1990(a)) to 

acquiring and enhancing visual inspection skills to 

improve airframe structural inspection (Shepherd, 

1993; Gramopadhye et al., 1992). Researchers have 

tended to concentrate on improving the overall 

training program either with training methodology 

189 

(e.g., Drury and Gramopadhye, 1990; Desormiere, 

1990) or with the training delivery system's 

technology for on-the-job training, classroom 

training, tutoring, and computer-based training 

(Gordon, 1994; Johnson et al., 1992; Drury et al., In 

Press). While there has been much study of 

individual skills, there bas been little on developing 

team skills. 

Task analysis of aircraft inspection and maintenance 

activities (Shepherd, 1990) reveals that the aircraft 

maintenance/inspection system is complex, requiring 

above-average coordination, communication, and 

cooperation among inspectors, maintenance 

personnel, supervisors, and members of other 

subsystems-planning, stores, and shops-to be 

effective and efficient. Many maintenance activities 

technicians or inspectors undertake can be performed 

more effectively and efficiently with a team. Though 

the airline industry widely recognizes advantages of 

teamwork (Hackman, 1990), individual AMTs, not 

the teams they work with, are held responsible for 

faulty work. The individual AMT licensing process • and concerns about personal liability often result in 

AMTs and supervisors being unwilling to share 

knowledge and responsibility across shifts or with 

less-experienced, less-skilled colleagues. This 

problem is exacerbated by the fact that experienced 

inspectors and mechanics are retiring and are being 

replaced with a younger, Jess-experienced workforce. 

The newer AMTs Jack the knowledge and skills of 

the experienced AMTs they replace and also are not 

trained to work as a team member. 

The FAA continually addresses the problem of 

individual development of initial AMT skills. The 

newly established Part 66 of the FAR specifically 

addresses significant technological advancements in 

the aviation industry, as well as the past decade's 

advancements in training and instructional 
Team Training 

methodologies. The FAA, through its Office of 

Aviation Medicine, has funded efforts to develop 

advanced training tools for future AMTs. New 

training technologies under development, e.g., 

intelligent tutoring systems and embedded training, 

will be available to A & P training schools. 

Application of new training technologies should help 

reduce tqe gap between AMTs' current skills and 

those skills necessary to maintain advanced systems. 

The effon invested in developing individual skills 

has led to a revised FAR, to new training tools (e.g., 

Johnson, 1990(b); Johnson 1992) applying advanced 

technology, and to development of advanced training 

delivery systems (Gramopadhye, Drury and Prabhu, 

In Press). The area now needing attention is 

development of team skills. In addition to 

fundamental skills, today's employers require 

creativity, an ability to communicate, and an ability 

to work in a team. Team skills are often not welldeveloped or pan of the background of AMTs now 

joining the workforce. The problem is made more 

urgent since the aviation maintenance workforce is 

much younger and less-experienced, usually without 

experience working on military aircraft. The younger 

workfare~ does not carry the passion for airplanes 

older workers expect. An FAA repon (FAA, 1991) 

stated, "People today join airlines for many reasons 

beyond the love of planes. This clear shift plus other 

changes in labor work force confound the longservice employee. Older employees are somewhat 

dismayed with the newer mechanics' acquired skills, 

their las'sez-faire attitude, and their high turnover." 

Inspectors and maintenance technicians are 

challenged to work autonomously while being pan of 

a team. In a typical maintenance environment, an 

inspector looks for and reporrs defects. A 

maintenance person repairs the reponed defect and 

works with the original inspector or the buy-back 

inspector to ensure that work meets standards. During 

the repair process, inspectors and maintenance 

technicians work as a team with colleagues from the 

same and the next shift, as well as with personnel 

from areas like planning or stores, to ensure that the 

task is completed (FAA, 1991). In any typical 

maintenance environment, a technician must learn to 

be a team member, to communicate, and to 

coordinate activities with other technicians and 

inspectors. However, AMTs joining the workforce 

lack team skills. The current A & P curriculum often 

encourages students to compete, so that new AMTs 

often are not prepared to work cooperatively. To 

190 

Chapter II 

prepare student AMTs for workplace realities, we 

need to find new ways to build students' 

technological, interpersonal, and sociotechnical 

competence while incorporating team training and 

communication skills into the curriculum. 

The present study's general objective was to present 

the imporrance of teamwork and team training in the 

aircraft inspection environment by focusing on teams 

and strategies to improve team performance. We 

expected results to help prepare new AMTs for 

teamwork in the aircraft inspection environment. The 

study's specific objectives were the following: 

• To understand the role of teamwork and team 

training in the aircraft inspection/maintenance 

environment 

• To evaluate the effectiveness of a team training 

activity with AMTs from an A & P school 

• To develop guidelines and suggestions for 

incorporating team training in the A & P school 

curriculum 

• To use results obtained from earlier activities to 

develop functional specifications for a computerbased tool for team training. 

To ensure that our project addressed the aviation 

community's needs, we conducted the project in 

cooperation with a major aircraft repair and 

overhauling facility and with an FAA-licensed A & P 

school. 

11.1.2 Literature on Teams 

Teams have rec.l'ived a great deal of attention in 

recent research literature (Salas, et al., 1992; Driksell 

and Salas, 1992; Glickman, et al., 1987). There is 

consensus among those· who study industrial and 

organizational behavior that teams/work groups will 

be the cornerstone of future American industry 

(Cannon-Bowers et al., 1992; Cummings, 1981; Shea 

and Guzzo, 1987). Teamwork will be essential 

because tomorrow's task demands are likely to 

exceed individual capabilities; hence, individuals will 

need to work together more. Teamwork will assume 

a critical role for achieve desired performance. Due 

to inherent complexities of studying teams in 

organizations, the abundant literature is fragmented, 

incomplete, and often contradictory. However, it is 

imporrant to glean from past work any findings that 

can help us understand teamwork, team performance, 

and strategies for improving team skills. 
I 

I 

Chapter II 

The review of the team literature that follows is 

limited to the objectives of this study and to a .greater 

extent resbicted to teams who perfqnn in a complex 

and dynamic environment similar to the environment 

of aircraft inspection/maintenance, which takes place 

at sites ranging from those of large international 

carriers, through startup and regional airlines, to the 

fixed based operators associated with general 

aviation (Drury et at., 1990). Previous FAA reports 

detail the complexity of the aircraft 

inspection/maintenance environment, clearly 

indicating above average coordination, cooperation 

and communication necessary to accomplish tasks. 

Additionally, the importance of teams has been 

emphasized in the Notional Plan for Aviation in 

Human Factors (FAA, 1991}, where both the industry 

and government grnups agreed that additional 

research needs to be conducted to evaluate teamwork 

in the aircraft maintenance/inspection environment. 

11.1.3 Team and Teamwork Defined 

A definition of what constitutes a team facilitates our 

discussion on teams in the aircraft inspection and 

maintenance environment. Throughout the literature, 

team and teamwork are defined differently. The 

following definition of teant is consistent with the 

nature of the effort required for aircraft 

inspection/maintenance tasks (Morgan et al., 1986 

p6): "a team is a distinguishable set of two or more 

individuals who interact interdependently and 

adaptively to achieve specified, shared and valued 

objectives." A number of principles have been 

proposed to ensure that teams work effectively in any 

situntion. Scholtes (1992) suggests that effective 

teamwork depends on the following ten essential 

ingredients: 

I. Clarity in team goals 

2. An improvement plan 

3. Clearly defined roles of team members 

4. Clear communicntion 

5. Beneficial team behavior 

6. Well-defined decision procedures 

7. Balanced participation 

8. Established ground rules . 

9. Awareness of the group process 

10. Use of scientific approach. 

For teams to be effective, its members must work 

collectively to achieve the overall task objective. To 

accomplish an objective, some sort of task 

dependency must exist among team members. 

According to Salas et at. (1992}, the completion of a 

task objective necessitates the following: 

191 

Team Training 

a) exchange: dynamic exchange of information and 

resources among team members 

b) coordination: coordinntion of different task 

activities and adjustments to changes in task structure 

c) organizational structore: some sort of 

organizational structore of members. 

Research in team and teamwork has shown that 

training facilitates the entire team process (Glickman 

et at., 1987; Salas et at., 1992; Swezey and Salas, 

1992). 

Most literature on teams in the aviation industry has 

focused on the CRM (Crew Resources Management) 

training program, which focuses on cockpit training 

for air crews (FAA, 1993; Helmreich, et al., 1989; 

Helmreich and Wilheim, 1991; Foushee and Manos, 

1981 ). CRM typically encompasses several team 

concepts, including team communication skills, 

interaction, situational awareness, assertiveness, and 

leadership skills. Although CRM programs have 

existed for more than a decade, there has been only 

limited use of the programs for maintenance and 

inspection crews. To date, little research has · 

evaluated teams working in the aircraft maintenance 

environment. However, since they realize the 

importance of teams, several aircraft carriers and 

repair facilities have developed in-house training 

programs. These programs often are part of larger 

management training programs, focusing on teaching 

management and non-management personnel to 

improve safety and efficiency (e.g., Robertson et al, 

1994; Taggart, i990). They are not specifically 

developed for maintenance and inspection personnel. 

U.lA Team Evolution 

To understand how training can provide measurable 

changes in team behavior that enhance the efficiency 

and effectiveness of teamwork in aircraft 

maintenance, we must examine the evolution of 

teams. Then we can develop effective intervention 

strategies that can impact teamwork. In recent years, 

several conceptual frameworks and theories have 

been proposed to explain the team-evolution process. 

In this section, we review salient frameworks and 

theories, drawing upon previous researchers' work to 

develop a new framework for understanding the team 

process in the aircraft maintenance environment. The 

theories described below are only representative; our 

aim in including them is to explain team performance 

and training. 

Hackman's (1983) nonnative model offers a 

comprehensive conceptualization of group process in 
Team Training 

the organizational environment. Though the model is 

not developed for a highly structured team, it 

emphasizes organizational input and the effort, skills, 

and strategies of team members bring to accomplish 

team goals. Gersick (1988) described a time and 

transition model for teams, focusing on the dynamic, 

evolving nature of team performance. The model 

shows )low exchange of information and resources 

among team members can result in effective team 

performance. In Gladstein's (1984) Group 

Effectiveness Model, group effectiveness is a 

function of different group processes, such as 

communication and strategy discussions, moderated 

by group task demands, such as task complexity and 

environmental uncertainty. This is one of the few 

models tested with a large sample of teams in the 

work environment. Morgan et al.'s (1986) Team 

Evolution and Maturation Model (TEAM) 

hypothesizes that teamwork develops through several 

phases, beginning with loosely organized groups of 

individuals and proceeding to become a highly 

effective team over time. This model conceptualizes 

a team as going through developmental phases and 

proceeding from ineptness and exploratory 

interactions to the final ievel of effective, efficient 

team performance. The model considers two 

distinguishable types of team activities through the 

steps of team evolution: task-related activities and 

team-related generic activities. Task-related activities 

are associated with developing operational skills to 

perform technical tasks; team-related activities are 

involved in developing team interaction, e.g., 

relationships, coordination, and interaction. 

Other models of team performance emphasize a task 

analytic approach to team training, e.g., Naylor and 

Dickinson, 1969; Shiflett et al., 1982. These models 

consider team performance as a function of the subtask the team has to perform. They imply that the 

organization and task complexity establish optimal 

work and communication and interact to determine 

individual and team training requirements for 

enhanced team performance. Tannenbaum et al. 

(1992) integrate previously described models in a 

framework for team performance and team training. 

Canon-Bowers et al. (In Press) state that, since teams 

operate in diverse work environment performing a 

wide variety of tasks, constructs such as teamwork 

and team training can only be understood in the 

context within which they occur. Tannenbaum et al. 

(1992) proposed framework explains this context. 

192 

Chapter II 

11.2 FR\\IE\\ ORK FOR TEA\1\\'0RK 

I" THE .\IRCI{AFT \IAIJ\TEN.\"'CE 

E:\ \'IIW\ \IEYI' 

Having reviewed various frameworks and 

theories, we now propose our framework for 

considering the team process in the aircraft 

maintenance environment. Drawing from task 

analysis of aircraft inspection and maintenance 

operations (Drury et al., 1990; FAA, 1991), from site 

visits to repair facilities, from observations made 

with training personnel and A & P school instructors, 

and from a detailed review of the team models, we 

developed the framework shown as Figure II.! 

(Chapter II - Appendix). This framework serves as a 

first step for understanding teamwork in aircraft 

inspection and maintenance operations; it could be 

seen as an extension of Tannenbaum et al.'s (1992) 

team effectiveness model. 

The framework illustrates the interaction among 

internal factors, external factors, the team process, 

training strategies, and outcome measures. External 

and internal factors effect the team process. External 

factors are categorized as follows: 

Oraanizational factors: organization's size, type 

(e.g., airline, general aviation, repair facility), 

reward structure, management structure, 

communication norms, and organizational climate. 

Environmental factors: level of environmental stress 

(work conduct in hangars or flight-line) and 

environmental uncertainty. 

EQuipment factors: automation, complexity, 

specialization, equipment availability, and safety. 

Task factors: task organization (type of aircraft 

check: A-, B-, C-, or Heavy-check), task type (e.g., 

avionics, power plant, hydraulics, sheet metal, 

frame), task complexity, and task structure. 

The internal factors, composed of individual and 

team skills, can be categorized as follows: 

Indjyidual skilJs factor: This represents individual 

team members' skills and is best represented by 

AMTs' knowledge, skills, and abilities. In an aircraft 

inspection/maintenance environment, the individual 

skills factor is determined by AMTs' experience 
Chapter II 

working on different aircraft types and with different 

aircraft systems. 

Team s11ms factor: The team members' ability to 

work together productively is dependent on their 

interpersonal skills, on the team's composition, on 

the number of people in the team, and on how long 

members have worked together. We identified team 

skills relevant to aircraft maintenance tasks and 

present them in Table 11.1 (Chapter II - Appendix). 

The name for each team skill is based on suggestions 

by Salas eta! (1992); they were established after a 

comprehensive review of the literature on teams. 

According to Morgan et a!. ( 1986), team skms that 

are isolated and identified can provide a framework 

for team performance assessments. Although attitude 

is not considered a team skill dimension per se, it is a 

"cognitive" entity that can be acquired through 

training (Gagne, 1988); hence, it is shown separately 

in Table 11.1 (Chapter II - Appendix). Previous 

studies have shown that attitude is important for 

teamwork and team performance. 

External and internal factors impact team interaction, 

as well as the team process. However, team 

development is evolutionary: a team matures over 

time (Morgan et a!., 1986). When viewed in light of 

Morgan et al.'s (1986) TEAM model, individual 

skills reflect task behavior and represent team 

members' abilities to perform assigned technical 

tasks; team skms reflect team members ability for 

successful interaction and coordination. Both skm 

acquisition and team evolution can be enhanced 

through training (Morgan eta!., 1987). Specific ways 

for imparting individual training to AMTs has been 

widely covered in the literature; hence, our effort 

focuses only on team training. 

AMTs are members of not only one team, but of 

several teams working on different, yet similar tasks. 

At an aircraft repair facility, an AMI may work on 

different subsystems of various aircraft and with 

different team members over a scheduled 

maintenance period. For such situations, it is critical 

to identify generic skills (Cannon-Bowers, eta!., In 

Press) and to train team members accordingly. 

Cannon-Bowers eta!. refer to these as "transportable 

team skills." At the same time, training AMTs on 

transportable skills, in itself, may not be sufficient to 

ensure successful team performance. For such 

performance, AMTs need training on task-specific 

team skills, focusing on aircraft inspection and 

maintenance tasks. Methodology for this type of 

193 

Team Training 

team training is outlined in the section on Team 

Training. 

The entire team's output can be determined by 

examining the changes in measures of individoal and 

team process and of task performance. 

Individual process measures: These measures 

identify changes in an individoal's task knowledge, 

skills and ability after he or she takes part in a team 

activity, also reflecting changes in an individoal's 

mental model and understanding of an entire task. 

Team process measures: These measures identify 

evolution of new team processes by changes in 

members' specific team skills, i.e., coordination, 

communication, leadership, and interpersonal skills. 

Task perfonnaoce measures: Performance of an 

aircraft inspection or maintenance task is measured 

on the dimensions of accuracy, speed, and safety. 

Accuracy measures the qoality of a job the team 

completed. Speed measures time required to 

accomplish a task. Safety refers to the team members 

ability to adhere to safety procedures by not 

endangering themselves or other team members. 

Measurement procedures used to evaluate teams must 

be sensitive to typical speed/accuracy tradeoffs. 

We used our understanding of teamwork to identify 

specific strategies for training AMTs in A & P 

schools. In the following section, we outline these 

strategies. Later in the report, we identify specific 

team projects which could be incorporated into A & 

P school curricula and report results of the study we 

conducted to evaluate how team training improves 

team skills for an aircraft maintenance task. 

It.3 n: \\1·1 R \1\1\C 

Team performance is a function of the average 

skills of its members. Individoal skills appear to 

be a necessary, but not sufficient, condition for 

effective team performance; and the correlation 

between average skj!! level and average team 

performance is typically small (Bass and Barett, 

1981; Teborg eta!., 1976). According to Steiner 

(1972), team performance is dependent on team 

members' ability to perform assigned tasks and on 

their ability to coordinate work flow and to 

communicate effectively. This process can be 

facilitated by team training. 
Team Training 

Development of a team training program follows 

classic training program development methodology. 

It begins with a thorough analysis of the training 

program's requirements and needs (goals). The next 

step is establishing knowledge, skills and abilities 

necessary for the job; these are used to specify the 

training program's behavioral objectives forming the 

basis for ~valuating the training program. The 

knowledge, skills, and abilities currently required for 

aircraft maintenance does not include team skills. 

Team training is instruction team members receive as 

a unit to enhance team performance (Nieva et a!., 

1978). It includes training strategies to enhance team 

skills. When team training must be combined with 

individual training in a single program, research 

shows team training to be most efficient and effective 

when team members first develop individual skills. 

Swezey and Salas' (1992) taxonomy identifies 

characteristics of team training to incorporate in 

every training program as communication, task 

organization, team decision-making, team 

organization, and information transmission. Specific 

strategies to enhance AMT team skills are outlined 

below. 

11.3.1 Lecture 

Lecture is most appropriate for transportable team 

skills and can be used to introduce basics of teams, 

teamwork, and the role of teams in enhancing 

performance. Lectures are most beneficial for team 

organization/collaboration in identifying the nature of 

interdependencies for team members and developing 

an understanding of the team's structure. AMTs can 

be taught how other members influence their 

performance, what contributions other AMTs make , 

the roles of inspectors, and cleanup crews, and for 

what conditions they must adapt their performance. 

For example, members should know what to do when 

particular equipment is unavailable, when a specific 

inspector is not available or when a member is 

assigned to a new task. Lecture can also be used to 

train AMTs in proper communication by giving 

examples of good and poor communication. AMTs 

can be taught what type of communication-written 

and oral-they should have with other members; to 

whom they must pass information, e.g., writing up a 

non-routine workcard or passing work to the next 

shift; and from whom they must receive instructions. 

Communication includes both technical and nontechnical information. Team members should be 

trained on how to provide and receive performance 

feedback on individual and team performance so that 

194 

Chapter II 

individual members and the team as a whole use it to 

enhance performance. 

11.3.2 Team Meetings 

Team meetings, i.e., group interaction methods, are 

another popular technique (Goldstein, 1986). This 

consists of bringing AMTs together to interact in a 

relatively unstructured environment. Team meetings 

can be effective for analyzing interpersonal problems 

and for developing effective understanding and 

coordination among team members. 

11.3.3 Role-Playing 

Role-playing can be used for training generic team 

skills. Members become aware of each other's roles 

(Cannon-Bowers, et al., In Press) by interacting with 

each other in role-playing situations. They can learn 

the knowledge, skills, and abilities each task requires. 

For example, a mechanic can become aware of skills 

an NOT inspector has and constraints under which he 

or she works. Role-playing helps each member 

develop a better understanding, e.g., mental model, 

of each task and of interdependencies between and 

among tasks. With role-playing, trainees have the 

opportunity to experience on-the-job problems and to 

explore specific solutions to them (Gordon, 1994). 

11.3.4 Task Demonstration 

Task demonstration has been successfully used for 

team training. A task demonstration assists trainees 

by showing where and how individual team members 

make inputs and can be most helpful for contextspecific skills (Gannon-Bowers, et al., In Press). A 

passive demonstration could be a computer 

simulation of a task or an illustration consisting of 

flow diagrams. A passive demonstration helps 

trainees identify critical task elements; determine 

how each team member contributes; understand the 

sequence of subtasks; establish step-by-step 

procedures; and identify requirements for 

coordination, equipment and tooling. For aircraft 

maintenance, when computer simulation of all tasks 

is not feasible, cross-training is possible with 

simulations of representative tasks sharing the same 

critical elements. 

11.3.5 Feedforward Training 

Feedforward training, proven effective for 

individuals (Drury and Gramopadhye, 1990), 

improves performance when applied to teams 
Chapter II 

(Fredericksen and White, 1989). Feedforward 

training can take the forms of physical guidance, 

demonstrations, or verbal advice. It advises team 

members about upcoming situations so that they are 

prepared. For example, trainees learn how a team 

should resolve conflicts arising due to equipment 

being unavailable, or how to respond when 

instruction procedures , e.g., on workcard, are not 

clear and are ambiguous, or when a member is 

assigned a different task. 

11.3.6 Team Decision-Making 

Team decision-making requires educating the team 

on how to utilize various pieces of information to 

reach an optimal decision (Hogan, eta!., 1991). The 

method involves training members on decisionmaking techniques, ranging from decision by 

consensus to brainstorming, to using nominal group 

techniques. Not all these techniques apply to or are 

relevant for training AMT teams. The team decisionmaking dimension is similar to communication 

because teams need to know what, why, where and 

how information can be accessed for optimal 

decisions (Swezey and Salas, 1992). 

11.3. 7 Feedback Training 

Feedback training, i.e., knowledge of results, is 

beneficial for individual skills training (Patrick, 

1992; Czaja and Drury, 1981), and a similar effect 

exists for teams (Dyer, 1984; Nieva, eta!., 1978). In 

fac~ practice without feedback degrades a team's 

proficiency. Cannon-Bowers eta! (In Press) write, 

"Feedback improves skill acquisition and subsequent 

task performance by reinforcing learning, by 

providing cues for goal setting and adjustment, and 

by reducing the negative effects of self-serving 

attributions and social loafing." 

The following factors are essential for providing 

effective feedback: 

Timing: Feedback should be timely. Team 

performance is generally superior when feedback is 

immediate, rather than delayed. 

Focus: Feedback's focus is important. Providing 

feedback on only certain aspects of a task results in 

performance improvements on only that aspect of the 

task. Team training should not emphasize one aspect 

of team performance more than others. 

Sequence: Initial feedback should be provided on one 

aspect of a task; later feedback, on all aspects of a 

195 

Team Training 

task. This sequence allows trainees to focus on all 

aspects of team tasks. 

Feedback Mix: The ratio of individual to team 

feedback also effects team performance. Individual 

feedback should be provided during the initial 

training session to train individuals to a criterion 

level of performance. Feedback on later sessions 

should address team aspects of performance. This 

strategy ensures that individual skills are suitably 

developed before team feedback is provided while 

also preventing individual members from developing 

misconceptions about their own performance when 

the team receives feedback. 

I 1.-t TE \\1 ·1 R \l"\1\(; STl In 

To test the effectiveness and usefulness of team 

training as a strategy for improving team 

performance for aircraft maintenance, we conducted 

a study with AMTs from an FAA-licensed A & P 

school. Current analyses are based on the hypothesis 

that teams successfully completing team training · 

exhibit specific interaction, communication, and 

coordination behaviors enhancing their performance. 

In this study, we addressed the following questions: 

• Does team training effectively improve overall 

team performance? 

• Do effective and less-effective teams display 

different types of team behaviors? 

• Can team training enhance 

interactive/communication behaviors? 

• We designed the experiment described below to test 

the hypothesis and to answer the questions. We do 

not provide complete details below, but eventually 

will publish them as a sequence of technical papers. 

llA.l Subjecbl 

The participants in this study were 24 male students 

AMTs between 20 and 30 years old from an FAAlicensed A & P school. All subjects were in the 

second year of a two-year curriculum. 

11A.2 Tuk 

The task consisted of two distinct sessions: the 

removal and the insrallation of a turbine engine from 

a Beechcraft airplane. Major phases in the removal of 

the engine are external preparation, engine 

preparation, and engine extraction. Major phases in 
Team Training 

engine installation are engine installation, engine 

preparation, and external preparation. Details of each 

phase are outlined in Table 11.2 (Chapter II -

Appendix). We selected this task based on its high 

potential for teamwork. It necessitates more than one 

person and requires a significantly high degree of 

coordination and communication between team 

member~ for its successful completion. 

11.4.3 Procedure 

Each subject completed a demographics form (Table 

11.3, Chapter II - Appendix) and was randomly 

assigned to one of eight three-person teams. Four 

teams served as the control group, and remaining 

four teams received team skills training (this was 

team training group). Initially, all subjects in the 

control group and the team training group received 

individual skills training that provided technical 

information on how a turbine engine works, on the 

theory of turbine engines, and on major steps for 

removing and installing the engine. Subjects also 

received detailed information about different tools 

and their proper uses; tools used are listed in the 

Chapter II - Appendix as Table 11.4. After 

individual skills training, teams in the training group 

received team training. Before starting the team 

training, teams in the training group performed a 

warm-up team exercise (see Chapter II - Appendix, 

Table 11.5). 

The team training program was developed in 

cooperation with trainers and key personnel of a 

major aircraft repair and overhaul facility and 

instructors from an A & P school. The training 

program used some, though not all, of the team 

training strategies we described above. We combined 

the team skills with team training research to develop 

a behaviorally based, team training program focused 

on improving specific team skills. First, we tested the 

team training program using AMTs from our partner 

repair facility for a specific aircraft maintenance task. 

However, we do not repon results of the field study 

at the aircraft repair facility; they are forthcoming in 

other papers. We modified and refined our team 

training program based on the field study's results 

and used the revised version in the current study. The 

training program had five stages, with each stage 

requiring 2-3 hours (see Chapter II -Appendix, 

Table 11.6). Teams remained intact through the 

entire team training process and the study's duration. 

Following team training, teams in the training group 

performed the engine removal and installation task. 

196 

Chapter II 

Teams in the control group performed the same task. 

Unlike the team training groups, control group teams 

performed the task directly after they received 

individual skills training. When they completed the 

entire task, we debriefed all teams and thanked them 

for participating. 

11.5 \11•-.\Sl RI:\C 'I EA\1\\ ORK 

SKILLS,TK\\1 ,\ I"'TI'l DE, .\ND 

T.\SK PERFOR\1 \:\CE 

11.5.1 Teamwork Skills 

A series of recent studies conducted with military 

teams offer insight into measuring the team 

process (Morgan, et al., 1986; Baker and Salas, 

1992). Studies in teamwork assessment show that it is 

possible to observe and record changes in team 

behavior and to discriminate more-effective from 

less-effective teams (Oser, et al., 1989). Our detailed 

review of teamwork measurement literature suggests 

that team process measures rely heavily on 

observation (Schiflett, et al., 1985; Morgan, et al., 

1986) and that team studies use behaviorally 

anchored rating scales for data collection. For the 

current study, assessment tools (rating scales) were 

developed and refined to measure teamwork skills 

and team task performance. 

We collected two types of data on the previously 

mentioned team skill dimensions by interviewing 

team members and instructors. One type of data 

reflected instructors' observations; the other, team 

members' perceptions. We collected the first type of 

data with the instructors' interviews (Chapter II -

Appendix, Tabfe 11.7). We collected the second type 

with the post-session interviews (Chapter II -

Appendix, Table 11.8). Both the interviews use a 

Likert-type, seven point, agree-disagree scale: 

trainees and instructors indicated their response to 

each item. Instructors and student AMTs completed 

the respective intervies on completion of each 

session, i.e., engine removal and engine installation. 

11.5.2 Team Attitude 

Attitude measures attempt to gauge the trainees' 

opinions about whether they believe that training and 

teamwork will improve team performance. One of 

the most popular attitude measurement 

questionnaires is the CMAQ (Cockpit Management 

Attitudes Questionairre) for assessing commercial 

aviators' attitudes about team training (Helmreich et 

al., 1986). In the current study, we used a modified 
I 

Chapter II 

version of an attitude questionnaire (Chapter II -

Appendix, Tables 11.9 and 11.10) in our interviews, 

administering it to student AMTs before the study's 

commencement and after its completion. 

11.5.3 Task Performance 

In addition to data on team behavior, data were also 

collected on speed, accuracy, and safety measures. 

We recorded this data using the data collection 

instrument in Chapter 11 - Appendix, Table 11.11. 

Data were collected on the above-listed task 

performance measures for each phase of the engine 

removal and engine installation tasks. Results are 

reported with the Task Performance Summary Table 

(see Chapter II- Appendix, Table 11.12). 

11.6 RESl LIS \"\D DISCl SSIO"\ 

This study's results are indicative since 

comparisons are based on only four teams per 

group (training, control). However, these results do 

generally indicate that we are heading in the right 

direction. The data collection instruments and task 

performance summary provided data for 24 

individuals from 8 teams. These data are reported in 

this section, divided into fiadings based on data from 

the instructors' evaluations, from self-evaluations, 

and from the task performance summary. 

Figures 11.2 and 11.3 (Chapter II -Appendix) show 

instructors' overall ratings for the trained and 

untrained teams on each team skill dimension. The 

instructor's ratings on the instructors' interview were 

mapped onto different team skills. The chart shows 

that teams which had team training were ranked 

equal to or better than teams which did not have team 

training on each team skill dimension for both engine 

removal and engine installation phases. These results 

suggest that teamwork skills of the teams receiving 

training were perceived to he much better than those 

of teams not receiving training. Since no data were 

collected on individual team members, it is not 

possible to assess each individual's relative 

performance. 

It is interesting to· note that performance differences 

between l)"ained and untrained teams are much larger 

on the engine removal phase (first session) than on 

the engine installation phase (second session). Teams 

which did not receive training showed improvement 

and better teamwork in the latter phase (engine 

installation). This could he because team interaction 

patterns are established, lessons are learned, and 

197 

Team Training 

communication norms develop. as the task proceeds. 

Experience helps refine the team's interaction 

process so that it works more effectively on 

subsequent tasks. Much of the team evolution and 

maturation process for teams not receiving training 

was completed "on-the-job," while a large portion of 

this process for trained teams was completed during 

training. Despite differences, the data indicate team 

evolution and maturation effects for both teams. 

These results add weight to the claim that effective 

team behaviors can he identified and enhanced by 

having teams engage in those behaviors in a training 

environment. 

To understand individual team members' perception 

of their team's performance, we analyzed the Postsession self-evaluation interview. Results are 

reported in the Chapter II - Appendix as Figures 

11.4 and 11.5. Although the instructors' analysis of 

trained and untrained teams revealed a large 

difference in various team behaviors, we did not find 

a similar large effect here. Nevertheless, results of the 

self-evaluation interview are that the trained group's 

mean score was higher than the control group's on 

five of six team skills measures on the engine 

removal task and on four of six measures on the 

engine installation task. To gauge teams' attitudes 

towards teamwork and their understanding of the 

principles of teamwork, we analyzed pre- and posttraining interviews. Figure 11.6 (Chapter 11 -

Appendix) shows that, although scores for both the 

trained and the untrained groups are comparable on 

the pre-training interviews, there are differences on 

the post-training interviews. The trained group's 

higher scores on six of eight questions reflect the 

effect of training jn and understanding of teamwork 

and team principles. 

To understand whether improved team performance 

translated into improved task performance, we 

collected task performance measures for both groups. 

The data for the trained and control groups are 

summarized in Table 11.12 (Chapter II - Appendix). 

Measure I relates to speed; measures 2, 3 and 4, to 

accuracy; and measures 5 and 6, to safety. Teams in 

the untrained (control) group required significantly 

more time to complete the engine removal task .. 

However, there was not a large difference on the 

engine installation task. This result could he 

attributed to the lack of coordination and 

communication among members of the control group 

present in the first stage and absent in the second. 

Over time, teams in the control group improved 

coordination and communication, resulting in 
Team Training 

reduced task time on the engine installation task. 

Similarly, the trained group made fewer errors for 

both engine removal and installation tasks and had 

superior scores on accuracy measures 2, 3 and 4. No 

significant differences were observed between the 

groups on safety measures. The most important result 

is that trained teams with effective team behaviors 

were ov~rall more effective and more efficient. 

Trained teams demonstrated more behavior involving 

coordination and communication skills, i.e., 

coordinating gathering information, conveying the 

right information to the right person at the right time 

in the right format, receiving relevant information; 

error-correction skills, i.e., providing team members 

with performance feedback and helping resolve 

errors; and interpersonal skills; i.e., leadership, 

displaying appreciation for help provided, and 

making team-building statements. These behaviors 

resulted in improved task performance. 

A correlation exists between successful team 

behavior and task performance. Though limited in its 

sample size, this study's results indicate that training 

AMTs on team skills improve coordination and 

communication skills. In turn, this translates into 

improved task performance. 

11.7 CONCLl SIONS 

This study was a first effort devoted expressly to 

evaluating the effect of team training in the 

aircraft maintenance environment. The study's 

implications are encouraging as to the potential team 

training has for improving team performance and 

overall task performance. We draw the following 

specific conclusions from this study: 

• It is possible to identify team skills and to train 

student AMTs in teamwork skills critical for 

successful team performance in the aircraft 

maintenance environment. 

• Teams which receive team training exhibit a 

larger percentage of behaviors related to team 

performance. Also, results suggest that members 

of teams which did not receive team training do 

not exhibit the high percentages of team 

behaviors as members of more-effective teams. 

Based on this study's results, training for student 

AMTs should emphasize generic and context-specific 

team skills, focusing on coordination, 

communication, interpersonal, and leadership skills. 

Our findings provide insight for developing future 

198 

Chapter II 

team training systems and for improving existing 

instructional technology. The elements of the team 

training program outlined in this study can easily be 

incorporated into A & P school curricula to prepare 

student AMTs for teamwork. Further, elements of the 

team training program can also be incorporated into 

formal methodology used to train AMTs at different 

aircraft sites. The operational setting for the current 

study provided the opportunity to observe teams in 

the field, rather than in a laboratory. Although results 

are encouraging, additional team research is needed 

to fully understand complex interactions existing in a 

team environment for different tasks and conditions. 

The following section outlines how team training can 

be incorporated in a typical A & P school curriculum 

and provides a functional description of a computerbased tool for team training which will be developed 

under Phase VI of this contract. 

li.S FUTl 11U: APPLICATIONS OF 

TEA !\I TRAINING\\ ITIIII\ A & I' 

SCHOOL CliJUUCliLU\1 

The previous study demonstrated team training's 

effectiveness for improving both teamwork skills 

and task performance for a specific aircraft 

maintenance task, using student AMTs. The results 

of the controlled study and recognition of the 

important role of teamwork establish a need to 

identify team projects which can train student AMTs 

in teamwork skills and prepare them for cooperative 

environments. This section outlines specific teamtraining projects which could be used in a typical 

FAA-licensed A & P school curriculum. Table 11.13 

(Chapter II - ;l,ppendix) outlines a typical A & P 

school curriculum, and Chapter II - Appendix, Table 

11.14 presents a condensed overview of various team 

projects which could be incorporated therein. " ------- ----

11.8.1 Computer-Based Tool for Team Training 

As computer-based technology becomes increasingly 

cheaper, the future will see an increased application 

of advanced technology in training. Over the past 

decade, instructional technologists have provided 

numerous technology-based training devices 

promising improved efficiency and effectiveness. 

Examples include computer simulation, interactive 

video discs, and other derivatives of computer-based 

applications (Johnson, 1990(a)). The compact disc 

read only memory (CD-ROM) and digital video 

interactive (DVI) are examples of other types of 

technologies which will provide future "multi-media" 
Chapter II 

training systems. Technologies such as ComputerAided Instruction (CAl), Computer-Based Training 

(CBT), and Intelligent Tutoring System (ITS) are 

being used today, ushering in a revolution in training. 

Several new technologies have found a place in 

maintenance trai~ting (Johnson, 1990(a), 1992; 

Shepherd, 1992). 

Hypermedia is a tooVinstructional system finding 

acceptance as a tool for lear~ting among learning 

theorists. Hypermedia involves non-linear 

organization of information, linking together discrete 

blocks (chunks) of information to create an 

information network. It can also be seen as a nonsequential method for presenting and accessing 

information in which users can move freely 

according to their needs. Hypermedia information is 

multimedia: text, graphics, animation, and audio. If 

information is only text, it is known as hypertext. 

Hypermedia systems have found extensive use in 

applications ranging from browsing to training. 

Jonassen and Gabringer (1990) list examples of 

hypermedia in instructional tools such as language 

learning, science teaching, and browsing in 

encyclopedias. Christensen, et al. (1993) developed a 

hypermedia-based instructional tool for teaching 

hypermedia system design. Koshy, et al. (In Press) 

developed a hypermedia version of a maintenance 

manual for diagnostic training. In each case, 

hypermedia was useful for learning and training 

applications. 

The current research effort was devoted expressly to 

facilitating understanding and to examining how 

team members interact and how team training can 

facilitate teamwork in the aircraft maintenance 

environment. Having met these goals, our next step is 

to consider training media which uses instructional 

techniques developed in this phase of the research in 

order to develop a training program enhancing team 

skills. Hypermedia has the potential to enhance 

learning and could prove to be useful for improving 

certain aspects of teamwork. In the next phase of our 

research, we propose to develop a hypermedia-based 

training tool designed to support learning teamwork 

in the aircraft inspection and maintenance 

environment. We provide a functional description of 

the proposed training tool below. 

11.8.2 Functional Description 

The Aircraft Maintenance Team Training (AMTT) 

software will be a computer-based hypermedia 

system for team training. It will be developed for 

199 

Team Training 

student AMTs, focusing on generic and contextspecific team skills. The system will be programmed 

using Visual Basicffool Book to operate on an IBMcompatible computer ( 486 DX2/66 Hz , 8 Mb of 

RAM), using Microsoft Windows and utilizing 

multiple media such as sound, text, animation and 

graphics. AMTT will consist of the two basic 

modules and other sub-modules outlined below. 

11,8.2.1 The Trainee's Module 

The Trainee's Module will train AMTs on various 

aspects of teamwork, including generic and contextspecific team skills. It will include the following 

basic elements: 

11.8.2.1.1 Team Overview Module 

Introductjon· This module will introduce trainees to 

the basics and o~ectives of teamwork (team 

mission). This module will use the Landing on the 

Moon exercise to demonstrate the importance of 

teamwork. The importance of and need for teamwork 

in aircraft inspection and maintenance will 3Iso be 

emphasized, identifying basic team skills and 

illustrating each skill's importance. 

Tools for Makjog Team [)ecisions: This submodule 

will introduce trainees to decision-making 

techniques, providing examples of using the 

techniques in the aircraft maintenance environment. 

Team Communication: This submodule will 

introduce trainees to aspects of written and 

verbal/nonverbal team communication, providing 

illustrations of appropriate and inappropriate 

communication in the aircraft maintenance 

environment. Specijically, communication examples 

will focus on: format, direction, frequency, length, 

conditions, context, and time. The importance of 

good communication for team performance will be 

emphasized. 

Team Feedback: This submodule will provide 

trainees with guidelines for providing, receiving, and 

using feedback to communicate with other AMTs 

clearly about how tasks are being performed. 

Team CoonJjnatjon· This submodule will focus on 

the coordination required for team members to ensure 

well-orchestrated teamwork. 

Team I &adersbjp· This submodule will focus on the 

critical role of team leadership for accomplishing 

team tasks. For example, team members will be 
Team Training 

shown how to handle information overload under 

stressful conditions, specific behaviors exhibiting 

leadership and assertiveness, and methods of 

motivating others. 

Team Evaluation: This submodule will expose the 

trainees to the instruments used to evaluate individual 

and team performance on a task. 

Each submodule will first introduce trainees to basic 

principles and the provides examples applying the 

principles to enhance teamwork in the aircraft 

maintenance environment. Trainees will make an 

active response as they are exposed to new material 

and will be provided with immediate feedback as to 

their answer's correctness. This stage will be 

followed by a question and answer session for the 

material. 

11.8.2.1.2 Team Building Exercise Module 

This module's objective is to demonstrate the 

application of basic principles ofteamwork 

emphasized in the Team Overview Module. Trainees 

will undertake a series of exercises requiring them to 

demonstrate their underslllnding of principles. The 

training will use training strategies such as roleplaying, feedforward, and feedback. For example, 

roles of various team members will be modeled for 

certain task situations, using knowledge from experts. 

Examples of how interactions could proceed, with 

examples of poor and good behavior, will be 

demonstrated via simulation. Trainees will comment 

on the behavior's appropriateness and will be asked 

for inputs or suggestions to improve team 

performance. Trainees will be given guidance and 

feedback during and after the session. 

11.8.2.1.3 Task Simulation Module 

This module will provide trainees with graphical 

demonstration, animation, and flow charts of 

different scenarios for select aircraft maintenance 

tasks. Team members using this module can interact 

cooperatively to identify ways to improve teamwork 

for the representative simulated aircraft maintenance 

tasks. 

11.8 2 2 The Instructors Module 

11.8.2.2.1 Assessment Module 

This module will provide the instructors with a 

means to assess trainees' understanding of using team 

principles and will allow instructors to evaluate 

trainee's and the team's performance while 

200 

Chapter II 

interacting with AMTT software. The module will 

provide the instructor with various data collection 

instruments used by both trainees and instructors. 

11.8.2.2.2 Report Generation Module 

The Report Generation Module will allow instructors 

to print reports of results. It will also allow 

instructors to generate printouts of data collection 

instruments and select material in the Team 

Overview Module. This will allow instructors to use 

the material in a classroom environment and to use 

data collection instruments for field study. 

Bass, B. M., and Barrett, B. V. (1981) People, work, 

and organizations. Boston: Allyn and Bacon. 

Cannon-Bowers, J. A., Tannenbaum, S., Salas, E., 

and Volpe, C. (In Press) Defining team 

competencies: Implication for training 

requirements and strategies. To appear in R. 

Guzzo and E. Salas (Eds.) Team effectiveness 

and decision-making in organizations, Frontier 

Series in Industrial and Organizational 

Psychology. 

Cannon-Bowers, J. A., Oser, R., and Flanagan, D. L. 

(1992) Work teams in industry: A selected 

review and proposed framework. In R. Swezey 

and E. Salas (Eds.), Teams: Their training and 

peiformance, Norwood, NJ: Ablex, 355-377. 

Christensen, M., Giamo, M., and Jones, T. (1993) 

Support for, teaching the design and 

implementation of multimedia/hypermedia 

systems. S/GCSE Bulletin, 25(1), 242-246. 

Cummings, T. G. (1981) Designing effective work 

groups. In P. C. Nystrom and W. Starbuck (Eds.) 

Handbook of organizational design (Vol 2.) 

London: Oxford University Press. 

Czaja, S. J. and Drury, C. G. (1981). Training 

programs for inspection, Human Factors, 23, 4, 

473-484. 

Desormiere, D. (1990) Impact of new general aircraft 

on the maintenance environment and work 

procedures. Proceedings of the Fifth FAA 

Meeting on Human Factors in Aircraft 

Maintenance and Inspection, Atlanta, GA, 124-

134. 
Chapter II 

Dickinson, T. L., Salas, E., Converse, S. A., and 

Tannenbaum, S. I. (1987) Impact of task work 

structure in team performance. In G. Lee (Ed.), 

Proceedings of psyclwlogy in the department of 

defense symposium. Colorado springs, CO: US 

Department of Commerce. (Quoted in Salas, E., 

Dickinson, T. L., Converse, S. A., and 

Tannenbaum, S. I. (1992) Toward an 

understanding of team performance and training. 

In R. Swezey and E. Salas (Eds.) Teams: Their 

training and performance, Norwood, NJ: Ablex, 

3-29). 

Driskell, J. E. and Salas, E. (1992) Collective 

behavior and team performance. Human 

Factors, 34, 277-288. 

Drury, C. G. and Gramopadhye, A. K. (1990) 

Training for visual inspection. In Proceedings of 

the Third Federal Aviation Administration 

Meeting on Human Factors in Aircraft 

Maintenance and Inspection: Training Issues, 

Atlantic City, New Jersey. 

Drury, C. G., Prabhu, P. V., and Gramopadhye, A. K. 

(1990) Task analysis of ~rcraft inspection 

activities: Methods and findings. Proceedings of 

the Human Factors Society 34th Annual 

Meeting, Sonta Monica, 1181-1185. 

Dyer, J. (1984) Team research and team training. A 

state-of-the-art review. In F. Muckier (Ed.) 

Human Factors Review, Santa Monica, CA, 

Human Factors Society, 285-323. 

FAA (1993) Human Factors in Aviation 

Maintenance-Phase Three, Volume I Progress 

Report, DOT/FAA/AM-93/15. 

Foushee, H. C., and Manos, K. L. (1981 ). 

Information transfer within the cockpit: 

Problems in intracockpit communication. In C. 

E. Billings and E. S. Cheaney (Eds.) Information 

Transfer Problems in the Aviation System, 

(NASA Technical Paper 1875; pp 63-71). 

Moffett Field, CA: NASA-Ames Research 

Center. 

Fredericksen, J. and White, B. 0989) An approach to 

training based upon principled task 

decomposition, Acta Psychologica, 71, 89-146. 

Gagne, R. M. (1985) The conditions of learning. 

New York: Holt, Rinehart and Winston. 

201 

Team Training 

Gersick, C. J. G. (1988) Time and transition in work 

teams: Towards a new model of group 

development, Academy and Management 

Review, 31,9-41. 

Gladstein, D. L. (1984). Groups in context: A model 

of task group effectiveness. Administrative 

Science Quarterly, 29,499-517. 

Glickman, A: S., Zimmer, S., Montero, R. C., 

Gurette, P. J., Campbell, W. J., Morgan, B. B., 

and Salas, E. (1987) The evolution of teamwork 

skills: An empirical assessment with implications 

for training. Technical Report No. 87-016, 

Human Factors Division, Naval Training 

Systems Center, Orlando, Florida. 

Goldstein, I. L. (1986) Training in organizations: 

Needs assessment, develapment, and evaluation. 

Monterey, CA: Brooks/Cole Publishing 

Company. 

Gordon, S. E. (1994) Systematic training program 

design: Maximizing effectiveness and minimizing 

liability. Prentice Hall: New Jersey. 

Gramopadhye, A. K., Drury, C. G., and Prabhu, P. V. 

(In Press) Training for Visual Inspection (Under 

review Applied Ergonomics) 

Hackman, J. R. (1990) Groups that work. San 

Francisco: Jossey and Bass. 

Hackman, J. R. (1983) A normative model of work 

team effectiveness (Tech. Report No. 2). New 

Haven, CT: Y>jle University. 

Helmreich, R. L., Foushee, H. C., Benson, R., and 

Russini, R. (1986) Cockpit management 

attitudes: Exploring the attitude-performance 

linkage. Aviation, Space, and Environmental 

Medicine, 51, 1198-1200. 

Helmreich, R. L., Chidester, T. R., Foushee, H. C., 

Gregorich, S. E., and Wilhelm, J. A. (1989). 

Critical issues in implementing and reinforcing 

cockpit resource management training, NASAIUT 

Technical Report No. 89-5. 

Helmreich, R. L. and Weilhem, J. A. ( 1991 ). 

Outcomes of crew resource management 

training. The International Journal of Aviation 

Psyclwlagy, I, 287-300. 
Team Training 

Hogan, J., Peterson, A., Salas, E., Reynolds, R., and 

Willis, R. (1991) Team Performance, Training 

Needs and Teamwork: Some Field Observations, 

Technical Report- 91-007, NTSC. 

Johnson, W. B. (1990(a)) Advanced Technology 

Training for Aviation Maintenance, Final Report 

of the Third FAA Meeting on Human Factors 

Issues in Aircraft Maintenance and Inspection, 

Atlantic City, New Jersey, 115-134. 

Johnson (1990(b)). Advanced Technology For 

Aviation Maintenance Training: An Industry 

Status Report and Development Plan, 

Proceedings of the Human Factor Society 34th 

Annual Meeting. 1171-1175. 

Johnson, W. B. (1992). Integrated information for 

maintenance training, aiding, and on-line 

documentation. Proceedings of the Human 

Factors Society 36th Annual Meeting, 87-91. 

Johnson, W. B., Norton, J. E., and Utaman, L. G. 

(1992) New technology for the schoolhouse and 

flightline maintenante environment, 

Proceedings of the Seventh FAA Meeting on 

Human Factor Issues on Aircraft Maintenance 

and Inspection, Atlanta, GA, 93-100. 

Jonassen, D. H. and Gabringer, R. S. (1990) 

Problems and issues in designing hypertext/ 

hypermedia for learning. In Jonassen, D. H. and 

Mandl, H. (Eds.) Designing Hypermedia for 

Learning. NATO AS! Series, Springer-Verlag, 

Berlin Heidelberg. 

Koshy, T., Gramopadhye, A. K., Kennedy, W. J., and 

Ramu, N. V. Application of hypertext 

technology to assist maintenance on the shop 

floor (In review Computers and Industrial 

Engineering) 

Latorella, K. A., Gramopadhye, A. K., Prabhu, P. V., 

Drury, C. G., Smith, M.A., and Shanahan, D. E. 

(1992) Computer-simulated aircraft inspection 

tasks for off-line experimentation. Proceedings 

of the Human Factors Society 36th Annual 

Meeting, 92-96. 

Morgan, B., Salas, E., and Glickman, A. (1987) 

Teamwork from team training: An assessment of 

instructional processes in navy training systems. 

Proceedings of the lnterservice/lndustry 

202 

Chapter II 

Training Equipment Conference and Exhibition, 

Washington DC 

Morgan, B. B., Glickman, A. S., Woodard, E. A., 

Blaiwes, A. S., and Salas, E. (1986) 

Measurement of team behavior in a navy 

environment. Technical report TR-86-014, 

Orlando, FL: Naval Training Systems Center, 

Human Factors Division. 

Naylor, J. C. and Dickinson, T. L. (1969) Task 

structure, work structure, and team 

structureperformance, Journal of Applied 

Psychology, 53, 163-177. 

Nieva, V. F., Fleishman, E. A., and Reick, A. (1978) 

Team dimensions: Their identity, their 

measurement, and their relationships (Contract 

No. DAHC19-78-C-0001), Washington, DC: 

Advanced Research Resources Organization. 

Oser, R., McCallum, G. A., Salas, E., and Morgan, B. 

B. (1989). Toward a definition of teamwork: An 

analysis of critical team behaviors. Technical 

report 89-004, Orlando, FL: Naval Training 

Systems Center. 

Patrick, J. (1992) Training Research and Practice, 

New York: Academic Press. 

Robertson, M., Taylor, J. Stelley, J., and Wagner, R. 

(1994) Evaluating a maintenance crew resource 

management training program: Effects of 

attitudes, behaviors, and performance. 

ProceedinKS of the Human Factors and 

Ergonomics Society 38th Annual Meeting, 1242-

1246. 

Salas, E., Dickinson, T. L., Converse, S. A., and 

Tannenbaum, S. I. (1992) Toward an 

understanding of team performance and training. 

In R. Swezey and E. Salas (Eds.) Teams: Their 

training and performance, Norwood, NJ: Ablex, 

3-29. 

Scholtes, R. P. (1992) The team handbook, Joiner 

Associates Inc., Madison, WI. 

Shea, G. P. and Guzzo, R. A. (1987) Group 

effectiveness: What really matters?, Sloan 

Management Review, 3, 25-31. 
Chapter II 

Shepherd, W. (1991) Human Factors in Aviation 

Maintenance Phase I: Progress Report, 

DOT/FAA/AM-91/16. 

Shepherd, W. and Parker, J. (1990) Human factors 

issues in aircraft maintenance and inspection: 

Training issues. Final Report of the Third FAA 

Meeting on Human Factors Issues in Aircraft 

Maintenance and Inspection, 52-71. 

Schiflett, S.C., Eisner, E. J., Price, S. J., and 

Schemmer, F. M. (1982) The definition and 

measurement of team functions (Final Report) 

Bethesda, MD: Advanced Research Resources 

Organization. 

Steiner, I. D. (1972) Group processes and 

productivity. New York: Academic Press. 

Swezey, R. W., and Salas, E. (1992) Guidelines for 

use in team-training development. In R. W. 

Swezey and Salas (Eds.), Teams: Their training 

and performance, 219-245. Norwood, NJ: 

Ablex. 

203 

Team Training 

Taggart, W. (1990) Introducing CRM into 

maintenance training. In Proceedings of the 

Third Federal Aviation Administration Meeting 

on Human Fcu;tors in Aircraft Maintenance and 

Inspection: Training Issues. Atlantic City, New 

Jersey. 

Tannenbaum, S.l., Beard, R. L., and Salas, E. (1992) 

Team building and its influence on team 

effectiveness: An examination of conceptual an 

empirical developments. In K. Kelley (Ed.), 

Issues, Theory, and Research in 

lndustriaVOrganizational Psychalogy, (117-153) 

Elsevier Science Publishers. 

Tebnrg, J. R., Castore, C. H., and DeNinno, J. A. 

(1976) A longitudinal field investigation of the 

impact of group composition on group 

performance and cohesion. Paper presented at 

the annual meeting of the Midwestern 

Psychalogical Association. Chicago. 

• 
THE INPUT 

ORGANilAIIONAL FfC10RS 

- size ol cogao 11za1too I 

- type (e.g.. <*Ina. general <Niallon. 

I8PC* lodiiYI • I I ICI' lageli 8 It slnJch.Je 

-qucltycR:Ieo 

-OIP llzaltoo d clmale 

- <Xlml'l'lJ"ic:a ""'""' 

ENIIIlONMENIAI. FfC10RS 

- - olerMOII l8l1lal stress (hanga. 

~) 

- enWa lfTl8l ltd lJ"k:ddka iiV 

EQUIM:NT FfC10RS 

- CII..IIOiiKJIIoli 

-con~ 

• speck 4zuflc::u 

- equipment odabllty 

-safely 

TASK FfC10RS 

-type ol check (A. B. Cor Heavy 

Checlc) '' -task type (e.g.. Cllllonlcs. -plant, 

hyQaiJicS. ._ melal frame) 

-task COl~ 

-task """"""" (e.g.. lll<lC8CUd. 

dlagnosllc corrtllnalton) 

coo 

bv: 

- kr1c:rAiedge 

--

-mental 

-- models 

coo 

repi8Sel lied bv: 

-Interpersonal sldls 

-cor •IPOSlflon of 

1he1eam 

-nootle!ol 

merrtleis In 1he 

teem 

-Jeng1h ol fl'ne 1he 

teem merrtleis 

ha\lebeal 

Chapter 11 - Appendix 

Team Training 

THE PROCESS 

TEAM PROCESS 

OVERTIME 

20S 

THE OUTPUT 

PROCESS MEASI.IIES 

ln<M:IuaiProceos Measures 

coo be I1'IIIOSll8d by changes In 1he 

-.g lor each member: 

- kr1c:rAiedge 

-----mental models 

Teem- Measures 

coo be I1'IIIOSll8d by changes In: 

-1eam sl<lls(ablly 1o co-ordlnale. 

communicate. lnlelpeisoiKllsl<lls) 

-de\elopn IBIII ol ,_ 1eam processes 

Task Pel tom o IC8 Meaues 

can be met1SUied In 1erms ollhe 

folowlng lor 1he .-all task a'ld each 

Sib-~ 

-sPeed 

-CJCCUOC'{ 

- safely Q.e.. elendidcllcn:lgellellr11gKl safely ol self 

a'ld olher 1eam membefs while 

perfarrlng 1he loslcJ. 
Appendix - Team Training 

Communication Cohesiveness Decision-Making 

Team Measures 

Interpersonal 

•Training 

DNo Training 

Leadership 

Figure 11.2 Evaluation of Team Performance Measures by Instructor- Eng1ne Removal 

Coordination Communication Decision-Making 

Team Measures 

Interpersonal 

•Training 

DNo Training 

Leadership 

Chapter II 

Figure 11.3 Evaluat1on of Team Performance Measures by Instructor- Eng1ne Installation 

206 
Chapter II 

Decision-Making 

Team Measures 

Figure 11.4 Self Evaluation - Eng1ne Removal 

Decision-Making 

T earn Measures 

Figure 11.5 Self Evaluat1on- Eng1ne Installation 

207 

Inte•pemmal 

Appendix- Team Training 

.-Ttaining 

CNo Training 

Leadership 

•Training 

ElNo Training 
Appendix - Team Training Chapter II 

l•rrg- Pre IIITrg- Post DNo Trg- Pre •No Trg- Post 

• 

208 
Chapter II 

Table 11.1 Team Sk1lls 

Team Skills 

1. Coordination 

2. Communication 

3. Cohesiveness 

4. Decision-Making 

5. Interpersonal 

6. Leadership 

7. Attitude 

Appendix - Team Training 

Description 

This refers to the team's ability to organize available resources 

and activities so as to accomplish the goal within the temporal 

constraints. 

The process by which the team members clearly and 

accurately exchange information, using established procedures 

and language. It also encompasses the team members' ability 

to receive and provide constructive feedback on the 

performance of other team member(s) so as to help achieve 

the team goal. 

This refers to the process by which all members of the team 

develop compatible models of the system and work together as 

one unit. 

This refers to the process by which teams can use judgement, 

analytical technique, and consensus methods to arrive at 

decisions by pooling together information and resources .. 

This refers to team members' abilities to employ cooperative 

behavior to resolve interpersonal problems and optimize 

member interactions. 

This refers to the ability to assign, plan, organize, and motivate 

members to accomplish the goal. 

209 
Appendix - Team Training 

Table 11.2 Task Decompos1t1on by Phases 

ENGINE REMOVAL 

1. External Preparation 

a) Set up tail stand 

b) Disconnect electric power 

c) Remove top cowling 

d) Disconnect actuator 

e) Remove bottom cowling 

2. Engine Preparation 

a) Remove hoses and fittings 

b) Disconnect electrical leads 

c) Disconnect engine controls 

d) Drain oil 

e) Remove propeller 

3. Engine Extraction 

a) Mount sling and hoist on 

engine 

b) Remove bulkhead bolts 

c) Djsconnect lower engine 

mounts 

d) Disconnect top V -brace 

e) Extract Engine 

ENGINE INSTALLATION 

1. Engine Installation 

a) Install Engine 

b) Connect top V-brace 

c) Connect lower engine mounts 

d) Put bulhead bolts 

e) Unmount sling and hoist from 

engine 

2. Engine Preparation 

a) Install propeller 

b) Fill oil 

c) Connect engine controls 

d) Connect electrical leads 

e) Put back hoses and fittings 

3. External Preparation 

a) Put back bottom cowling 

b) Connect actuator 

c) Put back top cowling 

d) Connect electric power 

e) Remove tail stand 

• 

210 

Chapter II 
i 

I 

Chapter II Appendix - Team Training 

T c1Cie 11 3 DemographiCS Form 

DEMOGRAPIDCS FORM 

The following information will remain confidential and is for research purposes only. Each team member 

should fill in all questions carefully and completely. 

I. Have you attended a technical or vocational school other than this school? 

Yes __ _ No __ _ 

2. If you answered yes to question I, what type of technical training did you receive? 

3. Have you ever worked in a team environment prior to this class? 

Yes __ _ No __ _ Not Sure __ _ 

4. If you answered yes to question 3, where did you work as a team member? 

Schooi ___ Work __ Other __ _ 

4 (a). What kind of work were you involved in as a team member? 

5. Have you ever been fully employed prior to attending this school? 

Yes __ _ No __ _ 

6. What kind of work did you do? 

7. Have you ever had any team training before? 

Yes __ _ No __ _ 

8. What skills did you learn? 

9. Sex: Male ____ Female __ _ 

10. Age: 17-20 21-30 31-40 41-50 51-60 61+ 

211 
Appendix - Team Training 

Table 11.4 Tool Descnpt1on 

STUDENT TOOL LIST - 1 TOOT . ._ 

I. Tool box 

No larger than 20 inches high X 20 inches long. (No 

Rollaways) 

2. Chain and lock 

3. Open-End Wrenches 

1/4 X 5/16" 

3/8 X 7/16" 

1/2 X 9/16" 

9/16 X 5/8" 

5/8 X 3/4" 

11/16 X 13/16" 

3/4 X 7/8" 

15/16xl" 

4. Box-End Wrenches 

1/4 X 5/16" 

3/8 X 7/16" 

1/2 X 9/16" 

9/16 X 5/8" 

11/16 X 13/16" 

3/4 X 7/8" 

15/16xl" 

5. Socket Set 3/8" Drive 

3/8" Regular 12 pt 

7/16" 

1/2" Deep 6 pi 

9/16" 

5/8" 

11116" 

3/4" 

13/16" 

15/8" Plug 

Ratchet 

3" Ext. 

Case 

6" Ext. 

7/8" Deep Spark Plug Socket 

Universal Joint 

6. Socket Set 1/4" Drive 

5/32" Regular 6 pt 

3/16" 

7/32" 

1/4" 

9/32" 

5/16" 

11/32" 

3/8" 

7/16" 

1/2" 

1/4" Deep 6 pt 

5/16" 

318" 

7/16" 

1/2" 

Ratchet 

Spinner Handle 

Ext. I 112" 

Ext. 3" 

Universal Joint 

7. Screw Drivers 

8. Punch 

Set of tenRange of Slotted and Phillips 

with a stubby of each. 

Pin Punch 

1116-118" 

Center Punch 

3/8" 

Prick Punch 

3/8" • 

Line-Up Tools 

3/16 X 9 & 5/32 X 7" 

9. Allen Wrenches 

212 

Long 

5164" 

3/32" 

7/64" 

1/8" 

9/64" 

5/32" 

3/16" 

7/32" 

1/4" 

Chapter II 
Chapter II 

Table 11.4 (contmued .) 

10. Adjustable Wrenches- 10" 

II. Measuring Tape 12ft. 

12. Hammer, Ball Peen 8 oz. 

13. Hammer, Plastic Tip 

14. Flash light 2 Cell 

IS. Pliers, Common 8" 

16. Pliers, Diagonal 7" 

17. Pliers, Longnose 8" 

18. Pliers, Duckbill 

19. Pocket Knife 4" 

20. Sheet Metal Snips 

Left 

Right 

21. lOX Magnifying GlaSs 

22 File Set - 8" or larger 

I- Bastard 

1- Roond 

I - Half Round 

I • Triangular 

23. File Handles 

24. File Card 

2S. I -Extension Type Inspector Mirror 

26. I -Mechanical Finger, 10- 14 inch 

27. I - Retrieving Magnet, 10 - 14 inch 

28. I - Thickness Gage Set 

.002 - .03S or better 

Appendix - Team Training 

OPIIQNAI. TOOLS 

I. Cold Chisels 

114, 3/8, In. 3/4 .. 

2. Allen Wrenches 

Short 

.050" 

1/16" 

S/64" 

3132" 

7/64" 

9/64" 

3. Adjustable Wrenches- 6" 

4. Machinist Square 

S. Hacksaw 

6. Hacksaw blades 

7. Pliers Arc Joint 9" 

8. Socket Set tn• Drive 

Socket Regular 12 pt 

1116" 

213 

tn· 

9/16" 

S/8" 

3/4" 

ll/16" 

13/16" 

118" 

IS/16" 

I" 

Deep 12pt 

tn• ExL 6" 

9/16" Ext. 3" 

S/8" 

3/4" 

ll/16" 

13/16" 

718" 

IS/16" 

Ratchet 

Universal Joint 
Appendix - Team Training Chapter II 

Table 11.5 Team ExerCise on Lost on the Moon 

Lost On The Moon Exercise 

Your spaceship has just crashed-landed on the moon. You were scheduled to rendezvous with a 

mother ship 200 miles away on the lighted surface of the moon, but the rough landing has ruined your ship 

and destroyed all the equipment on board, except for the IS items listed below. 

Ypur crew's survival depends on reaching the mother ship, so you must choose the items based on 

their importance for survival. Place number one by the most Important for survival. Place two by the 

second most Important, and so on through number I S, the least Important. 

Your Your NASA's Team Team Error 

Item Ranking Error Ranking Ranking Score 

I to IS Score I to IS I to IS 

Box of matches 

Food concentrate . 

Fifty feet of nylon rope 

Parachute silk 

Solar-powered portable 

heating unit 

Two . 4S caliber pistols 

One case of dehydrated milk 

Two I 00 pounds tanks of oxygen 

Stellar map (of the moon's 

constellations) 

Self-Inflating life raft • 

Magnetic compass 

Five gallons of water 

Signal flares 

First-aid kit containing Injection needles 

Solar-powered FM receiver -

transmitter 

YOUR TOTAL ERROR SCORE 

214 
Chapter II Appendix- Team Training 

Table 11.5 (continued .. ) 

Lost On The Moon - Team Rules 

I. Avoid arguing for your own ranking. Present your position as lucidly and logically as possible, but 

listen to the other members' reactions and consider them carefully before you press your point. 

2. Do not assume that someone must win and someone must lose when discussion reaches a stalemate. 

Instead, look for the next-most-acceptable alternative for all parties. 

3. Do not change your mind simply to avoid conflict and to reach agreement and hannony. When 

agreement seems to come too quickly and easily, be suspicious. Explore the reasons and be sure everyone 

accepts the solution for basically similar or complementary reasons. 

4. Avoid conflict-reducing techniques such as majority vote, averages, coin-flips and bargaining. When 

a dissenting member finally agrees, don't feel that he or she must be rewarded by having his or her own way 

on some later poinL 

5. Differences of opinion are natural and expected. Seek them out and try to involve everyone In the 

decision process. Disagreements can help the group's decision because with a wide range of lnfonnation and 

opinions, there is a greater chance that the group will hit upon more adequate solutions. 

Lost On the Moon - Scorinc 

Team I Team2 Team3 Team4 

Total Error Points 

Error points are absolute difference between your rank and NASA's (d~rdlng plus or minus signs) 

0 - 25 execellent 

26-32 cood 

33- 45 average 

46-55 fair 

56-70 poor 

71 - 112 very poor (suggest possible faking or use of earth bound logic) 

215 
Appendix - Team Training Chapter II 

Table 11.5 (continued ... ) Lost On The Moon Answers 

Item NASA's Reasoning NASA Team Em>• T .. m Error Toam Error Team Error 

R.onk I Points 1 Points J Points 4 Points 

R.onk R.onk Rank R.onk 

Box of matches No Oxygen on moon to 15 

sustain flame: wonhless 

Food concentrate Efficient means of 4 

supplying energy 

requirements 

Fifty feet of nylon rope Useful in scaling cliffs, 6 

tying injured together 

ParachUie silk Prorection from sun's rays 8 

Solar-powered portable Not needed unless on dark 13 

heating units ""' 

Two .45 caliber pistols Possible means of self II 

propulsion 

One case of dehydrated Bulkier duplication of food 12 

Pet milk concentrate 

Two 100 pound tanks Most pressing survival I 

of oxygen """ 

Stellar map (of the Primary means of 3 

moon's constellations) navigation 

Self-inflating life raft C02 bottle in military raft 9 

may be used for propulsion 

Magnetic compass Magnetic field on moon is 14 

not polarized: worthless 

Five ga]lons of water Replacement for 2 

tmnendous liquid loss on 

lighted side 

Signa] flares Distress signal when 10 

mocher ship is sighted 

First-aid kit containing Need1es for vitamins, 7 

injection needles medicines, etc. WiD fit 

aperture in NASA space • .... Solar-powered PM For communication with 5 

receiver transmitter mother ship, bu1 requires 

line of sighl (shon range) 

. 

216 
Chapter II Appendix • Team Training 

Table 11.6 Team Tra1n1ng Program 

Session 1 - Basics of Teamwolt 

Goals 

Provide trainees an understanding of teams, need for teamwoJt, Introduction to team concepts, 

and an outline of future sessions 

Major Elements 

• Initial attitude survey 

• Why there is a need for teams 

• Establish the need for consistency and clarity in goals: team goals and indMdual goals 

• Goals of team building 

• Team wolt exercise 

• Overview of future sessions 

Session 2: Decision Making 

Goals 

Introduce trainees to scientific approach to decision-making 

Major Elements 

• expose trainees to different tools for decision-making 

• Identify the merits and demerits of the tools 

• use of decision-making tools within the aircraft/maintenance environment context 

(which tool? when to use? How to use?) 

• exercise invoMng different tools 

• decision-making by consensus 

Session 3: Group Dynamics 1: Communication and Interpersonal 

Goals 

To provide each trainee with an understanding of the essential elements of communication 

Identify steps to minimize Interpersonal problems 

Major elements 

• establish need for oral communication and written communication 

• principles of good communication (format, terminology, direction, when, how, how 

much/little) 

• examples of appropriate forms of communications (written and oral) within the aircraft 

maintenance environment 

• importance of providing team members with positive and negative feedback and how 

to receive feedback (When to give? How It works? How to receive? ... ) 

• exercise invoMng correct and incorrect communication within the aircraft maintenance 

environment 

217 
Appendix - Team Training Chapter II 

Table 11.6 (continued ... ) Team Tra1n1ng Program 

Session 4: Group Dynamics 2: Coordination and Cohesiveness 

Goals 

To train on the Importance of coordination and cohesiveness in achieving the team goal 

Major Elements 

• Methods to eliminate barriers and behavioral problems 

• Demonstrate the Importance of coordination as It relates to aircraft maintenance and 

Inspection 

• provide examples of good and bad coordination and demonstrate the effects on 

task performance 

• Identify every member's role and explain Interdependency 

• Help establish accurate expectations of the contributions of other team members to 

overall performance 

Session 5: Team Activity 

Goals 

To demonstrate how team skills can improve team performance for an aircraft 

Inspection/maintenance task 

Major Elements 

• construct examples of team activity 

• Illustrate Importance of different team skills In accomplishing the activity 

• role play 

• provide feedback to teams 

218 
Chapter II Appendix - Team Training 

Table 11.7 INSTRUCTORS INTERVIEW PERFORMANCE ~1EASURE'.1EmS 

The purpose of this questionnaire Is to evaluate the effectiveness of team training on team 

perfonnance. The facilitator Is in a position to observe any improvements or lack of Improvements In team 

perfonnance, so please take time to consider each statemenL All responses will be kept confidential. 

Rate each statement on a scale of 1 - 7 

Number of times, 

Lowest, Poor, 

Never, etc. 

I 2 3 

Neutral 

4 

I. The team members worked well together. 

2. The team resolved conflicts effectively. 

3. All members of the team participated in the decisionmaking process 

5 

4. The team members discussed new ways to tackle the task. 

5. The team was effective in establishing ground rules. 

6. One person dominated the team. 

7. There was at least one person who was disruptive. 

8. There was at least one person who did not participate in 

team discussions. 

9. One member took charge of assigning the tasks and 

coordinating activities of other team members. 

10. Team members provided each other with performance 

fedback 

II. The team members worked cohesively. 

12. Team members responded well to team training. 

13. The team members follow the agenda 

(accomplished the objectives). 

14. There was a noticeable improvement due to team 

training. 

219 

6 

Team I 

Highest, Best, Always, 

Very, etc. 

7 

Team2 Team3 

• 

Team4 
Appendix - Team Training Chapter II 

Table 11.8 POST SESSION INTERVIEW 

Please rate the following statements on a scale of I - 7 by circling the response that best fits your opinion 

concerning the statement. All response will be kept confidential: 

Definitely Not Definitely 

I. 1lte team followed the agenda for 

the session: 2 3 4 5 6 7 

2. You were satisfied with the level of 

participation by team members. 2 3 4 5 6 7 

3. Everyone contributed and was 

involved in team decisions. 2 3 4 5 6 7 

4. You had a good attitude about your 

work and the task. 2 3 4 5 6 7 

5. Team members allowed personality 

conflicts to interfere with work.. 2 3 4 5 6 7 

6. You were satisfied with the level of the 

teams' achievement towards the 

established goal. 2 3 4 s 6 7 

7. Team members were able to settle 

conflicts effectively among themselves 2 3 4 s 6 7 

8. You feel the teams' perfonnance was 

very good. 2 3 4 s 6 7 

9. You feel the final result of the task 

was very good. 2 .3 4 s 6 7 

10. Your opinion was considered. 2 3 4 s 6 7 

It. One member took charge of assigning 

the tasks and coordinating the activities 

of other team members. 2 3 4 s 6 7 

• 

12. Team memben were aware of each others 

responsibilities. 2 3 4 s 6 7 

13. You were satisfied with the material 

used for team ttaining. 2 3 4 s 6 7 

14. You were satisfied with the material 

used for technical training. 2 3 4 s 6 7 

J S. If provided with another opportunity, 

you would want to participate in a 

team activity. 2 3 4 s 6 7 

16. If provided with another opportunity, 

you would participate in a team activity 

with the same group. 2 3 4 5 6 7 

220 
Chapter II Appendix - Team Training 

Table 11 9 PRE-TRAINING INTERVIEW 

Please circle the response that best reflects your opinion of each statement. All responses will be kept confidential. 

Strongly Strongly 

Disagree Nentral Agree 

I. I believe teamwork is the 

best way to accomplish 

work tasks in all situations. 2 3 4 5 6 7 

2. In team environments, it is 

important to follow an agenda. 2 3 4 5 6 7 

3. All team members should 

contribute to team decisions. 2 3 4 5 6 7 

4. If one team member doesn't 

understand, other team members 

should help him or her. I 2 3 4 5 6 7 

5. Team leaders should keep the team 

on track to accomplish goals. 2 3 4 5 6 7 

6. Team decisions are superior to 

individual decisions. 2 3 4 5 6 7 

7. All tasks are not suited for team 

environments. 2 3 4 5 6 7 

8. I am comfortable participating 

in team decisions. 2 3 4 5 6 7 

9. The success of the team is important 

to each individual. 2 3 4 5 6 7 

10. Training improves team 

performance. I 2 3 4 5 6 7 

221 
Appendix - Team Training Chapter II 

Table 11.10 POST-TRAINING INTERVIEW 

Please circle the response that best reflects your opinion of each statement. All responses will be kept confidential. 

Strongly Strongly 

Disagree Neutral Agree 

I. I believe teamwork is the 

best way to accomplish 

work tasks in all situations. 2 3 4 5 6 7 

2. In team environments, it is 

important to follow an agenda. I 2 3 4 5 6 7 

3. All team members should 

contribute to team decisions. I 2 3 4 5 6 7 

4. If one team member doesn't 

understand, other team members 

should help him or her. 2 3 4 5 6 7 

5. Team leaders should keep the team 

on track to accomplish goals. 2 3 4 5 6 7 

6. Team decisions are superior to 

individual decisions. I 2 3 4 5 6 7 

7. All tasks are not suited for team 

environments. 2 3 4 5 6 7 

8. I am comfortable participating 

in team decisions. I 2 3 4 5 6 7 

9. The success of the team is important 

to each individual. 2 3 4 5 6 7 

I 0. Training improves team 

performance. 2 3 4 5 6 7 

222 
Chapter II Appendix - Team Training 

Table 11.11 Data Collectton Instrument on Team Performance 

1. Total time to complete the entire task. 

-------------

I (a). Total time to complete the External Preparation Phase. 

-------------

I (b). Total time to complete the Engine Preparation Phase. 

-------------

I (c). Total time to complete the Engine Extraction Phase. 

2. Total number of mistakes made by the team while 

completing the entire task. 

-------------

2 (a). Total number of mistakes made by the team during 

External Preparation Phase. 

-------------

2 (b). Total number of mistakes made by the team during 

Engine Preparation Phase. 

-----------

2 (c). Total number of mistakes made by the team during 

Engine Extraction Phase. 

3. Number of times the instructor had to point out the 

mistakes being made .and correct them during the entire task. 

----------

3 (a). Number of times the instructor had to point out the 

mistakes being made and correct them during the External 

Preparation Phase. 

-------------

3 (b). Number of times the instructor had to point out the 

mistakes being made and correct them during the Engine 

Preparation Phase. 

-------------

3 (c). Number of times the instructor had to point out the . mistakes being made and correct them during the Engine 

Extraction Phase. 

4. Number of times team did not follow correct procedures 

during the entire task. 

-------------

4 (a). Number of times team did not follow correct procedures 

during the External Preparation Phase. 

-------------

4 (b). Number of times team did not follow correct procedures 

during the Engine Preparation Phase. 

-------------

4 (c). Number of times team did not follow correct procedures 

during the Engine Extraction Phase. 

223 
Appendix - Team Training Chapter II 

Table 11.12 Summary of Task Performance 

Engine Removal (averaaed over 4 teams) 

Task Performance Measures Training Group Control Group 

I. Total time taken to complete the task of engine 6 hrs 10 mlns. 7hrs 38 mlns. 

removal (hrs.lmlns.) 

2. Number of mistakes made by the team during engine 3 9 

removal 

3. Number of times the Instructor had to point out the 

mistakes being made and correct them during the task of 

3 6 

engine removal 

4. Number of times the. team did not follow correct 

procedures during the task of engine removal 

I 5 

5. Number of times safety of fellow team members was 

endanaered during the task of engine removal 

0 0 

6. Number of times safety procedures were not 3 I 

followed during the task of engine removal 

Engine Installation over 4 teams) 

Task Performance Measures Training Group Control Group 

1. Total time taken to complete the task of engine 

Installation (hrs.lmlns.) 

13 hrs 32 ntlns. 14 hrs 15 mlns 

2. Total number of mistakes made by the team durtng I 5 

engine Installation 

3. Number of dmes the Instructor had to point out the 2 6 

mistakes being made and correct them during the task of 

engine Installation . 4. Number of times the team did not follow correct 

procedures during the task of engine Installation 

2 4 

5. Number of times safety of fellow team members was 

endanaered during the task of engine Installation 

0 0 

6. Number of times safety procedures were not I 2 

followed during the task of engine Installation 

224 
Chapter II 

Table 11.13 AMP School Cumculum 

Fall Semester 

General Regulations 

Aircraft Drawings 

Year 1 

Ground Handling and Servicing 

Materials and Corrosion Control 

Assembly and Rigging 

Algebra, Geometty, and Trigonometty I 

Spring Semester 

Basic Aircraft Electrlcty 

Wood, Dope, Fabric, and Finishes 

Sheet Metal Layout and Repair 

Reciprocating Engine Overhaul 

Conceptual Physics I 

Summer 

Aircraft Environmental Systems 

Hydraulics and Pneumatic Systems 

Aircraft Electric Systems 

Aircraft Fuel Systems 

PersonaVInterpersonal Psychology 

Appendix - Team Training 

Year 2 

Fall Semester 

Bonded Structures & Welding 

Utilty & Warning Systems 

Landing Gear Systems 

Airframe Inspection 

Propellers and Components 

Professional Communications 

Spring Semester 

Lubricating Systems 

Ignition Systems 

Turbine Engine Overhaul 

Engine Inspection 

Engine Electrical, Instrument, and 

Fire Protection Elective 

Summer 

Powerplant Fuel Systems 

Induction Cooling and Exhuast 

Technology and Culture Elective 

• 

225 
Appendix - Team Training 

Table 11.14 Team ProJects 

Year 1 

Course: Ground Handling and Services 

Team project title: Aircraft towing 

Number of team members: 4 

Chapter II 

Description: Given an aircraft and aircraft towing equipment, the team will tow aircraft from the hangar to a 

preselected location within the areas marked for the landing gear. All the movement of aircraft will be 

conducted in a highly precautious and coordinated manner. Team members will have to follow standard 

operating procedures. 

Team project title: Aircraft operation 

Number of members In a team: 3 

Description: Given manufacturers' operating Instructions, team will locate, select, connect, and operate 

ground support equipment. Team will start and operate engine through normal operating range and perform 

shut down procedures. 

Course: Assembly and Rigging 

Team project title: Installing flight control 

Number of members In a team: 4 

Description: Team members will identify appropriate service manuals, tools, equipment, and forms. Team 

members will assign roles to remove, inspect, repair, and reinstall one flight control and make required 

maintenance record entries. All work performed needs to meet manufacturers' specifications. Team members 

will play the role of Inspector, buy-back Inspector, and maintenance personnel. 

Team project: Installing vertical stabilizer 

Number of members In team: 4 

Description: Team members will identify appropriate service manuals, tools, equipment, and forms. Team 

members will assign roles to remove, Inspect, repair, and reinstall vertical stabilizer and make required 

maintenance record entries. All work performed needs to meet manufacturers' specifications. T earn members 

will play the role of inspector, buy-back Inspector, and maintenance personnel. 

Team project: Aircraft control rigging (different sub-systems) 

Number of members in team: 3 

Description: Given an aircraft with cable operated flight control system, service manuals, tools, and 

equipment. The team will have to coordinate work and assign roles to Inspect the system for proper rigging, 

record the discrepancy, and make repairs, rig the flight controls, and record the work The members will play 

the role of a inspection and maintenance crew on a rigging check. 

226 
Chapter II 

Table 11.14 (continued 1 Team Proiects 

Year 2 

Coo~: Utility and Warning Systems 

Team project tide: Position Indicating and Warning Systems 

Nu~r of membe~ in a team: 4 

Appendix- Team Training 

Description: Given an aircraft with retractable landing gear and position Indicating and warning systems, 

ground support equipment, and the manufacture~· maintenance and service Instructions, the team will have 

~~with assigned roles of an inspector, buy-back Inspector, and maintenance pe~onnel. The team will 

first perform an operational check of the landing gear, Inspect components of the position indicating and 

warning system (Ins~), troubleshoot and repair malfunctions (maintenance crew), and ensure that the 

work meets standards (buy-back Inspector). 

Coo~: Landing Gear Systems 

Team project title: Aircraft Jacking 

N~r of membe~ In a team: 4 

Description: Given an aircraft with operational retractable landing gear, manufacture~· service manuals; 

other information, and ground support equipment, the team will have to assign roles and coordinate work to 

accomplish the following: jack the aircraft, check, inspect, repair, and service the landing gear so that"work Is 

accomplished within the allowed time frame. The team will have to ensure that the operation of the systems 

and the manufacture~· adjustment procedures are followed precisely and that the system meets "return-toservice" standards. 

Coo~: Airframe Inspection 

Team Project: Airframe Inspection and Maintenance 

N~r of membe~ In a team: 4 

Description: Given an operational aircraft ground support equipment and manufacture~' service manuals, 

the team will have me~~ with assigned roles of an inspector, buy-back Inspector, and maintenance 

personnel. The Inspector (lim team member) will perform an annual inspection of the aircraft, record 

conditions at the time of inspection, and make the appropriate aircraft record entries to communicate 

Information to other me~~ of the team (maintenance crew consisting of 2 team membe~). Team 

~~ responsible for maintenance activities will conduct maintenance and have it Inspected by another 

Inspector (fourth member of the team) to ensure that the maintenance work meets standards. 

Coo~: Turbine Engine Overhaul 

Team Project: Engine Overhaul 

Nu~rofmembe~:4 

Description: Given a turbojet or turboprop engine, manufacture~' maintenance manuals, special tools, and 

shop equipment, working as a team, the team will disassemble , clean, Inspect, identify repal~, and 

reassemble both cold and hot sections of the engine within a specified time frame. All activities and practices 

will be performed in accordance with manufacture~· maintenance instructions. 

Team Project: Engine Removal and Installation 

Nu~r of me~~= 4 

227 
Appendix - Team Training Chapter II 

Table 11.14 (continued ) Team Pro1ects 

Description: Given an aircraft with an operational turbojet engine, manufacturers' maintenance manuals, and 

engine removal and installation equipment, working as a team, the team will perform the engine removal and 

reinstallation procedures to meet manufacturers' standards and within the allocated time frame. 

Course: Reciprocating Engine Overhaul 

Team Project: Engine Overhaul 

Number of members: 4 

Description: Given a reciprocating engine, manufacturers' maintenance manuals, and special tools and shop 

equipment, working as a team, the team will disassemble , clean, inspect, identify repairs, and reassemble the 

engine within a specified time frame. All activities and practices will be performed in accordance with 

manufacturers' maintenance Instructions. 

Team Project: Engine Removal and Installation 

Number of members: 4 

Description: Given an aircraft with an operational reciprocating engine, manufacturers' maintenance manuals, 

and engine removal and installation equipment, working as a team, the team will perform the engine removal 

and reinstallation procedures to meet manufacturers' standards In the allocated time frame . 

• 

228 
CHAPTER 12 

TRAINING AND CERTIFICATION 

IN THE AIRCRAFT MAINTENANCE INDUSTRY 

TECHNICIAN RESOURCES FOR THE TwENTY-FIRST CENTURY 

Ray Goldsby 

HKS&A, Inc. 

12.0 I'\ I ROI>l l'IIO'\ 

The Federal Aviation Administration (FAA) is 

committed to exploring ways of restructuring the 

regulatory process as it pertains to training, 

qualification and certification of advanced skills 

(specialties) in the aviation maintenance industry. 

They recognize a need for a flexible, forward looking 

and more efficient system, geared to the rapid 

technological and industry changes taking place as 

we approach the 21st century. This project will 

evaluate the issues, analyze pertinent infonnation and 

present a plan for development of such a system. 

Included will be an evaluation of both US and 

international aviation maintenance technical training 

and qualification standard!\. and certification systems 

in other industries that require skill level standards. 

Pertinent information from other studies, such as 

Pilot and Aviation Maintenance Technician Blue 

Ribbon Panel, Aviation Maintenance Technician Job 

Task Analysis, and Human Resources in the 

Canadian Aircraft Maintenance Industry, will also be 

included. 

The system will be based on evaluation of other 

industries where individuals are certified to 

performance standards that are approved and kept 

current by recognized industry professional 

organizations. Candidates for certification are 

required to complete specific training and 

competency testing approved by the appropriate 

regulatory agency (the FAA, for purposes of this 

project), based upon the specific industry group's 

standards. Examples of this are found in the 

certification of medical technologists, electronic 

technicians, structural welders, and various other 

critical safety intensive professions. 

The initial focus of this study is to research 

alternative ways to develop industry input for 

training and certification standards for advanced 

229 

aircraft maintenance skills. This project will provide 

a basis for an implementation plan, development of 

the standards approval process and the selection of 

technical agencies that can validate, issue, and 

maintain these standards. The possibility of forming a 

national aviation industry forum that would provide 

information for industry standards development 

groups and advise the FAA will also he analyzed. 

This effort is an extension of the regulatory actions 

work being done by the Federal Aviation 

Administration on revising rules that specify the 

training and certification of aircraft maintenance 

personnel (Federal Air Regulations Parts 147 and 

65). Included in the final report will he an evaluation 

of the US system of certification for Aviation 

Maintenance Technicians (AMT) and Aviation 

Repair Specialists (ARS). 

The project will he accomplished in two phases. This 

first phase, that began in July of 1994 and completes 

at the end of March 1995, will include investigation, 

infonnation and data gathering. The second phase, 

April through December of 1995, will focus on 

development of proposals and the final report. 

• 

12.1 B \Ch.CIWl '\I> 

There is increasing evidence that validates FAA 

and Industry concern that the current background 

information and industry input into the FAA system 

for training, qualification and certification of aviation 

maintenance personnel may he insufficient. There is 

also concern that the FAA certification process is not 

geared for rapid revision and technical updates. 

Regulator actions have not kept pace with changing 

aviation technology and the industry's maintenance 

skill requirements. These concerns are focused on 

persons certified as Airframe and Powerplant 

Mechanics (A&P), and Repairmen, as prescribed in 

FAR Part 65. There must be sufficient input to ensure 
'I :I 

Training and Certification 

that aviation maintenance personnel will continue to 

meet the current and future needs of continuing air 

worthiness. It is necessary to explore means that will 

enhance the role of the industry's technical 

leadership working together with the FAA to keep 

the system current. 

In November 1989, a joint industry I FAA part 65 

review gr~up was formed to evaluate and review 

certification requirements for mechanics and 

repairmen. The review group's objective was to 

develop and present a unified position on 

recommended changes to part 65. The group was 

composed of representatives from several aviation 

associations and was coordinated by the Professional 

Aviation Maintenance Association (PAMA). FAA 

interests were represented by the Aircraft 

Maintenance Division (AFS-300) of the FAA. 

After conducting a series of panel discussions 

throughout the United States, the Indusl!y I fAA 

Part 65 Review Group Workim: Paper was published 

in January of 1991. This paper presented the issues 

on which there was general agreement and those 

issues that the group believed would require further 

discussion. 

During 1991, the FAA also conducted both a 

historical review of part 65, subparts D and E, and a 

survey of FAA regional offices on the certification of 

mechanics, holders of inspection authorizations, and 

repairmen. Results of both the historical review and 

the regional office survey showed clear support for a 

full review and update of part 65. 

Another major reason for review and revision of the 

Aircraft Mechanic and Repairman regulation is based 

upon the level of professionalism in these career 

fields. The Pilot and Avjatjon Maintenance 

Technician Blue Ribbon Panel Re.port pointed out 

that the US I&partment of Labor Dictionary of 

Occupatjonal Titles lists aircraft mechanics and 

repairers as semi-skilled. The panel recommended 

that this be reviewed. The FAA believes it is 

necessary to increase the level of professionalism 

within these occupations and have Aviation 

Maintenance Technicians and Aviation Repair 

Specialists recognized as highly skilled. 

230 

Chapter 12 

12.2 .\\ L\T10:\ 1:"/Dl STRY 

DY:\A\IICS \:\D RE<;LrL.\TORY 

CH.\\GE 

The Pilot & Aviation Maintenance Technician 

Blue Ribbon Panel Report (Aug. 1993) explains: 

The majority of new-hire AMTs come from FAAcertificated AMT schools, where they have 15 to 18 

months of structured training in a variety of subjects. 

Although the FAA recently revised the curriculum 

requirements for these schools, the new curriculum 

remains broad-based to fit a variety of technical 

disciplines, and it may not give AMTs the skills and 

competencies needed to maintain the increasingly 

sophisticated transport category aircraft. Therefore, 

new-hire AMTs working on newer aircraft willlu:J.ve 

to master skills that many AMT schools do not offer, 

if they are to become productive members of air 

transportation teams. 

Thus, the industry will face a problem with AMTs 

similar to the problem with pilots: A decreasing 

supply of qualified AMTs, combined with increasing 

skill and experience requirements, will yield a deficit 

not in the number of minimally qualified individuals 

but in the number with the necessary skills and 

experience. This gap will have to be bridged by 

additional focused and specialized training. Europe 

and Asia are effectively addressing the future skills 

shortages and becoming stronger competitors, 

causing dramatic increases in the amount of U. S. 

work done in foreign repair stations. 

The aviation industry will continue moderate growth 

well into the next century. At the same time the 

forces of compt'lition in the de-regulated air 

transportation environment mandate lowering prices 

to the consumer, with a resulting focus on lowering 

operating costs and the need to optimize maintenance 

processes and practices. This competition has also 

spurred the development of improved aircraft 

technology and operational efficiency. Today's 

aircraft are significantly more sophisticated, from 

both a materials and systems standpoint, than those 

built and certified when the current maintenance 

regulations were developed. 

The industry finds itself in a challenging situation. 

Significant changes are being made by air carriers 

with respect to internal maintenance programs and 

the contracting of second and third party agencies to 

maintain and modify their fleets. In the past most 

carriers completed a majority of maintenance work 
Chapter 12 

in-house, but it is now often more efficient and cost 

effective for them to have ml\ior work and 

modifications accomplished by others. The numbers 

of aireraft that are owned by leasing companies, 

maintained by various agencies worldwide and 

moved from operator to operator, have dramatically 

increased. Along with the international aspects of 

movement of aircraft within different fleets and 

maintenance programs, is the dramatic increase in the 

number of foreign certified repair stations and 

maintenance work begin done "off shore." These 

factors, mixed with numerous technology changes, 

have increased the complexity of aircraft 

maintenance. All of this has created both FAA and 

industry concern. 

The present maintenance regulatory system is 

cumbersome; it was not designed for rapid change. 

Changes due to new technology and the dynamics of 

the global business environment make it difficult for 

the rules that regulate training and qualification to 

keep pace. Finding methods that will allow for a 

more responsive regulatory system under the rules, 

while at the same time focusing on international 

harmonization, is essential. 

12.3 Ill!-. \\I\ 110\ IU LE \1.\KI:\(; 

\ll\ ISO In C0\1\IIT I"EE ( \R \(") 

PRO( I·-.,-, 

,....,.,eARAC was established (56 CFR 2190, 

J. January 22, 1991) to assist the FAA in the 

rulemaking process by providing input from outside 

the Federal Government on major regulatory issues 

affecting aviation safety. This process is designed to 

provide opportunity for those groups in the industry 

who are significantly affected by rulemaking to 

become involved in the process. Since affected 

parties are involved in the process the rules produced 

should be more complete, require less direct effort on 

the part of the FAA, have few elements of contention 

from the public when published and move rapidly 

from initial review to final effectivity. 

11te ARAC includes representatives of air carriers, 

manufacturers, general aviation, organized labor 

groups, universities, associations, airline passenger 

groups, and the general public. Formation of the 

ARAC bas given the FAA additional opportunities to 

solicit information directly from all elements of the 

industry. There are several working groups under 

ARAC that meet to exchange ideas about proposed 

231 

Training and Certification 

rules and existing rules that should be either revised 

or eliminated. 

Formed initially in November 1989, as the Joint 

Industry I FAA Part 65 Review Group, the Aviation 

Rule Making Advisory Committee Working Group 

for FAR Part 65 (ARAC- 65) has been meeting 

officially since May 24, 1991. The working group is 

made up of representatives from aviation industry 

professional organizations, aviation training 

providers, air transport labor unions, industry 

representatives, and the general public. One of the 

major objectives of ARAC is to shorten the time it 

takes to revise regulations by involving all interested 

parties in the process. This working group is 

responsible for regulatory review and recommending 

changes to FAR Part 65, Certification of Airmen 

Other Than Flight Crew Members, specifically the 

portion regulating mechanics, mechanics holding 

inspection authorizations and repairmen. Their 

efforts have yielded significant changes and upgrades 

to.FAR Part 65 which are scheduled to be released as 

a Notice of Proposed Rule Making (NPRM) in the 

winter of 1995/1996. Substantive recommended 

changes to Part 65 are outlined in Appendix 12-A. 

If the process remains on schedule, the new rule 

(consolidated as FAR Part 66) may become effective 

in mid-1998. This means that the process to review 

FAR part 65 will have been in the works for nine 

years. The process of evaluating and recomrneoding 

changes to Federal Air Regulations remains long and 

cumbersome. 

12.-1.\R \C- h5 \CTIO\ IU.<; \IWI\(; 

\D\ .\\CED OR SPH'I \L 

CEin IFIC \"110\ 

The ARAC- 65 working group has discussed and 

evaluated a significant number of issues 

regarding advanced certification. The group's 

consensus is that a new process needs to be 

developed and that the research project described 

herein is a necessary step toward reaching that 

objective. Suggested changes in the current Airframe 

and Powerplant, and Repairman Certificates reflect 

the complexity of today' s technology, and represent a 

wide range of input toward the develupment of an 

advanced certification process. Since the members 

represent a large cross section of the industry, their 

views may be considered as a reasonable 

representation of the industry's thinking on this issue. 
'lr :1 

Training and Certification 

The Airframe and Powerplant Certificate (A&P) is 

based on a broadly focused 1900 hour minimum 

curriculum specified in FAR Part 147. The Airframe 

or Powerplant privileges of the certificated may be 

issued separately under the current rule. The 

certification under the new Part 66 rule will be titled 

Aviation Maintenance Technician (AMT), will 

include a common set of privileges and be issued 

only as a ~inge certificate. Advanced certification 

will be provided with the addition of the Aviation 

Maintenance Technician- Transport (AMT-T) 

privilege. This certification will require an additional 

curriculum, approximately 600 hours above the 1900 

hours required in the current rule, that is specific to 

the current technology of Part 25 (air transport fixed 

wing) and 29 (air transport rotor craft) certified 

aircraft, along with additional competency testing. 

Persons may select the level of certification for which 

they wish to qualify. An AMT-T, however, will be 

required to return transport category aircraft to 

service once the rule becomes final. 

Through the creation of the AMT-T operators of 

aircraft certificated under FAR Parts 25 and 29 

(commercial airplanes and helicopters) will be 

assured that the holder of an AMT-T certificate 

possesses the knowledge and skill to approve these 

aircraft for return to service (or "sign off' of a 

maintenance release). This will allow operators to 

employ aviation maintenance personnel who will 

more quickly meet the requirements of their 

operating environment without having to attend 

extensive operator-sponsored training programs 

before performing maintenance on transport aircraft. 

Operators would be able to focus their training on 

aircraft type, aircraft differences, modifications, and 

technology upgrade of transport aircraft. Aviation 

maintenance technician training schools (certified 

under FAR Part 147) would be able to focus on the 

fundamental concepts and basic skills of aviation 

maintenance. They would also have the option of 

providing the additional knowledge and skill required 

for AMT-T certification. 

The Repairman Certificate is currently issued to an 

individual for a specific maintenance task(s), 

appliance or component repair I overhaul, for FAR 

Part 121 or 135 Operators under subpart J and L, 

Fixed Base Operators (FBO), or FAR part 145 

Certified Repair Stations. They are also issued by the 

FAA to those individuals constructing amateur-built 

aircraft for their own non-~ommercial use. The 

Repairman Certificate process has been significantly 

revised under the new FAR Part 66. 

232 

Chapter 12 

The new certification will grant specific repair and 

maintenance privileges to Aviation Repair Specialists 

(ARS). The ARS will be issued in three categories, 

defined as follows: 

I. ARS-I - May be issued by the FAA upon 

completion of an industry developed standardsbased training curriculum and appropriate 

competency testing and I or validation to an 

individual. The individual who has earned such 

certification may only exercise these privileges 

while employed at a Certified Repair Station, 

Part 121 or 135 Operator. This provides limited 

portability for this level of certification. The skill 

areas where ARS-I certification will be granted 

are to be determined based on the outcome of 

this project, and the Job Task Analysis project 

being completed by Northwestern University's 

Transportation Research Center. Also included 

may be areas with current standards such as nondestructive inspection (NDI). 

2. ARS-II - Issued as a replacement for today' s 

Repairman Certificate and will be issued under 

similar regulations. 

3. ARS-III- Issued by the FAA to amateur 

builders, producing "home built" aircraft for 

their own non-commercial use, as in the past. 

12.5 Sl (;(;ESTED SKILL .\REAS FOR 

All\'.\:\CED OR SPECIAL 

CERTIFICATIO"i (ARS-1) 

The working sessions of ARAC-65 generated 

presentatioN~ from various industry groups that 

stimulated discussion regarding advanced 

certification and appropriate skill areas. No firm 

decisions were made specifying what functional areas 

may be finally selected for advanced certification. It 

was concluded that there may eventually be new 

ARS-I categories beyond those listed below. The 

group agrees with the FAA that training, 

qualification, and certification will be based on 

nationally and internationally recognized standards 

developed by the aviation maintenance industry. The 

following skill areas have been selected as those that 

will be considered for advanced certification 

standards and ARS-I certification: 

• Aircraft Electronics (Avionics) 

• Composite Structural Repair 
Chapter 12 

• Non-destructive Inspection 

• Metal Structures Repair 

• Balloon and Glider Repair 

As rule making evaluation and change continues, 

there may he other skill areas identified and added to 

the list. There has been a good deal of work 

completed toward development of training, 

qualification and certification standards in the 

following areas: 

12.5.1 Aircraft Electronics 

In its broadest definition, aviation electronics, also 

known as avionics, encompasses all aircraft electrical 

I electronic systems and their components. The term 

"avionics" now goes beyond a more basic definition 

that once included only communication, navigation 

and auto-flight systems. 

One of the major changes in today's aircraft is the 

extensive use of digital electronic data processors, 

computers, electronic controls, and fly-by-wire 

technology. Aircraft have become fully integrated 

from a systems standpoint. While additional 

emphasis has been placed on avionics in the 

proposed Aviation Maintenance TechnicalTransport (AMT-T) rating in FAR Part 66, there is a 

large group within the aviation industry that strongly 

supports an ARS - I level avionics technician 

certification. Maintenance and alteration of these 

systems requires a highly specialized set of skills and 

knowledge that go beyond AMT and AMT-T 

requirements. 

The Association for Avionics Education (AAE), with 

the support of the Aircraft Electronics Association 

(AEA), is in the process of developing a training and 

qualification standard for Aviation Electronics 

Technicians. Their working documents have been 

presented to ARAC-65 on two occasions for review 

and comment. The ARAC-65 group has concluded 

that there will not he a separate avionics rating as part 

of AMT or AMT-T certification. They have 

encouraged AAE to continue with their standards 

development process, addressing aircraft electronics 

as an ARS-I certification. 

233 

Training and Certification 

12.5.2 Composite Structural Repair 

Composites are non metallic structures that include 

materials such as fiberglass, carbon fiber, kevlar", 

and graphite filament. They are usually chemically 

compounded or laminated with resins and bonded to 

metal, or other composite, support structures with 

adhesives to make light-weight, non-corroding, high-

. strength aircraft structural components. They are 

often formed and cured under heat and vacuum. 

Special equipment and working environments are 

often required to construct or repair composite 

structures. Special skills are required as improper 

handling or repair techniques can cause extensive 

damage and the materials themselves can create both 

worker health and environmental hazards. 

Moiit indicators point toward the increased use of 

composite materials in aircraft construction, 

particularly transport aircraft. Some aircraft currently 

in production are "all composite." It has become a 

very complex and highly specialized segment of 

aviation maintenance. The knowledge and skills 

necessary for composite maintenance require an 

expertise beyond the AMT and AMT-T certification 

requirements. 

The Commercial Aircraft Composite Repair 

Committee (CACRC), sponsored by the Society of 

Automotive Engineering (SAE), Is in the process of 

formulating a standard for this skill area. The format 

from Air Transport Association (ATA) Specification 

105 (Non Destructive Inspection) is being used as a 

modeL The CACRC group has gained international 

stature, based in representation from the European 

aviation maintenance community. They have been 

meeting for ove: two years developing their 

standards and have made a good deal of progress 

with the document. The group is close to the release 

of a draft that will include guidelines for composites 

materials handling, preventative maintenance, 

inspection, repair, alteration I fabrication, and 

protective coatings. 

12.5.3 Non-destructive Inspection 

Non-destructive inspection (NDI) has become a very 

highly specialized skill area that requires the use of 

sophisticated tooling and diagnostic equipment for 

the evaluation of defects and flaws. Technology 

ranges from magnetic particle and dye penetrant 

methods through x-ray, ultrasonic, eddy current and 

some currently emerging technologies. The 

technician is responsible for the setup and operation 
'I' I 

Training and Certification 

of these systems, plus the reading and interpretation 

of their output. Competency in non-destructive 

testing requires a high degree of both knowledge and 

skill. Proficiency also requires a good deal of handson practice and recurrent training. 

There have been recent improvements in nondestructive inspection technology. Sandia Laboratory 

in Albuq~rque, New Mexico has a dedicated facility 

and a staff, complete with air transport category 

aircraft, for the development and application of nondestructive testing technology. There are also human 

factors studies underway that are focused on 

improving visual inspection tools and processes. 

These studies are expected to produce human 

engineering results that will enhance techniques, 

therefore benefiting the technician's ability to 

conduct visual inspections. 

The Air Transport Association Non Destructive 

Inspection Sub Committee has developed 

Specification 105, Guidelines for Non Destructive 

Inspection. The document includes training curricula 

for the various NDI processes and associated 

inspection techniques. Also included are qualification 

standards for NDI personnel. ATA Specification 105 

represents a quality body of work that was developed 

with input from all elements of the aviation 

manufacturing and maintenance industry. 

The American Society for Non Destructive Testing 

standards have been in place for a number of years. 

They are kept current with state of the art processes 

and emerging technology. These standards specify 

training, qualification and certification of NDI 

specialists in each of the NDI processes, from the 

basics through the most complex radiography. Their 

standards are recognized by several industries other 

than air transport and they are considered as the 

model. 

While there are two other standards that are 

recognized in the non destructive inspection 

discipline, the aviation industry recognizes ATA 

Specification 105 and ASNT as the baseline. One, or 

both, of these could become standards that are 

accepted by the FAA for ARS-I certification. 

12.5.4 Metal Structures Repair 

Aircraft structure maintenance, modification and 

repair is an area of increasing focus and concern. 

Several factors are causing changes in the nature of 

work content and specialization of personnel within 

234 

Chapter 12 

this element of the maintenance industry. 

Specifically, the need to reduce operating costs is 

motivating the air transport community to conduct 

business differently: 

• Increasing amounts of modification and repair 

work (up to and including D check level) is 

being accomplished by second and third party 

maintenance providers. 

The number of aircraft classified as aging is 

increasing. By definition and structural status, 

these aircraft require extensive structural 

inspections, repairs and modifications in order to 

remain airworthy. 

• The size of the leased aircraft fleet is at an all 

time high, with continued growth forecast for the 

future. These aircraft move from operator to 

operator and are maintained by various AMOs 

around the world. 

Many airframe specialists are not certified 

because they are not required to return aircraft to 

service. They specialize in structures repair, and 

are not Airframe and I or Powerplant certificate 

holders. They usually work at AMOs and are 

covered under FAR Part 145 repair station 

certification. 

• Since a large percentage of the work done by 

second and third party maintenance providers is 

competitively bid, workload for these operations 

is cyclical with variable staffing demands. This 

has created a significant number of temporary 

contract airfi;raft maintenance personnel agencies. 

The workers in this field are assigned by contract 

to operatio~s worldwide that need maintenance 

staff. They are transient, moving from company 

to company and place to place as needed. Most 

of these workers are non certificated structures 

mechanics with training, qualifications and 

backgrounds that are supported only by resumes 

and word of mouth. 

An independent Structures Repair Committee (SRC) 

was formed by several participants involved in the 

CACRC is also in the process of developing a 

standard for aircraft metal structures repair 

specialists. The intended purpose is to create a 

document that will describe the training, 

qualifications, and certification of aircraft metal 

structures repair specialists as an ARS-I. They are at 

about the same point of development with the 
Chapter 12 

structures repair standards as CACRC is with the 

composite materials repair standard. Meetings to 

continue development work have been held as 

recently as February 14, 15, and 16, 1995. Progress 

continues to be made and this effort will continue to 

be evaluated as a part of this project. 

There is a strong body of thought within the industry 

that aircraft structures repair should be covered by a 

standard and require certification at the level required 

to meet ARS-1 certification. This was demonstrated 

in results from a recent survey that is discussed in 

section 12.6. 

12-5.5 Balloon and Glider Repair 

Balloon maintenance and repair although a relatively 

small segment of the industry, is currently asking for 

specialty status and fits under the ARS-I concept. 

Balloons are not true airframes, nor do they have 

conventional powerplants, yet under current 

definitions they fall under the same FAA rules as 

standard aircraft. Balloons must be maintained by A 

& P mechanics and lA' s under FAR Part 91 as 

general aviation aircraft. They may also be repaired 

by repairmen in certified,repair stations. It is the 

contention of many in balloon operations and 

maintenance that safety is compromised from lack of 

specific training, qualification and certification 

standards. Commercial operators contend that there 

should be a set of minimum standards for both repair 

facilities and maintenance personnel. 

A proposed standard, supported by several operators, 

was presented to the FAA at one of the Maintenance 

Regulatory Reviews in December of 1989. It 

included a minimum equipment list for hot - air 

balloon repair stations, and a minimum task list 

(qualifications) for certified balloon repairmen. 

The FAA concurs with the direction taken by the 

balloon industry and will encourage the completion 

of standards that may be accepted for ARS-1 

certification. 

While there is no specific information available at 

this time, the FAA has also recognized that a similar 

situation to the balloon sector also exists in the glider 

maintenance and repair sector. Means of having this 

sector develop acceptable ARS-I certification 

standards will be explored. 

235 

Training and Certification 

12-5.6 Other Potential Skill Areas 

There is general agreement in ARAC and the FAA 

that the skills listed above represent the areas of 

primary need and focus. Continuing research and 

investigation during the second phase of this project 

will focus on these and other skill areas that are 

potential candidates for specialist certification. 

Working with Northwestern University's 

Maintenance Job Task Analysis team's initial data 

should also serve to verify what the ARAC has 

accomplished. This data should also illuminate any 

other obvious areas where specialist certification 

needs to be considered. 

12.6 1:\H)({\J \L I· \R l.t5 REI'\IR 

S"l \TIO:\ TFC\1'\ICI \'\ Sl R\ I· 'I 

A member of CACRC, with agreement from the 

group, conducted a survey of a cross section of 

FAR 145 repair station operators. This survey was 

random, not intended to be formal nor statistically 

validated. However, it does provide worthwhile 

information, available nowhere else, on the subject of 

advanced certification for specific skill areas in 

certified repair stations. 

The survey was sent to 40 Part 145 repair stations, 

selected from the World Aviation Directory (WAD), 

who perform work on large transport category 

aircraft. It asked for information concerning the array 

of technicians employed at these facilities. The 

questions targeted A & P certified mechanics, and the 

four potential specialist groups considered for ARS-1 

certification by the ARAC-65 working group. 

Twenty-three ot the repair stations responded, which 

at over 57% is a very good response. They were 

asked to provide the following information: 

• 

• 

• 

• 

• 

• 

• 

• 

Total number of technicians employed 

Total number of certified A&Ps 

Total non-certified structural/ sheet metal 

technicians 

Total number of Avionics technicians 

Total number of Avionics technicians with FCC 

licenses 

Total number of Avionics technicians holding 

repairman certificates 

Total number of NDI technicians 

Total number of Composites technicians 

In addition, they were asked to respond to these 

questions: 
Training and Certification 

• What type of maintenance training does your 

company offer? 

• Would the company be better served by 

technicians trained to industry standards? 

Would !he company support development of 

specialist ratings in: 

-Avionics 

- Non Destructive Inspection 

- Stru!'tures 

Unlike the major air carriers, where at least 90% of 

maintenance personnel hold A&P certificates, the 

Part 145 operators employ maintenance staff where 

less than 50% hold A&P certification. Structures 

repair technicians represented almost 35% of the 

population of employees covered by the survey, none 

with certification of any type. It was also interesting 

to note that only 61% of the respondents conduct 

training for technicians in the specialties surveyed. 

This points out that there could be a significant gap 

in competencies between the air carrier and second or 

third party maintenance personnel. 

All respondents indicated that industry standards in 

the specialties listed above would benefit their 

operations. The survey shows that there is interest 

within the industry in the development of standards. 

Those responding were fully supportive of avionics 

and NDI standards and were within one percentage 

point of full support for composite and metal 

structures repair. 

In discussions with individuals from all areas of the 

industry, there seems to be general agreement that the 

development of such standards is a worthwhile and 

necessary undertaking. 

12.7 ESTABLISHED TIUI"'ll\(; .\l\D 

CERTIFIC.\TIOl\ ST.\l\D \RI>S 

Looking at systems and processes by which other 

industries and disciplines develop and maintain 

standards for training, qualification and certification 

of skills will provide examples of how this may best 

be accomplished in the aviation maintenance 

industry. A broad brush snapshot of other industries, 

with both technical and non-technical knowledge and 

skill requirements, has shown that there is a set of 

consistent characteristics. There are two general 

approaches to skill and knowledge certification: 

• Imposed and maintained by governmental 

agency (Federal, State, County, City or District) 

through rules and regulations. 

236 

Chapter 12 

• Self-imposed certification, based on standards 

that are designed to maintain specific levels of 

performance. In most cases the development of 

these standards and the resulting training, 

qualification and certification systems are under 

the auspices of non-profit professional 

organizations. Such standards are usually put in 

place for the purpose of ensuring public safety, 

elevating the professional standing and I or 

perception of a craft, career field, or profession, 

and in some cases to avoid or preclude imposed 

certification I regulation, i.e., American Welding 

Society, Professional Association of Diving 

Instruction, etc. 

There are various national organizations that have 

developed training and certification standards, for a 

wide range of skills, that are in continuous use today. 

Each organization has a board of directors, 

governors, or standards committee, consisting of 

recognized "senior" experts in the respective fields. 

While the actual skills for which the training and 

certification standards have been developed vary a 

great deal, the processes by which they were 

developed, applied, and maintained are similar. Some 

examples of these organizations and information 

pertinent to their successful, currently operational, 

training qualification and certification systems are as 

follows: 

American Red Cross <ARC) 

While far removed from the technical world of 

aviation maintenance, one of the best examples 

of a successful training and certification 

process, which has been effective for nearly a 

century, is ,the method used by the American 

Red Cross. This organization has a solid 

training and certification system that is 

recognized around the world. Their national 

headquarters establishes and maintains 

standards for training and certification of 

various public safety related skills such as: First 

Aid, First Aid Instructor, Jr. Life Saver, Sr. Life 

Saver, and Water Safety Instructor. 

The organization is completely self contained 

and accomplishes all training and certification 

through a comparatively small compensated 

staff and a large and complex national network 

of volunteers. Many organizations recognize 

Red Cross certification as pre-requisite for other 

training, such as Emergency Medical 

Technician, or as a job requirement as in Life 

Guards and Swimming Instructors. 
Chapter 12 

American Weldin&" Society CAWSl 

The FAA does not require additional 

certification for aircraft construction or repair 

welding beyond the Airframe and Powerplant 

ratings. Based on the most recent revision of 

FAR Part 147, A&P mechanics must he able to 

differentiate between acceptable and 

unacceptable welds, but are no longer required 

to demonstrate welding proficiency. (The state 

of the art has progressed well beyond basic 

acetylene gas and electric arc welding.) Many 

airlines and repair facilities, however, require 

welders (especially those performing "exotic" 

and critical welding) in component and engine 

repair shops to he A WS certified. 

The AWS was founded in 1919 to advance the 

science, technology and application of welding. 

It is a non-profit organization that conducts 

welder, welding inspector, and welding 

educator certification programs. The Society's 

over 42,000 members consist of educators, 

engineers, researchers, welders, inspectors, 

technicians, welding foremen, company 

officers, and supervisors. Disciplines include 

automatic, semi-au~Qmatic and manual welding, 

as well as brazing, soldering, ceramics, 

robotics, thermal spraying and lasers. (All of 

these processes are used in the aviation 

maintenance industry.) Activities include 

initiatives in research, safety and health, 

education, training, business, and government 

liaison. Their standards are considered as 

benchmarks in the welding craft. They also 

maintain a system of accredited education and 

test facilities in the fifty States and overseas 

locations. 

An example of their system and the process that 

relates to advanced certification for the aviation 

maintenance industry is their Certified Welder 

program (similar standards exist for Welding 

Inspector and Welding Educator qualification 

and certification). The Society's Certified 

Welder Program is established to identify all 

elements necessary to implement a National 

Registry of Certified Welders. 

The four key elements of the system include: 

I. Welder performance qualification 

standards. 

2. Standard welding procedure specifications. 

237 

Trainin& and Certification 

3. Accredited performance qualification test 

facilities. 

4. A WS welder certification requirements. 

The purpose of the Standard for A WS Certified 

Welders is: 

I. To determine the ability of welders to 

deposit sound welds in accordance with 

standardized requirements. 

2. To impose sufficient controls on the 

documentation and maintenance of 

certification to allow transfer between 

employers without re-qualification, where 

allowed by Standard of Contract 

documents. 

Specific specialties for advanced certification 

include: Chemical Plant, Petroleum Refinery 

Piping, and High Rise Construction. 

Application for certification is extensive and 

includes verification of background, experience 

and education. They also require medical 

certification of acceptable visual acuity 

completed not sooner than six months prior to 

testing and certification. 

The A WS standards are well-defined voluntary 

consensus standards, developed in accordance 

with the rules of the American National 

Standards Institute (ANSI). They provide an 

excellent basis on which to pattern the 

development of standards for training, 

qualification and certification of aviation 

maintenance skills. 

Ra!lio!oa-jcal Techpo!o&"ists I X-Ray 

Iechnjcians 

The system of training, qualification and 

certification of Radiological Technologists in 

the state of California is typical of processes for 

this discipline across the United States. 

The program is administered by California 

Health Services, Radiological Health Branch. 

This organization sets the standards for training 

and curriculum for Radiological Technologists. 

It is generally a 2 or 3 year program conducted 

by the state's community colleges. Successful 

completion of such a program qualifies the 

learner to take the state examination. The 

examinations are conducted by Comprehensive 

Personnel Services (CPS), a for profit 
Training and Certification 

organization that conducts these, and similar 

tests, for governmental agencies. CPS only does 

testing, they conduct no training or other related 

activities. 

There are also Limited Permit Technicians who 

are qualified with shorter duration, specific 

focus courses, often taught by business schools 

or medical technician schools. These courses 

ge~erally certify technicians to perform X-rays 

on specific parts of the body, such as podiatry, 

chest, etc. They are qualified through on the job 

training, and certified upon successfully passing 

a state administered test. 

Board Certified Radiologists (Physicians) 

automatically receive state certification. Other 

physicians may sit for and pass exams to gain 

certification. 

Schools apply to the California Health Services 

Administration for approval of their programs 

by completing an extensive application showing 

their curriculum content. Oversight is 

conducted by Inspectors from the California 

Health Services staff. Limited Permit Programs 

generally receive more scrutiny than the 

programs conducted at the community colleges. 

There is a National Society of Radiological 

Technologists and a California Registry of 

Radiological Technologists. The national 

organization sets the pattern for standards from 

which the California program is adapted. 

Changes are a regulatory process that may be 

driven by the California State Legislature. For 

instance, there is current interest in assuring 

quality in mammography. This is also being 

developed as a new advanced certification 

category. It will require additional training and 

examination after initial certification. 

Re-certification is required every 2 years. The 

re-certification is automatic if the application is 

timely. A continuing education requirement will 

become effective in July of 1996. 

The National Society of Radiological 

Technologists and the Society of Nuclear 

Medicine conduct conferences that often 

include post graduate programs (similar to 

Inspector Authorization renewal conducted at 

PAMA conferences). These groups are at the 

238 

Chapter !2 

level of industry organizations and do not 

develop standards for training, qualification and 

certification. 

Emen:ency Medical Technicians <EMD and 

Paramedics -fCaJifnmjq) 

EMT and Paramedic training programs are 

operated under standards, generally based on 

national guidelines, but developed and 

maintained by individual states. It is also a 

system that uses partnership between 

government regulatory agencies, where the 

public and private educational sector provides 

the training, qualification and certification for 

individuals entering a specific career field. 

The U.S. Department of Transportation issues 

national curriculum standards upon which 

California bases their curriculum requirements. 

The DOT has advisory standing with the states. 

The California Office of Emergency Medi~al 

Services Authority is the regulatory agency. 

They administer 3 programs: 

• EMT I Basic 

• EMT 2 Intermediate 

•EMT3 Paramedic 

EMT l & 2 certification is acquired through an 

approved training agency, usually Community 

Colleges or Junior Colleges. EMTs are 

generally classified as highly qualified first aid 

givers, but not as medical technologists. A 

standard 110 hours of instruction is required, 

usually provided by Community Colleges, in a 

4 to 5 mo11th course. Commercial schools may 

also be approved. EMT I & 2 may be 

administered at the County level, or through an 

association of counties in less populated areas. 

Trainees are given written and practical tests. 

The County agencies can accept the final exam 

from an approved training program, or they 

may administer their own tests. The California 

State Fire Marshal and California Highway 

Patrol also administer EMT I programs. 

State certification, granted after passing the 

initial written and skill examination, is good for 

two years. Continuing education credits, or a 

refresher class, is required to renew certification 

each subsequent two year period. 

Paramedic certification (EMT 3 - Paramedic) 

requires successful completion of EMT I & 2 
Chapterl2 

qualification, plus 1,000 hours of required 

training, usually provided by a Community 

(Junior) College. Persons with this certification 

are considered medical technologists who can 

carry out specific medical practices. These 

include intravenous injections, and operation of 

certain medical test and life support systems. 

State certification is by initial written and 

practical skill demonstration examination and 

remains current for two years. Currency is 

maintained by completing 48 hours of 

continuing education every two years, reported 

to the state board. 

In order to gain certification, schools submit 

their curriculum and qualifications to the State 

for approval. Approval allows schools to be 

included on an approved list and authorizes 

their programs for instruction. 

The California Office of Emergency Medical 

Services Authority goes through a full Office of 

Administrative Law process when changing 

their requirements or regulations. There is a 45 

day notice and solicitation of public comments, 

then a hearing, etc. 

There is a National Registry of EMTs and 

Paramedics. The National Registry is a not-forprofit, non-governmental organization. It is 

governed by a Board of Directors made up of 

users of their services and professional medical 

people. They have been in operation since 

1970. They conduct certification and recertification exams for those states and 

organizations who choose to use them. They 

conduct tests that some states use for 

certification. They feel they set the standards 

for the nation. They refer to the DOT standards, 

but base their standards on a job analysis. 

Changes to the standards are cyclical. 

Sometimes the DOT initiates a change to which 

they respond and sometimes technology or 

technique improvement requires change. 

There is also a National Association of EMT 

and Paramedics. Some state and local 

organizations provide forums and there are 

some private organizations that put on 

conferences and trade shows. 

239 

TraininG and Certification 

American Sailinv Assocjatjoo CASA> 

There are no government agencies, including 

the US Coast Guard, that require any type of 

certification for recreational, non-commercial, 

water vessel operators. There is no 

demonstration of skill necessary for commercial 

skippers operating water vessels under 500 tons 

displacement under Coast Guard regulations; 

passing of a written examination only meets the 

certification requirement. The ASA standard is 

an excellent example of a certification process 

that is maintained by a specific industry without 

any governmental regulatory oversight. 

The American Sailing Association is dedicated 

to promoting safe recreational sailing in the 

United States by administering an 

internationally recognized educational system. 

ASA is an association of sailors, professional 

sailing instructors, sailing schools and charter 

companies. 

ASA is a private, for profit, organizatioh 

recognized around the world. Their association 

with the International Sailing School 

Association (ISSA) allows for recognition of 

ASA certification by many national authorities, 

charter and insurance companies around the 

world. The group was formed to promote 

sailboat operations safety and ensure acceptable 

levels of proficiency for various levels of 

sailboat chartering and rental. 

Their Official International Log Book provides 

information about the standards and 

certification requirements for various levels of 

sailboat operational skills (including instructor 

certification). This group has developed and 

maintains standards of training and certification 

for non-commercial skippers who become 

certified in order to rent "bare boat" charter 

sailboats for pleasure cruising, or various other 

sail boats for personal recreation. The document 

is excellent. It is clear, brief and concise yet 

complete in all essential details. The Log Book 

is also used to record completion of the various 

levels of certification. Review of the Log Book 

is required by charter companies before a boat 

is released to a skipper. This system is very 

similar in oature to the requirements that a pilot 

must meet in order to rent an aircraft. 

The training system is progressive and 

encompasses both knowledge and skill 
Training and Certification 

requirements. All standards are considered as 

minimum for the respective certifications. 

There are pre-requisites for more advanced 

certifications. Starting with the entry level in 

the Basic Keelboat Sailing Standard that has no 

pre-requisites and is described as: "Able to sail 

a small boat of about 20 feet in length in light to 

moderate winds and sea conditions in familiar 

waters. without supervision. A preparatory 

Standard with no auxiliary power or navigation 

skills required." The skills advance through 

Basic Coastal Cruising, Advanced Coastal 

Cruising, though the most advanced Offshore 

Passage Making that has the prerequisites of all 

previous keelboat and navigation standards and 

is described as: "The sailor is able to safely act 

as skipper or crew of a sailing vessel on 

offshore passages requiring celestial 

navigation." 

All written testing on "Sailing Knowledge" 

must be passed with a score of 80% or higher 

and demonstration of skill competency, "Sailing 

Skill," is evaluated by an ASA certified . 

instructor. All certification is provisional until 

reviewed by the organitational headquarters 

who issue the final seal of approval. This 

process is very similar to FAA Airman 

certification as it relates to their system of 

written testing, an oral and practical test 

conducted by a designated examiner, followed 

by review and final certificate issue. 

Professional Assocjatjon of Divjng Instruction 

(FAliD 

PAD! is another example of a nongovernmental certification system. While not as 

complex as others, it serves the interest of 

public safety by ensuring at least basic 

knowledge before individuals may rent Self 

Contained Underwater Breathing Apparatus 

(SCUBA) or have air supply tanks filled. 

Approximately 28 hours of instruction, that 

includes at least one actual "deep water sea 

trial" (not in a swimming pool) dive. 

Certification includes both a written test and 

skills demonstration to the satisfaction of a 

PAD! certified instructor. Lack of recent 

experience requires re-certification to assure the 

diver remembers the safety factors and can 

properly use and operate SCUBA equipment. 

The system is very similar to the one that was 

developed by ASA and has all the basic 

240 

Chapter 12 

characteristics of agencies that are in the 

standards and certification arena. 

There is no question that excellent models for 

building an organization to develop standards of 

education, qualification and certification exist within 

the US. The organizations discussed in this chapter 

have provided information freely and would lend 

support to others wishing to develop such systems. It 

appears that the aviation maintenance-industry, by 

looking at the example set by others with similar 

charters and interests, could move toward the 

development of a national standards organization 

without a high degree of difficulty. 

12.8 TilE CANADIAN AIRCRAFT 

\L\INTENANCE SPECIALIST 

CEIHIFIC.\TION SYSTEI\1 

I

t was not possible to visit and meet with officials at 

Transport Canada in Montreal as planned. This 

visit and in depth discussions, will take place during 

Phase II of this project. There is, however, a good 

deal of information about the Canadian certification 

system that is pertinent to this phase of the project. 

There are aspects of the Canadian system that are 

directly applicable to the directions being taken in the 

US and may serve well as a model. 

The Canadian aviation regulatory and certification 

system is the responsibility of Transport Canada (TC) 

which is their equivalent of our FAA. While similar 

to the United States system in many ways, there are 

some differences that should be considered: 

• The Canadian40.viation maintenance industry is 

smaller than that of the USA. The current 

number of Aircraft Maintenance Engineers 

(AME), who are the equivalent of Airframe and 

Powerplant Mechanics (A & P), is about 32,000, 

versus about 148,000 A&Ps in the USA. 

• Transport Canada has recently revised the AME 

certification process, moving more toward a 

system similar to the FAA system. This moved 

Canada away from their former system that was 

closer to their European history and the 

International Civil Aviation Organization 

(ICAO) standards and practices. Under ICAO all 

maintenance certification authority is vested in 

the Approved Maintenance Organization 

(AMO). An AME is trained as a generalist with 

specific aircraft type-training requirements, 
Chapter 12 

return to service privileges, and is independently 

certified. 

• There is a group similar to ARAC in Canada; 

Canadian Aviation Regulatory Advisory 

Committee (CARAC) with a working group on 

maintenance certification and control. In 

activities much like those that have been 

conducted by the ARAC - 65 working group, the 

Canadians are moving toward broader AME 

licensing privileges and specialist licenses. It 

appears that their certification process will move 

even closer to that of the FAA than it is at 

present. 

• Apprenticeship programs are in place through 

which an individual may become certified as an 

AME. These individuals are under the 

supervision of a qualified trades person learning 

the principles, skills, tools and materials of the 

trade while observing, practicing and 

accomplishing work. They also attend short 

technical courses at a college or technical 

institute. 

A 1991 Price W atemouse $tudy, Human Resources jn 

the Canadian Aircraft Maintenance Industry, 

sponsored by Employment and Immigration Canada 

produced similar findings to those of the Pilot and 

Aviation Maintenance Technician Blue Ribbon 

Panel. 

Canada has also recognized the need for certified 

specialists in specific skill areas. They have in place 

the Canadian Aviation Maintenance Council 

(CAMC) which was formed for the following 

purposes, as stated in their introductory pamphlet: 

The council was created to address challenges facing 

the industry. These challenges were identified in a 

comprehensive human resources study prepared for 

the industry that inclulkd: 

• The need to overcome the lack of formal training 

programs available for non-licensed skilled 

tradespersons. 

• The need to meet ever - rising requirements for 

the entry into skilled trades. 

• The need to establish criteria to recognize skills 

of the aircraft maintenance workers. 

241 

Training and Certification 

• The need to increase retention of new recruits 

especially among smaller employers. 

The CAMC is a decision-making body. It manages 

current business, sets specific objectives, policies and 

procedures, and coordinates the efforts of various 

committees. The committees cover topics such as 

occupational standard, training programs, 

communications and financing, among others. The 

Council supports and encourages initiatives to 

develop the overall strength and economic well being 

of the Canadian Aviation Maintenance Industry both 

locally and internationally. 

The membership of the group covers the full industry 

spectrum, represented by an equal number of 

emnlover and emolovee or2anizations includin2: 

Air Transport Association of Canada 

Aerospace Industry Association of Canada 

Canadian Auto Workers 

International Association of Machinist and 

Aerospace Workers 

Canadian Federation of AME Associations 

CAMC has identified 22 occupational areas and is 

currently developing occupational standards for these 

thirteen aviation maintenance skills: 

Avionics 

Electrical Component 

Electroplating 

Gas Turbine Repair and Overhaul 

Interior Refinishing 

Machinist 

Mechanical Component 

Non-Destructive Testing 

Painting 

Reciprocating Engines and Propellers 

Structural Repair 

Weldin2 

To ensure high qualily standards, a technical 

committee, composed of knowledgeable 

tradespersons, is established for each skill area 

(''trade"). 

12.9 .IOI'\ r \\I\ I IO'\ 

RE(;t L \TIO'\S 1.1 \Ri h5 IU.\ ISIO'\ 

s r \Tl s 

J

oint Aviation Regulations (JAR) 65, which is the 

European Economic Communily (EEC) equivalent 

to FAR Part 65 has been in the process of 

development through seven revisions. It is being 

developed under the control of the Joint Aviation 

Authorily (JAA) which is the EEC regulatory body. 
Training and Certification 

The rule is not scheduled to become fully 

implemented until July of 1999. Harmonization with 

the FAR 65166 is on the agenda, but was not placed 

on the docket for 1995 11996 as of the last working 

group meeting in March of 1995. 

The JAR 65 approach is very different from that of 

both the USA and Canada, in that all maintenance 

certification authority will be vested in the Approved 

Maintenance Organizations. It seems apparent that 

JAA is committed to a model that will handle 

differences and variances that exist between the 

member nation states through accommodation. This 

suggests that the AMO will remain the basis for the 

total maintenance certification control program. 

Historically, many "flag" carriers have become 

accustomed to near regulatory control within their 

own country. These carriers seem hesitant to give up 

this level of influenced and control. The countries 

that have their own certification system are not 

comfortable with loosing their independence to a 

system of AMO control. 

Some countries place high value and specific 

requirements on structured formal training as part of 

certification, while others place emphasis on 

certification based in on-the-job training. In some 

cases maintenance personnel are trainined to a level 

of qualification with no certification requirement. It 

appears difficult for any consensus to be achieved in 

this environment without accommodating many 

divergent points of view. 

The USA and Canada, who have taken the approach 

of centralized certification control, through 

regulating training, qualifications and certification, 

feel that this is best for all concerned. Since there is a 

strong core of agreement between the two countries, 

and given the recent North American Free Trade 

Agreement (NAFTA), they are moving toward 

harmonization in North America, which may also 

include Mexico. 

Harmonization between JAA and the FAA may not 

be as simple, especially in the area of maintenance 

technician certification, as initially thought. It also 

appears that it has become a lower priority than it 

was only a few months ago. The challenges 

presented, and differences that exist, between the 

proposed JAA system and both the US and Canada 

do not appear to be approaching resolution in the 

near future. 

242 

Chapter 12 

12.10 ORGANIZ·\TIOI\S THAT ARE 

POTEYI'l.\1. CEIHIFICATIO~ 

STA~D \IW DEVELOPERS Al\D 

"KEEPERS OF TilE FL\!\IE" 

Several professional organizations have been 

suggested and I or discussed as having potential 

to become those who may develop and maintain 

aviation maintenance advanced certification 

standards. It is has also been suggested (for purposes 

of harmonization) that such organizations may need 

to be compliant with International Standards 

Organization (ISO) standards series 9000, and I or by 

the Board of Accreditation (RAB) that is part of the 

National Standards Institute (NSI). Following is a 

listing of possible organizations: 

Aircraft Electronics Association 

American Society for Nondestructive Testing 

Society of Automotive Engineers 

Air Transport Association 

Aircraft Industry Association 

Performance Review Institute 

National Aerospace and Defense Contractors 

Accreditation Program 

Commercial Aircraft Composite Repair Committee 

(and several others that may become interested) 

There is another point of view that suggests that it 

may not be in the national interest to specify one or 

more of these existing organizations to hold the 

"keeper of the flame" responsibility. It may be more 

advantageous to allow all recognized groups who 

develop, validate and maintain standards to prepare 

training, qualification, and certification standards for 

aviation maintonance advanced skills as they see fit. 

These standards, however, may be required to 

conform to a set of overall requirements, developed 

and maintained by a national steering, oversight, or 

executive committee. This committee, with 

membership consisting of high level industry 

"experts" would act as the "keeper of the flame" and 

endorse standards for aviation maintenance advanced 

skills and certification. The FAA, in tum, would 

accept certification standards that meet the specific 

requirements of this high level group for ARS-I 

certification. This approach bears some similarity to 

the CAMC system in Canada, which will be studied 

further. 

Determination of the industry and FAA views on this 

subject will be researched further; studied, reviewed, 

and reported upon in the next phase of this project. 
Chapter 12 

12.11 0 I HER RECL L \TOin· 

1\IPRO\ E\IE\ r U.E\11·.\ rs TO 

CO\SIIH.R 

During !he course of !his project, there are other 

areas that may be reviewed as having potential 

for creating an improved method for obtaining 

information from !he regulator's perspective and 

input from industry, while upgrading industry I 

government participation in rulemaking. 

Integration of ARS standards and FAR Part 145 

Future training scenarios 

AMT School self testing 

FAR Part 147 flexible curriculum 

More privileges for AMT (Annual inspections 

for part 91 aircraft, etc.) 

Harmonization - Canada, NAFT A, and rest of 

the World 

"Seamless'" maintenance training scenarios from 

primary ihrough recurrent. 

12.12 CO\CLL SIO\S 

The aircraft maintenance industry is in a state of 

change. While this S\'lte of change has been in 

process over !he last decade, the rate of change has 

increased over the past three to four years. All 

indications point toward the continuation of this 

trend, at perhaps even a faster and more dramatic 

rate. The regulatory process, as witnessed by the long 

overdue changes to FAR Part 147 and the changes 

currently in process for FAR Part 65, is slow to 

respond and has failed to keep pace with ongoing 

industry changes. 

While the ARAC process may be a starting point for 

regulatory management, it needs to continue to 

evolve. There is also an apparent need to conduct a 

more in depth evaluation of the need to convene a 

national aircraft maintenance standards oversight 

council, or committee. The membership may consist 

of high level aviation industry and FAA officials who 

have strong process orientation. The group would 

have the "Big Picture" of both the technology and 

maintenance processes with insight into how they 

may best be applied. It could also serve as the 

umbrella organization !hat provides oversight for 

other groups !hat have been qualified to issue and 

maintain training and qualification standards. This 

group could be similar to the board that has this type 

of function in Canada. 

As the study moves forward, support for this type of 

system continues to grow. The supporters of 

243 

Trainin& and Certification 

specialists, advanced skills certification and 

improvement of aircraft maintenance technician 

professionalism far outnumber !he dissenters. This 

majority is also cognizant of the need to harmonize 

regulations and standards, where possible, wiihin the 

international community. They also believe that 

regulatory congruence with Canada and other 

NAFfA countries' aviation maintenance regulations 

will be of significant benefit to North America as we 

move to toward harmonization wiih the EEC, 

Austral-Asia and Middle Eastern countries. 

There seems to be little doubt that a system of this 

type is needed. The next phase of this project will 

more completely explore the alternatives, 

opportunities and necessity for development of 

systems to provide advanced aviation technical 

training, qualification & certification. It will provide 

!he foundation of information necessary to begin 

putting the process in place, and will have established 

the multi - discipline network required to move 

forward. 

12.13 REFERE\CES 

Federal Register Part VI, Department of 

Transportation, Federal Aviation Administration, 

14 CFR Parts 65 and 66, Revision of 

Certification Requirements: Mechanics and 

Repairmen; Proposed Rule (Notice of Proposed 

Rule Making: Notice No. 94-27) 

Education and Training Standard for Entry Level 

Avionics (Aviation Electronics) Technicians and 

Engineering Technologist Programs. Association 

for Avionics Education Working Document, • Nolan Coleman, September 1994. 

ARAC- 65 Working Committee Meeting Minutes 

and attendee notes from meetings conducted 

1992, 1993, March 3-10, 1994, May 17-19, 

1994, July 6-7, 1994, September 15-17, 1994, 

November 28-30, 1994. 

US Department of Transportation Federal Aviation 

Administration - Pilot and Ayiatjon Maintenance 

Techojcjans for the Twenty-First Ceoturv· An 

Assessment of Ayailability and Quality. Pilot 

and Aviation Maintenance Technician Blue 

Ribbon Panel (Blue Ribbon Panel Report) 

Initiated by: Deputy Associate Administrator for 

Regulation and Certification. August 1993 
Training and Certification 

American Sailing Association - Official International 

Log Book Copyright 1993 by the American 

Sailing Association. 

American Welding Society Publications as follow: 

OC 1-<t 1992 - Guide to AWS Welding Inspector 

Qualification and Certification 

OC5-G 1992- Guide to AWS Welding Educator 

Qualification and Certification 

QC7 -9'3 - Standard for AWS Certified Welders 

QC7-93 Supplement C- Welder Performance 

Qualjficatjoo Sheet Metal Test ReQuirements 

QC7 -93 Supplement F - Chemical Pant and 

Petroleum Refinery Pipin~ 

QC7-93 Supplement G- AWS Performance 

Qualification Test 

Employment and Immigration Canada, Hlllnllll 

Resources in the Canadian Aircraft Maintenance 

lndustrv: Price Waterhouse, March, 1991. 

Canadian Aviation Maintenance Council, 

Introductory Pamphlet 

JAR 65 Draft, NPA 65-0 (dated 1.2.1995) 

APPENDIX 12-A- SUBSTANTIVE 

RECOMMENDED CHANGES TO PART 65 

• Removal of Gender -Specific Terms 

• Re-designation of the Term "Mechanic" 

Equivalency of Ratings 

• Replacement of Lost or Destroyed Certificates 

by Facsimile 

244 

Chapter 12 

• Demonstration of English-Language Proficiency 

and Removal of Exception Criteria for 

Applicants Employed Outside the United States 

Who Are Not Proficient in the English 

Language. 

• Establishment of a Requirement for Aviation 

Maintenance Technicians To Pass a Written Test 

on all Applicable Provisions of Chapter 14. 

Clarification of Requirement To Pass all 

Sections of the Written Test Before Applying 

for the Oral and Practical Tests 

Recognition of New Written Testing Methods 

• Specification of Experience Requirements in 

Hours 

• Establishment of Basic Competency 

Requirements 

• Use of Equipment-Specific Training to Qualify 

for Certificate Privileges 

• Use of Instructional Time by Aviation 

Maintenance Instructors to Satisfy Currency 

Requirements 

Establishment of Training Requirements for 

Certificated Aviation Maintenance Technicians 

Exercising the Privileges of their Certificates for 

Compensation or Hire 

Extension of Inspection Authorization Duration 

Expansiop of Inspection Authorization Renewal 

Options 
Phase V Report 

Appendix 

Modified Proceedings 

from Meeting 9 

245 
Phase V Report 

Ninth Federal Aviation Administration Meeting on 

Human Factors in Aircraft Maintenance and Inspection 

Sheraton Inn - Old Town 

Albuquerque, New Mexico 

Agenda 

"HUMAN FACTORS GUIDE FOR 

AVIATION MAINTENANCE 

AND INSPECTION" 

246 

Appendix 
Appendix Phase V Report 

Monday, November 7, 1994 

3:30p.m. - 4:30p.m. Registration - No other activities planned 

Tuesday, November 8, 1994 

7:30a.m. Continental Breakfast and Registration 

9:00a.m. Welcome and Meeting Objectives 

William T. Shepherd, Ph.D. 

Office of Aviation Medicine, FAA 

9:10a.m. Keynote Address FAA Perspective on Human Factors 

Research 

Ms. Darlene Freeman 

Associate Administrator for Aviation Standards, FAA 

9:30 a.m. The Human Factors Guide for Aviation Maintenance -

Introduction 

10:00 a.m. Break 

Michael Maddox, Ph.D. 

Sisyphus Associates 

10:20 a.m. Integrating Human Factors into a Maintenance Program 

Colin Drury, Ph.D. 

Dept. of Industrial Engineering, University of Buffalo 

11 :00 a.m. Workplace Safety 

James Bumene, Ph.D. 

Professional Ergonomist 

11:45 a.m. Lunch (in Hotel/Hosted) 

1:15 p.m. The "Guide"- Demonstration Exercise 

Michael Maddox, Ph.D. 

247 
Phase V Report 

Tuesday, November 8, 1994 cont. 

2:00 p.m. The Electronic Guide - A Demonstration 

Kiki Widjaja 

Galaxy Scientific Corporation 

2:40p.m. Human Factors in Maintenance and Engineering 

Mr. Edward Rogan 

Appendix 

Human Factors Manager, Engineering Quality & Training 

Services 

British Airways 

3:25 p.m. Break 

3:45p.m. , Shift Work and Scheduling 

Ted Baker, Ph.D. 

President, Shift Work Systems Inc. 

4:30 p.m. Adjourn 

6:30 p.m. Reception 

Wednesday, November 9,1994 

7:30a.m. Continental Breakfast 

8:30a.m. The "Guide"- Exercise Introduction 

Michael Maddox, Ph.D. 

8:45 a.m. Group Work on the Guide 

10:00 a.m. Break 

248 
Appendix 

10:20 a.m. Group Reports 

Wednesday, November 9, 1994 cont. 

ll :30 a.m. Human Factors Guide Evaluation/Discussion 

Michael Maddox, Ph.D. 

11:45 a.m. Lunch (in Hotel/Hosted) 

Phase V Report 

l :00 p.m. Performance Enhancement System (PENS): Update 

Charles Layton, Ph.D. 

Galaxy Scientific Corporation 

1:30 p.m. CD-ROM #2- A Demonstration 

William Johnson, Ph.D. 

Galaxy Scientific Corporation 

1 :50 p.m. Sandia NDI Research Overview 

Patrick Walter, Ph.D. 

NDI Validation Facility, Sandia Laboratory 

2:20 p.m. Closing Comments 

William T. Shepherd, Ph.D. 

Office of Aviation Medicine, FAA 

2:30p.m. Bus trip to Aging Aircraft NDI Validation Facility 

• 

3:00p.m. NDI Center Tours 

4:30 p.m. Transit to Hotel 

5:00p.m. Adjourn 

249 
Appendix Phose V Report 

HUMAN FACTORS 

IN AVIATION MAINTENANCE I 

Dr. Jon L. JOfdan, 

Federal Air Surgeon 

Aviation safety is dependent on the cooperation of 

operatoiS, manufactureiS, and the government 

(FAA). That triad is often referred to as "the three 

legged stool," with each leg representing one of these 

groups. I'm sure all of you have seen that stool 

depicted before in other seminaiS and workshops. 

The stool must have all three legs to be balanced and 

safe. 

The Office of Aviation Medicine Human FactoiS in 

Maintenance Resean:h Program bolsteiS the 

association and "the working together attitude" of 

government, manufactureiS, and operatoiS. Aviation 

Medicine does that through conferences like this one, 

through extensive publications, by distribution of CDROMs containing data and information, and from 

active participation in meetil'!ls worldwide. 

Aviation Standards is proud of the 6 year history of 

the Human FactoiS in Aviation Maintenance Research 

program. As Associate Admimstrator, I appreciate 

the opportunity_to be here, to speak with you, and to 

see first-hand why these Human FactoiS Workshops 

have become so popular. 

The historical basis for embarking on this research 

program can, most likely, be attributed to the 1988 

Aloha accident. The accident showed that human 

performance issues, are critical componets for safety, 

no different from adherence to Service Bulletins, use 

of proper inspection procedures, and maintenance 

training. The stark picture of the wrecked 737 moved 

Congress to quickly pass legislation that was already 

under consideration at the time. 1he Aviation Safety 

Act of 1988 specified that the Department of 

TllUISportation shall conduct safety research, which 

must atteud to "human performance" not only in the 

cockpit but also in maintenance operations. That 

legislation was significant. It gave us the financial 

wherewithal to focus resean:h on the aviation 

maintenance technician as the central and most 

critical part of the total aviation maintenance system. 

251 

The studies we have performed since 1988 have 

clearly shown that human performance in 

maintenance can have a strong impact on safety. 

In 1991 the FAA published the National Plan for 

Aviation Human FactoiS. That plan, authored by a 

panel of 50 human factoiS professionals from 

industry, academia, and government, also emphasized 

the need for maintenance research. 

Recommendations from the National Plan have 

guided the Aviation Medicine research, but just as 

importantly, feedback you have given us at these 

workshops has shaped our program. 

While safety is always our foremost concern at FAA, 

we cannot ignore the current tough economic times in 

the air transportation industry. We must work with 

industry to be sure that safety is never compromised 

by economics. This research program bas been 

successful partly because of its attention to economics 

and to providing cost-effective research solutions and 

products that enhance human performance. The 

results, therefore, are improvements in efficiency 

while maintaining the high safety standards of our 

industry. 

Six years of research, and now nine workshops, have 

already created a fesearch legacy. I would like to 

briefly highlight a bit of history and a few of the 

milestones that have been achieved in our research 

program, and the contributions made by you and your 

colleagues in maintenance and in human factors 

research. 

The first workshop, was held in the Fall of 1988. At 

that time we were formatting our research plans and 

establishing key industry liaisons. When I say "we" I 

mean not just FAA but all of us who have been 

working on maintenance human factors including you 

folks from industry, academia and other government 

agencies. I am really happy to say that "we" have 

been an effective team that has produced some solid 

results in the past few yeaiS. Please understand that 

very few of these results would have been achieved 

without your active participation. 
Phase V Repon 

The topic of Human Factors in Maintenance and 

Inspection was a new one at the time of the first 

meeting. While some maintenance research had been 

done in the US Military, Aviation Human Factors 

research was primarily a study of cockpit and air 

traffic control issues. Research in maintenance 

human factors was almost non-existent. A few key 

problems were identified at that first meeting. The 

topic that received the most attention was 

"communication," including just about every 

imaginable communication form ranging from design 

of job cards to use of understandable language in 

manuals, to holding workshops like this one. It was 

suggested at the first workshop that we have "at least 

one additional meeting of this kind .... ". This being 

the ninth meeting -- you certainly followed that 

recommendation! Meeting attendance has increased 

300%, and would probably be greater if we did not 

limit the number of workshop participants. 

In six years the research team has published over 150 

articles and conference papers. For four years now 

the prestigious Human Factors and Ergonomics 

Society has dedicated a complete session to the 

Aviation Medicine Maintenance Human Factors 

program. We presented again this Fall at their 

meeting in Nashville. We have supported the 

International Civil Aviation Organization (ICAO), by 

lecturing at 9 Regional Seminars on Human Factors 

in Aviation Maintenance. The research team has 

participated in a variety of Air Transport Association 

and FAA meetings as well. 

That first meeting produced some other 

recommendations about maintenance training, or the 

lack thereof, and FAA has paid attention to those 

recommendations. First, the Flight Standards Service 

modified FAR Part 147 dealing with curriculum in 

Aviation Maintenance Technicians Schools. Also, 

FAA established a Blue Ribbon Panel on the 

Shortage of Personnel for Aviation Occupations. 

Data from this human factors research program 

played a role in the conclusions and recommendations 

of the Blue Ribbon Panel. Likewise the panel 

provided ideas that have become topics of current 

human factors research. Our current study related to 

innovative maintenance certification practices is one 

such research example. 

The program has conducted research ranging from 

basic scientific studies at the University level and at 

the Civil Aeromedical Institute, to applied 

252 

Appendix 

development and evaluation in the airlines and 

schools. 

In 1989 the team was introducing such terms as 

"intelligent tutoring, and expert system trainers" for 

aviation maintenance. You built, evaluated, and 

distributed such systems at training schools and with 

cooperating airlines. Much of the "advanced 

technology training" language this research team was 

innovating in 1989 has become common training 

language used today. 

The 1988 meeting also placed importance on 

maintenance information. Your program has studied 

information collection, format, storage, retrieval and 

display. You have worked with airlines to redesign 

and evaluate job work cards. You have assisted the 

Flight Standards Service with design, development, 

and fielding of the pen computer-based Performance 

Enhancement System, or "PENS" as it's known 

today. That work is continuing with a look at other 

types of electronic support systems both for airlines 

and FAA inspectors. 

Your systems' analytic approach to the design and 

fielding of PENS has set an exemplary standard for 

new technology design development and 

implementation. The concept of "human-centered" 

design and extensive use of rapid prototypes have 

become very popular with users and with FAA 

management. Unequivocally, PENS is enhancing the 

work performance of Aviation Safety Inspectors. 

With respect to information technologies, your 

program was the first to publish and distribute 

information on CD-ROM. CD-ROMs represent the 

cutting edge of information technologies. There is no 

question that they, and similar systems will be critical 

to the efficiency and quality of future Aviation 

Maintenance operations, not to mention the 

performance of our own FAA oversight activities. 

While the Aviation Maintenance research program 

has compiled a good track record, it is you the people 

in this audience and your colleagues who deserve the 

credit. This research program is applied -- it is in 

tune with industry conditions and requirements. We 

feel that the industry has welcomed our input and has 

been very candid in reporting many opportunities for 

improvement. 

The list of airlines cooperating with our research 

program, is pretty impressive. Atlantic Southeast, 
Appendix 

Continental, Delta, Northwest, US Air, United, 

Southwest, TWA, British Airways, and Lufthansa 

have all taken an active role as host sites for data 

collection or as speakers at these workshops. 

Numerous others have permitted on-site operations to 

allow observation and measurement, and provided 

personnel time and advice. Manufacturers have also 

been generous with their time and counsel. 

The single most important characteristic of the 

research program is its attention to "real-world" 

aviation maintenance considerations. Again our 

thanks to you -- in industry -- for. your help. 

So much for the past; What about the Future of 

Maintenance Human Factors Research? 

Have all the human performance issues been 

addressed? Is the system "fixed?" Is the work of 

this research program over? The answer to all of 

these is "No." There are still plenty of untouched 

opportunities to enhance human performance in 

maintenance. 

As current or older technology aircraft continue to fly 

there must be increasing maintenance diligence and 

caution to ensure continuing airworthiness. These 

aircraft will require new maintenance practices and 

inspection procedures. These procedures will have to 

capitalize on new technology methods, which will 

come about through research. 

New aircraft will require new skills. Technicians 

already accustomed to aircraft built-in-test equipment 

will have to convey this knowledge to their coworkers. Changes in FAR Part 147, and plans for 

changes to Part 65, already have placed emphasis on 

new electronics-oriented skills for the aviation 

maintenance technician. In fact we see this as one of 

the most important areas for our research. Terms 

such as "the information super highway" are familiar 

to all of you. These terms are especially pertinent in 

aircraft maintenance where it is crucial that proper 

information is exchanged. The volume of dnta and 

information needed to maintain today's air carrier 

fleet is even now approaching overwhelming 

proportions and it wiU become even larger in the 

future. If technicians are not to spend all of their time 

looking up or finding repair information, they must 

hove assistance, electronic assistance, in getting the 

information needed to perform their jobs. 

Technical information and the way it's provided are 

changing. There will be ari increasing volume of 

253 

Phase V Report 

technical data, available to the technician. Such data 

must be provided at the aircraft. Otherwise, 

technicians would spend too much time traveling 

around obtaining needed information. Portable 

maintenance access terminals, band-held computers, 

and other such technologies promise to change the 

way technicians work. I think we are at the threshold 

of some dramatic changes in the way maintenance 

will be performed in the future. For the air carrier 

industry to stay safe, competitive and viable there will 

have to be some big improvements in efficiency of 

their operations across the board, but particularly in 

the realm of maintenance. We want our research to 

be on the forefront of this transition to help provide 

industry with the knowledge and tools needed to 

make their operations as safe as possible and also 

achieve new efficiencies. 

The work of human factors researchers, obviously, 

must position the industry to maximize the 

performance of humans in the maintenance system. 

That job is never completed. The good news is that 

the FAA and Industry recognize the importance of the 

investment in human performance research and 

development. We are on the right path and we have 

participation, and great support from government, 

industry, operators, and academia. We must 

continue, on course, with the Aviation Medicine 

Human Factors in Maintenance research and other 

related endeavors. 

For the next day and a half you'll be discussing a 

draft version of the Maintenance Human Factors 

Handbook. This handbook has been in preparation 

for the last few years and depends quite a bit on 

results obtained frpm the research program. We want 

the handbook to be a useful document for 

maintenance management. Typical research reports 

in scholarly journals are not of much use to people 

who work on the front lines in industry. These people 

have real world human factors problems they have to 

deal with every day and we've tried to aim our 

handbook at those types of problems. We want our 

handbook to be used, so during this meeting you're 

going to practice with some hypothetical human 

factors scenarios that might be found in a 

maintenance organization and show how the book can 

be put to good use. I hope the next couple of days 

will be rewarding and informative. 

Again, I appreciate the opportunity to participate in 

this meeting with you. I look forward to continued 

involvement with the program. 
Appendix Phase V Repon 

HUMAN FACTORS 

IN MAINTENANCE AND ENGINEERING 

Mr. Edward Rogan 

November 8, 1994 

Good afternoon, Ladies and Gentlemen. It really is 

a pleasure to l?e here and I would like to thank 

Bill for inviting me along to address the conference. 

As Bill said I am the Human Factors Manager 

Engineering at British Airways which is a fairly 

unique beast in our industry. The actual position was 

created in March of this year, so I'm a bit of a new 

kid on the block. Also from my introduction, you can 

see that career is all within aircraft engineering and 

maintenance and I have no psychology or Human 

Factors background. However, I am fortunate enough 

to have a Human Factors Consultant working with 

me who is very supportive in those areas engineers 

are traditionally very had, you know, things like 

spelling, grammar and punctuation to mention but a 

few. Again from my introduction you would have 

seen my background has been primarily in what we 

call Technical Services. I think in the States you 

would call it Engineering. In that time as a 

development engineer, I was continually frustrated 

when involved in a quality lapse investigation to 

conclude that the cause of the incident was human 

error. I was therefore very pleased that Human 

Factors gave us the opportunity to go deeper into 

these investigations and look for the actual cause of 

the quality lapse not the consequence. So this was 

one of the reasons why I was attracted to the subject 

of Human Factors. I believe the appointment of a 

Human Factors Manager Engineering shows a lot of 

commitment to the subject by British Airways, our 

task now is turning that commitment into action. One 

thing I should mention is that we believe not one 

person can have complete responsibility for Human 

Factors, we see everyone in the organisation as 

having this responsibility. We see Human Factors in 

terms of organisational culture where everybody 

understands and has bought into it as well as owning 

it, very much like the current Total Quality ethos at 

BA. 

So what am I going to talk about today? I would like 

to give a brief overview of some of the British 

255 

Airways Human Factors initiatives with which we are 

currently involved or planning to do in the future. I 

would then like to concentrate on one particular 

initiative called MESH. I will describe what MESH 

is, where it came from and what it looks like, which 

means another computef simulation or 

demonstration. I' II talk a bit about the trial we have 

ongoing at the moment in British Airways. I' II just 

share some very early results and I wouldn't like 

anyone to read too much into these results. Finally, I 

would like to talk about where we're going with 

MESH in the future and then take any questions you 

may have for me. 

So what Human Factors initiatives do we have at 

British Airways Engineering? First and most 

important of all, we need commitment from the top 

before we can change the culture of an organisation. 

It's a top-down approach. We have a Human Factors 

Steering Group which meets every two months and 

has done for the last two years and is attended by our 

General Managers. This high level involvement 

ripples through the organisation to say that we have 

• commitment here and that is essential for others to 

see before they buy-in. 

As I said before, we see an organisational culture 

change as a key element in any overall Human 

Factors programme. To change culture you must first 

make everyone aware of the subject and change some 

traditional attitudes towards human error. 

We have started to produce a quarterly Human 

Factors magazine with a circulation of about fifteen 

hundred, the third issue is due to be published next 

month. These are distributed to staff in our Hangars, 

workshops and offices throughout Engineering. The 

magazine not only allows staff to learn from past 

mistakes and incidents but also make them more 

aware of the subject by introducing them to some of 

Prof. Jim Reason's (University of Manchester) 

theories on human error. The magazine also serves as 
Phase V Report 

a way of communicating the progress of some of the 

other Human Factors initiatives which I will be 

talking about later. 

I would like to now talk about Human Factors 

education. We are currently involved with an 

education programme for managers which we call 

Emergent Leadership. This intensive programme 

encoura~es managers to re-examine many of their 

values (e.g., blame and fear free culture, openness 

and honesty, etc.) in order to change traditional 

attitudes and behaviour which should then lead to, 

what we would call, a Human Factors culture in the 

organisation. We are currently looking at a future 

programme for all grades similar to the CRM courses 

for flight crew. 

On Quality Lapse investigations, we are working 

with Boeing on a process and database known as 

MEDA, Maintenance Error Decision Aid. This is 

basically a process to actually discover the root 

causes, and take away some of the blame from the 

individuals involved in quality lapses. The process is 

currently paper-based, however, the plan is to 

develop a world-wide computer database of 

maintenance error involving many airlines . We see 

this as a worthwhile project with which to be 

involved. 

And finally MESH (Managing Engineering Safety 

Heahh), which we refer to as a proactive Human 

Factors tool as opposed to MEDA, which we term 

reactive because this is event-driven, i.e., an incident 

needs to have occurred before you can investigate it. 

MESH was conceived by Professor Jim Reason and 

Appendix 

his team from the University of Manchester. The 

system is an anonymous human factors computerbased evaluation or, if you prefer, a survey of staffs 

perceptions to Human Factors in their area. It is 

important to state that MESH is not a reporting tool 

and it is not iptended to replace our current reporting 

processes. MESH is a proactive safety management 

tool, the results from which can be used to identify 

and prioritise the human factors issues in a particular 

area. it's not, however,just a management tool. With 

the Total Quality culture that we have at BA 

Engineering, we see it very much as everyone getting 

involved and resolving their own local issues. Of 

course, there will be large, long term issues that 

affect the whole organisation which cannot be 

addressed locally. We must therefore also analyse the 

results from an organisational point of view. 

MESH looks at two types of factors, which we term 

Local and Organisational Factors. Local Factors, or 

LFs, are those factors that affect staff at the sharp end 

of the operation. These factors are actually specific to 

the area and have been selected by staff in that work 

area; typical examples of LFs are things like Parts 

Availability, Fatigue, Tools and Equipment, 

Knowledge, Skills and Experience, etc. 

Organisational Factors, or OFs, are constant 

throughout the engineering organisation, and 

although these affect everyone in the organisation, 

they are more visible to management. Organisational 

Factors are therefore rated by management grades; 

typical examples of OFs are training and selection, 

people management, provisions of tools and 

equipment, building and equipment maintenance, etc. 

• 

• Based on theory conceived by Prof. Jim Reason (Manchester University, England) 

• Anonymous Human Factors computer based evaluation 

• Proactive survey of staff's perceptions of Human Factors in their workplace 

• Results used to and 

MESH 

• Local and Orginisational Factors 

• Local Factors (LFs): 

• Affect those at the sharp end. 

• Specific to an area and selected by users. 

• Eg., Parts Availablility and Fatigue. 

• Organisational Factors (OFs): 

• Affect Everyone, but more visible to management. 

• Standard questions throughout organisation. 

• E ., Trainin and Selection and Pee le Mana ement. 

256 
Appendix 

I would like to now give a demonstration of the 

MESH software. Here's a scenario. One week ago, a 

licensed avionics engineer waited for seven hours for 

a spare part. The aircraft departed on time, but the 

installation of the part and the subsequent function 

tests needed to be done extremely quickly. The 

licensed aircraft engineer is rostered to do his MESH 

input today: You can see MESH has been designed to 

be used in the Windows environment. It is extremely 

user friendly. Not all of our technicians and aircraft 

engineers are actually PC literate, so we wanted to 

make it as simple as possible, this was an important 

specification for the system. Using the mouse, the 

licensed aircraft engineer would click on the MESH 

icon. The first question I'm asked is my Grade. Okay, 

I'm a Licensed Aircraft Engineer. All the grades in 

the list are particular to that area, so as I said before 

MESH is defined and designed for that area. The 

next question I am asked is my trade - in my case 

Avionics. The final personal question is what shift 

am I on- let's say A shift. Again the shifts are all 

applicable to this one area. I've used it though, I 

woyld like some more questions. The next screen 

also ask several questions b'\t this time they are 

related to the task. The system first asks you to think 

of a task in the last week that did not go to plan and 

which may have caused frustration or put pressure on 

the individual to complete. It then asks for the Date -

last week. ATA chapter- instruments. Airline 

operator- BA in this case. Source of work- let's say 

it was an AMS item. 

Now, because I have told the system that I am an 

aircraft engineer, the Local Factors evaluation 

appears. If I had said that I was a line manager then 

the Organisational Factors would have appeared. The 

first factor I am asked to rate is Knowledge, Skills 

and Experience. But what does this mean to me? By 

clicking on the example box next to the factor gives 

typical examples. These examples have been again 

derived by staff in that area and are therefore 

normally pertinent to typical tasks performed in that 

area. In this case Knowledge Skills and Experience 

wasn't a problem, so I click on No Problem. The next 

factors Morale, Fatigue, Time of Day, Manuals and 

Procedures and Tools and Equipment were not a 

problem in my example, so again I click on No 

Problem. Parts and Spares - well, this was my 

problem and by clicking on the example box, the 

system prompts me to think whether it was a quality, 

location, availability or delivery problem. In my case 

it was the time taken for delivery of the part. So for 

Parts I would rate this as quite a problem. The 

257 

Pluue V Report 

remaining factors were also not a problem in my case 

except maybe operational pressure which I will rate 

as a Nuisance. So the next screen that appears is the 

Anonymous Comment facility. The initial version of 

MESH didn't have this. This facility makes it easier 

to determine the real issues behind the MESH 

Profiles when these are analysed at a later date. In 

our case, you would normally put in something like -

waited for seven hours for a NA V receiver, Pt No. 

123ABC from SPUD stores. That's probably BA 

speak, but you would try and be specific as you can 

and the information is then stored in the data base 

along with all the other comments from that area. 

Now, it's important that feedback is given to the user, 

so we have the facility here that not only shows the 

Local Factors Profile for the individual's input but 

also for the whole area over the last month. I can also 

look at the Organisational Factor Profile for the 

month. I hope I have shown you how easy and quick 

the system is to use. There is, however, a Help 

facility for those who have forgotten how to use the 

system. The Help screens also give information 

regarding human error theory, again this has been 

based Prof. Reason's work. 

I am now going to click on the MESH Analysis icon. 

This tool is used by whoever is responsible for 

looking at the profiles and identifying the issues 

behind them. We plan to modify this part of the 

software over the next few months, however, I will 

demonstrate the software that we have been using 

over the last year. The Analysis tool gives us the 

ability to sort the c9llected MESH data in order that 

we may compare different time periods, work areas, 

shifts, trades etc. We can then use these results for 

trending purposes. Okay, let me exit the analysis tool 

and I'd like to now talk about the actual MESH trial 

at BA Engineering. 

MESH is currently installed in five work areas; two 

hangars, one workshop and two engineering offices. 

These are very different environments and this is one 

of the reasons for allowing the local area to define its 

own requirements for the system; what works in 

Hangar X probably will not work in Office Y. We 

have two MESH Champions per work area who train 

staff in the area on how to use the system. They also 

analyse MESH data from their area using the MESH 

analysis tool which I have just demonstrated. 

Existing computer hardware has been utilised in all 

these areas. More and more PCs are appearing in the 

hangars and other areas and therefore there shouldn't 
Phase V Report 

be any hardware costs associated with installing 

MESH. We have a monthly MESH Progress meeting 

to coordinate the trial, which is attended by all the 

champions, myself, the human factors consultant and 

my General Manager. 

The local process for managing MESH data is left to 

the local areas to develop, although, I do give a 

number\lf guidelines based on our experience to 

date; such as target number of MESH inputs per 

month. How they achieve this is very much up to 

them, because as I said before, all the environments 

are different and they know best how their staff need 

to be encouraged to use the system. For hangar areas, 

we found it best to go for a twenty percent sample. 

We would, therefore, ask a random sample of twenty 

percent of the workforce to enter data once a month 

for three months, and then they wouldn't have to 

enter data for another year. Then, there would be 

another group of staff who would do it for three 

months, and so it goes on. In areas where there isn't 

large numbers of staff employed, we normally ask 

everyone to input once a month in order to achieve a 

significant sample. 

I mentioned before that we don't see this as just a 

management tool and we actively encourage local 

TQ working groups to use MESH results in order to 

resolve Human Factors issues in their area. In some 

areas, the monthly MESH profiles are on the agenda 

of existing local TQ forums. MESH profiles make a 

very good agenda item, identifying and prioritising 

issues to talk about and, most importantly, resolve. I 

think it is important that we don't identify hundreds 

of issues and never get around to resolving any of 

them. MESH allows us to prioritise our efforts,-

concentrating on resolving maybe two or three issues 

identified as contributing to the high bar of the 

MESH profile. Feedback to MESH users is very, 

very important in order to motivate staff to use the 

• All users individually trained 

Appendix 

system. Actions taken and resolutions must regularly 

fed back and even if you can't take any action, it is 

important to say why not. If used correctly, MESH 

should help communication within an area. For 

example, MESH has identified a deep rooted 

problem which will take the organisation two years to 

resolve. By feeding back to staff information on what 

the organisation is planning to do about the problem 

and when can influence the attitude of staff and have 

a positive influence on the MESH profiles. The 

method of feedback varies according to the area, but 

is genera11y given in either newsletters, notice boards 

or the human factors magazine. We are considering 

giving feedback electronically, by having another 

icon in the MESH box saying, for example, MESH 

Feedback, so that a monthly update on MESH issues 

can be accessed by staff using the MESH PC. 

Another important use of MESH Profiles is looking 

at the effectiveness of past resolutions. Profiles can 

be used to support business cases to show not only 

that a problem exists, but also we now have a way of 

monitoring in the future the effectiveness of the 

resolution proposed in the business case. 

Now, I have stated that MESH is basically a local 

tool, but obviously there are going to be issues that 

affect the whole organisation. There are going to be 

times when Hangar A and Hangar B have the same 

problems and it would be a waste of their effort if 

they tried to resolve these individually. Therefore, we 

use our existing "quality forums" to monitor MESH 

profiles from an organisational point of view. In the 

future we hope to combine local profiles so that the 

highest forum in our organisation, the EQMRB 

(Engineering Quality Maintenance Review Board), 

can actually see what each business unit (or 

department) looks like. We think that such 

information will be very useful high level decisions. 

So, what do we have planned for MESH in the 

• Monthly MESH Input Roster 

• 20% sample of hangar staff 

• All workshop and office staff 

• MESH Results 

• Analysed by Champions in TQ groups 

• Issues identified/prioritized from MESH Profiles and 

Comments 

• Actions fed back to users - newsletters, noticeboards 

• 

258 
Appendix 

future? Well, it's not the best time to give this 

presentation at this moment because I'm actually in 

the process of analysing results from the trial over the 

last six months, and will be presenting my fmdings to 

our Human Factors Steering Group next month. As I 

stated before we are looking at a number of software 

modifications especially to the Analysis tool. 

We are also looking at networking the system, 

because at the moment we to collect MESH data 

from each area on disk and combine it on my PC. 

Obviously, that's not an efficient way of doing it. 

Also once MEDA (Human Factors Investigation tool) 

is producing results, I would like to see both systems 

259 

PluJse V Report 

working together as an integrated Human Factors 

tool, using proactive and reactive data. 

I think that it's clear that we've got to change some 

of the traditional engineering attitudes tu human 

error. However, I think there is one engineering 

attitude or tradition that I would like to keep and that 

is - we are pretty good as a profession at turning 

theory into practice. I see that as my job, turning 

Human Factors theory into practical applications. I 

believe that the Human Factors Guide that you have 

been working on over the past few years will be 

useful by giving practical advice on a misunderstood 

subject and I believe all our efforts will pay 

dividends in aircraft safety, quality and costs in the 

future. Thank you. 
Appendix Phase V Report 

OVERVIEW OF FAA NDI RESEARCH 

AT SANDIA LABS 

Patric/c Walter, PH.D. 

November 9, 1994 

Has anyone told you what Sandia means? In 

Spanish, it translates as watermelon. The 

mountain range to the north is the Sandias. The 

mountain range to the south is the Monsontos, which 

translates as apple. When the sun's setting at the right 

position, the mountains tum a pinkish hue for about 

thirty seconds every night. The mountains are full of 

deer, mountain lions, coyote, and bear. 

Just to orient you, Sandia National Labs is a 

Department of Energy Laboratory. Sandia's history 

has been one of designing nuclear weapons, doing 

underground nuclear testing, and maintaining the 

nuclear weapons stockpile. Actually, we're fairly 

busy today. One of the things we're doing is helping 

the Russians dismantle their stockpile of nuclear 

weapons. There's a segment of the program at our 

laboratory that works for other govermnent agencies. 

Sandia's big customer is the Department of Defense. 

The FAA came to Sandia after Congress passed the 

1988 Aviation Safety Act to ask if we would get 

involved with the inspection portion of their Aging 

Aircraft Program. We did. 

Last week, we hosted a meeting both of the AT A 

Inspection Network and its NDT Forum with about 

four hundred people attending, including fifty-one 

airlines from twenty-seven countries. What I'm going 

to do today is to go through the same presentation I 

gave at the forum a week ago today. There are a lot of 

human factors elements that tie into your programs 

aod other things I think will interest you. I am going 

to deviate from my previous presentation only in style 

to point out things that I believe relate to your 

interests. 

Our program's objective is to provide the FAA with a 

tool for independent and quantitative performance 

aod cost assessments of aircraft structural inspection 

techniques. If you want to introduce new inspection 

technologies to airlines today, economics is a big part 

of the process. I will point where I think we are 

saving the aviation industrY some dollars, but the 

program's overall goal is to encourage NDI technique 

261 

development and validation. We have test beds in a 

hangar that you are going to visit during the tour this 

afternoon. 

I am not going to give you a lot of verbiage; I am 

going to show you some pictures of our work 

activities. We were funded in September 1991, and 

for our first seventeen months we were essentially a 

facility organization. We acquired a hangar, probably 

a small hangar by most of your standards. Our hangar 

has 28,000 square feet. When we got it, the roof was 

caving in and the floor would not hold a transport 

aircraft. We leased the hangar, then rehabbed it and 

acquired test beds. We installed support equipment, 

scaffolding, and a catwalk so that we could walk 

along the top of our transport afi>lane. Since the FAA 

is responsible for the inspection program under the 

National Aging Aircraft Program, we worked with 

them to define some program infrastructure so that 

research could be focused towards Albuquerque. We 

wanted essentially to function as the hand-off to the 

aviation industry, so we defined items such as a 

validation process and procedures. In February 1993, 

we opened our facility doors. Soon after, researchers 

wanted to come arijl start working on our test beds. 

People came from industry and from universities with 

graduate students, so we had to next implement safety 

procedures. 

Researchers were making a lot of samples of flaws 

typically occurring in aircraft structure. We pooled all 

the information together and archived it. IfF AA 

researchers were looking for specific flaw samples, 

we could tell them where samples were available. We 

formalized an FAA Sample Defect Library which is 

published and routed as a DOT document. We also 

developed a database. All experimenters acquire data, 

and the common result of most advanced inspection 

techniques is an image. Visoal inspection is not about 

just a flashlight and mirror; in its broadest sense, it 

can encompass someone looking at an image of some 

NDT-determined flaw on a screen. When 

experimenters left our facility, they provided floppy 
Phase V Report 

disks containing inspection results. We archived these 

floppies into a database. 

We began our work with reliability experiments. I 

will subsequently tell you about work we are doing in 

reliability and then touch on human factors. We 

integrate'd economics into our program early on, and 

an economist works with us. We are initiating a 

program to assess results of experimentation in our 

facility to date. Thus far, when an experimenter has 

come to our facility, it has been a somewhat informal 

process. We are now developing rigorous protocols 

and acquiring blind samples we need to fill our 

Sample Defect Library. The core of any program has 

to be blind samples; blind implies unknown to the 

experimenters. We also work for the FAA in areas 

such as information systems and structural activities. 

We routinely interact with Boeing and Douglas. If 

Appendix 

you are here from an airline, chances are, I have been 

to your facility. The lab is going to get formal 

industry feedback as to results of our structured 

experiments. 

When I make a presentation at one of these meetings, 

I am typically asked: "Where's the beef?" What I 

want to do is to show you some things we've 

completed to date. In November 1992, we got our 

first test specimen for the Sample Defect Library 

(Figure A.l ). It is Sample I 00 and is from a Boeing 

737. It's from the same series as the Aloha, which 

means it has the cold bond joints and that terminating 

actions have not been performed on it. It's relatively 

low cycle, 46,000 cycles, 38,000 hours. We acquired 

it from Miranda, Arizona, and it was flown here for 

its last flight. 

+ 8737 Testbed Aircraft Acquired/ 

Configured - Nov. 1992 

Figure A.1 

262 
Appendix Phase V Report 

+ Hangar Acquired/Renovation Completed 

Jan.1993 

.«!. I 

Figure A.2 

Since you're going to be visiting our hangar later this 

afternoon, I can share a little humor (Figure A.2). 

When we acquired the Boeing airplane, there was still 

a lot of emotion from the Aloha accident. You can 

imagine how Boeing appreciated that there was a 

hangar in the airport for Aging Aircraft, and there was 

a Boeing airplane. Soon after, we brought in some 

Douglas DC-9 fuselage sections because there are 

some known problems around the bulkhead and with 

cracking around the windows. This also created a 

balance among manufacturers. 

At the 8th FAA/AAM Meeting on Human Factors 

Issues in Aviation Maintenance last year, Floyd 

Spencer presented the results of an eddy current 

reliability experiment to you, so I am going to 

mention it only briefly (Figure A.3, next page). When 

you go to the hangar, you are going to see the panels. 

Floyd is going to be standing by them in case you 

have any questions. One question that came up after 

Aloha was, "How well is inspection being done in 

airline maintenance facilities?" The way this system 

works is that fatigue modeling of the airframe is 

performed. Predictive crack row models are used, and 

263 

assigned inspection intervals must be frequent enough 

for one to find a crack before it gets to a critical 

length. On the Aloha plane's lap splices, cracks 

emanated from under the rivets, and so the question 

came up, "How well is eddy current lap splice 

inspection being done in aviation maintenance 

facilities today?" No one had ever quantified an 

answer before. The U.S. Air Force, through its 'have 

cracks, will travel' program, made a round robin in 

DOD facilities a number of years ago. Typically, 

Boeing and Douglas develops a predictive probability 

of detection (POD) modeL They then take the ninety 

percent POD and shift it to fifty percent to include 

degradation they assume will incur in going from 

their facility to American Airlines, Tramco, Dalfort, 

United or whomever. They degrade their curve 

conservatively to account for differences between 

their results and those they believe will occur in 

maintenance facilities. 

However, the question is: "How good is industry 

doing?: We have just completed a $1.5 million 

experiment for the FAA during which we visited nine 

maintenance facilities. When we called the facilities, 
Phase V Report Appendix 

+ Eddy Current Inspection Reliabi.lity 

Experiment Completed -·March 1994 

Figure A.3 

they really welcomed us. The concept of Sandia Labs 

being neutral on a three-legged stool-airline, OEM, 

FAA-appears to have been accepted. People in the 

field clearly distinguish among us, the FAA, the 

OEMs, and the airlines. There have been three DOT 

volumes published supporting this experiment. The 

last, which is full of human factors information, is at 

the FAA Tech Center right now awaiting publication. 

264 

The experimental parameters in this experiment 

include a Boeing 737 structure with enough 

inspection sites to keep an inspector working six to 

eight hours, day shifts, night shifts, crack angles, 

painted versus unpainted surfaces, accessibility, etc. 

We went into a facility, set up, followed developed 

protocols, and aslted the inspector to pull a job card 

and go to work. 
I 

I 

I 

I 

I 

Appendix 

Within the program, we have addressed some 

structural issues, and we have made some structural 

assessment on our B-737 airframe. I can tie in some 

human factors observations. The scanners make an XY scan; depending on the sensing technique, they 

paint an image of an aircraft's structure and its 

condition at the location being inspected (Figure 

A.4). Some people came in wearing backpacks and 

with vi;ual displays. Some would set up their 

equipment and after ten minutes, it would be on the 

floor, and their experiment was over. Other people 

were successful in adapting commercially available 

equipment to airframe structure. Some of it worked 

remarkably well. We did time trials, assessed the 

efficacy of inspection results, etc. Our results are 

available as a DOT report. 

As a byproduct of some of our work, the Coast Guard 

has asked to join our program. Their HU25A aircraft 

is a Falcon 20 derivative (Figure A.S, next page). 

This particular plane did a lot of reconnaissance over 

Haiti. The Coast Guard, like everybody else, is 

running short of money, and they cannot afford to 

maintain their entire fleet. They have park nine of 

Phase V Report 

their forty-one planes, so they donated one to us. The 

plane is made by Dessault. The American 

representative is Falcon Jet. There's an SSIP 

(Supplemental Structural inspection Program) 

coming on this aircraft, and SSIP's draft indicates that 

it will cost the Coast Guard a large sum of money. 

Dessault has met with us, and we think there is real 

potential for knocking the cost down by integrating 

some advanced inspection techniques. 

As I mentioned, we defined the deliverable from our 

validation process to be an inspection technique's 

reliability and cost-effectiveness. When we went 

around the country to various maintenance facilities 

with our eddy current inspection experiment, people 

used their own inspection equipment in-house. For 

example, when we went to United, we knocked on the 

door and asked if we could come in and take hangar 

space and lise your inspectors on day shifts, night 

shifts, etc. We said the we would need forklifts and 

that it really would be an inconvenience. To their 

credit, they said that it was okay, that they wanted us 

there, and that they would give us a week's worth of 

time, as we needed it. And then we said, "Well, listen, 

• Manual/Semi-Automated/Automated 

Scanner Assessment Completed 

April1994 

• 

Figure A.4 

265 
Phase V Report Appendix 

• HU25A Guardian Aircraft Donated to 

FAA Program By US Coast Guard 

June 1994 

WFO Agreement With Coast Gu.ard 

(Drawings. Records, Support) 

Figure A.S 

let us ask you something else. Here's a new piece of 

equipment, the MOL It's designed to do the same 

type of inspection work as the eddy current 

equipment, but it works a little differently. Instead of 

looking at a meter on impedance plane, it gives a 

visual image of the magnetic field. It might have 

some advantages in terms of reliability or costeffectiveness. If you want, we will provide a one-day 

training program. Then you can try it for another 

week, but then you will have to repeat the whole 

experiment, go through this whole process again, so 

now it is going to take two weeks of your time." 

Without exception United and everyone else said, 

"We want to give you another week's wonh of our 

time." We put together a training program. They 

repeated the experiments, and we acquired field 

reliability for this instrument to compare against the 

existing eddy current technique. We then worked with 

the Transponation Center at Nonhwestem University 

to perform a cost-benefit analysis. We presented it at 

the A TA meeting, and a repon is coming out on this 

now. The repon shows that if you can defer paint 

stripping, you can come up with some non-trivial 

economic advantages. 

266 

Another question that came up in using the B-737 as 

a test bed was, "What was its condition?" We 

transformed our facility into a maintenance facility 

and performed a heavy check on that airplane. Our 

procedure was to pick out structural elements of the 

heavy check that we thought we needed to 

incorporate into eur assessment, to initiate a cost 

estimate, and to identify the SIDS and 

supplementation inspections we wanted to 

incorporate. Then, we contacted some ex-airline 

employees who comprised a consulting company-exUnited, FedEx, and Continental folks-and brought 

them in The group included a retired foreman who 

had thiny years' experience. This company was Tech 

Ops. We acquired visual inspectors from 

Albuquerque Airpon; they had twenty to twenty-five 

years' experience and had bounced around in the 

industry, spending their career working on transpon 

aircraft. Then, we went down to Miami and hired a 

commercial NOT firm, Miami NDT, who contracts to 

third-pany maintenance facilities and has a large 

presence in NDT inspections in the Miami area. We 

brought all these people together, transformed one of 

our rooms into a maintenance planning room, put 

together a hundred and thiny-five job cards, and 
Appendix 

performed what we call a baseline of that airplane. 

This means that we inspected as well as can be done 

with the tools present in civil aviation today. This 

means a lot of visual inspection and a little bit of 

fairly simple NDT. Tech Ops held the "baton" during 

the entire process. 

Over the past year, fifty groups have worked in the 

facility. When you go over today, there will be two 

groups working, and those of you that have been in 

the inspection industry will be kind of interested. 

NASA Langley has four people here doing thermal 

wave imaging by putting heat pulses into airplanes 

and looking at temperature differences on the surface 

to detect corrosion and disbonds. They're also doing 

some ultrasonic work. There is also a commercial 

firm, Holographies, Inc., who are putting some 

vibratory inputs into our aircraft. They are able to use 

laser detection for things like subsurface structural 

damage, cracked frames, stringers, clips missing, etc. 

They want to rehearse this technique and ultimately 

give the FAA and industry the opportunity to assess 

it. 

The FAA, in setting up opr center, also funded a 

number of other locations across the country. They 

put up two principal centers, the one here in 

Albuquerque and a university consortium at Iowa 

State University. The other researchers undertake 

basic and applied research, and we have the role of 

technology transfer to industry, i.e., the industry 

hand-off. Based on industry feedback and FAA 

review, nine tasks at Iowa State's consortium look 

like they have the potential to contribute to the 

aviation industry. We're working with them to make 

working prototypes that we can demonstrate to 

industry in our center. Industry then has the 

opportunity to express its interest and state whether it 

wants this research to proceed. 

I don't know how many of you have heard about 

widespread fatigue damage, but it is a topic a number 

of folks are actively discussing right now. It involves 

small generalized cracking in airframes and has the 

potential to lower an airframe's capability to be 

damage-tolerant. I'm not here to debate that issue. 

TOGO (Technical Oversight Group On Aging 

Aircraft) an FAA/industry review group, has this 

issue as one of their concerns. We have been asked to 

be able better to assess technologies for detecting 

small cracks in airframes. The typical crack, at least 

around a rivet, an inspector looks for now is 

nominally a tenth of an inch, but interest has been 

expressed in looking at cracks in the forty and fifty 

267 

Phose V Repon 

mil range. To reach that goal, we need both blind 

specimens and industry and FAA researchers coming 

in to have their technologies assessed. Every task we 

are working on has an industry linkage. Concerning 

subsurface flaws, i.e., cracking in second and third 

layer structures, a requirement again exists for blind 

samples. We are working cooperatively with Boeing 

and Douglas to design those samples, and Northrop is 

also a player. Northrop represents the military side, 

but they also do R&D on improved inspection 

techniques. It looks like their technology will well 

lend itself to the civilian side. 

As you are aware, to inspect commercial transports, 

you open them up and go in with a flashlight and a 

mirror to visually inspect for corrosion. The Air 

Force is actively pursuing wide area and local 

nondestructive inspection techniques to detect 

corrosion. It is tremendous what service lives are 

predicted for Air Force planes. For example, the KC 

135 has to fly until the year 2030. Their cycle time is 

low, and the fatigue problem is not accentuated if you 

can prevent corrosion. The Air Force places a lot of 

emphasis on corrosion detection. We are leveraged 

with them for cooperation between our FAA program 

and their Air Force program. 

Visual experimentation is another activity at our 

facility. We have continuous requests to assess how 

well visual experimentation occurs in industry today. 

Colin Drury, who spoke earlier, is working on this 

task with us. On the earlier eddy current reliability 

experiment that I talked about, we were linked to the 

OAM through Bob Blanchard over at CAMI. We 

have tried to keep an involvement with this audience. 

Concerning the'visual experimentation, the ATA (Air 

Transport Association) has appointed an advisory 

group, including five airline people. We're planning a 

program over the next four to six months so that we 

should have between twenty-four and thirty-six airline 

inspectors through our facility. We are putting 

together a standard visual inspector's tool box to 

nomtalize the inspectors' tools, and we will complete 

experimentation on the transport aircraft before 

moving into the commuter area over the next year. 

We appreciate the AT A's support. 

Halon bottles are another interesting topic. The AT A 

has asked us to work in this area. American recently 

estimated a $10 million dollar savings for the industry 

in the next year if we are successful. Halon bottles in 

aircraft must be hydro-tested: there is a DOT 

regulation about it. The bottles are collected and sent 

to one of three centralized locations. The halon is 
Phase V Report 

captured, and the bottles are hydro-tested, reloaded, 

shipped back to the airline, and reinstalled. We think 

it's going to be-l don't want to say easy-certainly not 

difficult to heat the bottle by about forty degrees and 

to avoid disassembly and shipping by looking at 

acoustic emissions. 

A DC-9 wingbox inspection is another work activity. 

When I first became involved with this program, I 

went to an inspection facility and worked with 

inspectors for several days, crawling through the 

airplanes. The head of maintenance told me that if I 

wanted to serve the industry, I needed to assume that 

the average inspector has a minimum of a ninth-grade 

education and wants to do a good job. We have to 

design tools for the inspector. I believe another 

variable _is if one can save someone enough money, 

some constraints can lessen. In the case of the DC-9, 

Northwest Airlines expressed interest. In the DC-9's 

wingbox, there is a potential problem due to stress, 

corrosion, and cracking. The inspection procedure 

involves an extensive teardown. If the wingbox T-cap 

is in good shape, the unit can be reassembled. By 

working cooperatively with the technique's inventor, 

Northwestern University, we think we have an 

ultrasonic technique that can potentially replace a 

teardown with a forty-hour inspection, a savings of 

hundreds of hours per airplane. Since Northwest has a 

fleet of something like a 109 DC-9s, potential savings 

are in the millions of dollars. We hope soon to have a 

service bulletin revision from Douglas, who is fully 

cooperating in this project. 

The Air Force uses boron epoxy bonded repairs on a 

lot of their planes. Boron epoxy has some unique 

properties. It is directional in nature and has very high 

stiffness. We were interested in supporting the FAA 

by doing some work with boron epoxy, and the FAA 

supported a workshop at our facility. There were 

concerns about who would be the initial customer. 

Delta volunteered, and Lockheed expressed interest 

268 

Appendix 

in issuing a sc!rvice bulletin concerning the use of 

boron epoxy as a structural reinforcement. We now 

have a joint program with Delta and Lockheed on the 

L-1 011. Our parking lot at the hangar has some big 

slabs of L-1 011 we have cut out and hauled in. We 

are also working cooperatively with the Atlanta ACO. 

Textron is the boron epoxy supplier, and Warner 

Robbins AFB is also participating. 

Most of the program's focus to date has been on 

transports. Commuters can certainly benefit from this 

program, especially since some commuters contain 

inaccessible structure. We are going to bring a 

commuter airplane into our hangar in the next six 

months. By the time we finish the visual inspection on 

the transport, we will be ready to move to the 

commuter and use it as a test bed for other activities. 

Candidly, when we started this program a few years 

ago, we thought we had some tools to contribute, but 

the nature of that contribution was not totally clear. 

Now, we feel like we are delivering products to the 

aviation industry. I do not say that out of arrogance, 

but because the industry is indicating that we are 

contributing. The FAA is negotiating with us to 

expand our program and to acquire more access to 

other areas of Sandia. 

We feel we are coordinating well with Douglas and 

Boeing. As time goes by, we hope to work more with 

GE and Pratt to place more emphasis on engines in 

our program. We feel that we work well with 

individual airlines. We routinely have airline 

personnel in our facility, and we encourage and 

appreciate that. What we would like to do is to set up 

a more structured interface with the ATA and are 

talking with them about how we might develop an 

optimal interface. We need to get more involved in 

commuter issues; our commuter aircraft is coming in. 

As I told participants at the ATA Forum, we're trying 

to listen to everybody, and it is a big industry. 
Appendix PluJse V Report 

9th Federal Aviation Administration Meeting on 

Human Factors Issues in Aviation Maintenance 

List of Attendees 

Robert (Bob) F. Aaron, Jr. 

Chairman, Maintenance Committee 

NW A - Airline Pilots Association 

I05 S. Joyce St 

Golden, CO 8040 I 

Leon F. Alberdi 

Human Factors Consultant 

I 06 Refonna St. Coyoacan 

Mexico City, Mexico 04000 

Lawrence (fony) Antoine 

Technical Training Coordinator 

Delta Air Lines 

Dept.590 

I500 Aviation Blvd 

Atlanta, GA 30320 

Ted Baker, Ph.D. (SPEAKER) 

President 

ShiftWork Systems, Inc. 

One Kendall Square 

Building 200, 4th Floor 

Cambridge, MA 02139 

Steven Berry 

Manager of Technical Support 

Westair Airlines 

5588 Air Tertninal Drive 

Fresno, CA 93727 

Clarence Braddock 

Director of Maintenance 

Piedmont Airlines 

5443 Airport Terminal Road 

Salisbury, MD 2I801 

David Buchanan 

Inspector 

IAMINWA 

2740 Flag Ave., N. 

New Hope, MN 55427 

James Burnette, Ph.D. CPE 

(SPEAKER) 

29IO Oak Ridge Rd 

Oak Ridge, NC 273I 0 

Jerry Burrow 

269 

Manager, Quality Audit 

Continental Airlines 

P.O. Box I2788 

Houston, TX 77217 

Alexander A. Chironis 

Inspector - Aircraft 

Continental Airlines 

10475 Dona Ave. 

West Los Angeles, CA 90064 

Joseph C. Cotter 

Quality Control Supervisor 

United Parcel Service 

80 I Grade Ln - Hangar 

Louisville, KY 402I3 

Daniel Condray 

Maintenance Standardization Manager 

Airbus Industrie 

I Rond Point Maurice Bellonte 

3I707 Blagnac Cedex 

France 

James T. Crouse 

Artomey 

Speise, Krause & Madole 

300 Convent, Ste 2600 

San Antonio, TX 78205 

Bruce R. David 

Manager - Airframe Production 

United Airlines 

1100 Airport Blvd. 

Oakland, CA 94621 

Alan Diehl 

Sup. Engin. Psych. 

U.S. Air Force Operation Test & Evaluation Center 

8500 Gibson Blvd SE 

Kirkland AFB, NM 87117-5558 

Dutch Drescher 

Fit. Safety 

IAM/NWA 

215 E. 98th St. 

Bloomington, MN 55420 
Phase V Report 

David Driscoll 

Manager - Regulatory Compliance 

USAir, Inc. 

R.I.D.C. Park, Building 4 

173 Industry Drive 

Pittsburgh, PA 15275 

Dr. Colin Drury (SPEAKER) 

Professor 

Dept. of Industrial Engineering 

State University of New York at Buffalo 

342 Bell Hall 

Buffalo, NY 14260-2050 

Steve H. Eberhardt 

Director, Quality Assurance & Training 

Northwest Airlines, Inc. 

I 000 Inner Loop Rd 

Atlanta, GA 30337-6072 

Vernon S. Ellingstad 

Deputy Director, Research & Engineering 

National Transportation Safety Board 

490 L'Enfant Plaza E., SW 

Washington, DC 20594 

Mr. Koichi Emori 

All Nippon Airways Co LTD 

1-6-6, Haneda Airport Ota-ku 

Tokyo 144, Japan 

Mica Endsley 

Texas Tech University 

Dept. of Industrial Engineering 

Lubbock, 1J( 79409 

Raynard M. Fenster, Ph.D. 

Sender Safety Scientist 

UNLVIHRC 

4505 S. Maryland Pkwy 

Las Vegas, NV 89154-4009 

David V. Finch 

Consultant 

12 Rectory Close 

Windsor, Berks, SL4 5ER 

England 

Wrensey C. Gill 

Manager, Human Factors Division 

FAA/Office of Integrated Safety Analysis 

400 7th Street SW 

Washington, DC 20590 

270 

Raymond P. Goldsby 

Director 

HKS&A,Inc. 

503 Seaport Court, Suite 105 

Redwood City, CA 94063 

Gary Goodman 

Director of Quality Assurance 

SkyWest Airlines, Inc. 

444 South River Rd 

St. George, UT 84770 

Willard Gregory 

Appendix 

Manager, Reliability and Maintainability/ Human 

Factors Engineering 

GE Aircraft Engines 

One Neumann Way MD T-25 

Cincinnati, OH 45215-6301 

Franklin Haag 

Maintenance Training Director 

Airbus Service Company, Inc. 

Training Center 

P.O. Box 660037 

Miami Springs, FL 33266-0037 

David J. Hall 

Deputy Regional Manager 

Civil Aviation Authority 

Safety Regulation Group, Sipson House, Sipson Rd. 

Sipson, West Drayton 

Middlesex, England 

Lon Haney 

Human Factors Psychologist 

Idaho National Engineering Laboratory (INEL) 

P.O. Box 1625 ' 

Idaho Falls, Idaho 83415 

Max Henderson 

Associate Professor 

Embry-Riddle Aeronautical University 

600 South Clyde Morris Blvd. 

Daytona Beach, FL 32114-3900 

M.C. Hutchinson 

Engineer 

Boeing 

MS9U-EA 

P.O. Box 3707 

Seattle, W A 98124 
Appendix 

Robert C. Johnson 

Chief, Operational Logistics Branch 

Air Force Armstrong Lab, Logistic Research Division 

2698 G Street 

Wright-Patterson AFB, OH 45433-7604 

William Johnson, Ph.D. (SPEAKER) 

Galaxy sCientific Corporation 

2310 Parklake Drive, Suite 325 

Atlanta, GA 30345 

Dr. Jon Jordan 

Federal Aviation Administration 

800 Independence Avenue, SW 

Washington, DC 20591 

Barbara Kanki 

Research Psychologist 

NASA Ames Research Center 

MS262-4 

Moffett Field, CA 94035 

Pete Kelly 

America West Airlines 

4000 E. Sky Harbor Blvd 

Phoenix, AZ 85034 

David F. King 

Principal Inspector of Air Accidents 

Air Accidents Investigation Branch 

Dept ofTranportation 

BldgT75 

ORA Farnborough 

Hampshire GUI4 6TD 

England 

David C. Kline 

Manager, Qualtiy ControVChief Inspector 

American Trans Air 

7337 W. Washington St 

Indianapolis, IN 46251 

271 

Robert Lapp 

Sky West Airlines 

444 South River Rd 

St. George, UT 84770 

Kelly M. Lee 

Human Factors Engineer 

Computer Sciences Corporation 

120 E. Jones Rd, 420 FLTSJENH 

Edwards AFB, CA 93535 

Fred Liddell 

Phose V Repon 

Co-Chairman; lAM/FAA Conformance CommitteeTWA 

lAM & A W Air Transport Local 1650 

P.O. Box 9067 

Vivion Rd & Cliffview Dr 

Riverside, MO 64168 

A. C. MacArthur 

Director Aircraft Maintenance 

Monarch Aircraft Engineering L TO 

Luton Airport 

Luton, Beds. LU2 9LX 

England 

Michael Maddox, Ph.D. (SPEAKER) 

Sisyphus Associates 

P.O.Box911 

Madison, NC 27025 

ArupK. Maji 

Associate Professor 

University of New Mexico 

Dept. of Civil EnJiineering 

Albuquerque, NM 87131 

John Mapel 

Program Analyst 

Federal Aviation Administration 

Office of Aviation Safety 

400 7th Street SW, Room 2227 

Washington, DC 20590 
Phase V Report 

Randy Marlar 

Vice President, Maintenance & Engineering 

American Trans Air, Inc. 

7661 N. Perimeter Road 

Indianapolis, IN 46251 

James A. Mcintyre 

Chairman, Human Factors Working Group 

Internatio~al Society of Air Safety Investigators 

(ISASI) 

49 SouthRd 

White Picket Farm 

Deerfield, NH 03037-1713 

Carlton E. Melton, Ph.D. 

2710 Walnut Rd 

Norman, OK 73072 

Dean B. Minett 

Asst. AirpoRt Mgr., Director of Maintenance FBO 

Andrews University Airpark 

Griggs Drive 

Berrien Springs, Michigan 49104-0930 

Paul Mollenhauer 

Human Factors Engineer, Senior 

Lockheed Aeronautical Systems Company 

86 South Cobb Dr, MZ 0976 

Atlanta, GA 30063 

Lou Moore 

Director, Base Maintenance 

American Trans Air, Inc. 

7661 N. Perimeter Road 

Indianapolis, IN 46241 

Suzanne Morgan 

Galaxy Scientific Corporation 

2310 Parklake Drive, Suite 325 

Atlanta, GA 30345 

272 

Dal Mortensen 

Director, Quality Assurance 

United Airlines 

Maintenance Operations Center (SFOQA) 

San Francisco International Airport 

San Francisco, CA 94128 

Duncan Nielsen 

Appendix 

General Manager - Line Maintenance for North 

America 

United Airlines 

Maintenance Operations Center - SFOLM 

San Francisco Int'l Airport 

San Francisco, CA 94128 

Michael Oppedisano 

Director of Quality Control 

National Helicopter Corporation 

Republic Airport 

cs 9100 

Farmingdale, NY 11735-9100 

lame P. Ouellette 

Program Leader, Technical Publications 

GE Aircraft Engines 

1000 Western Ave. 

(IMZ 13206) 

Lynn,MA 01910 

Walter J. Overend 

2439 Fair Oaks Drive 

Jonesboro, GA 30236 

Peter V. Pope 

Engineering Training Manager 

Britannia Airways L TO 

London Luton Airport 

Luton 

Beds. LU29ND U.K. 
Appendix 

Charles Puckett 

Aerospace Engineer 

Federal Aviation Administration AFS-612 

P.O. Box 25082 

Oklahoma City, OK 73125 

Ramon A. Raoux 

Supt. Coirective Action 

Transport Canada Aviation 

200 Kent Street 7th Aoor 

Ottawa Ontario, Canada K2P 2J8 

Edward Rogan (SPEAKER) 

Human Factors Mgr Engineering 

Quality & Training Services 

British Airways 

Technical Block A (S352) 

PO Box 10 - Heathrow Airport 

Hounslow- Middlesex TW6 2JA 

England 

Stephen M. Rokicki, Ph.D. 

Consultant- Human Factors & Ergonomics 

P.O. Box345 

Corrales, NM 87048 

Adel AI Saghir 

Aircraft Planning Engineer 

Kuwait Aircraft Engineers & Pilots Association 

Post Box 62n Hawally 

32037 Hawally, Kuwait 

Sean P. Sanfilippo 

Aircraft Maintenance Technician/Student 

1402 Montego Drive 

San Jose, CA 95120 

AI Schafer 

Supervisor, Technical Training 

Mesaba Airlines 

7501 26th Ave SO 

Minneapolis, MN 55450 

Vic Seavers 

Manager Maintenance Training & Sales 

Northwest Airlines 

DeptF8871 

5101 Northwest Drive 

St. Paul, MN 55III-3034 

Saleh AI Shammari 

Quality Control Engineer 

Kuwait Aircraft Engineers & Pilots Assoication 

Post Box 6277 

273 

Hawally, 32037 Kuwait 

Sam Shaw 

Principal Maintenance Inspector 

Federal Aviation Administration 

Salt Lake City FSDO 

116 North 2400 West 

Salt Lake City, UT 84II6 

William Shepherd, Ph.D. 

Federal A viaton Administration 

Office of Aviation Medicine 

400 7th Street, Room 21028 

Washington, DC 20591 

Aoyd W. Spencer 

Phase V Repon 

Distinguished Member of Technical Staff 

Sandia National Laboratories 

P.O. Box 5800 

1515 EubankSE 

Albuquerque, NM 87185-0829 

John G. Spiciarich 

Director - Inspection 

Trans World Airlines, Inc. 

P.O. Box 20126 

Kansas City, Missouri 64195 

Mary D. Stearns 

Project Manager 

US DOTNolpe Center, DTS-45 

55 Broadway 

Cambridge, MA 02178 

• George P. Sweeney Ill 

Mgr., Human Factors Development- Right 

Operations 

Northwest Airlines, Inc. 

DeptN7260 

5105 Northwest Drive 

St. Paul, MN 55II6 

James C. Taylor 

Adjunct Professor 

Institute of Safety and Systems Mgt. - USC 

ISSMIUSC 

Los Angeles, CA 90089-0021 

Richard Thackray, Ph.D. 

Galaxy Scientific Corp. 

2324 NW 57th Street 

Oklahoma City, OK 73ll2 
Phase V Report 

Tove Titlow 

Program Analyst 

Federal Aviation Administration 

Integrated Safety Analyst 

800 Independence A venue SW 

Washington, DC 20590 

Paul Upp 

Manager -. Planning & Control 

United Airlines 

MS-OAKOM 

1100 Airport Dr. 

Oakland, CA 94612 

Diane Walter 

Aviation Psychology Consultant 

Boeing Commercial Airplane 

P.O. Box 3707 

Mail Stop 2J-21 

Seattle, W A 98124 

Jean Watson . ' 

• 

. ' .~. l 1<~-

; "'~·· . 

~ ·., . 

274 

Federal Aviation Administration 

Office of Aviation Medicine 

400 7th Street, Room 2102B 

Washington, DC 20591 

Kiki Widjaja (SPEAKER) 

Galaxy Scientific Corporation 

2310 Parklake Drive, Suite 325 

Atlanta, GA 30345 

Harvey S. Wiltzen 

Superintendent of Standards 

Transport Canada 

9700 Jasper Avenue 

lith Floor 

Edmonton, Alberta T6J3Y9 .;;_ 

JanZanutto 

Engine Shop Manager 

Westair Airlines 

5588 Air Tenninal Dr. 

Fresno, CA 93727 

Appendix