**Handbook of System Reliability in**

**Airframe and Engine Inspection**

**C. G. Drury**

University at Buffalo, State University of New York

Department of Industrial Engineering

Buffalo, NY 14260

Draft

December 2005

Prepared for:

Federal Aviation Administration

Washington, D.C.

Contract #2004-G-037

**Table of Contents**

#  {#section .Chapter-heading}

[**Chapter 1: Purpose and Scope of the
Handbook**](#__RefHeading___Toc119901141)

[1.1 Purpose](#__RefHeading___Toc119901142)
................................................................................................

[1.2 Scope](#__RefHeading___Toc119901143)
.................................................................................................

[1.3 Why the Inspector is Critical to NDI
Effectiveness](#__RefHeading___Toc119901144)

[1.4 How to Use the Handbook](#__RefHeading___Toc119901145)

**Chapter 2: Inspection
Reliability**.................................................................

2.1 Inspection Reliability
Defined............................................................\...

2.2 The PoD
Curve................................................................................

2.3 Inspection as Detecting Signals in Noise: The ROC
Curve.............................

2.4 The Basis for NDI
Inspection...............................................................

2.5 Transition to Chapter
3.......................................................................

**Chapter 3: Human and Equipment Roles in Inspection**
.................................\...

3.1 Generic Tasks of Inspection
.........................................................................

3.2 Levels of Functioning in Inspection
...............................................................

3.3 Critical Functions: Search and Decision
..........................................................

3.3.1 Search
..............................................................................................

3.3.2 Decision
..................................................................................

3.3.3 Search Plus Decision
...................................................................

3.4 Path Control
..................................................................................

3.5 Factors Affecting Inspection
...............................................................

3.5.1 Task Factors
...............................................................................

3.5.2 Operator Factors
..........................................................................

3.5.3 Machinery Factors
.......................................................................

3.5.4 Environment Factors
.............................................................................

3.5.5 Social Factors
......................................................................................

[**Chapter 4:** **Methodology for Generating NDI Good
Practices**](#__RefHeading___Toc119901068)

[4.1 Site Visits](#__RefHeading___Toc119901069)

[4.2 Hierarchical Task Analysis](#__RefHeading___Toc119901070)

[4.3 Transition to Chapters 5-12](#__RefHeading___Toc119901071)

[**Chapter 5: Generic Good Practices**
..................................................................](#__RefHeading___Toc116795751)

[5.1 Speed and Accuracy: How closely should an inspector
inspect?](#__RefHeading___Toc116795752)\...\...\...\...\...\...\...\...\...\...\...\...\...\...\...\....

[5.2 Time Limits in Inspection performance
...........................................................](#__RefHeading___Toc116795753)

[5.2.1 The Vigilance
Literature](#__RefHeading___Toc116795754).......................................................................

[5.2.2 Field Studies of
Vigilance](#__RefHeading___Toc116795755).....................................................................

[5.2.3 Conclusions on Vigilance
Effects............................................................](#__RefHeading___Toc116795756)

[5.3 Documentation
Design](#__RefHeading___Toc116795757)...............................................................................

[5.4 Training and Selection
............................................................................](#__RefHeading___Toc116795758)

[5.4.1
Training](#__RefHeading___Toc116795759)..........................................................................................

[5.4.2
Selection](#__RefHeading___Toc116795760)..........................................................................................

[5.5 Workplace Design, Posture and Inspection
Performance.....................................](#__RefHeading___Toc116795761)

[5.6 Equipment
Usability............................................................................](#__RefHeading___Toc116795762)

[5.7 Control/Display
Design...........................................................................](#__RefHeading___Toc116795763)

[5.7.1 Control-Display Integration](#__RefHeading___Toc116795764)
..................................................................

[5.7.2 Controls](#__RefHeading___Toc116795765)
.......................................................................................\...

[5.7.3 Visual Displays](#__RefHeading___Toc116795766)
..............................................................................\...

[5.7.4 Glare Reduction](#__RefHeading___Toc116795767)
................................................................................

[5.8 Human Computer
Interaction............................................................................](#__RefHeading___Toc116795768)

[5.8.1 Organization of
Functionality](#__RefHeading___Toc116795769)...............................................................\...

[5.8.2 Windowing Operations](#__RefHeading___Toc116795770)
........................................................................

[5.8.3 User Dialog Styles](#__RefHeading___Toc116795771)
...........................................................................\...

[5.8.4 Use of Color in Human-Computer
Interfaces](#__RefHeading___Toc116795772)
..............................................

[5.4.5 On-line Aiding/Help](#__RefHeading___Toc116795773)
...........................................................................

[5.8.6 Error
Management](#__RefHeading___Toc116795774)..............................................................................

[5.9 Automation and Function Allocation
.........................................................](#__RefHeading___Toc116795775)

[5.10 The Visual Environment
.......................................................................](#__RefHeading___Toc116795776)

[5.10.1
Glare](#__RefHeading___Toc116795777)..............................................................................................

[5.10.2 Reflectance](#__RefHeading___Toc116795778)
......................................................................................

[5.10.3 Current Lighting Practice](#__RefHeading___Toc116795779)
.......................................................................

[5.11 Interpersonal System Design
...................................................................](#__RefHeading___Toc116795780)

5.12 [Transition to Chapters 6-12](#__RefHeading___Toc116795781)
..........................................................................

[**Chapter 6: Visual Inspection**
...........................................................................](#__RefHeading___Toc122942372)

[6.1 Visual Inspection Defined
....................................................................](#__RefHeading___Toc122942373)

[6.2 Characteristics of Visual
Inspection](#__RefHeading___Toc122942374)..............................................................

[6.3 Overall Visual Good Practices
..............................................................](#__RefHeading___Toc122942375)

[6.4 Detailed Good Practices
......................................................................](#__RefHeading___Toc122942376)

[6.5 Checklist for Visual Inspection
.............................................................](#__RefHeading___Toc122942377)

[**Chapter 7: Borescope Inspection**](#__RefHeading___Toc123017299)

[7.1 Borescope Inspection Defined](#__RefHeading___Toc123017300)

[7.2 Characteristics of Borescope
Inspection](#__RefHeading___Toc123017301)

[7.3 Overall Borescope Good Practices](#__RefHeading___Toc123017302)

[7.3.1 Borescope Physical
Design........................................................................\...](#__RefHeading___Toc123017303)

[7.3.2 Automation and Borescope Use](#__RefHeading___Toc123017304)

[7.4 Detailed Human Factors Good
Practices](#__RefHeading___Toc123017305)

[7.5 Checklist for Audit of Borescope Inspection
Systems](#__RefHeading___Toc123017306)

[**Chapter 8: Radiographic Inspection**](#__RefHeading___Toc123016259)

[8.1 Radiographic Inspection Defined](#__RefHeading___Toc123016260)

[8.2 Characteristics of Radiographic
Inspection](#__RefHeading___Toc123016261)

[8.3 Overall Radiographic Inspection Good
Practices](#__RefHeading___Toc123016262)

[8.3.1 Automation for Radiographic
Inspection](#__RefHeading___Toc123016263)

[8.3.2 Digital Radiographic Inspection](#__RefHeading___Toc123016264)

[8.4 Detailed Good Practices for Radiographic
Inspection](#__RefHeading___Toc123016265)

[8.5 Checklist for Audit of Radiographic Inspection
Systems](#__RefHeading___Toc123016266)........................................

[**Chapter 9: Fluorescent Penetrant
Inspection**](#__RefHeading___Toc122928095)

[9.1 Characteristics of
FPI](#__RefHeading___Toc122928096).................................................................................

[9.2 Overall FPI Good
Practices...........................................................................](#__RefHeading___Toc122928097)

[9.2.1 Operator Selection, Training and Turnover in
FPI](#__RefHeading___Toc122928099)...........................................

[9.2.2 Hardware Design for
FPI](#__RefHeading___Toc122928100)..........................................................................

[9.2.3 Software and Job Aids for
FPI](#__RefHeading___Toc122928101)..................................................................

[9.2.4 Interpersonal System Design in
FPI](#__RefHeading___Toc122928102)..............................................................

[9.2.5 Environmental
Control](#__RefHeading___Toc122928103).............................................................................

[9.3 Detailed Good Practices in FPI](#__RefHeading___Toc122928104)

[9.4 Checklist for Audit of FPI Inspection
Systems](#__RefHeading___Toc122928105).....................................................

[**Chapter 10: Magnetic Particle
Inspection**](#__RefHeading___Toc123017484)................................................................

[10.1 Magnetic Particle Inspection
Defined](#__RefHeading___Toc123017485)...............................................................

[10.2 Characteristics of Magnetic Particle
Inspection](#__RefHeading___Toc123017486)....................................................

[10.3 Documentation for
MPI](#__RefHeading___Toc123017487)...............................................................................

[10.4 Detailed Good Practices for
MPI](#__RefHeading___Toc123017488)....................................................................

[10.5 Checklist for Audit of Magnetic Particle Inspection
Systems](#__RefHeading___Toc123017489)

[**Chapter 11: Eddy Current
Inspection**](#__RefHeading___Toc122926948).....................................................................

[11.1 Eddy Current Inspection
Defined](#__RefHeading___Toc122926949)..................................................................\...

[11.2 Characteristics of Eddy Current
Inspection](#__RefHeading___Toc122926950).........................................................

[11.3 Overall Eddy Current Good
Practices](#__RefHeading___Toc122926951)...............................................................

[11.3.1 Automation for EC](#__RefHeading___Toc122926952)
...................................................................................

[11.3.2 Automation and Final
Decision](#__RefHeading___Toc122926953).....................................................................

[11.3.3 Documentation and Task
Characteristics](#__RefHeading___Toc122926954)..........................................................

[11.4 Detailed Good Practices for Eddy Current
Inspection](#__RefHeading___Toc122926955).............................................

[11.5 Checklist for Audit of Eddy Current Inspection
systems](#__RefHeading___Toc122926956)..........................................

[**Chapter 12: Ultrasonic
Inspection**.............................................](#__RefHeading___Toc123018184)............................

[12.1
Introduction...................................................................................](#__RefHeading___Toc123018185)
.........

[12.2 Ultrasonic Inspection
Defined............................................................](#__RefHeading___Toc123018186)...........

[12.3 Characteristics of Ultrasonic
Inspection.............................................](#__RefHeading___Toc123018187)............\...

[12.4
Methodology..............................................................................](#__RefHeading___Toc123018188).............

12.5 Overall Ultrasonic Good
Practices..................................................................

[12.5.1 Automation for Ultrasonic
Inspection.......................................](#__RefHeading___Toc123018190)...................

[12.5.2 Automation and Final
Decision..........................................](#__RefHeading___Toc123018191)
....................

[12.6 Checklist for Audit of Ultrasonic Inspection
Systems.....................](#__RefHeading___Toc123018192).......................

**ACROYMNS**

A&P Airframe and Power Plant (license)

AAM FAA's Office of Aviation Medicine

AC Advisory Circular

AD Airworthiness Directives

AECMA European Association of Aerospace Industries

ANCOVA Analyses of Covariance

ANOVA Analysis of Variance

AMT Aviation Maintenance Technician

ASME American Society of Mechanical Engineers

ASNT American Society of Nondestructive Testing

ASSIST Automated System of Self Instruction for Specialized Training

ATA Air Transport Association

BASI Bureau of Air Safety Investigation

BLS Bureau of Labor Statistics

CAC Canadian Airports Council

CAE Conclusion, Analysis and Evidence

CASA Iowa State University\'s Center for Nondestructive Evaluation

CASR Center for Aviation Systems Reliability

CBT Computer-based Training

CCD Change Coupled Device

CRT Cathode Ray Tube

CTSB Canadian Transportation Safety Board

DDA Documentation Design Aid

EC Eddy Current

ECRIRE Sandia Labs Program

EFT Embedded Figures Test

ERNAP Ergonomics Audit Program

FAA Federal Aviation Administration

FPI Fluorescent Penetrant Inspection

GLM General Linear Models

GNMM Global Nuclear Materials Management

HIT Harris Inspector Test

HTA Hierarchical Task Analysis

ID Index of Difficulty

ILS International Illuminating Society

IQI Image Quality Indicator

MAUS IV Eddy Current Inspection System, Boeing

MRM Maintenance Resource Management

MRO Maintenance Repair Operator

MPI Magnetic Particle Inspection

MSG-3 Maintenance Safety Group

NDE Non-Destructive Evaluation

NDI Non-Destructive Inspection

NRR Non-routine Repair

NTSB National Transportation Safety Board

OASIS On-line Aviation Safety Inspection System

OEM Original Equipment Manufacturer

OSHA Occupational Safety and Health Administration

OWAS OVAKO Work Analyzing System

PERS Proactive Error Reporting System

PoD Probability of Detection

PoFA Probability of False Alarm

QA Quality Assurance

ROC Relative Operating Characteristics

SAOC Speed-Accuracy Operation Characteristics

SATO Speed-Accuracy Trade-Off

SDT Signal Detection Theory

SE Simplified English

SFD Source-to-film-distance

SHELL Software / Hardware / Environment / Liveware / Liveware system

SNL/AANC Sandia National Laboratories /Aging Aircraft Nondestructive
Inspection

Validation Center

TAM Tool Aerospace Material

TLX Task Load Index

TOMES Task / Operator / Machine / Environment / Social system

UFOV Useful Field of View

USA United States of America

UV Ultraviolet

UT Ultrasonics