# Presentation: 532432

## Non-Prescriptive Tools for Effective Fatigue ManagementDOT Human Factors Coordinating CommitteeJune 23, 2004

**Steven R. Hursh, Ph.D.**
**Program Manager, Biomedical Modeling and Analysis**
**Science Applications International Corporation, 443-402-2701**
**Professor, Johns Hopkins University School of Medicine**
**Hurshs@saic.com**

- The Sleep, Activity, Fatigue, and Task Effectiveness (SAFTETM) Model
- and Fatigue Avoidance Scheduling Tool (*FAST*TM)

## Outline

- FRA role and objectives in fatigue management initiatives
- The **SAFTE** Model of fatigue
- The Fatigue Avoidance Scheduling Tool (***FAST***)
- Update on calibration for accidents and incidents
- On-the-Job Performance Study
- Fatigue Assessment Dashboard
- The Incident Fatigue Assessment Protocol (**IFAP**)
- FAST Schedule Design Wizards and XIMES Interoperability

## The FRA Role

- Partnership between Human Factors R & D Program (Volpe) and Office of Safety.
- Supported and coordinated with DOT human factors coordinating committee
- Facilitate development of fatigue management tools for the railroad environment
- Make available funds to advance this tool
- Facilitate and finance necessary studies to calibrate the tool for the railroad environment
- Encourage system-wide assessments combined with site specific solutions
- Maintain a hands-off approach to applications of the tool
- Ensure the confidentiality of information
- Disseminate information about fatigue management solutions

## FRA Office of Safety Objectives

- Advance a non-prescriptive approach to fatigue management.
- Help develop a fatigue and performance model as a non-prescriptive tool to be used at the discretion of management and labor.
- Help develop methods to collect and analyze information (data) that might be used to assess fatigue contributions to accidents and incidents.
- Integrate tools so they can be mutually supporting: *FAST*, Actigraphs, XIMES RAS, PERCLOS
- Continue to promote cooperative approaches to fatigue mitigation solutions.

## Comparative Status of the SAFTE Model

- The DOD and DOT sponsored a comparison of six fatigue models from around the world in Seattle.
- All models attempted to predict the results from four standard scenarios.
**While all models can be improved, the **_**SAFTE**__ __**model had less error than any model**_** tested and was combined with a convenient and logical user interface, the fatigue avoidance scheduling tool** - ***FAST***.

## Model Comparison – Independent EvaluationScenario 2 – Restricted Sleep (data from Dinges Lab – U. Penn)

| Model Author | RRMSE (% Error)
Low score is better | Rank |
| --- | --- | --- |
| Spencer/Belyavin | 90.76 | 4 |
| Moore-Ede | 99.99 | 7 |
| Dawson/Fletcher | 74.57 | 2 |
| Jewett/Kronauer | 92.27 | 5 |
| Hursh (SAFTE 1) | 81.21 | 3 |
| Hursh (SAFTE 2)* | 70.91 | 1 |
| Folkard/Akerstedt | 93.87 | 6 |

- * “Just prior to the Fatigue and Performance Workshop, SAFTE Model was revised and optimized using data from Scenario 5, which is similar to scenario 2.  The ....RRMSE value [was] 70.91%...This constitutes a substantial improvement with respect to the earlier predictions from this and all the other models. (page 27)”

## General Model Deficiencies

- Estimate of prediction error – population variance.
- Representation of individual differences.
- Representation of countermeasures.
- Translation into task or job performance and risk assessment.

- Now included in ***FAST***
- Proposal to Army
- Ongoing with AF and Army
- Lapse Index added, other measures are part of DOT effort.

**Gap**

**Remedy**

## Fatigue Avoidance Scheduling Tool (FASTTM)

***FAST*****TM **is a fatigue assessment tool based on the SAFTETM model
- Developed for the US Air Force and the US Army.
- NTI and SAIC in a Phase III SBIR program.
- DOT/FRA sponsored work has lead to a enhancement for transportation applications.
- Auto Sleep algorithm
- Schedule Grid data entry tool
- Wizards and Dashboard - funded
- DOT field calibration underway.

## SAFTE Model and FAST Predict Performance Effectiveness

- Effectiveness is a measure of speed of making correct responses.
- Effectiveness estimates are at a 1 min. resolution.
- Effectiveness is highly correlated with other measures of fatigue:
- Lapse likelihood
- Reaction time
- Average cognitive throughput
- Driving simulator performance
- Tool predicts both average person and population variance estimate.

## FAST Graphical Screen Options

***FAST***** Graphical Screen Options**

**Effectiveness**

**Sleep Periods in ****Blue**

**Work Periods in ****Red**

## Lapses Increase with Decreasing Effectiveness from FAST (revised)Sleep Dose Response Study – Experimental & Recovery Days - WRAIR Data

**High**

**Low**

- Days of Increasing Sleep Debt

## Lapse Index

## Commercial Applications of FASTTM

*FAST*TM is optimized for the average person, including truck drivers, studied under a range of schedules.  It has a variety of applications:
- Problem Definition and Assessment
- Work Schedule Design and Evaluation
- Generic Schedules (shift-schedules)
- Individualized Schedules (work assignments)
- Safety and Accident Investigation Tool
- Training and Awareness
- Voluntary Self-management
- DOT Experience:  Schedule evaluation and accident fatigue assessment underway

## Complements Other Existing Tools

- FAST provides an objective fatigue and performance estimate that can be used _in conjunction with other available tools_.
- Other Fatigue Risk Management Tools:
- Fatigue monitoring devices – actigraphs and PERCLOS
- Return on Investment Models
- Operational/Terminal Management Models
- Staffing Tools
- Team decision-making tools
- Other Fatigue Estimation Tools

## FRA Initiatives

- Calibrate ***FAST*** for railroad environment
- Analysis of accidents and incidents with two railroad partners, _pilot study_
- Analysis of accidents and incidents with two railroad partners, _large sample study_
- Analysis of locomotive engineer performance
- Enhancement of tool for accident and incident investigations
- Fatigue indicators “dashboard” based on NTSB workshop
- Input wizards for irregular and shift schedules
- XIMES interoperability
- Incident fatigue analysis protocol

## Fatigue Risk Management Tools

***FAST*** TM – _Fatigue Avoidance Scheduling Tool_
- Prospective forecasting of fatigue risk under proposed work/rest schedules.
- Retrospective assessment of fatigue leading up to an event.
- Uses the SAFTE model of fatigue developed by the DOD.
**IFAP****TM** – _Incident Fatigue Assessment Protocol_
- Questionnaire and schedule assessment software to aid event analysis.
- Integrates data into FAST for fatigue assessment.

## Validation Schematic

**Components of the Model**

**Laboratory Cognitive**
**Performance Predictions**

**Task **
**Performance Predictions**

**Safety & Accident**
** ****Predictions**

**Subjective Fatigue**
** ****Predictions**

***Confirming Validation:***

***Intrinsic Process Validation:***

***Primary Predictive Validation:***

**DOD & DOT Projects will contribute to these three areas of validation**

## SAFTE MODELPredictions and Data 

**Total Sleep Deprivation (WRAIR 72 hr Study)**

## Decline of Performance 

**with Total Sleep Deprivation**

## SAFTE Model (Revised)Walter Reed Army Institute of Research Sleep Restriction Study

- Sleep duration is total observed according to EEG measurement.
- Model parameters are constant across conditions of the experiment.

## Congruence of SAFTE Model to Sleep Deprivation Data

## Missed Horns AnalysisITRI Locomotive Simulator 

- Chart 3

## SAFTE/FAST Analysis of Incident DataPilot Study

- Pilot study of 50 events from two railroads.
- Report here data for only one railroad.
- For each event, there were usually two operators: engineer and conductor.
- Results are tentative because of small sample size; large study of 500 events is pending railroad approval.
- Results illustrates the analysis approach.

## Important Data for FAST

- Work starts and stops
- Call times
- Commute times
- Particular sleep habits
- Actual sleep times
- Quality of sleep environment
- Schedule predictability

-     Considered
-     Considered
-     Considered
-     Not considered
-     Not considered
-     Not considered
-     Not considered

## Important Data Not Considered

- Schedule delays or misinformation
- Training and experience
- Medical conditions and sleep disorders
- Medications and/or drug use
- Observations of operator performance and appearance
- Concurrent stress, family issues, or work demands
- Crew resource management (communication problems)

## Fatigue Analysis Overview

***FAST*** is a tool for evaluating and combining fatigue factors in a schedule and providing an objective and impartial performance prediction.
- _The model is designed to predict fatigue, not incidents and accidents._
- Many human factors accidents occur without fatigue.
- Some non-human factors accidents may have a fatigue component that was not recorded.
- However, our hypothesis was that the model should indicate an _increased probability of events at the extremes of predicted fatigue (low effectiveness)._

## RR-X Incident Effectiveness (Least Effective Crewmember)

## RR-X Incident Effectiveness Probabilities(Least Effective Crewmember)

## FAST Can Discriminate Human Factors vs Nonhuman Factors Events

**FAST **with effectiveness criterion at 67%** **
- Percent correct is 68%
- A’ = .77
- Comparisons:
- Materials Testing (detection of cracks in airplane wings)
-     Ultrasound - 0.68
-     Eddy Current - 0.93
-  Medical Imaging (detecting tumors)
-         Ultrasound, adrenal gland - 0.83
-  Weather Forecasting
-     Extreme cold - 0.89
-     Rain    - 0.82
-     Fog - 0.76
-     Storms - 0.74
-     Temperature Intervals - 0.71
-  Polygraph Lie Detection
-     Real crimes - 0.70 to 0.92

*Promising but more data needed.*

- Receiver Operator Curve

## On-the-Job Performance StudyPurpose and Process

- To determine if the ***FAST*** software can make valid predictions of **operator performance changes** that result from fatigue and circadian variations.
- Analysis Steps:
- Collaborate with two or more Class 1 railroads to collect event recorder data.
- Correlate performance measures from engineers along with work schedule information and actigraph measures of sleep.
- Determine if the ***FAST*** predictions are consistent with general variations in performance resulting from sleep patterns and time-of-day effects.
- Determine if interventions reduce performance changes.
- Status:  Looking for collaborators – could use historical data, if available.
- FAA-CAMI (Tom Nesthus) is planning a study of fatigue in coordination with the International Flight Inspection Office to test the predictions of FAST.
- Actigraphs, Logbooks, ARES Performance Battery, and perhaps oral temperature.
- The IFIO missions are typically 3-wks out (Europe, Far East, or wherever).

## Supported by Transportation Research Board Research Recommendations

- (1) Measurements of effectiveness of safety interventions of any sort. Relationship between fatigue and operator performance.
- (7) Sensitivity of fatigue models to performance-related measures? Relationship between fatigue and operator performance.
- (8) Incidence of fatigue in train crews based on work schedule data.

## NTSB Workshop on Fatigue Analysis in Transportation Accidents

- Rosekind and Dinges – Expert Consultants
- Recommended in-depth assessment of operator schedules, sleep need and habits, medical history, and medications.
- Recommended calculation of fatigue indicators.
- FRA now sponsoring work to add fatigue indicators to ***FAST***, along with performance prediction.
- Feature will provide immediate fatigue explanation of low predicted performance to aid the accident investigator.

## Other Initiatives

- Fatigue Indicators Dashboard
- Incident Fatigue Assessment Protocol - Update
- Schedule Design Wizard
- FAST – XIMES RAS interoperability
- (Details follow)

## All of the indicators can be computed from variables available within FAST.  All the performance indicators are computed as transforms from Effectiveness shown on the graph. 
Fatigue factors within a dangerous range are indicated by a red flag based on criteria established at NTSB workshop.  
The performance indicator will change color from green to yellow to red.  When performance is less than green, the flagged fatigue factors will provide an explanation.  
All these values would be continuously updated as the cursor is moved along the time line of the schedule, giving an instantaneous assessment at any moment or at some critical event.  

| Performance Indicators |  |  |  | Fatigue Factors |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Effectiveness
(PVT speed)
(% normal) | Mean Cognitive
(% normal) | Lapse Probability
(times normal) | Reaction Time
(% normal) | Acute Sleep Debt
(Hrs in last 24 hrs) | Cumulative Sleep Debt
(Total Hours prior to current day) | Continuous Hours Awake | Time of Day | Circadian Disruption
(Hrs out of phase) |
| <90%=Y
<75%=R | <90%=Y
<75%=R | >2=Y
>4=R | >110%=Y
>133%=R | ≥8 hrs=R | ≥ 8 hrs=R | ≥ 17 hrs=R | 0000-0600, 1500-1700
=R | ≥ 3 hrs=R |
|  |  |  |  |  |  |  |  |  |

- All of the indicators can be computed from variables available within ***FAST***.  All the performance indicators are computed as transforms from Effectiveness shown on the graph.
- Fatigue factors within a dangerous range are indicated by a red flag based on criteria established at NTSB workshop.
- The performance indicator will change color from green to yellow to red.  When performance is less than green, the flagged fatigue factors will provide an explanation.
- All these values would be continuously updated as the cursor is moved along the time line of the schedule, giving an instantaneous assessment at any moment or at some critical event.

## When activated, the dashboard will appear as a window within the schedule window and update with the movement of the mouse.  A left mouse button depression within the schedule area will clear the table; release the mouse button and it will rewrite with new values appropriate for the new cursor position.  The “dashboard” can be dragged to any convenient place within the graphical window.  A right mouse click will allow the user to print or copy the dashboard to the clipboard for paste into another application.

**Schedule Name: Date:Time**

## Incident Fatigue Assessment Protocol

- Computer-based investigation _questionnaire_ of operators and witnesses
- Computerized version of Technical Bulletin (proof of principal)
- Keppen Associates initiative to test and improve questionnaire
- Computer-based schedule recording and analysis
- Access database creation
- Automatic porting to ***FAST***
- Potential to be ported to PDA

## Incident Fatigue Assessment Protocol(IFAP) Schedule Entry Screen

## Schedule Design Wizards – Funded (Volpe Project)

- Shift-schedule wizard
- Irregular schedule wizard
- Dialog data entry with options to branch to tabular or grid entry (“TurboTax” metaphor)
- Descriptive results output
- Graphical
- Tabular
- Fatigue factors
- Narrative

## FAST and XIMES RAS Interoperability – Funded(Volpe Project)

**XIMES RAS** is an Austrian developed shift schedule design tool.
- Assists in design and descriptive analysis of shift schedules
- Contains rule of thumb schedule evaluation
- Does not assess fatigue potential
- Initiative will create standard schedule file format that can be shared by these two tools and an other schedule analysis software.
- Schedules created with **XIMES RAS ** can be imported into ***FAST*** for sleep assessment and fatigue analysis.

## Recommendations

- Continue to advance our knowledge and understanding of software modeling of fatigue as _one tool_ to address fatigue.
- Continue to seek cooperation of railroad partners in collecting essential data.
- Continue to seek performance indices to quantify the role of fatigue in railroad operations.
- Continue to coordinate and strengthen transfer of our knowledge to other modes.
- Welcome partnerships with other modes to fill data gaps.
- Continue to strengthen our working relationships with AAR’s Work Rest Task Force and NARAP.

## End of Presentation

**Steven R. Hursh, Ph.D.**
**Science Applications International Corporation, 443-402-2701**
**Professor, Johns Hopkins University School of Medicine**
**Hurshs@saic.com**

**Non-Prescriptive Tools for ****Effective Fatigue Management**

## Modeling Approaches

*Fatigue Audit InterDyneTM (FAID) **Model*: Drew Dawson (University of South Australia)* - *fatigue model currently being applied as a fatigue management tool in Australia and by the Union Pacific.
*Sleepwake Predictor:  *Torbjorn Åkerstedt (Karolinska Institute) & Prof. Simon Folkard (University of Wales)
*Interactive Neurobehavioral Model:* Megan Jewett, (Harvard University)
*System for Aircrew Fatigue Evaluation (SAFE):* Spencer and Belyavin (QinetiQ, Inc, UK)
*Circadian Alertness Simulator (CAS): *Martin Moore-Ede* *(Circadian Technologies, Inc., USA)
*Sleep, Activity, Fatigue and Task Effectiveness (SAFTE) Model: *Steven Hursh (SAIC and Johns Hopkins University, USA).