Detection, prediction and modelling of mental fatigue in naturalistic environment

Operator mental fatigue in workplace can result in serious mistakes which have dangerous and life-threatening consequences. Fatigue assessment and prediction are, therefore, considered critical safety requirements that cut across modes and operations of numerous high-risk environments and industries such as nuclear and transportation. However, robust, accurate and timely assessment of fatigue (or alertness) is still a challenging task for many reasons. The majority of operator fatigue studies are still being carried out in simulation environments, overlooking operator's naturalistic behaviour and fatigue growth. Moreover, most of the available systems rely on using a single fatigue-related data source, which is clearly a major drawback that affects operation, performance, accuracy and reliability of the system in case this source fails. With multi-data sources in an integrated system, the system might stop working in the event of losing one or more data sources or at least becomes inaccurate or unreliable. Furthermore, paying no attention to human individual differences working as an operator in mission-critical jobs related to fatigue growth and in response to fatigue deleterious effect is another serious issue with the current fatigue assessment and prediction systems. The research work presented in this thesis proposes a novel fatigue assessment approach, which addresses the aforementioned issues with fatigue detection and prediction system. This is achieved by developing and realising algorithms based on data collected from participants in naturalistic environments. Numerous experiments have been conducted to cover a wide range of fatigue-related tasks which are broadly grouped into two categories: biological and behavioural (performance) experiments. The biological-based experiments employ various data types such as heart rate, skin temperature, skin conductance and heart rate variability. These fatigue-related data types are used to build the proposed fatigue detection system, and the obtained results have demonstrated high accuracy and reliability (94.5% accuracy in naturalistic environments). The behavioural-based category includes two experiments: keyboard typing and driving task. The typing experiments have been carried out using computer keyboard and smartphone virtual keyboard, and have confirmed enhanced operator fatigue detection accuracy (94%). The driving experiments were conducted in naturalistic driving environments, and the used algorithms have demonstrated a new framework for driver fatigue detection using smartphone inertial sensors based on a novel vehicle heading algorithm. A prototype system was designed and built with a modular structure so as to allow the addition of multiple fatigue-related biological and behavioural sources. This modular structure was tested under different situations that involve losing one or more sources. In addition, the circadian rhythm, which is a main input to fatigue/alert regulators, was customised for each operator and modelled based on biological data collected from wearable devices. The constructed model captures individual differences of operators, which is a challenge in current systems. Such multi-source, modular and non-intrusive approach for fatigue/alertness assessment and prediction is expected to be of superior performance, low-cost and favourable by users compared to existing systems. Furthermore, it addresses other challenges of current fatigue systems by carrying out fatigue assessment in naturalistic environments and considering operator individual differences in response to fatigue. In addition, the modular structure of the proposed system helps improving robustness and accuracy against losing one or more input sources (accuracy for 4 sources: 91%, 3 sources: 87%, 2 sources: 77%). Following the proposed approach will definitely enhance the reliability of fatigue assessment systems, improve operator safety, productivity and reduce financial fatigue impacts. Moreover, the proposed system has proven to be non-intrusive in nature and of low implementation cost. The results obtained after testing the proposed system have been very promising to support the aforementioned benefits.