Detecting Sleepy Drivers by Pattern Recognition based Analysis of Steering Wheel Behaviour

Using steering wheel behaviour based approaches for sleepiness monitoring might have the advantage of being cheap, non-intrusive, and robust even under extreme demanding environmental conditions (e.g. high background noise, temperature, or humidity). Twelve healthy young adults completed 7 overnight driving sessions (1 8 a.m.) in our real car driving simulation lab. The combinations of observed and self-rated sleepiness (Karolinska Sleepiness Scale; KSS) were considered as ground truth of sleepiness. Steering angle, lane deviation and pedal movement were recorded. In order to investigate sleepiness-induced changes in steering wheel behaviour, spectral and state space domain features are computed. Using advanced signal processing procedures for feature extraction, we computed 3 feature sets in the time, frequency and state space domain (a total number of 1251 features) to capture fatigue impaired steering patterns. Within the time domain we extracted class distribution measures, peak amplitudes and distances, and zero crossing distances (e.g. maximum of peak amplitude, mean distance between consecutive zero crossings). Each feature set was separately fed into 5 pattern recognition methods (e.g. SVM, KNN, MLP). We yielded a recognition rate of 86.1% in classifying slight from strong fatigue. Sleepiness and Steering Drivers are constantly judging the situation ahead and applying small, smooth, steering adjustments to correct for small road bumps and crosswinds by turning the steering wheel in small increments. This steering behavior can be impaired when drivers are becoming fatigued. A wide variety of steering wheel metrics have been suggested to measure steering behavior, from standard deviation of steering wheel angle, steering wheel velocity (Peters, Kloeppel, & Alicandri, 1999), steering wheel action rate (Verwey, 2000), to more advanced metrics such as high frequency component of steering wheel angle (Östlund, Nilsson, Carsten et al., 2004) and steering entropy (e.g. Nakayama, Futami, Nakamura, & Boer, 1999). Until recently, the analysis of steering wheel movement was based on finding single features correlating with fatigue. The detection task was reduced to a simple thresholding of these features, but large interand intra-individual differences in fatigued driving patterns are an issue. Due to this overwhelming complexity, a sufficient and accurate classification of individual steering samples was not achieved. Thus, we propose a multivariate, data fusion, and machine learning based approach including the computation of a large, highly redundant feature set modelling fine grained temporal steering behavior structures (cf. Sayed & Eskandarian, 2001). This approach showed its superiority in many pattern recognition tasks (e.g. Ruvolo & Movellan, 2008). Feature Extraction for Steering Behavior. Features of steering wheel movement can be divided into time, frequency and state space domains. This assignment follows the first processing step of computing frame level descriptors, independent of feature characteristics of the second, contour describing, functional based processing step: (a) time domain features. Within the time domain the following features can be extracted: regression descriptors (e.g. regression slope, intercept, maximum of regression error), class Jarek Krajewski, Martin Golz, and David Sommer 286 distribution measures (e.g. number of values within steering angle bin 0.0-0.1), peak amplitudes and distances (e.g. mean distance of peaks; maximum of peak amplitude), entropy, zero crossing distances and slope (e.g. maximum of distance between consecutive zero crossings; mean velocity of steering angle in zero crossings) (b) frequency domain features. To capture fine temporal changes of spectral descriptors we performed a framing and windowing of the signal (frame size = 512, frame shift = 256, hanning window), and computed the power spectral density per frame. This procedure results in 803 frequency domain features as e.g.: relative and absolute power spectral density (PSD) of raw and first derivate contours in 30 spectral bands (e.g. minimum of relative PSD of first derivate of 0.6-0.7Hz spectral band contour), band energy ratios (PSD 1-5hz/ PSD 0-1hz), spectral flux (e.g. max of spectral flux = Euclidean distance of PSD between consecutive frames), and long term average spectrum descriptors (e.g. skewness of PSD distribution). (c) state space features. To extract the nonlinear properties of the steering angle signal, we computed a three-dimensional state space (steering wheel position, steering wheel velocity, steering wheel acceleration), and a reconstructed phase space (d = 3,4 ; τ = 1; steering angle t0, steering angle t1, steering angle t2). The geometrical properties of the resulting attractor figures were described by trajectory based descriptor contours (angle between consecutive trajectory parts, distance to centroid of attractor, length of trajectory leg). The temporal information of the contours was captured by computing functionals.