# A System for Driver Analysis Using Smartphone as Smart Sensor

Rui Daniel Vilaça, Rui Esteves Araújo

► To cite this version:

Rui Daniel Vilaça, Rui Esteves Araújo. A System for Driver Analysis Using Smartphone as Smart Sensor. 8th Doctoral Conference on Computing, Electrical and Industrial Systems (DoCEIS), May 2017, Costa de Caparica, Portugal. pp.103-110, 10.1007/978-3-319-56077-9_9 . hal-01629584

HAL Id: hal-01629584

https://inria.hal.science/hal-01629584

Submitted on 6 Nov 2017

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# A System for Driver Analysis Using Smartphone as Smart Sensor

Rui Daniel Vilaça¹, Rui Araújo² and Rui Esteves Araújo³

¹ Faculty of Engineering University of Porto,
Rua Dr. Roberto Frias, 4200-465 Porto, Portugal
rdvilaca@fe.up.pt

² MCA & Associados, Lda
Rua D. Manuel II, s/44, 4050-345 Porto, Portugal
ruka.araujo@gmail.com

³INESC TEC and Faculty of Engineering University of Porto, Portugal
raraujo@fe.up.pt

**Abstract.** This work is focused on the development of system able to keep tracking driver's behavior without a black box device mounted inside the car. Firstly, we intend to explore the data from GPS (Global Positioning System), accelerometer, gyroscope and magnetometer for a full characterization of the vehicle dynamics. Secondly, we develop an event detector that determines and classifies distinct kind of maneuvers, like turns, lane change, U-turns, among others. Finally, we developed a simple aggressiveness classifier using fuzzy logic. Experiments have been conducted and the initial results of the system were found to be encouraging on the implementation of a non-intrusive system for driver analysis.

**Keywords:** Intelligent Systems, Smartphone sensor, Driver Behavior, Vehicle Dynamics.

## 1 Introduction

According to a study from the World Health Organization [1] the injuries caused by traffic accidents are one of the main causes of death, the majority being among young people between the ages of 15 and 29 years old. This circumstance motivates the scientific community to the continuous development of technologies and technical solutions that contribute to attenuate the numbers related to road accidents. One possible solution is the development of driver support and/or evaluation systems in on-board computers or mobile computing devices, which can provide almost instantaneous support to the driver and contribute to the driver's awareness about driving risks.

In recent years, there has been a highly significant growth in the usage of mobile phones with computer-like capabilities. These units, commonly known as smartphones, stand out by their high capacity of processing and connection to several
high-speed networks, as well as for the ability of sensing information from movement and position.

In this context, the use of the provided sensors is an opportunity for the development of an intelligent system capable of monitoring, evaluation and assistance to the driver of road vehicles.

Studying the specialized literature, we found works with similar objectives, such as: driver evaluation [2][3][4][5][6], driving support (in real-time or not) [7][8], increase vehicle performance [7] or traffic control [9][10]. These types of works can also help reduce pollution and the number of road accidents.

This paper presents a system with the objective of evaluating the aggressiveness of the driver, focusing mainly on the characterization of vehicle dynamics, using one smartphone as a smart sensor. The basic system requirements are: Plug and Play – easy usability for the driver; No extra costs or additional devices and Non-intrusive.

At this early stage the challenge of characterizing the dynamics of the vehicle with a non-oriented smartphone is solved. As a complement, two applications were developed for validation of results and emulation of a simple aggressive classifier.

## 2 Smart System

In the developed countries, most of the population has, at least, one smartphone. Also, these machines have an increasing processing capacity and more types of sensors with better quality in terms of signal acquisition. Such capabilities make it a platform with high potential for implementation of smart systems.

Generally, this type of equipment allows the acquisition of image, recording of sound with the microphone, smartphone movement characterization with the accelerometer and gyroscope, location and orientation through GPS and magnetometer and detection of use with the presence sensor.

Thus, for the system in question, due to the objective of creating a non-intrusive system and to preserve the users' privacy, the use of the camera and microphone is ruled out, focusing only on GPS, accelerometer, gyroscope and magnetometer.

The GPS operates with 1 Hz sample frequency and allows to obtain the position in terms of the Geographic Coordinate System (longitude, latitude, altitude), and speed. Its use requires significant battery consumption.

In the remaining sensors, the sampling frequency can reach up to hundreds of Hz. The sampling frequency is a trade-off between the precision of information and battery consumption. For this system, a sampling rate of 10 Hz was defined.

The accelerometer and gyroscope allow obtaining, respectively, the acceleration in m/s and the angular velocity in rad/s, represented in terms of the smartphone coordinate system. Due to the no strict orientation requirements of the smartphone in order to ease the user experience, it is necessary to perform a virtual orientation of the signals to characterize the movement of the vehicle. In addition, the signals from the accelerometer require a more efficient filtering due to high sensitivity to small movements. The use of the gyroscope comes with the problem that not all smartphones have one.
The use of the magnetometer allows the measurement of the magnetic field in µT around of the device, which is not necessarily the Earth's magnetic field due to the environment around it, a notorious situation in a motorized vehicle due to the electromagnetic materials. Calibrating the magnetometer to the surrounding environment allows obtaining a more precise orientation of the vehicle.

For data storage and processing, the operating systems available for smartphones allow a real-time processing of the data or store in files for further processing performed locally or on an external machine.

## 3 Related Work

A recent bibliography research allows us to conclude that there are several scientific papers describing systems with characteristics similar to the one presented in the present work. However, there are aspects related to the central objective of the work, the operating conditions or signal processing techniques that deserve a more detailed discussion and that contribute to support the present work.

The authors present, in [2], a work divided in two parts, with the main objective being the development of a system adaptable to mobile devices with the ability to detect aggressive behaviour. It is defined as aggressive behaviour: speeding, abrupt acceleration and braking, nonconformity with the minimum distance for the front vehicle and overtaking in heavy traffic. In the first part, it is validated experimentally a mathematical model based on the initial hypothesis that the aggressiveness operates as a linear filter on the probability distribution function of the input signals. The mainly conclusions were that aggressiveness affects the mean, standard deviation and range of values of lateral and longitudinal acceleration, but maintains the waveform of the acquired signals. Also, it said that the results are independent of the road type and the driver.

In the second part the model is tested through the development of an aggressive classifier, a GPS data logger is used for data acquisition and a success rate of 92.3% is achieved identifying the aggressive lap in a total of 10 laps of the same driver.

From this work, it is important to retain the aggressive behaviours, the statistical indicators affected with the aggressiveness and the independence before the road pavement and the driver. In addition, it serves as validation of the hypothesis of creating a non-intrusive aggressiveness classifier applicable to smartphones.

The document [3] explains the development of MIROAD, a smartphone application for detecting aggressive driving. It differentiates itself from this work using the camera and requiring a specific positioning of the smartphone. For detection of driving patterns, lateral and longitudinal movement is characterized by the fusion of accelerometer, gyroscope and magnetometer data.

The system allows detecting the following events: normal/aggressive right/left shift, normal/aggressive U-turn, aggressive right/left lane change, aggressive acceleration/braking, speeding and device removal. For detection and classification, the authors used triggers and pattern recognition through dynamic time wrapping (DTW).
This work presents a lot of similarities to the one developed previously, providing useful information regarding signal acquisition and processing, and event detection and classification. The main difference is the restriction of the smartphone position so that is oriented with the vehicle. The removal of this restriction is the novelty of the work developed here.

Finally, in [4] it is presented a tool applicable to mobile applications, V-sense, which aims to detect and differentiate different maneuvers. The maneuvers detected are: lane change, turns and driving in curvy roads. Through the accelerometer, it is calculated the distribution of the gravity force and, from there, the angles of the rotation to orient the smartphone coordinate system with the vehicle's one. With the oriented gyroscope signal the maneuver classification is done considering the bumps duration and the value of maximum and minimum. In some cases, the horizontal displacement calculated through GPS and accelerometer data is used as decisive factor.

The major differentiation of this work is the calibration of the signal to the vehicle coordinate system and the maneuvers classification with the oriented gyroscope signal and horizontal displacement.

## 4 System Architecture

The system developed in this work presents a modular architecture where two parts are clearly differentiated. The first one is responsible for characterizing the movement of the vehicle during a recorded trip and, it is called the “Vehicle Black-Box”. The second one is responsible for extracting useful information from processed signals, consisting of two main modules: Event Detector and Aggressiveness Classifier. In Fig.1 it is represented a layout with the architecture and information flow of the system.

### SYSTEM LAYOUT

<figure><img src="image_2.png" /><figcaption>Fig.1. System architecture and information flow.</figcaption></figure>

One of the central aspects for orientation manipulation is the definition of the necessary coordinate systems, which is represented in Fig.2. The correct definition, followed by a precise virtual orientation of the signals, gives the system freedom relatively to the smartphone's position. Furthermore, it allows one to introduce, in future work, an automatic mechanism that detect phone handling while driving.
COORDINATE SYSTEMS

<figure><img src="image_3.png" /><figcaption>Fig.2. Representation of coordinate systems: Smartphone, vehicle and earth.</figcaption></figure>

4.1 Vehicle Black Box

This part of the system is responsible for characterizing the dynamics of the vehicle
during a trip using the signals acquired from smartphone sensors. For correct
operation, it is necessary to place the device in a stable place during the entire
recording session. A sampling of the last value read by each sensor is done at 10 Hz.
Although GPS has a maximum sampling frequency of 1 Hz, this method is applied to
obtain a full data sample at determined times instead of having individual sensor
samples in different instants of time.

From the GPS data, in a X-size sample, the velocity signal of 1 Hz is transformed
into a signal of 10 Hz using a linear extrapolation:

$$
R(i) = \frac{1}{2*R+1} \sum_{k=i-R}^{i+R} V(k), \forall i \le X. \quad (1)
$$

A reference signal for the acceleration and jerk of the vehicle is also calculated
through the first and second derivatives. The orientation of the vehicle is calculated
using the pair (longitude, latitude) through the slope of the line drawn using three
consecutive instances.

A filter is applied to the signals from accelerometer based on the exponential
average,

$$
R(i) = \alpha * A(i) + (1 - \alpha) * R(i - 1), \forall 1 < i \le X, R(1) = k, \quad (2)
$$

eliminating most of the noise without removing relevant information. The gyroscope
disregards the use of filters due to the low susceptibility to noise.

The next step consists in the virtual orientation of the signals to the vehicle
coordinate systems. For this, every time the speed is zero or constant, the distribution
of acceleration in the three axes according to the smartphone coordinate system is
analysed and a weighted average for such distribution is calculated. It is assumed that
the obtained vector corresponds to the gravity force and it is completely represented
in the Z axes of the vehicle. With this assumption, the rotation matrix for the vehicle
XY plane is calculated. In the new oriented signals, the rotation speed of the vehicle is
represented in the Z axe of the vehicle coordinates system and the module of vehicle's
acceleration is represented by the module of X and Y values.

In order to differentiate the longitudinal and lateral acceleration, all values of the
acceleration derived from the GPS speed with a significant value are selected. Those
values represent the times when it can be assumed that all the acceleration is
longitudinal. For those times, it is calculated the angle between the acceleration vector
in XY plane and the X axe of the vehicle. The longitudinal acceleration is derived to
compute the jerk.

Having properly characterized the position (longitude, latitude), speed, longitudinal and lateral acceleration, rotation speed and jerk, its assumed that the vehicle dynamics are completely characterized and some travel statistics are computed, such as average, standard deviation, maximum and minimum values.

## 4.2 Event Detector

Using the previously calculated vehicle dynamics, the different events are detected. It is considered as an event. the U-turn, curve, turn and lane change. The direction of the change, i.e., left or right, is also differentiated.

For the event detection, an algorithm like the one presented in [4] is used. Rotation speed is used as trigger value and when the ±δ is exceeded, it is defined that an event started tini instants ago, and will only end when it has tend consecutive instants below the limits as is shown in Fig.3.

<figure><img src="image_4.png" /><figcaption>Fig.3. Example for the event detector algorithm.</figcaption></figure>

To classify the event, additional information such as horizontal displacement, vehicle rotation during the entire event, number and type of bumps - time slots with a consecutive minimum duration at which the rotation speed module is greater than β. Taking all the information calculated, the classification is done using a decision tree with number of bumps, total rotation, type of bumps and horizontal displacement as decision factors.

## 4.3 Aggressiveness Classifier

In this last module, we intend to assign an aggressiveness index to each driver according to their behaviour. For this a level of aggressiveness is attributed to all the events detected in the previous module. It also analyses the percentage of time it was in excessive of speed, at this stage only comparing to the maximum limit applied to the fastest roads of the country.

The fuzzy logic system used has two linguistic variables, speed and square of the jerk. The first uses six membership functions and the second four. The output linguistic variable has three membership functions, assigning an aggressiveness index between 0 and 100. With the new event index, the driver's aggressiveness index is
updated through an exponential average, giving more importance to the most recent
events.

# 5 Results

For the initial tests of the system two different sets of trips were used. One of the sets consisted of three trips recorded on a previously defined circuit. The other corresponded to a set of trips recorded by three different drivers during different periods of time.

The previously defined circuit consists of twenty maneuvers previously identified with the desired output for each one. The driver was asked to perform three laps, two with normal behavior and one with aggressive behavior. For all of them, the system detected all the maneuvers performed, and obtains a seventy-five percent success in identifying the type of event in the first two cases and sixty percent in the most aggressive lap. In the aggressiveness classifier, for the two normal laps, the mean score was thirty while for the aggressive one the mean score was seventy-five points.

The other set of trips allowed to intensively teste the event detector, presenting
very favorable results. Two examples are shown in Fig.4 with their respective rotation
speed. As for the level of aggressiveness, the temporal variation of the behavior of the
two drivers for different coefficients applied in the exponential average used in the
formula are presented in Fig.5. The red line, the one with higher coefficient, shows
two distinct profiles, the left one with an inconsistent driving style, and the right one
with a steadier driving style. The other lines, with smaller coefficient, do not show
that difference because the “mean behavior” is similar in both drivers. Calibrating this
parameter allows to set whether to have an analysis with a higher weight in the most
recent history, or to allow older trips to significantly influence the index.

<figure><img src="image_5.png" /><figcaption>Fig.4. Event detector example – Turn to right and left curve followed by right curve.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>**Fig.5.** Evolution over time of aggressiveness level for two different drivers with three different coefficients in exponential average.</figcaption></figure>
## 6 Conclusion

This paper allows concluding that it is possible to characterize the dynamics of a vehicle using the sensors of the smartphone. The accuracy obtained in this process is sufficient to apply it as a basis in the development of applications with the purpose of monitoring and / or assessment of driving style.

Our tests have confirmed this claim by showing that different experiment can indeed capture distinct driving responses from the participants. These responses can be used by the aggressive classifier to determine a score for the driver during a trip. This leads us to believe that our system can indeed help the driver to improve his driving performance.

For future work, it is necessary to develop one algorithm to detect the smartphone movement during the trip and take actions for correcting the orientation of the acquired signals.

## References

1. World Health Organization. Road traffic injuries, May 2016. [Online] Accessed: 3006-2016. Available in: http://www.who.int/mediacentre/factsheets/fs358/en/.

2. Ana Belén Rodríguez González, Mark Richard Wilby, Juan José Vinagre Díaz and Carmen Sánchez Ávila. Modeling and detecting aggressiveness from driving signals. IEEE Transactions on intelligent transportation systems, 15(4):1419–1428, 2014.

3. Derick A Johnson and Mohan M Trivedi. Driving style recognition using a smartphone as a sensor platform. In 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC), pages 1609–1615. IEEE, 2011.

4. Dongyao Chen, Kyong-Tak Cho, Sihui Han, Zhizhuo Jin, and Kang G Shin. Invisible sensing of vehicle steering with smartphones. In Proceedings of the 13th Annual Int. Conference on Mobile Systems, Applications, and Services, pages 1–13. ACM, 2015.

5. G Zylius, V Vaitkus, and P Lengvenis. Driving style analysis using spectral features of accelerometer signals. Proceedings of 9th International Conference, ITELMS, 2014.

6. Haluk Eren, Semiha Makinist, Erhan Akin, and Alper Yilmaz. Estimating driving behavior by a smartphone. In Intelligent Vehicles Symposium (IV), 2012 IEEE, pages 234–239. IEEE, 2012.

7. Rui Araújo, Ângela Igreja, Ricardo de Castro, and Rui Esteves Araújo. Driving coach: A smartphone application to evaluate driving efficient patterns. In Intelligent Vehicles Symposium (IV), 2012 IEEE, pages 1005–1010. IEEE, 2012.

8. João GP Rodrigues, Mariana Kaiseler, Ana Aguiar, João P Silva Cunha, and João Barros. A mobile sensing approach to stress detection and memory activation for public bus drivers. IEEE Transactions on Intelligent Transportation Systems, 16(6):3294–3303, 2015.

9. Kuang-Shih Huang, Po-Jui Chiu, Hsin-Mu Tsai, Chih-Chung Kuo, Hui-Yu Lee, e Yu-Chiang Frank Wang. Redeye: Preventing collisions caused by red-light running scooters with smartphones. IEEE Transactions on Intelligent Transportation Systems;

10. Chenqi Wang e Hsin-Mu Tsai. Detecting urban traffic congestion with single vehicle. In 2013 Int. Conf. on Connected Vehicles and Expo (ICCVE), pages 233–240. IEEE, 2013.