# A Review for the Driving Behavior Recognition Methods Based on Vehicle Multisensor Information

Authors: Dengfeng Zhao; Yudong Zhong; Zhijun Fu; Junjian Hou; Mingyuan Zhao
Journal: Journal of Advanced Transportation
 (2022)

---

## Abstract

The frequent traffic accidents lead to a large number of casualties and large related financial losses every year; this serious state is owed to several factors; among those, driving behavior is one of the most imperative subjects to discuss. Driving behaviors mainly include behavior characteristics such as car-following, lane change, and risky driving behavior such as distraction, fatigue, or aggressive driving, which are of great help to various tasks in traffic engineering. An accurate and reliable method of driving behavior recognition is of great significance and guidance for vehicle driving safety. In this paper, the vehicle multisensor information, vehicle CAN bus data acquisition system, and typical feature extraction methods are summarized at first. And then, several driving behavior recognition models based on machine learning and deep learning are reviewed. Through a detailed analysis of the features of random forests, support vector machines, convolutional neural networks, and recurrent neural networks used to build driving behavior recognition models, the following findings are obtained: the driving behavior model constructed by traditional machine learning model is relatively mature but it is greatly affected by feature extraction, data scale, and model structure, which affects the accuracy of the final driving behavior recognition. Deep learning model based on a neural network has achieved high accuracy in identifying driving behavior, and it may gradually become the mainstream of constructing the driving behavior model with the development of big data, artificial intelligence technology, and computer hardware. Finally, this paper points out some content that needs to be further explored, to provide reference and inspiration for scholars in this field to continue to study the driving behavior recognition model in depth.


---

## Body

## 1. Introduction

Due to the uncertainty of driver’s state, the variability of road conditions, and the complexity of traffic environment, driving behavior has become an important factor affecting vehicle traffic safety. Driver behavior states include fatigue driving, distracted driving, and drunk driving. Vehicle states mainly include acceleration, deceleration, turning, lane change, and following. With the development of sensing technology and assisted driving technology, all kinds of high-performance sensors have been widely used in vehicles. Generally, they can be divided into vision class (such as camera) and timing class (such as speed, accelerometer). At the same time, with the popularization of CAN bus, on-board terminal, and Internet of vehicles technology, the use of on-board CAN bus to collect data of driver’s state and vehicle’s state under natural driving conditions has become the mainstream way, and massive and accurate on-board sensor data can provide strong support for driving behavior analysis. By integrating all kinds of data information and constructing models to identify or predict driving behaviors and taking corrective measures to abnormal and dangerous driving behaviors, it will be beneficial to improve the safety of vehicle driving and restrain the occurrence of road traffic accidents caused by drivers.Driving behavior recognition mainly includes regular behavior recognition and risky driving behavior recognition; regular driving behaviors have appeared in the process of driving such as car-following and regular lane change behavior, and risky driving behavior recognition refers to the phenomenon of nonstandard driving (distraction, etc), abnormal driving behavior of drivers (abrupt change, too closely car-following, etc.), and obvious changes in physiological parameters (fatigue, etc). Driving behavior recognition is a pattern recognition process, the selection of appropriate information fusion method is the basis of accurate identification of driving behavior, and the construction of a reasonable mathematical model is the key to accurate identification of driving behavior. Driving behavior with the external drive of multidimensional, complexity, randomness, and uncertainty, as well as road conditions, traffic conditions, vehicle status, and behavior factors such as the coupling correlation and multilevel characteristics, leads to the driving cause of chaos and nonlinear; how to construct the driving behavior recognition model has been the technical difficulty and research hotspot. There are many methods for constructing the recognition models of driving behavior (as shown in Figure1). Most of the existing models are traditional models based on empirical rule inference and mathematical statistical analysis, such as Hidden Markov Model (HMM) [1–3], Gaussian Mixture Model (GMM) [4–6], random forest (RF) [7–10], support vector machine (SVM) [11–13], Naive Bayes (NB) [14–16], Fuzzy Neural Network (FNN) [17], and K-Nearest Neighbor (KNN) [18, 19]. Although these traditional machine learning methods have been widely applied in the construction of a driving behavior model, there are still some limitations, such as relying on expert experience artificial extracting features, cannot consider driving time sequence and correlation, the higher dimensional instability problem such as large noise data is sensitive, and at the same time, the restriction of the model structure; these can lead to driving behavior recognition performance which is uneven, and the accuracy is low but difficult to increase greatly.Figure 1
The common recognition methods of driving behavior.With the progress of big data and artificial intelligence technologies, deep learning models with more hidden layers have attracted more attention from researchers. Deep learning model is a neural network model with multiple hidden layers and large-scale parameters such as neuron connection weights and thresholds. This kind of model can automatically extract the deep temporal and spatial features of multidimensional, complex, uncertain, incomplete, and coupled temporal data, which has great advantages in feature learning. Meanwhile, it can be organically integrated with the classifier to realize end-to-end data learning and significantly improve the recognition accuracy. Some researchers try to adopt deep neural network (DNN) [20, 21], convolutional neural network (CNN) [22–29], and recurrent neural network (RNN) [30–37] to construct a driving behavior recognition model, which has achieved good results. In recent years, with the widespread application of on-board sensors and CAN bus technology in cars, driving behavior data in the natural driving process have been collected and stored, which provides massive data samples for the construction of deep learning model, and the recognition methods of driving behavior are gradually evolving to deep learning model. However, deep learning algorithms require more data to establish a validate model, and the training time may be longer than traditional machine learning algorithms. Subsequently, with the establishment of lots of driving behavior recognition models including traditional models and deep learning models, it becomes difficult and confusing to choose a suitable one that meets the needs when developing a driving behavior recognition model.Previous reviews have addressed a subset of problems of different driving behaviors such as drowsiness [38], distraction [38], lane changing [39], and car-following [40], but their recommendations are limited to specific areas. For example, Kaplan et al. [38] have presented a review of driving behavior analysis for safe driving wherein driving behaviors cover drowsiness and distraction; they focus on identification technologies of drowsiness and distraction and not comparisons of driving behavior recognition models. Koesdwiady et al. [39] and Saifuzzaman and Zheng [40] have provided a review of specific driving behavior, without going into the details of different models and performance comparisons. None of these studies specifically focus on the comparison of driving behavior recognition methods. In the most related article to our work, Abou et al. [41] provide a review of four traditional machine learning algorithms such as SVM, NN, EL, and EB for driving behavior analysis. The authors make statistics on the use frequency of models and performance indicators, analyze the distribution of commonly used performance evaluation indicators, and briefly analyze the differences on performances of four models. Mozaffari et al. [42] present a survey only on different deep learning approaches such as RNN and CNN for autonomous vehicle behavior prediction; the authors mainly introduce the structure of models, and there is no comparison with the performance and characteristics of traditional machine learning models. However, to the best of our knowledge, systematic and comparative review papers focusing on introducing and surveying the difference of models including machine learning-based methods and deep learning-based methods are not available in this field yet.Therefore, we present a review of such studies that is aimed at summarizing the difference between traditional machine learning and deep learning-based approaches applied in driving behavior studies in recent years. In addition, some other contents are also summarized: firstly, the common data processing and information fusion methods are introduced based on the CAN bus data of vehicle operation. Then, several driving behavior recognition models based on machine learning and deep learning are reviewed. Finally, the advantages and disadvantages of several driving behavior recognition models are compared and analyzed, and the corresponding conclusions and prospects are given. The purpose of this paper is to provide a reference for the researchers in the field of driving behavior recognition and theoretical support for the development of the traffic safety field.The main contributions of this paper are given as follows:(i)
Elaboration on the recent research on driving behavior recognition model and algorithm, to help researchers develop optimal methods and establish driving behavior recognition models with higher accuracy and performance(ii)
Understanding the characteristics of the information of vehicle sensors, help researchers to find the reason sensor data, to know about the main feature processing methods and to design vehicle multisensor data sampling system for driving behavior data(iii)
Establish gaps in the recognition model of the current state of the art that can be addressed by future driving behavior research, and an overview of research opportunities in the future is also providedThe structure of the paper is as follows: the vehicle sensor information and vehicle CAN bus data acquisition system is introduced in Section 2. The research progress of several driving behavior recognition models is reviewed in Section 3. At last, the paper gives some conclusions and prospects.


## 2. The Information of Vehicle Sensors

With the development of intelligent, networked, and electric technology of vehicles, all kinds of on-board sensor devices such as GPS, gyroscope, camera, radar, and CAN bus have been widely used in modern vehicles. These devices can not only provide reliable information input for controlling the safe operation of the vehicles but also accurately record the behavior state information of the driver and the vehicle. This vehicle sensor information can be divided into six categories according to the dimensions of driver and vehicle: driver status information, driver control vehicle information, vehicle control state information, vehicle driving state information, and road environment state information. This state information contains abundant characteristic information of driving behavior, the data information with high correlation with driving behavior characteristics can be mined, and then, the accurate identification of different driving behaviors can be realized by the information fusion method. Various types of data information are shown in Table1, and the typical on-board sensors and CAN bus structure are shown in Figure 2.Table 1
The main vehicle sensor information.
The information of vehicle sensorsMain dataData acquisition device or sensorDriversStatus information of driversCamera data on fatigue, distraction, drunk driving, or ill drivingVideo surveillance systemControl information of driversAcceleration pedal opening, brake pedal opening, hand brake, steering wheel angle, etc.Vehicle-mounted terminals such as dashcamVehiclesControl status informationSwitch signal, torque, speed, voltage, current, etc.Vehicle data recorderVehicle driving informationSpeed, acceleration, turn signal, brake signal, location, distance, etc.Vehicle data recorder, gyroscope, GPS units, radarRoadsStatus information of roadTrack, lane line, signal lamp, intersection, zebra and other dataCamerasEnvironmentEnvironment informationTemperature, snow, rainVehicle data recorderFigure 2
The schematic diagram of onboard CAN bus structure.All kinds of on-board sensors have different perception results for drivers and vehicle states. Mining on-board multisensor data with high correlation with driving behavior characteristics is the key to accurately identifying drivers’ driving behaviors. The relevant features of information from data for driving behavior recognition can be extracted by using the feature extraction method [43]. The method can obtain new feature representation by mapping original features into the new space, including feature reduction and feature selection. The commonly used feature extraction methods are shown in Figure 3.Figure 3
The feature extraction method.Feature dimension reduction is to reduce the dimension of data by mapping data points from a high-dimensional space to a low-dimensional space using a mapping method. Feature selection is a method to find the optimal feature subset by eliminating irrelevant or redundant features, including filter, wrapper, and embedded. Filter directly selects features from the data set, while wrapper selects subsets from the initial feature set to train and optimize the learner until the best subset is selected. The performance of this method is better than filter, but it requires multiple training of the learner, and the calculation cost is high. Embedded is an automatic feature selection in the process of training the learner by closely combining the feature selection process with the training of the learner. For example, the convolutional neural network can automatically extract the deep spatiotemporal features of the data through convolution and pooling operations and seamlessly connect with the classifiers.


## 3. The Recognition Model of Driving Behavior

Driving behavior recognition is a pattern recognition process. Accurate identification or prediction of driving behavior plays an important role in the development of high-performance driver assistance system and improvement of driver-caused accidents. Driving behavior recognition models can be generally divided into two categories: one is the traditional machine learning model, which is usually based on empirical rule reasoning and mathematical statistical analysis methods, such as HMM, GMM, SVM, RF, NB, FNN, and KNN. Among them, RF and SVM are widely used methods for constructing the recognition models of driving behavior. The other is the deep learning model. Deep learning model is a neural network model with multiple hidden layers and large-scale neuron connection weights, thresholds, and other parameters, including deep neural network (DNN), convolutional neural network model (CNN), and recurrent neural network model (RNN). CNN and RNN can automatically extract the deep spatiotemporal features of driving behavior data and integrate feature extraction and recognition prediction into a model to achieve end-to-end learning with high recognition accuracy. They are widely used in the construction of the recognition models of driving behavior. Deep learning models become popular with the development of big data sampling technology and computer hardware. The basic principles and characteristics of random forest, support vector machine, convolutional neural network, and recurrent neural network, and their research situations are summarized as follows.


### 3.1. The Random Forest

Random forest model is a classification and regression method based on a decision tree (the principle is shown in Figure4). In this method, the multiple training sets are generated by constructing bagging integrated random sampling with putting back. A decision tree is constructed for each training set, and the output results are determined by comprehensive voting of these decision trees. In the training process of the decision tree, the optimal attribute is selected from several randomly selected attributes for node segmentation, so as to reduce the variance of the model and avoid overfitting. This method is suitable for processing high-dimensional multifeature data and has the characteristics of fast learning and high classification accuracy [44].Figure 4
The principle of random forest method.Random forest has been widely used in many fields and achieved great success. Some researchers used the random forest model to construct the recognition model of driving behavior. In terms of fatigue driving behavior recognition, McDonald et al. [45] applied the random forest model to design an algorithm based on steering wheel angle data to detect fatigue driving behavior. Mårtensson et al. [46] pointed out that the random forest algorithm is superior to other mainstream classifiers in identifying tired driving, with good accuracy and robustness and high computational efficiency, and can adapt to the training environment of small samples. Cai et al. [47] adopted the random forest model to identify the fatigue driving behavior based on vehicle running data through manual extraction of features, with an accuracy of only 78.5%. Dong and Lin [48] used the random forest model to identify fatigue driving behavior based on facial image data, with an accuracy of 91%. Some scholars also use random forest to build the recognition models of distracted driving behavior, such as Ahangari et al. [7] used random forest classification model to identify 6 common distracted driving behaviors, with an accuracy rate of 76.5%. Yao et al. [49] established a random forest model to identify distracted driving behavior based on machine vision and behavioral data, with a scoring accuracy of 90%. It can be seen from the above literature that the accuracy of driving behavior recognition varies widely, which is closely related to sensor data fusion and feature extraction. Some scholars also use the random forest model to identify driving behaviors such as car-following [50, 51], lane change [8], and being followed [9] or to evaluate driving safety risks [10] and identify drivers [52, 53], driving style [54], and driving posture [55]. Other researchers [56] combined random forest and other methods to construct a driving behavior recognition model.In conclusion, using the random forest method to construct the recognition model of driving behavior has been widely used. This method can be applied to the identification of various driving behaviors, and the recognition accuracy is greater than 75%. However, the recognition results fluctuate greatly under the influence of different application scenarios. Now, its application fields and corresponding advantages and disadvantages are analyzed as shown in Table2.Table 2
The characteristics analysis of the random forest model.
Types of driving behaviorApplicationsAdvantages and disadvantagesFatigue driving [45–48]Used for identification of fatigue driving behavior with high accuracy fluctuationAdvantages: it can be used for the identification of various driving behaviors. The algorithm is simple and easy to implement, the calculation cost is small, and it is not easy to appear overfitting. It is suitable for the classification of high-dimensional and multifeature small sample data, and the accuracy is usually more than 75%.Disadvantages: big data applications are flawed and do not handle unbalanced data well. Manual intervention is needed to extract features, and the recognition results vary wide.Distracted driving [7] [49]Used for distracted driving behavior recognitionCar-following [50] [51]Used for identification of following driving behaviorLane change [8] [56]Used for lane change driving behavior recognitionOthers [9, 10] [52] [53–55]Used for acceleration and deceleration, turning, lane change, being followed, and driver, driving style, driving posture recognition


### 3.2. Support Vector Machines

Support vector machine is a binary supervised learning model, which was first used to solve the binary problem of linearly fractionable data. Its main goal is to find a hyperplane that can best divide the data set into desired classes. Its basic principle is to find an optimal hyperplane that meets the requirements of data classification interval, so that the maximum distance between the hyperplane and the two types of sample points can be achieved under the condition that the classification accuracy is ensured [57]. As shown in Figure 5, H is the optimal hyperplane, which maximizes the classification interval margin (the distance from H1 to H2) in the set of hyperplanes for data classification, where the distance from H1 to H is equal to the distance from H2 to H. Points on H1 and H2 are called support vectors. For the nonlinearity problem, the hyperplane H does not exist in the original space; SVM transforms the nonlinearity inseparability in the low-dimensional space into linear separability in the high-dimensional space by importing the kernel function and constructs the optimal hyperplane in the high-dimensional space to realize the nonlinear classification.Figure 5
The hyperplane of the support vector machine.SVM is characterized by the strong learning ability of small samples and good model generalization performance and has been used by some scholars to build the recognition model of driving behavior. For example, Wang et al. [11] used the SVM model to identify the driving behavior in the process of car-following, with an accuracy of 80.8%. Savaş and Becerikli [58] established the fatigue driving behavior recognition model using SVM, achieving the fatigue driving recognition rate of 97.93%. However, You et al. [59] used SVM to establish the identification model of fatigue driving behavior, and the accuracy was only 80.83%. Amsalu et al. [12] constructed a SVM model for identifying the driving intentions at intersections and estimated driver intentions by using the data of natural driving states of vehicles, with the recognition accuracy reaching 97%. Tomar and Verma [60] established a SVM model to identify the driving behavior of lane change and car-following by preprocessing and constructing features of real vehicle track sample data, with an accuracy of 98.41%. Liao et al. [61] believed that the support vector machine model has certain advantages to identify distracted driving behaviors after comparing the performance of different models, but there are problems such as large fluctuation of recognition results and unstable performance. At the same time, they pointed out that the deep learning model has high accuracy to identify distracted driving behaviors. Some researchers improved the performance of recognition models by combining SVM with other methods. For example, Yang et al. [13] classified and evaluated the safety levels of different driving behaviors by combining the SVM and decision tree model. Wang et al. [62] proposed a method for the classification of fatigue driving behavior by using CNN to extract features and then SVM, with an accuracy of 94%.To sum up, the SVM method to build the recognition model of driving behavior has been widely used. This method can be used to identify various driving behaviors such as fatigue, distraction, following, and lane change, and the recognition accuracy is usually above 80%. The analysis of its application fields and corresponding advantages and disadvantages is shown in Table3.Table 3
The characteristics analysis of SVM.
Types of driving behaviorApplicationsAdvantages and disadvantagesFatigue driving [58, 59] [62]Used to identify tired driving behaviorAdvantages: Suitable for high-dimensional feature small sample classification problem, the model generalization performance is good. The computer has low memory usage and fast computing speed. Kernel function has better effect on nonlinear decision making.Disadvantages: SVM relies on a limited number of support vectors for classification and is sensitive to noise and abnormal data, so it is not suitable for nonlinear classification of large-scale samples. At the same time, feature extraction and classification learning are separated, and the accuracy of recognition is greatly affected by feature extraction.Distracted driving [61, 63, 64]Used to identify distracted driving behaviorCar-following [11, 60]Used to identify car-following driving behaviorChange lanes [65, 66]Used to identify lane change driving behaviorOthers [12, 13, 67, 68]Used to identify sharp deceleration, sharp steering, lane departure, and other abnormal driving behavior


### 3.3. Convolutional Neural Network

As a representative of deep learning model, the convolutional neural network is a feedforward neural network with a deep structure that includes convolutional computation, which is proposed by biological receptive field mechanism, and generally consists of input layer, convolutional layer, pooling layer, full connection layer, and output layer (as shown in Figure6). The function of the convolution layer is to extract the features of a local region, and the function of the convergence layer is to select features and reduce the number of features. CNN performs feature extraction on input information by combining multiple convolutional layers and aggregation layers and then realizes mapping with output targets at the full connection layer. Each convolutional layer contains multiple feature mappings. Through multilayer processing, the initial shallow feature representation is gradually transformed into a deep feature representation. CNN adopts convolution instead of full connection, which has the properties of local connection, weight sharing, and convergence. Compared with a full connection feedforward neural network, it can reduce the number of hyperparameters on a large scale.Figure 6
The structure of the convolutional neural network.CNN took the lead in a large number of successful applications in the fields of image recognition and speech processing [69, 70]. Xing et al. [23] built a CNN model for fatigue driving recognition to automatically extract features from drivers’ facial images and identify fatigue state, with an accuracy rate of 87.5%. Jabbar et al. [71] built the CNN model to automatically extract the depth features of the driver’s face in the two states of wearing glasses or not to identify the fatigue state of the driver, with a recognition accuracy of 88% and 85%, respectively. At the same time, there are also a large number of successful cases to identify distracted driving behaviors by extracting driver attitude images through CNN. For example, Eraqi et al. [24] established a CNN model and used driver attitude images to identify distracted driving behaviors, with recognition accuracy reaching 90%. Baheti et al. [25] established the distracted driving recognition CNN model in the improved convolutional neural network VGG-16 model framework, and the recognition accuracy reached 95.54%. Majdi et al. [72] applied CNN and the decision tree algorithm to construct DriveNet, a behavior recognition model of distracted driving, with recognition accuracy reaching 95%. Liu et al. [73] jointly applied ResNet50, Inception V3, and Xception for model pretraining and adopted transfer learning to extract driving behavior characteristics to construct a hybrid CNN model for identifying distracted driving behavior, with recognition accuracy reaching 96.74%. In recent years, many researchers have developed several new convolutional network models based on the CNN basic model, such as VGG, ResNet, and InceptionNet, and began to apply them to driving behavior recognition. For example, Srinivasan et al. [74] made a comparative analysis of the recognition effects of CNN models such as VGG16, ResNet, and Xception in the recognition of distracted driving behavior and analyzed that ResNet convolutional neural network model had the highest recognition accuracy. Some researchers also built CNN models to extract vehicle state data features to predict lane changing and following driving behaviors. For example, Azadani and Boukerche [26] designed a continuous-time convolution network system architecture model to identify lane-changing behaviors, with a recognition accuracy of 95.3%. Xie et al. [75] constructed a CNN driving behavior recognition model to recognize lane change, braking, and other driving behaviors with an accuracy of 87.67%, which is better than KNN and FR. Lee et al. [76] adopted CNN for lane change driving behavior recognition, with an average accuracy of 89.87%, better than HMM (81.14%). To carry out engineering applications, some researchers carried out lightweight designs on the CNN model. For example, Qin et al. [77] constructed a CNN lightweight model for the identification of distracted driving, and the recognition accuracy reached 95.59%. At present, CNN tends to use a smaller convolutional kernel, deeper structure, and fewer aggregation layers and gradually develop into fully convolutional network.In conclusion, the driving behavior recognition model constructed based on CNN has achieved significant application effects in the recognition of various driving behaviors such as fatigue, distraction, and lane change, with high accuracy of more than 85%. The application fields and corresponding advantages and disadvantages are summarized in Table4.Table 4
The characteristic analysis of CNN.
Types of driving behaviorApplicationsAdvantages and disadvantagesFatigue driving [23, 69, 70, 78–82]Used to identify tired driving behaviorAdvantages: suitable for big data and high-dimensional data samples, reduce the number of parameters through weight sharing and aggregation, automatic feature extraction, and organic integration with classifier to achieve end-to-end learning, high accuracy, and antinoise interference.Disadvantages: restricted by the model structure, the input and output dimensions cannot be arbitrarily changed, requiring the sample length to be fixed. It is difficult to process time series data, and overfitting is easy to occur when the data sample size is large.Distracted driving [24, 25, 72–74, 77, 83–85]Used to identify distracted driving behaviorChange lanes [26] [75] [76] [86] [87] [88]Used to identify lane change driving behaviorOthers [83] [89] [90] [91] [92]Used to identify the driving behavior, driver, driving style, and so on


### 3.4. Recurrent Neural Network

Recurrent neural network (RNN) is a new network structure based on deep neural network, which establishes association learning on time series data through cyclic calculation of the weight of hidden layers along the timeline and ensures that the network has a certain short-term memory ability. The hidden layer neuron of RNN can not only accept the information of other neurons but also accept its information, forming a network structure with loops, as shown in Figure7. By using self-feedback neurons, RNN can process time-series data of any length and theoretically approximate any nonlinear dynamic system. The internal structure of the simple cyclic neural network is shown in Figure 8. X and O represent the input vector and output vector, respectively; U and V are the weight matrix from the input layer to the hidden layer and from the hidden layer to the output layer, respectively; and W is the weight matrix between the hidden layer at different moments.Figure 7
The principle of recurrent neural network.Figure 8
The schematic diagram of RNN neuron connection.The driver state and vehicle state collected by vehicle sensors are high-dimensional nonlinear data with time-series relationships, and RNN has an excellent performance in processing such data. Relevant scholars have carried out a large number of explorations using RNN to construct driving behavior models and achieved good results. For example, Li et al. [31] and Wei et al. [93] used RNN to predict lane change behavior, and the accuracy reached 96% and 93.5%, respectively. EdDoughmi et al. [32] identified fatigued driving behaviors by inputting the collected video stream data of drivers’ faces into the 3D-RNN model established, with an accuracy of 92%. To solve the problem of long-term dependence caused by gradient explosion and gradient disappearance of RNN, researchers have developed novel RNN based on gated mechanisms, such as Long Short-Term Memory Network (LSTM) and Gated Recurrent Unit (GRU) network. LSTM solves the problem of long-range dependence by introducing an input gate, forgetting gate, and output gate to control the path of information transmission. Some researchers used LSTM to research the identification of driving behavior. For example, Griesbach et al. [34] applied LSTM to predict lane change behavior, and the accuracy reached 90%. Zhang et al. [94] constructed a DeepConvLSTM model based on RNN framework to identify different driving behaviors by introducing LSTM gating mechanism, with an accuracy of 95.19%, far better than the random forest model (87.39%). Some researchers also use LSTM to identify tired driving behavior and distracted driving behavior. For example, Yarlagadda et al. [95] used LSTM to predict driver fatigue state, and the recognition accuracy reached 97.25%. Sun et al. [96] adopted LSTM model to identify distracted driving behavior, and the accuracy reached 91%, better than SVM. GRU is different from LSTM in principle. It solves the problem of long-term dependence by setting a gate to control the balance between input and forgetting. Some researchers have also applied GRU to identify driving behaviors, such as Tang et al. [97] who built a DeepConvGRU model to identify driving behaviors by introducing GRU gating mechanism. The accuracy reached 95.08%. Yan et al. [98] constructed a GRU model to predict lane change behavior, and the accuracy reached 94.76%, better than LSTM (93.47%).To give full play to the advantages of various deep learning models, some researchers combine RNN and other models to improve the performance of driving behavior recognition. For example, Mafeni et al. [99] jointly applied BI-LSTM and InceptionV3 CNNs to construct a distracted driving model (as shown in Figure 9) and identified distracted driving behaviors through driving posture pictures, with an accuracy of 93.1%. Wollmer et al. [33] used RNN combined with random forest FR to identify distracted driving behavior, and the accuracy reached 95%. Zhang et al. [100] propose a composite model that combines a multiscale convolutional neural network (MSCNN) and bi-LSTM to identify driving behaviors through vehicle track data. The highest accuracy was 97.75%. Zhang et al. [101] built a vehicle driving behavior recognition model based on joint data enhancement with a multiview convolutional neural network, and the model recognition accuracy was better than CNN, RNN, LSTM, CNN+LSTM, and other models. Xing et al. [102] jointly applied LSTM and BI-RNN to establish a lane change behavior recognition model; the accuracy reached 96.1% and shortens the prediction time cycle.Figure 9
The schematic diagram of the union model [99].Other researchers have further improved the performance of recurrent neural networks by increasing their depth and using them in driving behavior recognition research, such as Stacked Recurrent Neural Networks (SRNN) [103–109], Bidirectional Recurrent Neural Network (Bi-RNN) [100, 102, 110–112], Recursive Neural Networks [113], and Graph Neural Network (GNN) [114, 115], which have been applied in driving behavior recognition and achieved high recognition accuracy.The above studies are only some typical application cases. There are many successful application cases of RNN in the construction of the driving behavior recognition model [116–124]. It can be seen that RNN can be used for all kinds of driving behavior recognition, is suitable for high-dimensional and big data sample learning, and can extract deep temporal and spatial features with an accuracy rate of more than 90%. However, a large sample size of the data is required for training RNN.Although the above model has been proved to be effective in driving behavior recognition by some studies, it does not perform well in all driving behavior recognition tasks. In this problem, there is no general model for driving behavior recognition, and the performance of the model depends heavily on the task for which it is applicable. Therefore, researchers not only need to understand the characteristics of candidate models but also need to understand driving scenarios and conduct corresponding tests to correctly select models and effectively apply them to practical driving behavior recognition tasks.The test part mainly includes the following three types: real vehicle test, simulation verification, and vehicle in-loop test. The advantages of the real vehicle test are real and objective, but the disadvantages are that the high-risk scene data cannot be obtained and the manpower and material resources are heavily invested. The advantages of simulation verification are easy to test various complex working conditions, with high efficiency, but the scene cannot be completely close to the real situation. Vehicle in-loop test is a relatively popular method at present, which can integrate the advantages of real vehicle tests and simulation tests and can not only reproduce all kinds of actual scenes but also exhaust high-risk and complex scenes. After the driving behavior scene test, massive test data samples can be obtained by the driving behavior scene test, which is imported into the deep learning model for training and computation after data preprocessing, to complete the recognition of driving behavior.All in all, the deep learning models could be mainstream model for its high accuracy and stable robustness if a lot of data set are available to train the models. The advantages of the deep learning model are as follows: strong learning ability and can be transplanted to the framework of many platforms; the neural network of deep learning has a large number of layers and a wide width. Theoretically, it can be mapped to any function, so it can solve very complex problems. Deep learning is highly data-dependent, and the larger the amount of data, the better its performance. The disadvantage is that the model design of deep learning is very complex and requires a large amount of data and computational power, so the cost is very high. Deep learning has high requirements on computer hardware, and ordinary CPU can no longer meet the requirements of deep learning. However, with the development of big data, computing technology, and hardware, the application of deep learning in driving behavior will be greatly developed and its application will be more and more extensive.


## 3.1. The Random Forest

Random forest model is a classification and regression method based on a decision tree (the principle is shown in Figure4). In this method, the multiple training sets are generated by constructing bagging integrated random sampling with putting back. A decision tree is constructed for each training set, and the output results are determined by comprehensive voting of these decision trees. In the training process of the decision tree, the optimal attribute is selected from several randomly selected attributes for node segmentation, so as to reduce the variance of the model and avoid overfitting. This method is suitable for processing high-dimensional multifeature data and has the characteristics of fast learning and high classification accuracy [44].Figure 4
The principle of random forest method.Random forest has been widely used in many fields and achieved great success. Some researchers used the random forest model to construct the recognition model of driving behavior. In terms of fatigue driving behavior recognition, McDonald et al. [45] applied the random forest model to design an algorithm based on steering wheel angle data to detect fatigue driving behavior. Mårtensson et al. [46] pointed out that the random forest algorithm is superior to other mainstream classifiers in identifying tired driving, with good accuracy and robustness and high computational efficiency, and can adapt to the training environment of small samples. Cai et al. [47] adopted the random forest model to identify the fatigue driving behavior based on vehicle running data through manual extraction of features, with an accuracy of only 78.5%. Dong and Lin [48] used the random forest model to identify fatigue driving behavior based on facial image data, with an accuracy of 91%. Some scholars also use random forest to build the recognition models of distracted driving behavior, such as Ahangari et al. [7] used random forest classification model to identify 6 common distracted driving behaviors, with an accuracy rate of 76.5%. Yao et al. [49] established a random forest model to identify distracted driving behavior based on machine vision and behavioral data, with a scoring accuracy of 90%. It can be seen from the above literature that the accuracy of driving behavior recognition varies widely, which is closely related to sensor data fusion and feature extraction. Some scholars also use the random forest model to identify driving behaviors such as car-following [50, 51], lane change [8], and being followed [9] or to evaluate driving safety risks [10] and identify drivers [52, 53], driving style [54], and driving posture [55]. Other researchers [56] combined random forest and other methods to construct a driving behavior recognition model.In conclusion, using the random forest method to construct the recognition model of driving behavior has been widely used. This method can be applied to the identification of various driving behaviors, and the recognition accuracy is greater than 75%. However, the recognition results fluctuate greatly under the influence of different application scenarios. Now, its application fields and corresponding advantages and disadvantages are analyzed as shown in Table2.Table 2
The characteristics analysis of the random forest model.
Types of driving behaviorApplicationsAdvantages and disadvantagesFatigue driving [45–48]Used for identification of fatigue driving behavior with high accuracy fluctuationAdvantages: it can be used for the identification of various driving behaviors. The algorithm is simple and easy to implement, the calculation cost is small, and it is not easy to appear overfitting. It is suitable for the classification of high-dimensional and multifeature small sample data, and the accuracy is usually more than 75%.Disadvantages: big data applications are flawed and do not handle unbalanced data well. Manual intervention is needed to extract features, and the recognition results vary wide.Distracted driving [7] [49]Used for distracted driving behavior recognitionCar-following [50] [51]Used for identification of following driving behaviorLane change [8] [56]Used for lane change driving behavior recognitionOthers [9, 10] [52] [53–55]Used for acceleration and deceleration, turning, lane change, being followed, and driver, driving style, driving posture recognition


## 3.2. Support Vector Machines

Support vector machine is a binary supervised learning model, which was first used to solve the binary problem of linearly fractionable data. Its main goal is to find a hyperplane that can best divide the data set into desired classes. Its basic principle is to find an optimal hyperplane that meets the requirements of data classification interval, so that the maximum distance between the hyperplane and the two types of sample points can be achieved under the condition that the classification accuracy is ensured [57]. As shown in Figure 5, H is the optimal hyperplane, which maximizes the classification interval margin (the distance from H1 to H2) in the set of hyperplanes for data classification, where the distance from H1 to H is equal to the distance from H2 to H. Points on H1 and H2 are called support vectors. For the nonlinearity problem, the hyperplane H does not exist in the original space; SVM transforms the nonlinearity inseparability in the low-dimensional space into linear separability in the high-dimensional space by importing the kernel function and constructs the optimal hyperplane in the high-dimensional space to realize the nonlinear classification.Figure 5
The hyperplane of the support vector machine.SVM is characterized by the strong learning ability of small samples and good model generalization performance and has been used by some scholars to build the recognition model of driving behavior. For example, Wang et al. [11] used the SVM model to identify the driving behavior in the process of car-following, with an accuracy of 80.8%. Savaş and Becerikli [58] established the fatigue driving behavior recognition model using SVM, achieving the fatigue driving recognition rate of 97.93%. However, You et al. [59] used SVM to establish the identification model of fatigue driving behavior, and the accuracy was only 80.83%. Amsalu et al. [12] constructed a SVM model for identifying the driving intentions at intersections and estimated driver intentions by using the data of natural driving states of vehicles, with the recognition accuracy reaching 97%. Tomar and Verma [60] established a SVM model to identify the driving behavior of lane change and car-following by preprocessing and constructing features of real vehicle track sample data, with an accuracy of 98.41%. Liao et al. [61] believed that the support vector machine model has certain advantages to identify distracted driving behaviors after comparing the performance of different models, but there are problems such as large fluctuation of recognition results and unstable performance. At the same time, they pointed out that the deep learning model has high accuracy to identify distracted driving behaviors. Some researchers improved the performance of recognition models by combining SVM with other methods. For example, Yang et al. [13] classified and evaluated the safety levels of different driving behaviors by combining the SVM and decision tree model. Wang et al. [62] proposed a method for the classification of fatigue driving behavior by using CNN to extract features and then SVM, with an accuracy of 94%.To sum up, the SVM method to build the recognition model of driving behavior has been widely used. This method can be used to identify various driving behaviors such as fatigue, distraction, following, and lane change, and the recognition accuracy is usually above 80%. The analysis of its application fields and corresponding advantages and disadvantages is shown in Table3.Table 3
The characteristics analysis of SVM.
Types of driving behaviorApplicationsAdvantages and disadvantagesFatigue driving [58, 59] [62]Used to identify tired driving behaviorAdvantages: Suitable for high-dimensional feature small sample classification problem, the model generalization performance is good. The computer has low memory usage and fast computing speed. Kernel function has better effect on nonlinear decision making.Disadvantages: SVM relies on a limited number of support vectors for classification and is sensitive to noise and abnormal data, so it is not suitable for nonlinear classification of large-scale samples. At the same time, feature extraction and classification learning are separated, and the accuracy of recognition is greatly affected by feature extraction.Distracted driving [61, 63, 64]Used to identify distracted driving behaviorCar-following [11, 60]Used to identify car-following driving behaviorChange lanes [65, 66]Used to identify lane change driving behaviorOthers [12, 13, 67, 68]Used to identify sharp deceleration, sharp steering, lane departure, and other abnormal driving behavior


## 3.3. Convolutional Neural Network

As a representative of deep learning model, the convolutional neural network is a feedforward neural network with a deep structure that includes convolutional computation, which is proposed by biological receptive field mechanism, and generally consists of input layer, convolutional layer, pooling layer, full connection layer, and output layer (as shown in Figure6). The function of the convolution layer is to extract the features of a local region, and the function of the convergence layer is to select features and reduce the number of features. CNN performs feature extraction on input information by combining multiple convolutional layers and aggregation layers and then realizes mapping with output targets at the full connection layer. Each convolutional layer contains multiple feature mappings. Through multilayer processing, the initial shallow feature representation is gradually transformed into a deep feature representation. CNN adopts convolution instead of full connection, which has the properties of local connection, weight sharing, and convergence. Compared with a full connection feedforward neural network, it can reduce the number of hyperparameters on a large scale.Figure 6
The structure of the convolutional neural network.CNN took the lead in a large number of successful applications in the fields of image recognition and speech processing [69, 70]. Xing et al. [23] built a CNN model for fatigue driving recognition to automatically extract features from drivers’ facial images and identify fatigue state, with an accuracy rate of 87.5%. Jabbar et al. [71] built the CNN model to automatically extract the depth features of the driver’s face in the two states of wearing glasses or not to identify the fatigue state of the driver, with a recognition accuracy of 88% and 85%, respectively. At the same time, there are also a large number of successful cases to identify distracted driving behaviors by extracting driver attitude images through CNN. For example, Eraqi et al. [24] established a CNN model and used driver attitude images to identify distracted driving behaviors, with recognition accuracy reaching 90%. Baheti et al. [25] established the distracted driving recognition CNN model in the improved convolutional neural network VGG-16 model framework, and the recognition accuracy reached 95.54%. Majdi et al. [72] applied CNN and the decision tree algorithm to construct DriveNet, a behavior recognition model of distracted driving, with recognition accuracy reaching 95%. Liu et al. [73] jointly applied ResNet50, Inception V3, and Xception for model pretraining and adopted transfer learning to extract driving behavior characteristics to construct a hybrid CNN model for identifying distracted driving behavior, with recognition accuracy reaching 96.74%. In recent years, many researchers have developed several new convolutional network models based on the CNN basic model, such as VGG, ResNet, and InceptionNet, and began to apply them to driving behavior recognition. For example, Srinivasan et al. [74] made a comparative analysis of the recognition effects of CNN models such as VGG16, ResNet, and Xception in the recognition of distracted driving behavior and analyzed that ResNet convolutional neural network model had the highest recognition accuracy. Some researchers also built CNN models to extract vehicle state data features to predict lane changing and following driving behaviors. For example, Azadani and Boukerche [26] designed a continuous-time convolution network system architecture model to identify lane-changing behaviors, with a recognition accuracy of 95.3%. Xie et al. [75] constructed a CNN driving behavior recognition model to recognize lane change, braking, and other driving behaviors with an accuracy of 87.67%, which is better than KNN and FR. Lee et al. [76] adopted CNN for lane change driving behavior recognition, with an average accuracy of 89.87%, better than HMM (81.14%). To carry out engineering applications, some researchers carried out lightweight designs on the CNN model. For example, Qin et al. [77] constructed a CNN lightweight model for the identification of distracted driving, and the recognition accuracy reached 95.59%. At present, CNN tends to use a smaller convolutional kernel, deeper structure, and fewer aggregation layers and gradually develop into fully convolutional network.In conclusion, the driving behavior recognition model constructed based on CNN has achieved significant application effects in the recognition of various driving behaviors such as fatigue, distraction, and lane change, with high accuracy of more than 85%. The application fields and corresponding advantages and disadvantages are summarized in Table4.Table 4
The characteristic analysis of CNN.
Types of driving behaviorApplicationsAdvantages and disadvantagesFatigue driving [23, 69, 70, 78–82]Used to identify tired driving behaviorAdvantages: suitable for big data and high-dimensional data samples, reduce the number of parameters through weight sharing and aggregation, automatic feature extraction, and organic integration with classifier to achieve end-to-end learning, high accuracy, and antinoise interference.Disadvantages: restricted by the model structure, the input and output dimensions cannot be arbitrarily changed, requiring the sample length to be fixed. It is difficult to process time series data, and overfitting is easy to occur when the data sample size is large.Distracted driving [24, 25, 72–74, 77, 83–85]Used to identify distracted driving behaviorChange lanes [26] [75] [76] [86] [87] [88]Used to identify lane change driving behaviorOthers [83] [89] [90] [91] [92]Used to identify the driving behavior, driver, driving style, and so on


## 3.4. Recurrent Neural Network

Recurrent neural network (RNN) is a new network structure based on deep neural network, which establishes association learning on time series data through cyclic calculation of the weight of hidden layers along the timeline and ensures that the network has a certain short-term memory ability. The hidden layer neuron of RNN can not only accept the information of other neurons but also accept its information, forming a network structure with loops, as shown in Figure7. By using self-feedback neurons, RNN can process time-series data of any length and theoretically approximate any nonlinear dynamic system. The internal structure of the simple cyclic neural network is shown in Figure 8. X and O represent the input vector and output vector, respectively; U and V are the weight matrix from the input layer to the hidden layer and from the hidden layer to the output layer, respectively; and W is the weight matrix between the hidden layer at different moments.Figure 7
The principle of recurrent neural network.Figure 8
The schematic diagram of RNN neuron connection.The driver state and vehicle state collected by vehicle sensors are high-dimensional nonlinear data with time-series relationships, and RNN has an excellent performance in processing such data. Relevant scholars have carried out a large number of explorations using RNN to construct driving behavior models and achieved good results. For example, Li et al. [31] and Wei et al. [93] used RNN to predict lane change behavior, and the accuracy reached 96% and 93.5%, respectively. EdDoughmi et al. [32] identified fatigued driving behaviors by inputting the collected video stream data of drivers’ faces into the 3D-RNN model established, with an accuracy of 92%. To solve the problem of long-term dependence caused by gradient explosion and gradient disappearance of RNN, researchers have developed novel RNN based on gated mechanisms, such as Long Short-Term Memory Network (LSTM) and Gated Recurrent Unit (GRU) network. LSTM solves the problem of long-range dependence by introducing an input gate, forgetting gate, and output gate to control the path of information transmission. Some researchers used LSTM to research the identification of driving behavior. For example, Griesbach et al. [34] applied LSTM to predict lane change behavior, and the accuracy reached 90%. Zhang et al. [94] constructed a DeepConvLSTM model based on RNN framework to identify different driving behaviors by introducing LSTM gating mechanism, with an accuracy of 95.19%, far better than the random forest model (87.39%). Some researchers also use LSTM to identify tired driving behavior and distracted driving behavior. For example, Yarlagadda et al. [95] used LSTM to predict driver fatigue state, and the recognition accuracy reached 97.25%. Sun et al. [96] adopted LSTM model to identify distracted driving behavior, and the accuracy reached 91%, better than SVM. GRU is different from LSTM in principle. It solves the problem of long-term dependence by setting a gate to control the balance between input and forgetting. Some researchers have also applied GRU to identify driving behaviors, such as Tang et al. [97] who built a DeepConvGRU model to identify driving behaviors by introducing GRU gating mechanism. The accuracy reached 95.08%. Yan et al. [98] constructed a GRU model to predict lane change behavior, and the accuracy reached 94.76%, better than LSTM (93.47%).To give full play to the advantages of various deep learning models, some researchers combine RNN and other models to improve the performance of driving behavior recognition. For example, Mafeni et al. [99] jointly applied BI-LSTM and InceptionV3 CNNs to construct a distracted driving model (as shown in Figure 9) and identified distracted driving behaviors through driving posture pictures, with an accuracy of 93.1%. Wollmer et al. [33] used RNN combined with random forest FR to identify distracted driving behavior, and the accuracy reached 95%. Zhang et al. [100] propose a composite model that combines a multiscale convolutional neural network (MSCNN) and bi-LSTM to identify driving behaviors through vehicle track data. The highest accuracy was 97.75%. Zhang et al. [101] built a vehicle driving behavior recognition model based on joint data enhancement with a multiview convolutional neural network, and the model recognition accuracy was better than CNN, RNN, LSTM, CNN+LSTM, and other models. Xing et al. [102] jointly applied LSTM and BI-RNN to establish a lane change behavior recognition model; the accuracy reached 96.1% and shortens the prediction time cycle.Figure 9
The schematic diagram of the union model [99].Other researchers have further improved the performance of recurrent neural networks by increasing their depth and using them in driving behavior recognition research, such as Stacked Recurrent Neural Networks (SRNN) [103–109], Bidirectional Recurrent Neural Network (Bi-RNN) [100, 102, 110–112], Recursive Neural Networks [113], and Graph Neural Network (GNN) [114, 115], which have been applied in driving behavior recognition and achieved high recognition accuracy.The above studies are only some typical application cases. There are many successful application cases of RNN in the construction of the driving behavior recognition model [116–124]. It can be seen that RNN can be used for all kinds of driving behavior recognition, is suitable for high-dimensional and big data sample learning, and can extract deep temporal and spatial features with an accuracy rate of more than 90%. However, a large sample size of the data is required for training RNN.Although the above model has been proved to be effective in driving behavior recognition by some studies, it does not perform well in all driving behavior recognition tasks. In this problem, there is no general model for driving behavior recognition, and the performance of the model depends heavily on the task for which it is applicable. Therefore, researchers not only need to understand the characteristics of candidate models but also need to understand driving scenarios and conduct corresponding tests to correctly select models and effectively apply them to practical driving behavior recognition tasks.The test part mainly includes the following three types: real vehicle test, simulation verification, and vehicle in-loop test. The advantages of the real vehicle test are real and objective, but the disadvantages are that the high-risk scene data cannot be obtained and the manpower and material resources are heavily invested. The advantages of simulation verification are easy to test various complex working conditions, with high efficiency, but the scene cannot be completely close to the real situation. Vehicle in-loop test is a relatively popular method at present, which can integrate the advantages of real vehicle tests and simulation tests and can not only reproduce all kinds of actual scenes but also exhaust high-risk and complex scenes. After the driving behavior scene test, massive test data samples can be obtained by the driving behavior scene test, which is imported into the deep learning model for training and computation after data preprocessing, to complete the recognition of driving behavior.All in all, the deep learning models could be mainstream model for its high accuracy and stable robustness if a lot of data set are available to train the models. The advantages of the deep learning model are as follows: strong learning ability and can be transplanted to the framework of many platforms; the neural network of deep learning has a large number of layers and a wide width. Theoretically, it can be mapped to any function, so it can solve very complex problems. Deep learning is highly data-dependent, and the larger the amount of data, the better its performance. The disadvantage is that the model design of deep learning is very complex and requires a large amount of data and computational power, so the cost is very high. Deep learning has high requirements on computer hardware, and ordinary CPU can no longer meet the requirements of deep learning. However, with the development of big data, computing technology, and hardware, the application of deep learning in driving behavior will be greatly developed and its application will be more and more extensive.


## 4. Summaries and Prospects

This paper reviews and summarizes driving behavior recognition methods based on vehicle sensor information fusion. On-board sensor data contains abundant driving behavior information; based on the two main factors of driving behavior and vehicle control, driving behavior information can be divided into the driver state information, the driver control vehicle information, vehicle control state information, the vehicle state information, and road environment state information, as well as the corresponding data acquisition system. The characteristics of common data level, feature level, and decision level information fusion methods are analyzed to guide for selecting appropriate information fusion methods and the basic principles and main characteristics of feature extraction methods. Driving behavior recognition methods are classified into traditional machine learning methods and deep learning methods. Random forest, support vector machine, and its application in fatigue, distraction, following, lane change, and other driving behavior recognition are introduced. The application of convolutional neural network (CNN) and recurrent neural network (RNN) in the construction of driving behavior recognition models is analyzed. The characteristics of the four driving behavior recognition models described in this paper are briefly summarized as shown in Table5.Table 5
The comparisons of characteristics of driving behavior recognition models.
The model’s nameFRSVMCNNRNNCharacteristicsCan be used for all kinds of driving behavior recognition, features need manual selectionCan be used for all kinds of driving behavior recognition, features need manual selectionCan be used for all kinds of driving behavior recognition, and depth features are automatically extracted by the convolutional layerCan be used for all kinds of driving behavior recognition and automatic feature extraction of time series samplesRecognition accuracy>70%, distribution of discrete>75%, distribution of discrete>85%>90%Calculate power demandMiddleLowHighHighEfficiencyMiddleHighMiddleLowAdaptabilityStatic modelStatic modelStatic modelDynamic adaptive modelDefectsUnable to obtain time-series features, features extracted manually, feature consistency is poorUnable to obtain time-series features, features extracted manually, feature consistency is poorTime series features cannot be obtained, the sample input is fixed length, and the features have no clear meaningFeatures have no clear meaningApplicationsSuitable for pattern recognition and classification and applies to 1D time-series data and multidimensional dataSuitable for pattern recognition classification, multifeature, small sampleSuitable for pattern recognition and classification, large samples of fixed lengthSuitable for timing data of different lengths, large sampleWith the development of automobile intelligence and networked technology, the research of driving behavior recognition methods based on vehicle sensor information fusion will have a broad application prospect in the field of driving assistance. The development and large-scale application of high-performance onboard sensor technology enable real-time collection and storage of massive driving behavior data reflecting the natural and real driving state. At the same time, the rapid development of big data technology and artificial intelligence technology provides technical support for the research of driving behavior recognition models. How deep excavating multisource, heterogeneous, vast amounts of on-board sensor data information; information on different hierarchical classification using information fusion technology and characteristics of different processing technology; and comprehensive utilization of the characteristics of different driving behavior recognition models, to more accurately identify all types of driving behavior, will serve as a research hotspot in the field of driving safety technology. In particular, it will provide useful help for the development of high-performance advanced driver assistance systems and autonomous driving technology. At the same time, how to combine the characteristics of driving behavior data in the natural driving state and jointly apply multiclass models to establish the compound driving behavior recognition model, to construct a higher precision driving behavior recognition model, is also an important research direction in the future. As abnormal driving behaviors may lead to instantaneous accidents, how to further improve the identification efficiency and develop a lightweight model that can accurately identify driving behaviors and meet the requirements of real-time online identification, low computational power, and engineering needs is also an important research direction. Driving behavior is greatly affected by the driver’s state, and the driving behavior shown by the same driver may differ greatly. At the same time, driving behavior is also susceptible to the influence of road, environment, and vehicle, so there are certain uncertainties. How to improve the robustness and generalization ability of the driving behavior recognition model remains to be further studied.



---

*Source: 7287511-2022-10-07.xml*