Systems and methods for utilizing machine learning and feature selection to classify driving behavior

A device may receive vehicle operation data associated with operation of a plurality of vehicles, and may process the vehicle operation data to generate processed vehicle operation data. The device may extract multiple features from the processed vehicle operation data, and may train machine learning models, with the multiple features, to generate trained machine learning models that provide model outputs. The device may process the multiple features, with a feature selection model and based on the model outputs, to select sets of features from the plurality of features, and may process the sets of features, with the trained machine learning models, to generate indications of driving behavior and reliabilities of the indications. The device may select a set of features, from the sets of features, based on the indications and the reliabilities, where the set of features may be calculated by a device associated with a particular vehicle.

RELATED APPLICATION

This application claims priority to European Patent Application No. 19205766.9, filed on Oct. 28, 2019, entitled “SYSTEMS AND METHODS FOR UTILIZING MACHINE LEARNING AND FEATURE SELECTION TO CLASSIFY DRIVING BEHAVIOR,” which is hereby expressly incorporated by reference herein.

BACKGROUND

Driver behavior classification is a problem of great interest in the intelligent transportation community and is a key component of several applications. First of all, driving style, in combination with other factors such as road type and traffic congestion, has a substantial impact on fuel consumption. Thus, detecting high consumption maneuvers and coaching drivers to avoid them can lead to significant reductions in cost and carbon dioxide emissions. Furthermore, driver behavior classification is closely related to road safety since aggressive drivers tend to operate vehicles in an unsafe manner (e.g., with excessive speed, short car-following distance, erratic lane changes, and imprudent maneuvers), thus increasing risks of road accidents.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Vehicle operation data (e.g., driving data) is generally collected using one or multiple tracking devices installed inside a vehicle. Such tracking devices generally include an inertial measurement unit (IMU), a three-axis accelerometer, a global positioning system (GPS) device, and/or the like. More advanced tracking devices have access to vehicle sensors, via an engine control module (ECM), that collect information associated with pedal positions, steering angle, engine revolutions-per-minute, and/or the like.

Driving behavior may be classified utilizing all vehicle operation data from all vehicle tracking devices and collecting the vehicle operation data from the vehicle tracking devices at a maximum sampling frequency. However, transmission and storage of such vehicle operation data in some domains (e.g., a fleet management company monitoring several millions of vehicles) can be daunting and expensive. Furthermore, vehicle tracking devices have limited computational power and are unable to perform advanced computations required for classification of driving behavior. Thus, current techniques for classifying driving behavior waste computing resources (e.g., processing resources, memory resources, and/or the like), communication resources, networking resources, and/or the like associated with capturing vehicle operation data, transmitting the vehicle operation data, storing the vehicle operation data, and/or the like.

Some implementations described herein provide a vehicle platform that utilizes machine learning and feature selection to classify driving behavior. For example, the vehicle platform may receive vehicle operation data associated with operation of multiple vehicles, and may process the vehicle operation data to generate processed vehicle operation data for a particular time period. The vehicle platform may extract multiple features from the processed vehicle operation data, and may train machine learning models, with the multiple features, to generate trained machine learning models. The vehicle platform may generate model outputs based on training the machine learning models, and may process the multiple features, with a feature selection model and based on the model outputs, to select sets of features from the multiple features. The vehicle platform may process the sets of features, with the trained machine learning models, to generate indications of driving behavior and reliabilities of the indications of driving behavior, and may select a set of features, from the sets of features, based on the indications of driving behavior and the reliabilities of the indications of driving behavior. A user device associated with a particular vehicle may be capable of calculating the set of features for the particular time period.

In this way, the vehicle platform identifies particular features of vehicle operation data to be captured by vehicle devices for a particular time period, and utilizes the particular features to classify driving behavior. A quantity of the particular features and the particular time period ensure that the vehicle devices can process the vehicle operation data and transmit the particular features to the vehicle platform. Thus, the vehicle platform conserves computing resources (e.g., processing resources, memory resources, and/or the like), communication resources, networking resources, and/or the like that would otherwise be wasted in capturing vehicle data, transmitting the vehicle data, storing the vehicle data, and/or the like.

FIGS. 1A-1Kare diagrams of one or more example implementations100described herein. As shown inFIG. 1A, user devices105may be associated with vehicles110and a vehicle platform115. In some implementations, user devices105may include sensors that capture vehicle operation data (e.g., data indicating acceleration, speed, movement, and/or the like) associated with vehicles110and/or may receive the vehicle operation data from vehicle sensors of vehicles110, an engine control module (ECM) of vehicles110, and/or the like.

As further shown inFIG. 1A, and by reference number120, vehicle platform115may receive, from user devices105associated with vehicles110, vehicle operation data associated with operation of the vehicles. In some implementations, the vehicle operation data may include data identifying accelerations of vehicles110, speeds of vehicles110, distances of vehicles110from other vehicles, pedal positions associated with vehicles110(e.g., brake pedals and accelerator pedals), steering angles associated with vehicles110, engine revolutions-per-minute (RPMs) of vehicles110, and/or the like.

In some implementations, vehicle platform115may receive the vehicle operation data from user devices105, from inertial measurement units associated with vehicles110, from three-axis accelerometers associated with vehicles110, from global positioning system (GPS) devices associated with vehicles110, from ECMs associated with vehicles110, from video cameras associated with vehicles110, and/or the like. Vehicle platform115may periodically receive the vehicle operation data, may continuously receive the vehicle operation data, may receive the vehicle operation data based on a request, and/or the like. Vehicle platform115may store the vehicle operation data in a data structure (e.g., a database, a table, a list, and/or the like) associated with vehicle platform115.

As shown inFIG. 1B, and by reference number125, vehicle platform115may process the vehicle operation data to generate processed vehicle operation data for a particular time period. The particular time period may be a time period measured in seconds (e.g., ten seconds, thirty seconds, sixty seconds, and/or the like), in minutes (e.g., two minutes, five minutes, and/or the like), and/or the like. In some implementations, when processing the vehicle operation data, vehicle platform115may apply a moving average to the vehicle operation data to generate a low-pass version of the vehicle operation data, and may add the low-pass version of the vehicle operation data to the vehicle operation data to generate the processed vehicle operation data. For example, the vehicle operation data may include raw accelerometer data that includes a non-negligible amount of noise (e.g., mid-to-high frequency noise due to engine vibration, rugged road vibration, and/or the like). Vehicle platform115may process the raw accelerometer data, by applying a low-pass filter that employs the moving average, to generate a low-pass version of the accelerometer data. Vehicle platform115may add the low-pass version of the accelerometer data and the raw accelerometer data to generate the processed vehicle operation data.

In this way, vehicle platform115may generate processed vehicle operation data that includes values for each of a quantity of input variables that represent filtered data as well as raw data. For example, the input variables may include an x-axis raw acceleration variable, a y-axis raw acceleration variable, a z-axis raw acceleration variable, an x-axis filtered acceleration variable, a y-axis filtered acceleration variable, a z-axis filtered acceleration variable, a GPS speed variable, and/or the like.

As shown inFIG. 1C, and by reference number130, vehicle platform115may extract a plurality of features from the processed vehicle operation data. In some implementations, when extracting the plurality of features, vehicle platform115may compute, based on the processed vehicle operation data, a plurality of statistics that correspond to the plurality of features. For example, vehicle platform115may compute a value of a statistical variable for each input variable included in the processed vehicle operation data, and may extract a feature for each combination of a particular input variable and a different statistical variable associated with the particular input variable.

The statistical variables may include a mean variable, a variance variable, a standard deviation variable, a maximum variable, a minimum variable, a range variable, an interquartile range variable, a skewness variable, a kurtosis variable, a slope variable, a median variable, a twenty-fifth percentile variable, a seventy-fifth percentile variable, and/or the like. In one example, vehicle platform115may extract a feature associated with a mean of the x-axis raw acceleration, a feature associated with a variance of the x-axis raw acceleration, a feature associated with a standard deviation of the x-axis raw acceleration, and/or the like for all statistical variables associated with the x-axis raw acceleration. Vehicle platform115may extract a feature associated with a mean of the y-axis raw acceleration, a feature associated with a variance of the y-axis raw acceleration, a feature associated with a standard deviation of the y-axis raw acceleration, and/or the like for all statistical variables associated with the y-axis raw acceleration. In this way, vehicle platform115may extract a set of features based on commonly available and readily obtainable and/or determinable information and based on statistics that can be efficiently calculated without requiring a large amount of computing resources.

As shown inFIG. 1D, and by reference number135, vehicle platform115may train machine learning models, with the plurality of features, to generate trained machine learning models. The trained machine learning models may generate indications of driving behaviors (e.g., normal driving behavior, reckless driving behavior, and/or the like) based on the features. In some implementations, when training the machine learning models with the plurality of features, vehicle platform115may utilize a nested cross-validation to tune a plurality of parameters for the machine learning models and to evaluate the machine learning models. Vehicle platform115may perform a preliminary analysis, with a first random forest model (e.g., a baseline random forest model), to tune the plurality of parameters, and may utilize a second random forest model (e.g., an evaluation random forest model) to classify the plurality of parameters and to generate the trained machine learning models. As further shown inFIG. 1D, and by reference number140, vehicle platform115may generate model outputs based on training the machine learning models. The model outputs may include indications of reliabilities associated with the indications of driving behaviors.

When utilizing the nested cross-validation to tune the plurality of parameters for the machine learning models and to evaluate the machine learning models, vehicle platform115may employ a nested leave-one-out cross-validation for a driver, of a plurality of drivers, associated with the vehicle operation data. For example, the nested leave-one-out cross-validation may include an outer loop that generates multiple training and/or test splits by iteratively selecting data from one driver as a test set and selecting remaining data as a training set. Where d is the number of drivers, for each of the d splits, vehicle platform115may determine a reliability of data relative to the test driver, and may average the reliabilities to generate a test reliability. The nested leave-one-out cross-validation may include an inner loop that iterates over the drivers selected in the training set and generates training and/or validation splits. For each of the d−1 splits, vehicle platform115may determine reliabilities associated with the validation set, and may average scores of different splits of the inner loop to generate a validation accuracy.

In some implementations, the machine learning models may include random forest machine learning models. The random forest machine learning models may be utilized by vehicle platform115to determine whether a driver is exhibiting normal driving behavior or aggressive driving behavior based on vehicle operation data.

In some implementations, vehicle platform115may train the machine learning models, with historical feature information (e.g., information identifying the plurality of features), to determine indications of driving behavior. For example, vehicle platform115may separate the historical feature information into a training set, a validation set, a test set, and/or the like. The training set may be utilized to train the machine learning models. The validation set may be utilized to validate results of the trained machine learning models. The test set may be utilized to test operation of the machine learning models.

In some implementations, vehicle platform115may train the machine learning models using, for example, an unsupervised training procedure and based on the historical feature information. For example, vehicle platform115may perform dimensionality reduction to reduce the historical feature information to a minimum feature set, thereby reducing resources (e.g., processing resources, memory resources, and/or the like) to train the machine learning models, and may apply a classification technique to the minimum feature set.

In some implementations, vehicle platform115may use a logistic regression classification technique to determine a categorical outcome (e.g., that particular historical feature information indicates particular driving behaviors). Additionally, or alternatively, vehicle platform115may use a naïve Bayes classifier technique. In this case, vehicle platform115may perform binary recursive partitioning to split the historical feature information into partitions and/or branches and use the partitions and/or branches to determine outcomes (e.g., that particular historical feature information indicates particular driving behaviors). Based on using recursive partitioning, vehicle platform115may reduce utilization of computing resources relative to manual, linear sorting and analysis of data points, thereby enabling use of thousands, millions, or billions of data points to train the machine learning model, which may result in a more accurate model than using fewer data points.

Additionally, or alternatively, vehicle platform115may use a support vector machine (SVM) classifier technique to generate a non-linear boundary between data points in the training set. In this case, the non-linear boundary is used to classify test data into a particular class.

Additionally, or alternatively, vehicle platform115may train the machine learning models using a supervised training procedure that includes receiving input to the machine learning models from a subject matter expert, which may reduce an amount of time, an amount of processing resources, and/or the like to train the machine learning models relative to an unsupervised training procedure. In some implementations, vehicle platform115may use one or more other model training techniques, such as a neural network technique, a latent semantic indexing technique, and/or the like. For example, vehicle platform115may perform an artificial neural network processing technique (e.g., using a two-layer feedforward neural network architecture, a three-layer feedforward neural network architecture, and/or the like) to perform pattern recognition with regard to patterns of the historical feature information. In this case, using the artificial neural network processing technique may improve an accuracy of the trained machine learning models generated by vehicle platform115by being more robust to noisy, imprecise, or incomplete data, and by enabling vehicle platform115to detect patterns and/or trends undetectable to human analysts or systems using less complex techniques.

In some implementations, rather than training the machine learning models, vehicle platform115may receive a trained machine learning models from another device (e.g., a server device). For example, a server device may generate the trained machine learning models based on having trained machine learning models in a manner similar to that described above, and may provide the trained machine learning models to vehicle platform115(e.g., may pre-load vehicle platform115with the trained machine learning models, may receive a request from vehicle platform115for the trained machine learning models, and/or the like).

As shown inFIG. 1E, and by reference number145, vehicle platform115may process the plurality of features, with a feature selection model and based on the model outputs, to select sets of features from the plurality of features. In some implementations, when processing the plurality of features, vehicle platform115may estimate an importance (e.g., a Gini importance) of each of the plurality of features, and may select the sets of features from the plurality of features based on the importance estimated for each of the plurality of features. For example, starting from a trained random forest model with a quantity (F) of trees, vehicle platform115may calculate a Gini importance (Imp) for a feature (Xi) as:

iG⁡(n)=∑k=1C⁢NnkNn⁢(1-NnkNn)
an impurity decrease generated by split snon a node n may be defined as:

Δ⁢i⁡(sn,n)=iG⁡(n)-NnLNn⁢iG⁡(nL)-NnRNn⁢iG⁡(nR)
where C may correspond to a number of classes, Nnkmay correspond to a number of samples of a class k falling in a node n's subtree, nLand nRmay correspond to, respectively, left and right children of the node n, and NnLand NnRmay correspond to a number of training samples falling in left and right subtrees of the node n.

In some implementations, when processing the plurality of features, vehicle platform115may rank each of the plurality of features, based on an average Gini importance associated with each of the plurality of features, to generate rankings of the plurality of features. Vehicle platform115may filter the rankings of the plurality of features to generate filtered rankings of the plurality of features, and may determine validation accuracies associated with the plurality of features based on the filtered rankings of the plurality of features. Vehicle platform115may select the sets of features, from the plurality of features, based on the validation accuracies associated with plurality of features.

When filtering the rankings of the plurality of features, vehicle platform115may execute a filtering model to remove highly correlated variables as follows:

function FILTERRANKING (r, c)i ← 0while i <= len(r) doiterate on the rankingtop ← r[i]j ← i+1while j <= len(r) doother ← r[j]if Corr(top, other) > c thenr ← r \ otherelsej ← j+1return r.
As shown above, the filtering model may begin with a feature ranked highest based on the Gini importance, as described above, and may remove all features that have a correlation (e.g., an absolute Pearson correlation) that exceeds a given threshold (c). The filtering model may select a next highest ranked feature from the remaining features, and may iterate the method until all features have been selected or filtered out. Vehicle platform115may re-rank the remaining features by training a first random forest model on the remaining features. In some implementations, vehicle platform115may assess several different values for the threshold c, each of which may generate a different filtered ranking.

As shown inFIG. 1F, and by reference number150, vehicle platform115may process the sets of features, with the trained machine learning models, to generate indications of driving behavior and reliabilities of the indications. In some implementations, each of the indications of driving behavior may include information identifying aggressive driving behavior, information identifying normal driving behavior, information identifying drowsy driving behavior, information identifying performance driving behavior, information identifying economical driving behavior, and/or the like. In some implementations, vehicle platform115may train a second random forest model based on different subsets of features. For example, vehicle platform115may begin with a feature ranked highest based on the Gini importance, as described above, and may incrementally add features in decreasing order of importance. At each increment, vehicle platform115may train the second random forest model, may utilize tuning parameters with the second random forest model, and may evaluate a reliability of the second random forest model based on independent data via cross-validation, as described above.

As shown inFIG. 1G, and by reference number155, vehicle platform115may select a set of features, from the sets of features, based on the indications of driving behavior and the reliabilities of the indications. In some implementations, vehicle platform115may utilize a feature selection model to select the set of features from the sets of features. For example, given the plurality of features and a set of filtering thresholds (cSet), the feature selection model may perform the following steps:procedure FEATURESELECTION(allFeatures, cSet)ref Ranking, _←train (baseRF, allFeatures)reliabilities←Øfor c in cSet dofr←FILTERRANKING(refRanking, c)filteredRanking, _←train(baseRF, fr)features←Øreliabilitiesc←Øfor f in filteredRanking dofeatures←features∪f_, a←train(evalRF, features)reliabilitiesc←reliabilitiesc∪areliabilities←reliabilities∪reliabilitiescreturn (features, reliabilities)
where baseRF may correspond to a first (e.g., a baseline) random forest model and evalRF may correspond to a second (e.g., an evaluation) random forest model. The train(⋅) function may train a specified random forest model based on a particular set of features (e.g. allFeatures) and may return a ranking of input features and a validation accuracy.

In one example, vehicle platform115may utilize features based on combinations of the input variables and the statistical variables, as described above, and may select a particular quantity (e.g., three, six, and/or the like) of features, such as a standard deviation of a z-axis, a mean of a vehicle speed, a skewness of a y-axis filtered acceleration, and/or the like. The selected features may be representative of discriminative driving characteristics. For example, the standard deviation of the z-axis (e.g., a longitudinal acceleration) may indicate extreme braking and accelerations, the mean of the vehicle speed (e.g., an average speed) may be associated with a tendency of speeding, and the skewness of y-axis filtered acceleration (e.g., a lateral acceleration) may indicate harsh cornering in curves and turns. In this way, vehicle platform115may select a quantity of features that may be determined by a device with limited computing power, such as by user device105and/or a computing device associated with vehicle110.

As shown inFIG. 1H, and by reference number160, vehicle platform115may provide, to user device105associated with vehicle110, a request to calculate the selected set of features for a particular time period. The particular time period may be time period measured in seconds (e.g., ten seconds, thirty seconds, sixty seconds, and/or the like), in minutes (e.g., two minutes, five minutes, and/or the like), and/or the like. As further shown inFIG. 1H, and by reference number165, user device105may receive vehicle operation data, from vehicle110and/or sensors associated with vehicle110, for the particular time period and based on the request. For example, user device105may calculate the vehicle operation data (e.g., based on sensors associated with user device105) and/or may receive the vehicle operation data from a inertial measurement unit associated with vehicle110, from a three-axis accelerometer associated with vehicle110, from a GPS device associated with vehicle110, from an ECM associated with vehicle110, from video cameras associated with vehicle110, and/or the like.

In some implementations, the selected set of features may be different for different vehicles110and/or based on conditions associated with the different vehicles110. For example, a vehicle110may not include a device that is capable of capturing the vehicle operation data required to determine the set of features. In such an example, vehicle platform115may receive an indication of vehicle operation data that is available to vehicle110and may determine a particular set of features based on the available vehicle operation data. Other factors may influence selection of the set of features, such as a type of vehicle110calculating the set of features (e.g., a truck, a car, a motorcycle, and/or the like); a location of vehicle110(e.g., a highway, a city road, a country road, and/or the like); environmental conditions associated with vehicle (e.g., sunny, rainy, snowy, icy, and/or the like); and/or the like. In some implementations, vehicle platform115may receive information identifying one or more of the other factors and may determine a particular set of features based on the one or more of the other factors. With reference toFIG. 1H, vehicle platform115may then provide, to user device105, a request to calculate the particular set of features for the particular time period.

As further shown inFIG. 1H, and by reference number170, user device105may calculate the selected set of features for the particular time period based on the vehicle operation data. Alternatively, or additionally, user device105may automatically calculate the selected set of features on a periodic basis (e.g., every quantity of seconds) and may continuously and periodically provide the calculated set of features to vehicle platform115in near real-time relative to calculating the selected set of features.

As shown inFIG. 1I, and by reference number175, vehicle platform115may receive, from user device105, data identifying the selected set of features for the particular time period. In some implementations, vehicle platform115may receive a single data point for each feature of the selected set of features over the particular time period. Alternatively, or additionally, vehicle platform115may continuously and periodically receive data points for each feature of the selected set of features when such features are provided on a periodic basis.

As shown inFIG. 1J, and by reference number180, vehicle platform115may process the selected set of features, with one or more of the trained machine learning models, to generate an indication of driving behavior associated with vehicle110(e.g., a drive of vehicle110). In this way, vehicle platform115may distinguish between safe or normal driving behavior and aggressive driving behavior while using only a limited set of selected features. The limited set of features may be calculated with limited computing resources (e.g., by user device105and/or by devices of vehicle110) to provide an effective and accurate indication of driving behavior. This may reduce computations and communications required to determine driving behavior associated with vehicle110, which may conserve computing resources, communication resources, networking resources, and/or the like that would otherwise be wasted in capturing all available vehicle operation data, transmitting the vehicle operation data, storing the vehicle operation data, and/or the like.

As shown inFIG. 1K, and by reference number185, vehicle platform115may perform one or more actions based on the indication of driving behavior. In some implementations, the one or more actions may include vehicle platform115providing, to user device105, the indication of driving behavior associated with vehicle110. For example, vehicle platform115may provide the indication for display on user device105or on another device associated with vehicle110; may provide the indication for display to a driver of vehicle110, to an employer of the driver, to an owner of vehicle110, and/or the like. In this way, vehicle platform115may enable the user of user device105, the driver of vehicle110, an employer of the driver, a parent of the driver, and/or the like, to be aware of adverse driving behavior. This may enable the driver to effectively adjust and/or improve driving techniques, which may improve road safety, conserve fuel, conserve resources that would otherwise be wasted policing poor driving behavior, handling vehicle accidents, and/or the like.

In some implementations, the one or more actions may include vehicle platform115providing the indication of driving behavior to a device associated with an insurer of vehicle110. In this way, vehicle platform115may enable the insurer to adjust coverage and/or cost of coverage associated with vehicle110, an owner of vehicle110, a driver of vehicle110, and/or the like. The insurer may provide guidance and/or training to the driver of vehicle110(e.g., in exchange for not increasing coverage costs) in order to reduce risks of accidents, tickets, and/or the like, which may improve road safety, conserve fuel, conserve resources that would otherwise be wasted handling insurance claims, policing poor driving behavior, handling vehicle accidents, and/or the like.

In some implementations, the one or more actions may include vehicle platform115providing, to user device105, an instruction to address the driving behavior. For example, vehicle platform115may instruct the driver of vehicle110(e.g., in near real-time) to adjust a driving speed, to stop making reckless turns, and/or the like. In this way, vehicle platform115may enable the driver of vehicle110to perform corrective driving actions based on the indication of driving behavior associated with vehicle110, which may improve road safety, conserve fuel, conserve resources that would otherwise be wasted policing poor driving behavior, handling vehicle accidents, and/or the like.

In some implementations, the one or more actions may include vehicle platform115causing vehicle110to be disabled or the speed of vehicle110to be reduced to a speed limit based on the indication of driving behavior (e.g., that the driver was speeding). In this way, vehicle platform115may prevent vehicle110from being operated in a dangerous manner that risks death or injury (e.g., to the driver of vehicle110, passengers of vehicle110, other drivers, and/or the like), that risks damage to property (e.g., damage to vehicle110, damage to other vehicles, damage to physical property, and/or the like), and/or the like. This may conserve resources that would otherwise be wasted in treating injuries, repairing damage, handling vehicle accidents, handling legal actions, and/or the like.

In some implementations, the one or more actions may include vehicle platform115determining a driver risk score based on the indication of driving behavior. In this way, vehicle platform115may enable a driver of vehicle110, an employer of the driver, an owner of vehicle110, and/or the like to monitor quality and assess risk associated with the driver and/or operation of vehicle110; to be aware of changes to driving behavior associated with vehicle110; to provide an objective and consistent mechanism for evaluating driver performance, rewarding good driver performance, addressing bad driver performance, etc.; and/or the like.

In some implementations, the one or more actions may include vehicle platform115retraining the machine learning models based on the indication of driving behavior. In this way, vehicle platform115may improve the accuracy of the machine learning models in determining indications of driving behaviors, reliabilities of the indications, and/or the like, which may improve speed and efficiency of the machine learning models and conserve computing resources, network resources, and/or the like.

In some implementations, the one or more actions may include vehicle platform115causing a driver training program, focused on the driving behavior, to be scheduled for the driver of vehicle110. In this way, vehicle platform115may automatically arrange and/or facilitate training that may improve driving behavior, without requiring manual administrative actions or other resources to arrange for and/or facilitate the driver training program. This may conserve resources that would otherwise be wasted treating injuries, repairing damage, handling vehicle accidents, handling legal actions, and/or the like caused by the driving behavior.

In some implementations, the one or more actions may include vehicle platform115causing vehicle110to operate in an autonomous mode until the driver of the particular vehicle completes particular training. For example, vehicle110may include an autonomous mode, and vehicle platform115may cause the autonomous mode to be engaged, and prevent the drive from operating vehicle110until receiving an indication that the driver has completed the particular training. In this way, vehicle platform115may conserve resources that would otherwise be wasted treating injuries, repairing damage, handling vehicle accidents, handling legal actions, and/or the like caused by the driver.

In this way, several different stages of the process for classifying driving behavior is automated via machine learning and feature selection, which may remove human subjectivity and waste from the process, and which may improve speed and efficiency of the process and conserve computing resources (e.g., processing resources, memory resources, and/or the like), communication resources, networking resources, and/or the like. Furthermore, implementations described herein use a rigorous, computerized process to perform tasks or roles that were not previously performed or were previously performed using subjective human intuition or input. For example, currently there does not exist a technique that utilizes machine learning and feature selection to classify driving behavior. Finally, the process for classifying driving behavior conserves computing resources, communication resources, networking resources, and/or the like that would otherwise be wasted in capturing all available vehicle operation data, transmitting the vehicle operation data, storing the vehicle operation data, and/or the like.

As indicated above,FIGS. 1A-1Kare provided merely as examples. Other examples may differ from what was described with regard toFIGS. 1A-1K. The number and arrangement of devices and networks shown inFIGS. 1A-1Kare provided as an example. In practice, there may be additional devices and/or networks, fewer devices and/or networks, different devices and/or networks, or differently arranged devices and/or networks than those shown inFIGS. 1A-1K. Furthermore, two or more devices shown inFIGS. 1A-1Kmay be implemented within a single device, or a single device shown inFIGS. 1A-1Kmay be implemented as multiple, distributed devices. Additionally, or alternatively, a set of devices (e.g., one or more devices) ofFIGS. 1A-1Kmay perform one or more functions described as being performed by another set of devices ofFIGS. 1A-1K.

FIG. 2is a diagram of an example environment200in which systems and/or methods described herein may be implemented. As shown inFIG. 2, environment200may include user device105, a vehicle platform115, and a network230. Devices of environment200may interconnect via wired connections, wireless connections, or a combination of wired and wireless connections.

User device105includes one or more devices capable of receiving, generating, storing, processing, and/or providing information, such as information described herein. For example, user device105may include a mobile phone (e.g., a smart phone, a radiotelephone, etc.), a laptop computer, a tablet computer, a desktop computer, a handheld computer, a gaming device, a wearable communication device (e.g., a smart watch, a pair of smart glasses, a heart rate monitor, a fitness tracker, smart clothing, smart jewelry, a head mounted display, etc.), a GPS device, a device included in vehicle110(e.g., an inertial measurement unit, a three-axis accelerometer, a GPS device, an ECM, a video camera, and/or the like) or a similar type of device. In some implementations, user device105may receive information from and/or transmit information to vehicle platform115.

Vehicle platform115includes one or more devices that utilize machine learning and feature selection to classify driving behavior. In some implementations, vehicle platform115may be designed to be modular such that certain software components may be swapped in or out depending on a particular need. As such, vehicle platform115may be easily and/or quickly reconfigured for different uses. In some implementations, vehicle platform115may receive information from and/or transmit information to one or more user devices105.

In some implementations, as shown, vehicle platform115may be hosted in a cloud computing environment210. Notably, while implementations described herein describe vehicle platform115as being hosted in cloud computing environment210, in some implementations, vehicle platform115may not be cloud-based (i.e., may be implemented outside of a cloud computing environment) or may be partially cloud-based.

Cloud computing environment210includes an environment that hosts vehicle platform115. Cloud computing environment210may provide computation, software, data access, storage, etc., services that do not require end-user knowledge of a physical location and configuration of system(s) and/or device(s) that hosts vehicle platform115. As shown, cloud computing environment210may include a group of computing resources220(referred to collectively as “computing resources220” and individually as “computing resource220”).

Computing resource220includes one or more personal computers, workstation computers, mainframe devices, or other types of computation and/or communication devices. In some implementations, computing resource220may host vehicle platform115. The cloud resources may include compute instances executing in computing resource220, storage devices provided in computing resource220, data transfer devices provided by computing resource220, etc. In some implementations, computing resource220may communicate with other computing resources220via wired connections, wireless connections, or a combination of wired and wireless connections.

As further shown inFIG. 2, computing resource220includes a group of cloud resources, such as one or more applications (“APPs”)220-1, one or more virtual machines (“VMs”)220-2, virtualized storage (“VSs”)220-3, one or more hypervisors (“HYPs”)220-4, and/or the like.

Application220-1includes one or more software applications that may be provided to or accessed by user device105. Application220-1may eliminate a need to install and execute the software applications on user device105. For example, application220-1may include software associated with vehicle platform115and/or any other software capable of being provided via cloud computing environment210. In some implementations, one application220-1may send/receive information to/from one or more other applications220-1, via virtual machine220-2.

Virtual machine220-2includes a software implementation of a machine (e.g., a computer) that executes programs like a physical machine. Virtual machine220-2may be either a system virtual machine or a process virtual machine, depending upon use and degree of correspondence to any real machine by virtual machine220-2. A system virtual machine may provide a complete system platform that supports execution of a complete operating system (“OS”). A process virtual machine may execute a single program and may support a single process. In some implementations, virtual machine220-2may execute on behalf of a user (e.g., a user of user device105or an operator of vehicle platform115), and may manage infrastructure of cloud computing environment210, such as data management, synchronization, or long-duration data transfers.

FIG. 3is a diagram of example components of a device300. Device300may correspond to user device105, vehicle platform115, and/or computing resource220. In some implementations, user device105, vehicle platform115, and/or computing resource220may include one or more devices300and/or one or more components of device300. As shown inFIG. 3, device300may include a bus310, a processor320, a memory330, a storage component340, an input component350, an output component360, and a communication interface370.

FIG. 4is a flow chart of an example process400for utilizing machine learning and feature selection to classify driving behavior. In some implementations, one or more process blocks ofFIG. 4may be performed by a device (e.g., vehicle platform115). In some implementations, one or more process blocks ofFIG. 4may be performed by another device or a group of devices separate from or including the device, such as a user device (e.g., user device105).

As shown inFIG. 4, process400may include receiving vehicle operation data associated with operation of a plurality of vehicles (block410). For example, the device (e.g., using computing resource220, processor320, communication interface370, and/or the like) may receive vehicle operation data associated with operation of a plurality of vehicles, as described above. The vehicle operation data may include data identifying accelerations of the plurality of vehicles, speeds of the plurality of vehicles, distances of the plurality of vehicles from other vehicles, pedal positions of the plurality of vehicles, steering angles of the plurality of vehicles, engine revolutions-per-minute of the plurality of vehicles, and/or the like. The vehicle operation data may be received from user devices associated with the plurality of vehicles, inertial measurement units associated with the plurality of vehicles, three-axis accelerometers associated with the plurality of vehicles, global positioning system (GPS) devices associated with the plurality of vehicles, engine control modules associated with the plurality of vehicles, video cameras associated with the plurality of vehicles, and/or the like.

As further shown inFIG. 4, process400may include processing the vehicle operation data to generate processed vehicle operation data for a particular time period (block420). For example, the device (e.g., using computing resource220, processor320, memory330, and/or the like) may process the vehicle operation data to generate processed vehicle operation data for a particular time period, as described above. In some implementations, processing the vehicle operation data to generate the processed vehicle operation data may include process400applying a moving average, based on the particular time period, to the vehicle operation data to generate a low-pass version of the vehicle operation data; and adding the low-pass version of the vehicle operation data to the vehicle operation data to generate the processed vehicle operation data.

As further shown inFIG. 4, process400may include extracting a plurality of features from the processed vehicle operation data (block430). For example, the device (e.g., using computing resource220, processor320, storage component340, and/or the like) may extract a plurality of features from the processed vehicle operation data, as described above. In some implementations, extracting the plurality of features from the processed vehicle operation data may include process400computing a plurality of statistics based on the processed vehicle operation data.

As further shown inFIG. 4, process400may include training machine learning models, with the plurality of features, to generate trained machine learning models (block440). For example, the device (e.g., using computing resource220, processor320, memory330, and/or the like) may train machine learning models, with the plurality of features, to generate trained machine learning models, as described above. In some implementations, training the machine learning models with the plurality of features may include process400utilizing a nested cross-validation to tune the plurality of features for the machine learning models and to evaluate the machine learning models; performing a preliminary analysis, with a first random forest model, to tune the plurality of features; and utilizing a second random forest model to classify the plurality of features and to generate the trained machine learning models. Each of the machine learning models may include a random forest machine learning model.

As further shown inFIG. 4, process400may include generating model outputs based on training the machine learning models (block450). For example, the device (e.g., using computing resource220, processor320, storage component340, input component350, output component360, communication interface370, and/or the like) may generate model outputs based on training the machine learning models, as described above.

As further shown inFIG. 4, process400may include processing the plurality of features, with a feature selection model and based on the model outputs, to select sets of features from the plurality of features (block460). For example, the device (e.g., using computing resource220, processor320, memory330, storage component340, and/or the like) may process the plurality of features, with a feature selection model and based on the model outputs, to select sets of features from the plurality of features, as described above. In some implementations, processing the plurality of features, with the feature selection model and based on the model outputs, to select the sets of features, may include process400estimating an importance of each of the plurality of features based on an average Gini importance associated with each of the plurality of features; and selecting the sets of features from the plurality of features based on the importance estimated for each of the plurality of features.

In some implementations, processing the plurality of features, with the feature selection model and based on the model outputs, to select the sets of features, may include process400ranking each of the plurality of features based on an average Gini importance associated with each of the plurality of features to generate rankings of the plurality of features; filtering the rankings of the plurality of features to generate filtered rankings of the plurality of features; determining validation accuracies associated with plurality of features based on the filtered rankings of the plurality of features; and selecting the sets of features, from the plurality of features, based on the validation accuracies associated with plurality of features.

As further shown inFIG. 4, process400may include processing the sets of features, with the trained machine learning models, to generate indications of driving behavior and reliabilities of the indications of driving behavior (block470). For example, the device (e.g., using computing resource220, processor320, memory330, and/or the like) may process the sets of features, with the trained machine learning models, to generate indications of driving behavior and reliabilities of the indications of driving behavior, as described above. Each of the indications of driving behavior includes information identifying one of aggressive driving behavior or normal driving behavior.

As further shown inFIG. 4, process400may include selecting a set of features, from the sets of features, based on the indications of driving behavior and the reliabilities of the indications of driving behavior, wherein a user device associated with a particular vehicle is capable of calculating the set of features for the particular time period (block480). For example, the device (e.g., using computing resource220, processor320, storage component340, and/or the like) may select a set of features, from the sets of features, based on the indications of driving behavior and the reliabilities of the indications of driving behavior, as described above. In some implementations, a user device associated with a particular vehicle may be capable of calculating the set of features for the particular time period.

In some implementations, process400may further include providing, to the user device, a request to calculate the set of features for the particular time period, where the user device is capable of calculating the set of features for the particular time period and for the particular vehicle based on the request; receiving, from the user device, data identifying the set of features for the particular time period; and processing data identifying the set of features, with one of the trained machine learning models, to generate an indication of driving behavior associated with the particular vehicle.

In some implementations, process400may further include performing one or more actions based on the indication of driving behavior associated with the particular vehicle. The one or more actions may include providing, to the user device, the indication of driving behavior associated with the particular vehicle; providing the indication of driving behavior to a particular device associated with an insurer of the particular vehicle; providing, to the user device, an instruction to address the driving behavior; causing the particular vehicle to be disabled based on the indication of driving behavior; determining a driver risk score, for a driver of the particular vehicle, based on the indication of driving behavior; retraining at least one of the machine learning models based on the indication of driving behavior; causing a driver training program, focused the driving behavior, to be scheduled for the driver of the particular vehicle; causing the particular vehicle to operate in an autonomous mode until the driver of the particular vehicle completes particular training; and/or the like.