Understanding driver awareness through brake behavior analysis

Systems and methods for analyzing a driver's brake behavior are provided. In some aspects, the system includes a memory that stores instructions for executing processes for analyzing the driver's brake behavior, and a processor configured to execute the instructions. In aspects, the processes include: receiving data from one or more multi-modal sensors, determining whether each braking event of a plurality of braking events is a hard braking event; analyzing the one or more physiological signals of the driver to determine whether each of the hard braking events is a reactive hard braking event or an intended hard braking event; applying a Lasso regression analysis and a classification framework for each of the hard braking events to identify features that indicate whether a respective hard braking event is a reactive hard braking event or an intended hard braking event; and generating a signal notifying a vehicle of the features.

TECHNICAL FIELD

Aspects of the present disclosure relate to methods and systems for analyzing a driver's brake behavior.

BACKGROUND

Driving is a highly dynamic activity where a driver's awareness of the traffic environment may play an essential role in performance. Easy, smooth driving may depend on the driver's ability to develop situation-specific expectations. In some circumstances, however, infrequent or unexpected situations may not be taken into account. Understanding these unexpected situations may provide important insight on driver situational awareness and accident prevention. For example, braking is one of the fundamental driving maneuvers, as drivers use braking to control vehicle speed, maintain the distance to the frontal traffic participants, and stop at traffic lights, among other uses. Braking is also closely related to a driver's awareness of the surrounding traffic situations. In most instances, a driver's braking may be less (or moderately) intense indicating routine driving events. However, in other instances, a driver's braking may be more intense indicating a hard braking event, which may be the result of an intentional maneuver, e.g., decelerating to make a turn or exit from a highway, or may be the result of an unexpected situation, e.g., decelerating to avoid an expected object on the roadway.

Despite the popular dichotomy of human awareness control, driving often involves top-down and bottom-up processes, where the top-down processes are associated with prediction using learned knowledge, and the bottom-up processes relate to reaction to salient features in the environment. Furthermore, handling unexpected events conventionally involves a bottom-up process, while implementing planned actions conventionally involves a top-down process. In a naturalistic driving condition, the difference between the unexpected events and implemented planned actions may be captured using driver physiological signals. Current methodologies, however, do not correlate driver physiological signals to distinguish between different types hard braking events, and/or further analyze road features to determine what may have been the cause of the different types of hard braking events.

SUMMARY

In some aspects, the present disclosure may be used to analyze braking events, such as hard braking because: (1) understanding a driver's hard braking due to an unexpected traffic scene may provide useful insights to prevent accidents for future advanced driver assistance systems (ADAS) development; and (2) focusing on the hard braking events may ensure the association between driver physiological signals and driver awareness. In some aspects, different stimulus, such as distractions and secondary tasks, may lead to significant changes in driver physiological signals, and by analyzing hard braking events, it may be determined whether these changes in these driver physiological signals are caused by unexpected traffic conditions. Although the present disclosure relates to analyzing hard braking events, it should be understood by those of ordinary skill in the arts that the framework described herein may be used to analyze other driving maneuvers such as, but not limited to, steering wheel control and/or other ADAS functionalities.

In some aspects, the present disclosure is directed to analyzing data from a vehicle and data from wearable devices worn by the driver to detect physiological signals, and this analysis may be used to identify unexpected situations during hard braking. Based on a naturalistic driving dataset, two types of hard braking behavior may be identified: (1) reactive hard braking; and (2) intended hard braking. Reactive hard braking may be a result of a driver reacting to unexpected situations that usually lead to deviated physiological signals due to stress. Intended hard braking may be a result of a planned maneuver that includes stable physiological signals. In some aspects, the present disclosure may be used to identify the different situations in which these two types of hard braking occur. For example, aspects of the present disclosure may be used to identify features that are representative of these two types of hard braking, including a road environment. Furthermore, aspects of the present disclosure may be used to extract features from a Lidar depth scanner to represent the road scene, and further apply a Lasso regression and a classification framework to analyze the extracted features. In some aspects, the Lasso regression model may achieve a high correlation of 0.77 between a prediction and a ground truth, and the classification framework may achieve an F-score of 0.76. In further aspects, selected Lidar features may serve as high level road scene representation that may facilitate next generation ADAS to prevent accidents in unexpected traffic scenarios.

Reactive hard braking events may be in response to, but are not limited to, the following situations:1. Approaching an intersection while the traffic light changes from green to yellow.2. A frontal vehicle suddenly braking or cutting into traffic.3. Merging into busy highway from a high speed ramp.4. Stopping for traffic light after turning into a difficult illumination condition where the driver faces direct sun light.5. A driver failing to check both side mirrors before start vehicle at the stop sign.

Intended hard braking events, vents may be in response to, but are not limited to, the following situations:1. Following another car.2. Approaching an intersection with a red light.3. Executing a sharp turn and/or a “U-turn.”4. Stopping at a stop sign on a steep downhill.5. Entering a traffic jam in highway.

Furthermore, aspects of the present disclosure may be directed to identifying physiological signals to define driving maneuvers that are closely related to the top-down and bottom-up driver attention process. Such an approach provides an understanding of a driver's awareness during driving maneuvers and its connection to the driving environment. Furthermore, aspects of the present disclosure may be directed to analyzing a Lidar depth scanner to represent the driving environment in order to provide an accurate distance measurement between frontal road objects and the vehicle. In further aspects, the present disclosure may be directed to defining reactive and intended hard braking to study hard braking behavior.

DETAILED DESCRIPTION

The following includes definitions of selected terms employed herein. The definitions include various examples and/or forms of components that fall within the scope of a term and that may be used for implementation. The examples are not intended to be limiting.

A “processor,” as used herein, processes signals and performs general computing and arithmetic functions. Signals processed by the processor may include digital signals, data signals, computer instructions, processor instructions, messages, a bit, a bit stream, or other computing that may be received, transmitted and/or detected.

A “bus,” as used herein, refers to an interconnected architecture that is operably connected to transfer data between computer components within a singular or multiple systems. The bus may be a memory bus, a memory controller, a peripheral bus, an external bus, a crossbar switch, and/or a local bus, among others. The bus may also be a vehicle bus that interconnects components inside a vehicle using protocols, such as Controller Area network (CAN), Local Interconnect Network (LIN), among others.

A “memory,” as used herein may include volatile memory and/or non-volatile memory. Non-volatile memory may include, for example, ROM (read only memory), PROM (programmable read only memory), EPROM (erasable PROM) and EEPROM (electrically erasable PROM). Volatile memory may include, for example, RAM (random access memory), synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), and/or direct RAM bus RAM (DRRAM).

An “operable connection,” as used herein may include a connection by which entities are “operably connected”, is one in which signals, physical communications, and/or logical communications may be sent and/or received. An operable connection may include a physical interface, a data interface and/or an electrical interface.

A “vehicle,” as used herein, refers to any moving vehicle that is powered by any form of energy. A vehicle may carry human occupants or cargo. The term “vehicle” includes, but is not limited to: cars, trucks, vans, minivans, SUVs, motorcycles, scooters, boats, personal watercraft, and aircraft. In some cases, a motor vehicle includes one or more engines.

Generally described, the present disclosure provides systems and methods for estimating saliency in a drive scene.

Turning toFIG. 1, a schematic view of an example operating environment100of a vehicle data acquisition system110according to an aspect of the disclosure is provided. The vehicle data acquisition system110may reside within a vehicle102. The components of the vehicle data acquisition system110, as well as the components of other systems, hardware architectures, and software architectures discussed herein, may be combined, omitted or organized into various implementations.

The vehicle102may generally include an electronic control unit (ECU)112that operably controls a plurality of vehicle systems. The vehicle systems may include, but are not limited to, the vehicle data acquisition system110, among others, including vehicle HVAC systems, vehicle audio systems, vehicle video systems, vehicle infotainment systems, vehicle telephone systems, and the like. The data acquisition system110may include a plurality of multi-modal sensors including, but not limited to, an interior camera120, a road camera121, a Lidar laser scanner122, and a vehicle controller area network (CAN) reader123. In further aspects, the vehicle data acquisition system110may also collect data from multi-modal sensors attached to a driver, such as a chestband and/or a wristband worn by the driver. In some aspects, the plurality of multi-modal sensors may be connected to the ECU112to provide images of the environment surrounding the vehicle102and/or provide data related to operation of the vehicle102. In further aspects, the ECU112may also collect data from the multi-modal sensors attached to the driver.

In some aspects, the interior camera120may capture images and/or video of the driver, while the road camera121may capture images and/or video of the road. In some aspects, the interior camera120and the road camera121may be installed below a rear mirror of the vehicle. In further aspects, the interior camera120and the road camera121may capture video images of the driver and the road, respectively, at 15 frames per second (FPS).

In further aspects, the Lidar laser scanner122may operate as illustrated inFIG. 3. For example, the Lidar laser scanner122may operate at a distance of between 0 and 200 meters. The Lidar laser scanner122may include a 145 degree horizontal field of view (FOV), and may be installed in a middle of a frontal bumper of the vehicle102to capture a distance between frontal objects and the vehicle102.

In further aspects, the CAN reader123may provide vehicle information including, but not limited to, speed, braking, throttle, steering, yaw rate, and velocity, etc. In further aspects, the driver may wear the chestband to record an electrocardiogram (ECG) signal and a breath rate, and the wristband to record a galvanic skin response GSR.

In some aspects, each of the signals from the interior camera120, road camera121, Lidar laser scanner122, CAN reader123, as well as the chestband/wristband are synchronized to a same system clock. In this way, the data collected by the multi-modal sensors may include both internal and external signals capturing the interaction between the driver and the driving environment. For example, physiological signals may be used to determine a driver's mental state, e.g., stress, anxiety, comfort, etc., that is often difficult to directly estimate from driver face, gaze and gesture.

Table I summarizes some of the multi-modal signals, and their respective frequencies, collected by the interior camera120, road camera121, Lidar laser scanner122, and vehicle controller area network (CAN) reader123, respectively, in addition to the wristband sensor and/or chestband sensor worn by the driver.

The data acquisition system110may also include a processor114and a memory116that communicate with the interior camera120, road camera121, Lidar laser scanner122, vehicle controller area network (CAN) reader123, wristband sensor, chestband sensor, head lights124, tail lights126, communications device130, and automatic driving system132.

The ECU112may include internal processing memory, an interface circuit, and bus lines for transferring data, sending commands, and communicating with the vehicle systems. The ECU112may include an internal processor and memory, not shown. The vehicle102may also include a bus for sending data internally among the various components of the vehicle data acquisition system110.

The vehicle102may further include a communications device130(e.g., wireless modem) for providing wired or wireless computer communications utilizing various protocols to send/receive electronic signals internally with respect to features and systems within the vehicle102and with respect to external devices. These protocols may include a wireless system utilizing radio-frequency (RF) communications (e.g., IEEE 802.11 (Wi-Fi), IEEE 802.15.1 (Bluetooth®)), a near field communication system (NFC) (e.g., ISO 13157), a local area network (LAN), a wireless wide area network (WWAN) (e.g., cellular) and/or a point-to-point system. Additionally, the communications device130of the vehicle102may be operably connected for internal computer communication via a bus (e.g., a CAN or a LIN protocol bus) to facilitate data input and output between the electronic control unit112and vehicle features and systems. In an aspect, the communications device130may be configured for vehicle-to-vehicle (V2V) communications. For example, V2V communications may include wireless communications over a reserved frequency spectrum. As another example, V2V communications may include an ad hoc network between vehicles set up using Wi-Fi or Bluetooth®.

The vehicle102may include head lights124and tail lights126, which may include any conventional lights used on vehicles. The head lights124and tail lights126may be controlled by the vehicle data acquisition system110and/or ECU112for providing various notifications. For example, the head lights124and/or tail lights126may be activated or controlled to provide desirable lighting when scanning the environment of the vehicle102.

FIG. 2illustrates an example network200for managing the data acquisition system110. The network200may be a communications network that facilitates communications between multiple systems. For example, the network200may include the Internet or another internet protocol (IP) based network. The network200may enable the data acquisition system110to communicate with a mobile device210, a mobile service provider220, or a manufacturer system230. The data acquisition system110within the vehicle102may communicate with the network200via the communications device130. The data acquisition110may, for example, data collected by the multi-modal sensors.

The manufacturer system230may include a computer system, as shown with respect toFIG. 9described below, associated with one or more vehicle manufacturers or dealers. The manufacturer system230may include one or more databases that store data collected by the front camera120, roof camera121, and/or the rear camera122. The manufacturer system230may also include a memory that stores instructions for executing processes for analyzing a driver's brake behavior and a processor configured to execute the instructions.

In some aspects, a braking event may be defined as a temporal sequence, where a brake is engaged. In some aspects, an intensity of a braking event I may be defined as a maximum deceleration during braking and may be calculated based on the velocity recorded by the CAN reader123, as shown in Equations (1) and (2):
I=|min(Δ([vt0,vt1, . . . ,vtn]))  (1),
where
Δ([x0,x1, . . . ,xn])=[x1−x0, . . . ,xn−xn-1]  (2).

In some aspects, during routine driving, most of the braking events include low to medium brake intensity, thereby indicating smooth braking. As such, any braking event falling below a threshold intensity value may be considered low to medium brake intensity event, whereas any braking events falling above the threshold intensity value may be considered a hard braking event. For example,FIG. 4illustrates an example graph400demonstrating a braking intensity (m/s) and a number of braking events for each braking intensity. As shown inFIG. 4, most braking events have a brake intensity falling below an intensity of about 12 m/s. Thus, in some aspects, a threshold intensity value405may be set to 12 m/s; however, it should be understood by those of ordinary skill in the art that this is merely an example threshold intensity value, and that other threshold intensity values are further contemplated according to aspects of the present disclosure. In some aspects of the present disclosure, the manufacturer system230may be configured to define braking events exceeding the threshold intensity value405as hard braking events and to further analyze these hard braking events, while braking events falling below the threshold intensity value405may be disregarded.

The manufacturer system230may also extract physiological signals of a driver, such as GSR and breath rate, during a known hard braking event to determine whether it was a result of a reactive hard braking event. In turn, the manufacturer system230may compare the extracted physiological signals with physiological signals of a driver during an intended hard braking event.FIG. 5illustrates a plurality of example graphs comparing the physiological signals of the driver between a reactive hard braking event and an intended hard braking event. For example, as shown inFIG. 5, graphs500a-500dillustrate that the GSR signal increases and the breath rate decreases during the reactive hard braking event; whereas, the GSR signal remains stable and the breath rate slightly increases during the intended hard braking event.

FIG. 6illustrates a plurality of example graphs plotting the brake intensity versus physiological signals of a driver during hard braking events, according to aspects of the present disclosure. For example,FIG. 6illustrates graphs600a,600bplotting the brake intensity versus a GSR signal and a breath rate signal, respectively, during hard braking events. As shown inFIG. 6, as the brake intensity increases, the GSR signal increases and the breath rate signal decreases during some hard braking events. On the other hand, during other hard braking events, both the GSR and breath rate signals remain stable. Thus, in some aspects, a first threshold value610may be set for the GSR signal and a second threshold value615may be set for the breath rate signal. It should be understood by those of ordinary skill in the art that the physiological signals and the threshold values shown inFIG. 6are merely example physiological signals and corresponding threshold values, and that other the physiological signals and threshold values are further contemplated according to aspects of the present disclosure.

In some aspects, the first and second threshold values610,615may be used to distinguish between reactive hard braking events and intended hard braking events. For example, for GSR signals that increase beyond the first threshold value610during the hard braking event, the hard braking event may correspond to a reactive hard braking event; whereas for GSR signals that remain below the first threshold value610during the hard braking event, the hard braking event may correspond to an intended hard braking event. Similarly, for the breath rate signals, for breath rate signals that decrease beyond the second threshold value615during the hard braking event, the hard braking event may correspond to a reactive hard braking event; whereas for breath rate signals that remain above the second threshold value615during the hard braking event, the hard braking event may correspond to an intended hard braking event.

In further aspects of the present disclosure, the manufacturer system230may apply a Lasso regression analysis and a classification framework to explore the representation of road environment that are discriminative between reactive hard braking and intended hard braking. Through the Lasso regression analysis and the classification framework, the manufacturer system230may identify road features that are representative of unexpected road changes. As discussed with respect to Table 1, both road video and Lidar scanner signals may be recorded. In some aspects, the Lidar scanner signal may provide an accurate distance measurement between the a frontal object and a vehicle, e.g., the vehicle102ofFIG. 1.

In some aspects, the Lidar scanner signals may include over 1000 depth measurements per scan covering 145 degree horizontal field of view and 3.2 degree vertical view. In some aspects, the Lidar scanner signals may be down-sampled to a fixed-length. For example, the Lidar scanner signals may be down-sampled to a fixed-length, e.g.,291signals, covering a 145 degree horizontal field of view with a 0.5 degree resolution. In further aspects, as shown inFIG. 3, a center frontal direction may be set as the 0 degree, a direction to the left of the vehicle may be set as a negative degree direction, e.g., −72.5 to 0 degrees, and a directions to the right of the vehicle as may be set as a positive degree direction, e.g., 0 to 72.5 degrees.FIG. 7illustrates a comparison of the down-sampled Lidar scanner signal and the original Lidar scanner signal, as well as a corresponding road image.

In further aspects of the present disclosure, the manufacturer system230may estimate four statistics over a one second window before the start of each of the hard braking events for each of the 291 down-sampled Lidar signals to generate 1164 dimensional vectors, i.e., road features, that may be used as independent variables for the Lasso regression and classification framework analysis. In some aspects, the statistics may include a mean, a standard deviation, a maximum, and a minimum. In further aspects, when building linear regression models, hard braking events may be labeled as dependent variable, where a value of one may represent a reactive hard braking event and a value of zero may represent an intended hard braking event.

In further aspects of the present disclosure, a least absolute shrinkage and selection operator regression, e.g., Lasso regression, may be applied to analyze features that are correlated with the reactive hard braking events and the intended hard braking events. A person of ordinary skill in the arts would understand that Lasso is a regularized linear regression model. Given N observations {(x1, y1) . . . (xN, yN)}, where ximay correspond to an independent variable and yimay correspond to a dependent variable, the Lasso regression may solve equation (3) for the intercept and coefficient vectors:

minβ0,β_⁢{12⁢⁢N⁢∑i=1N⁢⁢(yi-β0-xiT⁢β_)+λ⁢∑j=1n⁢⁢βj},(3)
where |βj| may be the L1norm. The term λΣj=1n|βj| may serve as a penalty for high relevant independent variables in the model. As a result, the optimization may reduce the number of non-zero coefficients, and may select highly relevant independent variables, where λ may control the degree of penalty. In some aspects, a higher value of λ may increase the penalty, and lead to fewer non-zero coefficients, for example, when the independent variables are highly correlated.

In further aspects, setting the λ value may be determined by cross-validation. In some aspects, a 10 fold cross-validation may be used to select the λ value. In such an example, a resulting Lasso model may achieve an R-square value of 0.603, thereby indicating a high correlation between the prediction and the ground truth value (p=0.777), as illustrated inFIG. 8A.

In further aspects of the disclosure, these results indicate that it may be possible to determine a traffic environment that may lead to unexpected braking events. For example, out of the 1164 Lidar features, 34 may be selected by the Lasso model, where features with a positive coefficient may be more related to reactive hard braking, and thus indicative of unexpected road events. In some aspects, features with a higher positive coefficient may indicate that unexpected hard braking events may happen when a frontal object is present, such as when a driver is following another vehicle. Alternatively, reactive hard braking may occur when a traffic light changes or when a driver fails to check cross traffic at a stop sign. On the other hand, the features having a higher negative coefficient may be related to objects in an adjacent lane, and as the distance to the adjacent road object decreases, the chance of an unexpected braking event increases.

In further aspects of the present disclosure, the manufacturer system230may analyze the Lidar features to classify hard braking events as either reactive hard braking events or intended hard braking events. For example, the manufacturer system230may use a binary classification using logistic regression classifier to identify Lidar features that are useful to distinguishing between reactive hard braking events and intended hard braking events. In some aspects, a chi-square feature selection may be used to reduce the Lidar features. For example, for the feature selection, chi-square statistics may be used to compare each feature variable and a target variable based on a relationship between the feature variable and the target variable. In some aspects, features with high values for the chi-squared statistic may be important features of which the target variable are dependent.

In further aspects of the present disclosure, selected features may be evaluated using a logistic regression classifier. In some instances, the data may be not balanced, as such, three metrics including precision (P), recall (R) and F-score (F) may be used to evaluate the classifiers. For example, precision may be a fraction of the classified samples with a correct label, and recall may be a fraction of the samples being classified correctly. In some aspects, the precision and recall for both classes may be calculated, and then these two metrics may be used to calculate the F-score using equation (4), which may be used as a single measurement to evaluate the performance of the classifiers for the unbalanced dataset.

FIG. 8Bshows the classification performance using the weighted F-score as the number of feature changes. Table II shows a sample classification performance with 10 selected features.

In some aspects, an average of 0.76 F-score may be achieved with the logistic regression classifier using 10 selected features, where the 10 selected features by Chi Square feature selection include: a mean feature of the Lidar signal along directions (in degree) 1 and 1.5; a standard deviation feature along direction −34; a max feature along direction 0, 0.5, 1, 1.5, 2, 3; and a minimum feature along direction −13.5.

In some aspects, the manufacturer system230may generate a signal notifying the vehicle102of the features that indicate whether a respective hard braking event is a reactive hard braking event or an intended hard braking event. The manufacturer system230may notify the vehicle102of these features, such that the ADAS functionality of the vehicle102may be activated upon detection of a similar feature in the future. For example, the manufacturer system230may generate a signal notifying the vehicle102that a reactive hard braking event may have been caused by, for example, a swerving vehicle and/or a red traffic light, such that an ADAS functionality, for example, a braking system, may be activated whenever a car swerves and/or a red light is encountered. In this way, the functionality of the vehicle102may be controlled based on the features causing reactive hard braking events and/or intended hard braking events. Additionally, or alternatively, the manufacturing system230may generate a signal notifying the vehicle102of the features that may have caused a reactive hard braking event and/or an intended hard braking event, such that the vehicle102may generate a warning signal, for example, an audible or a visual warning, to alert the driver. In this way, the driver may respond to the alert to avoid potentially hazardous driving situations, e.g., avoid an accident.

Aspects of the present invention may be implemented using hardware, software, or a combination thereof and may be implemented in one or more computer systems or other processing systems. In an aspect of the present invention, features are directed toward one or more computer systems capable of carrying out the functionality described herein. An example of such a computer system900is shown inFIG. 9.

Computer system900includes one or more processors, such as processor904. The processor904is connected to a communication infrastructure906(e.g., a communications bus, cross-over bar, or network). Various software aspects are described in terms of this example computer system. After reading this description, it will become apparent to a person skilled in the relevant art(s) how to implement aspects of the invention using other computer systems and/or architectures.

Computer system900may include a display interface902that forwards graphics, text, and other data from the communication infrastructure906(or from a frame buffer not shown) for display on a display unit930. Computer system900also includes a main memory908, preferably random access memory (RAM), and may also include a secondary memory910. The secondary memory910may include, for example, a hard disk drive912, and/or a removable storage drive914, representing a floppy disk drive, a magnetic tape drive, an optical disk drive, a universal serial bus (USB) flash drive, etc. The removable storage drive914reads from and/or writes to a removable storage unit918in a well-known manner. Removable storage unit918represents a floppy disk, magnetic tape, optical disk, USB flash drive etc., which is read by and written to removable storage drive914. As will be appreciated, the removable storage unit918includes a computer usable storage medium having stored therein computer software and/or data.

Alternative aspects of the present invention may include secondary memory910and may include other similar devices for allowing computer programs or other instructions to be loaded into computer system900. Such devices may include, for example, a removable storage unit922and an interface920. Examples of such may include a program cartridge and cartridge interface (such as that found in video game devices), a removable memory chip (such as an erasable programmable read only memory (EPROM), or programmable read only memory (PROM)) and associated socket, and other removable storage units922and interfaces920, which allow software and data to be transferred from the removable storage unit922to computer system900.

Computer system900may also include a communications interface924. Communications interface924allows software and data to be transferred between computer system900and external devices. Examples of communications interface924may include a modem, a network interface (such as an Ethernet card), a communications port, a Personal Computer Memory Card International Association (PCMCIA) slot and card, etc. Software and data transferred via communications interface924are in the form of signals928, which may be electronic, electromagnetic, optical or other signals capable of being received by communications interface924. These signals928are provided to communications interface924via a communications path (e.g., channel)926. This path926carries signals928and may be implemented using wire or cable, fiber optics, a telephone line, a cellular link, a radio frequency (RF) link and/or other communications channels. In this document, the terms “computer program medium” and “computer usable medium” are used to refer generally to media such as a removable storage drive918, a hard disk installed in hard disk drive912, and signals928. These computer program products provide software to the computer system900. Aspects of the present invention are directed to such computer program products.

Computer programs (also referred to as computer control logic) are stored in main memory908and/or secondary memory910. Computer programs may also be received via communications interface924. Such computer programs, when executed, enable the computer system900to perform the features in accordance with aspects of the present invention, as discussed herein. In particular, the computer programs, when executed, enable the processor904to perform the features in accordance with aspects of the present invention. Accordingly, such computer programs represent controllers of the computer system900.

In an aspect of the present invention where the invention is implemented using software, the software may be stored in a computer program product and loaded into computer system900using removable storage drive914, hard drive912, or communications interface920. The control logic (software), when executed by the processor904, causes the processor904to perform the functions described herein. In another aspect of the present invention, the system is implemented primarily in hardware using, for example, hardware components, such as application specific integrated circuits (ASICs). Implementation of the hardware state machine so as to perform the functions described herein will be apparent to persons skilled in the relevant art(s).

FIG. 10illustrates a flowchart for a method of analyzing a driver's brake behavior. The method1000may include receiving data from one or more multi-modal sensors1010. The data may include one or more characteristics of a vehicle and one or more physiological signals of the driver. The method1000may also include determining whether each braking event of a plurality of braking events is a hard braking event1020. The method1000may further include analyzing the one or more physiological signals of the driver to determine whether each of the hard braking events is a reactive hard braking event or an intended hard braking event1030. The method may also include applying a Lasso regression analysis and a classification framework for each of the hard braking events to identify features that indicate whether a respective hard braking event is a reactive hard braking event or an intended hard braking event1040. The method may include generating a signal notifying the vehicle of the features that that indicate whether a respective hard braking event is a reactive hard braking event or an intended hard braking event1050.