Patent Description:
The various embodiments of the invention relate generally to automotive systems and, more specifically, to a method for managing cognitive load while driving.

Oftentimes, drivers fail to focus an appropriate amount of attention on the task of driving. For example, drivers may not adjust their focus to adequately address complex driving situations attributable to traffic, the manner in which others are driving, pedestrians, road conditions, weather conditions, volume of traffic, and the like. Further, drivers typically engage in multiple, secondary in-vehicle activities that divert their attention from the primary task of driving. Such secondary in-vehicle activities may include listening to loud music, participating in conversations, texting, soothing a crying child, and so forth.

"Distracted" driving attributable to complex driving situations and secondary in-vehicle activities increases the likelihood of collisions and accidents. For example, a driver who is driving on a winding road at night while talking with a passenger and operating an entertainment system is more likely to become involved in an automobile accident than a driver who is focused solely on the task of driving along a straight road during the day. Moreover, because using in-vehicle technologies while driving has become widespread, the frequency of injuries from accidents caused by distracted driving has increased. Some examples of prevalent in-vehicle technologies are navigation systems and entertainments systems.

In general, the three primary types of driver distractions are visual distractions, manual distractions, and cognitive distractions. Many adverse driving conditions and in-vehicle activities lead to multiple types of driver distractions. For example, texting is associated with a visual distraction that causes the driver to take his or her eyes off the road, a manual distraction that causes the driver to take his or her hands off the steering wheel, and a cognitive distraction that causes the driver to take his or her mind off the task of driving.

Because the impact of cognitive distractions on a driver is more difficult to assess than the impact of visual distractions and manual distractions, most drivers are oblivious to the amount of mental resources required to perform activities and tasks. As a result, drivers typically fail to modify the driving environment or their actions to reduce their cognitive load when their level of driver distraction becomes dangerously high.

As the foregoing illustrates, more effective techniques that enable drivers to better understand their levels of cognitive load while driving would be useful.

Publication <CIT> discloses a system for determining a workload level for a driver of a vehicle. The system includes a transceiver, a positioning unit, and a controller. The transceiver is capable of receiving data from a remote location. The data includes a remote workload level and a remote geographic position associated with the remote workload level. The positioning unit is capable of determining a current position of the vehicle. The controller is configured to compare the current position of the vehicle with the remote geographic location. If the current position of the vehicle is within a predetermined range of the remote geographic position, then a workload level for the vehicle will include at least in part the remote workload level. Publication <CIT> discloses a communication system for conducting wireless communication between communication units mounted in vehicles, wherein workload information indicating a workload of a driver of each vehicle is generated. When a communication request for requesting communication is transmitted from the communication unit of a first vehicle, it is determined whether the workload of a driver of the first vehicle and the workload of a driver of a second vehicle are lower than a predetermined level. When the workload of the driver of the first vehicle and the workload of the driver of the second vehicle are determined lower than the predetermined level, an incoming call is notified to the driver of the second vehicle. Publication <CIT> pertains to a driver safety manager.

At least one advantage of the disclosed techniques is that conveying cognitive load levels to drivers and/or adjusting vehicle behavior based on cognitive load levels may increase driver safety. In particular, if the driver is exhibiting elevated cognitive loads typically associated with distracted driving, then a cognitive load driving assistant may take action to reduce the complexity of the driving situation and/or the secondary tasks that the driver is performing, thereby allowing the driver to devote appropriate mental resources to the primary driving task.

So that the manner in which the above recited features of the various embodiments can be understood in detail, a more particular description, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments and are therefore not to be considered limiting in scope, for the various embodiments may admit to other equally effective embodiments.

In the following description, numerous specific details are set forth to provide a more thorough understanding of the various embodiments. However, it will be apparent to one of skill in the art that the various embodiments may be practiced without one or more of these specific details.

<FIG> illustrates a passenger compartment <NUM> of a vehicle that is configured to implement one or more aspects of the various embodiments. As shown, the passenger compartment <NUM> includes, without limitation, a windshield <NUM> and a head unit <NUM> positioned proximate to a dashboard <NUM>. In various embodiments, the passenger compartment <NUM> may include any number of additional components that implement any technically feasible functionality. For example and without limitation, in some embodiments the passenger compartment <NUM> may include a rear-view camera.

As shown, the head unit <NUM> is located in the center of the dashboard <NUM>. In various embodiments, the head unit <NUM> may be mounted at any location within the passenger compartment <NUM> in any technically feasible fashion that does not block the windshield <NUM>. The head unit <NUM> may include any number and type of instrumentation and applications, and may provide any number of input and output mechanisms. For example, and without limitation, the head unit <NUM> typically enables the driver and/or passengers to control entertainment functionality. In some embodiments, the head unit <NUM> may include navigation functionality and/or advanced driver assistance functionality designed to increase driver safety, automate driving tasks, and the like.

The head unit <NUM> may support any number of input and output data types and formats as known in the art. For example, and without limitation, in some embodiments, the head unit <NUM> may include built-in Bluetooth for hands-free calling and audio streaming, universal serial bus (USB) connections, speech recognition, rear-view camera inputs, video outputs for any number and type of displays, and any number of audio outputs. In general, any number of sensors, displays, receivers, transmitters, etc. may be integrated into the head unit <NUM> or may be implemented externally to the head unit <NUM>. External devices may communicate with the head unit <NUM> in any technically feasible fashion.

While driving, the driver of the vehicle is exposed to a variety of stimuli that are related to either the primary driving task and/or any number of secondary tasks. For example, and without limitation, the driver could see lane markers <NUM>, a pedestrian <NUM>, a cyclist <NUM>, and a police car <NUM> via the windshield <NUM>. In response, the driver could steer the vehicle to track the lane markers <NUM> while avoiding the pedestrian <NUM> and the cyclist <NUM> and apply the brake pedal to allow the police car <NUM> to cross the road in front of the vehicle. Further, and without limitation, the driver could concurrently participate in a conversation <NUM>, listen to music <NUM>, and attempt to soothe a crying baby <NUM>. Challenging driving environments and secondary activities typically increase the cognitive load of the driver and may contribute to an unsafe driving environment for the driver and for objects (other vehicles, the pedestrian <NUM>, etc.) in the proximity of the vehicle. In general, the head unit <NUM> includes functionality to enable the driver to efficiently perform both the primary driving task and certain secondary tasks as well as functionality designed to increase driver safety while performing such tasks.

<FIG> is a more detailed illustration of the head unit <NUM> of <FIG>, according to various embodiments. As shown, the head unit <NUM> includes, without limitation, a processor <NUM> and a system memory <NUM>. The processor <NUM> and the system memory <NUM> may be implemented in any technically feasible fashion. For example, and without limitation, in various embodiments, any combination of the processor <NUM> and the system memory <NUM> may be implemented as a stand-alone chip or as part of a more comprehensive solution that is implemented as an application-specific integrated circuit (ASIC) or a system-on-a-chip (SoC).

The processor <NUM> generally comprises a programmable processor that executes program instructions to manipulate input data. The processor <NUM> may include any number of processing cores, memories, and other modules for facilitating program execution. The processor <NUM> may receive input from drivers and/or passengers of the vehicle via any number of user input devices <NUM> and generate pixels for display on the display device <NUM>. The user input devices <NUM> may include various types of input devices, such as buttons, a microphone, cameras, a touch-based input device integrated with a display device <NUM> (i.e., a touch screen), and other input devices for providing input data to the head unit <NUM>.

The system memory <NUM> generally comprises storage chips such as random access memory (RAM) chips that store application programs and data for processing by the processor <NUM>. In various embodiments, the system memory <NUM> includes non-volatile memory such as optical drives, magnetic drives, flash drives, or other storage. In some embodiments, a storage <NUM> may supplement or replace the system memory <NUM>. The storage <NUM> may include any number and type of external memories that are accessible to the processor <NUM>. For example, and without limitation, the storage <NUM> may include a Secure Digital Card, an external Flash memory, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

As shown, the system memory <NUM> includes, without limitation, an entertainment subsystem <NUM>, a navigation subsystem <NUM>, and an advanced driver assistance system (ADAS) <NUM>. The entertainment subsystem <NUM> includes software that controls any number and type of entertainment components, such as an AM/FM radio, a satellite radio, an audio and video computer files player (e.g., MP3 audio files player), an optical media player (e.g., compact disc (CD) player), and so forth. In some embodiments, any number of entertainment components may be included in the head unit <NUM> and any number of entertainment components may be implemented as stand-alone devices. The navigation subsystem <NUM> includes any number and type of applications that enable a driver to efficiently navigate the vehicle. For example, the navigation subsystem <NUM> may include maps, direction routing software, and the like.

The ADAS <NUM> includes functionality that is designed to increase driver safety and/or automate driving tasks. For example, and without limitation, in various embodiments, the ADAS <NUM> may provide hill descent control, automatic parking, and the like. Notably, the functionality included in the ADAS <NUM> may supplement, enhance, and/or automate functionality provided by other components included in the vehicle to decrease the likelihood of accidents or collisions in challenging conditions and/or driving scenarios.

In general, challenging driving environments and distractions may strain that ability of the driver to devote adequate attention to the primary driving task. For example, suppose that the driver is driving the vehicle during low light conditions along a congested, windy road while texting on a cell phone. In such a scenario, the driver may not devote enough mental resources to the primary driving task to operate the vehicle in a safe manner. However, many drivers do not recognize when their cognitive loads increase past a comfortable level and they begin to exhibit unsafe driving behaviors associated with distracted driving. For this reason, the ADAS <NUM> includes, without limitation, a cognitive load driving assistant <NUM>.

In general, the cognitive load driving assistant <NUM> continually estimates the current cognitive load of the driver and determines whether the current cognitive load indicates an abnormally stressful driving environment and/or an abnormal number of distractions. If the cognitive load driving assistant <NUM> determines that the current cognitive load indicates an abnormally stressful driving environment and/or an abnormal number of distractions, then the cognitive load driving assistant <NUM> attempts to indirectly or direct modify the driving environment to reduce the cognitive load of the driver. For example, and without limitation, the cognitive load driving assistant <NUM> could notify the driver of an atypically high cognitive load and suggest alternate driving routes that are less congested than the current driving route.

The cognitive load driving assistant <NUM> may process any type of input data and implement any technically feasible algorithm to estimate current cognitive load and/or determine whether the current cognitive load negatively impacts the driver's ability to safely operate the vehicle. As shown, and without limitation, the head unit <NUM>, including the cognitive load driving assistant <NUM>, receives data via any number of driver-facing sensors <NUM> and non-driver-facing sensors <NUM>. The driver-facing sensors <NUM> may include devices capable of detecting and relaying physiological data associated with the driver. More specifically, the driver-facing sensors <NUM> may measure physiological change in the body related to cognitive load. In a complementary fashion, the non-driver-facing sensors <NUM> may include any devices capable of detecting and relaying that data that does not reflect the physiology of the driver but are related to the driving environment.

In general, the driver-facing sensors <NUM> and the non-driver-facing sensors <NUM> may include any type of sensors designed to measure any characteristic and may be implemented in any technically feasible fashion. In particular, the driver-facing sensors <NUM> may, without limitation, track specific features of the driver, such as hands, fingers, head, eye gaze, feet, facial expression, voice tone, and the like. For example, and without limitation, the driver-facing sensors <NUM> could include sensors that measure brain activity, heart rate, skin conductance, steering-wheel grip force, muscle activity, skin/body temperature, and so forth. Further, the driver-facing sensors <NUM> may include, without limitation, microphones that detect conversational context, conversational turn taking, voice tone and affect, other auditory distractions, and the like. For example, and without limitation, the driver-facing sensors <NUM> could detect that the driver is engaged in conversation with a passenger, the driver is currently speaking, the driver's voice tone indicates that the driver is drowsy, and two other passengers are engaged in a second conversation. In some embodiments, and without limitation, the driver-facing sensors <NUM> may include visual imagers that detect head position and orientation, facial features, hands movements, etc. In some embodiments, and without limitation, the driving facing sensors <NUM> may include depth sensors that detect finger and hand gestures, body posture, and as forth and/or eye gaze and pupil size tracking sensors.

In a complementary fashion, and without limitation, the non-driver-facing sensors <NUM> may track any features of the vehicle and/or environment surrounding the vehicle that are relevant to the driver. For example, and without limitation, the non-driver-facing sensors <NUM> may track vehicle control elements, such as the position of the throttle, the position of the clutch, gear selection, the location of the brake pedal, the angle of the steering wheel, and so forth. The non-driver-facing sensors <NUM> may include any number of sensors for tracking vehicle speed, position, orientation, and dynamics, such as inertial and magnetic sensors. Further, the non-driver-facing sensors <NUM> may include devices that detect and/or track stationary and/or moving objects surrounding the vehicle. Such detection sensors may include, without limitation, a front-mounted visible light imager, an infrared imager, a radio detection and ranging (RADAR) sensor, a light detection and ranging (LIDAR) sensor, a dedicated short range communication (DSRC) sensor, thermal and motion sensors, depth sensors, sonar and acoustic sensors, and the like. In some embodiments, and without limitation, the non-driver-facing sensors <NUM> may include remote sensors that provide information regarding local weather, traffic, etc..

The driver-facing sensors <NUM> and the non-driver-facing sensors <NUM> may be deployed in any technically feasible fashion. For example, and without limitation, the driver-facing sensors <NUM> and the non-driver-facing sensors <NUM> may include any number and combination of vehicle-integrated sensors, vehicle-integrated imagers, wearable devices (affixed to or worn by the driver), and remote sensors. In one example, and without limitation the, driver-facing sensors <NUM> could include steering wheel-mounted sensors that measure heart rate, skin conductance, and grip force, while the non-driver facing sensors <NUM> could include include a front-mounted visible light imager, an infrared imager, and a LIDAR sensor.

In some embodiments, the cognitive load driving assistant <NUM> may receive additional input data, referred to herein as advanced driver assistance system (ADAS) data. Such ADAS data may include, without limitation, data received from a global navigation satellite system (GNSS) receiver <NUM>, data received from the navigation subsystem <NUM>, and data received from the entertainment subsystem <NUM>. The global navigation satellite system (GNSS) receiver <NUM> determines global position of the vehicle. The GNSS receiver <NUM> operates based on one or more of the global positioning system of manmade Earth satellites, various electromagnetic spectrum signals (such as cellular tower signals, wireless internet signals, and the like), or other signals or measurements, and/or on a combination of the above items. In various embodiments, the cognitive load driving assistant <NUM> accesses global positioning data from GNSS receiver <NUM> in order to determine a current location of the vehicle. Further, in some embodiments, the cognitive load driving assistant <NUM> accesses data provided by the navigation subsystem <NUM> in order to determine a likely future location of the vehicle. In some embodiments, the cognitive load driving assistant <NUM> accesses data provided by entertainment subsystem <NUM> to assess the impact of secondary tasks, such as listening to music, on the cognitive load of the driver.

In yet other embodiments, the cognitive load driving assistant <NUM> may receive and transmit additional ADAS data including, and without limitation, automotive vehicle-to-everything (V2X) data <NUM>. The vehicle-to-everything (V2X) data <NUM> may include vehicle-to-vehicle (V2V) data, vehicle-to-infrastructure (V2I) data, and so forth. The V2X data <NUM> enables the vehicle to communicate with other objects that include V2X capabilities. For example, the vehicle may communicate with other vehicles, smartphones, traffic lights, laptops, road-side V2X units, and so forth.

After receiving the input data, the cognitive load driving assistant <NUM> computes any number of cognitive metrics that relate to the current cognitive load of the driver. Subsequently, the cognitive load driving assistant <NUM> determines whether the cognitive metrics indicate that the driver may be unable to devote a typical and/or safe amount of mental resources to the primary task of driving. In general, the cognitive load driving assistant <NUM> may compute any number of cognitive metrics and assess whether the cognitive metrics indicate an elevated current cognitive load in any technically feasible fashion. For example, and without limitation, for a subset of the driver-facing sensors <NUM>, the cognitive load driving assistant <NUM> could compute a current value for a cognitive metric and compare the current value to historical values for the cognitive metric. Substantially in parallel, for each of the remaining driver-facing sensors <NUM>, the cognitive load driving assistant <NUM> could compare current sensor data to historical sensor data. The cognitive load driving assistant <NUM> could then determine whether the results of the various comparisons indicate an elevated current cognitive load.

For example, and without limitation, the cognitive load driving assistant <NUM> could compute a weighted average of the deviations of the values of any number of cognitive metrics and any number of driver-facing sensors <NUM> from historical values to determine an average deviation. If the average deviation exceeds a certain preset limit, then the cognitive load driving assistant <NUM> could determine that the current cognitive load is elevated. In another example, the cognitive load driving assistant <NUM> could compare the value of a primary cognitive load metric to historical values of the primary cognitive load metric to determine whether the current cognitive load may be elevated. Additionally, the cognitive load driving assistant <NUM> could compare the values of any number of driver-facing sensors <NUM> to historical values to provide a confidence measurement.

In general, the cognitive load driving assistant <NUM> may compute a current cognitive load based on any number, including one, of cognitive metrics and sensor data. Further, the cognitive load driving assistant <NUM> may determine historical values for cognitive metrics, cognitive loads, and/or sensor data in any technically feasible fashion. For example, and without limitation, in some embodiments the cognitive load driving assistant <NUM> may store the current cognitive load and other relevant data, referred to herein as a "driving context" in any available memory (e.g., the system memory <NUM>). The driving context may include any number and type of data such as time of day, the location of the vehicle, detailed sensor readings, and so forth. Subsequently, the cognitive load driving assistant <NUM> may retrieve previously stored cognitive loads and driving contexts to determine historical cognitive loads at any level of situational granularity. For example and without limitation, in some embodiments, the cognitive load driving assistant <NUM> may compute an average cognitive load based on all historical cognitive loads. In other embodiments, and without limitation, the cognitive load driving assistant <NUM> may compute an average cognitive load based on the historical cognitive loads in similar driving contexts (e.g., the same time of day and/or location).

In some embodiments and without limitation, the cognitive load driving assistant <NUM> may transmit and/or receive cognitive loads and, optionally, driving contexts to other a cognitive load database <NUM> that is included in a cloud <NUM> (e.g., encapsulated shared resources, software, data, etc.). The cognitive load driving assistant <NUM> and other cognitive load driving assistants included in other vehicles may then retrieve information from the cognitive load database <NUM>. The cognitive load driving assistant <NUM> may analyze such data as part of evaluating the current cognitive load, detecting situations that involve high cognitive loads, and so forth.

In some embodiments, the cognitive load driving assistant <NUM> may transmit and/or receive cognitive loads and, optionally, driving contexts with other cognitive load driving assistants <NUM> as V2X data <NUM>. In general, the cognitive load driving assistant <NUM> may be configured to transmit and store data relevant to the cognitive load of the driver in any technically feasible fashion. Similarly, the cognitive load driving assistant <NUM> may be configured to receive and process data relevant to the cognitive loads of other drivers as well as any additional factors that may influence the cognitive load of the other drivers in any technically feasible fashion.

After determining the current cognitive load of the driver and assessing other relevant data, the cognitive load driving assistant <NUM> may perform any number of actions designed to increase the safety of the driver. As previously detailed, such relevant data may include, without limitation, such current location of the vehicle, time of day, data provided by the navigation subsystem <NUM> and the entertainment subsystem <NUM>, cognitive loads of drivers along the planned driving route, and so forth. The actions may directly or indirectly modify the driving task and any secondary tasks that may distract the driver.

For example, and without limitation, the cognitive load driving assistant <NUM> could provide feedback to the driver via the display device <NUM>. The feedback could include the current cognitive load, historical cognitive loads, and suggestions for reducing the complexity of the primary driving task, such as easier (less congested) driving routes or lanes. In some embodiments, and without limitation, the cognitive load driving assistant <NUM> may reduce human machine interface (HMI) complexity to reduce distractions. For example, and without limitation, the cognitive load driving assistant <NUM> could block incoming cellular phone calls, lower the volume of music, block non-critical alerts (e.g., low windshield washer fluid alert, etc.), and the like.

In some embodiments, the cognitive load driving assistant <NUM> may perform actions designed to preemptively increase driving safety. For example, and without limitation, suppose that the cognitive load driving assistant <NUM> detects elevated cognitive loads associated with other drivers in the proximately of the vehicle or along the driving route specified by the navigation subsystem <NUM>. To increase the vigilance of the driver, the cognitive load driving assistant <NUM> may alert the driver to expect potentially hazardous situations (e.g., accidents, dangerous curves, etc.) and/or distracted drivers.

In some embodiments and without limitation, the cognitive load driving assistant <NUM> may work in conjunction with the navigation subsystem <NUM> and/or other elements included in the ADAS <NUM> to increase driving safety based on one or more predictive heuristics. In some embodiments, the cognitive load driving assistant <NUM> could configure the navigation subsystem <NUM> to avoid locations associated with elevated cognitive loads. For example, and without limitation, if elevated historical cognitive loads are associated with a particular exit to an airport, then the cognitive load driving assistant <NUM> could configure the navigation subsystem <NUM> to preferentially select an alternative exit to the airport. According to the invention, upon detecting elevated cognitive loads of the driver or nearby drivers, the cognitive load driving assistant <NUM> modifies one or more ADAS parameters to increase the conservatism of the ADAS <NUM>. For example, and without limitation, the cognitive load driving assistant <NUM> could configure preemptive braking to activate at an earlier time or could decrease the baseline at which the ADAS <NUM> notifies the driver of a lane departure from the current driving lane.

The cognitive load driving assistant <NUM> may configure the vehicle to provide feedback to the driver in any technically feasible fashion. For example, and without limitation, the cognitive load driving assistant <NUM> may configure the vehicle to provide any combination of visual feedback, auditory feedback, haptic vibrational feedback, tactile feedback, force feedback, proprioceptive sensory feedback, and so forth. Further, the cognitive load driving assistant <NUM> may configure any features of the vehicle in any technically feasible fashion. For example, the cognitive load driving assistant <NUM> may configure the entertainment subsystem <NUM>, the navigation subsystem <NUM>, applications included in the ADAS <NUM>, and any control mechanisms provided by the vehicle via any number of control signals or via any type of interface.

As described above, in some embodiments the cognitive load driving assistant <NUM> receives cognitive load data and/or related data from other vehicles (e.g., via the cognitive load database <NUM>, the V2X data <NUM>, etc.). In operation, the cognitive load driving assistant <NUM> may leverage such shared data in any technically feasible fashion to optimize driving safety either at the current time or at a future time. For example, and without limitation, instead of comparing the current cognitive load to a personalized average cognitive load, the cognitive load driving assistant <NUM> could compare the current cognitive load to a baseline cognitive load based on collective cognitive loads of many drivers normalized for time, location, and other factors. In general, the cognitive load driving assistant <NUM> attempts to maintain the current cognitive load below the threshold represented by the baseline cognitive load.

In another example, and without limitation, the cognitive load driving assistant <NUM> may examine the average cognitive load of drivers in close proximity to the vehicle or along a driving route associated with the vehicle to detect a preponderance of elevated cognitive loads that indicates a complex situation, such as an accident. Upon detecting such an area of elevated cognitive loads, the cognitive load driving assistant <NUM> may generate a sensory warning designed to cause the driver to become more vigilant, generate a new driving route that avoids areas of elevated cognitive load, and so forth. In yet another example, and without limitation, the cognitive load driving assistant <NUM> may generate a "heat map" based on collective cognitive loads. The cognitive load driving assistant <NUM> may then suggest altering the driving environment based on the heat map. In particular, the cognitive load driving assistant <NUM> may recommend lane changes to lanes associated with lower cognitive loads; interact with the navigation subsystem <NUM> to optimize the driving route, and the like.

In general, the cognitive load driving assistant <NUM> may be configured to process any type of input data and/or compute any number of metrics related to cognitive load. Further, the cognitive load driving assistant <NUM> may be configured to increase driving safety and/or improve the driving experience based on the processed data and metrics in any technically feasible fashion. Although the cognitive load driving assistant <NUM> is described in the context of the head unit <NUM> herein, the functionality included in cognitive load driving assistant <NUM> may be implemented in any technically feasible fashion and in any combination of software and hardware. For example, and without limitation, each of the processor <NUM> and the system memory <NUM> may be embedded in or mounted on a laptop, a tablet, a smartphone, a smartwatch, a smart wearable, or the like that implements the cognitive load driving assistant <NUM>. In other embodiments, and without limitation, the cognitive load driving assistant <NUM> may be implemented as a stand-alone unit that supplements the functionality of existing vehicle safety systems. Such a stand-alone unit may be implemented as a software application that executes on any processor.

<FIG> is a more detailed illustration of the cognitive load driving assistant <NUM> of <FIG>, according to various embodiments. As shown, the cognitive load driving assistant <NUM> includes, without limitation, a pupillometery engine <NUM>, a body state engine <NUM>, a cognitive load analyzer <NUM>, a current driving context <NUM>, and a cognitive load feedback engine <NUM>. In alternate embodiments and without limitation, any number of components may provide the functionality included in the cognitive load driving assistant <NUM> and each of the components may be implemented in software, hardware, or any combination of software and hardware.

In operation, the pupillometry engine <NUM> receives pupil data from a pupil sensor <NUM> that measures the sizes of the driver's pupils via eye tracking tools. Based on the pupil data, the pupillometry engine <NUM> computes a pupil-based metric that reflects the cognitive load of the driver. The pupillometry engine <NUM> may compute the pupil-based metric in any technically feasible fashion. For example, and without limitation, the pupilloemetry engine <NUM> may analyze the pupil data to identify specific rapid changes in pupil size that are associated with increased cognitive load.

Operating substantially in parallel to the pupillometry engine <NUM>, the body state engine <NUM> receives sensor data from a heart rate sensor <NUM>, a galvanic skin response (GSR) sensor <NUM>, and a blood pressure (BP) sensor <NUM>. Based on the sensor data, the body state engine <NUM> computes a body-based metric that reflects the cognitive load of the driver. The body state engine <NUM> may compute the body-based metric in any technically feasible fashion. For example, and without limitation, the body state engine <NUM> may evaluate the heart rate in conjunction with the skin rate to determine a level of psychophysiological arousal. Further, the body state engine <NUM> may evaluate the BP to estimate an amount of blood flow in the front part of the brain. In general, the body state engine <NUM> may evaluate any type of sensor data in any combination to compute any number of metrics that reflect the cognitive load of the driver.

As shown, the cognitive load analyzer <NUM> receives the pupil-based metric and the body-based metric and computes a current cognitive load <NUM> that approximates the cognitive load of the driver. The cognitive load analyzer <NUM> may compute the current cognitive load <NUM> in any technically feasible fashion. For example, and without limitation, the cognitive load analyzer <NUM> may compute the current cognitive load <NUM> as a weighted average of the pupil-based metric and the body-based metric. In various embodiments, the cognitive load analyzer <NUM> may perform any number of comparison operations between the current value of any number of metrics and any number and type of corresponding baseline values to determine the current cognitive load <NUM>. Further, the cognitive load analyzer <NUM> may determine that the value of a particular metric is erroneous based on the values of other metrics. In some embodiments, the cognitive load analyzer <NUM> may compute the current cognitive load <NUM> based on a subset of metrics and compute a confidence value based on a different subset of metrics.

While the cognitive load driving assistant <NUM> evaluates data received via the driver-facing sensors <NUM>, the cognitive load driving assistant <NUM> also generates a current driving context <NUM> that includes data received via the non-driver-facing sensors <NUM>, data received via the GNSS receiver <NUM>, and the V2X data <NUM>. The current driving context <NUM> described the current driving environment. As shown, the current driving context <NUM> includes, without limitation, driving task parameters <NUM>, secondary task parameters <NUM>, vehicle parameters <NUM>, and environmental parameters <NUM>. In general, the driving task parameters <NUM> directly influence a driving task load that represents the mental resources required to perform the primary driving task. By contrast, the secondary task parameters <NUM> directly influence a secondary task load that represents the mental resources required to perform secondary tasks, such as operating the entertainment subsystem <NUM> or talking on a cellular phone. The vehicle parameters <NUM> and the environmental parameters <NUM> reflect circumstances that impact the mental resources required to perform the driving task and/or the secondary tasks. For example, and without limitation, the vehicle parameters <NUM> and the environmental parameters <NUM> could include the location of the vehicle, the condition of the road, the weather, the lighting conditions, and so forth.

As shown, the cognitive load feedback engine <NUM> receives the current cognitive load <NUM> and the current driving context <NUM> and generates, without limitation, feedback signals <NUM>, driving adjustment signals <NUM>, entertainment subsystem adjustment signals <NUM>, and navigation subsystem adjustment signals <NUM>. In operation, the cognitive load feedback engine <NUM> evaluates the current cognitive load <NUM> relative to a baseline cognitive load to determine whether the current cognitive load <NUM> is elevated. The cognitive load feedback engine <NUM> may determine the baseline cognitive load in any technically feasible fashion. For example, and without limitation, the baseline cognitive load could be a predetermined constant value. In some embodiments, the cognitive load feedback engine <NUM> may dynamically compute the baseline cognitive load based on any number and type of historical data associated with any number of drivers and any number of driving contexts.

If the cognitive load feedback engine <NUM> determines that the current cognitive load <NUM> is elevated relative to the baseline cognitive load, then the cognitive load feedback engine <NUM> may endeavor to reduce the current cognitive load <NUM>. Notably, the cognitive load feedback engine <NUM> may examine the current driving context <NUM> to determine how to optimize the driving environment to reduce the driving task load and/or the secondary tasks loads. In general, the cognitive load feedback engine <NUM> may generate any number of control signals in any technically feasible fashion that is consistent with the capabilities and interfaces implemented in the vehicle. Such control signals may provide, without limitation, any combination of visual feedback, auditory feedback, haptic vibrational feedback, tactile feedback, force feedback, proprioceptive sensory feedback, and so forth.

For example, and without limitation, the cognitive load feedback engine <NUM> could transmit the feedback signals <NUM> that configure the display device <NUM> to provide visual feedback regarding the current cognitive load <NUM>, historical cognitive loads, and recommendations for reducing the driving task and/or secondary tasks loads. If the vehicle is equipped with the advanced driving features, then the cognitive load feedback engine <NUM> could increase the conservatism of the vehicle via the driving adjustment signals <NUM>, such as decreasing a baseline at which the ADAS <NUM> notifies the driver of a lane departure. In some embodiments, the cognitive load feedback engine <NUM> may configure the entertainment subsystem <NUM> via the entertainment subsystem adjustment signals <NUM> to reduce distractions associated with an in-vehicle audio system. In yet other embodiments, the cognitive load feedback engine <NUM> may configure the navigation subsystem <NUM> via the navigation subsystem adjustment signals <NUM> to replace a current driving route with a new driving route that is less congested, thereby lowering the mental resources required to perform the primary driving task.

<FIG> illustrates the relationship between the current driving context <NUM> and the current cognitive load <NUM> of <FIG>, according to various embodiments. As shown, the current driving context <NUM> includes the driving task parameters <NUM>, the secondary task parameters <NUM>, the vehicle parameters <NUM>, and the environmental parameters <NUM>. In general, the driving task parameters <NUM> directly influence a driving task load <NUM> that represents the mental resources required to perform the primary driving task. By contrast, the secondary task parameters <NUM> directly influence a secondary task load <NUM> that represents the mental resources required to perform secondary tasks, such as talking on a cell phone.

Together, the driving task parameters <NUM>, secondary task parameters <NUM>, vehicle parameters <NUM>, and environmental parameters <NUM> contribute to the current cognitive load <NUM>. In particular, as the driving task load <NUM> and/or the secondary task load <NUM> increases, the current cognitive load <NUM> increases (depicted as an increasing cognitive load <NUM>) within an overall cognitive load <NUM>. The overall cognitive load <NUM> represents the total cognitive load of the driver and, within the overall cognitive load <NUM>, a baseline cognitive load <NUM> reflects the typical cognitive loads of the driver.

As shown, initially the current cognitive load <NUM> exceeds the baseline cognitive load <NUM>. In response, the cognitive load feedback engine <NUM> analyzes the current driving context <NUM> and transmits the navigation subsystem adjustment signal <NUM> "reroute via less congested roads" to the navigation subsystem <NUM>, and the entertainment subsystem adjustment signal <NUM> "mute the audio system" to the entertainment subsystem <NUM>. Subsequently, as a result of the reduction in the driving task load <NUM> and the secondary task load <NUM> attributable to, respectively, the navigation subsystem adjustment signal <NUM> and the entertainment subsystem adjustment signal <NUM>, the current cognitive load <NUM> decreases and no longer exceeds the baseline cognitive load <NUM>.

As the foregoing example illustrates, in general, if the current cognitive load <NUM> exceeds the baseline cognitive load <NUM>, then the cognitive load feedback engine <NUM> attempts to adjust the current driving context <NUM> to either directly or indirectly reduce the current cognitive load <NUM>. Accordingly, the level of driver distraction is reduced and the safety of the driver and surrounding drivers is increased.

<FIG> is a flow diagram of method steps for managing cognitive load while driving, according to various embodiments. Although the method steps are described in conjunction with the systems of <FIG>, persons skilled in the art will understand that any system configured to implement the method steps, in any order, may be used.

As shown, a method <NUM> begins at step <NUM>, where the cognitive load driving assistant <NUM> included in a vehicle receives sensor data via the driver-facing sensors <NUM> and the non-driver-facing sensors <NUM>. The driver-facing sensors <NUM> may include any number of sensors that monitor characteristics of the driver. For example and without limitation, the driver-facing sensors <NUM> may include the pupil sensor <NUM>, the heart rate sensor <NUM>, the galvanic skin response (GSR) sensor <NUM>, the blood pressure (BP) sensor <NUM>, and the like. By contrast, the non-driver-facing sensors <NUM> monitor data that is not directly related to the driver, such as environmental data and vehicle data.

At step <NUM>, the cognitive load driving assistant <NUM> computes the current cognitive load <NUM> based on the driver-facing sensor data. At step <NUM>, the cognitive load driving assistant <NUM> computes the current driving context <NUM> based on the non-driver-facing sensor data in conjunction with other relevant environmental and vehicle data. The additional data may include any type of data received in any technically feasible fashion. For example, and without limitation, the additional data could include a location of the vehicle based on data received via the GNSS receiver <NUM> and locations of other vehicles based on V2X data <NUM>. As persons skilled in the art will recognize, the cognitive load driving assistant <NUM> typically performs steps <NUM> and steps <NUM> substantially in parallel.

At step <NUM>, the cognitive load driving assistant <NUM> transmits the current cognitive load <NUM> and the current driving context <NUM> to the cognitive load database <NUM> included in the cloud <NUM>. Sharing cognitive data in this manner enables other cognitive load driving assistants <NUM> included in other vehicles to alert other drivers when the current cognitive load <NUM> indicates that the driver of the vehicle may pose a safety risk.

At step <NUM>, the cognitive load feedback engine <NUM> computes the baseline cognitive load <NUM> based on historical cognitive load data in conjunction with historical driving contexts. The historical cognitive load data and the historical driving contexts may be stored in any memory, in any technically feasible fashion, and include any amount of data associated with any number of drivers. For example, and without limitation, the historical cognitive load data could be stored in the cognitive load database <NUM> and include data for many drivers. The cognitive load feedback engine <NUM> may compute the baseline cognitive load <NUM> in any technically feasible fashion. For example, and without limitation, the cognitive load feedback engine <NUM> could compute the baseline cognitive load <NUM> as the average of all historical cognitive loads associated with the driver.

At step <NUM>, the cognitive load feedback engine <NUM> compares the current cognitive load <NUM> to the baseline cognitive load <NUM>. If, at step <NUM>, the cognitive load feedback engine <NUM> determines that the current cognitive load <NUM> is not greater than the baseline cognitive load <NUM>, then the method <NUM> returns to step <NUM> where the cognitive load driving assistant <NUM> receives new sensor data. If, however, at step <NUM>, the cognitive load feedback engine <NUM> determines that the current cognitive load <NUM> is greater than the baseline cognitive load <NUM>, then the method <NUM> proceeds directly to step <NUM>.

At step <NUM>, the cognitive load feedback engine <NUM> provides feedback to the driver indicating the elevated current cognitive load <NUM>. The cognitive load feedback engine <NUM> may provide the feedback in any technically feasible fashion and may include any additional data for reference. For example, and without limitation, the cognitive load feedback engine <NUM> could display an "evaluated cognitive load" warning via the dashboard-mounted display device <NUM>. The warning could include the current cognitive load <NUM> and an indication of how the current cognitive load <NUM> relates to the baseline cognitive load <NUM>. In another example, and without limitation, the cognitive load feedback engine <NUM> could audibly warn the driver that the current cognitive load <NUM> indicates a dangerous driving situation.

At step <NUM>, the cognitive load feedback engine <NUM> performs corrective actions designed to reduce the driving task load <NUM> and/or the secondary task load <NUM> based on the current driving context <NUM> and/or the historical driving contexts. For example, and without limitation, the cognitive load feedback engine <NUM> could determine that the current driving route is challenging and, in response, interact with the navigation subsystem <NUM> to suggest a less congested route for the vehicle. In another example, and without limitation, the cognitive load feedback engine <NUM> could determine that the number of secondary tasks that the driver is performing significantly exceeds the number of secondary tasks that the driver typically performs and, in response, interact with the entertainment subsystem <NUM> to mute the speakers.

The method <NUM> then returns to step <NUM> where the cognitive load driving assistant <NUM> receives new sensor data. The cognitive load driving assistant <NUM> continues to cycle through steps <NUM>-<NUM>, assessing the current cognitive load <NUM> to detect and attempt to minimize situations associated with elevated cognitive loads until the vehicle or the cognitive load driving assistant <NUM> is turned off.

In one embodiment, a cognitive driving assistant analyzes driver-facing sensor data and provides feedback regarding elevated driver cognitive loads to enable drivers to recognize and react to dangerous driving environments. In operation, the cognitive driving assistant processes driver-facing sensor data to compute a current cognitive load. Substantially in parallel, the cognitive driving assistant processes non driver-facing sensor data along with other relevant data, such as GNSS data, to generate a current driving context. The current driving context includes driving parameters, vehicle parameters, environmental parameters, and secondary task parameters.

Because the impacts of different "distractions," such as talking on a cellular phone, vary between individual drivers, a cognitive load feedback engine analyzes the current cognitive load of the driver with respect to historical cognitive loads of the driver in similar driving contexts. For example, if a current time included in the current driving context indicates night time lighting conditions, then the cognitive load feedback engine could compare the current cognitive load of the driver to historical cognitive loads in other driving contexts that indicate night time lighting conditions. If the cognitive load feedback engine determines that the current cognitive load is greater than the "baseline" cognitive load in similar driving contexts, then the cognitive load feedback engine initiates corrective action. The corrective action may include any type of passive feedback, such as an audible warning, or any type of active control, such as disabling a ringer of a cellular phone.

In some embodiments, the cognitive load feedback engine transmits the current cognitive load and/or the current driving context to a cognitive load database stored in a public cloud. Such information enables other cognitive load feedback engines operating in other vehicles to preemptively identify dangerous driving situations. For example, if the current cognitive load of the driver is elevated, then a cognitive load feedback engine in a second vehicle located in the immediate vicinity of the vehicle could notify the driver of the second vehicle that a distracted driver is nearby.

At least one advantage of the disclosed approach is that because the cognitive load feedback engine enables drivers to adjust driving and/or secondary task behavior based on cognitive loads, driver safety may be increased. In particular, educating drivers on their cognitive load levels and/or the cognitive load levels of nearby drivers provides drivers with an opportunity to increase their concentration on the primary driving task during challenging driving situations and/or reduce their concentration on secondary tasks. Consequently, driver safety may be increased for the driver as well as nearby drivers.

Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.

Aspects of the present invention may be embodied as a system, method or computer program product.

Aspects of the present invention are described above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions/acts specified in the flowchart and/or block diagram block or blocks. Such processors may be, without limitation, general purpose processors, special-purpose processors, application-specific processors, or field-programmable processors or gate arrays.

Claim 1:
A method for managing cognitive load while driving, the method comprising:
computing (<NUM>, <NUM>) a first cognitive load (<NUM>) associated with a first driver based on data received via one or more sensors (<NUM>, <NUM>), wherein the first driver and the one or more sensors are associated with a first vehicle;
receiving a second cognitive load that is associated with a second driver of a second vehicle; and
modifying, based on the first cognitive load and the second cognitive load, a driving task, wherein the step of modifying the driving task comprises modifying one or more parameters of an advanced driver assistance system (<NUM>) to increase a conservatism of the advanced driver assistance system (<NUM>).