Automated vehicle control take-over alert timing based on infotainment activation

A system for automated operation of a vehicle includes an infotainment-device and a controller. The infotainment-device is operable to provide an infotainment-activity to an operator of a vehicle. The controller is operable to estimate a take-over-interval for an operator to prepare for a mode-transition from automated-control of the vehicle by the controller to manual-control of the vehicle by the operator. The take-over-interval is determined based on the infotainment-activity of the operator. The controller is operable to notify the operator that the mode-transition is needed at least the take-over-interval prior to a take-over-time.

TECHNICAL FIELD OF INVENTION

This disclosure generally relates to a system for automated operation of a vehicle, and more particularly relates to a system that estimates a take-over-interval for an operator to prepare to assume manual-control of the vehicle, where the take-over-interval is determined based on an infotainment-activity of the operator.

BACKGROUND OF INVENTION

Passenger vehicles (e.g. automobiles) with various degrees of automation have been proposed. Vehicles equipped with automated speed control, commonly known as cruise control, are well-known. Fully automated or autonomous vehicles where the operator is not engaged with any aspect of operating the vehicle are being developed. However, it is contemplated that unexpected traffic scenarios or situations will arise when it will be preferable to have the operator assume manual control of the vehicle.

SUMMARY OF THE INVENTION

If a vehicle is fully automated, the operator may have nothing to do with actually driving the vehicle. That is, the operator may do little more than designate a destination for an automated vehicle, and the automated vehicle takes care of all further operation. This will allow the operator to engage in a variety of infotainment activities such as participate in a conference call, draft a memo, play a video-game such as a virtual-reality game, or watch a movie or sporting event. However, if a situation arises where the operator must assume manual-control of the vehicle, and the operator is engrossed in some infotainment-activity that might delay or slow the preparation of the operator to assume manual control, then it is advantageous for the vehicle automation to give some advanced warning that manual-control is needed.

In accordance with one embodiment, a system for automated operation of a vehicle is provided. The system includes an infotainment-device and a controller. The infotainment-device is operable to provide an infotainment-activity to an operator of a vehicle. The controller is operable to estimate a take-over-interval for the operator to prepare for a mode-transition from automated-control of the vehicle by the controller to manual-control of the vehicle by the operator. The take-over-interval is determined based on the infotainment-activity. The controller is operable to notify the operator that the mode-transition is needed at least the take-over-interval prior to a take-over-time.

DETAILED DESCRIPTION

FIG. 1illustrates a non-limiting example of a system10for automated operation of a vehicle12. In general, the system10includes a controller14configured to autonomously operate (i.e. drive) the vehicle12using vehicle-controls16to control steering and acceleration/braking of the vehicle12. The controller14may include a processor (not specifically shown) such as a microprocessor or other control circuitry such as analog and/or digital control circuitry including an application specific integrated circuit (ASIC) for processing data as should be evident to those in the art. The controller14may include memory (not specifically shown), including non-volatile memory, such as electrically erasable programmable read-only memory (EEPROM) for storing one or more routines, thresholds and captured data. The one or more routines may be executed by the processor to perform steps for operating the vehicle12under automated-control20or manual-control18as in more detail below.

The system10may include an object-detection device22operable to detect an object24such as an other-vehicle, pedestrian, roadway marker, and the like. The controller14receives information from the object-detection device22that is useful to maneuver the vehicle12. The object-detection device22may include, but is not limited to, an imager device such as a camera, a radar device, and/or a lidar device, as will be recognized by those in the art. The system10may also include a global-positioning system or GPS (not shown) to provide further information to maneuver the vehicle12, as will be recognized by those in the art. The vehicle-controls16include manual-controls26so an operator28is able to manually operate or drive the vehicle12when the need arises, and may include control-override devices30so the controller14can control the steering, acceleration, and braking of the vehicle12even if the operator28is, for example, moving the steering-wheel (not shown) of the vehicle12.

The system10may include an infotainment-device32operable to provide an infotainment-activity34to the operator28of the vehicle12. As used herein, infotainment refers to any form of information or entertainment that can be received or perceived by the operator28, and the operator28does so by way of the infotainment-device32. In some instances, infotainment may include interaction by the operator28as will be explained in more detail. By way of example and not limitation, infotainment that is merely received by the operator28includes music, movies, sporting-events, and audio-books. Examples of infotainment-activities that may elicit some sort of response or interaction by the operator28includes video-games, surfing the web, designating a new or revised destination on a navigation-device of the vehicle12, text messaging, typing a memo, or making a presentation during a phone or video conference call.

As any of the infotainment-activities suggested may distract or occupy the operator28to the point where more than one or two seconds notice is necessary for the operator28to be prepared to take or assume the manual-control18of the vehicle12, the controller14is advantageously configured or operable to estimate a take-over-interval38for the operator28to prepare for a mode-transition40from automated-control20of the vehicle12by the controller14to manual-control18of the vehicle12by the operator28. The take-over-interval38is determined based on the infotainment-activity34engaged by the operator28. The controller14is also operable to notify the operator28by way of a notification-device42that the mode-transition40is needed. The notification-device42is activated at least the take-over-interval38prior to a take-over-time44, which is the time that the controller14estimates that the operator28need to prepare for the mode-transition40to manual-control18from automated-control20.

Some non-limiting examples of situations when the mode-transition40to manual-control18from automated-control20is necessary include construction-zones or accident-sites where the travel-path of the vehicle12is diverted from an expected path that is based on stored roadway information. For example, if an accident is blocking the roadway, the vehicle12may need to be manually steered off the roadway into an area that is not well-known by the controller14because the area is not included in a roadway database. Another example is when weather conditions such as snow and/or ice obscure roadway markers or obstacles, or accumulate on the vehicle12in a way that obscures the view of the roadway and surrounding area by a camera or lidar device that is part of the object-detection device22.

Further examples of scenarios or situations that may require additional time for the operator to prepare for the mode-transition40include when the operator is playing a video-game that includes or necessitates wearing a display-headset such as a virtual-reality headset that must be removed before the operator is prepared for the mode-transition40. Another example is when the operator28is participating in a conference-call in such a way that the operator28is operating a personal-computing-device (e.g. a lap-top PC or tablet-type PC) that must be deactivated or otherwise placed in a secure location before the operator28is prepared for the mode-transition40.

To better accommodate the operator28during the infotainment-activity34selected, the interior of the vehicle12may be reconfigurable to be more comfortable. For example, a seat occupied by the operator28may be rotatable so the operator28can sit sideways in the vehicle12, or may be able to be pushed back so far that the steering wheel does not interfere with the use of a keyboard or other input-control device (e.g. mouse, trackball, or touch-pad) for a personal-computing device (PC). For example, if the operator28is participating in a conference-call includes positioning a seat (e.g. adjusting the seat-position) of the vehicle12to other than what may typically be referred to as a manual-control-position, then the seat-position must be returned to the manual-control-position for the operator to prepare for the mode-transition.

While some may not consider certain infotainment-activities such as listening to music as distracting, most would agree that other infotainment-activities such as watching a sports-event are certainly distracting. Accordingly, when the infotainment-activity34is characterized as distracting, the infotainment-activity is preferably terminated for the operator28to prepare for the mode-transition40. The characterization of the infotainment-activity34as distracting may be based on a distraction level that is preassigned to each of the various infotainment-activities. It is contemplated that the distraction level may be modified based on prior behavior exhibited by the operator28. For example, if the operator28previously took longer than unexpected to turn-off a sports-event, the take-over-interval38associated with watching a sports-event may be increased.

Prior to making the mode-transition40from automated-control20to manual-control18, there is preferably some determination that the operator28is indeed ready to take manual-control of the vehicle12. The system may include an operator-detection device46operable or useable by the controller14to determine a readiness-state48(e.g. ready/not ready) of the operator28. The operator-detection device46may include, but is not limited to, a camera to determine a body-orientation such as a facing-direction and/or a head-pose of the operator28, a capacitive based pedal-proximity sensor to determine a foot-position of the operator28(e.g. determine if the foot of the operator covering the accelerator pedal and/or the brake-pedal), and a hand-wheel-proximity sensor to determine if the operator's hands are in contact with the steering-wheel. If the operator28has a head-pose that indicates that the operator28is looking at the roadway, a facing-direction that is normal for manual-control18(e.g. body facing forward), the feet are covering or in contact with the brake-pedal and/or accelerator-pedal, and the operator28is touching the steering wheel, then the operator is likely ready to assume manual-control18of the vehicle12.

By way of example and not limitation, if the infotainment-activity34is something that is easily stopped and the operator28is in a position suitable for assuming manual control except for moving the hands and feet a few centimeters to make contact with the manual-controls26, the take-over-interval may be five seconds (5 s). However, if the operator28is holding a PC or input-control (e.g. keyboard, game-control-device) and the seat is moved back such that the operator28cannot readily operator the steering wheel or the brakes, a suitable value for the take-over-interval38may be twenty seconds (20 s)

Even if the operator28is in the proper position described above for manual-control18, but the operator has not terminated the infotainment-activity34(e.g. did not pause or turn-off a movie), then the mode-transition40may be delayed when the operator28does not terminate the infotainment-activity34. If the assumption of manual-control18by the operator28is imperative, the controller14may be configured to terminate the infotainment-activity34when the operator28does not terminate the infotainment-activity34. If the infotainment-activity34is something that the controller14is unable to terminate, the controller14may be configured to stop the vehicle12when the operator28does not terminate the infotainment-activity34. If the roadway on which the vehicle is traveling includes a shoulder, the controller may be further configured to steer the vehicle12to a shoulder of a roadway when the operator28does not terminate the infotainment-activity34.

FIG. 2illustrates a non-limiting example of a method200for automated operation of the vehicle12.

Step210, AUTOMATED CONTROL, may include the operator28engaging the automated-control20by, for example, a voice command from the operator28that is detected by a microphone connected to the controller14, and/or the operator28operating an input-control to designate a destination to a navigation device for the vehicle12.

Step215, MODE-TRANSITION?, may include the controller14determining that an upcoming roadway is not sufficiently defined in a data-base for automated-control, or detecting via vehicle-to-vehicle (V2V) or vehicle-to-infrastructure (V2I) that an accident has occurred on a roadway that the vehicle12must travel. If a mode-transition40is not necessary (NO), Step215may be repeated until a situation is detected that warrants a mode-transition40. If such a situation is detected (YES), the method200proceeds to step220. Step215may also include determining the take-over-time44based on how soon the vehicle12will arrive at a location where the mode-transition40from automated control20to manual-control18is necessary.

Step220, INFOTAINMENT ACTIVITY, may include the controller14determining if the infotainment-activity34is activated, and which of the activities is activated.

Step225, DETERMINE TAKE-OVER-INTERVAL, may include recalling from memory a previously stored value for the take-over-interval38for the particular infotainment-activity34that is activated. Step225may also include further estimating (i.e. increasing) the take-over-interval38if the operator-detection device46indicates that the operator28is well out-of-position for the operator28to assume manual-control18. For example, even though the operator28is watching a moving which could be terminated quickly, the seat in which the operator28resides may have been moved way-back or may be inclined, so some additional time is needed for the readiness-state48of the operator28to become READY.

Step230, NOTIFY OPERATOR, may include the controller14activating one or more options of the notification-device42at a time corresponding to the take-over-interval38prior to the take-over-time44.

Step235, WAIT TAKE-OVER-INTERVAL, may include the controller14activating a timing counter to wait for the take-over-interval38to expire after the operator28has been notified by the notification-device42.

Step240, READY AT TAKE-OVER-TIME?, may include the controller14verifying that the operator28is properly positioned and is at least in close contact to the manual-controls26(i.e. the operator28is READY) before proceeding to step255where the operator28assumes manual-control18of the vehicle12. However, if for some reason the operator28is NOT READY, the method200may execute some optional steps which are described below.

Step245, TURN-OFF INFOTAINMENT-DEVICE, is an optional step that may include the controller14shutting off any or all options of the infotainment-device32so the operator is not distracted. This step may also include an audible notification that, for example, the movie has been stopped so the operator28does not wonder why the movie has stopped. If the operator28is properly positioned for manual-control18, and the only reason the controller14indicated NOT READY was because the infotainment-activity34had not been stopped by the operator28, the method may optionally proceed to step255.

Step250, STOP VEHICLE, is another optional step that may include the controller14stopping the vehicle12, possibly steering the vehicle12to a shoulder on the roadway to get out of the way of other traffic. This step may be an advantageous alternative if the operator28is, for example, unable or unwilling to interrupt a conference-call and would rather have the vehicle12stop rather than have the controller forcibly terminate the conference-call.

Step255, MANUAL CONTROL, may include the notification-device issuing an audible message announcing, for example, “Manual Control Resumed”.

Accordingly, a system10for automated operation of a vehicle12, a controller14for the system10and a method200of operating the system10is provided. The system10is generally configured to estimate or anticipate how much time it will take the operator28to stop or interrupt the infotainment-activity34that the operator28is engaged with when a situation arises that requires the operator28to assume manual-control18if the vehicle12is to continue traveling