SEAT CONTROL DEVICE, STATE ESTIMATION DEVICE, AND NON-TRANSITORY COMPUTER-READABLE MEDIUM

A seat control device is for a subject vehicle equipped with a driver monitor. The driver monitor is configured to monitor a driver's state. The seat control device is configured to control a state of a driver's seat on which a driver is to be seated. The seat control device includes a control grasping unit and a reclining control unit. The control grasping unit is configured to grasp whether the subject vehicle travels under an automated driving control that does not oblige the driver to monitor surroundings. The reclining control unit is configured to, when the subject vehicle travels under the automated driving control, control a reclining state of the driver's seat, such that the driver is included in a detection range of the driver monitor.

TECHNICAL FIELD

The present disclosure relates to a seat control device, a non-transitory computer-readable medium storing a seat control program, a state estimation device, and a non-transitory computer-readable medium storing a state estimation program.

BACKGROUND

Conventionally, an automated driving device to assist a driver to drive a vehicle has been developed.

SUMMARY

According to an aspect of the present disclosure, a subject vehicle is equipped with a driver monitor to monitor a driver's state. A seat control device for the subject vehicle is configured to control a state of a driver's seat.

DETAILED DESCRIPTION

Hereinafter, examples of the present disclosure will be described.

A control device according to an example of the preset disclosure enables automated driving without obligation on a driver to monitor surroundings by performing all driving operations. This control device determines whether the driver's state is abnormal based on an image captured by an occupant monitoring device that monitors the driver's state and a detection signal. When the control device determines that an abnormality occurs and that the automated driving can be hardly continued, the control device performs vehicle control to stop the vehicle at an evacuation site such as a roadside or a parking area.

When the driver is obligated to monitor the surroundings, the driver's head position, the face direction, and the like may generally fall within a predetermined region. However, when the driver is not obligated to monitor the surroundings, the driver may lose a driving posture or may operate a mobile terminal. Consequently, when there is no obligation to monitor the surroundings, there is a possibility that the driver moves more significantly than when there is an obligation to monitor the surroundings. As a result, there is a possibility that monitoring of the driver's state is hardly continued.

According to an aspect of the present disclosure, a seat control device is for a subject vehicle equipped with a driver monitor to monitor a driver's state. The seat control device is configured to control a state of a driver's seat on which a driver is to be seated. The seat control device comprises: a control grasping unit configured to grasp whether the subject vehicle travels under an automated driving control that does not oblige the driver to monitor surroundings; and a reclining control unit configured to, when the subject vehicle travels under the automated driving control, control a reclining state of the driver's seat, such that the driver is included in a detection range of the driver monitor.

According to, another aspect of the present disclosure, a non-transitory computer-readable medium stores a seat control program for a subject vehicle equipped with a driver monitor to monitor a driver's state. The seat control program is configured to control a state of a driver's seat on which a driver is to be seated. The seat control program comprises instructions configured to, when executed by a processor, to cause the processor to: grasp whether the subject vehicle travels under an automated driving control that does not oblige the driver to monitor surroundings; and control, when the subject vehicle travels under the automated driving control, a reclining state of the driver's seat, such that the driver is included in a detection range of the driver monitor.

In these examples, the reclining state of the driver's seat is controlled such that the driver is included in the detection range of the driver monitor. Therefore, even when the driver who has been released from the obligation to monitor the surroundings loses the driving posture, the driver is enabled to remain within the detection range of the driver monitor by the reclining control. As a result, the configuration enables to suppress interruption of the driver's state monitoring in the automated driving period in which obligation to monitor the surroundings does not exist.

According to an aspect of the present disclosure, a state estimation device is for a subject vehicle configured to travel under an automated driving control that does not oblige a driver to monitor surroundings. The state estimation device is configured to estimate a driver's state and comprises: a monitoring state grasping unit configured to grasp, based on a driver captured image acquired by capturing a detection range defined to include at least a head of the driver seated on a driver's seat, head information, which relates to a shape of a head in the driver captured image, and face information, which relates to parts of a face in the driver captured image; and a driver state estimation unit configured to, when the subject vehicle travels under the automated driving control, estimate the driver's state by combining the head information and the face information.

According to another aspect of the present disclosure, a non-transitory computer-readable medium stores a state estimation program for a subject vehicle configured to travel under an automated driving control that does not oblige a driver to monitor surroundings. The state estimation program is configured to estimate a driver's state. The state estimation program comprises instructions configured to, when executed by a processor, to cause the processor to: grasp, based on a driver captured image acquired by capturing a detection range defined to include at least a head of the driver seated on the driver's seat, head information, which relates to a shape of a head in the driver captured image, and face information, which relates to parts of a face in the driver captured image; and estimate, when the subject vehicle travels under the automated driving control, a state of the driver by combining the head information and the face information.

In these examples, when the subject vehicle travels under the automated driving control, the driver's state is estimated by combining the head information and the face information. Therefore, even when the driver, who has been released from the obligation of monitoring the surroundings, looks down to operate a mobile device and the like, the face information, which is hardly grasped, can be supplemented with the head information. As a result, the configuration enables to suppress interruption of the driver state monitoring in the automated driving period in which obligation to monitor the surroundings does not exist.

Hereinafter, embodiments of the present disclosure will be described.

An HCU (Human Machine Interface Control Unit) according to an embodiment of the present disclosure shown inFIGS.1and2is an interface control device used in a vehicle (hereinafter referred to as subject vehicle Am). An HCU100constitutes a human machine interface (HMI) system10of the subject vehicle Am together with multiple display devices, an audio device24, an operation device26, and the like. The HMI system10has an input interface function for receiving operations performed by an occupant such as the driver of the subject vehicle Am, and an output interface function for presenting information to the driver.

The HCU100is communicably connected to a communication bus99of an in-vehicle network1mounted on the subject vehicle Am. The HCU100is one of nodes connected to the in-vehicle network1. A surrounding monitoring sensor30, a locator35, a driving control ECU (Electronic Control Unit)40, a driving assistance ECU50a, an automated driving ECU50b, and the like are connected as nodes to the communication bus99of the in-vehicle network1. Furthermore, a seat actuator28, a driver monitor29, and the like are connected to the communication bus99. These nodes connected to the communication bus99are capable of communicating with each other. It should be noted that specific nodes among the multiple devices and the multiple ECUs and the like may be directly electrically connected with one another and can communicate with one other without using the communication bus99.

The surrounding monitoring sensor30is an automated sensor that monitors a surrounding environment of the subject vehicle Am. The surrounding monitoring sensor30includes, for example, one or multiple of camera units31, millimeter wave radars32, lidars33, and sonars34. The surrounding monitoring sensor30can detect a moving object and a stationary object from a detection range in the surroundings of the subject vehicle. The surrounding monitoring sensor30provides detection information of an object in the surroundings of the subject vehicle to the driving assistance ECU50a, the automated driving ECU50b, and the like.

The locator35includes a global navigation satellite system (GNSS) receiver, an inertial sensor, and the like. The locator35further includes a map database storing three-dimensional map data and two-dimensional map data. The locator35reads map data around the current location from the map database, and provides the map data, as locator information, to the driving assistance ECU50a, the automated driving ECU50b, and the like together with position information and direction information of the subject vehicle Am.

The driving control ECU40is an electronic control device mainly including a microcontroller. The driving control ECU40has at least functions of a brake control ECU, a drive control ECU, and a steering control ECU. The driving control ECU40continuously controls a braking force of each wheel, an output of an in-vehicle power source, and a steering angle based on any one of control commands based on a driving operation of the driver, a control command of the driving assistance ECU50a, and a control command of the automated driving ECU50b.

The driving assistance ECU50aand the automated driving ECU50bconstitute an automated driving system50of the subject vehicle Am. The subject vehicle Am serves as an automated driving vehicle including an automated driving function by mounting the automated driving system50, and can travel by the automated driving function.

The driving assistance ECU50aimplements a driving assistance function for assisting a driving operation of a driver in the automated driving system50. The driving assistance ECU50aenables advanced driving assistance of about level 2 or partial automated driving at automated driving levels defined by the Society of Automotive Engineers. automated driving performed by the driving assistance ECU50ais automated driving with a surroundings monitoring obligation that obliges monitoring of the surroundings of the subject vehicle by visual observation of the driver.

The driving assistance ECU50ais a computer mainly including a control circuit including a processing unit, a RAM, a storage unit, an input and output interface, a bus connecting the processing unit, the RAM, the storage unit, and the input and output interface, and the like. The driving assistance ECU50aimplements driving assistance functions such as ACC (Adaptive Cruise Control), LTC (Lane Trace Control) and LCA (Lane Change Assist) by executing programs in the processor. In one example, the driving assistance ECU50aimplements driving assistance control for causing the subject vehicle Am to travel along the subject vehicle lane Lns in which the vehicle Am is traveling, in cooperation with each of the functions ACC and LTC.

The automated driving ECU50bis an in-vehicle ECU that implements an automated driving function capable of substituting a driving operation of the driver. The automated driving ECU50bcan perform automated driving of level 3 or higher in which the system is a control subject. The automated driving performed by the automated driving ECU50bis automated driving that does not need to monitor the surroundings of the subject vehicle, that is, eyes-off automated driving with no surroundings monitoring obligation.

The automated driving ECU50bis a computer mainly including a control circuit including a processor51, a RAM52, a storage53, an I/O interface54, a bus connecting them, and the like. The automated driving ECU50bhas the higher calculation capability than the driving assistance ECU50a, and can execute at least travel control corresponding to ACC, LTA, and LCA. The automated driving ECU50bimplements, in a preset automated driving area (hereinafter referred to as AD area, seeFIG.5), and when a traffic congestion around your vehicle is detected, automated driving control (hereinafter referred to as traffic-congestion level 3) that is implemented only when traveling in a traffic congestion.

In the automated driving system50, a traveling control state of the automated driving function is switched between multiple controls including at least driving assistance control with surroundings monitoring obligation by the driving assistance ECU50aand driving assistance control with no surroundings monitoring obligation by the automated driving ECU50b. In the following description, the automated driving control of level 2 or lower by the driving assistance ECU50amay be described as “driving assistance control”, and the automated driving control of level 3 or higher by the automated driving ECU50bmay be described as “automated driving control”.

In an automated driving period in which the subject vehicle Am travels under the automated driving control of the automated driving ECU50b, a specific action (hereinafter, referred to as a second task) other than predetermined driving can be permitted by the driver. The second task is legally permitted for the driver until a request for a driving operation occurs, that is, a request for driving takeover occurs. This request is performed by the automated driving ECU50band the HCU100in cooperation. Assumed examples of the second task include viewing entertainment content such as moving image content, operation of a device such as a smartphone, and action such as eating.

The seat actuator28is a driving device that adjusts a seat position of a driver's seat DS. The seat actuator28has a function to move a position of a seating surface of the driver's seat DS in a front-rear direction and a left-right direction and a function to change an angle of a backrest of the driver's seat DS relative to the seating surface (hereinafter referred to as reclining angle). Additionally, the seat actuator28includes position sensors that detect the position and the reclining angle of the seating surface.

The seat actuator28adjusts the position and the reclining angle of the seating surface based on an operation by the driver and a control signal input from the HCU100. In addition, the seat actuator28provides, as seat information indicating the state of the driver's seat DS, the position information of the seating surface and the reclining angle information detected by each position sensor to the HCU100 and the like.

The driver monitor29is mounted on the subject vehicle Am and monitors the state of the driver of the subject vehicle Am. The driver monitor29includes a near-infrared light source, a near-infrared camera, and a control unit that controls the near-infrared light source and the near-infrared camera. The driver monitor29is installed, for example, on a top surface of a steering column or on a top surface of an instrument panel, such that the near-infrared camera faces an upper side of the backrest of the driver's seat DS, specifically, faces the headrest. The near-infrared camera is integrated with a meter display21or a center information display (hereinafter, referred to as CID)22, and the like, and is provided to a screen of any of the devices.

The driver monitor29uses the near-infrared camera to capture the driver's head radiated with near-infrared light by the near-infrared light source. A detection range (imaging range) of the near-infrared camera is predefined in a range approximately centered on the headrest portion, such that at least the driver's head seated in the driver's seat DS is included. An image captured by the near-infrared camera (hereinafter referred to as a driver captured image) is subjected to image processing or image analysis by a control unit. The control unit may be capable of extracting information such as a position of the driver's eye point and a line of the sight direction from the captured image of the driver. The driver monitor29provides, as driver state information, at least one of the captured image of the driver and the information extracted from the image to the HCU100, the automated driving ECU50b, and the like.

Next, the details of each of a plurality of display devices, the operation device26, and the HCU100included in the HMI system10will be described in order.

The display device presents information by implementing image display or the like through the driver's vision. The display device includes the meter display21, the CID22, and a head-up display (hereinafter, referred to as HUD)23. The CID22has a function of a touch panel and detects a touch operation on a display screen by the driver or the like. The audio device24includes a plurality of speakers installed in a vehicle interior in a placement surrounding the driver's seat DS, and causes the speakers to reproduce a notification sound, a voice message, or the like in the vehicle interior.

The operation device26is an input unit that receives an operation of user, such as a driver or the like. For example, a user operation related to activation and stop of the automated driving function is input to the operation device26. As an example, a driver input (hereinafter referred to as level 3 transition operation) to instruct the transition from the driving assistance control to the automated driving control is input to the operating device26. The operation device26includes a steering switch provided on a spoke portion of the steering wheel, an operation lever provided on a steering column portion, a voice input device for recognizing an utterance content of the driver, and the like.

The HCU100comprehensively controls information presentation using a plurality of display devices and the audio device24. In addition, HCU100 further has a function of a seat control device that cooperates with the seat actuator28to control the state of the driver's seat DS where the driver is seated and a function of a state estimation device that cooperates with the driver monitor29to estimate the driver's state.

The HCU100mainly includes a control circuit including a processing unit11, a RAM12, a storage unit13, an I/O interface14, a bus that connects them, and the like. The processing unit11is a hardware combined with the RAM12, and executes arithmetic processing. The processing unit11includes at least one arithmetic core, such as a central processing unit (CPU) or a graphics processing unit (GPU). The processing unit11may further include a field-programmable gate array (FPGA), a neural network processing unit (NPU), an IP core having another dedicated function, and the like. The RAM12may include a video RAM for generating image data. The processing unit11accesses the RAM12and executes various processes for performing the presentation control method, the sheet control method, the state estimation method, and the like of the present disclosure. The storage unit13includes a non-volatile computer-readable medium. The storage unit13stores various programs (such as a seat control program and a state estimation program) executed by the processing unit11.

The HCU100includes a plurality of functional units by causing the processing unit11to execute each program stored in the storage unit13. The HCU100includes functional units such as an information acquisition unit81, an information cooperation unit82, a driver information grasping unit85, a driver state estimation unit86, a presentation control unit88, and a reclining control unit89(seeFIG.2).

The information acquisition unit81acquires vehicle information (such as vehicle speed information, and the like) indicating the state of the subject vehicle Am from the communication bus99. In addition, the information acquisition unit81acquires operation information indicating the contents of the user operation from the CID22, the operation device26, and the like.

The information cooperation unit82cooperates with the automated driving ECU50band enables sharing of information between the automated driving system50and the HCU100. The information cooperation unit82provides the automated driving ECU50bwith operation information grasped by the information acquisition unit81, driver estimation information (described later) estimated by the driver state estimation unit86, and the like.

The information cooperation unit82recognizes the operation state of automated driving by the automated driving system50by acquiring control state information indicating the state of the automated driving function. Based on the control state information, the information cooperation unit82grasps whether the driving control in operation is the driving assistance control or the automated driving control, in other words, whether the driving control performed by the automated driving function requires the driver to have the obligation to monitor the surroundings. In addition, the information cooperation unit82grasps a transition schedule from the driving assistance control to the automated driving control, when the subject vehicle Am travels under the driving assistance control. The information cooperation unit82grasps a transition schedule from the automated driving control to the driving assistance control, when the subject vehicle Am travels under the automated driving control.

The driver information grasping unit85acquires, from the driver monitor29, driver state information including at least one of the driver captured image and the extracted information. The driver information grasping unit85grasps the operating state of the driver monitor29based on the driver state information. As an example, the driver information grasping unit85grasps, when the driver's upper body is out of the detection range of the driver monitor29, whether the driver monitor29is capable to detect the driver or not, in other words, grasps interruption of the driver detection by the driver monitor29. When the driver information grasping unit85grasps interruption of the driver detection by the driver monitor29, the driver information grasping unit85notifies the presentation control unit88of interruption of driver monitoring.

The driver information grasping unit85grasps, based on the driver state information, head information and face information which are based on the driver captured image. The head information is information related to the shape of the head in the captured image of the driver, and is information mainly extracted from a contour shape of the head. The face information is information related to parts of the face (such as, eyes, nose, mouth, and the like) in the captured image of the driver, and is information mainly extracted from the positional relationship among the parts.

The head information and face information may be generated by either the driver information grasping unit85or the driver monitor29. For example, when the captured image of the driver is provided from the driver monitor29to the driver information grasping unit85, the head information and the face information may all be prepared by the driver information grasping unit85. Further, at least a part of the head information and face information may be generated by a control unit of the driver monitor29and may be provided, as the driver state information, to the driver information grasping unit85.

The driver state estimation unit86grasps the state and the behavior of the driver based on the information grasped by the driver information grasping unit85. The driver state estimation unit86grasps whether the driver is monitoring the surroundings of the subject vehicle Am, whether the driving posture is distorted, and the like. The driver state estimation unit86grasps the content of the second task performed by the driver during the automated driving period. Note that the driver state estimation unit86may be configured to further determine whether the steering wheel is being held, whether the physical condition of the driver is abnormal.

The driver state estimation unit86cooperates with the information cooperation unit82and the driver information grasping unit85to perform a weighting change process (FIG.3) and a state estimation process (FIG.4). The weighting change process and the state estimation process are started by the HCU100that has completed its initial processing after the power of the subject vehicle Am is turned on, and are repeated until the power of the subject vehicle Am is turned off.

In the weighting change process, the driver state estimation unit86changes weighting of the head information and the face information in the driver state estimation between the case, in which the subject vehicle Am travels under the driving assistance control, and the case, in which the subject vehicle Am travels under the automated driving control. In the weighting change process, it is determined whether the level 3 automated driving control is being performed based on the control state information (S21).

When the level 3 automated driving control is being performed by the automated driving system50(S21: YES), the driver state estimation unit86selects settings for level 3 (S22). In the settings for level 3, the weighting of the head information is increased, and majority of the head information in the driver state estimation is higher than in the settings for level 2. In other words, influence of the head information on the driver state estimation is greater when the automated driving control is performed than when the driving assistance control is performed.

On the other hand, when the level 2 driving assistance control is being performed by the automated driving system50(S21: NO), the driver state estimation unit86selects setting for level 2 (S23). In the settings for level 2, the weighting of the head information is decreased, and majority of the face information in the driver state estimation is higher than in the settings for level 3.

In the state estimation process, the driver state estimation unit86estimates the state of the driver in the period in which the subject vehicle Am travels under the automated driving control or the driving assistance control. The driver state estimation unit86continues to grasp the driver's state by repeating the state estimation process. In the state estimation process, the driver state estimation unit86reads the settings of the weighting of the head information and the face information based on the weighting change process (S31). The step of reading the settings of the weighting need not be executed every time, and may be executed only once at a predetermined time.

The driver state estimation unit86cooperates with the driver information grasping unit85and grasps the face information and the head information based on the latest captured image of the driver (S32). The driver state estimation unit86combines the face information and the head information with a weighing that reflects the settings of the weighting as read, and estimates the driver's state (S33). Specifically, the driver state estimation unit86estimates, as the driver's state, the line of sight of the driver, whether the driver operates a mobile device, whether the driver is sleeping, whether the driver is drinking alcohol, and the like. The estimation result of the driver's state by the driver state estimation unit86is provided, as the driver estimation information, to the presentation control unit88, the automated driving ECU50b, and the like.

The presentation control unit88comprehensively controls provision of information to the driver using respective display devices and the audio device24. The presentation control unit88provides contents and presents information in accordance with the operating state of the automated driving (seeFIG.5), based on the control state information acquired by the information cooperation unit82, the driver's state information acquired by the driver information grasping unit85, and the like.

Specifically, when approach to a traffic congestion is detected, while the subject vehicle Am is driving in the AD area under the level 2 driving assistance control, the presentation control unit88provides a level 3 approach notification by using the HUD23, the meter display21, and the like. The level 3 approach notification is a notification that indicates to the driver that the level 3 automated driving (traffic-congestion level 3) is likely to be available, and displays a message such as “second task available in traffic congestion ahead”.

Further, when it is grasped that the subject vehicle Am enters a traffic congestion period, the presentation control unit88issues a level 3 availability notification. The level 3 availability notification indicates to the driver that the level 3 automated driving control becomes available. In the level 3 availability notification, for example, a message such as “second task is now available” is displayed.

After the level 3 availability notification is issued, when an input of the level 3 transition operation by the driver is grasped by the information acquisition unit81, the presentation control unit88issues a level 3 start notification. The level 3 start notification indicates start of the level 3 automated driving control to the driver. In the Level 3 start notification, a message such as “Please pay attention to information from the vehicle. Automated driving will be canceled depending on surrounding road conditions” is displayed. Specifically, when the information cooperation unit82recognizes that the automated driving ECU50bexecutes the automated driving control, the presentation control unit88enables reproduction of video contents by the CID22and the like.

When the driver monitor29cannot detect the driver after starting the automated driving control, the presentation control unit88issues a posture improvement request in response to an interruption notification of the driver monitoring which is acquired from the driver information grasping unit85. For example, when most of the driver's head falls outside the detection range of the driver monitor29due to distortion of the driving posture (see frame-out inFIG.5), the posture improvement request is issued. The posture improvement request is the interruption notification indicating to the driver that the detection by the driver monitor29is interrupted. The posture improvement request is a notification that requests the driver to improve the driving posture, such that the head returns to the detection range of the driver monitor29. In the posture improvement request, for example, a message such as “Please take correct driving posture” is displayed on the CID22and is also played back by the audio device24inside the vehicle. When the driver improves the driving posture in response to the posture improvement request, and when the driver's head returns to the detection range of the driver monitor29(see frame-in inFIG.5), the presentation control unit88ends the posture improvement request.

When the automated driving ECU50bpredicts that the automated driving control ends due to, for example, end of a traffic congestion section, the presentation control unit88requests a driving takeover request. The driving takeover request is a notification indicating to the driver that the level 3 automated driving control is scheduled to end and that the control of the driving operation needs to be transferred. In the driving takeover request, for example, a message such as “automated driving is to be canceled. Please hold the steering wheel” is displayed on at least one of the HUD23and the meter display21. Further, in response to the driving takeover request, the video content played by the CID22is stopped.

When the level 3 automated driving control (traffic-congestion level 3) is terminated, the presentation control unit88issues a level 3 termination notification. The level 3 termination notification is a notification that indicates to the driver the transition from the automated driving control to the driving assistance control. In the level 3 termination notification, a message indicating the end of the automated driving, such as “automated driving canceled”, is displayed on at least one of the HUD23and the meter display21.

The reclining control unit89grasps the position of the seating surface of the driver's seat DS and the reclining angle of the backrest based on the seat information acquired from the reclining control unit89. In addition, the reclining control unit89is configured to adjust the position and the reclining angle of the seating surface of the driver's seat DS by outputting a driving signal (control command) to the seat actuator28. The reclining control unit89controls at least the reclining state of the driver's seat DS in the automated driving period in which the subject vehicle Am travels under the automated driving control, such that the driver is included in the detection range of the driver monitor29. The reclining control unit89adjusts the reclining angle and the position of the seating surface, such that the driver is located at a predetermined position where the driver monitor29is enabled to implement the driver's state monitoring appropriately. The predetermined position is a position of the driver such that the head and the face of the driver are located close to the center of the detection range.

The reclining control unit89implements reclining control of the driver's seat DS to avoid interruption of the driver monitoring in a period from issuance of the level 3 start notification, which indicates the start of the automated driving control, until the start of the driving takeover request (see the dot range inFIG.5). The reclining control unit89starts the reclining control when the driver monitor29cannot not detect the driver or when the driver state estimation unit86cannot estimate the driver state. As an example, the reclining control is implemented, when the frame out of the driver's head occurs (seeFIG.5), thereafter, the driving posture of the driver does not improve by the posture improvement request, and consequently, the driver monitor29continues to be unable to detect the driver.

In a case where the transition schedule from the automated driving control to the driving assistance control is grasped, the reclining control unit89performs the reclining control at a timing at which the driving takeover request (seeFIG.5) is started. As an example, the reclining control unit89performs control to return the reclining state to the state before starting the automated driving control. Specifically, the reclining control unit89stores respective positions of the seating surface and the backrest of the driver's seat DS during the driving assistance period and implements control to return the positions and the reclining angle of the seating surface to the stored positions.

Next, details of the seat control process for implementing the reclining control will be described below based onFIG.6and with reference toFIGS.2and5. The seat control process is started by the HCU100, for example, in response to start of the travelling under the driving assistance control.

In S101, the information cooperation unit82grasps whether the subject vehicle Am travels under the automated driving control without obligation to monitor the surroundings and determines whether the automated driving control is started. When it is determined in S101that traveling under the automated driving control is to be started, the process proceeds to S102.

In S102, the reclining control unit89stores the reclining position or the like before starting the automated driving control based on the seat information acquired from the seat actuator28. The process proceeds to S103. Note that, the process of storing the reclining position or the like may be performed in a period from issuance of the level 3 available notification until issuance of the level 3 start notification.

In S103, the information cooperation unit82determines whether the schedule to end the level 3 automated driving control exists. When it is determined in S103that the schedule to end the automated driving control exists, in other words, when it is determined that a schedule of transition to the level 2 driving assistance control or lower or a manual driving exists, the process proceeds to S104.

In S104, the driver state estimation unit86determines whether the driving posture is appropriate or not. The condition for determining the driving posture in S104may be made stricter than the condition for determining the driving posture in S106, which will be described later. When it is determined in S104that the driver's head is positioned appropriately within the detection range of the driver monitor29, the seat control process ends. On the other hand, when most of the driver's head is out of the detection range of the driver monitor29, or when the face of the driver is directed downward, it is determined that the driving posture is inappropriate, and the process proceeds to S105.

In S105, the reclining control unit89implements the reclining control. In S105, the information stored in S102is read out, and the respective positions in the driver's seat DS are adjusted to be in the state before the level 3 automated driving control is started. Upon completion of the reclining control, the seat control process ends.

On the other hand, when it is determined in S103that the schedule to transit to the driving assistance control or the manual driving does not exist and that the level 3 automated driving control is to be continued, the process advances to S106. Also in S106, the determination whether the driving posture is appropriate or not is made. In S106, the driver information grasping unit85grasps the state in which the driver monitoring by the driver monitor29continues, and determines whether the frame out of the driver's head from the detection range of the driver monitor29occurs. When it is determined in S106that the driver's head is located within the detection range, the process proceeds to S110. On the other hand, when it is determined in S106that most of the driver's head is outside the detection range, the process proceeds to S107. In S107, the presentation control unit88starts the posture improvement request to indicate that the driver monitor29is unable to recognize the driver.

In S108, it is determined whether the driver has improved the driving posture in response to the posture improvement request. When it is determined in S108that the driver's driving posture has been improved and that the driver's head is located approximately at the center of the detection range, the process proceeds to S110. On the other hand, when it is determined in S108that the driving posture has not been improved and that interruption of the driver monitoring continues, the process proceeds to S109.

In S109, the reclining control unit89controls the reclining state of the driver's seat DS such that the detection range of the driver monitor29includes the driver's head. The reclining control in S109may control to return the position and the reclining angle of the seating surface to the respective positions stored in S102, similarly to the reclining control in S105. Alternatively, the reclining control unit89may implement reclining control to position the driver's head at the center of the detection range with reference to the position information of the driver's head grasped by the driver state estimation unit86.

Herein, when interruption of the driver monitoring continues in S109although the reclining control is implemented, the automated driving ECU50bdetermines that it is difficult to continue the level 3 automated driving control and starts transition to MRM (Minimal Risk Maneuver) control. The MRM control is automated evacuation control that sets, as an evacuation place, a place in the lane on which the vehicle travels or the roadside (shoulder) in the road on which the vehicle travels and stops the subject vehicle Am at the evacuation place.

In S110, the information cooperation unit82determines whether the level 3 automated driving control is terminated. When it is determined in S110that the automated driving control is to be continued, the process returns to S103. On the other hand, when it is determined in S110that the automated driving control is to be terminated, the current seat control process is terminated.

In the present embodiment described so far, the reclining state of the driver's seat DS is controlled such that the driver is included in the detection range of the driver monitor29. Therefore, even when the driver who has been released from the obligation to monitor the surroundings loses the driving posture, the driver is enabled to remain within the detection range of the driver monitor29by the reclining control. As a result, the configuration enables to suppress interruption of the driver state monitoring in the automated driving period in which obligation to monitor the surroundings does not exist.

The above configuration enables to avoid a situation where the automated driving control is forcibly terminated and where transition to the MRM control is started, due to the inappropriate driving posture. In other words, the configuration facilitates to continue the level 3 automated driving. Therefore, the reclining control of the present embodiment is capable of contributing to improving the convenience of the automated driving.

In addition, the present embodiment controls the reclining state of the driver's seat DS when the start of the automated driving control is detected, such that the driver is located at the predetermined position where the driver monitor29is capable of implementing the driver's state monitoring appropriately. The above configuration further enables to suppress interruption of the driver monitoring after the start of the level 3 automated driving.

Furthermore, the present embodiment implements the reclining control to return the reclining state to the state before the start of the automated driving control, when the schedule to transition from the automated driving control to the driving assistance control is grasped. The configuration enables, when takeover of the driving is implemented, the driver to smoothly start monitoring of the surroundings of the subject vehicle.

Furthermore, the present embodiment starts the reclining control by the reclining control unit89, when the state in which the driver monitor29is unable to detect the driver continues even after the posture improvement request is issued. The configuration enables to avoid causing of anxiety, discomfort, and the like to the driver due to sudden implementation of the reclining control.

In addition, the present embodiment estimates the driver's state by combining the head information and the face information, when the subject vehicle Am travels under the automated driving control. Therefore, even when the driver, who has been released from the obligation of monitoring the surroundings, looks down to operate a mobile device and the like, the face information, which is hardly grasped, can be supplemented with the head information. As a result, the configuration enables to suppress interruption of the driver state monitoring in the automated driving period in which obligation to monitor the surroundings does not exist.

Further, the driver state estimation unit86of the present embodiment changes the weighting of the head information and the face information in the driver state estimation between the case where the subject vehicle Am travels under the driving assistance control that obliges the surrounding monitoring and the case where the subject vehicle Am travels under the automated driving control. Therefore, the state of the driver who is monitoring the surroundings and the state of the driver who is not monitoring the surroundings can be estimated by appropriately processing for the respective states. Therefore, the accuracy of estimating the driver state can be improved.

Furthermore, in the present embodiment, when the subject vehicle Am travels under the automated driving control, the weighing of the head information in the driver state estimation is increased to be higher than the weighing of the head information when the subject vehicle Am travels under the driving assistance control. Therefore, the accuracy of the driver state estimation can be ensured even in the automated driving period in which the face information is hardly extracted.

In addition, in this embodiment, the reclining state of the driver's seat DS is controlled such that the head is located in the detection range of the driver monitor29when the driver state estimation is interrupted. As described above, the configuration enables to further suppress interruption of the driver state monitoring by combining the complementation of the face information with the head information and the reclining control.

In the above embodiment, the information cooperation unit82corresponds to a “control grasping unit”, the driver information grasping unit85corresponds to a “monitoring state grasping unit”, the presentation control unit88corresponds to a “notification control unit”, and the HCU100corresponds to a “seat control device” and a “state estimation device”. Further, the posture improvement request corresponds to an “interruption notification”.

Other Embodiments

Although one embodiment of the present disclosure has been described above, the present disclosure is not construed as being limited to the above-mentioned embodiments, and can be applied to various embodiments and combinations within a scope that does not depart from the spirit of the present disclosure.

In the reclining control of the embodiment described above, both the position of the seating surface and the reclining angle are the adjusted object. On the other hand, in a first modification of the above embodiment, only the reclining angle is the adjusted object in the reclining control. Further, as a driving position control in conjunction with the reclining control, the position of the steering wheel, and the like may be adjusted together with the reclining angle.

In the reclining control of the embodiment described above, the adjustment to return the state of the driver's seat DS to the position stored at the time of the transition to the automated driving control is implemented. On the other hand, in a second modification of the above embodiment, the reclining control is implemented to limit respective adjustment ranges of the position of the seating surface and the reclining angle so that the driver's head does not fall out of the detection range.

In third and fourth modifications of the above embodiment, the process of storing the position at the time of the transition to level 3 is omitted. In the third modification, a reference state is set in advance such that the driver is to be located at a predetermined position where the state can be monitored appropriately. The reclining control unit89implements the reclining control to adjust the reclining angle and the like to the reference state. In the fourth modification, the reclining control is performed to adjust the reclining angle and the like with reference to the information extracted from the driver image, such that the driver's face is located at the center of the detection range.

In the fifth modification of the above embodiment, issuance of the posture improvement request is omitted. In the firth modification, the reclining control is immediately started in response to interruption of the driver monitoring by the driver monitor29. In the sixth modification of the above embodiment, the reclining control is performed simultaneously with the posture improvement request.

In the above embodiment, the driver's head information is also used to estimate the driver's state in the driving assistance period. On the other hand, in the seventh modification of the above embodiment, the driver's state estimation in the driving assistance period is performed using only the driver's face information. Furthermore, the driver state estimation in the automated driving period is implemented using only the driver's head information.

The HCU100of the above embodiment is capable of estimating the driver's state even in the driving assistance control at the automated driving of level 1 or in the manual driving. The driver state estimation unit86may select the setting of the weighting for the level 2 under the level 1 driving assistance control or under the manual driving, or may select the setting of the weighting different from that for level 2.

In the above embodiment, the criteria for determining whether the driving posture is appropriate or not at the end of level 3 (see S104inFIG.6) as scheduled is stricter than the criteria for determining whether the driving posture is appropriate or not in the period in which the level 3 continues (see S106inFIG.6). However, the criteria used at the end of level 3 as scheduled and the criteria used in the period in which the level 3 continues may be substantially the same. Furthermore, the criteria used in the period in which the level 3 continues may be stricter than the criteria used at the end of the level 3 as scheduled.

In the above embodiment, the respective functions of the seat control device and the state estimation device installed in the HCU100may be implemented by an in-vehicle ECU that is other than the HCU100. For example, the function of the seat control device may be implemented by the driving assistance ECU50a, the automated driving ECU50b, the control unit of the seat actuator28, and/or the like. Further, the function of the state estimation device may be implemented by the driving assistance ECU50a, the automated driving ECU50b, the control unit of the driver monitor29, and/or the like. Furthermore, the function of the state estimation device may be implemented by a dead man determining device provided separately from the driver monitor29. Moreover, the respective functions of the seat control device and the state estimation device may be implemented by a plurality of ECUs. In these embodiments, a system implementing a plurality of ECUs corresponds to a “seat control device” and a “state estimation device.”

The automated driving ECU50bmay be configured to perform automated driving control in a driving control mode different from the traffic-congestion level 3, such as area-limited control (hereinafter referred to as area level 3) that is performed only within a specific area. The automated driving ECU50bmay perform automated driving control at Level 4 or higher.

In an eighth modification of the above embodiment, the respective functions of the automated driving ECU50band the HCU100are implemented by one in-vehicle ECU. In the eighth modification, the in-vehicle ECU that serves as both the automated driving ECU50band the HCU100corresponds to the “seat control device” and the “state estimation device.”

In the aforementioned embodiments, the respective functions provided by the HCU can be also provided by software and hardware for executing the software, only software, only hardware, and complex combinations of software and hardware. Furthermore, in a case where such function is provided by an electronic circuit as hardware, each function can also be provided by a digital circuit including a large number of logic circuits or an analog circuit.

Each processing unit according to the above-described embodiments may be individually mounted on a printed circuit board, or may be mounted on an application specific integrated circuit (ASIC), an FPGA, or the like. The configuration of the storage medium (non-transitory tangible computer-readable storage medium or non-transitory tangible storage medium) that stores various program may be changed as appropriate. Further, the storage medium is not limited to the configuration provided on the circuit board, and may be provided in the form of a memory card or the like. The storage medium may be inserted into a slot portion, and electrically connected to the control circuit of the automated driving ECU or the control circuit of the HCU. Further, the storage medium may include an optical disk which forms a source of programs to be copied into the automated driving ECU or the HCU, or a hard disk drive therefor.

The vehicle on which the automated driving system and the HMI system are mounted is not limited to a general passenger vehicle for private use, and may be a rental vehicle, a manned taxi vehicle, a ride-sharing vehicle, a cargo vehicle, a bus, or the like. The vehicle on which the automated driving system and the HMI system are mounted may be a right-hand drive vehicle or a left-hand drive vehicle. Additionally, a traffic environment in which the vehicle travels may be a traffic environment on a premise of left-hand traffic or a traffic environment on a premise of right-hand traffic. The seat control and the driver state estimation according to the present disclosure may be optimized according to the road traffic laws of respective countries and regions, and further according to the position of the vehicle's steering wheel, and the like.

The control unit and the method thereof described in the present disclosure may be realized by a dedicated computer constituting a processor programmed to execute one or a plurality of functions embodied by a computer program. Alternatively, the device and the method thereof described in the present disclosure may be implemented by a dedicated hardware logic circuit. Alternatively, the device and the method thereof described in the present disclosure may be implemented by one or more dedicated computers configured by a combination of a processor executing a computer program and one or more hardware logic circuits. The computer programs may be stored, as instructions to be executed by a computer, in a tangible non-transitory computer-readable medium.