System and method for controlling vehicle based on condition of driver

A system and method for controlling a vehicle based on a driver status are disclosed. The vehicle control system includes various sensing devices (including a camera, a vehicle dynamics sensor, a vehicle around view monitoring (AVM) camera, a periphery surveillance sensor, and a navigation device) and an electronic control unit (ECU). The ECU may analyze a driver status through the driver's face and pupils recognized by output signals of the sensing devices. If the driver has no intention to drive the vehicle, the ECU may control the vehicle to stop on a road shoulder, resulting in guarantee of safer driving.

CROSS-REFERENCE TO RELATED APPLICATION

BACKGROUND

Embodiments of the present disclosure relate to a system and method for controlling a vehicle based on a driver status, and more particularly to a system and method for controlling a vehicle based on a driver status, which analyze a driver status by monitoring the driver's face and pupils during vehicle driving, and control the vehicle to stop on a road shoulder when it seems that the driver has no intention to drive the vehicle, resulting in safer and more stable vehicle driving.

2. Description of the Related Art

Generally, an Advanced Driver Assistance System (ADAS) may autonomously recognize some of many situations capable of occurring during vehicle driving, and may determine what situations have occurred, such that the ADAS can autonomously control mechanical devices based on the determined situations. The ADAS has been designed to assist or help a vehicle driver in a complicated vehicle control process, and has been intensively researched and developed to complete autonomous driving technology.

A representative example of the ADAS embedded in the vehicle may include an Autonomous Emergency Brake (AEB) system, a Lane Keeping Assist System (LKAS), an Advanced Smart Cruise Control (ASCC) system, an Active Blind Spot Detection (ABSD) system, an Around View Monitoring (AVM) system, etc. The AEB system may control the vehicle to autonomously decelerate or stop although the driver does not depress a brake pedal in a dangerous situation having a higher possibility of collision with peripheral objects. The LKAS determines whether a vehicle deviates from a current lane or is scheduled to deviate from the current lane. If the vehicle has deviated from the current lane or is scheduled to deviate from the current lane, the LKAS performs lane keeping control in a manner that the vehicle can maintain the current lane, or performs lane departure warning to inform the driver of a lane departure state or a lane departure expectation state. The ASCC system may control the vehicle to travel on a road at a predefined speed, while simultaneously maintaining a constant distance to a vehicle ahead. The ABSD system may determine whether there is a high possibility of collision in a blind spot, such that the ABSD system may guide a host vehicle to safely perform lane change when high possibility of collision is decided. The AVM system may visually display peripheral situations of a host vehicle, such that the driver of the host vehicle can recognize the peripheral situations.

CITED REFERENCES

Patent Documents

SUMMARY

Therefore, it is an aspect of the present disclosure to provide a system and method for controlling a vehicle based on a driver status, analyze a driver status by monitoring the driver's face and pupils during vehicle driving, and control the vehicle to stop on a road shoulder when it seems that the driver has no intention to drive the vehicle, resulting in safe and more stable vehicle driving.

In accordance with an aspect of the present disclosure, a system and method for controlling a vehicle based on a driver status are disclosed. The vehicle control system includes various sensing devices (including a camera, a vehicle dynamics sensor, a vehicle around view monitoring (AVM) camera, a periphery surveillance sensor, and a navigation device) and an electronic control unit (ECU). The ECU may analyze a driver status through the driver's face and pupils recognized by output signals of the sensing devices. If the driver has no intention to drive the vehicle, the ECU may control the vehicle to stop on a road shoulder.

If the analyzed driver status is considered normal, the ECU may analyze the driver's pupils, such that the ECU may also control the vehicle based on movement of the driver's pupils.

DETAILED DESCRIPTION

Advantages and features of the present disclosure and a method of achieving the advantages and features of the present disclosure will be clearly understood from embodiments described hereinafter in conjunction with the accompanying drawings. However, the present disclosure is not limited to the following embodiments and may be realized in various different forms. These embodiments are provided only to completely disclose the present disclosure and for a person having ordinary skill in the art to which the present disclosure pertains to completely understand the category of the disclosure. That is, the present disclosure is defined only by the claims. The same reference numbers will be used throughout this specification to refer to the same parts.

A system and method for controlling a vehicle based on a driver status according to an embodiment of the present disclosure will hereinafter be described with reference to the attached drawings.

FIG.1is a block diagram illustrating a system for controlling a vehicle according to an embodiment of the present disclosure.

Referring toFIG.1, the system for controlling the vehicle according to the present disclosure may include a sensing device10, an electronic control unit (ECU)20, a driver carelessness warning device30, an acceleration/deceleration control device40, and a steering control device50. The sensing device10may include a driver surveillance camera11, a vehicle dynamics sensor12, a vehicle around view monitoring (AVM) camera13, a periphery surveillance sensor14, and a navigation device15.

The driver surveillance camera11may be any one of various well-known image sensors such as a Far Infrared Ray (FIR) camera, and may be provided in an indoor space of the vehicle. The driver surveillance camera11may acquire images of the driver's face and pupils, and may thus transmit driver monitoring information (for example, a change in the driver's facial expression, eye blinking of the driver, movement of the driver's pupils, the driver's gaze position, etc.) to the ECU20.

The vehicle dynamics sensor12may be implemented as any of various well-known sensors, for example, a wheel speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, etc. The vehicle dynamics sensor12may be arranged at proper positions of a host vehicle, for example, a wheel, a steering wheel, or the like, such that the vehicle dynamics sensor12may sense a driving speed, acceleration, yaw angular speed, information as to whether the driver grasps a steering angle, etc. of the host vehicle, and may transmit the recognized information to the ECU20.

The vehicle AVM camera13may be implemented as any of various well-known image sensors, for example, an FIR camera, a CMOS camera (or a CCD camera), etc. The vehicle AVM camera13may be arranged at an upper end of a windshield of a host vehicle, may sense and project various kinds of lights, for example, infrared light, visible light, etc., within the range of a predefined angle and a predefined distance with respect to a forward region of the host vehicle, may acquire an image of an external object located ahead of the host vehicle, an image of a vehicle lane, an image of a traffic sign, etc., and may transmit the recognized vehicle omnidirectional information to the ECU20.

The periphery surveillance sensor14may be implemented as any of various well-known sensors, such as a radar sensor, etc. The periphery surveillance sensor14may be implemented as a plurality of periphery surveillance sensors14. The peripheral surveillance sensors14may be respectively installed at a center part of a front surface, a center part of a rear surface, and a corner part of the host vehicle, may emit electromagnetic waves within the range of a predetermined angle with respect to a forward region of each periphery surveillance sensor, may receive electromagnetic waves reflected from peripheral objects located in the vicinity of the host vehicle, and may detect an angle, a distance, a relative speed, a relative acceleration, etc. between the host vehicle and each peripheral object, such that the peripheral surveillance sensors14may transmit the recognized vehicle peripheral information to the ECU20.

The navigation device15may confirm a current position of the host vehicle through a Global Positioning System (GPS), may display a map to a destination, may confirm a distance to the destination and a total consumption time to be consumed to arrive at the destination, and may display a route to the destination for driver convenience, such that the navigation device15may allow the driver to easily drive the host vehicle. In addition, when a roadside assistance service is needed due to occurrence of an emergency situation or a faulty operation of the vehicle running on a road or when an emergency rescue service is needed due to occurrence of a traffic accident, the navigation device15may be a machine or program capable of informing the vehicle of a current GPS position. The navigation device15may transmit information about a current road and lane on which the host vehicle is running, information about a shoulder of the road, etc. to the ECU20.

The ECU20may recognize the driver's face and pupils upon receiving signals detected by the sensing device10, may analyze a driver status through the recognized driver's face and pupils, and may determine whether the driver has an intention to continuously drive the host vehicle. If it is determined that the driver has no intention to continuously drive the host vehicle, the ECU20may control the host vehicle to stop on a road shoulder, resulting in guarantee of safer driving.

If the analyzed driver status is considered normal, the ECU20may analyze movement of the driver's pupils, such that the ECU20may also control the vehicle based on the movement of the driver's pupils.

The driver carelessness warning device30may warn the driver of a careless driving status upon receiving a control signal from the ECU20. Upon receiving a control signal from the ECU20, the acceleration/deceleration control device30may increase engine power of the host vehicle or may generate a braking pressure of the host vehicle. The steering control device50may generate a steering angle of the steering wheel upon receiving a control signal from the ECU20.

The present disclosure may analyze a driver status through the driver's face and pupils. If it seems that the driver has no intention to drive the vehicle, the present disclosure may control the vehicle to stop on a road shoulder, resulting in guarantee of safe driving.

In addition, if it seems that the driver status can be considered normal, the present disclosure may analyze movement of the driver's pupils, such that the vehicle can be controlled based on such movement of the driver's pupils. A detailed description thereof will hereinafter be given with reference to the attached drawings.

A method for controlling a vehicle based on a driver status using the above-mentioned system according to the embodiment of the present disclosure will hereinafter be described with reference toFIGS.2to4.

FIG.2is a flowchart illustrating a method for controlling the vehicle using the vehicle control system according to an embodiment of the present disclosure.

Referring toFIG.2, the system for controlling the vehicle based on a driver status according to the present disclosure may acquire images about the driver's face and pupils through the driver surveillance camera11, such that the system may recognize driver monitoring information (e.g., a change in the driver's facial expression, eye blinking of the driver, movement of the driver's pupils, the driver's gaze position, etc.) through the acquired images (S210).

The system may analyze a driver status using the recognized driver monitoring information (S220). For example, if the driver's face is wrinkled or distorted, the system may determine that there is something wrong with the driver's health. In addition, the system may recognize occurrence or non-occurrence of drowsy driving by analyzing movement of the driver's pupils, the number of eye blinking times of the driver's eyes, and a closed or open state of the driver's eyes. In more detail, if the driver's pupils frequently and repeatedly move in up, down, right and left directions, this means that the driver feels sleepy. If the driver's eyes blink slowly, this means that the driver is in a drowsy driving status. In addition, if it seems that the driver's eyes are closed for a little while, this means that the driver fell asleep. In addition, for example, if the vehicle frequently deviates from a current lane, if the driver does not grasp the steering wheel, and if a change in distance between the vehicle and a peripheral vehicle suddenly increases, this means that the driver feels sleepy or fell asleep.

If the analyzed driver status is considered abnormal (S220), the system may warn the driver of drowsy or careless driving through the driver carelessness warning device30(S232). In this case, the driver carelessness warning device30may receive a control signal from the ECU20, such that the driver carelessness warning device30may sequentially perform a first warning action based on visual sensation→a second warning action based on audio-visual sensation→a third warning action based on audio-visual sensation and tactile sensation (e.g., steering wheel vibration) according to lapse of time.

If it is determined that the driver has no intention to drive the vehicle irrespective of the warning actions shown in step S232(e.g., if the driver's eyes do not move for a predetermined time, or if the driver does not move the steering wheel or does not depress an accelerator or brake pedal for a predetermined time), the system may recognize rear-view and side-view regions of the host vehicle and information about peripheral objects of the host vehicle through the vehicle dynamics sensor12, the vehicle AVM camera13, the periphery surveillance sensor14, and the navigation device15, may analyze a forward driving route of the host vehicle, and may determine whether the host vehicle can move toward a road shoulder on the basis of the analyzed result. If it is determined that the host vehicle can move toward the road shoulder, the system may guide the host vehicle to move toward the road shoulder and may control the host vehicle to stop on the road shoulder. On the other hand, if it is determined that the host vehicle is unable to move toward the road shoulder, the system may determine whether there is a high possibility of rear-end collision. If there is a high possibility of rear-end collision, the system may decelerate the vehicle or may control the vehicle to stop driving (S234). A method for guiding the host vehicle to move toward the shoulder and controlling the host vehicle to stop on the shoulder will hereinafter be described with reference toFIGS.3and4.

If the analyzed driver status is considered normal (S220), the system may analyze the driver's pupils (S242), such that the system may control the vehicle based on movement of the driver's pupils (S244). A method for analyzing movement of the driver's pupils will hereinafter be described with reference toFIG.5.

FIG.3is a flowchart illustrating a method for directing a vehicle to move toward a road shoulder and controlling the vehicle to stop on the road shoulder according to an embodiment of the present disclosure.FIG.4is a conceptual diagram illustrating a method for directing a vehicle to move toward a road shoulder according to an embodiment of the present disclosure.

Referring toFIG.3, if it is determined that the driver has no intention to drive the vehicle irrespective of the warning actions shown, the system may detect the presence or absence of a shoulder on a current road (S310). In more detail, if the system receives information about the presence of a shoulder in a forward region of the host vehicle, information about horizontal/vertical distances from the host vehicle to a start point of the shoulder, and information about horizontal/vertical distances from the host vehicle to the end point of the shoulder from the navigation device15, or if the system recognizes the presence of a right solid lane and a road edge of a next lane using the vehicle AVM camera13, the system may determine the presence of the shoulder.

Subsequently, the system may recognize a current position of the host vehicle (S320). The system may determine whether a current position of the host vehicle is in the lane next to the shoulder (i.e., the lane nearest to the shoulder) upon receiving the horizontal distance from the host vehicle to the start point of the shoulder from the navigation device15. If the horizontal distance from the host vehicle to the start point of the shoulder is equal to or shorter than a predetermined distance, the system may determine that the host vehicle is located in the lane next to the shoulder. Alternatively, if the presence of a road edge of the next lane is decided by information acquired by the vehicle AVM camera13, the system may determine that the host vehicle is traveling in the lane nearest to the shoulder.

If it is determined that the host vehicle does not travel in the lane nearest to the shoulder (S320), the system may recognize the presence or absence of objects located in backward and peripheral regions of the host vehicle using the vehicle dynamics sensor12, the vehicle AVM camera13, and the periphery surveillance sensor14, and may determine whether the host vehicle can perform lane change based on the recognized result. If it is determined that the host vehicle can perform lane change, the system may control the host vehicle to move into the next lane (S322).

If the host vehicle travels in the lane nearest to the shoulder (S320or S322), the system may calculate a width of a shoulder lane, and may determine whether the shoulder lane has a sufficiently large width (e.g., a minimum width of 2.8 meters) in which the host vehicle can stop (S330).

If it is determined that the shoulder lane width is equal to or longer than 2.8 meters (S330), the system may detect the presence or absence of another vehicle traveling on the shoulder using the vehicle dynamics sensor12, the vehicle AVM camera13, and the periphery surveillance sensor14(S340).

Upon receiving information about the distance from the host vehicle to the end point of the shoulder from the navigation device15(S340) or upon receiving the end point of a road edge recognized by the vehicle AVM camera13(S340), the system may calculate a vehicle stop point where the host vehicle will stop driving using the received information (S350). Finally, the system may control the host vehicle to move toward the stop point calculated in step S350, and may control the host vehicle to stop at the calculated stop point (S360).

After completion of step S360, the system may turn on an emergency light, or may transmit vehicle information, driver information, and driver status information of the host vehicle to a central control center, such that the driver of the host vehicle may request an ambulance or police from the central control center. In this case, the system may confirm the driver's health information in advance. If the system determines that the driver has any health problem on the basis of health history information, the system may request an ambulance from the central control center. If the system determines that the driver does not awake after lapse of a predetermined time, the system may request the police from the central control center.

FIG.5is a flowchart illustrating a method for analyzing the driver's pupils according to an embodiment of the present disclosure.

Referring toFIG.5, if the analyzed driver status is considered normal, the system may first perform zero calibration once on the image of the driver's pupils (S510). In this case, when the driver gazes at a left lane with respect to a current lane, position information of the driver's pupils captured by the camera may be stored. When the driver looks forward in the current lane, position information of the driver's pupils captured by the camera may be stored. In addition, when the driver gazes at a right lane with respect to the current lane, position information of the driver's pupils captured by the camera may be stored.

Thereafter, meaningless movement irrelevant to vehicle control from among various movements of the driver's pupils may be filtered out (S520). Current gaze position information of the driver may be transmitted to the vehicle AVM camera13(S522), and the vehicle AVM camera13may detect the presence or absence of any object within regions gazed at by the driver (S524). If the detected object is not identical to a vehicle lane or a road, the system may determine that movement of the driver's pupils is meaningless movement irrelevant to vehicle control (S526).

Finally, the system may detect the presence or absence of meaningful movement of the driver's pupils requesting vehicle control (S530). In more detail, if movement of the driver's pupils is not identical to meaningless movement irrelevant to vehicle control (S520), if a vehicle lane is detected in the regions gazed at by the driver, and if the driver gazes at a vehicle lane region in which the host vehicle is scheduled to enter, for a predefined time, the system may determine occurrence of movement of the driver's pupils requesting vehicle control, such that the vehicle may control the host vehicle to perform lane change. Alternatively, if the system determines that the driver was surprised while driving on the basis of movement of the driver's pupils (e.g., dilation of the driver's pupils), the system may perform vehicle braking control.

As is apparent from the above description, a system and method for controlling a vehicle based on a driver status according to the embodiments of the present disclosure can analyze a driver status by monitoring the driver's face and pupils during vehicle driving, and can control the vehicle to stop on a road shoulder when it seems that the driver has no intention to drive the vehicle, resulting in safer and more stable vehicle driving.

If the analyzed driver status is considered normal, the system and method for controlling the vehicle may monitor the driver's pupils, and may control the vehicle based on movement of the driver's pupils as necessary.