MONITORING AND WARNING SYSTEM FOR VEHICLES

There is provided a monitoring and warning system for vehicles including a first image sensor, a second image sensor, a warning unit, an expression recognition unit and a sign recognition unit. The first image sensor captures a first image frame toward a driver seat direction. The second image sensor captures a second image frame toward a forward direction. The expression recognition unit recognizes a sleep expression according to the first image frame to accordingly control the warning unit to trigger a first warning. The sign recognition unit recognizes a stop sign according to the second image frame to accordingly control the warning unit to trigger a second warning.

DETAILED DESCRIPTION OF THE EMBODIMENT

Referring toFIG. 1, it shows an application schematic diagram of the monitoring and warning system for vehicles according to an embodiment of the present disclosure. The monitoring and warning system for vehicles according to the embodiment of the present disclosure may be coupled to or integrated in the central control system of a vehicle9and coupled to the dynamic system thereof. The sign recognition algorithm and sign information may be built in the monitoring and warning system for vehicles according to the embodiment of the present disclosure such that the system may give a warning to a driver, force the power output to be decreased or restrain the power output from increasing according to the stop sign of traffic signs. For example in one embodiment, the power output may be gradually decreased with a predetermined rate till the driver notices or till the break is triggered so as to prevent accidents from an emergency break. In another embodiment, the acceleration may be suppressed by suppressing the power output so as to attract the attention of the driver. The expression recognition algorithm and expression information may be built in the monitoring and warning system for vehicles according to the embodiment of the present disclosure such that the system may give a warning to a driver according to the sleep expression of the driver. Accordingly, it is able to construct an intelligent vehicle control system to increase the driving safety. In addition,FIG. 1only shows a part of elements of the vehicle9, e.g. a rear-view mirror91, a central control panel92and a driver seat93and other elements are omitted. In one embodiment, the sign recognition algorithm and the expression recognition algorithm may use the same algorithm but different feature information so as to perform the recognition of different objects. In other embodiments, it is possible to adopt different algorithms.

Referring toFIG. 2, it shows a schematic block diagram of the monitoring and warning system for vehicles1according to a first embodiment of the present disclosure. The monitoring and warning system for vehicles1according to the first embodiment of the present disclosure may include an image sensor11, a memory unit12, a sign recognition unit13and a warning unit14, and may be coupled to a dynamic system94of a vehicle9.

Referring toFIGS. 1 and 2, the monitoring and warning system for vehicles1according to the first embodiment of the present disclosure is described below.

The image sensor11may be a CCD image sensor, a CMOS image sensor or the like, and is configured to capture an image frame F1toward a forward direction Fd of the vehicle9, and thus the image sensor11is preferably disposed on a rear-view mirror91, e.g. the image sensor11shown inFIG. 1. However, the image sensor11is not limited to be disposed on the rear-view mirror91and may be disposed at other positions without any limitation as long as it is able to capture images of the traffic sign5in front of the vehicle9.

The memory unit12previously saves sign information associated with traffic signs and stop sings. As the monitoring and warning system for vehicles1according to the first embodiment of the present disclosure is configured to detect the stop sign, the sign information saved in the memory unit12may be the shape feature of the traffic sign5and position information of the stop sign at the traffic sign5. For example, the stop sign (e.g. red light) may locate at the most left or top position of the traffic sign5.

The sign recognition unit13may recognize a traffic sign5according to the image frame F1and control the warning unit14to trigger a warning according to a stop sign. The sign recognition unit13may be implemented by a digital signal processor (DSP) or other processors capable of processing digital image data. After receiving the image frame F1, the sign recognition unit13utilizes the built-in algorithm to calculate the object feature in the image frame F1and compares the object feature with the sign features previously saved in the memory unit12so as to recognize the traffic sign5. Next, the sign recognition unit13may identify the stop sign according to the position that is lighting currently (e.g. the most left or top). In addition, the sign recognition unit13may further recognize a traffic sign of interest in a plurality of traffic signs. For example, when the sign recognition unit13recognizes a plurality of traffic signs according to the comparison result of the object feature and the sign features, the closest traffic sign in the plurality of traffic signs may be served as the traffic sign of interest, wherein the term “closest” may be identified according to values of the size, contrast and/or sharpness of the traffic sign.

The warning unit14is controlled by the sign recognition unit13to trigger a warning, wherein the warning unit14may trigger the warning through a lamp signal141, an image142and/or a sound143. The lamp signal141may be generated at any position inside the vehicle9within a view range of the driver8;

the image142may be the picture shown by a liquid crystal display or other displays that are, for example, integrated with the central control panel92; and the sound143may be generated by the vehicle radio or additionally speaker. It is appreciated that the disposition and generation of the above lamp signal141, image142and sound143are only intended to describe the present disclosure, not to limit the present disclosure.

In the present disclosure, the sign recognition unit13may further be configured to detect a power output of a dynamic system94of the vehicle9. In one embodiment, when the sign recognition unit13has recognized the stop sign and detects that the power output is decreasing (e.g. decreasing at a predetermined rate within each predetermined time interval), the warning unit14is not controlled to trigger the warning. When the sign recognition unit13has recognized the stop sign and detects that the power output is not decreasing, the warning unit14is controlled to trigger the warning so as to warn the driver8to break or release the accelerator. In another embodiment, when the sign recognition unit13has recognized the stop sign and detects that the power output is not decreasing, the sign recognition unit13actively controls the dynamic system94to decrease the power output or restrains the power output from increasing thereby increasing the driving safety. In an alternative embodiment, when the sign recognition unit13has recognized the stop sign, the sign recognition unit13immediately controls the warning unit14to trigger the warning to warn the driver8to brake or release the accelerator regardless of the current driving condition.

Referring toFIG. 3, it shows a schematic block diagram of the monitoring and warning system for vehicles2according to a second embodiment of the present disclosure. The monitoring and warning system for vehicles2according to the second embodiment of the present disclosure may include an image sensor21, a memory unit22, an expression recognition unit23and a warning unit24.

Referring toFIGS. 1 and 3together, the monitoring and warning system for vehicles2according to the second embodiment of the present disclosure is described hereinafter.

The image sensor11may also be a CCD image sensor, a CMOS sensor or the like, and is configured to capture an image frame F2toward a driver seat direction of a driver seat93, and thus the image sensor21is preferably disposed on a rear-view minor91, e.g. the image sensor21shown inFIG. 2. However, the image sensor21is not limited to be disposed on the rear-view mirror91and may be disposed at other positions without any limitation as long as it is able to capture face images of the driver8.

The memory unit22previously saves expression information associated with the sleep expression. For example, an expression recognition algorithm may divide a face image into a plurality of blocks and calculate the feature of every block associated with the sleep expression to be previously recorded. In other embodiments, if the expression recognition unit23may further recognize other expressions or perform the ID identification, the memory unit22may also save the related expression information or facial information.

The expression recognition unit23is configured to recognize a sleep expression according to the image frame F2to accordingly control the warning unit22to trigger a warning. The expression recognition unit23may be implemented by a digital signal processor or other processors capable of processing digital image data. After receiving the image frame F2, the expression recognition unit23utilizes the built-in algorithm to calculate the expression feature in the image frame F2and compares the expression feature with the expression features previously saved in the memory unit12so as to recognize the sleep expression. In addition, the expression recognition unit23may further identify whether there is a driver8in the image frame F2. In one embodiment, when the expression recognition unit23identifies that a driver8is contained in the image frame F2, the expression recognition unit23starts to recognize the sleep expression of the driver8immediately or after a predetermined time interval, wherein the predetermined time interval may be, but not limited to, 30 minutes or an hour, or it may be set by the user.

The warning unit14is controlled by the expression recognition unit23to trigger a warning, wherein the warning unit14may trigger the warning through a lamp signal241, an image242, a sound243, a vibration244and/or a current stimulation245. The lamp signal241may be generated at any position inside the vehicle9within a view range of the driver8; the image242may be the picture shown by a liquid crystal display or other displays that are, for example, integrated with the central control panel92; the sound243may be generated by the vehicle radio or additionally speaker; the vibration244and the current stimulation245may be transmitted to the driver8through, for example, the driver seat93. It is appreciated that the disposition and generation of the above lamp signal241, image242, sound243, vibration244and current stimulation245are only intended to describe the present disclosure, not to limit the present disclosure.

Referring toFIG. 4, it shows a schematic block diagram of the monitoring and warning system for vehicles3according to a third embodiment of the present disclosure. The monitoring and warning system for vehicles3according to the third embodiment of the present disclosure may simultaneously recognize a traffic sign, a stop sign and a sleep expression. The monitoring and warning system for vehicles3includes a first image sensor31, a second image sensor31′, a memory unit32, an expression recognition unit33, a sign recognition unit33′ and a warning unit34, and may be coupled to a dynamic system94of a vehicle9. In this embodiment, the first image sensor31and the expression recognition unit33may respectively correspond to the image sensor21and the expression recognition unit23in the second embodiment; and the second image sensor31′ and the sign recognition unit33′ may respectively correspond to the image sensor11and the sign recognition unit13in the first embodiment. The memory unit32may save the expression information associated with sleep expressions and the sign information associated with traffic signs and stop signs. Therefore, the first image sensor31is preferably disposed on a rear-view mirror91, e.g. at the position21; and the second image sensor31′ is preferably also disposed on the rear-view mirror91, e.g. at the position11, but not limited to.

Similar to the second embodiment, in this embodiment the first image sensor31is configured to capture a first image frame F toward a driver seat direction of the driver seat93. The expression recognition unit33is configured to recognize a sleep expression according to the first image frame F to accordingly control the warning unit34to trigger a first warning. Similarly, the warning unit34may trigger the first warning through a lamp signal, an image, a sound, a vibration and/or a current stimulation. In addition, the expression recognition unit33may further identify whether there is a driver8in the first image frame F. In addition, the method of recognizing a sleep expression according to the first image frame F captured by the first image sensor31and controlling the warning unit34to trigger the first warning is similar to the aforementioned second embodiment and thus details thereof are not repeated herein.

Similar to the first embodiment, in this embodiment the second image sensor31′ is configured to capture a second image frame F′ toward a forward direction Fd of the vehicle9. The sign recognition unit33′ is configured to recognize a traffic sign5according to the second image frame F′ and controls the warning unit34to trigger a second warning according to a stop sign of the traffic sign5. Similarly, the warning unit34may trigger the second warning through a lamp signal, an image and/or a sound. In addition, the sign recognition unit33′ may identify a traffic sign of interest in a plurality of traffic signs and further recognize the stop sign (e.g. red light) generated by the traffic sign of interest.

In addition, the sign recognition unit33′ may further detect a power output of a dynamic system94of the vehicle9. In one embodiment, when the sign recognition unit33′ has recognized the stop sign and detects that the power output is decreasing, the warning unit34is not controlled to trigger the second warning. When the sign recognition unit33′ has recognized the stop sign and detects that the power output is not decreasing, the warning unit34is controlled to trigger the second warning so as to warn the driver8to break or release the accelerator. In another embodiment, when the sign recognition unit33′ has recognized the stop sign and detects that the power output is not decreasing, the sign recognition unit33′ actively controls the dynamic system94to decrease the power output thereof or restrains the power output from increasing thereby increasing the driving safety. In an alternative embodiment, when the sign recognition unit33′ has recognized the stop sign, the sign recognition unit33′ immediately controls the warning unit34to trigger the second warning to warn the driver8to brake or release the accelerator regardless of the current driving condition.

In addition, the method of recognizing a stop sign according to the second image frame F′ captured by the second image sensor31′ and controlling the warning unit34to trigger the second warning and/or to change the power output is similar to the aforementioned first embodiment and thus details thereof are not repeated herein.

In one embodiment, the expression recognition unit33and the sign recognition unit33′ may be implemented by a single digital processor. That is, the expression recognition unit33and the sign recognition unit33′ may be implemented by a single element or by two elements.

It is appreciated that the above traffic sign5is only exemplary, and the traffic sign5may be other types, e.g. lamps arranged in a vertical line. The sign information of different types of traffic signs may be previously saved in the memory units12and32.

In addition, inFIGS. 2-4the symbols “FOV” and “FOV'” are referred to the field of view of the image sensors.

In addition, the monitoring and warning system for vehicles1-3according to the embodiment of the present disclosure may cooperate with the ID identification system. For example, the expression recognition units23and33may be configured to identify a user ID so to as further increase the practicality thereof.

As mentioned above, the conventional sleep detection device is mainly configured to detect the blinking speed and needs to capture images at a higher frame rate. Therefore, the present disclosure further provides a monitoring and warning system for vehicles (FIGS. 2-4) that may simultaneously detect a stop traffic sign and a sleep expression to accordingly warn the driver to decrease the power output or to concentrate thereby broadening the application of the vehicle control system. And since the present disclosure may detect the sleep expression according to a single image, the frame rate of the image sensor is decreased.