Patent Description:
Riders of motorcycles have increased safety concerns than drivers of automotive vehicles. Additionally, due to the open area in which the riders are positioned when driving the motorcycles and the two-wheeled nature of the motorcycles, the availability of standard safety systems, such as airbags, is reduced.

Therefore, motorcycle riders must be careful to avoid traffic incidents. Specifically, the motorcycle riders must be careful to see the surrounding vehicle and make the motorcycle seen by drivers of the surrounding vehicles. One such instance in particular occurs when, at a stop, the single rear brake light of a motorcycle can be confused with one of the brake lights of a vehicle located in front of the motorcycle.

However, viewing angles of the riders who ride cycles such as bicycles or the motorcycles are limited, and accordingly, the riders cannot check all the surrounding environment of the cycles that are being driven, causing various accidents.

Accordingly, the inventors of the present disclosure propose a method for allowing the riders of the cycles to accurately perceive the surrounding environment.

It is noted that <CIT> discloses a motorcycle helmet that includes electronic components operating within the motorcycle helmet. At least a portion of the electronic components is embedded within an outer shell of the helmet or an inner shell of the helmet. The electronic components include an internally mounted Global Positioning System (GPS) transceiver that is part of a GPS subsystem of the electronic components.

It is an object of the present disclosure to solve all the aforementioned problems.

It is another object of the present disclosure to allow a rider of a cycle in operation to perceive surrounding environment of the cycle.

It is still another object of the present disclosure to allow the rider of the cycle to ride the cycle safely as a result of perceiving the surrounding environment.

It is still another object of the present disclosure to transmit information acquired by the rider of the cycle to one or more nearby vehicles over V2X communication.

In accordance with one aspect of the present disclosure, there is provided a method as claimed in independent claim <NUM>. Embodiments of the method are claimed in dependent claims <NUM> - <NUM>.

In accordance with still yet another aspect of the present disclosure, there is provided a blind-spot monitoring device as claimed in independent claim <NUM>. Embodiments of the blind-spot monitoring device are claimed in dependent claims <NUM>-<NUM>.

In addition, recordable media readable by a computer for storing a computer program to execute the method of the present disclosure is further provided.

The following drawings to be used to explain example embodiments of the present disclosure are only part of example embodiments of the present disclosure and other drawings can be obtained based on the drawings by those skilled in the art of the present disclosure without inventive work.

Detailed explanation on the present disclosure to be made below refer to attached drawings and diagrams illustrated as specific embodiment examples under which the present disclosure may be implemented to make clear of purposes, technical solutions, and advantages of the present disclosure. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention.

Besides, in the detailed description and claims of the present disclosure, a term "include" and its variations are not intended to exclude other technical features, additions, components or steps. Other objects, benefits and features of the present disclosure will be revealed to one skilled in the art, partially from the specification and partially from the implementation of the present disclosure. The following examples and drawings will be provided as examples but they are not intended to limit the present disclosure.

Moreover, the present disclosure covers all possible combinations of example embodiments indicated in this specification. It is to be understood that the various embodiments of the present disclosure, although different, are not necessarily mutually exclusive. For example, a particular feature, structure, or characteristic described herein in connection with one embodiment may be implemented within other embodiments without departing from the spirit and scope of the present disclosure. In addition, it is to be understood that the position or arrangement of individual elements within each disclosed embodiment may be modified without departing from the spirit and scope of the present disclosure. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present disclosure is defined only by the appended claims, appropriately interpreted, along with the full range of equivalents to which the claims are entitled. In the drawings, similar reference numerals refer to the same or similar functionality throughout the several aspects.

The headings and abstract of the present disclosure provided herein are for convenience only and do not limit or interpret the scope or meaning of the embodiments.

To allow those skilled in the art to carry out the present disclosure easily, the example embodiments of the present disclosure by referring to attached diagrams will be explained in detail as shown below.

<FIG> is a drawing schematically illustrating a blind-spot monitoring device for monitoring one or more blind spots of a cycle using a smart helmet of a rider of the cycle in accordance with one example embodiment of the present disclosure. By referring to <FIG> , the blind-spot monitoring device <NUM> includes a memory <NUM> for storing instructions to monitor the blind spots of the cycle by referring to sensor information acquired from one or more sensors installed on the smart helmet worn by the rider of the cycle, and a processor <NUM> for performing processes to monitor the blind spots of the cycle by referring to the sensor information acquired from the sensors installed on the smart helmet according to the instructions in the memory <NUM>. Throughout the present disclosure, the cycle may include a unicycle, a bicycle, a tricycle, a two-wheeled motorcycle, a one-wheeled or three-wheeled motor vehicle, etc..

Specifically, the blind-spot monitoring device <NUM> may typically achieve a desired system performance by using combinations of at least one computing device and at least one computer software, e.g., a computer processor, a memory, a storage, an input device, an output device, or any other conventional computing components, an electronic communication device such as a router or a switch, an electronic information storage system such as a network-attached storage (NAS) device and a storage area network (SAN) as the computing device and any instructions that allow the computing device to function in a specific way as the computer software.

The processor of the computing device may include hardware configuration of MPU (Micro Processing Unit) or CPU (Central Processing Unit), cache memory, data bus, etc. Additionally, the computing device may further include OS and software configuration of applications that achieve specific purposes.

However, such description of the computing device does not exclude an integrated device including any combination of a processor, a memory, a medium, or any other computing components for implementing the present disclosure.

A method for monitoring the blind spots of the cycle by referring to the sensor information acquired from the sensors installed on the smart helmet by using the blind-spot monitoring device <NUM> in accordance with one example embodiment of the present disclosure is described by referring to <FIG> as follows.

First, if at least one video image of one or more <NUM>-st blind spots is acquired from at least one camera or at least one radar sensor <NUM> installed on the smart helmet worn by the rider, the blind-spot monitoring device <NUM> performs a process of instructing an object detector to detect one or more objects on the video image, at a step of S1. Herein, the radar sensor may include a LiDAR sensor, a laser sensor, an ultrasonic sensor, etc., and may include any sensors capable of scanning their surrounding environment and acquiring images corresponding to the surrounding environment.

As one example, by referring to <FIG> , the blind-spot monitoring device <NUM> transmits the video image to the object detector <NUM>. Then, the object detector <NUM> may input the video image into a convolutional layer <NUM>, to thereby allow the convolutional layer <NUM> to apply its at least one convolution operation to the video image and thus to generate at least one feature map. And, the object detector <NUM> may input the feature map into a region proposal network <NUM>, to thereby allow the region proposal network <NUM> to output one or more proposal boxes corresponding to one or more objects on the feature map. Thereafter, the object detector <NUM> may input the feature map into a pooling layer <NUM>, to thereby allow the pooling layer <NUM> to output at least one feature vector by applying its pooling operation to one or more regions, corresponding to the proposal boxes, on the feature map. And, the object detector <NUM> may input the feature vector into a fully connected layer <NUM>, to thereby allow the fully connected layer <NUM> to apply its fully-connected operation to the feature vector, and may input at least one output from the fully connected layer <NUM> respectively into a classification layer <NUM> and a regression layer <NUM>, to thereby allow the classification layer <NUM> and the regression layer <NUM> to respectively generate class information and regression information on each of the objects corresponding to each of the proposal boxes, and as a result, may detect the objects on the video image.

Herein, the object detector may have been learned by a learning device.

That is, the learning device may input at least one training image into the convolutional layer <NUM>, to thereby allow the convolutional layer <NUM> to generate at least one feature map for training by applying its at least one convolution operation to the training image, and may input the feature map for training into the region proposal network <NUM>, to thereby allow the region proposal network <NUM> to output one or more proposal boxes for training, corresponding to one or more objects for training, on the feature map for training. And, the learning device may input the feature map for training into the pooling layer <NUM>, to thereby allow the pooling layer <NUM> to generate at least one feature vector for training by applying its pooling operation to one or more regions, corresponding to the proposal boxes for training, on the feature map for training, and may input the feature vector for training into the fully connected layer <NUM>, to thereby allow the fully connected layer <NUM> to apply its fully-connected operation to the feature vector for training. Thereafter, the learning device may input at least one output for training from the fully connected layer <NUM> respectively into the classification layer <NUM> and the regression layer <NUM>, to thereby allow the classification layer <NUM> and the regression layer <NUM> to respectively generate class information for training and regression information for training on each of one or more objects for training corresponding to each of the proposal boxes for training. And, the learning device may allow a loss layer to calculate one or more losses by referring to the class information for training and the regression information for training and their respective corresponding GTs, and may update at least one parameter of at least one of the fully connected layer <NUM> and the convolutional layer <NUM> via backpropagation using the losses such that the losses are minimized. And, as a result of repeating the above processes, the learning device may learn the object detector.

Next, the blind-spot monitoring device <NUM> performs a process of confirming one or more <NUM>-st objects, located in the <NUM>-st blind spots, among the objects detected by the object detector, at a step of S2.

Herein, the <NUM>-st blind spots are the blind spots corresponding to the smart helmet, and may be areas, within a preset distance from the smart helmet, that cannot be visually perceived by the rider.

Next, if the sensor information is acquired from at least part of the sensors <NUM>, for example, a GPS sensor, an acceleration sensor, and a geomagnetic sensor, installed on the smart helmet, then the blind-spot monitoring device <NUM> confirms a smart helmet orientation and a cycle traveling direction at a step of S3, by using the sensor information from at least part of the GPS sensor, the acceleration sensor, and the geomagnetic sensor.

And, the blind-spot monitoring device <NUM> performs a process of confirming one or more <NUM>-nd objects, located in one or more <NUM>-nd blind spots corresponding to the cycle, among the <NUM>-st objects, by referring to the smart helmet orientation and the cycle traveling direction at a step of S4. Herein, the <NUM>-nd blind spots are areas near the cycle that cannot be visually perceived by the rider.

That is, the blind-spot monitoring device <NUM> performs a process of calculating at least one angular difference between the smart helmet orientation and the cycle traveling direction, a process of converting one or more <NUM>-st locations of the <NUM>-st objects into one or more relative locations corresponding to the cycle traveling direction by using the angular difference, and a process of determining at least part of the <NUM>-st objects, corresponding to at least part of the relative locations matching the <NUM>-nd blind spots, as the <NUM>-nd objects.

As one example, by referring to <FIG> , one or more sensing angles <NUM> of the camera or the radar sensor installed on the smart helmet <NUM> worn by the rider of the cycle <NUM> may correspond to the blind spots of the smart helmet, that is, one or more rear areas of the smart helmet, and the blind spots <NUM> of the cycle <NUM> are one or more preset areas set as areas where the rider cannot perceive visually depending on a traveling direction of the cycle <NUM>.

Herein, the smart helmet orientation of the smart helmet <NUM> may be changed by the rider, and in such a case, the <NUM>-st objects located in the <NUM>-st blind spots, which are the blind spots of the smart helmet detected by the object detector, may not be the objects detected from the <NUM>-nd blind spots which are the blind spots of the cycle.

Therefore, the blind-spot monitoring device <NUM> determines an angle between the smart helmet orientation RD and the cycle traveling direction BD on a basis of the smart helmet <NUM>, converts the <NUM>-st locations of the <NUM>-st objects into the relative locations on a basis of the cycle traveling direction BD by referring to the determined angle, and determines at least part of the <NUM>-st objects, whose relative locations are in the <NUM>-nd blind spots <NUM>, as the <NUM>-nd objects, to thereby detect the objects located in the blind spots <NUM> of the cycle.

Next, the blind-spot monitoring device <NUM> performs a process of displaying the <NUM>-nd objects, at a step of S5, located in the <NUM>-nd blind spots, which are the blind spots of the cycle, via a head-up display installed on the smart helmet, or a process of sounding an alarm representing that the <NUM>-nd objects are located in the <NUM>-nd blind spots via at least one speaker installed on the smart helmet, to thereby allow the rider to safely drive the cycle by perceiving that the objects, that is, one or more pedestrians, one or more vehicles, or one or more other cycles, are located in the blind spots of the cycle.

Also, at a same time, the blind-spot monitoring device <NUM> may confirm one or more rider blind spots at a step of S6 by referring to a viewing angle of the rider wearing the smart helmet and the sensing angles of the radar sensor or the camera taking the video image.

That is, by referring to <FIG> again, the rider blind spots <NUM> may be determined, which are out of ranges of the sensing angles <NUM> of the camera or the radar sensor installed on the smart helmet and the viewing angle <NUM> of the rider wearing the smart helmet.

Therefore, the blind-spot monitoring device <NUM> may transmit, at a step of S7, (i) rider blind-spot information on the rider blind spots acquired by referring to the viewing angle of the rider wearing the smart helmet and the sensing angles of the camera taking the video image or the radar sensor, and (ii) a cycle location, a cycle traveling direction, and a cycle traveling speed acquired by referring to the sensor information from the smart helmet, to one or more nearby vehicles and one or more nearby smart helmets of one or more nearby cycles. Herein, the blind-spot monitoring device <NUM> may transmit the rider blind-spot information, the cycle location, the cycle traveling direction, and the cycle traveling speed over V2X (vehicle to everything) communication.

Then, a specific nearby vehicle located in one of the rider blind spots may alert a specific nearby driver of the specific nearby vehicle using a probability of a traffic accident between the specific nearby vehicle and the cycle, where the probability may be determined by referring to (i) a vehicle location, a vehicle traveling direction, and a vehicle traveling speed acquired from the sensor information of the specific nearby vehicle and (ii) the cycle location, the cycle traveling direction, and the cycle traveling speed acquired from the blind-spot monitoring device <NUM>. Also, at least one specific nearby smart helmet, corresponding to at least one specific nearby cycle located in the rider blind spots, among the nearby smart helmets, may determine a possibility of a traffic accident between the specific nearby cycle and the cycle by referring to (i) a nearby cycle location, a nearby cycle traveling direction, and a nearby cycle traveling speed acquired from sensor information of the specific nearby smart helmet and (ii) the cycle location, the cycle traveling direction, and the cycle traveling speed acquired from the blind-spot monitoring device <NUM>, and may thus alert at least one specific nearby rider corresponding to the specific nearby cycle.

Herein, if the specific nearby driver of the specific nearby vehicle located in the rider blind spots operates a steering wheel of the specific nearby vehicle to move into a nearby front area of the cycle by referring to (i) the vehicle location, the vehicle traveling direction, and the vehicle traveling speed acquired from sensor information of the specific nearby vehicle and (ii) the cycle location, the cycle traveling direction, and the cycle traveling speed acquired from the blind-spot monitoring device <NUM>, the specific nearby vehicle located in the rider blind spots may perform a process of preventing the steering wheel from rotating or of vibrating the steering wheel in order to alert the specific nearby driver.

As one example, by referring to <FIG>, a vehicle <NUM> traveling near the cycle <NUM> may perform a process of confirming whether the vehicle <NUM> is located in one of the rider blind spots <NUM> by referring to the rider blind-spot information received from the blind-spot monitoring device <NUM>, and a process of, if the vehicle <NUM> is determined as located in said one of the rider blind spots <NUM>, confirming a dangerous area <NUM> having a probability of a traffic accident larger than a preset threshold in case the vehicle traveling direction of the vehicle <NUM> is changed to pointing to an area where the rider of the cycle <NUM> is not paying attention, that is, to the nearby front area of the cycle where the rider is not visually observing, and thus preventing the traffic accident between the vehicle <NUM> and the rider <NUM> by stopping the vehicle <NUM> from entering the dangerous area <NUM>.

Also, in case that the specific nearby vehicle located in the rider blind spots is an autonomous vehicle, if a driving plan of the autonomous vehicle is determined as representing moving into the nearby front area of the cycle by referring to (i) an autonomous vehicle location, an autonomous vehicle traveling direction, and an autonomous vehicle traveling speed acquired from sensor information of the autonomous vehicle and (ii) the cycle location, the cycle traveling direction, and the cycle traveling speed, then the autonomous vehicle located in the rider blind spots may prevent itself from changing lanes due to the driving plan of the autonomous vehicle.

As one example, by referring to <FIG> , the autonomous vehicle <NUM> may determine whether to change the lanes, and if lane-changing is to be performed, may determine a lane-changing direction by referring to the driving plan <NUM> and at least one signal from at least one steering sensor <NUM> at a step of S11, and while the autonomous vehicle <NUM> drives itself by referring to the autonomous vehicle location, the autonomous vehicle traveling direction, and the autonomous vehicle traveling speed using the sensor information acquired from at least one location sensor and at least one speed sensor <NUM> at a step of S12, if the rider blind-spot information, the cycle location, the cycle traveling direction, and the cycle traveling speed are acquired via a V2x communication part <NUM>, the autonomous vehicle <NUM> may determine that itself is located in one of the rider blind spots by referring to the rider blind-spot information.

And, if the autonomous vehicle <NUM> is determined as located in said one of the rider blind spots, the autonomous vehicle <NUM> may determine whether a driving environment of the autonomous vehicle <NUM> is dangerous, at a step of S13.

That is, in case that the driving plan of the autonomous vehicle is determined as representing moving from said one of the rider blind spots into the nearby front area of the cycle by referring to (i) the autonomous vehicle location, the autonomous vehicle traveling direction, and the autonomous vehicle traveling speed acquired from the sensor information of the autonomous vehicle and (ii) the cycle location, the cycle traveling direction, and the cycle traveling speed, then the autonomous vehicle may determine that the probability of the traffic accident is larger than the preset threshold because the rider of the cycle cannot perceive the autonomous vehicle <NUM>.

Then, the autonomous vehicle <NUM> may operate an electronic steering device <NUM> to make the steering wheel difficult to move or to vibrate the steering wheel, to thereby allow the driver of the autonomous vehicle <NUM> to perceive a dangerous situation. Also, the autonomous vehicle <NUM> may operate an autonomous driving system <NUM> to stop the autonomous vehicle <NUM> from changing the lanes toward the cycle, in order to prevent the traffic accident. Also, the autonomous vehicle <NUM> may operate an alarming device <NUM> to alert the driver of the autonomous vehicle or the rider of the cycle using light, sound, etc., in order to prevent the traffic accident.

The present disclosure as defined in the appended claims has an effect of preventing traffic accidents by allowing the rider of the cycle being driven to perceive surrounding environment of the cycle.

The present disclosure has another effect of improving a driving quality of the rider by allowing the rider of the cycle to perceive the surrounding environment of the cycle.

The present disclosure as claimed in claims <NUM> and <NUM> has in addition the effect of allowing the nearby vehicles to safely travel by transmitting information acquired by the rider of the cycle to the nearby vehicles over the V2X communication, and as a result, reducing the traffic accidents on a roadway.

Claim 1:
A method performed by a blind-spot monitoring device (<NUM>) for monitoring at least one blind spot of a cycle (<NUM>) using a smart helmet (<NUM>) to be used for a rider of the cycle, wherein the at least one blind spot (<NUM>) of the cycle (<NUM>) is one or more preset areas set as areas where the rider cannot perceive visually depending on a traveling direction of the cycle (<NUM>) and the smart helmet having at least one camera (<NUM>) or at least one radar installed thereon, the method comprising steps of:
(a) if at least one video image of one or more <NUM>-st blind spots corresponding to the smart helmet worn by the rider is acquired from the at least one camera or radar, detecting (S1) by an object detector (<NUM>) one or more objects on the video image and confirming (S2) one or more <NUM>-st objects located in the one or more <NUM>-st blind spots among the detected objects; and
(b) determining (S3) a smart helmet orientation (RD) and a cycle traveling direction (BD) by using sensor information acquired from at least part of a GPS sensor, an acceleration sensor, and a geomagnetic sensor installed on the smart helmet, confirming (S4) one or more <NUM>-nd objects, among the one or more <NUM>-st objects, located in one or more <NUM>-nd blind spots corresponding to the blind spots (<NUM>) of the cycle by referring to the smart helmet orientation and the cycle traveling direction, wherein confirming (S4) one or more <NUM>-nd objects, among the one or more <NUM>-st objects, is performed by:
calculating at least one angular difference between the smart helmet orientation (RD) and the cycle traveling direction (BD) converting one or more <NUM>-st locations of the one or more <NUM>-st objects into one or more relative locations corresponding to the cycle traveling direction by using the at least one angular difference, determining at least part of the one or more <NUM>-st objects, corresponding to at least part of the relative locations matching the one or more <NUM>-nd blind spots, as the <NUM>-nd objects, and the method further comprising, subsequent to confirming (S4), the steps of
displaying (S5) the <NUM>-nd objects via a head-up display installed on the smart helmet or
sounding an alarm representing that the <NUM>-nd objects are located in the one or more <NUM>-nd blind spots via at least one speaker installed on the smart helmet