Road user identification system for identifying a road user for safer autonomous vehicle road navigation

A road user identification system for identifying a road user is provided. The system includes a digital camera generating a digital image of a road with the road user on the road. The system further includes a computer operably coupled to the digital camera that receives the digital image. The computer has a digital image based classification model with a neural network machine learning algorithm that analyzes the digital image and determines a first probability value. The first probability value indicates a probability that the road user has a first wearable marker that is associated with a specific type of road user. The computer stores a road user identifier corresponding to the specific type of road user in a memory device when the first probability value is greater than a threshold probability value.

BACKGROUND

Driver assistance and autonomous vehicle systems have been developed to prevent or mitigate collisions between vehicles and road users such as pedestrians and cyclists. However, these systems face challenges with respect to a detection accuracy of road users and a confidence in the detection of road users which can slow down the response time of these systems.

The inventor herein has recognized that it would be advantageous to provide an improved road user identification system that is configured to detect wearable markers to quickly identify types of road users to improve the detection response time of a road user, and the overall vehicle movement response time to safely navigate around and to not contact the road user.

SUMMARY

A road user identification system for identifying a road user for safer autonomous road navigation in accordance with an exemplary embodiment is provided. The road user identification system includes a digital camera generating a digital image of a road with the road user on the road. The road user identification system further includes a computer operably coupled to the digital camera that receives the digital image. The computer has a digital image based classification model with a neural network machine learning algorithm that analyzes the digital image and determines a first probability value. The first probability value indicates a probability that the road user has a first wearable marker that is associated with a specific type of road user. The computer stores a road user identifier corresponding to the specific type of road user in a memory device when the first probability value is greater than a threshold probability value.

A road user identification system for identifying a road user for safer autonomous road navigation in accordance with another exemplary embodiment is provided. The road user identification system includes a radar system generating radar reflection data representative of a road with the road user on the road. The road user identification system further includes a computer operably coupled to the radar system that receives the radar reflection data. The computer has a radar reflection data based classification model with a neural network machine learning algorithm that analyzes the radar reflection data and determines a first probability value. The first probability value indicates a probability that the road user has a first wearable marker that is associated with a specific type of road user. The computer stores a road user identifier corresponding to the specific type of road user in a memory device when the first probability value is greater than a threshold probability value.

A road user identification system for identifying a road user for safer autonomous road navigation in accordance with another exemplary embodiment is provided. The road user identification system includes a lidar system generating lidar reflection data representative of a road with the road user on the road. The road user identification system further includes a computer operably coupled to the lidar system that receives the lidar reflection data. The computer has a lidar reflection data based classification model with a neural network machine learning algorithm that analyzes the lidar reflection data and determines a first probability value. The first probability value indicates a probability that the road user has a first wearable marker that is associated with a specific type of road user. The computer stores a road user identifier corresponding to the specific type of road user in a memory device when the first probability value is greater than a threshold probability value.

DETAILED DESCRIPTION

Referring toFIGS.1and2, a vehicle30that identifies and avoids a road user20on a road in accordance with an exemplary embodiment is illustrated. As shown, the road user20is a cyclist riding a bicycle22on the road40.

For purposes of understanding, a few terms will be defined hereinafter.

The term “road user” means a human or an animal that is on a road and is not enclosed within a vehicle.

The term “wearable marker” means a predetermined marker that is placed on or in an article of clothing with specific detection properties that indicates a specific type of road user. In particular, a wearable marker can identify at least one of the following types of road users: a cyclist, an adult, a child, a senior citizen for example. It is noted that the wearable marker is distinct from the article of clothing even though it is attached or fixedly coupled to the clothing. Further, each type of wearable marker has a distinct detection signature and may have a distinct shape or a distinct reflection characteristic. In an exemplary embodiment, the wearable marker can be detected utilizing at least one of a digital camera280, a radar system282, or a lidar system284.

Referring toFIG.1, the road user20is wearing a wearable marker52that indicates a cyclist. Referring toFIG.4, the road user60is wearing a wearable marker62that indicates a senior citizen. Referring toFIG.5, the road user70is wearing a wearable marker72that indicates a child. Referring toFIG.6, the road user80is wearing a wearable marker82that indicates an adult. Referring toFIG.7, the road user90is wearing a wearable marker92that indicates a pedestrian. Referring toFIG.8, the road user100is wearing a wearable marker102that indicates an animal.

The inventor herein has recognized that a wearable marker allows the road user identification system260in the vehicle30to more quickly identify a specific type of road user and to control movement of the vehicle30to avoid contacting the road user. Further, the system260can determine an estimated initial velocity of the road user, an estimated acceleration of the road user, and a time interval based on the road user identifier. For example, a cyclist will have an estimated initial velocity that is greater than an estimated initial velocity of a pedestrian, and an estimated acceleration that is greater than an estimated acceleration of the pedestrian. Further, because the cyclist is likely moving faster than the pedestrian, the time interval associated with the cyclist could be more than a time interval associated with a pedestrian to model a more imminent path intersection.

Referring toFIGS.1and2, the vehicle30includes a vehicle body250, a road user identification system260, a vehicle brake controller262, a vehicle powertrain controller264, and a vehicle steering controller266.

The road user identification system260is disposed within the vehicle body250and is provided to control movement of the vehicle30such that the vehicle30navigates safely around and does not contact a road user. The road user identification system260includes a digital camera280, a radar system282, a lidar system284, and a computer286.

The digital camera280is coupled to the vehicle body250. The digital camera280is provided to receive light reflected off of the road40and objects thereon and to generate a plurality of digital images of the road40and objects thereon utilizing the reflected light. Further, the digital camera280sends the plurality of digital images to the computer286. In an exemplary embodiment, the digital camera280has a detection range290in front of the vehicle30.

The radar system282is coupled to the vehicle body250. The radar system282is provided to generate radio wave pulses and to generate radar reflection data representative of the road40and objects thereon from the radio wave pulses being reflected off of the road and objects and back to the radar system282. Further, the radar system282sends the radar reflection data to the computer286. In an exemplary embodiment, the radar system282has a detection range292in front of the vehicle30.

The lidar system284is coupled to the vehicle body250. The lidar system284is provided to generate laser beam pulses and to further generate lidar reflection data representative of the road40and objects thereon from the laser beam pulses being reflected off of the road and objects and back to the lidar system282. Further, the lidar system284sends the lidar reflection data to the computer286. In an exemplary embodiment, the lidar system284has a detection range294around the vehicle30.

Referring toFIGS.2and3, the computer286is provided to control the operation of the vehicle30and to implement the methods described herein. In particular, the computer286sends and receives data from the digital camera280, the radar system282, and the lidar system284. Also, the computer286generates control signals that are received by the vehicle brake controller262, the vehicle powertrain controller264, and the vehicle steering controller266. As shown, the computer286is operably coupled the digital camera280, the radar system282, and the lidar system284, the vehicle brake controller262, the vehicle powertrain controller264, and the vehicle steering controller266. The computer286includes a microprocessor340operably communicating with a memory342. The microprocessor340implements the methods performed by the computer286, and the memory342stores data utilized in the methods described herein. Further, the memory342stores a digital image based classification model350, a radar reflection data based classification model352, a lidar reflection data based classification model354, and a movement prediction and determination module356.

The digital image based classification model350has a neural network machine learning algorithm that analyzes a digital image and determines a first probability value. The first probability value indicates a probability that a road user has a wearable marker that is associated with a specific type of road user (e.g., a cyclist, an adult, a child, or a senior citizen). The digital image based classification model350is initially trained using: (i) a plurality of images of road users wearing a specific wearable marker, and (ii) respective tags indicating the type of road user. Although only one digital image based classification model has been illustrated herein for purposes of simplicity, it is noted that in an alternative embodiment, there could be a distinct digital image based classification model for identifying each type of road user (e.g., a cyclist, an adult, a child, or a senior citizen).

The radar reflection data based classification model352has a neural network machine learning algorithm that analyzes radar reflection data and determines a first probability value. The first probability value indicates a probability that a road user has a wearable marker that is associated with a specific type of road user (e.g., a cyclist, an adult, a child, or a senior citizen). The radar reflection data based classification model352is initially trained using: (i) a plurality of radar reflection data of road users wearing a specific wearable marker, and (ii) respective tags indicating the type of road user. Although only one radar reflection data based classification model has been illustrated herein for purposes of simplicity, it is noted that in an alternative embodiment, there could be a distinct radar reflection data based classification model for identifying each type of road user (e.g., a cyclist, an adult, a child, or a senior citizen).

The lidar reflection data based classification model354has a neural network machine learning algorithm that analyzes lidar reflection data and determines a first probability value. The first probability value indicates a probability that a road user has a wearable marker that is associated with a specific type of road user (e.g., a cyclist, an adult, a child, or a senior citizen). The lidar reflection data based classification model354is initially trained using: (i) a plurality of lidar reflection data of road users wearing a specific wearable marker, and (ii) respective tags indicating the type of road user. Although only one lidar reflection data based classification model has been illustrated herein for purposes of simplicity, it is noted that in an alternative embodiment, there could be a distinct lidar reflection data based classification model for identifying each type of road user (e.g., a cyclist, an adult, a child, or a senior citizen).

Referring toFIG.1, the vehicle brake controller262is disposed within the vehicle30and is provided to control a brake system of the vehicle30in response to control signals from the computer286. As shown, the vehicle brake controller262is operably coupled to the computer286.

The vehicle powertrain controller264is disposed within the vehicle30and is provided to control a powertrain of the vehicle30in response to control signals from the computer286. As shown, the vehicle powertrain controller264is operably coupled to the computer286.

The vehicle steering controller266is disposed within the vehicle30and is provided to control a steering system of the vehicle30in response to control signals from the computer286. As shown, the vehicle steering controller266is operably coupled to the computer286.

Referring toFIGS.1-3and9, a flowchart of a method for identifying and avoiding a road user utilizing the road user identification system260with the digital camera280in accordance with another exemplary embodiment will be explained.

At step500, a digital camera280generates a digital image of a road40with the road user20on the road40. After step500, the method advances to step502.

At step502, a computer286has a digital image based classification model350with a neural network machine learning algorithm that analyzes the digital image and determines a first probability value. The first probability value indicates a probability that the road user20has a first wearable marker52that is associated with a specific type of road user (e.g., a cyclist). After step502, the method advances to step503.

At step503, the computer286stores a road user identifier corresponding to the specific type of road user in a memory device342when the first probability value is greater than a threshold probability value. After step503, the method advances to step504.

At step504, the computer286determines a first position of the road user20relative to a vehicle30at a first time utilizing the first wearable marker52in the digital image when the first probability value is greater than a threshold probability value. After step504, the method advances to step506.

At step506, the computer286has a movement prediction and determination module356that determines an estimated initial velocity of the road user, an estimated acceleration of the road user, and a time interval, based on the road user identifier. In an exemplary embodiment, the movement prediction and determination module356utilizes a table having a plurality of records that are indexed by the type of road user, and each record has an estimated initial velocity value, an estimated acceleration value, and a time interval value. The module356accesses the table utilizing the type of road user as an index, and retrieves the associated estimated initial velocity value, estimated acceleration value, and time interval value from a specific record in the table. After step506, the method advances to step508.

At step508, the computer286determines a second position of the road user20at a second time utilizing the following equation: second position=first position+(estimated initial velocity of the road user*time interval from the first time to the second time)+½ (estimated acceleration of the road user*(time interval from the first time to the second time)2). After step508, the method advances to step510.

At step510, the computer286determines a desired trajectory of the vehicle30based on the second position of the road user20and a position of the vehicle30to navigate safely around the road user20. After step510, the method advances to step512.

At step512, the computer286generates control signals based on the second position to induce at least one of a vehicle brake controller262, a vehicle powertrain controller264, and a vehicle steering controller266to control movement of the vehicle30such that the vehicle30traverses the desired trajectory and navigates safely around the road user20.

Referring toFIGS.1-3and10, a flowchart of a method for identifying and avoiding a road user utilizing the road user identification system260with the radar system282in accordance with another exemplary embodiment will be explained.

At step600, the radar system282generates radar reflection data representative of a road40with the road user20on the road40. After step600, the method advances to step602.

At step602, the computer286has a radar reflection data based classification model352with a neural network machine learning algorithm that analyzes the radar reflection data and determines a first probability value. The first probability value indicates a probability that the road user20has a first wearable marker52that is associated with a specific type of road user20(e.g., a cyclist). After step602, the method advances to step604.

At step603, the computer286stores a road user identifier corresponding to the specific type of road user in a memory device342when the first probability value is greater than a threshold probability value. After step603, the method advances to step604.

At step604, the computer286determines a first position of the road user20relative to a vehicle30at a first time utilizing the radar reflection data corresponding to the first digital marker52when the first probability value is greater than a threshold probability value. After step604, the method advances to step606.

At step606, the computer286has a movement prediction and determination module356that determines an estimated initial velocity of the road user20, an estimated acceleration of the road user20, and a time interval, based on the road user identifier20. After step606, the method advances to step608.

At step608, the computer286determines a second position of the road user20at a second time utilizing the following equation: second position=first position+(estimated initial velocity of the road user20*time interval from the first time to the second time)+½ (estimated acceleration of the road user20*(time interval from the first time to the second time)2). After step608, the method advances to step610.

At step610, the computer286determines a desired trajectory of the vehicle30based on the second position of the road user20and a position of the vehicle30to navigate safely around the road user20. After step610, the method advances step612.

At step612, the computer286generates control signals based on the second position to induce at least one of a vehicle brake controller262, a vehicle powertrain controller264, and a vehicle steering controller266to control movement of the vehicle30such that the vehicle30traverses the desired trajectory and navigates safely around the road user20.

Referring toFIGS.1-3and11, a flowchart of a method for identifying and avoiding a road user utilizing the road user identification system260with the lidar system284in accordance with another exemplary embodiment will be explained.

At step700, the lidar system284generates lidar reflection data representative of a road40with the road user20on the road40. After step700, the method advances to step702.

At step702, the computer286has a lidar reflection data based classification model354with a neural network machine learning algorithm that analyzes the lidar reflection data and determines a first probability value. The first probability value indicates a probability that the road user20has a first wearable marker52that is associated with a specific type of road user20(e.g., a cyclist). After step702, the method advances to step703.

At step703, the computer286stores a road user identifier corresponding to the specific type of road user in a memory device342when the first probability value is greater than a threshold probability value. After step703, the method advances to step704.

At step704, the computer286determines a first position of the road user20relative to a vehicle30at a first time utilizing the lidar reflection data corresponding to the first digital marker52when the first probability value is greater than a threshold probability value. After step704, the method advances to step706.

At step706, the computer286has a movement prediction and determination module356that determines an estimated initial velocity of the road user20, an estimated acceleration of the road user20, and a time interval, based on the road user identifier20. After step706, the method advances to step708.

At step708, the computer286determines a second position of the road user20at a second time utilizing the following equation: second position=first position+(estimated initial velocity of the road user*time interval from the first time to the second time)+½ (estimated acceleration of the road user*(time interval from the first time to the second time)2). After step708, the method advances to step710.

At step710, the computer286determines a desired trajectory of the vehicle30based on the second position of the road user20and a position of the vehicle30to navigate safely around the road user20. After step710, the method advances to step712.

At step712, the computer286generates control signals based on the second position to induce at least one of a vehicle brake controller262, a vehicle powertrain controller264, and a vehicle steering controller266to control movement of the vehicle30such that the vehicle30traverses the desired trajectory and navigates safely around the road user20.