Vehicle lane map estimation

A computer can receive, from a vehicle sensor, data about a plurality of second vehicles, define two or more vehicle clusters based on location data of second vehicles, each cluster including two or more of the second vehicles determined to be traveling in a same lane, identify two or more lane boundaries according to clusters, and use lane boundaries to generate a lane map.

BACKGROUND

Sensors are used in vehicles to detect road features, e.g., lane markings. These sensors may assist an operator to improve driving safety, e.g., avoiding an unintended lane change. Alternatively, in a semi-autonomous or in an autonomous vehicle with limited monitoring or without monitoring of operation of the vehicle by occupants of the vehicle, these sensors may be used to determine location of lane markings relative to the vehicle on the road. However, vehicle sensors do not always provide adequate data to identify lane markings, e.g., because of sensor faults, environmental conditions, etc.

DETAILED DESCRIPTION

Introduction

FIG. 1illustrates an exemplary lane-identification system100. A computer112in a first vehicle110(sometimes referred to for convenience as a “host” vehicle110) comprises a processor and a memory, the memory storing instructions such that the processor is programmed for various operations, including as described herein. For example, the computer112can receive, from a vehicle sensor111, data about a plurality of second vehicles120, define two or more vehicle clusters L1, L2, L3(seeFIG. 2) based on location data of second vehicles120, each cluster including two or more of the second vehicles120determined to be traveling in a same lane, identify two or more lane boundaries201,202,203according to clusters L1, L2, L3, and use lane boundaries201,202,203to generate a lane map.

The lane map describes respective positions of the lane boundaries201,202,203either relative to a location C0of the vehicle110or based on geolocation coordinates like GPS latitude and longitude coordinates. The lane map may include multiple discrete points representing an estimated location of a lane boundary, e.g. the boundary201. Further, a curve fitting technique can be used to fit a curve between the multiple discrete points, e.g. a Bezier curve fitting formula. The lane map may be used to warn an operator of the vehicle110about an unintended lane change, navigate a vehicle in a lane without operator monitoring, or provide this information of the lane map to other vehicles, e.g., through a vehicle-to-vehicle communication interface.

Exemplary System Elements

The vehicle110includes the computer112that generally includes the processor and the memory, the memory including one or more forms of computer-readable media, and storing instructions executable by the processor for performing various operations, including as disclosed herein. For example, the computer112may include programming to operate one or more of vehicle brakes, propulsion (e.g., control of acceleration in the vehicle110by controlling one or more of an internal combustion engine, electric motor, hybrid engine, etc.), steering, climate control, interior and/or exterior lights, etc., as well as to determine whether and when the computer112, as opposed to a human operator, is to control such operations. Further, the computer112may include programming to determine one or more lane boundaries and to receive data relating to determining one or more lane boundaries, and/or to receive a lane boundary determination, from other vehicles120(i.e., from computers therein) and/or a remote computer140.

The computer112may include or be communicatively coupled to one or more wired or wireless communications networks, e.g., via a vehicle communications bus, Ethernet, etc. Via a vehicle communication network, the computer112may send and receive data to and from controllers or the like included in the vehicle110for monitoring and/or controlling various vehicle components, e.g., electronic control units (ECUs). As is known, an ECU can include a processor and a memory and can provide instructions to actuators to control various vehicle110components, e.g., ECUs can include a powertrain ECU, a brake ECU, etc. In general, the computer112may transmit messages to various devices in the vehicle110and/or receive messages from the various devices, e.g., controllers, actuators, sensors, etc., including sensors111. Alternatively or additionally, in cases where the computer112actually comprises multiple devices, the vehicle communication bus may be used for communications between devices represented as the computer112in this disclosure.

In addition, the computer112may be configured for communicating with a remote computer140, e.g. a cloud server, via a network130, which may utilize various wired and/or wireless networking technologies, e.g., cellular, BLUETOOTH®, wired and/or wireless packet networks, etc.

Referring toFIG. 1, sensors111may include a variety of devices known to provide data to the computer112. For example, a radar fixed to a front bumper (not shown) of the vehicle110may provide locations of the second vehicles120travelling in front of the vehicle110, relative to the location of the vehicle110. As another example, the sensors111may include a global positioning system (GPS) sensor disposed in the vehicle110that provides geographical coordinates or geolocation of the vehicle110. As another example, the sensors111may include a Light Detection And Ranging (LIDAR) sensor disposed on a top of the vehicle110that provides relative locations, sizes, and shapes of the second vehicles120surrounding the vehicle110, including the second vehicles120travelling next to or behind the vehicle110. The sensors111may include a camera, e.g. a front facing camera providing images from a forward direction of the vehicle110. A computing device, e.g., the computer112, may be programmed to receive image data from the camera, implement image processing techniques to detect the lane markings201,2020,203, and create the lane map. The camera may provide an image or images of one or more objects, for example, the front facing camera may detect the second vehicles120in front of vehicle110. Based on camera images, the computer112may determine a relative distance of the vehicle110from one of more lane markings adjacent the vehicle110, e.g., with reference forFIG. 2, the distances DR0/DL0from the center C0of the vehicle110to the lane marking203/202on a right/left side of the vehicle110.

Data provided from different sensors111may have different properties, range, accuracy, and failure conditions. For example, radar and LIDAR can provide distance to objects, e.g., one or more second vehicles120. A radar can work in even when environmental conditions are poor for visual sensing, e.g. fog, heavy precipitation, etc., and depending on a type of radar may have a long range, e.g., 250 meters from the vehicle110. A camera may provide information like type of the object, e.g., lane markings201,202,203, which cannot be provided by the radar or the LIDAR. On the other hand, a camera may fully or partially fail to provide information in bad environmental conditions, e.g., rain, fog, etc. For example, when a road is snow-covered, a camera may detect locations of the second vehicles120but not lane markings201,202,203.

The second vehicles120may include sensors such as radar, GPS sensor, LIDAR sensors etc., and may provide data to computing devices in respective second vehicles120. The second vehicles120may include vehicle-to-vehicle communication interfaces such as are known that provide sensor data and/or other second vehicle120data to the host vehicle110. Additionally or alternatively, a second vehicle120could provide second vehicle120data to a remote computer140via a network130. The host vehicle110may receive second vehicle120data from the remote computer140. For example, the second vehicles120may include cameras operable to detect lane markings201,202, and203. The cameras of the second vehicles120may be operable to determine distances from the second vehicles120to adjacent lane markings of the second vehicles120, e.g., the distances DR/DL from the center C of the second vehicle120B to the lane marking203/202on a right/left side of the second vehicle120B. As stated above, the distances DL and/or DR from the center C of the second vehicle120B may be sent to the vehicle110or the remote computer140through a wireless interface of the second vehicle120B.

FIG. 2illustrates the host vehicle110and the second vehicles120travelling in clusters L1, L2, and L3, separated by lane boundaries202and203. The lane boundaries may be curved or straight lines.

Multiple vehicles travelling between the same two lane boundaries may be in a same cluster, e.g., the host vehicle110, the second vehicle120B, and the second vehicles120D are in the cluster L2.

As described further below, the lane boundaries may be represented by a fitted curve or a fitted line between multiple discrete points on the lane boundary. The lane map includes the information of a plurality of lane boundaries, e.g.201,202, and203.

Geolocations of a lane boundary or geolocations of discrete points representing a lane boundary may be defined relative to the center C0of the host vehicle110, e.g., when a camera in the host vehicle110includes a processor programmed to detect the lane markings201,202,203.

As stated above, the sensors111, e.g. a camera of the host vehicle110, may fully or partially fail to provide usable image data for various reasons, e.g., bad environmental conditions like rain, fog, etc., or obstacles like other vehicles. For example, as illustrated inFIG. 2, a second vehicle120B could limit a field of view of a camera in the host vehicle110. With such a limited view, the camera in the vehicle110may not obtain an image of a section205of the lane boundary202.

As another example, geolocations of a lane boundary or geolocation of discrete points representing a lane boundary may be defined based on GPS geolocations, e.g., determined by GPS sensors of the second vehicles120and communicated via the network130. Based on the geolocations of the second vehicles, a cluster path between the centers of vehicles in a cluster may be determined, e.g., a cluster path for the cluster L2may be a fitted curve between the center C0of the host vehicle110, the center C of the second vehicle120B, and the center C of the second vehicle120D. Lane boundaries separating the adjacent cluster paths may be determined to be between the two adjacent cluster paths, for example in the middle, e.g., the lane boundary202may be a curve between the cluster paths of the cluster L1and L2. Alternatively or additionally, the second vehicles120may send the distances of their centers to the adjacent lane boundaries to the host vehicle110, e.g., the host vehicle may receive information indicating a distance DL of the second vehicle120B from the lane boundary202. In such a configuration, a geolocation of a discrete point on the lane boundary202may be determined, instead of assuming the lane boundary202is in the middle of the cluster paths of L1and L2.

Processes

FIG. 3illustrates an example process300for creating a lane map in the system ofFIG. 1.

The process300begins in a block301, in which the computer112receives sensor data. As discussed above, such data may include data received from the remote computer140, the second vehicles120, and/or sensor data received from sensors111included in the vehicle110.

Second vehicle120sensor data may include respective locations of each second vehicle120, e.g. geolocations determined by GPS sensors in the second vehicles120, lateral distance of the second vehicles120from adjacent lane boundaries, e.g. the distance DL of the second vehicle120B from the lane boundary202, yaw angle of the second vehicles120, lane maps generated by computers in the second vehicles120. Data received from the remote computer140may include data regarding road and/or environmental conditions, e.g., a lane closure or narrow lane due to construction, traffic conditions, police deployment on a side of the road, temporary change of number of lanes, precipitation likely to reduce road friction, affect visibility, etc. Additionally or alternatively, the data received from the remote computer140may include data, which are received by the remote computer140from the second vehicles120.

Sensor111data may include geolocation of the vehicle110, e.g. from a GPS sensor is included in the vehicle110. Further, vehicle110sensor111data may be used to determine locations of respective second vehicles120relative to the vehicle110, e.g. a LIDAR in the vehicle110may measure relative distances of the second vehicles120within a predetermined distance from the vehicle110. A LIDAR may illuminate the second vehicles120with a laser light and calculate distance from the LIDAR to the second vehicles120by measuring time for a laser signal to return. Alternatively or additionally, image data from a camera in the vehicle110may be used to measure a relative distance to the second vehicles120. As an example, a computing device receiving image data from a stereo camera may calculate the distance to a second vehicle120using known triangulation techniques. Alternatively, a computing device receiving image data from a mono camera may calculate distance to a second vehicle120using, for example, projected height techniques. In a projected height technique, a computing device may detect the object and estimate an actual size of the object, measure the number of pixels in the received image data, estimate the distance to the object based on known calibration parameters of the camera, e.g. focal point, etc. A yaw angle of the vehicle110, e.g., computed based on a yaw rate received from a yaw rate sensor in a vehicle, may also be used.

Next, in a block305, the computer112defines vehicle120clusters based on the received data at the block301, each cluster including two or more second vehicles120determined to be traveling in a same lane. Cluster definition, may include data received from the sensors111within the vehicles, and/or the data received via the network130. A process400for defining vehicle clusters is described below with respect toFIG. 4.

Next, in a block310, the computer112identifies lane boundaries, such as boundaries201,202,203, and204illustrated inFIG. 2. Such identification of lane boundaries may take into account cluster data and sensor data received from the second vehicles120. A process500is described below with respect toFIG. 5for identifying lane boundaries.

Next, in a block315, the computer112generates the lane map describing the position of the lane boundaries, e.g.,201,202,203, and204. As stated above, the lane map may be based on relative location data, e.g., relative location of the lane boundaries from the center C0of the vehicle110. Alternatively or additionally, a lane map may be based on geolocation coordinates, i.e., the lane boundaries can be specified by GPS coordinates. A lane map may further include confidence data for each section of a lane boundary, e.g., depending on a confidence parameter based on the sensor data used to estimate the lane boundaries.

In the block315, the computer112may take into account data indicating updates regarding infrastructure into account, e.g., a blocked lane may be removed from the lane map, or a position of a lane boundary may change when a change in lane width has been enforced by the infrastructure in a construction zone.

In the block315, the computer112may take a received lane map from a second vehicle120into account, e.g., a difference between a lane boundary based on calculations in the computer112versus the received lane map from a second computer may be considered in determining a confidence level for the lane, e.g. a difference more than a predetermined value may reduce the confidence in an accuracy of lane map calculated by the computer112.

Next, in a block320, the computer112may broadcast an updated lane map through the network130for receipt by one or more vehicles120and/or the remote server140.

Next, in a block325, the computer112may identify a cluster in which the host vehicle110is travelling. Such a determination may take into account the lane map and at least one of relative distance of the vehicle110to one or more second vehicles120, geolocation of the vehicle110relative to the lane map, and yaw angle of the vehicle110. For examples, the block325may include comparing a geolocation of the host vehicle110to each respective lane boundary of the lane map, identify two adjacent lane boundaries between which the geolocation of the host vehicle110is positioned, and assign the vehicle110to the identified lane.

Next, in a block330, the computer112may cause actuation of one or more vehicle110components, e.g., by sending an instruction to one or more vehicle110ECUs to control one or more of vehicle110steering, propulsion, and/or braking. Such actuation may be based on the lane map generated as described with respect to the block325and/or vehicle110data, e.g. a geolocation of the vehicle110that can be compared to geolocations specified by the lane map. In another example, the block330may include actuating a lane departure warning. The lane departure warning may warn a driver of the vehicle110when the vehicle110is about to make an unintended lane change, i.e., the vehicle110is approaching a lane boundary and the vehicle110turn signal is off. A lane departure warning can include actuating a haptic output of a steering wheel, providing an audible warning etc. Additionally or alternatively, the block330may include lane keeping assistance that includes actuating one or more of steering, propulsion, and braking to maintain the vehicle110in a lane, i.e., between two specific lane boundaries, e.g., the boundaries202,203. Lane keeping assistance may be implemented in form of a control loop, e.g., a proportional integral derivative (PID), taking sensor data from the vehicle110as input, e.g., identified cluster L2of the vehicle110, and distance DL and DR of the vehicle110from adjacent lane boundaries202and203, and actuate an ECU of the vehicle110, e.g., a steering controller. The lane keeping assistance may keep the vehicle110in the middle of the lane L2, i.e. keeping DL equal to DR, which is sometime referred to as lane centering function.

Following the block330, the process300ends.

FIG. 4illustrates the details of an exemplary process400for defining vehicle cluster(s), e.g., as mentioned above concerning the block305of the process300.

The process400begins with a block401, in which the computer112in an iterative action determines a drive path of each second vehicle120. The determination of a drive path for a second vehicle120may include using known techniques to interpolate between multiple locations of a respective second vehicle120.

Next, in a block405, the computer112identifies relevant second vehicles120for defining vehicle cluster(s), e.g., a second vehicle120with a lateral acceleration greater than a predetermined threshold may be determined to be changing lanes and therefore not useful for cluster definition.

Next, in a block410, the computer112may identify vehicles120in a same lane, e.g. by comparing drive paths of relevant second vehicles120determined in the block405, e.g. using known techniques such as supervised learning techniques such as support vector machine, neural network or regression analysis, unsupervised learning techniques such as clustering, or other forecasting models. Supervised learning and unsupervised learning include various known types of machine learning tasks. Supervised learning is a machine learning task of inferring a function from labeled training data. Unsupervised learning is a machine learning task of inferring a function to describe hidden structure from unlabeled data. As another example, a statistical model may take drive path of each relevant second vehicle120, and iteratively compare to drive paths of other relevant second vehicles120to identify the relevant second vehicles120, which follow one another substantially on the same drive path.

Next, in a block415, the computer112generates vehicle120clusters. One or more second vehicles120identified by the block410as travelling in a same lane is referred to as a vehicle cluster. When multiple lanes are available for driving, then multiple vehicle clusters may be identified. The block415may include identifying data for vehicle clusters, e.g., the computer112may include programming to fit a curve between centers C of selected second vehicles120determined in the block410to drive in a same lane, e.g., a polynomial of third degree such as Y=aX+bX2+cX3may be used to represent the vehicle cluster. Y and X represent longitudinal and lateral coordinates. Parameters a, b, and c of such a polynomial may determine a lane curvature for the lane, in which the respected cluster of vehicles travel. Additionally, a yaw angle of a leading second vehicle120of a vehicle cluster, i.e., determined to be in front of a vehicle cluster, may have a more significant value in estimating or extrapolating the curvature of an upcoming section of a lane.

Following the block415, the process400ends.

FIG. 5illustrates an exemplary process500for defining lane boundaries, e.g. as mentioned above concerning the block310of the process300.

The process begins with a block501, in which the computer112identifies adjacent vehicle clusters. “Adjacent vehicle cluster,” as that term is used herein, refers to two clusters of vehicles having only one lane boundary in between them, e.g., the illustrated lanes L1and L2have only the lane boundary202in between them. Such identification may be an iterative process to sort the vehicle clusters, e.g. by computing the polynomials of all vehicle clusters.

Next, in a block505, which the computer112may estimate width of lane(s), i.e., width of a lane in which a respective cluster of second vehicles120travel. In one embodiment, estimation of lane width may include computing an average distance between the curves fitted between the centers C of second vehicles120two adjacent cluster paths. Such estimation may be dependent on an assumption that vehicles in a cluster drive on average substantially in a middle of the respective lane, e.g., a center C of second vehicles is assumed to be within a 5% deviation from a center of the respective lane. In another example, an estimation of a lane width may depend on distances DL and DR of the second vehicles, when the second vehicles120provide the distances DL and DR. Alternatively, the second vehicles120may provide lane width estimation, e.g., from lane marking detecting sensors in second vehicles120. The block415may further take into account infrastructure data form the server130, e.g., as described above concerning road and/or environmental conditions, road layout, etc. As an example, road layout data such as a change in number of lanes, or lane width may be taken into account for defining vehicle clusters.

Next, in a block510, the computer112, identifies a plurality of points on each lane boundary between adjacent vehicle clusters. For example, to identify the lane boundary202between vehicle clusters L1and L2, multiple points forming the lane boundary202are identified in the block510. Such identification may be based, as stated above, on an assumption that the second vehicles120travel substantially in middle of lanes. In this case, multiple points in the middle of respective cluster paths of L1and L2may be included on the lane boundary202. Alternatively, second vehicle120sensor data may include distances DL and/or DR from lane boundaries, i.e. a marking of lanes of travel of second vehicles120. In such a configuration, in the block510, the computer112estimates points on lane boundaries using the distances DL and/or DR of second vehicles120, e.g., point206inFIG. 2on the lane boundary202may be identified based on sensor data DL of the second vehicle120B on the vehicle cluster L2.

Next, in a block515, the computer112fits a curve between multiple points identified in the block510on a respective lane boundary. Next, in a block520, the fitted curve of a respective lane boundary may be extrapolated based on sensor data, e.g., yaw rate, from second vehicles in vehicle clusters enclosing the respective lane boundaries, e.g., the lane boundary202may be extrapolated based on yaw rate data of second vehicles on the vehicle clusters L1and L2, especially based on sensor data from lead vehicles of vehicle clusters L1and L2. Lead vehicles are the second vehicles120in front of a vehicle cluster and more likely first vehicles of a vehicle cluster reacting to a change in lane curvature of the lane in which the vehicle cluster is travelling.

Following the block520, the process500ends.

With regard to the media, processes, systems, methods, etc. described herein, it should be understood that, although the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, such processes could be practiced with the described steps performed in an order other than the order described herein. It further should be understood that certain steps could be performed simultaneously, that other steps could be added, or that certain steps described herein could be omitted. In other words, the descriptions of systems and/or processes herein are provided for the purpose of illustrating certain embodiments, and should in no way be construed so as to limit the disclosed subject matter.