System and method for generating and communicating lane information from a host vehicle to a vehicle-to-vehicle network

A method of generating and communicating lane information from a host vehicle to a vehicle-to-vehicle (V2V) network includes collecting visual data from a camera, detecting a lane within the visual data, generating a lane classification for the lane based on the visual data, assigning a confidence level to the lane classification, generating a lane distance estimate from the visual data, generating a lane model from the lane classification and the lane distance estimate, and transmitting the lane model and the confidence level to the V2V network.

FIELD

The invention relates generally to a driver assistance system for motor vehicles, and more particularly to a driver assistance system for generating and communicating lane information from a host vehicle to a vehicle-to-vehicle (V2V) network.

BACKGROUND

Motor vehicle sensing systems are known which can identify to a host vehicle other proximate motor vehicles and warn an operator of the host vehicle of the other vehicle's movements which may intersect the driving path of the host vehicle. Other motor vehicle sensing systems are known which can utilize data from geographic positioning systems (GPS) to identify to a host vehicle the host vehicle position on a road. GPS data may also be used by other proximate motor vehicles to determine the position of the proximate motor vehicles on the road. Yet other motor vehicle sensing systems are known which can utilize the data received from the above noted sensing systems and institute changes such as to reduce a host vehicle driving speed, apply brakes, provide audio and visual warning signals and the like.

However, GPS systems may have a positional error and cannot, on their own, accurately map surrounding vehicles as the locations of the lanes of the road are unknown to the GPS system. Therefore, there is a need in the art for a system and method for accurately generating and communicating lane information over V2V networks.

SUMMARY

In one aspect of the present invention, a method of generating and communicating lane information from a host vehicle to a vehicle-to-vehicle (V2V) network includes collecting visual data from a camera. The method further includes detecting a lane within the visual data. The method further includes generating a lane classification for the lane based on the visual data. The method further includes assigning a confidence level to the lane classification. The method further includes generating a lane distance estimate from the visual data. The method further includes generating a lane model from the lane classification and the lane distance estimate and transmitting the lane model and the confidence level to the V2V network.

In another aspect of the present invention, the camera of the method includes a front camera mounted to a front-facing surface of the host vehicle.

In yet another aspect of the present invention, detecting a plurality of lanes further includes determining a position, a width, a curvature, a topography, a distance of each of the plurality of lanes relative to a reference position on the host vehicle, and a color and a shape of a plurality of lane markers for the plurality of lanes.

In yet another aspect of the present invention, generating a lane classification further includes comparing the color and the shape of the plurality of lane markers to a library of colors and shapes of known lane markers.

In yet another aspect of the present invention, generating a lane distance estimate further includes mathematically interpolating from the visual data the distance from a lane edge relative to a reference position on the host vehicle.

In yet another aspect of the present invention, the V2V network includes at least one remote V2V equipped vehicle.

In yet another aspect of the present invention, the method includes scanning a predetermined area for remote V2V equipped vehicles within a predefined range of the host vehicle.

In yet another aspect of the present invention, transmitting the lane model and confidence level further includes periodically transmitting the lane model and confidence level over the V2V network.

In yet another aspect of the present invention, transmitting the lane model and confidence level further includes transmitting the lane model and confidence level immediately upon determining that the remote V2V equipped vehicle is within the predefined range of the host vehicle.

In yet another aspect of the present invention, a method of generating and communicating lane information from a host to data vehicle-to-vehicle (V2V) network includes optically scanning a predefined area of road surface surrounding the host vehicle. The method further includes tracking a plurality of lanes. The method also includes detecting remote V2V equipped vehicles. The method also includes encoding information about the plurality of lanes into a mathematical lane model and communicating the mathematical model over the V2V network.

In yet another aspect of the present invention, optically scanning further includes collecting optical data from a plurality of cameras mounted to the host vehicle.

In yet another aspect of the present invention, tracking a plurality of lanes further includes determining a position, a width, a curvature, a topography, a distance of each of the plurality of lanes relative to a reference position on the host vehicle, and a color and a shape of a plurality of lane markers for the plurality of lanes.

In yet another aspect of the present invention, tracking a plurality of lanes further includes comparing the color and the shape of the plurality of lane markers to a library of colors and shapes of known lane markers.

In yet another aspect of the present invention, detecting remote V2V equipped vehicles further includes transmitting V2V data packets and receiving V2V data packets sent by remote V2V equipped vehicles over the V2V network.

In yet another aspect of the present invention, communicating the mathematical lane model further includes encoding the mathematical lane model to create an encoded mathematical lane model that conforms to a communications protocol and transmitting the encoded mathematical lane model over the V2V network.

In yet another aspect of the present invention, a system for generating and communicating lane information from a host vehicle to a vehicle-to-vehicle (V2V) network includes a camera. The system further includes a V2V sub-system having a receiver and a transmitter. The system further includes a controller in communication with the camera and the V2V sub-system, the controller having memory for storing control logic and a processor configured to execute the control logic. The control logic further includes a first control logic for collecting visual data from the camera. The control logic further includes a second control logic for detecting a lane within the visual data. The control logic further includes a third control logic for generating a lane classification for the lanes based on the visual data. The control logic further includes a fourth control logic for assigning a base confidence level to the lane classification. The control logic further includes a fifth control logic for generating a lane distance estimate from the visual data. The control logic further includes a sixth control logic for generating a base lane model from the lane classification and the lane distance estimate. The control logic further includes a seventh control logic for generating a formatted lane model and a formatted confidence level. The control logic further includes an eighth control logic for selectively transmitting the formatted lane model and the confidence level to the V2V network.

In yet another embodiment of the present invention, the camera includes a plurality of cameras attached to the host vehicle.

In yet another embodiment of the present invention, the base and formatted lane models include lane positioning, lane markings, lane curvature, speed, and trajectory data for the host vehicle.

In yet another embodiment of the present invention, the seventh control logic further includes aligning the base lane model and base confidence level to a standardized communications protocol.

In yet another embodiment of the present invention, the selectively transmitting further includes periodically transmitting the formatted lane model and formatted confidence level over the V2V communications network and when a V2V equipped vehicle appears within a predefined range of the host vehicle, automatically transmitting the formatted lane model and formatted confidence level over the V2V communications network.

Further aspects, examples, and advantages will become apparent by reference to the following description and appended drawings wherein like reference numbers refer to the same component, element or feature.

DETAILED DESCRIPTION

With reference toFIGS. 1 and 2, a system and method for generating and communicating camera and lane position information is generally indicated by reference to lane position system10. The system10is used with a host vehicle12having a vision sub-system14and a vehicle-to-vehicle (V2V) communication sub-system16. The vision sub-system14and the V2V communication sub-system16are in communication with a controller18.

The vision sub-system14includes one or more optical sensors or cameras22. The camera22is operable to collect visual information in a predefined field of view24surrounding the host vehicle12. In the example provided, the camera22is illustrated as a front facing camera with a field of view24projected in a forward arc relative to the host vehicle12. However, it should be appreciated that the vision sub-system14may include a plurality of cameras, including surround-view cameras, rear facing cameras, etc. Visual data from the camera22is communicated to the controller18.

The V2V sub-system16includes a transmitter20operable to transmit wireless data from the V2V sub-system16of the host vehicle12. The V2V sub-system16may also include a receiver21operable to receive wireless data sent by remote V2V equipped vehicles over the V2V communications network or by vehicle-to-infrastructure systems. As will be described below, the V2V data transmitted by the transmitter20may include GPS data, camera data, and/or object lists.

The controller18is a non-generalized, electronic control device having a preprogrammed digital computer or processor28, memory or non-transitory computer readable medium30used to store data such as control logic, instructions, image data, lookup tables, etc., and a plurality of input/output peripherals or ports32. The processor28is configured to execute the control logic or instructions. The controller18may have additional processors or additional integrated circuits in communication with the processor28, such as perception logic circuits for analyzing the visual data or dedicated V2V circuits. Alternatively, the functions of the controller18may be distributed across the vision sub-system14and/or the V2V sub-system16.

Turning now toFIG. 3, and with continued reference toFIGS. 1 and 2, a method for generating and communicating camera and lane position information is generally indicated by reference number100. For illustrative purposes, the method100will be described with the host vehicle12operating on an exemplary road segment34, shown inFIG. 2. The road segment34has lanes L1, L2, L3, to Ln. It should be appreciated that the road segment34may have as few as one lane without departing from the scope of the present disclosure. The lanes L1to Lnare defined by lane markings36. The lane markings36may be reflective paint, reflectors, traffic cones or barrels, grooves, etc. Additionally, the lane markings16may be solid lines, dashed lines, dashed and solid lines, or any other type of lane marking36. The road segment34is illustrated as being partially curved but may have any shape and have any topography without departing from the scope of the present disclosure.

In the present example, the road segment34is populated with two remote V2V equipped vehicles26, and one non-communicative vehicle40. It should be appreciated that the road segment34may be populated by any number and combination of remote V2V equipped vehicles26and non-communicative vehicles40. The non-communicative vehicles40may be vehicles without V2V systems or may be remote V2V equipped vehicles26that are disposed outside a communication range of the host vehicle12.

The method100begins at block102where the camera22continuously captures visual data of the road segment34and sends the visual data to the controller18. The visual data may be in a forward arc or a surround view relative to the host vehicle12, depending on the number and type of cameras22mounted on the host vehicle12. In the present example, the visual data includes the lane markings36for the portion of the road segment34within the field of view24of the camera22. At block104the controller18processes the visual data for any possible lane markings36identifiable within the visual data. In one aspect, to detect the presence of lane markings36within the visual data, the controller18compares an optical intensity profile of the visual data to a library of known optical intensity profiles for known lane markings36. The optical intensity profiles may include information about lane marking width, periodicity, direction relative to the host vehicle12, color, curvature, etc. Additionally, the library includes reference information corresponding to road markings that are not lane markings36. In one aspect, the reference information includes optical intensity profiles corresponding to pedestrian crosswalks, parking space markings, roadwork markings, etc.

At block106, the controller18continuously generates a base lane classification from the visual data processed at block104. The base lane classification is based on the comparison of any lane markings36identified within the visual data to the data in the library of known lane markings36. In an example, the base lane classification indicates that a lane marking36corresponding to one or more of the lanes L1to Ln, is a dashed line, a dashed and solid line, a solid line, or a double solid line, etc. The base lane classification further includes information about lane width, lane density, road curvature, and lane marking36color.

A base lane tracking list is generated at block108from the visual data processed at block104. The base lane tracking list includes a count of the lanes detected within the visual data by the controller18. In the example ofFIG. 2, the base tracking list includes lanes L1to Ln, wherein n=4, indicating that four lanes have been detected, though it should be understood that the base tracking list may include any lanes L1to Lnwithin the field of view24from the camera22. Thus, the base tracking list may include all of the lanes L1to Lnin the road segment34, or only a portion of the lanes L1to Lnin a road segment.

At block110, a base lane classification confidence level is generated. To generate the base lane classification confidence level, the controller18determines a level of similarity between the lane markings36detected within the visual data to the reference lane markings36within the library. For lane markings36with a high degree of similarity to the reference lane markings36within the library, a high base confidence level is assigned. For lane markings36with a low degree of similarity to the reference lane markings36, a low base confidence level is assigned. It should be appreciated that the base confidence level may be based on a continuum. For example, a solid line lane marking36within the visual data that has been heavily damaged by erosion, or that has been partially covered by skid-marks from tires may approximate the periodicity of the dashed paint of a dashed lane marking36. In this example, the base lane classification may be assigned a low base lane confidence level. However, with further reference to the example, because the lane markings36are continuously captured by the visual data from the camera22, as the host vehicle12travels along the road segment34, the damaged lane marking36may exhibit less damage at some points along its length than at other points. For the less damaged sections of the lane marking36of the example, the controller18may assign a high base lane confidence level, indicating a high probability that the lane marking36is a solid line.

While the processes of blocks106,108, and110are discussed as occurring in a particular sequence, it should be appreciated that any of the processes described as occurring within blocks106,108, and110may be performed independently of one another, and in any order.

At block112, the controller18uses the base lane classifications, base lane tracking list, and/or base lane classification confidence levels to estimate a distance of each of the lanes L to Lnfrom a reference position44on the host vehicle12. In the example provided, the reference position44is an edge of a tire, though it should be appreciated that any reference position may be used. To determine the estimated distance of each of the lanes L1to Lnfrom the reference position44on the host vehicle12, the controller extrapolates an extent of the lane markings36from the visual data. That is, because the visual data from the cameras22is limited to the predefined area24surrounding the host vehicle12, the road markings36extend beyond a field of view of the cameras22. Thus, in order to accurately determine a position of the road markings36, the controller18extrapolates from the position of the host vehicle12on the road segment34, and from the visual data, a predicted position of the road markings36. In one aspect, in addition to using the visual data, and the base lane classifications, base lane tracking list, and base lane confidence levels, the controller18compiles the position of the host vehicle12, an angular position of a steering wheel of the host vehicle12, a speed of the host vehicle12, etc. to extrapolate the predicted position of the road markings36relative to the reference position44on the host vehicle. At block114, the controller18combines the base lane classifications, base lane tracking list, and base lane confidence levels into a lane model. The lane model is a numerical formulation of the lanes detectable by the host vehicle12and the relationship of the host vehicle12to the lanes.

At block116the controller18retrieves V2V data from receiver21of the V2V sub-system16. The V2V data includes information about remote V2V equipped vehicles26within a predetermined range. For example, the remote V2V equipped vehicles26may be within a predetermined one kilometer radius of the host vehicle12, whereas the non-communicative vehicle40may be outside the predetermined one-kilometer radius of the host vehicle. Additionally, the controller18selectively chooses which, if any, of the remote V2V equipped vehicles26within the predetermined range of the host vehicle12to designate as target vehicles46. In an aspect, the target vehicles46of the remote V2V equipped vehicles26are those remote V2V equipped vehicles26for which the visual data collected by the host vehicle12is relevant. For example, the target vehicles46may be traveling along the same road segment34as the host vehicle12, in the same direction, and/or the target vehicles46may be traveling along a course and heading that intersects with the course and heading of the host vehicle12. It should be understood that while inFIG. 2, two target vehicles46are depicted along the road segment34, there could be any number of target vehicles46, including zero target vehicles46.

At block118the lane model data is aligned to the standardized V2V communications protocol. The standardized V2V communications protocol includes limitations on the size of data packets transmitted over the V2V communications network, as well as limitations on the types of data and the frequency with which the data may be transmitted. Each of the data packet size, data type and frequency with which the data is transmitted is limited to prevent the V2V communications network from becoming overloaded or otherwise inoperative.

At block120, the controller18determines for which of any target vehicles46detected the lane model is relevant. To determine for which of the target vehicles46the lane model is relevant, the controller18analyzes the lane model with respect to locational information retrieved by the V2V sub-system16about each of the target vehicles46. The locational information may include global positioning system (GPS) location information, heading information, speed information, etc. pertaining to each of the target vehicles46. Additionally, the locational information may include lane position information including lane density, lane width, road curvature, lane marking color, and other information pertaining to the lanes in which the target vehicles46are operating. If the controller18determines that no target vehicles46have been identified, the method proceeds to block122where the controller18commands the transmitter20of the V2V sub-system to periodically transmit a basic safety message (BSM) and the lane model to the V2V communications network. The BSM includes information such as GPS location, heading, speed, etc. for the host vehicle12. The periodicity with which the BSM and lane model are transmitted is defined by a standardized V2V communications protocol wherein the frequency with which the BSM and lane model is transmitted is limited to prevent the V2V communications network from becoming overloaded or otherwise inoperative. For example, the frequency with which the BSM and lane model are transmitted may be once per second.

However, if target vehicles46are identified by the controller18, the method proceeds to step124where the controller18commands the transmitter20to transmit the BSM and the lane model to the target vehicles46immediately as they are identified. That is, while the system10transmits the BSM and the lane model at a predetermined frequency, when a target vehicle46is first identified, or when a request for information is received from the target vehicle46, the BSM and the lane model are transmitted immediately to the target vehicle46as well.

By generating and transmitting lane information that is detected by imaging sensors, the system10allows for the accurate mapping of surrounding vehicles within lanes. The lane information may then be used by advanced driver assistance systems to provide increased levels of autonomous driving.