Vision sensing compensation

A lane-control system suitable for use on an automated vehicle comprising a camera, a lidar-sensor, and a controller. The camera captures an image of a roadway traveled by a host-vehicle. The lidar-sensor detects a discontinuity in the roadway. The controller is in communication with the camera and the lidar-sensor and defines an area-of-interest within the image, constructs a road-model of the roadway based on the area-of-interest, determines that the host-vehicle is approaching the discontinuity, and adjusts the area-of-interest within the image based on the discontinuity.

TECHNICAL FIELD OF INVENTION

This disclosure generally relates to a lane-control system suitable for use on an automated vehicle, and more particularly relates to a system that compensates the vision sensing.

BACKGROUND OF INVENTION

It is known to apply lane-keep assist and/or lane-centering methods to vehicles traveling roadways. These methods rely on the continuous feed of information from a vision system mounted on the vehicle. The loss of vision system data may cause the various lane-keep-assist and lane-centering methods perform sub-optimally, or fail to perform their intended function.

SUMMARY OF THE INVENTION

In accordance with one embodiment, a lane-control system suitable for use on an automated vehicle is provided. The lane-control system includes a camera, a lidar-sensor, and a controller. The camera is used to capture an image of a roadway traveled by a host-vehicle. The lidar-sensor is used to detect a discontinuity in the roadway. The controller is in communication with the camera and the lidar-sensor and defines an area-of-interest within the image, constructs a road-model of the roadway based on the area-of-interest, determines that the host-vehicle is approaching the discontinuity, and adjusts the area-of-interest within the image based on the discontinuity.

DETAILED DESCRIPTION

Roadways traveled by a host-vehicle12are seldom flat and smooth, and often contain irregularities such as pot holes, debris, and discontinuities from bridge overpasses and on-ramps that may have transient influences on the suspension and/or trajectory of the host-vehicle12. Described herein is a lane-control system10that anticipates such a transient-event and maintains control of the host-vehicle12through the transient-event when the vision-system data may otherwise become disrupted.

FIG. 1illustrates a non-limiting example of a lane-control system10, hereafter referred to as the system10, which is suitable for use on an automated vehicle, for example a host-vehicle12. As used herein, the term ‘automated vehicle’ is not meant to suggest that fully automated or autonomous operation of the host-vehicle12is required. It is contemplated that the teachings presented herein are applicable to instances where the host-vehicle12is entirely manually operated by a human and the automation is merely providing a lane-keep-assist (LKA) or a lane-centering (LC) to the human, and possibly operating the brakes of the host-vehicle12to prevent the host-vehicle12from entering a travel-path of an approaching vehicle.

The system10includes a camera14used to capture an image16of a roadway18traveled by the host-vehicle12. Examples of the camera14suitable for use on the host-vehicle12are commercially available as will be recognized by those in the art, one such being the APTINA MT9V023 from Micron Technology, Inc. of Boise, Id., USA. The camera14may be mounted on the front of the host-vehicle12, or mounted in the interior of the host-vehicle12at a location suitable for the camera14to view the area around the host-vehicle12through the windshield of the host-vehicle12. The camera14is preferably a video type camera14or camera14that can capture images16of the roadway18and surrounding area at a sufficient frame-rate, of ten frames per second, for example.

The system10also includes a lidar-sensor20that may detect a discontinuity22in the roadway18. The discontinuity22may be of any magnitude measurable by the lidar-sensor20(a few millimeters to several hundred millimeters for example) and may include, but is not limited to, a pothole, road debris, dip, peak, or a pavement transition, for example. The discontinuity22may span the entire width of the roadway18, or may span only a portion of the roadway18. The lidar-sensor20may detect the discontinuity22at a range in excess of two-hundred meters (200 m), based on the reflectivity of the discontinuity22and an unencumbered line-of-sight between the discontinuity22and the lidar-sensor20.

The system10also includes a controller24in communication with the camera14and the lidar-sensor20. The controller24may include a processor (not specifically shown) such as a microprocessor or other control circuitry such as analog and/or digital control circuitry including an application specific integrated circuit (ASIC) for processing data as should be evident to those in the art. The controller24may include a memory (not specifically shown), including non-volatile memory, such as electrically erasable programmable read-only memory (EEPROM) for storing one or more routines, thresholds, and captured data. The one or more routines may be executed by the processor to perform steps for determining if a detected instance of the discontinuity22in the roadway18is going to be in the intended path of the host-vehicle12based on signals received by the controller24from the lidar-sensor20and camera14as described herein.

The controller24may define an area-of-interest26(FIG. 2) within the image16to identify features of the roadway18including, but not limited to, lane-markings28, road-edges30, vanishing points, and other features characteristic of the roadway18that may be used for LKA and/or LC, and will be understood by one skilled in the art.FIG. 2. illustrates the area-of-interest26within the image16as captured by the camera14. A bottom-edge of the area-of-interest26may be at any position in front of the host-vehicle12, and preferably is located at a position that is 1.5 seconds ahead of the host-vehicle12, regardless of a velocity (not shown) of the host-vehicle12.

The controller24is further configured to construct a road-model32of the roadway18based on the features detected in the area-of-interest26, as shown inFIG. 2. The road-model32may be updated with new information from the camera14and the lidar-sensor20as the host-vehicle12moves along the roadway18. The road-model32may be updated at rate equal to the frame-rate of the camera14, or may be updated at a slower rate to satisfy any computational limitations. Kalman filters may be used to track the lane-edges, for example, and will be recognized by those skilled in the art. The controller24may also assume some or all control of the vehicle-controls34(FIG. 1) of the host-vehicle12based on the road-model32as will be understood by one skilled in the art of autonomous controls.

The controller24is further configured to determine that the host-vehicle12is approaching the discontinuity22in anticipation of the aforementioned transient-event. Once the discontinuity22is detected by the lidar-sensor20, the controller24may track the discontinuity22and determine a time-of-arrival to the discontinuity22based on the velocity of the host-vehicle12and based on the distance (not specifically shown) to the discontinuity22. The time-of arrival may be updated at a rate equal to a clock-speed (not shown) of the controller24, or at a slower rate to satisfy any computational limitations, or may be varied based on the velocity of the host-vehicle12. The time-of-arrival may be stored in the memory of the controller24and may be associated with a track of the discontinuity22.

The controller24is further configured to adjust the area-of-interest26within the image16, in anticipation of the host-vehicle12arriving at the discontinuity22, and based on the discontinuity22to create an adjusted-area-of-interest36(FIG. 3). The adjusted-area-of-interest36may be used to compensate for any missing data and/or induced-noise in the image16that occurs, such as an instantaneous change in the position of the lane-markings28in the image16, while the host-vehicle12is traveling over the discontinuity22thus creating a disruption in a field-of-view of the camera14as the host-vehicle12reacts to the discontinuity22. This disruption in the field-of-view of the camera14may lead to sub-optimal steering control where the steering commands from the controller24fluctuate.FIG. 3illustrates the adjusted-area-of-interest36compared to a transient-area-of-interest38, where the transient-area-of-interest38illustrates a view of the camera14at a moment in time after the host-vehicle12has reached the discontinuity22, and the host-vehicle12is reacting to (e.g. pitching, jouncing, rolling, etc.) the discontinuity22by pointing the camera14in an upward and leftward direction, for example. In the non-limiting example illustrated inFIG. 3, as the host-vehicle12reaches the discontinuity22the adjusted-area-of-interest36is shifted rightward and downward relative to the transient-area-of-interest38to align with a previous-area-of-interest40to compensate for an impending-reaction of the host-vehicle12to the discontinuity22. The previous-area-of-interest40is defined as the last area-of-interest26used to update the road-model32before the host-vehicle12reaches the discontinuity22. As the host-vehicle12passes over the discontinuity22, the road-model32may then be updated with information from the adjusted-area-of-interest36, eliminating any disruption in updating the road-model32that may have occurred if the transient-area-of-interest38were relied upon, thereby enabling the controller24to steer the host-vehicle12as if there were no discontinuity22. The duration of the transient-event may last for several seconds depending on the magnitude of the discontinuity22, during which the road-model32is updated according to the adjusted-area-of-interest36until a transient-response of the host-vehicle12falls below a user defined threshold and a stable host-vehicle12reaction to the roadway18is re-established.

Using the adjusted-area-of-interest36to update the road-model32is beneficial because it requires less computational resources than some prior-art-systems that may track a reference-object in the image16, so that the area-of-interest26can be adjusted in response to a movement of the reference-object. This tracking of the reference-object typically requires a significant amount of computational resources when the discontinuity22is sufficiently large such that the controller24is required to search the entire image16in order to re-locate the reference-object. In contrast, the system10described herein anticipates how the area-of-interest26will move in a future image16as a result of the discontinuity22, which reduces the image-processing burden on the controller24because the entire image16need not be searched for the reference-object.

The controller24may use a transformation, such as an Affine-transformation and/or a Perspective-transformation to create the adjusted-area-of-interest36where the image16is rotated to account for an angle change in the field-of-view of the camera14and determine the real-world coordinates of the lane-markings28. The controller24may determine the type of transformation required based on the discontinuity22. That is, if the discontinuity22is detected to be a sudden vertical drop in the roadway18the controller24may use the Affine-transformation, for example. It the discontinuity22is detected to be a ramp, the controller24may use the Perspective-transformation, for example.

The controller24may use a dynamic-model42(FIG. 1) of the host-vehicle12to anticipate the reaction of the host-vehicle12to the discontinuity22. The dynamic-model42estimates a dynamic-response of the host-vehicle12to various inputs, including, but not limited to, a suspension-input, a steering-input, a velocity-input, a wheel-speed input, and a cargo-load-input. The dynamic-model42may also include components such as aerodynamic, geometric, mass, motion, tire, and roadway18specific components that may describe the motion of the host-vehicle12under a variety of conditions, and will be understood by one skilled in the art.

Accordingly, a lane-control system10, a camera14, a lidar-sensor20, and a controller24for the lane-control system10is provided. The lane-control system10is an improvement over other lane control systems because it anticipates discontinuities22in the roadway18and compensates for erroneous vision-system inputs.

While this invention has been described in terms of the preferred embodiments thereof, it is not intended to be so limited, but rather only to the extent set forth in the claims that follow. Moreover, the use of the terms first, second, upper, lower, etc. does not denote any order of importance, location, or orientation, but rather the terms first, second, etc. are used to distinguish one element from another. Furthermore, the use of the terms a, an, etc. do not denote a limitation of quantity, but rather denote the presence of at least one of the referenced items.