Patent Description:
It is known to equip an automated vehicle to detect an object in the travel-path of the automated vehicle. Normally, the automated-vehicle will take various actions to avoid running over most objects. However, in some circumstances such as during high-speed travel on a crowded roadway, it may be preferable to run-over an object such as tumbleweed rather than perform an abrupt braking and/or lane-change maneuver to avoid hitting the tumbleweed.

In accordance with one embodiment, an object classification system for an automated vehicle is provided. The system includes a lidar and a controller. The lidar is mounted on a host-vehicle. The lidar determines spot-distances indicated by light-beams that were emitted by the lidar and reflected toward the lidar from an area proximate to the host-vehicle. The controller is in communication with the lidar. The controller determines a lidar-outline of an object in the area based on spot-distances, determines an object-distance to the object based on spot-distances within the lidar-outline of the object, determines a backdrop-distance to a backdrop based on spot-distances outside of the lidar-outline of the object, determines a transparency-characteristic of the object based on instances of spot-distances from within the lidar-outline of the object that correspond to the backdrop-distance, and operates the host-vehicle to avoid the object when the transparency-characteristic is less than a transparency-threshold.

In another embodiment, an object classification system for an automated vehicle is provided. The system includes a camera and a controller. The camera is mounted on a host-vehicle. The camera renders an image of an area proximate to the host-vehicle. The image is based on light detected by a plurality of pixels in the camera, where each pixel detects a pixel-color of light from the area. The controller is in communication with the camera. The controller determines a camera-outline of an object based on the image, determines a backdrop-color of a backdrop outside of the camera-outline of the object, determines a transparency-characteristic of the object based on instances of pixel-color within the camera-outline that correspond to the backdrop-color, and operates the host-vehicle to avoid the object when the transparency-characteristic is less than a transparency-threshold.

In another embodiment, an object classification system for an automated vehicle is provided. The system includes a lidar, a camera, and a controller. The controller is in communication with the lidar and the camera. The controller determines a transparency-characteristic of an object using a combination of the aforementioned steps with regard to the lidar and the camera.

Further features and advantages will appear more clearly on a reading of the following detailed description of the preferred embodiment, which is given by way of non-limiting example only and with reference to the accompanying drawings.

<FIG> illustrates a non-limiting example of an object classification system <NUM>, hereafter referred to as the system <NUM>, which is suitable for use on an automated vehicle, e.g. a host-vehicle <NUM>. As used herein, the term automated vehicle may apply to instances when the host-vehicle <NUM> is being operated in an automated-mode <NUM>, i.e. a fully autonomous mode, where a human-operator (not shown) of the host-vehicle <NUM> may do little more than designate a destination in order to operate the host-vehicle <NUM>. However, full automation is not a requirement. It is contemplated that the teachings presented herein are useful when the host-vehicle <NUM> is operated in a manual-mode <NUM> where the degree or level of automation may be little more than providing an audible or visual warning to the human-operator who is generally in control of the steering, accelerator, and brakes of the host-vehicle <NUM>. For example, the system <NUM> may merely assist the human-operator as needed to change lanes and/or brake to avoid a collision with an object <NUM>.

The system <NUM> includes an object-detector <NUM> that may include a lidar <NUM> and/or a camera <NUM>, where either or both are preferably mounted on the host-vehicle <NUM>. While <FIG> may be interpreted to mean that the lidar <NUM> and the camera <NUM> are part of a unified assembly, this is not a requirement. That is, the lidar <NUM> and the camera <NUM> may be mounted at distinct spaced-apart locations on the host-vehicle <NUM>. It is also contemplated that the host-vehicle <NUM> may be equipped with multiple instances of the lidar <NUM> and/or the camera <NUM>. That is, the fact that the following discussion only considers that the host-vehicle <NUM> is equipped with one instance of the lidar <NUM> and/or one instance of the camera <NUM> does not restrict the system <NUM> from being equipped with multiple instances of either device.

<FIG> illustrates a non-limiting example of a scenario <NUM> that the host-vehicle <NUM> may encounter. The lidar <NUM> determines or is used to determine instances of spot-distances <NUM> to spots <NUM> illuminated by the light-beams in an area <NUM> proximate to, i.e. forward of and within one-hundred meters (<NUM>) of the host-vehicle <NUM>. That is, the spot-distances <NUM> are indicated by light-beams <NUM>, which are typically infrared-light and were emitted by a laser-source (not shown) the lidar <NUM> and reflected toward the lidar <NUM> from the area <NUM> proximate to the host-vehicle <NUM>. By way of example and not limitation, the object <NUM> illustrated in <FIG> is a tumbleweed, which is a structural part of the above-ground anatomy of a number of species of plants that, once it is mature and dry, detaches from its root or stem, and tumbles away in the wind. As shown, the light-beams <NUM> may illuminate a spot 32A on the object <NUM>, or may pass through the object <NUM> and illuminate a spot 32B on a roadway <NUM> behind (relative to the host-vehicle <NUM>) the object <NUM>.

The system <NUM> may include a controller <NUM> (<FIG>) in communication with the lidar <NUM> and/or the camera <NUM>, whichever is provided. The controller <NUM> may include a processor (not specifically shown) such as a microprocessor or other control circuitry such as analog and/or digital control circuitry including an application specific integrated circuit (ASIC) for processing data as should be evident to those in the art. The controller <NUM> may include memory (not specifically shown), including non-volatile memory, such as electrically erasable programmable read-only memory (EEPROM) for storing one or more routines, thresholds, and captured data. The one or more routines may be executed by the processor to perform steps for classifying the object <NUM> based on signals received by the controller <NUM> from the lidar <NUM> and/or the camera <NUM> as described herein.

In an embodiment of the system <NUM> that includes the lidar <NUM>, the controller <NUM> determines (i.e. the controller <NUM> is configured to or programed to determine) a lidar-outline <NUM> of the object <NUM> in the area <NUM> based on the spot-distances <NUM>. Those familiar with the operation of lidars will recognize that the object <NUM> will be illuminated with many more instances of the light-beams <NUM> than the two illustrated in <FIG>. That <FIG> shows only two instances of the light-beams <NUM> is done only to simplify the illustration. The spot-distances <NUM> that are similar will be grouped together so that the controller <NUM> is able to determine the lidar-outline <NUM> of the object <NUM>. While <FIG> may be interpreted as suggesting that the lidar <NUM> is a three-dimensional lidar that indicates both an azimuth-angle (side-to-side or left/right) and an elevation-angle (up/down) for each of the spots <NUM>, this is not a requirement. The teachings presented herein are applicable to systems that use two-dimensional lidars that only indicate an azimuth-angle of the spots <NUM>. Those in the art will recognize that the lidar-outline <NUM> for a two-dimensional lidar may merely be a left and right boundary rather than a line that encloses the object <NUM> as shown in <FIG>.

The controller <NUM> then determines an object-distance <NUM> to the object <NUM> from the host-vehicle <NUM> based on spot-distances <NUM> within the lidar-outline <NUM> of the object <NUM>. If some of the light-beams <NUM> pass completely through the object <NUM>, as could be the case when the object <NUM> is a tumbleweed, and illuminate a backdrop <NUM> behind the object <NUM> (relative to the host-vehicle <NUM>), the spot-distances <NUM> to those spots, e.g. the spot 32B, will be distinguishable from the spots <NUM> on the object <NUM>, e.g. the spot 32A. In other words, the spot-distances <NUM> from within the lidar-outline <NUM> will be noisy, i.e. be highly variable, because some of the light-beams <NUM> passed through the object <NUM> so are not reflected by the object <NUM>. If the lidar <NUM> is mounted on the host-vehicle <NUM> relatively close to the surface of the roadway <NUM> so the light-beams <NUM> are substantially parallel to the surface of the roadway <NUM>, the spot 32B may be much more far-removed from the object <NUM> than is suggested by <FIG>. Indeed, the spot 32B may be so far-removed that no reflection of the light-beam is detected when the light-beam passes through the object <NUM>. That is, the backdrop <NUM> may be the sky or a horizon that is beyond the detection range of the lidar <NUM>.

The controller <NUM> then determines a backdrop-distance <NUM> to the backdrop <NUM> based on the spot-distances <NUM> to spots <NUM> outside of the lidar-outline <NUM> of the object <NUM>, e.g. the distance to spot 32C. If the roadway <NUM> curves upward relative to the host-vehicle <NUM>, the backdrop-distance <NUM> may correspond to the distance to the spot 32C on the roadway <NUM>. However, if the roadway <NUM> curves downward, or is level and the lidar <NUM> is located close to surface of the roadway <NUM>, the backdrop-distance <NUM> may be infinity because the backdrop <NUM> is the sky.

The controller <NUM> then determines a transparency-characteristic <NUM> of the object based on instances of the spot-distances <NUM> from within the lidar-outline <NUM> of the object <NUM> that correspond to the backdrop-distance <NUM>. In other words, the transparency-characteristic <NUM> is an indication of how many or what percentage of the light-beams <NUM> that are directed toward the object <NUM> (i.e. inside the lidar-outline <NUM>) end up passing through the object <NUM> and thereby indicate the backdrop-distance <NUM> rather than the a distance comparable to the object-distance <NUM>.

The controller <NUM> may then operate the host-vehicle <NUM> to avoid the object <NUM> when the transparency-characteristic <NUM> is less than a transparency-threshold <NUM>. The transparency-threshold may be fifty-five percent (<NUM>%) for example, however it is contemplated that empirical testing may be needed for various configurations of the lidar <NUM> and testing of various example of the object <NUM> that can be run-over by the host-vehicle <NUM> is necessary. If the transparency-characteristic <NUM> is greater than the transparency-threshold <NUM>, then it is presumed that the object <NUM> is, for example, a tumbleweed or something that could be run-over by the host-vehicle <NUM> without causing excessive damage to the host-vehicle <NUM> if the actions necessary to avoid the object <NUM> are not preferable. For example, if the host-vehicle <NUM> is being followed at close range (e.g. less than <NUM>) by a following-vehicle (not shown), then sudden braking by the host-vehicle <NUM> may be ill-advised. Similarly, if there is an approaching-vehicle (not shown) traveling in the on-coming lane <NUM>, then it may be ill-advised for the host-vehicle <NUM> to swerve into the on-coming lane <NUM> to avoid running-over the object <NUM>.

<FIG> illustrates a non-limiting example of a scenario <NUM> that is similar to the scenario <NUM> of <FIG> and that the host-vehicle <NUM> may encounter. The camera renders an image <NUM> of the area <NUM> proximate to the host-vehicle <NUM>, e.g. forward of and in the line-of-site from the host-vehicle <NUM>. The image <NUM> is based on light detected by a plurality of pixels <NUM> in the camera <NUM>, where each pixel detects a pixel-color <NUM> of light from the area <NUM>, as will be recognized by those in the art. While the camera <NUM> could be a black and white type camera, a color camera is preferable as it makes it easier to distinguish parts of the object <NUM> from the backdrop <NUM>.

When the system <NUM> is equipped with the camera <NUM>, either with or without the lidar <NUM>, the controller <NUM> determines a camera-outline <NUM> of the object <NUM> based on the image <NUM>. For example, the portion of the pixels <NUM> that have an object-color <NUM> that is distinct from a backdrop-color <NUM> of the backdrop <NUM> outside of the camera-outline <NUM> of the object <NUM>. For example, if the object <NUM> is a tumbleweed, then the object-color <NUM> may be tan or light-brown. In contrast, if the back-drop is the roadway <NUM> then the backdrop-color may be dark-grey, e.g. the color of asphalt. In further contrast, if the backdrop <NUM> is the sky, then the backdrop-color may be blue or white or grey depending on the weather conditions.

The controller <NUM> then determines the transparency-characteristic <NUM> of the object <NUM> based on, for example, a percentage of instances of the pixel-color <NUM> within the camera-outline <NUM> that correspond to the backdrop-color <NUM>. In other words, if the object <NUM> is relatively transparent as is the case for a typical example of a tumbleweed, then the backdrop-color <NUM> would be detected in the image <NUM> inside of the camera-outline <NUM>. As described above, the controller <NUM> may operate the host-vehicle <NUM> to avoid the object <NUM> when the transparency-characteristic <NUM> is less than the transparency-threshold <NUM>.

If the system <NUM> is equipped with both the lidar <NUM> and the camera <NUM>, the decision to operate the host-vehicle <NUM> to avoid the object <NUM> may be based on either data from the lidar <NUM> or the camera <NUM> indicating that the transparency-characteristic <NUM> is less than the transparency-threshold <NUM>, or both the lidar <NUM> or the camera <NUM> indicating that the transparency-characteristic <NUM> is less than the transparency-threshold <NUM>. It is contemplated that empirical testing for various configurations of the lidar <NUM> and the camera <NUM> will yield which decision rules are preferable.

Accordingly, an object classification system (the system <NUM>), a controller <NUM> for the system <NUM>, and a method of operating the system <NUM> is provided. The transparency-characteristic <NUM> of the object <NUM> may be just one of several characteristics that could be considered in combination to determine if the object <NUM> can be, if necessary, run-over by the host-vehicle <NUM> without causing excessive damage to the host-vehicle <NUM>, where the necessity to do so may be determined by the presence of other vehicles proximate to the host-vehicle <NUM>.

Claim 1:
An object classification system (<NUM>) for an automated vehicle, said system (<NUM>) comprising:
a lidar (<NUM>) mounted on a host-vehicle (<NUM>), said lidar (<NUM>) determines spot-distances (<NUM>) indicated by light-beams (<NUM>) that were emitted by the lidar (<NUM>) and reflected toward the lidar (<NUM>) from an area (<NUM>) proximate to the host-vehicle (<NUM>);
a controller (<NUM>) in communication with the lidar (<NUM>), wherein the controller (<NUM>) is configured to
determine a lidar-outline (<NUM>) of an object (<NUM>) in the area (<NUM>) based on spot-distances (<NUM>),
determine an object-distance (<NUM>) to the object (<NUM>) based on spot-distances (<NUM>) within the lidar-outline (<NUM>) of the object (<NUM>),
determine a backdrop-distance (<NUM>) to a backdrop (<NUM>) based on spot-distances (<NUM>) outside of the lidar-outline (<NUM>) of the object (<NUM>),
determine a transparency-characteristic (<NUM>) of the object (<NUM>) based on instances of spot-distances (<NUM>) from within the lidar-outline (<NUM>) of the object (<NUM>) that correspond to the backdrop-distance (<NUM>) rather than a distance comparable to the object-distance (<NUM>), and
operate the host-vehicle (<NUM>) to avoid the object (<NUM>) when the transparency-characteristic (<NUM>) is less than a transparency-threshold (<NUM>).