Method and system for obstacle detection

A method and system for detecting the presence of an object or the distance between the system and an object supports the provision of rapid and reliable obstacle detection. A transmitter emits generally linear beams of electromagnetic radiation with a transmitted radiation pattern within a defined spatial zone. A camera collects an image of the defined spatial zone. A data processor detects a presence of an object in the collected image based on an observed illumination radiation pattern on an object formed by at least one of the generally linear beams. A distance estimator estimates a distance between the object and the optical device based on a change (e.g., an observed shift in one or more positions of generally linear beams) from the transmitted radiation pattern to the observed illumination radiation pattern.

FIELD OF THE INVENTION

This invention relates to a method and system for obstacle detection for a vehicle.

BACKGROUND OF THE INVENTION

In the prior art, scanning lasers may use mechanical components (e.g., a mirror or other movable member) to scan a beam over a field of view. The mechanical components may require the scanning laser to cycle through an entire range of beam orientations, rather than supporting random access to any particular orientation of the scanning beam. For obstacle detection applications of a vehicle, the response time for collecting image data should be rapid over a wide field of view to facilitate early recognition and avoidance of obstacles.

Although micro-electromechanical systems may be applied to improve reliability and improve the response time of the scanning lasers, the cost of micro-mechanical systems may not be appropriate for some applications. Further, because a micro-electromechanical system may generate multiple lines of light from a common source, the intensity of the light near the source may exceed a target level (e.g., a desired safety level). For the foregoing reasons, there is a need for an economical, safe, rapid and reliable method and system for obstacle detection.

SUMMARY OF THE INVENTION

A method and system for obstacle detection facilitates rapid and reliable detection of an object and/or estimation of a distance between the system and the object. A transmitter emits generally linear beams of electromagnetic radiation with a transmitted radiation pattern within a defined spatial zone. A camera collects one or more images of the defined spatial zone. A data processor detects a presence of an object in the collected image based on an observed illumination radiation pattern on an object formed by at least one of the generally linear beams. A distance estimator estimates a distance between the object and the camera based on a material change (e.g., an observed shift in one or more positions) of the generally linear beams from the transmitted radiation pattern to the observed illumination radiation pattern.

Like reference numbers in different drawings indicate like elements.

DESCRIPTION OF THE PREFERRED EMBODIMENT

In accordance with one embodiment of the invention,FIG. 1is a block diagram of an obstacle detection system11. One or more objects16or obstacles of a given size in a field of regard must be known with sufficient temporal resolution to enable safe passage of a vehicle at a given speed, acceleration, and heading. The obstacle detection system11comprises a transmitter50and a receiver34. The transmitter (e.g.,50) and the receiver (e.g.,34) may collectively be referred to as an optical device or optical unit herein. The transmitter50further comprises a group of transmitter modules30(e.g., a laser line emitters).

Each transmitter module30includes an electromagnetic energy source10(e.g., a laser) that provides electromagnetic energy (e.g., light, ultraviolet light or infrared radiation) to a line generator12. For example, the electromagnetic energy source10may provide a generated beam, a columnar beam or another input beam of electromagnetic energy to the line generator12. The line generator12accepts the input beam and outputs a generally linear segment or strip. An output of the line generator12is coupled to the splitter14. The splitter14splits or divides the generally linear segment or strip into two or more linear segments or strips that are generally parallel to one another or diverge from one another at known angles. In one illustrative example, an upper linear segment makes an upper angle with respect to a generally horizontal plane, and a lower segment makes a lower angle with respect to the generally horizontal plane. The upper angle may equal, but does not need to equal, the lower angle.

Each transmitter module30may be spaced apart from other transmitter modules30such that an aggregate radiation pattern is produced that consists of a series of generally parallel lines spaced vertically apart by known amounts from each other. Advantageously, the stacking of multiple transmit modules30along the vertical height of the vehicle allows the vehicle to detect obstacles of various heights in the path of the vehicle, including obstacles lying on the ground or near the ground plane, obstacles at a higher horizontal plane, and obstacles that are in an intermediate horizontal plane between the ground plane and the higher horizontal plane. The higher horizontal plane may be equal to or greater than a maximum height of the vehicle, for instance. An example of an obstacle lying in an intermediate horizontal plane might be a cable or wire (in the path of the vehicle) that is suspended in space between two poles or supports.

In one embodiment, the laser line emission points or their corresponding transmitter modules30are distributed vertically along the vehicle for several reasons: (1) the number of transmitted lines of a radiation pattern to cover the full field of regard is minimized, (2) eye safety issues are reduced by not having electromagnetic radiation (e.g., all light) emitted from a single point or concentrated source, (3) the system11may be more reliable since the failure of a single emission point or transmitter module30may still allow the vehicle to continue operating at a slower speed, while avoiding obstacles in a less robust manner (e.g., potentially missing the detection of obstacles associated with the failed emission points).

The transmitter50may pulse the entire aggregate radiation pattern or components (e.g., line segments or strips) of the radiation pattern to enhance eye safety and reduce power consumption. The transmitter50transmits the aggregate radiation pattern (or components thereof) toward an object16or obstacle. Although the aggregate radiation pattern may form a generally triangular shape in a horizontal plane, the radiation pattern may have any other shape in the horizontal plane as viewed from the top.

The receiver34receives a reflection from the object16or obstacle or captures an illumination radiation pattern incident on the object16or obstacle. The receiver34comprises a filter33that filters received electromagnetic radiation (e.g., the illumination radiation pattern) in a field of view on a frequency-selective basis. The filter33is optically coupled to a camera18. Although the filter33may comprise a bandpass filter33tuned to the same frequency as one or more transmitter modules30, the filter33may have a band-reject frequency response, a notch frequency response, or any other suitable filtering frequency versus magnitude (e.g., relative signal amplitude in decibels) response. The camera18may comprise a detector, such as a charge-coupled device (CCD), a complementary metal-oxide semiconductor (CMOS) image array, a shutter-controlled aperture, and memory for storing images. The camera18has a sufficient number of pixels (e.g., greater than or equal to 1 megapixels) to meet the minimum obstacle size observation requirement (if any)(e.g., greater than or equal to 0.1 meter) within the filed of regard. The camera18has a short enough image capture time (e.g., 20 Hz sampling rate for the images) to minimize blurring from motion. Further the camera18may provide one or more stored images to a data processor20.

The data processor20may comprise a distance estimator22for estimating a distance of the object16from the receiver34, which is mounted on a work vehicle. For example, the distance estimator22may comprise a pattern detector for detecting a pattern of the illumination radiation pattern or a shift in the observed illumination radiation pattern with respect to a reference illumination radiation pattern. The data processor20has sufficient processing capacity or throughput to process one or more images quickly enough so that obstacle detection system can gather enough images to meet the obstacle detection requirement within the field of regard. In one embodiment, the reference illumination radiation pattern represents an illumination radiation pattern that is present when the transmitter50with known positional coordinates (e.g., x, y and z coordinates) transmits the aggregate radiation pattern toward a wall or generally planar surface that is spaced apart from the transmitter50by a known distance and orientation. Because each generally linear radiation beam, each segment or strip has an upper angle and a lower angle with respect to a generally horizontal plane, the relative position of each line of the generally linear radiation pattern with respect to the reference radiation pattern reveals the location of an obstacle in front of the vehicle within a certain maximum range. As shown inFIG. 6A, which is considered in conjunction withFIG. 1here, the beams spread out from each transmission module30or emission point at a symmetric angle to the horizontal axis. The upper transmission angle (θ) and the lower transmission angle (θ) added together form a beam angle (e.g., 2θ). The beam angle and the horizontal plane fan-out width are related to the specified minimum obstacle size and the image capture rate of the receiver34(or camera18) such that in a first zone between dnear(40) and dfar(42), any obstacle greater in size than the specified minimum size will be observed breaking the laser light plane in one of the images captured by the receiver34.

The intensity of the transmitted lines (e.g. laser lines) are either safe for human vision or are operated in areas where people must wear optical filtering glasses or other eye protection to attenuate the frequency of the transmitted lines. The transmitter50transmits with a transmission power that is sufficient or powerful enough to generate a reflection from dark surfaces encountered up to a distance dfar(42) from the vehicle. The line width of the radiation pattern is adequate for the detection of obstacles breaking the line.

InFIG. 1, consistent with the environment around the vehicle, the transmitter50needs to produce an adequate number of laser lines with a specified thickness, intensity, horizontal fan-out, and vertical inclination or declination to ensure that an obstacle approaching the vehicle will perturb at least one of the laser lines for a sufficiently long interval to be observed by the camera18while the obstacle is in the field of regard. The receiver34and processor must have a sufficient number of pixels, a short enough image capture time, and a short enough image processing time to sample and process laser line perturbations in the field of regard in real time.

Unless a wall or ceiling is present, upwardly directed lines of the transmitted radiation pattern are likely to fall outside the field of view of the camera18. If the near, downward directed lines are not visible in the field of view of the camera18and if such downward directed lines are expected to be visible, a nonreflective or dark obstacle may be present or a negative obstacle may be present. A negative obstacle refers to a hole, a depression in the terrain, a staircase or a stairwell in a building, for example. A hole depression, or stairwell may not reflect light transmitted by the transmitter50to the same extent as flat ground or a generally planar floor, for example.

The obstacle detection system111ofFIG. 2is similar to the obstacle detection system11ofFIG. 1, except the obstacle detection system111ofFIG. 2comprises a transmitter150and a receiver134instead of transmitter50and receiver34, respectively. Like reference numbers inFIG. 1andFIG. 2indicate like elements.

The transmitter150ofFIG. 2comprises a clock52coupled to a controller54. In turn, the controller54is coupled to a group of transmitter modules30and the receiver134. The controller54may control the time and duration of transmissions from one or more transmitter modules30. For example, the controller54may stagger temporally the transmission of electromagnetic radiation from each transmitter module30to reduce the aggregate instantaneous output of the transmitter50and to enhance eye safety. In one embodiment, the controller54may cause the transmitter50to transmit pulses, rather than to continuously transmitting. Although the transmitter modules30may transmit electromagnetic radiation (e.g., laser light) continuously, a controller54, data processor, a logic circuit may be used to pulse the transmit modules.

The receiver134communicates with the controller54, the clock52or both of the transmitter150to determine a reception time window for receiving the transmitted electromagnetic signal of the transmitter150. As illustrated inFIG. 2, the transmitter150includes a band pass filter module56coupled between the camera18and the data processor20. The band pass filter module56may filter stored image data in the digital domain to be frequency selective of the transmitted radiation pattern. For example, if each transmitter module30transmits at a different frequency of electromagnetic radiation simultaneously or serially, the band pass filter module56may be dynamically changed or adjusted to isolate the contribution of each transmitter module30or emission source to the illumination radiation pattern.

In an alternate embodiment, the receiver134ofFIG. 2may be replaced by the receiver34ofFIG. 1. In another alternate embodiment, both the filter33and the band pass filter module56may be incorporated into the receiver134.

The obstacle detection system211ofFIG. 3is similar to the obstacle detection system111ofFIG. 2, except the obstacle detection system211ofFIG. 3further includes a pulse coordination module60and a wireless communications device58. The wireless communications device58is coupled to the pulse coordination module60. Like reference numbers inFIG. 2andFIG. 3indicate like elements.

The wireless communications device58is arranged to receive an electromagnetic status signal from other work vehicles or robots in a geographic area to coordinate obstacle detection. For example, assume that multiple work vehicles in a geographic area have an architecture similar to that ofFIG. 3for an obstacle detection system211. When the controller54seeks to command one or more transmitter modules30to transmit electromagnetic radiation (e.g., optical energy or laser light), the controller54may use the wireless communications device58to first listen or scan for the status signal (e.g., radio frequency signal or microwave signal) of other work vehicles to determine whether or not the transmitter50of another work vehicle is currently transmitting in the geographic area. If another transmitter50of another work vehicle is currently transmitting (e.g., optical energy or laser light) in the geographic area (e.g., within a optical communications range or up to several kilometers), the pulse coordination module60may delay transmission of (e.g., optical energy or laser light) to the other work vehicle to avoid interference or an aggregate level of transmission of electromagnetic energy that may otherwise exceed safe human eye levels, safety standards or regulations. However, if another transmitter50of another vehicle is not currently transmitting, the controller54is permitted to command the transmitter50to transmit and the pulse coordination module60sends a busy status or transmission-in-progress status signal to prevent other vehicles from interfering with the transmission or exceeding a target maximum eye safety level.

In one embodiment, when multiple work vehicles are operating near each other, erroneous observations may be eliminated by giving each vehicle a time slice in which it can pulse its lasers and capture an image. The time slice coordination may be accomplished over a wireless communications device58(e.g., in accordance with Bluetooth, 802.11b, or another communications standard or protocol).

In another embodiment, when vehicles are within communications range of each other, each vehicle's location and pose information is exchanged with the other vehicles via the wireless communication devices58. Each vehicle can negotiate a time slot of one or more time windows to use and synchronize its time slot. The link between a wireless communications device58may also be used to share mapping information, particularly if the vehicles are in an exploratory mode.

FIG. 4is a flow chart of a method for detecting an obstacle. The method ofFIG. 4begins in step S100.

In step S100, a transmitter (e.g.,50,150, or250) emits a plurality of generally linear beams of electromagnetic radiation with a transmitted radiation pattern within a defined spatial zone. The electromagnetic radiation may be transmitted over a certain frequency range, such as visible light, a portion of the visible light spectrum, infra-red light, a portion of the infra-red light spectrum, ultraviolet light, a portion of the ultra-violet light spectrum, or any combination of the foregoing frequency ranges. Step S100may be executed in accordance with various techniques, which may be applied cumulatively or individually. Under a first technique, the transmitter (50,150or250) transmits one or more pulses of the electromagnetic radiation. Under a second technique, the transmitter250coordinates the transmission time of pulses with other optical units in a common geographic zone to avoid interference. Under a third technique, the transmitter (50,150or250) directs the transmitted electromagnetic radiation within a defined spatial zone, which comprises a field of regard. The field of regard is defined by a three-dimensional spatial zone extending in a direction of travel of the vehicle and having a height and width that is greater than the vehicle height and the vehicle width by a minimum spatial clearance. Under a fourth technique, the transmitter (50,150or250) directs the transmitted electromagnetic radiation within a depth of a three-dimensional spatial zone. The three-dimensional spatial zone is defined by the at least one of the vehicular speed or velocity of the vehicle, minimum vehicle stopping distance plus a safety interval corresponding to the vehicular speed, maximum oncoming speed of oncoming objects, sampling rates of the collection of the image, and resolution of the image. Under a fifth technique, the transmitter (50,150or250) emits a first group of generally linear beams from a first transmitter module and emits a second group of generally linear beams from a second transmitter module spaced apart from the first transmitter module. The first transmitter module and the second transmitter module may be spaced apart by a vertical dimension where the generally linear beams are substantially horizontally oriented or spaced apart by a horizontal dimension where the generally linear beams are substantially vertically oriented. In general, the first transmitter module and the second transmitter module may be aligned in a generally linear array, which has an array axis oriented approximately ninety degrees from the longitudinal axis of the generally linear beams transmitted.

In step S102, a receiver (e.g.,34or134) collects one or more images of the defined spatial zone. In one example, the defined spatial zone includes at least the field of regard. The receiver (34or134) or camera18has a field of view that meets or exceeds the field of regard in spatial scope.

In one embodiment, step S102may be executed to collect multiple images (e.g., including first image data and second image data) of a defined spatial zone. For example, a controller54instructs the receiver34to collect a first image data of an image when a pulse of the electromagnetic radiation is not emitted or transmitted by the transmitter50(or by a particular transmitter module30), and a second image data of substantially the same image when a pulse of the electromagnetic radiation is emitted or transmitted by the transmitter50(or by a particular transmitter module30). The first image data and the second image data are taken from substantially the same position (e.g., x, y and z coordinates) or the first image data and the second image data are registered by determining the change in displacement (e.g., expressed as a vector or multidimensional quantity) between the first coordinates of the vehicle associated with the first image data and the second coordinates of the vehicle associated with the second image data. In an alternate embodiment, a location-determining receiver (e.g., Global Positioning System receiver with differential correction)(not shown) may be co-located with the obstacle detection system and the vehicle to provide the first coordinates, the second coordinates, or the relative displacement between the first and second coordinates. The data processor20subtracts the registered first image data from the second image data to derive the image (e.g., observed illumination radiation pattern or reflection on the object) to compensate for variation in illumination of at least one of the image and the object16(or obstacle).

In step S104, a data processor20detects a presence of an object16or obstacle in the collected image and in the defined spatial zone based on an observed illumination radiation pattern on an object16formed by at least one of the generally linear beams. For example, if no object is present in the defined spatial zone (or some defined portion thereof), none of the transmitted linear beams are reflected or form an illumination radiation pattern on the object16. Instead, the transmitted linear beams propagate through space until they attenuate or reach an obstacle outside of the defined spatial zone. However, if an object is present in the define spatial zone, one or more of the transmitted linear beams are reflected or form an illumination radiation on the object16. For example, if a short object is present in a first zone within the defined spatial zone, only lower linear beams form an illumination radiation pattern on the short object, whereas if a taller object is present in the first zone, both lower, middle and upper linear beams form an illumination radiation pattern on the taller object. If the horizontal length of the transmitted linear beams exceed the object width or dimension and if the coverage of beams exceeds a height of the object, the dimensions in image space or color space may be estimated. The spacing between the transmitted linear beams is proportional to the smallest detectable object, which may be significant for avoiding small flying objects or projectiles.

In step S106, a data processor20or distance estimator22estimates a distance between the object16and the optical device based on an observed material change (e.g., shift in one or more relative positions or relative heights) of generally linear beams from the transmitted radiation pattern to the observed illumination radiation pattern. In step S106, the distance may be estimated in accordance with various techniques that may be applied alternately or cumulatively.

Under a first technique, the distance estimator22estimates the distance by determining a reference illumination radiation pattern resulting from the transmitted radiation pattern being incident on a planar obstruction at a known fixed distance; and by determining a distance based on an observed shift between the reference illumination radiation pattern and the observed illumination radiation pattern.

Under a second technique, the estimating of a distance in step S106is applied to a first zone from the obstacle detection system11mounted on a vehicle. The first zone begins at a first distance (or first radius) spaced apart from the vehicle and extending to a second distance (or second radius) spaced apart from the first distance.

Under a third technique, the material change between the transmitted radiation pattern and the illumination radiation pattern represents a shift in the relative spacing or relative height of the linear beams of the illumination pattern. If the transmission angle between adjacent beams is fixed and if the linear beams of the illumination radiation pattern are closer together than those of the reference object located further away, the object16is closer than the reference object. Conversely, if the transmission angle between adjacent beams is fixed and if the linear beams of the illumination radiation pattern are farther apart than those of a closer reference object, the object16is farther away than the reference object. In one embodiment, the image data on adjacent beam spacing is proportional to the real world beam spacing dimensions that may be determined (1) by defining the optical characteristics of the receiver (34or134), (2) by empirical measurements of the obstacle detection system, or (3) by field measurements of the obstacle detection system11. Further, the relationship between (1) relative spacing of linear beams in the transmitted radiation pattern and the illumination radiation pattern and (2) the distance between the obstacle detection system and the object16may be empirically determined or experimentally measured and stored as a look-up table, a relational database, a file, tabular information or otherwise in data storage (e.g., memory) associated with the data processor20for future reference.

Under a fourth technique, the transmission angle (θ) of the generally linear beam with respect to the horizontal plane is known; the transmission height (HT) of the transmitter module30is known, the reception height (HR) of the receiver (34or134) or lens of the camera18is known; the image data (e.g., bitmap or color space) spacing (SI) between adjacent linear beams is measured from the illumination radiation pattern; and the real world spacing (SR) between the adjacent linear beams is derived from the image data spacing based on experimental measurements, empirical data, optical characteristics of the receiver (34or134) or otherwise. The data processor20or the distance estimator22estimates the distance of the object from the obstacle based on a trigonometric equation that considers one or more of the following: the transmission angle (θ), the transmission height (HT), the reception height (HR), image data spacing between lines of the illumination radiation pattern (SI), real world spacing (SR) between lines of the illumination radiation pattern. For example, distance (d) between the obstacle detection system and the object may be determined in accordance with the following equation:

d=h/tan θ, where d is the distance, θ is the transmission angle measured from a generally horizontal axis, and h is an observed height (HO) of the illumination on The object less the reception height (HR) (e.g., mounting height) of the camera or receiver. Alternatively, h is estimated as one-half of the real world spacing (SR).

In an alternate embodiment, the method ofFIG. 4relates to the operation of a primary obstacle detection procedure that may be supplemented by supplemental obstacle detection procedures including one or more of the following. Under a first supplemental procedure, an auxiliary or secondary obstacle detection procedure detects an obstacle via a transmission of an ultrasonic signal and a receipt of a reflection in a near range closer than the first zone. Under a second procedure, an auxiliary or tertiary obstacle detection procedure generates a radiation pattern (e.g., optical energy or laser light) in a generally planar vertical “virtual wall” extending from at least one side of the vehicle to determine spatial constraints during turning of the vehicle.

FIG. 5is a block diagram of a method for detecting an obstacle. The method ofFIG. 5begins in step S101.

In step S101, a transmitter50emits a plurality of generally linear beams of electromagnetic radiation in a transmitted radiation pattern within a defined spatial zone. The electromagnetic radiation may be transmitted over a certain frequency range, such as visible light, a portion of the visible light spectrum, infra-red light, a portion of the infra-red light spectrum, ultraviolet light, a portion of the ultra-violet light spectrum, or any combination of the foregoing frequency ranges. In step S101, the transmitted radiation pattern has a transmission angle (θ) with respect to a generally horizontal plane or axis and a height above ground.

Step S101may be executed in accordance with various techniques, which may be applied cumulatively or individually. Under a first technique, the transmitter (50,150, or250) transmits one or more pulses of the electromagnetic radiation. Under a second technique, the transmitter250coordinates the transmission time of pulses with other optical units in a common geographic zone to avoid interference. Under a third technique, the transmitter (50,150or250) directs the transmitted electromagnetic radiation within a defined spatial zone, which comprises a field of regard. The field of regard is defined by a three-dimensional spatial zone extending in a direction of travel of the vehicle and having a height and width that is greater than the vehicle height and the vehicle width by a minimum spatial clearance. Under a fourth technique, the transmitter (50,150or250) directs the transmitted electromagnetic radiation within a depth of the three-dimensional spatial zone. The three-dimensional spatial zone is defined by the at least one of the vehicular speed or velocity of the vehicle, minimum vehicle stopping distance plus a safety interval corresponding to the vehicular speed, maximum oncoming speed of oncoming objects, sampling rates of the collection of the image, and resolution of the image. Under a fifth technique, the transmitter (50,150or250) emits a first group of generally linear beams from a first transmitter module and emits a second group of generally linear beams from a second transmitter module spaced apart from the first transmitter module.

In step S102, the receiver34collects one or more images of the defined spatial zone. The collection of the image may comprise filtering of the image to attenuate signals outside a frequency range. In one embodiment, step S102may be executed to collect multiple images (e.g., including first image data and second image data) of a defined spatial zone. For example, a controller54instructs the receiver34to collect a first image data of an image when a pulse of the electromagnetic radiation is not emitted or transmitted by the transmitter50(or by a particular transmitter module30), and a second image data of substantially the same image when a pulse of the electromagnetic radiation is emitted or transmitted by the transmitter50(or by a particular transmitter module30). The first image data and the second image data are taken from substantially the same position (e.g., x, y and z coordinates) or the first image data and the second image data are registered by determining the change is displacement (e.g., expressed as a vector or multidimensional quantity) between the first coordinates of the vehicle associated with the first image data and the second coordinates of the vehicle associated with the second image data. In an alternate embodiment, a location-determining receiver (e.g., Global Positioning System receiver with differential correction)(not shown) may be co-located with the obstacle detection system and the vehicle to provide the first coordinates, the second coordinates, or the relative displacement between the first and second coordinates. The data processor20subtracts the registered first image data from the second image data to derive the image (e.g., an observed illumination radiation pattern or reflection associated with the object16) to compensate for variation in illumination of at least one of the image and the object16(or obstacle).

In step S110, the data processor20processes the image to identify pixels associated with the illumination of the object16with one or the generally linear beams.

In step S112, the data processor20determines an observed height of the identified pixels. Once the observed height of the pixels are determined in the image data (e.g., in the bitmap or in color space), the observed height of the pixels may be scaled to a real world or actual height of the corresponding points on the object with reference to the optical characteristics of the camera18, test measurements, or empirical measurements or studies. In one embodiment, the determination of the observed height comprises determining the real world height of one or more points on the object16at which the radiation pattern strikes the object16, where the points correspond to the pixel locations or coordinates.

In step S114, the distance estimator22determines a distance between the object16and the obstacle detection device (e.g.,11,111or211) based on one or more of the following factors: (1) transmission angle (θ) of the generally linear beam with respect to a horizontal plane or axis, (2) the transmission height (HT) of the transmitter module30, (3) the reception height (HR) of the receiver (34or134) or lens of the camera18, (4) the transmission coordinates of the transmitter module30, (5) the reception coordinates of the receiver (34or134) or lens of the camera18, (6) the observed height of the identified pixels, (7) the observed height (e.g., real world height) of the points on the object16corresponding to the identified pixels, (8) the image data (e.g., bitmap or color space) spacing (SI) between adjacent linear beams is measured from the illumination radiation pattern; and (9) the real world spacing (SR) between the adjacent linear beams is derived from the image data spacing based on experimental measurements, empirical data, optical characteristics of the receiver (34or134) or otherwise. In accordance with one illustrative example of carrying out step S114, the data processor20or the distance estimator22estimates the distance of the object from the obstacle based on a trigonometric equation that considers one or more of the above factors. For example, distance (d) between the obstacle detection system and the object may be determined in accordance with the following equation:

d=h/tan θ, where d is the distance, θ is the transmission angle measured from a generally horizontal axis, and h is an observed height (HO) of the illumination on the object less the reception height (HR) (e.g.. mounting height) of the camera or receiver. Alternatively, h is estimated as one-half of the real world spacing (SR).

In the design or configuration of the obstacle detection system (11,111, or211), transmission angle (θ) may be selected in accordance with various techniques, that may be applied separately or cumulative. Under a first technique, the transmission angle is determined in conformity with minimum vehicle stopping distance plus a safety interval. Under a second technique, the transmission angle is determined in conformity with one or more of the following: distance related to the maximum expected speed of any oncoming mobile objects, sampling rate of the collection of data, and resolution requirements.

FIG. 6Ashows the side view of a mobile vehicle38equipped with an obstacle detection system (11,111or211ofFIG. 1,FIG. 2, orFIG. 3). As illustrated, the mobile vehicle38has three transmitter modules30, including a first transmitter module31, a second transmitter module33, and a third transmitter module35. The first transmitter module31is separated from the second transmitter module33by a known (e.g., fixed vertical) distance; the second transmitter module33is separated from the third transmitter module35by a known (e.g., fixed vertical) distance.

As used herein, field of regard refers to a three dimensional volume in the direction of travel of a vehicle. The field of regard in a linear direction is defined spatially to be a window that is at least the width and the height (e.g., or another maximum transverse cross-section dimension) of the vehicle, each extended by a clearance safety factor. The depth or distance (which is designated dnear(40)) of the window forming the volumetric field of regard is defined by the vehicle stopping distance (e.g., minimum stopping distance or worst case stopping distance) plus a safety interval. On level terrain with a defined surface, the minimum vehicle stopping distance or worst case stopping distance depends upon at least the instantaneous speed and acceleration of the vehicle, unless a maximum speed and acceleration are assumed. The depth or distance (which is designated dfar (42)) is related to the maximum expected speed (or actual detected speed) of any oncoming mobile objects16, the sampling rate of the obstacle detection system (e.g.,11,111or211), the resolutions of the obstacle detection system, or any combination of the foregoing items.

In the illustrative example ofFIG. 6A, a wall, fence, building side, or other vertical planar surface355is located a distance dfar(42) from a front side of the mobile vehicle38. The aggregate radiation pattern44is used to detect an obstacle within a range or first zone from dnear(40) to dfar(42). However, if an obstacle is located between the vehicle and dnear(40), an auxiliary obstacle detector is used to detect the obstacle. For example, an ultrasonic obstacle detector may be used to detect an obstacle located in a second zone between the vehicle and dnear(40).

The transmitter modules30transmit an aggregate radiation pattern44that is composed of a first linear beam91, a second linear beam93, a third linear beam95, a fourth linear beam97, a fifth linear beam98and a sixth linear beam99. Although six linear beams (91,93,95,97,98, and99) are illustrated inFIG. 6A, it is understood that any number of linear beams may fall within the scope of the invention and that the actual number of linear beams may depend upon factors such as the smallest obstacle size to be detected, for example. In one embodiment, each linear beam has a transmission angle (θ) with respect to a horizontal plane or axis. However, in alternative embodiments the transmission angle of each linear beam may differ. InFIG. 6A, a primary beam pair361comprises a first linear beam91and a fourth linear beam97; a secondary beam pair359comprises a second linear beam93and fifth linear beam98, and a tertiary beam pair357comprises a third linear beam95and sixth linear beam99. The primary beam pair361emanates from a common transmitter module30; the secondary beam pair359originates from another common transmitter module30, and the tertiary beam pair357emanates from yet another common transmitter module30.

FIG. 6Bshows a reference illumination radiation pattern or first illumination radiation pattern that occurs when the transmitter (50,150or250) transmits electromagnetic radiation that is incident upon a planar surface355spaced apart from the vehicle by dfar(42). Like reference numbers inFIG. 6AandFIG. 6Bindicate like elements. The first linear beam91, the second linear beam93, the third linear beam95, the fourth linear beam97, the fifth linear beam98and the sixth linear beam99are shown as projected onto and looking toward the substantially planar surface355from the frontward-facing perspective of the vehicle38.

FIG. 6Cshows a top view of the radiation pattern44ofFIG. 6Ain a horizontal plane. The radiation pattern44is associated with the primary obstacle detection procedure, consistent with any of the embodiments ofFIG. 1throughFIG. 5, inclusive. The primary radiation pattern associated with the primary obstacle detector extends from dnear(40) to dfar(42) in a first zone43. The secondary radiation pattern associated with the secondary obstacle detector (e.g., ultrasonic obstacle detector) extends from the vehicle to dnear(40) in a second zone41.

The diagram ofFIG. 7Ais similar to the diagram ofFIG. 6A, except the diagram ofFIG. 7Afurther includes an obstacle or object116(e.g., person on a scooter or skateboard) between dnear(40) and dfar(42). The illumination radiation pattern144is obstructed by the object116where the transmitted electromagnetic radiation or light strikes the object116. Accordingly, the transmitted electromagnetic radiation144comprises a first obstructed linear beam191, a second obstructed linear beam193, a third obstructed linear beam195, a fourth obstructed linear beam197, a fifth obstructed linear beam198, and a sixth obstructed linear beam99. The first obstructed linear beam191corresponds to the first linear beam91; the second obstructed linear beam193corresponds to the second linear beam93; the third obstructed linear beam195corresponds to the third linear beam95; the fourth obstructed linear beam197corresponds to the fourth linear beam97; the fifth obstructed linear beam198corresponds to the fifth linear beam98; and the sixth obstructed linear beam199corresponds to the sixth linear beam99. An obstructed primary beam pair461is composed of obstructed linear beams (191and197); an obstructed secondary beam pair459is composed of obstructed linear beams (193and198); a obstructed tertiary beam pair457is composted of obstructed linear beams (195and199). The obstructed linear beams (191,193,195,197,198, and199) interact with or reflect from the object116to form an illumination radiation pattern.

FIG. 7Bshows an observed illumination radiation pattern or a second illumination radiation pattern that occurs when the transmitter50transmits electromagnetic radiation144that is incident upon the object116between dnear(40) and dfar(42). Like reference numbers inFIG. 7AandFIG. 7Bindicate like elements.

The receiver34may receive the reference illumination radiation pattern or the data processor20may reference the reference illumination radiation pattern in data storage (e.g., memory). The receiver34receives the observed illumination radiation pattern. The data processor20may compare the reference illumination radiation pattern to the observed illumination radiation pattern to determine if a shift or change in the radiation pattern has occurred. The extent of the change or shift indicates the relative position or distance of the obstacle with respect to the vehicle.

FIG. 8AandFIG. 8Bshow a side-by-side comparison between the received illumination radiation patterns ofFIG. 6BandFIG. 7B. Like reference numbers inFIG. 6AthroughFIG. 8B, inclusive, indicate like elements. If the reference illumination radiation pattern ofFIG. 8Ais established based on a substantially planar surface355spaced apart from the vehicle by dfar, the object116associated with the observed illumination radiation pattern ofFIG. 8Bis closer than dfarto the vehicle38because the line spacing of the observed illumination radiation pattern is lesser than that of the radiation pattern ofFIG. 8A, for corresponding beam pairs. For example, the obstructed linear beams (191and197) of obstructed primary beam pair461ofFIG. 8Bare closer together than in the corresponding linear beams (91and97) of the primary beam pair361ofFIG. 8A. Similarly, the obstructed linear beams (193and198) of obstructed secondary beam pair459ofFIG. 8Bare closer together than in the corresponding linear beams (93and98) of the secondary beam pair359ofFIG. 8A. The degree of shift between the position of the primary beam pair361and the obstructed primary beam pair461may provide an indication of the distance of the object116, if the distance between the vehicle and the planar surface355is known or if dfaris known. The degree of shift between the position of the secondary beam pair359and the obstructed secondary beam pair459may provide an indication of the distance of the object116, if the distance between the vehicle and the planar surface355is known or if dfaris known.

FIG. 9is a block diagram of a obstacle detection system comprising a primary obstacle detector350and a secondary obstacle detector352that are coupled to a vehicular control unit354. The primary obstacle detector350is consistent with the obstacle detection system inFIG. 1orFIG. 2. Like reference numbers inFIG. 1,FIG. 2andFIG. 9indicate like elements.

The primary obstacle detector350and the secondary obstacle detector352are coupled to a vehicle control unit354for communications therewith. The secondary obstacle detector352may supplement or complement the primary obstacle detector350. For example, the primary obstacle detector350may have a different coverage pattern or coverage range with the respect to the vehicle (or a different electromagnetic frequency range), than the secondary obstacle detector352. In on illustrative configuration, the primary obstacle detector350is based on an optical obstacle detection system ofFIG. 1,FIG. 2, orFIG. 3and covers an area between dnear(40) and dfar(42), whereas the secondary obstacle detector352is based on an ultrasonic obstacle detection scheme and covers an area between the vehicle and dnear(40).

The secondary obstacle detector352may comprise an ultrasonic acoustic sensor. Ultrasonic acoustic sensors generally have a wide beamwidth and are susceptible to multipath reflections, such that the secondary obstacle detection is most effective in a first zone from the vehicle to dnear(40). Ultrasonic sensors are well suited for identifying or resolving negative obstacles. In one embodiment, the secondary obstacle detector352has a maximum range of approximately dnear(40). Suitable ultrasonic sensors are manufactured and available through Polaroid Corporation of Waltham, Mass.

The obstacle detection system ofFIG. 10is similar to the obstacle detection system ofFIG. 9, except the obstacle detection system ofFIG. 10further includes a tertiary obstacle detector356. Like reference numbers inFIG. 9andFIG. 10, indicate like elements. In one embodiment, the tertiary obstacle detector356has a different coverage pattern or coverage range with respect to the vehicle, than the primary obstacle detector350and the secondary obstacle detector352. For example, the tertiary obstacle detector356may comprise a flare-side obstacle detector that forms a generally vertical radiation pattern extending from a side of the vehicle38. The generally vertical radiation pattern is transmitted backwards from two opposite sides of the vehicle38, at a side angle. The side angle and the generally vertical radiation pattern are selected to assist the vehicle in maneuvering and making turns without striking or contacting obstacles on an inner radius or outer radius during the turn.

In one illustrative example, the tertiary obstacle detector356creates a radiation pattern that comprises a generally vertical radiation pattern (e.g., a virtual optical wall) that is projected vertically to the ground and is associated with a camera18to detect perturbations in the generally vertical radiation pattern, indicating the presence of an obstacle. The tertiary obstacle detector356is well-suited for detecting an obstacle where the vehicle turns and the area when the turn occurs has not been recently observed by forward looking sensors or data collection devices.

FIG. 11shows a top view of the radiation patterns associated with the primary obstacle detector350, the secondary obstacle detector352, and a tertiary obstacle detector356. Like reference number inFIG. 9andFIG. 11indicate like elements.

The first radiation pattern548is associated with the primary obstacle detector350and extends from dnear(40) to dfar(42) in a first zone43. The second radiation pattern550is associated with the secondary obstacle detector352and extends from the vehicle to dnear(40) in a second zone41. The third radiation pattern554associated with the tertiary obstacle detector356extends backwards from each side of the vehicle38.

The first radiation pattern548in the first zone43ofFIG. 10is substantially the same as the radiation pattern of the first zone43ofFIG. 6C. The first radiation pattern548is formed from multiple emission sources of the primary obstacle detector350. Accordingly, multiple individual radiation patterns of transmitter modules30form an overall radiation pattern or first radiation pattern548for the first zone43.

The secondary obstacle detector352has a second radiation pattern550in a second zone41. The second radiation pattern550is an aggregate of multiple secondary obstacle detectors352. The tertiary obstacle detector356produces a third radiation pattern554from each side of the vehicle.

Although a single camera18was used in each receiver34referenced herein, in alternate embodiments multiple cameras may be used. Multiple cameras may support redundancy and stereo image processing, for example.

The vehicle may comprise a small robot for in-building or exterior use. Although any dimensions may apply to the invention and fall within its scope, in an illustrative example the vehicle is 1 m long, 1 m high, and 0.5 meters long; dnear(40) and dfar(42) are set at 2 meters and 4 meters, respectively; and detectable obstacle size is 0.01 meters. The transmitter50may not rely upon moving parts to attain robustness and reliability over prior art designs. Other obstacle detection technologies such as stereo vision and ultrasonics may be used to augment other obstacle detection procedure or technology, and for redundant obstacle detection.