Patent ID: 12253594

DETAILED DESCRIPTION

Introduction

Radar systems measure velocity of a point on an object in a radial direction, this is referred to as range-rate. Multiple points (e.g., a radar point cloud) or multiple frames (e.g., scans of the radar) of measurements are needed to extract a velocity profile for an object. Radar systems often provide, during each frame or scan, a sparse point cloud (e.g., including only a few points) for some objects and may even provide only a single point detection objects that are far away, so called far range objects. With only a limited or single point in the radar point cloud, it is difficult to accurately initialize the velocity for a track. The number of radar points in one scan might not be enough to have a stable solution. In other words, the velocity computation may be erratic or not consistent with actual movement of the object.

A typical process for computing velocity includes solving a least square problem using range-rates and azimuth angles computed for different points within the radar point cloud. A change of positions of the points is calculated across two or more consecutive scans of the radar. However, instability in the velocity results from these existing techniques cannot provide a stable result that is accurate and predictable from one frame to the next. For example, when multiple the reflection points from an object are close to each other in position at every scan, the reflection points might actually contain nearly linear dependent information. However, if the reflection points are far away from each other at each scan, a change of position of those points from consecutive scans can yield a corrupted velocity because the extent of the object, which is also captured by the change of position, is typically ignored for simplicity. That is the change in position between two points may be seemingly impossible given limitations on size of other objects, especially vehicles, on a road. Existing algorithms and techniques may try to solve this problem by examining consistency of the change of position information of radar points from consecutive cycles to attempt to lower weightages of the inconsistent information in the ultimate assumption about the initial object motion and direction.

This document describes an object tracker that performs stable velocity initialization for radar tracks, using multiple hypotheses, including when only sparse radar point clouds are available. With just a single point per scan, the tracker creates multiple hypotheses for predicted movement of an object. A least square function can be applied to each hypothesis to derive each respective initial velocity, which are tracked using a Kalman Filter during a hypotheses tracking period. When hypotheses are initialized and tracked during the hypotheses tracking period, their track error scores are computed. The hypotheses that have low evidence (e.g., high error score) are discarded during the hypotheses tracking period. When the hypotheses tracking period ends, a single hypothesis with high evidence (e.g., low error score) is used to initialize the track's velocity. Parallel hypothesis evaluation enables the tracker to initialize track velocity quickly and accurately by merely selecting the best hypothesis, which may enable safer driving.

Example Environment

FIG.1illustrates an example environment100for stable radar track velocity initialization using multiple hypotheses, for example, by a vehicle102. Although illustrated as a passenger truck, the vehicle102can represent other types of motorized vehicles (e.g., a car, motorcycle, bus, tractor, semi-trailer truck), non-motorized vehicles (e.g., a bicycle), railed vehicles (e.g., a train), watercraft (e.g., a boat), aircraft (e.g., an airplane), spacecraft (e.g., satellite), and the like. The depicted environment100includes the vehicle102traveling on a roadway. The vehicle102is equipped with a perception system104for detecting an object106(or other like it) present on or near the roadway, which can impact how or whether the vehicle102can continue to travel.

The perception system104is used to relay information to other systems of the vehicle102about objects detected in the environment100, such as an object106in a travel path of the vehicle102. A region of interest associated with the perception system104at least partially surrounds the vehicle102. This region, when monitored by the perception system104, is referred to as a field of view108(also referred to as an instrumented field of view). The perception system104functions based on an input of point cloud sensor data, which can be obtained from a single type of sensor or a variety of different types of sensors. For ease of description, the point cloud sensor data obtained by the perception system104is described primarily as being radar data obtained from a radar system of the vehicle102. The perception system104can be installed on, mounted to, or integrated with any part of the vehicle102, such as in a front, back, top, bottom, or side portion of the vehicle102, a bumper, a side mirror, part of a headlight and/or taillight, or at any other interior or exterior location of the vehicle102where object detection using point cloud sensor data is desired. Careful selection or re-positioning of components of the perception system104and/or the radar system to which it connects can further cause the field of view108to have a particular shape or size.

Included in the perception system104are a processor110and a radar interface112. The processor110executes software and/or firmware that configures the perception system104to perform various functions in furtherance of object tracking. For example, information output from the processor110can take the form of tracks116; each of the tracks116is to a different object detected in the field of view108. The tracks116may include fields of information, including bounding box dimensions, size and position measurements, classifications, and other data characterizing a vehicle, pedestrian, traffic sign, or other object that appears in a region being monitored by the perception system104. The processor110is configured to generate the tracks116based on point cloud sensor data114that the processor110receives from the radar interface112.

The radar interface112can include a combination of hardware and software executing thereon or on the processor110. The radar interface112operably connects the processor110of the perception system104to an output of a radar based sensor source, including the radar system of the vehicle102and/or external radar sources. The radar interface112may obtain point cloud sensor data from radar systems of other vehicles using vehicle-to-vehicle communications, e.g., for improving size or resolution of the field of view108. The radar interface112may provide the point cloud sensor data in compressed or uncompressed form, or in any other format suitable for object tracking. A communication channel is shared between the radar interface112and a radar system. The communication channel can include an application programming interface (API), or other function executed by the processor110. The radar interface112is one example sensor interface. Another interface to another type of sensor (e.g., lidar, camera) can also be used by the perception system104to obtain additional point cloud sensor data that is of a different sensor type.

An object tracker118, including an initializer120, is an example component of the perception system104that may be implemented at least partially in executable code that, when executed, configures the processor110to generate the tracks116to objects in the field of view108. The tracks116are output from the object tracker118to other systems of the vehicle102, which rely on the tracks116for situational awareness of potential obstacles. The initializer120, when executed, configures the processor110to initialize a respective velocity of each of the tracks116in a stable manner, by using multiple hypotheses.

Relying on the initializer120, the object tracker118is configured to perform stable velocity initialization for the tracks116, using multiple hypotheses, including when only sparse radar point clouds are available. With just a single point per scan, the initializer120creates multiple hypotheses for direction and speed of an object. For instance, the object tracker118obtains from the radar interface112, the point cloud sensor data114, which is indicative of radar returns that reflect off of objects in the environment100. The object tracker118performs object tracking techniques. Using the point cloud sensor data114, a track to the object106is added to the tracks116. However, rather than use other techniques to initialize a velocity measurement for the object106, the initializer120is configured to initialize the velocity measurement more accurately and in a stable way. The initializer120is configured to create at least two hypotheses for predicted movement of the object106. For each of the at least two hypotheses, an initial velocity and a first associated level of evidence supporting the initial velocity of that hypothesis is determined. The initializer120next generates a fused hypothesis by combining the at least two hypotheses. More specifically, the fused hypothesis includes an initial velocity and a first associated level of evidence of the fused hypothesis being based on an aggregate of the initial velocities and the first associated levels of evidence of the at least two hypotheses. The initializer120includes the fused hypothesis among the at least two hypotheses. A least square function can be applied by the initializer120to each hypothesis to derive each respective initial velocity.

Responsive to including the fused hypothesis among the at least two hypotheses, the initializer120is configured to select, based on the first associated levels of evidence for the at least two hypotheses, a single best hypothesis for initializing the velocity measurement for the object106. For example, the initializer120updates the hypotheses using a Kalman Filter during a hypotheses tracking period. During the hypothesis tracking period, each of the hypotheses are initialized by the initializer120and tracked, which includes the initializer120computing their track error scores. The hypotheses that have low evidence (e.g., high error score) may be discarded during the hypotheses tracking period. When the hypotheses tracking period ends, a single hypothesis with high evidence (e.g., low error score) initializes the track's velocity. Parallel hypotheses evaluation performed by the initializer120in this way enables the object tracker118to quickly initialize a velocity of a track by merely selecting the best hypothesis, which ultimately enables safer driving because the tracks116, which are used by other systems of the vehicle102for control or safety functions, include information more accurately, but as quick or nearly as fast as if initialized in other ways.

Example Vehicle Configuration

FIG.2illustrates an example vehicle102-1including a system104-1configured to perform stable radar track velocity initialization using multiple hypotheses. The vehicle102-1is an example of the vehicle102.

Included in the vehicle102-1is a perception system104-1, which is an example of the perception system104, shown in greater detail. The vehicle102-1also includes vehicle-based systems210that are operatively and/or communicatively coupled to the perception system104-1via link202, which may be one or more wired and/or wireless links including vehicle-based network communications for interconnecting the components of the vehicle102-1. In some examples, the link202is a vehicle communication bus.

The vehicle-based systems210use vehicle data, including object tracking data provided on the link202by the perception system104-1, to perform vehicle-based functions, which in addition to other functions may include functions for vehicle control. The vehicle-based systems210can include any conceivable device, apparatus, assembly, module, component, subsystem, routine, circuit, processor, controller, or the like, which uses radar data to act on behalf of the vehicle102-1. As some non-limiting examples, the vehicle-based systems210may include a system for autonomous control, a system for safety, a system for localization, a system for vehicle-to-vehicle communication, a system for use as an occupant interface, and a system for use as a multi-sensor tracker. Upon receipt of the object tracking data (e.g., the tracks116), functions provided by the vehicle-based systems210use portions of the object tracking data, including velocity measurements of objects detected in the field-of-view108, to configure the vehicle102-1to safely drive without colliding with the detected objects.

The tracks116are examples of the object tracking data that is output on the link202to the vehicle-based systems210. One of the tracks116can include information about movement of the object106, such as a velocity, position, and the like, to enable the vehicle-based systems210to control or assist with braking, steering, and/or accelerating the vehicle102-1to avoid a collision with the object106. A system for autonomous control can use the tracks116received via the link202to autonomously or semi-autonomously drive the vehicle102-1safely on a road. A system for use as an occupant interface can use the information in the tracks116allow an operator or passengers to have situational awareness to make driving decisions or provide operator inputs to a controller for providing more buffer to avoid the objects. The tracks116or information contained therein may be provided to other vehicles using a system for vehicle-to-vehicle communication, to allow operators, passengers, or controllers of other vehicles to also avoid the objects being tracked or have confidence that the vehicle102-1is aware of their presence based on receipt of the tracks116. By improving situational awareness for the vehicle102-1and other vehicles in the environment100, the vehicle102-1can drive in a safer manner under manual, autonomous, or semi-autonomous control.

The perception system104-1includes a processor110-1, as an example of the processor110, a radar system206, and a computer-readable medium (CRM)208. The radar system206can include any quantity of radar devices, antennas, and other components to provide the point cloud sensor data114that covers the field of view108. The radar system206may include a radar chip, an antenna or antenna array such as a multiple input multiple output (MIMO). The radar system206can include various transmitter/receiver elements, timing/control elements, and analog-to-digital converters. As already mentioned, although described primarily in the context of radar, other sensor systems may be adopted by the perception system104-1to perform point cloud based object tracking.

Some examples of the processor110-1include, a controller, a control circuit, a microprocessor, a chip, a system, a system-on-chip, a device, a processing unit, a digital signal processing unit, a graphics processing unit, and a central processing unit. The processor110-1may include multiple processors, one or more cores, embedded memory storing software or firmware, a cache or any other computer element that enables the processor110-1to execute machine-readable instructions for generating the tracks116.

Machine-readable instructions that are executed by the processor110-1can be stored by the CRM208. The CRM208may also be used to store data managed by the processor110-1during execution of the instructions. In some examples, the CRM208and the processor110-1are a single component, such as a system on chip including the CRM208configured as a dedicated storage for the processor110-1. In some examples, access to the CRM208is shared by other components of the perception system104-1(e.g., the radar system206) that are connected to the CRM208. The processor110-1obtains instructions from the CRM208; execution of the instructions configure the processor110-1to perform object tracking operations, such as radar based object tracking, which result in communication of the tracks116to the vehicle-based systems210and other components of the vehicle102-1over the link202.

In this example, the CRM208includes instructions for configuring the processor110-1to provide a radar interface112-1, which is an example of the radar interface112. In addition, the CRM208includes instructions for executing an object tracker118-1, including an initializer120-1, which are examples of, respectively, the object tracker118and the initializer120fromFIG.1.

In operation, the object tracker118-1is configured to obtain from the radar interface112-1, the point cloud sensor data114generated by the radar system206. The point cloud sensor data114conveys information about radar returns detected from objects in the environment100. The object tracker118-1is configured to process the point cloud sensor data114to establish the tracks116; with each of the tracks116corresponding to a particular object in the environment100. As the object tracker118-1generates the tracks116, they are reported to the vehicle-based systems210. In establishing each of the tracks116, the object tracker118-1calls on the initializer120-1to initialize a velocity measurement for each track prior to that track being introduced into the tracks116.

To initialize the velocity measurement of one of the tracks116, the initializer120-1generates and maintains hypotheses212, which are based in part on solutions to least linear square (LLS) problems. Solving the LLS problems is simplified by being based on a weighted least square principle. The hypotheses212initially include at least two hypotheses corresponding to solutions of different LLS problems, and another hypothesis derived from fusing the solutions of all the different LLS problems into one. For example, the hypotheses212may be derived from solving seven different LLS problems to capture seven different potential radar point distributions possible from the point cloud sensor data114. The seven LLS problems produce seven solutions, which correspond to seven of the hypotheses212. An eighth hypothesis is added to the hypotheses212, and, as described below in greater detail, is formed from fusing the seven solutions. Then, a high degree of accuracy is achieved from tracking, for just a short time, the hypotheses212in order to select a best one.

After the initializer120-1establishes the hypotheses212, the velocity measurement of the tracks116is still not initialized, immediately. Instead, all the hypotheses212are tracked using a motion model (e.g., a simple constant motion model) for a few frames or scans of the radar system206(e.g., less than or equal to eight scans). The quantity of frames scanned when tracking the hypotheses212may be tunable or adjusted for different modes or characteristics of the radar system206, the perception system104-1, and/or the vehicle102-1. During each scan of this hypotheses tracking period, a track error score is computed and assigned to each of the hypotheses212.

The track error score is computed based on an accumulated error between each of the hypotheses212and the velocity measurement associated to the best hypothesis. The track error score is used by the initializer120-1to select the best hypothesis to use for initializing the velocity measurement of one of the tracks116. Unlike other hypothesis tracking techniques that consider associations between every hypothesis and all measurements at every scan, complexity of velocity initialization is reduced. Rather than consider every association at each scan, the initializer120-1is configured to only considers a single best association per scan. The track error score enables the initializer120-1to arrive at the single best hypothesis sooner by eliminating, after each scan of the hypotheses tracking period, each of the less likely hypotheses from the hypotheses212until arriving at one.

After the initializer120-1determines the velocity measurement of a new track, it is added to the tracks116. The tracks116may be measurement updated for a current track reporting period. By using the velocity measurement from the initializer120-1, the tracks116can be output with high accuracy and low latency. This may enable the vehicle-based systems210that receive the tracks on the link202to perform functions of the vehicle102-1with increased safety and greater precision.

Process of Stable Radar Track Velocity Initialization Using Multiple Hypotheses

FIG.3-1illustrates a flow diagram of an example process300for stable radar track velocity initialization using multiple hypotheses. For ease of description, the process300is described primarily in the context of being performed by the perception systems104and104-1. Operations (also referred to as steps) of the process300are numbered, however, this numbering does not necessarily imply a specific order of operations. The steps of the process300may be rearranged, skipped, repeated, or performed in different ways than the specific way it is shown in the diagram ofFIG.3-1.FIG.3-2andFIG.3-3illustrate example scenarios330-1and330-2of a host vehicle initializing radar track velocity measurements, in the context ofFIG.3-1. In each example, it is assumed that only a single point from the point cloud sensor data114is associated with each detectable object, after four consecutive scans or frames of the radar system206. One of ordinary skill can easily generalize this example and apply the process300to a different scenario where more than one point in the point cloud sensor data114is associated with each detectable object, for example, by calculating an average position of all the points within the point cloud sensor data114that are associated with a detectable object.

Hypotheses Initialization Period (e.g., Steps302Through308)

In operation, the perception system104obtains from the radar system206, the point cloud sensor data114indicative of radar returns that reflect off of objects in the environment100. Using the point cloud sensor data114, the perception system104establishes a track to the object106in the environment including initializing a velocity measurement for the object106. The perception system104may execute the initializer120in executing the object tracker118. When called on by the object tracker118, the initializer120executes the process300to initialize the velocity measurement of the object106.

At302, hypotheses for at least two different distributions of point cloud sensor data are created. For example, the initializer120creates multiple hypotheses for predicted movement of the object106. There are 2 kinds of information can be used to solve for the initial velocity, as defined by Equation 1 for defining range rate and azimuth angle information, and Equation 2 for defining change of position:
vxcos(θi)+vysin(θi)={dot over (R)}c,   Equation 1
ΔTivx=xi−xi-1,ΔTivy=yi−yi-1,   Equation 2
In each of these Equations 1 and 2, the subscript i indicates each different scan, and a constant velocity is assumed within a plurality of scans (e.g., four scans), referred to a scanning period.

ConsiderFIG.3-2, the tracks116produced for the scenario330-1are directed to four different objects in the field of view108of the vehicle102. Each of the tracks116includes bounding box dimensions for a different object. The bounding box is based on information inferred from single point observations in the point cloud sensor data114, over the scanning period, which in this case is four scans. For example, a bounding box332-1is derived for a first object based on consecutive points334-1and another bounding box332-2is derived for a second object using consecutive points334-2. For a third object and a fourth object in the field of view108, respectively, a third bounding box332-3can be derived from consecutive points334-3and a fourth bounding box332-4can be derived from consecutive points334-4.

In this scenario, when determining velocity measurements for each of the objects based on the consecutive, single points observed over the scanning period, various considerations are accounted for. For example, one of the tracks116corresponds to the bounding box332-1; a change of longitudinal position (e.g., as indicated by a X axis) of this track may not be particularly useful in estimating direction or speed, but a change of lateral position (e.g., as indicated by a Y axis) may be useful. For a different one of the tracks116, the opposite may be true; one of the tracks116corresponds to the bounding box332-2, and a change of lateral position of this track may not be useful for determining velocity, however, a change of longitudinal position may be useful for such estimations. For the third object associated with the bounding box332-3, a positional change in both directions, e.g., seemingly concurrent changes in lateral and longitudinal position, may be useful to derive an initial velocity estimate for a third track of the tracks116. In contrast, for the fourth object associated with the bounding box332-4, a positional change in both directions is not useful to derive an initial velocity estimate for a fourth track of the tracks116.

This demonstrates that, if an amount of angular separation θibetween the consecutive points being used to derive a particular bounding box is too small (e.g., the amount of angular separation θiis less than a separation threshold), then the consecutive points for that bounding box may not be reliable for determining velocity. Likewise, the consecutive points are not useful to estimate velocity, in some cases, even if the amount of angular separation θiis sufficiently large. Velocity (vx, vy) of an object can be derived from the consecutive points based on Equation 1, where vxis a longitudinal velocity of an object, vyis a lateral velocity of an object, and {dot over (R)}cis a radial range rate to the object, compensated for host velocity. As the amount of angular separation θiapproaches zero, solving for the velocity (vx, vy) is difficult. Even if the amount of angular separation θiis sufficiently large between the consecutive points, inconsistent change in position by some of the points (e.g., when some points span a first edge of an object and the other consecutive points span a second edge that is orthogonal to the first edge) causes the velocity (vx, vy) to also be unreliable.

When executed by the processor110at302, the initializer120of the object tracker118may generate the following seven hypotheses, referred to as Hypothesis 1 to Hypothesis 7:1. ΔTivx=xi−xi-1, ΔTivy=yi−yi-1;2. vxcos(θi)+vysin(θi)={dot over (R)}c, ΔTivx=xi−xi-1;3. vxcos(θi)+vysin(θi)={dot over (R)}c, ΔTivy=yi−yi-1;4. vxcos(θi)+vysin(θi)={dot over (R)}c, ΔTivx=xi−xi-1, ΔTivy=yi−yi-1;5. Assuming same motion (e.g., curvature) as the vehicle102and the object being tracked, vxcos(θi)+vysin(θi)={dot over (R)}c, ΔTivx=xi−xi-1, ΔTivy=yi−yi-1;6. Assuming some radial motion as the vehicle102and the object being tracked (e.g., a velocity vector of the object points to a center of a front bumper of the vehicle102), vxcos(θi)+vysin(θi)={dot over (R)}c, ΔTivx=xi−xi-1, ΔTivy=yi−yi-1; and7. Assuming some cross radial motion between the vehicle102and the object being tracked (e.g., a velocity vector of the object points perpendicular to the radial direction), vxcos(θi)+vysin(θi)={dot over (R)}c, ΔTivx=xi−xi-1, ΔTivy=yi−yi-1.
An assumption in Hypothesis 5 listed above is that an object is moving on the same road as the vehicle102, so they are moving with the same motion or curvature.FIG.3-3shows a scenario330-2where the vehicle102has a circular motion and detects an object336in a field of view. A motion direction or curvature of the host vehicle102can be obtained from a vehicle state estimator (VSE) (e.g., executed by a controller of the vehicle102). Based on the VSE of the vehicle102, a motion direction α of the object336can easily be computed based on geometry (e.g., α=γ=arcsin [xi*curvature]).

At304, an initial velocity for each of the hypotheses is determined. For example, the initializer120determines, for each of the multiple hypotheses212, an initial velocity of that hypothesis. An LLS problem is assigned to each hypothesis as a weighted LLS based on Equations 3 and 4, as shown below, and where diag[·] refers to a corresponding diagonal matrix:

[w1000⋱000w3⁢n+4][cos⁢θ1sin⁢θ1⋮⋮⋮⋮Δ⁢T100Δ⁢T1⋮⋮100110011001]Equation⁢3[vxvy]=[w1000⋱000w3⁢n+4][Range⁢Rate1⋮⋮xk+1-xkyk+1-yk⋮vx⁢1vy⁢1vx⁢2vy⁢2vx⁢3vy⁢3]diag[w1w2w3w4w5w6w7w8w9w10][cos⁢θ1sin⁢θ1cos⁢θ2sin⁢θ2Δ⁢T100Δ⁢T1100110011001]Equation⁢4[vxvy]=diag[w1w2w3w4w5w6w7w8w9w10][Range⁢Rate1⋮⋮xk+1-xkyk+1-yk⋮vx⁢1vy⁢1vx⁢2vy⁢2vx⁢3vy⁢3]

The term (vx1, vx1) of Equation 3 assumes that the track has the same motion (curvature) as the vehicle102. The velocity calculated based on the assumption that the track is moving at radial direction is (vx2, vy2). The velocity calculated based on the assumption that the track is moving at cross-radial direction is (vx3, vy3). The terms (vx1, vy1), (vx2, vy2) and (vx3, vy3) are only used as regularization. In practice, these terms may have small weightages when compared to other information. Practical values of these weights w3n-1, w3n, w3n+1, w3n+2, w3n+3, w3n+4are approximately one tenth of the smallest weights of any other information.

Equation 4 shows data from two consecutive cycles. The weights are the inverse of variances and can be obtained from sensor specifications or data mining. Equation 4 shows that solving different hypotheses is equivalent to setting different weights to zero while keeping the others. For example, the Hypothesis 2 listed above is equivalent to zeroing the terms w5, w6, w7, w8, w9, and w10in Equation 4, while setting the terms w1, w2as the inverse of the range rate variance, and while computing the terms w3, w4based on range and azimuth angle variances. For

w1=w2=1σrr2,
where σrr2is the range rate variance from sensor specification. For w3and all weights related to change of longitudinal position, it can be calculated by the inverse of the sum of longitudinal position variances of 2 consecutive scans. For

w3=1σxk2+σxk+12,σxk2=σθ2*yk+(cos⁡(θk))2*σr2+σp2.
Where σθ2is the azimuth angle variance from sensor specification, σr2is the rate variance from sensor specification, σp2a is a tunable constant representing the uncertainty of a scattering center of the radar point cloud. Same for w4and all weights related to change of lateral position, it can be calculated by the inverse of the sum of lateral position variances of 2 consecutive scans. For

w4=1σyk2+σyk+12,σyk2=σθ2*xk+(sin⁡(θk))2*σr2+σp2.

Velocity of each hypothesis can be obtained by solving Equation 3. Position can also be estimated using the velocity solved from Equation 3, and the Equations 5 and 6, which assume a nearly constant velocity during the scanning period (e.g., four or more consecutive cycles for each hypothesis).

{xesti=xi+Δ⁢Ti→4⁢vxyesti=yi+Δ⁢Ti→4⁢vyEquation⁢5{xest=Σi⁢xest⁢_⁢i4yest=Σi⁢yest⁢_⁢i4Equation⁢6

At306, a track error associated with each of the hypotheses is determined. For example, the initializer120determines, for each of the multiple hypotheses212, a first associated level of evidence supporting the initial velocity of that hypothesis. The level of evidence is based on the track error scores, which are computed as accumulated position and range rate errors with a point or points from the point cloud sensor data114associated with one of the tracks116. The track error scores can be determined from Equation 7. In Equation 7, Δxris the position error in range direction (e.g., azimuth angle), Δxois the position error in the direction perpendicular to the range direction, Δ{dot over (r)} is the range rate error between the hypothesis and the related point or points from the point cloud sensor data114. If multiple points from the point cloud sensor data114correlate to a track, an average position and range rate can be used. βr, βo, βrrare the weights for difference errors.
Scoreerr=βrΔxr+βoΔxo+βrrΔ{dot over (r)}Equation 7
At this step, track error score is calculated using the latest radar point cloud.

At308, another hypothesis is associated with the hypotheses by fusing all the hypothesis initially created into one, fused hypothesis. For example, the initializer120generates a fused hypothesis by combining the multiple hypotheses212into one. An initial velocity and a first associated level of evidence of the fused hypothesis may be based on an aggregate of the initial velocities and the first associated levels of evidence of the multiple hypotheses. The initializer120then includes the fused hypothesis among the multiple hypotheses212.

For instance, all seven of the Hypotheses 1 to 7 can be fused to create a new hypothesis, which is referred to as Hypothesis 8. Following Equations 8 and 9, the Hypothesis 8 is determined. In Equation 8, the term Ximhis i-th hypothesis generated at302, and in Equation 9, the weight for that hypothesis is computed as the evidence E. The term XFin Equation 8 is the fused hypothesis (e.g., Hypothesis 8) that results from summation of the weighted hypotheses.

XF=∑iEi⁢Xim⁢hEquation⁢8E=1ScoreerrEquation⁢9
Following step308the hypothesis initialization period is ended, and all eight hypotheses are generated with initial positions, initial velocities, and initial track error scores.
Hypotheses Tracking Period (e.g., Steps310Through324)

The hypotheses tracking period includes steps310through324, and by definition, is a time period after the hypothesis initialization period and before the track is set to a mature status and finally initialized. Prior to finally initializing and outputting a new track among the tracks116, which are mature, a new track is tracked similarly, but not reported among the tracks116, until the new track is stable and ready for reporting.

At310, a best hypothesis from the hypotheses is selected to estimate a velocity measurement for a track. A temporary track state for the new track that enables tracking during this time may be set to the best hypothesis so far (e.g., the hypothesis with a lowest track error score). For instance, to enable tracking until maturity, the initializer120initializes a velocity measurement of each new track before it can enter a hypotheses tracking period for that track. Responsive to including the fused hypothesis among the multiple hypotheses212, the initializer120selects, based on the first associated levels of evidence for the multiple hypotheses212, a first best hypothesis.

At312, the hypotheses are time updated. During the hypotheses tracking period, each hypothesis may be tracked using a Kalman Filter with a constant-velocity motion model, which enables the initializer120to solve each hypothesis for a current cycle time of the hypotheses tracking period. The initializer120time updates the initial velocity of each of the multiple hypotheses212.

At314, the track score associated with each of the hypotheses is updated. During the hypotheses tracking period, data association is performed for the track to determine the track error scores for the hypotheses. For example, prior to accumulating the track error score for each hypothesis, data association between the track and an associated point or points from the point cloud sensor data114can be used to compute the track error score of each hypothesis using Equation 7. For instance, the initializer120determines, for each of the multiple hypotheses212, a second associated level of evidence supporting the updated initial velocity of that hypothesis.

At316, any of the hypotheses that do not satisfy an evidence threshold are terminated. For example, the initializer120eliminates from the multiple hypotheses212any having values for the second associated level of evidence that do not satisfy an evidence threshold. The evidence threshold includes an evidence ratio computed for each of the multiple hypotheses. The evidence ratio computed for each of the multiple hypotheses can be a unique evidence ratio among all the multiple hypotheses. The evidence ratio for each hypothesis can be determined from Equation 10:

ρE⁢i=EiΣn⁢EnEquation⁢10

If the evidence ratio ρEifor a hypothesis is smaller than an evidence threshold the initializer120terminates the hypothesis. The evidence threshold may be determined using Equation 11:

ρth=0.2⁢5(Nvalid-1)Equation⁢11
In Equation 11, Nvalidrepresents the number of hypotheses still active (i.e., not terminated) during the current cycle time of the hypotheses tracking period. If none of the hypotheses has an evidence ratio smaller than the evidence threshold ρth, then the initializer120can attempt to terminate a single hypothesis with a smallest evidence ratio, however, this may be conditioned on its evidence ratio being smaller than 2*ρth. If termination of a hypothesis with a smallest evidence ration is not appropriate, no hypothesis is terminated in the current cycle time of the hypotheses tracking period.

At318, whether there is enough evidence from the hypotheses to initialize the track is determined. In other words, the initializer120checks whether sufficient evidence exists to finally initialize the track using one of the hypotheses. The initializer120may use one or more criteria for checking whether the evidence for any of the hypotheses is sufficient to finalize the velocity measurement initialization. One criterion may be that the hypotheses tracking period is ended. The duration of the hypotheses tracking period may depends on a specific application, however, because this period adds delay in the track initialization, a suitable duration may be different from one vehicle or implementation to the next. An example, duration is equivalent to eight radar scans in a twenty hertz system. Another criterion may be that there are only two hypotheses still active; all but two hypotheses are terminated up to now. A third criterion can be that there are only three hypotheses alive and all of them have evidence ratio ρEilarger than 0.3, which indicates that all three hypotheses produce similar results. The process300arrives at324, when at318, it is determined that sufficient evidence exists (e.g., at least one criteria is satisfied), and the process300carries on to320, when at318, it is determined that insufficient evidence exists (e.g., none of the criteria is satisfied).

At320, the remaining hypotheses are measurement updated. For example, responsive to determining that more than two remain in the multiple hypotheses212, after eliminating any having values for the second associated levels of evidence that do not satisfy the evidence threshold, the initializer120measurement updates the predicted movement of the object for each of the remaining multiple hypotheses212.

As indicated above at312, during the hypotheses tracking period, each hypothesis may be tracked using a Kalman Filter with a constant-velocity motion model, which enables the initializer120to solve each hypothesis for a current cycle time of the hypotheses tracking period. At320, each hypothesis is measurement updated using the information derived from the point cloud sensor data114, which is associated to the track.

At322, which is similar to step310, a best hypothesis from the hypotheses is selected to be used as the track state for the new track to further enable tracking during the hypotheses tracking period. This temporary track state enables the data association to occur during the hypotheses tracking period. For example, responsive to determining that more than two remain in the multiple hypotheses212, after eliminating any having values for the second associated levels of evidence that do not satisfy the evidence threshold, the initializer120selects, based on the second associated levels of evidence for the remaining multiple hypotheses212, a second best hypothesis to replace the first best hypothesis selected previously.

At324, the best hypothesis is used to initialize the velocity measurement for the track. If the conditions at318indicate the evidence is sufficient, the track state will be initialized using the best hypothesis. For example, responsive to determining that only two hypotheses remain in the multiple hypotheses212after eliminating any having values for the second associated levels of evidence that do not satisfy the evidence threshold, the initializer120selects, based on the second associated levels of evidence for the two hypotheses that remain, a third best hypothesis to replace the first or second best hypothesis selected previously.

During the hypotheses tracking period, the new track is measurement updated and track errors are recomputed, until the new track reaches maturity, when it is then reported among the tracks116. A flag may prevent the new track from being included and output among the tracks116. When the flag indicates that the new track is initialized and otherwise ready for inclusion in an output from the perception system104, the new track is added to the tracks116. At324, the object tracker118and/or the initializer120may change the flag to indicate the new track is mature and ready for inclusion among the tracks116. For example, the object tracker118may cause the perception system104to output, for the vehicle-based systems210, the track to the object including a velocity parameter initialized to the initial velocity of the third best hypothesis.

Example Results

FIG.4illustrates a graph400of velocity measurement initializations during a time period of stable radar track velocity initialization using multiple hypotheses. The results of the graph400come from performance testing a perception system, such as the perception system104, as an object makes a straight crossing path across a host vehicle travel path. In the graph400, lines402and404show a global positioning system (GPS) reported longitudinal and lateral velocity, respectively. The lines402and404represent a true velocity of the object. Lines410and412track the longitudinal and lateral velocity, respectively, when initialized using previous track velocity initialization techniques (i.e., the old way, without using multiple hypotheses). In contrast, lines406and408show distinct advantages from using multiple hypotheses to initialize velocity measurements (i.e., the new way). The lines406and408show the longitudinal and lateral velocities being tracked from using multiple hypotheses, which allows the track velocities to settle to values much closer to the true velocity (e.g., the lines402and404), in less time and/or with less variance, which causes the results to be more stable than the lines410and412. As the object travels somewhat perpendicular to the travel path of the host vehicle, comparing the lines406and408relative to the lines410and412shows that the example perception system is able to achieve velocity track initialization as fast or nearly as fast as other velocity initialization techniques, but with much higher accuracy relative to the lines402and404.

Further Examples

Some further examples in view of the techniques described above include:

Example 1. A system comprising a processor configured to: obtain point cloud sensor data indicative of signal returns that reflect off of objects in an environment; and establish, using the point cloud sensor data, a track to an object in the environment including initializing a velocity measurement for the object by: creating multiple hypotheses for predicted movement of the object; determining, for each of the multiple hypotheses, an initial velocity and a first associated level of evidence supporting the initial velocity of that hypothesis; generating a fused hypothesis by combining the multiple hypotheses, an initial velocity and a first associated level of evidence of the fused hypothesis being based on an aggregate of the initial velocities and the first associated levels of evidence of the multiple hypotheses; including the fused hypothesis among the multiple hypotheses; and responsive to including the fused hypothesis among the multiple hypotheses, select, based on the first associated levels of evidence for the multiple hypotheses, a first best hypothesis; time update the initial velocity of each of the multiple hypotheses; determine, for each of the multiple hypotheses, a second associated level of evidence supporting the updated initial velocity of that hypothesis; eliminate from the multiple hypotheses any of the multiple hypotheses having values for the second associated level of evidence that do not satisfy an evidence threshold; responsive to determining that more than two hypotheses remain in the multiple hypotheses after eliminating any of the multiple hypotheses having values for the second associated levels of evidence that do not satisfy the evidence threshold: measurement update the predicted movement of the object for each of the remaining multiple hypotheses; and select, based on the second associated levels of evidence for the remaining multiple hypotheses, a second best hypothesis to replace the first best hypothesis selected previously; and responsive to determining that only two hypotheses remain in the multiple hypotheses after eliminating any of the multiple hypotheses having values for the second associated levels of evidence that do not satisfy the evidence threshold: select, based on the second associated levels of evidence for the two hypotheses that remain, a third best hypothesis to replace the first or second best hypothesis selected previously; and output, for a vehicle system, the track to the object including a velocity parameter initialized to the initial velocity of the third best hypothesis.

Example 2. The system of any other example, wherein the processor is further configured to, until only two hypotheses remain: time update the initial velocity of each of the multiple hypotheses; determine, for each of the multiple hypotheses, the second associated level of evidence supporting the updated initial velocity of that hypothesis; and eliminate from the multiple hypotheses any of the multiple hypotheses having values for the second associated level of evidence that do not satisfy the evidence threshold.

Example 3. The system of any other example, wherein the second best hypothesis to replace the first best hypothesis selected previously is selected further in response to determining a hypotheses tracking period ends.

Example 4. The system of any other example, wherein the hypotheses tracking period of time comprises multiple frames of a radar system from which the point cloud sensor data is obtained.

Example 5. The system of any other example, wherein the hypotheses tracking period of time comprises approximately fifteen frames of the radar system.

Example 6. The system of any other example, wherein the processor is configured to determine, for each of the multiple hypotheses, the first associated level of evidence supporting the initial velocity of that hypothesis by determining accumulated position and range rate errors of one or more points of the point cloud sensor data.

Example 7. The system of any other example, wherein the processor is configured to use a constant motion model to time update or to measurement update the multiple hypotheses.

Example 8. The system of any other example, wherein the evidence threshold comprises an evidence ratio computed for each of the multiple hypotheses.

Example 9. The system of any other example, wherein the evidence ratio computed for each of the multiple hypotheses comprises a unique evidence ratio among all the multiple hypotheses.

Example 10. The system of any other example, wherein the point cloud sensor data comprises point cloud radar data.

Example 11. A method, comprising: obtaining, by an object tracker and from a radar system, point cloud sensor data indicative of radar returns that reflect off of objects in an environment; and establishing, using the point cloud sensor data, a track to an object in the environment including initializing a velocity measurement for the object by: creating multiple hypotheses for predicted movement of the object; determining, for each of the multiple hypotheses, an initial velocity and a first associated level of evidence supporting the initial velocity of that hypothesis; generating a fused hypothesis by combining the multiple hypotheses, an initial velocity and a first associated level of evidence of the fused hypothesis being based on an aggregate of the initial velocities and the first associated levels of evidence of the multiple hypotheses; including the fused hypothesis among the multiple hypotheses; and responsive to including the fused hypothesis among the multiple hypotheses, selecting, based on the first associated levels of evidence for the multiple hypotheses, a first best hypothesis for initializing the velocity measurement for the object.

Example 12. The method of any other example, further comprising time updating the initial velocity of each of the multiple hypotheses; determining, for each of the multiple hypotheses, a second associated level of evidence supporting the updated initial velocity of that hypothesis; eliminating from the multiple hypotheses any of the multiple hypotheses having values for the second associated level of evidence that do not satisfy an evidence threshold; responsive to determining that more than two hypotheses remain in the multiple hypotheses after eliminating any of the multiple hypotheses having values for the second associated levels of evidence that do not satisfy the evidence threshold: measurement updating the predicted movement of the object for each of the remaining multiple hypotheses; and selecting, based on the second associated levels of evidence for the remaining multiple hypotheses, a second best hypothesis to replace the first best hypothesis selected previously.

Example 13. The method of any other example, further comprising time updating the initial velocity of each of the multiple hypotheses; determining, for each of the multiple hypotheses, a second associated level of evidence supporting the updated initial velocity of that hypothesis; eliminating from the multiple hypotheses any of the multiple hypotheses having values for the second associated level of evidence that do not satisfy an evidence threshold; responsive to determining that only two hypotheses remain in the multiple hypotheses after eliminating any of the multiple hypotheses having values for the second associated levels of evidence that do not satisfy the evidence threshold: selecting, based on the second associated levels of evidence for the two hypotheses that remain, a third best hypothesis to replace the first or second best hypothesis selected previously; and outputting, for a vehicle system, the track to the object including a velocity parameter initialized to the initial velocity of the third best hypothesis.

Example 14. The method of any other example, further comprising time updating the initial velocity of each of the multiple hypotheses; determining, for each of the multiple hypotheses, a second associated level of evidence supporting the updated initial velocity of that hypothesis; eliminating from the multiple hypotheses any of the multiple hypotheses having values for the second associated level of evidence that do not satisfy an evidence threshold; responsive to determining that more than two hypotheses remain in the multiple hypotheses after eliminating any of the multiple hypotheses having values for the second associated levels of evidence that do not satisfy the evidence threshold: measurement updating the predicted movement of the object for each of the remaining multiple hypotheses; and selecting, based on the second associated levels of evidence for the remaining multiple hypotheses, a second best hypothesis to replace the first best hypothesis selected previously; and responsive to determining that only two hypotheses remain in the multiple hypotheses after eliminating any of the multiple hypotheses having values for the second associated levels of evidence that do not satisfy the evidence threshold: selecting, based on the second associated levels of evidence for the two hypotheses that remain, a third best hypothesis to replace the first or second best hypothesis selected previously; and outputting, for a vehicle system, the track to the object including a velocity parameter initialized to the initial velocity of the third best hypothesis.

Example 15. The method of any other example, wherein the second best hypothesis to replace the first best hypothesis selected previously is selected further in response to determining a hypotheses tracking period of time is expired.

Example 16. The method of any other example, wherein the hypotheses tracking period of time comprises multiple frames of the radar system.

Example 17. The method of any other example, wherein the evidence threshold comprises a unique evidence ratio computed for each of the multiple hypotheses.

Example 18. The method of any other example, wherein the object tracker is configured to determine, for each of the multiple hypotheses, the first associated level of evidence supporting the initial velocity of that hypothesis by determining accumulated position and range rate errors of an associated portion of the point cloud sensor data.

Example 19. The method of any other example, wherein the object tracker is configured to use a constant motion model to time update or to measurement update the multiple hypotheses.

Example 20

A computer-readable storage medium comprising instructions that, when executed, cause a processor to execute an object tracker configured to: obtain, from a radar system, point cloud sensor data indicative of radar returns that reflect off of objects in an environment; and establish, using the point cloud sensor data, a track to the object in an environment including initializing a velocity measurement for the object by: creating at least two hypotheses for predicted movement of the object; determining, for each of the at least two hypotheses, an initial velocity and a first associated level of evidence supporting the initial velocity of that hypothesis; generating a fused hypothesis by combining the at least two hypotheses, an initial velocity and a first associated level of evidence of the fused hypothesis being based on an aggregate of the initial velocities and the first associated levels of evidence of the at least two hypotheses; including the fused hypothesis among the at least two hypotheses; and responsive to including the fused hypothesis among the at least two hypotheses, selecting, based on the first associated levels of evidence for the at least two hypotheses, a single best hypothesis for initializing the velocity measurement for the object.

Example 21. A system comprising means for performing the method of any previous example.

Example 22. A system comprising a processor configured to perform the method of any previous example.

Example 23

A computer readable medium including instructions that, when executed, cause a processor to perform the method of any previous example.

CONCLUSION

While various embodiments of the disclosure are described in the foregoing description and shown in the drawings, it is to be understood that this disclosure is not limited thereto but may be variously embodied to practice within the scope of the following claims. From the foregoing description, it will be apparent that various changes may be made without departing from the scope of the disclosure as defined by the following claims. In addition to radar systems, problems associated with stable radar track velocity initialization can occur in other systems (e.g., image systems, lidar systems, ultrasonic systems) that identify and process tracks from a variety of sensors. Therefore, although described to improve radar tracking, the techniques of the foregoing description can be adapted and applied to other problems to effectively detect and track objects in a scene using other types of object trackers.

The use of “or” and grammatically related terms indicates non-exclusive alternatives without limitation unless the context clearly dictates otherwise. As used herein, a phrase referring to “at least one of” a list of items refers to any combination of those items, including single members. As an example, “at least one of: a, b, or c” is intended to cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiples of the same element (e.g., a-a, a-a-a, a-a-b, a-a-c, a-b-b, a-c-c, b-b, b-b-b, b-b-c, c-c, and c-c-c or any other ordering of a, b, and c).