Associating observations in a multi-sensor system using an adaptive gate value

In one aspect, a method to assign observations includes receiving first observations of a first sensor system, receiving second observations of a second sensor system and assigning a set of pairs of the first and second observations predicted to correspond to the same physical position. The assigning includes using a likelihood function that specifies a likelihood for each assigned pair. The likelihood is dependent on the assignment of any other assigned pairs in the set of assigned pairs. The assigning also includes determining the set of assigned pairs for the first and second observations based on the likelihood function. The likelihood function uses a gate value determined from estimating a true volume using nearest neighbor distances determined from the first and second observations.

BACKGROUND

In multi-sensor tracking systems a process of associating sets of observations from at least two sensor systems is problematic in the presence of bias, random errors, false observations, and missed observations. For example, one tracking system may share data with the other system and transmit its current set of its observations to the other sensor system. The second sensor system determines which data points received from the first sensor system correspond to the airplanes it is also tracking. In practice, the process is difficult because each sensor system has an associated error, making it problematic to assign a data point from one sensor system to the other sensor system. Examples of errors may include misalignment between the two sensor systems, different levels of tolerances, and/or one sensor system may observe certain types of airplanes while the other sensor system does not observe the same types of aircraft. Therefore, based on each particular sensor system having errors due to bias, random errors, false observations, and missed observations, it is difficult to use a set of data received from the first sensor system and directly overlay it with a set of data from the second sensor system.

SUMMARY

In one aspect, a method to assign observations includes receiving first observations of a first sensor system, receiving second observations of a second sensor system and assigning a set of pairs of the first and second observations predicted to correspond to the same physical position. The assigning includes using a likelihood function that specifies a likelihood for each assigned pair and determining the set of assigned pairs for the first and second observations based on the likelihood function. The likelihood function uses a gate value determined from estimating a true volume using nearest neighbor distances determined from the first and second observations.

In another aspect, an article includes a machine-readable medium that stores executable instructions to assign observations. The instructions cause a machine to receive first observations of a first sensor system, receive second observations of a second sensor system and assign a set of pairs of the first and second observations predicted to correspond to the same physical position. The instructions causing a machine to assign includes instructions causing a machine to use a likelihood function that specifies a likelihood for each assigned pair and determine the set of assigned pairs for the first and second observations based on the likelihood function. The likelihood function uses a gate value determined from estimating a true volume using nearest neighbor distances determined from the first and second observations.

In a further aspect, an apparatus includes circuitry to receive first observations of a first sensor system, receive second observations of a second sensor system and assign a set of pairs of the first and second observations predicted to correspond to the same physical position. The circuitry to assign includes circuitry to use a likelihood function that specifies a likelihood for each assigned pair and determine the set of assigned pairs for the first and second observations based on the likelihood function. The likelihood function uses a gate value determined from estimating a true volume using nearest neighbor distances determined from the first and second observations.

DETAILED DESCRIPTION

Referring toFIG. 1, a multi-sensor system10includes a sensor system12and a sensor system14each of which is configured to observe the position and velocity of objects (e.g., airplanes16). The multi-sensor system10also includes a multi-sensor processing system26. Observations may occur through sensing, prediction, or a combination of sensing and prediction. In one example, the sensor system12and the sensor system14are radar systems. In other examples, one or more of the sensor systems12,14may be an electro-optic sensor system, an infrared sensor system and so forth. For simplicity, not all airplanes16that are used herein are depicted inFIG. 1.

Associated with, or included within, the sensor system12is a representation18of positions of airplanes16that are observed by sensor system12. Sensor system14is associated with a similar representation20. Representations18and20may differ for a variety of reasons. For example, there may be a misalignment between the sensor systems12and14, errors between the sensor systems12and14, or one sensor system may not observe all of airplanes16that the other sensor system observes. The errors for either sensor system12or14may be due to bias, random errors, false observations, missed observations, or other sources. Representation18includes observations22representing positions of the corresponding airplanes16observed by the sensor system12. Representation20includes observations24representing positions of the corresponding airplanes16observed by the sensor system14. The physical position of any airplane16is an example of a physical parameter, observation of which is represented by observations22and24.

In one example, the sensor system12may provide the observations22to the sensor system14. A problem may arise with the sharing of observations when correlating which the observations24correspond to which the observations22. This may be particularly difficult in the situation where the sensor system12includes the observations22corresponding to airplanes16that the sensor system14does not also observe. Persistent bias within or between sensor systems12and14also makes this problem particularly difficult.

The observations22are assigned to observations24by calculating a likelihood function indicative of differences between observations22and observations values24using the multi-sensor processing system26.

Prior attempts to determine the likelihood function relied on Global Nearest Neighbor (GNN) techniques, but the GNN techniques do not take in to account the random and correlated bias components. On the other hand, a Global Nearest Pattern (GNP) technique determines a likelihood function that does account for the random and correlated bias components. For example, one such method to determine the likelihood function using a GNP technique is described in U.S. Pat. No. 7,092,924 which is assigned to the same assignee as this patent application and is incorporated herein in its entirety. An observation assignment using GNP accounts for persistent errors and the possibility that observations to be assigned may not constitute a perfect subset of the observation set to which they are being assigned.

Referring toFIG. 2, a data set28includes observations22, observations24, and assignments32corresponding to associations made between observations22and observations24predicted to correspond to the same physical parameter. In this embodiment, the assignments in the data set28account for both random and correlated bias components. The assignments32of data set28are made based on errors having both random and correlated bias components, generating a common pattern, and assigning observations22to observations24accordingly. In this example, a persistent offset exists between observations22and observations24, as indicated inFIG. 2.

The likelihood function of the GNP assumes that there are N objects (airplanes16in this example) in space being observed by sensor system A (e.g., sensor system12in the example ofFIG. 1) and sensor system B (e.g., sensor system14in the example ofFIG. 1), and each sensor system observes a potentially different subset of the objects. Sensor system A determines observations (observations22in the example ofFIG. 1) on m of these, sensor system B determines observations (observations24in the example ofFIG. 1) on n of these subject to:
m≦n≦N
False observations in sensor system A or sensor system B correspond to observations of a non-existent object. The observations are transformed to a D-dimensional common reference frame. The likelihood function in GNP is:

Ja=-(X_⁡(CMBias)-1⁢X_T)-∑i=1m⁢{δ⁢⁢Xi⁡(Si)-1⁢δ⁢⁢XiT+L⁢⁢N⁡(Si)-L⁢⁢N⁡(Cmin)G-L⁢⁢N⁢⁢(Cmin)}⁢⁢if⁢⁢{a⁡(i)≠0a⁡(i)=0}(1)
where α(i) is an association vector (i=1 to m) and is equal to an index of sensor system B object with which ith sensor system A object is associated (i.e., 0 for non-association). Siis a residual error covariance equal to CMA(i)+CMB(α(i)), where CMA(i) is a D×D error covariance matrix for sensor system A and CMB(α(i)) is a D×D error covariance matrix for sensor system B.Xis an estimate of the relative bias and

X_=[∑i=1m⁢(SVA⁡(i)-SVB⁡(a⁡(i))·(Si)-1]·[(CMBIAS)-1+∑i=1m⁢(Si)-1]-1
where each sum over those i for which a(i)≠0. SVA(i) is a D-dimensional state vector for sensor system A and SVB(α(i)) is a D-dimensional state vector for the sensor system B. δXiis a state vector difference where δXi=SVA(i)−SVB(α(i))−X. CMbiasis a D×D intersensor bias covariance matrix, G is a gate value. Cminis a minimum determinant of a residual error matrix where Cmin=MIN{1,MIN(|Si|)i} is a normalization constant.

In prior art approaches, the gate value, G, (also known as a cost of non-association) is determined using:

G=2⁢⁢L⁢⁢N⁡[βT⁢PAB(2⁢π)D2⁢βNA⁢βNB](2)
where D is the dimensionality of the data space, βTis a true object density, PAB=PAPBand is a true probability that any given object is observed by both sensor system A and sensor system B, BNA=PB(1−PA) βT+βFAand is a true density of sensor system A observations having no corresponding sensor system B observations, BNB=PA(1−PB)βT+βBand is a true density of sensor system B observations having no corresponding sensor system A observations, PAis a true probability that any given object is observed by the sensor system A, PBis a true probability that any given object is observed by the sensor system B, βFAis a true false observation density of sensor system A and βFBis a true false observation density of sensor system B.

However, the equation for the gate value G in Equation (2) cannot be used in a practical application since most of the parameters used to determine the gate value are truth parameters which are unknown. Therefore, as described further herein, the prior art gate value is discarded in favor of a more useful gate value. In order to determine a more practical gate value, the following assumptions and analysis is made.

βFAand βFBare set to zero, on the assumption that no false tracks are likely, and PAand PBare estimated, based on a priori information, to be P′Aand P′Brespectfully so that

βTis substituted with NT/VTwhere NTis the total number of truth objects and VTis the true volume in which the truth objects are randomly distributed. NTis approximated by NB/P′Bwhere NBis the number of objects observed by sensor system B so that

VTis the only unknown truth parameter, and is taken to be the volume over which the sensor B observations are distributed (VB). VBis estimated using the average, measured nearest neighbor distance of each observation of sensor B. Since the nearest neighbor distance does not seem to be calculable for mixed units (e.g., position and velocity) the total volume is represented as a product of position volume and velocity volume. Therefore, VTis KVVB—posVB—velwhere KVis a proportionality constant, VB—posis the position volume and VB—velis the velocity volume (i.e., the volume in velocity space over which the truth objects are randomly distributed) so that

By an analysis of nearest neighbor distance in multi-dimensional space it has been determined that the position and velocity volumes can be determined according to

VB_pos={NB⁡[dB_posK⁡(Dpos)]Dpos/21}⁢⁢for⁢⁢{Dpos>0Dpos=0},⁢andVB_vel={NB⁡[dB_velK⁡(Dvel)]Dvel/21}⁢⁢for⁢⁢{Dvel>0Dvel=0},
where K(D) is a constant equal to Γ(1+1/D)Γ1/D(D/2+1)/√{square root over (π)} (where Γ( ) is the standard Gamma Function), and Dposand Dvelare the number of position space and velocity space dimensions respectively. The average measured position space nearest neighbor distance, the average measured velocity space nearest neighbor distance for sensor B are calculated according to

dB_pos=(NB)-1⁢∑i=1NB⁢⁢d~B_pos⁡(i),⁢anddB_vel=(NB)-1⁢∑i=1NB⁢⁢d~B_vel⁢(i),
respectively, where dB—pos(i) is the measured nearest neighbor distance of ith sensor system B object in position space and equal to dB—pos(i)=Min{MAG[XB—pos(i)−XB—pos(i)]}i=1:n, and XB—pos(i) is the sensor B measured position of the ith object. dB—vel(i) is a measured nearest neighbor distance of ith sensor system B object in velocity space and equal to dB—vel(i)=Min {MAG[XB—vel(i)−XB—vel(i)]}i=1:nand XB—vel(i) is the sensor B measured velocity of the ith object.

KVis set equal to 1 based on an analysis of nearest neighbor distance and confirmed by simulation results. Thus, the gate value becomes

G=2⁢⁢LN[PB′·VB_pos·VB_vel[2⁢⁢π]D/2·[1-PA′]·[1-PB′]·NB](3)
where D=Dpos+Dvel. Note that the number of position space dimensions (Dpos) and the number of velocity space dimensions (Dvel) must either be equal to each other, or one must be zero.

Thus, using equation (3), the cost function in equation (1) is based on known parameters. In one example, P′Aand P′Bmay range from 50% to 100% with little affect on the results of the cost function. Applicants have determined that P′Aand P′Bof 70% is preferred.

In one example, the dimensionality parameter, D. is 6 for two radar system where three dimensions are position dimensions in x, y and z space and three dimensions are velocity dimensions in x, y and z space. In another example, when one of the sensor systems is an electro-optic system, D is equal to 2 for angle/angle dimensions (azimuth and elevation).

Referring toFIG. 3, in one example, a process to associate observations from multi-sensor systems is a process100. The multi-sensor processing system26receives observations from the sensor systems12,14(102). Based on the observations received, the multi-sensor processing system26determines parameters (104). For example, the multi-sensor processing system26determines state vectors, SVA(i) and SVB(α(i)), and error covariance matrices, CMA(α(i)) and CMB(α(i). The multi-sensor processing system26determines intersensor bias covariance matrix, CMbias(106). The multi-sensor processing system26determines the gate value (108). For example, equation (3) is used to determine the gate value. The multi-sensor processing system26determines associations (108). For example, the multi-sensor processing system26uses the parameters determined in processing blocks104,106and108in the cost function in equation (1) to determine an association vector.

Referring toFIG. 4, the multi-sensor processing system26may be configured as a multi-sensor processing system26′, for example. The multi-sensor processing system26′ includes a processor202, a volatile memory204and a non-volatile memory206(e.g., hard disk). The non-volatile memory206stores computer instructions214, an operating system210and data212. In one example, the computer instructions214are executed by the processor202out of volatile memory204to perform the process100.

Process100is not limited to use with the hardware and software ofFIG. 4; it may find applicability in any computing or processing environment and with any type of machine or set of machines that is capable of running a computer program. Process100may be implemented in hardware, software, or a combination of the two. Process100may be implemented in computer programs executed on programmable computers/machines that each includes a processor, a storage medium or other article of manufacture that is readable by the processor (including volatile and non-volatile memory and/or storage elements), at least one input device, and one or more output devices. Program code may be applied to data entered using an input device to perform process100and to generate output information.

The processes described herein are not limited to the specific embodiments described. For example, in the problem of associating sets of observations from multi-sensor systems in the presence of inter-sensor observation bias, random observations errors, false observations and missed observations, Global Nearest Pattern (GNP) association provides a preferred solution both for the association of one sensor's observations to those of the other (or to non-assignment), and for the unknown bias between the two sensors. Global Nearest Neighbor (GNN) association assumes no inter-sensor observation bias, and solves for an association of one sensor's observations to those of the other (or to non-assignment). In the absence of inter-sensor observation bias, GNN yields the same preferred association result as the GNP result. Thus, the gate value of equation 3 may be used by both GNN and GNP techniques.

In another example, the process100is not limited to the specific processing order ofFIG. 3, respectively. Rather, any of the processing blocks ofFIG. 3may be re-ordered, combined or removed, performed in parallel or in serial, as necessary, to achieve the results set forth above.