Abstract:
A method of associating targets from at least two object detection systems. An initial prior correspondence matrix is generated based on prior target data from a first object detection system and a second object detection system. Targets are identified in a first field-of-view of the first object detection system based on a current time step. Targets are identified in a second field-of-view of the second object detection system based on the current time step. The prior correspondence matrix is adjusted based on respective targets entering and leaving the respective fields-of-view. A posterior correspondence matrix is generated as a function of the adjusted prior correspondence matrix. A correspondence is identified in the posterior correspondence matrix between a respective target of the first object detection system and a respective target of the second object detection system.

Description:
BACKGROUND OF INVENTION 
       [0001]    An embodiment relates to object sensor fusion. 
         [0002]    Vision-imaging systems are used in vehicles for enhancing applications such as object detection systems and other vision/positioning systems. Such systems utilize a camera to capture the image and then extract objects from the image. Such objects may be other vehicles, pedestrians, or even objects within a road of travel. 
         [0003]    Radar systems are also used to detect objects within the road of travel. Radar systems utilize radio waves to determine the range, altitude, direction, or speed of objects. A transmitter transmits pulses of radio waves which bounce off any object in their path. The pulses bounced off the object returns a tiny part of the wave&#39;s energy to a receiver which is typically located at the same location as the transmitter. 
         [0004]    Detecting objects by cooperatively utilizing vision-imaging systems and radar systems would add confidence as to the detection and position of an object in the path of travel of a vehicle. Data associated or data correspondence between different modalities may be readily performed when the vehicle is traveling along a flat, non-pitched road; however, if the road of travel is not flat or the road pitch changes abruptly due to the structure of the road or due to harsh braking, then fusion of the data from complementary sensors of different modalities becomes increasingly difficult. 
       SUMMARY OF INVENTION 
       [0005]    An advantage of an embodiment is the integral merging of data from two separate object detection systems which reduces offsets between the two object detections system so that the positions of each may be merged for providing greater confidence as to the objects&#39; positions in the road of travel. The technique described herein offer robust and accurate association of data from the different modalities. The technique provides special handling targets leaving and entering the field-of-view of each modality such that assumptions may be made as to the correspondence between the respective targets within the captured field-of-views. Moreover, the data association technique described herein may be utilized as a subsystem to a fusion architecture technique that fuses data from different sensors through target tracking 
         [0006]    An embodiment contemplates a method of associating targets from at least two object detection systems. An initial prior correspondence matrix is generated based on prior target data from a first object detection system and a second object detection system. Targets are identified in a first field-of-view of the first object detection system based on a current time step. Targets are identified in a second field-of-view of the second object detection system based on the current time step. The prior correspondence matrix is adjusted based on respective targets entering and leaving the respective fields-of-view. A posterior correspondence matrix is generated as a function of the adjusted prior correspondence matrix. A correspondence is identified in the posterior correspondence matrix between a respective target of the first object detection system and a respective target of the second object detection system. 
     
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         [0007]      FIG. 1  is a schematic illustration of a vehicle equipped with object detection systems. 
           [0008]      FIG. 2  is an exemplary image captured by a vehicle driven on a flat and straight road surface. 
           [0009]      FIG. 3  is an exemplary image captured by a vehicle traveling along the curved and pitched road surface. 
           [0010]      FIG. 4  is an exemplary target correspondence graph between a radar system and a vision system. 
           [0011]      FIG. 5  is a block diagram illustrating the general concept of a target association technique. 
           [0012]      FIG. 6  is a flowchart of a method for the recursive target association technique. 
           [0013]      FIG. 7  is a block diagram of a fusion technique incorporating the recursive target association technique. 
       
    
    
     DETAILED DESCRIPTION 
       [0014]    There is shown in  FIG. 1  a vehicle  10  equipped with a first object detection system  12  and a second object detection system  14 . Each system utilizes a different type of sensor to detect objects. Each sensor within each respective system is able to detect and track multiple targets in its respective field-of-view. For the exemplary purposes herein, the first object detection system  12  includes a vision-based system and the second object detection system  14  includes a radar-based system. It should be understood that the respective systems are exemplary and that any two or more different object detection systems may be used. 
         [0015]    A field-of-view for the vision system  12  is shown generally at  16 . A set of targets detected by the vision system  12  is identified as V 1 , V 2 , and V 3 . Each respective target identified by vision system  12  includes measurements such as longitudinal displacement, lateral displacement, longitudinal velocity, and lateral velocity. 
         [0016]    A field-of-view of the radar system  14  is shown generally at  18 . A set of targets detected by the radar system  14  is identified as R 1 , R 2 , and R 3 . Each respective target identified by vision system  12  includes measurables such as longitudinal displacement, lateral displacement, longitudinal velocity, and lateral velocity. 
         [0017]    The idea is to determine a correspondence between radar targets and vision targets in their common field-of-view.  FIG. 1  shows an example of a correspondence between the radar targets and the vision targets V 1 →R 1 , V 2 →R 2 , and V 3 →R 3 . Since the measurables constantly change due to change of vehicle direction, speed, and occlusion with other vehicles, the respective targets must be tracked by each respective detection system at each instant of time. 
         [0018]    Accurate association of data from different modalities is significant for target tracking, however, data association is a challenging task. Targets of vision systems in certain situations, such as a non-flat road surface, curvatures, and abrupt pitch angle changes of the host vehicle (e.g., caused by either the road or harsh braking) may result in a deviation from their true value significantly. Therefore, a distance between the vision targets may be significantly large. A commonly known technique for the corresponding problem is a technique known as the nearest neighbor matching. In the nearest neighbor matching, for every radar target, each radar target is assigned to a vision target having the nearest distance to it. The issue with the nearest matching method is that the decision is based only on current data. 
         [0019]    The nearest neighbor matching method fails for scenarios shown in  FIGS. 2-4 .  FIG. 2  illustrates an image of a vehicle  17  driven on a flat and straight road surface  19  with no pitch changes at a time t 1 . Under such driving conditions, there is significant correspondence between the vision targets and radar targets in the time displaced images if the road of travel continued as a flat and straight road surface with no pitch changes. 
         [0020]      FIG. 3  illustrates an image of the vehicle  17  traveling along the road of travel at a later time. As shown in the image, the road  19  includes pitch changes as well as curves. In addition to changes in the position of a vehicle captured within the image, the range to the target captured by the vision system becomes unpredictable. As shown in the graph illustrated in  FIG. 4 , a significant distance between radar target measurements and vision target measurements are cause by the non-flat and curved road surface. The y-axis represents distance and the x-axis represents time. The tracking of targets by the vision system is represented by  20  and the tracking of the target by the radar system is represented by  22 . As shown in  FIG. 4 , the correspondence of the target between the vision system and the radar system is significant at time t 1 . That is, the target as determined by the vision system and the radar system are measured at a substantially same longitudinal range. As time elapses, correspondence between the target tracked by the radar system and the vision system deteriorates and the correspondence becomes increasingly uncertain. This is the result of the road transitioning from a straight and flat road surface to a pitched and non-flat road surface. Correspondence of the target between the vision system and radar system gains confidence only after a time period elapses from t 2  (i.e., when the road of travel transitions to a consistent road surface). As shown in  FIG. 4 , there is a lapse in association in targets from time period t 1  and t 2 . 
         [0021]    The technique described herein overcomes the deficiencies of target association between radar systems and vision systems. To find a correspondence between radar targets and vision targets a Boolean matrix A of size M×N is maintained at each time step. M is the number of vision targets whereas N is the number of radar targets. The Boolean matrix A is updated at each step of time for re-determining which respective radar targets are associated with which respective vision targets. To identify a correspondence between a respective vision target V j  and a respective radar target R k , a match between a respective vision target V j  and a respective radar target R k  is identified when a correspondence element a jk =1. It is assumed that each vision target only corresponds to at most one object. Therefore, for each N column in the Boolean matrix A, each column is summed and the largest summation is identified for a corresponding match between a respective vision target and a respective radar target. A formula for identifying whether there is a correspondence between a vision target and a radar target is represented as follows: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       ∑ 
                       
                         k 
                         = 
                         1 
                       
                       N 
                     
                      
                     
                       a 
                       jk 
                     
                   
                   = 
                   
                     { 
                     
                       
                         
                           1 
                         
                       
                       
                         
                           0 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   1 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where the sum is equal to 0 if V j  is an outlier, and equal to 1 otherwise. 
         [0022]    It is first assumed that there is a same set of targets in a sequence of captured images detected for the radar system and vision system. It is also assumed a tracker exists in each individual sensor. Based on target identity (i.e., the signal of a target identifier), historical matrices A(t−1), A(t−2), . . . , A(1), at time step t−1, t−2, . . . , 1, respectively, are known. 
         [0023]    A rigid 2D transformation T is defined which is regarded as a transformation from a radar coordinate frame to the vision coordinate frame. [T(R k )] is defined as the radar target position at the vision coordinate frame. The radar targets (R 1 , R 2 , . . . , R N ) are modeled as Gaussian mixture model (GMM) distribution with parameters {R k |k=1, . . . , N} where R k  represents the measurement of location of the radar (i.e., longitudinal and lateral displacement of targets). The formula for the GMM with parameters can be represented by the formula: 
         [0000]    
       
         
           
             
               
                 
                   
                     p 
                      
                     
                       ( 
                       v 
                       ) 
                     
                   
                   = 
                   
                     
                       ∑ 
                       
                         k 
                         = 
                         1 
                       
                       
                         N 
                         + 
                         1 
                       
                     
                      
                     
                       
                         π 
                         k 
                       
                        
                       
                         p 
                          
                         
                           ( 
                           
                             v 
                             | 
                             k 
                           
                           ) 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where p(v|k) is the 2D Gaussian distribution, i.e., 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       p 
                        
                       
                         ( 
                         
                           v 
                           | 
                           k 
                         
                         ) 
                       
                     
                     = 
                     
                       
                         
                           1 
                           
                             2 
                              
                             π 
                              
                             
                                 
                             
                              
                             
                               σ 
                               2 
                             
                           
                         
                          
                         
                           exp 
                           ( 
                           
                             - 
                             
                               
                                 
                                    
                                   
                                     v 
                                     - 
                                     
                                       T 
                                        
                                       
                                         ( 
                                         
                                           R 
                                           k 
                                         
                                         ) 
                                       
                                     
                                   
                                    
                                 
                                 2 
                               
                               
                                 2 
                                  
                                 
                                   σ 
                                   
                                     2 
                                      
                                     
                                         
                                     
                                   
                                 
                               
                             
                           
                           ) 
                         
                          
                         
                             
                         
                          
                         for 
                          
                         
                             
                         
                          
                         k 
                       
                       = 
                       1 
                     
                   
                   , 
                   … 
                    
                   
                       
                   
                   , 
                   N 
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where v is the random variable of a possible vision target location, and σ is a standard deviation. 
         [0024]    In another embodiment p(v|k) is the 2D Gaussian distribution, i.e., 
         [0000]    
       
         
           
             
               
                 p 
                  
                 
                   ( 
                   
                     v 
                     | 
                     k 
                   
                   ) 
                 
               
               = 
               
                 
                   
                     1 
                     
                       2 
                        
                       
                         πσ 
                         x 
                       
                        
                       
                         σ 
                         y 
                       
                     
                   
                    
                   
                     exp 
                      
                     
                       ( 
                       
                         
                           - 
                           
                             
                               
                                  
                                 
                                   x 
                                   - 
                                   
                                     x 
                                     ′ 
                                   
                                 
                                  
                               
                               2 
                             
                             
                               2 
                                
                               
                                 σ 
                                 x 
                                 2 
                               
                             
                           
                         
                         - 
                         
                           
                             
                                
                               
                                 y 
                                 - 
                                 
                                   y 
                                   ′ 
                                 
                               
                                
                             
                             2 
                           
                           
                             2 
                              
                             
                               σ 
                               y 
                               2 
                             
                           
                         
                       
                       ) 
                     
                   
                    
                   
                       
                   
                    
                   for 
                    
                   
                       
                   
                    
                   k 
                 
                 = 
                 1 
               
             
             , 
             … 
              
             
                 
             
             , 
             N 
           
         
       
     
         [0000]    where v=(x,y) is the random variable of a vision target location; x and y longitudinal and lateral offsets of the vision target, respectively; x′ and y′ are longitudinal and lateral offsets of the k-th projected radar target T(R k ) in the vision system frame; and σ x  and σ y  are standard deviation in longitudinal and lateral directions, respectively. In this embodiment, several specific characteristics of vision system such as rough longitudinal measurement but accurate azimuth angle measurement can be modeled. 
         [0025]    Given a vision target at range r, the standard deviation is set as σ x =αr and σ y =βr, respectively, where the constants α and β are determined by the performance specification of the vision system. 
         [0026]    Next a component is modeled for noise or outliers as a uniform distribution which is represented by the formula: 
         [0000]    
       
         
           
             
               
                 
                   
                     p 
                      
                     
                       ( 
                       
                         x 
                         | 
                         
                           N 
                           + 
                           1 
                         
                       
                       ) 
                     
                   
                   = 
                   
                     1 
                     Γ 
                   
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where Γ is the area of vision field-of-view. A coefficient for the GMM is represented by the formula: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       π 
                       k 
                     
                     = 
                     
                       
                         1 
                         - 
                         w 
                       
                       N 
                     
                   
                   , 
                 
               
               
                 
                   ( 
                   5 
                   ) 
                 
               
             
           
         
       
     
         [0000]    for k=1, . . . , N and π N+1 =w where w denote the probability of a target being an outlier. The vision targets (V 1 , V 2 , . . . , V M ) are viewed as independent and identically distributed samples of the GMM. The correspondence matrix can be treated as a Boolean random matrix. Each realization A represents the correspondences between radar and vision targets. This is verified as follows: 
         [0000]        ā   jk   =E ( A   jk )= p ( A   jk =1).  (6)
 
         [0000]    ā jk  is the correspondence prior distribution which is usually set to be non-informative at time step  0 . This assumes that each vision target is assigned to every radar target at equal probability at time step  0 . This is represented by the formula: 
         [0000]        ā   jk =π k  for all j.  (7)
 
         [0000]    Given the prior ā jk , the posterior â jk , which is the resulting correspondence, can be computed. After the vision targets are observed V=(V 1 , V 2 , . . . , V M ) the posterior â jk  is represented by the following formula: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         a 
                          
                       
                       jk 
                     
                     = 
                     
                       
                         p 
                          
                         
                           ( 
                           
                             
                               A 
                               jk 
                             
                             = 
                             
                               1 
                               | 
                               V 
                             
                           
                           ) 
                         
                       
                       = 
                       
                         
                           
                             
                               a 
                               _ 
                             
                             jk 
                           
                           · 
                           
                             p 
                              
                             
                               ( 
                               
                                 
                                   V 
                                   j 
                                 
                                 | 
                                 k 
                               
                               ) 
                             
                           
                         
                         
                           
                             ∑ 
                             
                               k 
                               = 
                               1 
                             
                             
                               N 
                               + 
                               1 
                             
                           
                            
                           
                             
                               
                                 a 
                                 _ 
                               
                               jk 
                             
                             · 
                             
                               p 
                                
                               
                                 ( 
                                 
                                   
                                     V 
                                     j 
                                   
                                   | 
                                   k 
                                 
                                 ) 
                               
                             
                           
                         
                       
                     
                   
                   , 
                   
                     
 
                   
                    
                   
                     
                       for 
                        
                       
                           
                       
                        
                       j 
                     
                     = 
                     1 
                   
                   , 
                   … 
                    
                   
                       
                   
                   , 
                   M 
                   , 
                   
                     k 
                     = 
                     1 
                   
                   , 
                   … 
                    
                   
                       
                   
                   , 
                   N 
                 
               
               
                 
                   ( 
                   8 
                   ) 
                 
               
             
           
         
       
     
         [0000]    wherein Â=[â jk ] is the posterior probability of matrix, p(V j |k) is the probability that the j-th vision target V j  is associated with the k-th target R k , defined in Eq. (3), and 1/Σ k=1   N+1 ā jk ·p(V j |k) is the normalization factor to make p(A jk =1|V) a probability distribution. 
         [0027]    As a result, the vision target V j  is assigned to the k j −th radar target where k j =arg max k  (â jk ). It should be noted that if k j =N+1, then the vision target is an outlier and is not assigned to any radar target. 
         [0028]      FIG. 5  illustrates a block diagram illustrating the general concept of the target association technique. Detected radar target data  24  and vision target data  26  are input to a Bayesian framework model for data association inference at time t 1 . Also input to the Bayesian framework model  30  is the prior correspondence matrix (Ā)  28  determined at a last time period. A recursive data association technique is applied to the input data for generating a posterior correspondence matrix (Â)  32 . As a result, the posterior correspondence matrix (Â)  32  that is generated for the current time period is utilized as the prior correspondence at the next determination stage. 
         [0029]      FIG. 6  illustrates flowchart of a method for the recursive target association technique. In step  40 , at time step zero, the prior correspondence matrix is initialized as a non-informative correspondence matrix. This treats all correspondence as equal at the initiation of routine since there are no pre-assumptions of any relationships between any vision targets and any radar targets. This is represented as ā jk =π k , for all j=1, . . . , M, and k=1, . . . , N+1. 
         [0030]    In step  41 , a determination is made if a new frame of data has arrived from each of the sensing systems. A data frame includes data from both the radar system (R 1 , R 2 , . . . , R N ) and the vision system. (V 1 , V 2 , . . . , V M ). If the determination is made in step  41  that a new frame of data has not arrived, then the routine continues to check for the arrival of a new data frame from both the vision systems and the radar systems. If the determination is made in step  41  that new frame of data has arrived, then the routine proceeds to step  42 . 
         [0031]    In steps  42 , the data frame is analyzed for determining whether a target leaves or enters the field-of-view of either the radar sensor or the vision sensor and adjusts the prior correspondence matrix Ā. 
         [0032]    In step  43 , the data frame is analyzed for determining whether a respective vision target j showed up in the in current target list, but does not show up in the previous target list. If the respective vision target j showed up in the current target list, but is not showing up in the previous target list, then the respective row (i.e., respective vision target j) is initialized as non-informative. This is represented as ā j′k =π k  for all k, where j′ is the new added row index. 
         [0033]    In step  44 , the data frame is analyzed for determining whether a respective radar target k showed up in the previous target list, but does not show up in the current target list. If the respective radar target k showed up in the previous target list, but is not showing up in the current target list, then the respective column (i.e., respective radar target k) is removed from the prior correspondence matrix (Ā). 
         [0034]    In step  45 , the data frame is analyzed for determining whether a respective vision target j showed up in the previous target list, but does not show up in the current target list. If the respective vision target j showed up in the previous target list, but is not showing up in the current target list, then the respective row (i.e., respective vision target j) is removed from the prior correspondence matrix (Ā) 
         [0035]    In step  46 , the data frame is analyzed for determining whether a respective radar target k showed up in the in current target list, but does not show up in the previous target list. If the respective radar target k showed up in the current target list, but is not showing up in the previous target list, then a respective zero-value column is appended in the correspondence matrix (Ā). 
         [0036]    In step  47 , the posterior matrix is computed. The posterior correspondence matrix (Â) is determined using the formula shown in eq. (8) for all j and k. 
         [0037]    In step  48 , assignments relating to target correspondence are determined based on the posterior correspondence matrix (Â). That is, the column having the largest value closest to 1 is assigned the target correspondence between a respective radar target and a respective vision target (e.g., 
         [0000]    
       
         
           
             
               ∑ 
               
                 k 
                 = 
                 1 
               
               N 
             
              
             
               
                 a 
                  
               
               jk 
             
           
         
       
     
         [0000]    is equal to 1 or 0). 
         [0038]    In step  49 , the assignments of every vision target to a radar target are output. An index list (k 1 , k 2 , . . . , k m ) is outputted corresponding to vision targets (V 1 , V 2 , . . . , V m ). Each element in the index list represents the radar target index to which the vision target is assigned. The j-th vision target V j  is assigned to the k j -th radar target and k j =arg max k  (â jk ). 
         [0039]    In step  50 , sensor registration is performed. Sensor registration includes removing error and bias of data from the different sets of sensors. This includes minimizing the target matching error between two sensors through estimating a 2D rigid transformation. The transformation between the two sensors is recursively estimated by using the prior value to derive updated value at every time step. This is performed by updating a 2D rigid transformation from T to where T is the past transformation and is the updated transformation. The rigid transformation is determined by the following formula: 
         [0000]        T ′=(1−η) T+ηT*  
 
         [0000]    where T* is the current estimate of the transformation based on the matching between the radar targets and vision targets and is computed by the formula 
         [0000]    
       
         
           
             
               T 
               * 
             
             = 
             
               arg 
                
               
                   
               
                
               
                 
                   min 
                   T 
                 
                  
                 
                   ( 
                   
                     
                       
                         a 
                          
                       
                       jk 
                     
                      
                     
                       
                         
                            
                           
                             
                               V 
                               j 
                             
                             - 
                             
                               T 
                                
                               
                                 ( 
                                 
                                   R 
                                   k 
                                 
                                 ) 
                               
                             
                           
                            
                         
                         2 
                       
                       
                         2 
                          
                         
                           σ 
                           
                             2 
                              
                             
                                 
                             
                           
                         
                       
                     
                   
                   ) 
                 
               
             
           
         
       
     
         [0000]    and the predefined small learning factor (η) is 0&lt;η&lt;1. 
         [0040]    A solution for the transformation T* as shown above may be derived utilizing three parameters: t x , t y , ε, of the past transformation. Parameter t x  represents the x-offset from the radar frame to the vision frame, parameter t y  represents the y-offset from the radar frame to the vision frame, and ε represents the angular displacement offset from the radar frame to the vision frame. The transformation parameters are defined by the following two equations: 
         [0000]    
       
      
       x′=x−εy+t 
       x  
      
     
         [0000]        y′=εx−y+t   y . 
         [0000]    The above formulas transform a point (x, y) to the new position (x′, y′). 
         [0041]    It is assumed that V j =(x′ j , y′ j ) and R k =(x k , y k ). x′ j  and y′ j  are the longitudinal and lateral displacements of the j-th vision targets. x k  and y k  are longitudinal and lateral displacements of the k-th radar targets. T* can then be computed. T*=(t* x , t* y , ε*) and transformation parameters used to derive T* can be computed as follows: 
         [0000]    
       
         
           
             
               
                 t 
                 x 
                 * 
               
               = 
               
                 
                   
                     ∑ 
                     
                       j 
                       , 
                       k 
                     
                   
                    
                   
                     
                       
                         a 
                          
                       
                       
                         j 
                         , 
                         k 
                       
                     
                      
                     
                       ( 
                       
                         
                           
                             x 
                             j 
                             ′ 
                           
                           + 
                         
                         ∈ 
                         
                           
                             y 
                             k 
                           
                           - 
                           
                             x 
                             k 
                           
                         
                       
                       ) 
                     
                   
                 
                 
                   
                     ∑ 
                     
                       j 
                       , 
                       k 
                     
                   
                    
                   
                     
                       a 
                        
                     
                     
                       j 
                       , 
                       k 
                     
                   
                 
               
             
             , 
             
               
 
             
              
             
               
                 t 
                 y 
                 * 
               
               = 
               
                 
                   
                     ∑ 
                     jk 
                   
                    
                   
                     
                       
                         a 
                          
                       
                       jk 
                     
                     ( 
                     
                       
                         
                           y 
                           j 
                           ′ 
                         
                         - 
                       
                       ∈ 
                       
                         
                           x 
                           k 
                         
                         - 
                         
                           y 
                           k 
                         
                       
                     
                      
                     
                         
                     
                     ) 
                   
                 
                 
                   
                     ∑ 
                     
                       j 
                       , 
                       k 
                     
                   
                    
                   
                     a 
                     
                       j 
                       , 
                       k 
                     
                   
                 
               
             
             , 
             
               
 
             
              
             
               
                 ∈ 
                 * 
               
                
               
                 = 
                 
                   
                     
                       
                         ∑ 
                         jk 
                       
                        
                       
                         
                           
                             
                               a 
                                
                             
                             jk 
                           
                            
                           
                             ( 
                             
                               
                                 y 
                                 j 
                                 ′ 
                               
                               - 
                               
                                 t 
                                 y 
                               
                               - 
                               
                                 y 
                                 k 
                               
                             
                             ) 
                           
                         
                          
                         
                           x 
                           k 
                         
                       
                     
                     + 
                     
                       
                         ( 
                         
                           
                             x 
                             j 
                             ′ 
                           
                           - 
                           
                             t 
                             x 
                           
                           - 
                           
                             x 
                             k 
                           
                         
                         ) 
                       
                        
                       
                         y 
                         k 
                       
                     
                   
                   
                     
                       ∑ 
                       
                         j 
                         , 
                         k 
                       
                     
                      
                     
                       
                         
                           a 
                            
                         
                         
                           j 
                           , 
                           k 
                         
                       
                        
                       
                         [ 
                         
                           
                             x 
                             k 
                             2 
                           
                           + 
                           
                             y 
                             k 
                             2 
                           
                         
                         ] 
                       
                     
                   
                 
               
             
             , 
           
         
       
     
         [0000]    wherein Â=[â jk ] is the posterior matrix, y′ j  is a lateral displacement of the j-th vision target, ε angular displacement offset, y k  is the lateral displacement of the k-th radar target, x′ j  is the longitudinal displacement of the j-th vision target and x k  is the longitudinal displacement of the k-th radar target. 
         [0042]    In step  51 , the prior correspondence matrix at the next time step Ā t+1 =[ā jk ] is set to the posterior correspondence matrix at the current time step Â t =[â jk ] where ā jk =â jk , for all j and k. That is, the computed posterior correspondence matrix at time step t is utilized as input as the prior correspondence matrix for computing the next posterior correspondence matrix for the next time step t+1. 
         [0043]    A return is made to step  41  for awaiting a new frame of data and recursively updating the correspondence matrix. 
         [0044]      FIG. 7  illustrates a block diagram of an alternative embodiment integrating the recursive target association technique into a target fusion algorithm. 
         [0045]    In block  60 , a first object detection system (e.g., radar) senses for objects within the first object detection system&#39;s field-of-view. Similarly, in block  61 , a second object detection system (e.g., vision) senses for objects within the second object detection system&#39;s field-of-view. 
         [0046]    In block  62 , targets are identified within the first object detection system&#39;s field-of-view. For example, the targets relate to objects captured by the radar detection system. 
         [0047]    In block  63 , targets are identified within the second object detection system&#39;s field-of-view. For example, the targets relate to objects captured by the vision detection system. 
         [0048]    Both the radar targets  62  and the vision targets  63  are regarded as independent and identical distribution samples. 
         [0049]    The data association technique may be used to associate targets from the radar detection system and vision detection system directly as described in  FIG. 6 . However, the integration technique as described herein utilizes a third variable for integration with the radar target respectively, and utilizes the third variable for integration with the vision target respectively. Thereafter, the results from both integrations are fused. 
         [0050]    In block  64 , the feedback data as it relates to fused targets from a motion model is utilized as inputs for data association technique with radar targets, respectively, and with vision targets, respectively. The feedback data from the target motion model is identified as a predicted fusion target based on its last known position and motion analysis. Details of the predicted fusion target will be discussed in detail later. 
         [0051]    In block  65 , detected radar targets  62  and the predicted fusion targets  64  are provided as inputs to a first data association module that utilizes the recursive target association technique. Data association as described in  FIG. 6  is applied on the radar targets  62  and the predicted fusion targets  64  for generating a first posterior correspondence matrix. 
         [0052]    In block  66 , detected vision targets  63  and the predicted fusion targets  64  are provided as inputs to a second data association module that utilizes the recursive target association technique. Data association as described in  FIG. 6  is applied on the vision targets  63  and the predicted fusion targets  64  for generating a second posterior correspondence matrix. 
         [0053]    In block  67 , the outputs from the recursive target association technique in block  65  and the outputs from the recursive target association technique  66  is provided to a fusion system that utilizes a target tracking using sensor fusion. The fusion system fuses data from each posterior correspondence matrix for estimating the location of a given target. A fusion technique that can be used in block  67  is described in U.S. Pat. No. 7,460,951 having an issue date of Dec. 2, 2008, which is herein incorporated by reference in its entirety. 
         [0054]    In block  68 , the fusion targets are identified from the output of the fusion system in block  67 . The fusion targets are modeled as a Gaussian mixture model (GMM) distribution. 
         [0055]    In block  69 , a delay is applied to the fusion target data prior to proceeding to block  70 . 
         [0056]    In block  70 , a target motion model is applied to the fusion targets. The target motion model utilizes the fusion target data and predicts a future position of the targets based on the last known position and a trajectory of the target. Motion tracking involves locking onto a target and following the object through multiple frames and predicting a path of the target based on the trajectory of the target. In each fusion target, a longitudinal offset (x), a lateral offset (y), a longitudinal speed (v_x), and a lateral speed (v_y) of the target are estimated state variables. Therefore, if the current state of the target is known, then the future state variables at time dT late may be estimated using the following formulas: 
         [0000]    
       
      
       x′=x+dTv 
       — 
       x  
      
     
         [0000]    
       
      
       y′=y+dTv 
       — 
       y  
      
     
         [0000]    
       
      
       v 
       — 
       x′=v 
       — 
       x  
      
     
         [0000]    
       
      
       v 
       — 
       y′=v 
       — 
       y  
      
     
         [0000]    Therefore, the target motion model transforms a target&#39;s current state to its future state s′ as represented by the following formula: 
         [0000]        s′=F ( s )+ w    
         [0000]    where w is a random variable of appropriate dimension, typically with a zero mean and a covariance matrix Q. 
         [0057]    After a target motion model of the target is generated, a predicted position of the target is determined in block  64  and is used as an input to the data association techniques in blocks  65  and  66 . 
         [0058]    While certain embodiments of the present invention have been described in detail, those familiar with the art to which this invention relates will recognize various alternative designs and embodiments for practicing the invention as defined by the following claims.