Abstract:
This invention is related to a Doppler-Vision-Radar Traffic Surveillance System comprising of multiple Doppler radars, circuitry for processing radar signals, and data recording and displaying devices. Although the system is mainly designed for roadside traffic surveillance, it can be used in different applications, such as mounted on a host vehicle or on a UAV. The system will provide continuous surveillance of all incoming and leaving traffic.

Description:
TECHNICAL FIELD 
       [0001]    This invention relates to a Doppler-vision-radar traffic surveillance system. 
       BACKGROUND OF THE INVENTION 
       [0002]    Doppler Radar Based Traffic Surveillance Systems: A traditional radar based traffic surveillance system uses a Doppler radar for vehicle speed monitoring which measures a vehicle speed at line-of-sight (LOS). In  FIG. 1 , the speed of an approaching (or a leaving) vehicle is calculated in terms of Doppler frequency f D  by 
         [0000]    
       
         
           
             
               
                 
                   
                     v 
                     t 
                   
                   = 
                   
                     
                       f 
                       D 
                     
                     
                       K 
                        
                       
                           
                       
                        
                       
                         cos 
                          
                         
                           ( 
                           
                             φ 
                             t 
                           
                           ) 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   1 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where K is a Doppler frequency conversion constant and φ t  is the angle between the vehicle velocity vector ν t , and the LOS. Although an advantage of a Doppler radar based system is its long detection range, there are several difficulties associated with the traditional radar based system: (1) the Doppler radar beam angle is too large to precisely locate vehicles within the radar beam, i.e., no precise line-of-sight (LOS) angular information of the moving vehicle is available; (2) the angle between the vehicle velocity vector and the LOS, φ t , is unknown and therefore, needs to be small enough for a reasonable speed estimation accuracy; (3) since all velocity vectors on the equal-Doppler cone in  FIG. 1  generate the same speed, the Doppler radar cannot differentiate the vehicles with the same speed but moving in different directions defined by the same equal-Doppler cone. Therefore, precise vehicle speed and location information cannot be derived in a traditional Doppler radar based traffic surveillance system. 
         [0003]    This invention overcomes the shortcoming of the traditional Doppler radar based system lacking the information of the LOS angle and the angle between the vehicle velocity vector and the LOS by using multiple Doppler radars with a special configuration to obtain the precise information of the LOS angle and the angle between the vehicle velocity vector and the LOS. The reason this patent is called “Doppler-Vision-Radar Traffic Surveillance System” is because it uses two moving radars with a specially designed motion pattern to obtain the moving vehicles&#39; LOS angle information which normally can only be obtained by a vision system. The precise angle between the vehicle velocity vector and the LOS is used to calculate the vehicle speed and the precise LOS angle is used in the Doppler-Vision-Radar Traffic Surveillance System to pinpoint the moving vehicles. 
       SUMMARY 
       [0004]    A Doppler-vision-radar traffic surveillance system to monitor traffic may include a first movable Doppler radar to generate a first radar beam along the direction of a first motion ray, a second movable Doppler radar to generate a second radar beam along the direction of a second motion ray, a third fixed Doppler radar to generate a third radar beam along a direction ray, a data processing device to process Doppler radar information, a tracking device to continuously point the surveillance system to the moving vehicle, and a recording device to continuously record the complete information of the moving vehicle. 
         [0005]    The surveillance system may find the line-of-sight (LOS) angle of a moving vehicle by finding a velocity vector perpendicular to the LOS vector. 
         [0006]    The surveillance system may find the velocity vector perpendicular to the LOS vector by scaling the motion vector of one moving radar and subtracting the motion vector of the other moving radar from the scaled motion vector. 
         [0007]    The surveillance system may scale the motion vector of one moving radar by multiplying it with a ratio of Doppler differences. 
         [0008]    The surveillance system may find the Doppler differences by subtracting the Doppler of the fixed radar from the Doppler of two moving radars. 
         [0009]    The surveillance system may find the vehicle heading angle information, which us the same as road structure information, using a training procedure. 
         [0010]    The surveillance system may use radar data from multiple vehicles in the training procedure. 
         [0011]    The surveillance system may find the vehicle speed information by jointly using three radars. 
         [0012]    The surveillance system may track the moving vehicle by continuously pointing to the vehicle using the vehicle LOS angle information. 
         [0013]    The surveillance system may record the moving vehicle speed and LOS angle information onto a recording device. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0014]    The invention may be understood by reference to the following description taken in conjunction with the accompanying drawings, in which, like reference numerals identify like elements, and in which: 
           [0015]      FIG. 1  illustrates the speed measurement of an approaching vehicle and a leaving vehicle with a Doppler radar; 
           [0016]      FIG. 2  illustrates the operational setup of the surveillance system; 
           [0017]      FIG. 3  illustrates the lay out of the surveillance system; 
           [0018]      FIG. 4  illustrates the functional flow chart of the surveillance system; and 
           [0019]      FIG. 5  illustrates three radar configuration and the two moving radar motion pattern. 
       
    
    
     DETAILED DESCRIPTION 
       [0020]    While the term “traffic surveillance” is used herein, it may also refer to other traffic applications, such as “traffic monitoring”, etc. The invention discussed here may be applied to the case of more than three radars. 
         [0021]    A Doppler-vision-radar traffic surveillance system is shown in  FIG. 2  where  1 —the sensor system which may include a sensor suite/recording device or apparatus,  2 —a target tracking device,  3 —a first moving Doppler radar motion ray,  4 —a second moving Doppler radar motion ray, and  5 —a radar direction ray connecting the sensor apparatus  1  to a moving vehicle  6 . 
         [0022]      FIG. 3  shows the layout of the sensor apparatus  1  where  7 —a first moving Doppler radar,  8 —a second moving Doppler radar,  9 —a fixed or stationary Doppler radar,  10 —a data processing device, such as a computer, laptop, personal computer, PDA or other such device, and  11 —data recording device, such as a hard drive, a flash drive or other such device. 
         [0023]    The functional flow chart of the system is shown in  FIG. 4 . In the following, we will describe the functional blocks. 
       1. Analog-to-Digital Conversion (ADC) for Doppler Radars 
       [0024]    Doppler radars illustrated in this patent are continuous wave (CW) radars. Analog-to-digital conversion (ADC) may be performed in  101 ,  102  and  103  to convert analog signals to digital signals. If digital signals are directly available from the radars, this ADC step may be skipped. 
       2. Calculate Doppler Frequency of the Moving Vehicle 
       [0025]    Assume the current time is k in discrete time. The Doppler frequencies of the moving vehicle  p ,  6  in  FIG. 3 , induced by both moving Doppler radars may be given by (steps  104  and  106  in  FIG. 4 ) 
         [0000]        f   D     k     1   =K   1 [ν tk  cos(φ tk )+ν r1k  cos(θ r1k )]  (1)
 
         [0000]      and 
         [0000]        f   D     k     2   =K   2 [ν tk  cos(φ tk )+ν r2k  cos(θ r2k )].  (2)
 
         [0000]    where K 1  and K 2  may be Doppler conversion constants for the first and second moving Doppler radars ( 7  and  8  in  FIG. 3 ), and θ r1k , θ r2k  and θ tk  are depicted in  FIG. 3  without time index k. A fixed Doppler radar  9  may be used to sense the moving vehicle motion (step  105  in  FIG. 4 ) 
         [0000]        f   D     k     3   =K   3 ν tk  cos(φ tk )  (3)
 
         [0000]    where K 3  may be the Doppler conversion constant for the fixed Doppler radar ( 9  in  FIG. 3 ). 
       3. Calculate Doppler Differences 
       [0026]    In steps  107  and  108  of  FIG. 4 , since all three radars  7 ,  8 ,  9  may be located together and assuming that the distance from the sensor suite to the moving vehicle  6  may be much larger than the distance between radars  7 ,  8 ,  9 , the following Doppler differences may be obtained as 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       Δ 
                        
                       
                           
                       
                        
                       
                         f 
                         
                           D 
                           k 
                         
                         13 
                       
                     
                     = 
                     
                       
                         
                           
                             f 
                             
                               D 
                               k 
                             
                             1 
                           
                           
                             K 
                             1 
                           
                         
                         - 
                         
                           
                             f 
                             
                               D 
                               k 
                             
                             3 
                           
                           
                             K 
                             3 
                           
                         
                       
                       = 
                       
                         
                           v 
                           
                             r 
                              
                             
                                 
                             
                              
                             1 
                              
                             
                                 
                             
                              
                             k 
                           
                         
                          
                         
                           cos 
                            
                           
                             ( 
                             
                               θ 
                               
                                 r 
                                  
                                 
                                     
                                 
                                  
                                 1 
                                  
                                 
                                     
                                 
                                  
                                 k 
                               
                             
                             ) 
                           
                         
                       
                     
                   
                    
                   
                     
 
                   
                    
                   and 
                 
               
               
                 
                   ( 
                   4 
                   ) 
                 
               
             
             
               
                 
                   
                     Δ 
                      
                     
                         
                     
                      
                     
                       f 
                       
                         D 
                         k 
                       
                       23 
                     
                   
                   = 
                   
                     
                       
                         
                           f 
                           
                             D 
                             k 
                           
                           2 
                         
                         
                           K 
                           2 
                         
                       
                       - 
                       
                         
                           f 
                           
                             D 
                             k 
                           
                           3 
                         
                         
                           K 
                           3 
                         
                       
                     
                     = 
                     
                       
                         v 
                         
                           r 
                            
                           
                               
                           
                            
                           2 
                            
                           
                               
                           
                            
                           k 
                         
                       
                        
                       
                         cos 
                          
                         
                           ( 
                           
                             θ 
                             
                               r 
                                
                               
                                   
                               
                                
                               2 
                                
                               
                                   
                               
                                
                               k 
                             
                           
                           ) 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   5 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where the impact of the moving vehicle may have been removed. Eqs. (4) and (5) may actually recover the substantially independent motion Doppler signals of the first and second moving Doppler radars  7 ,  8 , except for the conversion constants. 
       4. Calculate Doppler Ratio 
       [0027]    In step  109 , the Doppler ratio may be calculated as 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       Δ 
                        
                       
                           
                       
                        
                       
                         f 
                         
                           D 
                           k 
                         
                         13 
                       
                     
                     
                       Δ 
                        
                       
                           
                       
                        
                       
                         f 
                         
                           D 
                           k 
                         
                         23 
                       
                     
                   
                   = 
                   
                     
                       
                         
                           v 
                           
                             r 
                              
                             
                                 
                             
                              
                             1 
                              
                             
                                 
                             
                              
                             k 
                           
                         
                          
                         
                           cos 
                            
                           
                             ( 
                             
                               θ 
                               
                                 r 
                                  
                                 
                                     
                                 
                                  
                                 1 
                                  
                                 
                                     
                                 
                                  
                                 k 
                               
                             
                             ) 
                           
                         
                       
                       
                         
                           v 
                           
                             r 
                              
                             
                                 
                             
                              
                             2 
                              
                             
                                 
                             
                              
                             k 
                           
                         
                          
                         
                           cos 
                            
                           
                             ( 
                             
                               θ 
                               
                                 r 
                                  
                                 
                                     
                                 
                                  
                                 2 
                                  
                                 
                                     
                                 
                                  
                                 k 
                               
                             
                             ) 
                           
                         
                       
                     
                     . 
                   
                 
               
               
                 
                   ( 
                   6 
                   ) 
                 
               
             
           
         
       
     
       5. Scale Doppler Vector of Moving Doppler Radar Two 
       [0028]    In step  110 , the Doppler vector of moving radar two may be scaled as 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         v 
                         _ 
                       
                       _ 
                     
                     
                       r 
                        
                       
                           
                       
                        
                       2 
                        
                       
                           
                       
                        
                       k 
                     
                   
                   = 
                   
                     
                       
                         Δ 
                          
                         
                             
                         
                          
                         
                           f 
                           
                             D 
                             k 
                           
                           13 
                         
                       
                       
                         Δ 
                          
                         
                             
                         
                          
                         
                           f 
                           
                             D 
                             k 
                           
                           23 
                         
                       
                     
                      
                     
                       
                         
                           v 
                           _ 
                         
                         
                           r 
                            
                           
                               
                           
                            
                           2 
                            
                           
                               
                           
                            
                           k 
                         
                       
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   7 
                   ) 
                 
               
             
           
         
       
     
         [0000]    6. Subtract the Doppler Vector of Moving Radar One from the Scaled Doppler Vector 
         [0029]    In step  111 , the Doppler vector of moving radar one may be subtracted from the scaled Doppler vector of moving radar two 
         [0000]        ν   r12k =   θ     r12k − ν   r1k .  (8)
 
       7. Find the Directional Vector Pointing to Moving Vehicle 
       [0030]    In step  112  of  FIG. 4 , the direction vector  n   k  pointing to the moving vehicle perpendicular to  ν   r12k  may be found, and a pointing direction angle α k  (step  113  in  FIG. 4 ) may be calculated. This pointing direction angle is the LOS angle of the moving vehicle. 
       8. Road Structure Learning 
       [0031]    From  FIG. 5 , the following may be obtained: 
         [0000]      φ tk =α k −δ k .  (9)
 
         [0000]    Eq. (3) may be rewritten as 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           f 
                           
                             D 
                             k 
                           
                           3 
                         
                         = 
                         
                           
                             K 
                             3 
                           
                            
                           
                             v 
                             tk 
                           
                            
                           
                             cos 
                              
                             
                               ( 
                               
                                 
                                   α 
                                   k 
                                 
                                 - 
                                 
                                   δ 
                                   k 
                                 
                               
                               ) 
                             
                           
                         
                       
                     
                   
                   
                     
                       
                         = 
                         
                           
                             
                               λ 
                               1 
                             
                              
                             
                               cos 
                                
                               
                                 ( 
                                 
                                   α 
                                   k 
                                 
                                 ) 
                               
                             
                           
                           + 
                           
                             
                               λ 
                               2 
                             
                              
                             
                               sin 
                                
                               
                                 ( 
                                 
                                   α 
                                   k 
                                 
                                 ) 
                               
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   10 
                   ) 
                 
               
             
           
         
       
     
         [0000]    where λ 1  and λ 2  may be 
         [0000]      λ 1   =K   3 ν tk  cos(δ k ),λ 2   =K   3 ν tk  sin(δ k ).  (11)
 
         [0000]    By modeling the vehicle&#39;s kinematics with a constant velocity model and the road structure with a straight line model (step  115  in  FIG. 4 ), Eq. (11) may become 
         [0000]      λ 1   =K   3 ν t  cos(δ),λ 2   =K   3 ν t  sin(δ).  (12)
 
         [0000]    We may assume that the vehicles may follow the road lane markings and the vehicle&#39;s heading angle (λ k  in  FIG. 5 ) may reflect the road structure which may be learnt from the traffic flow (step  114  in  FIG. 4 ). 
         [0032]    With a collection of N Doppler frequencies of the fixed Doppler radar, a least square approach may be used to calculate {circumflex over (λ)} 1  and {circumflex over (λ)} 2  using Eq. (10). The road structure may be calculated by (steps  114  of  FIG. 4 ) 
         [0000]    
       
         
           
             
               
                 
                   
                     δ 
                     ^ 
                   
                   = 
                   
                     
                       
                         tan 
                         
                           - 
                           1 
                         
                       
                        
                       
                         ( 
                         
                           
                             λ 
                             2 
                           
                           
                             λ 
                             1 
                           
                         
                         ) 
                       
                     
                     . 
                   
                 
               
               
                 
                   ( 
                   13 
                   ) 
                 
               
             
           
         
       
     
         [0033]    Note: A different moving vehicle heading direction may result in different signs of angles in Eq. (9). 
       9. Calculate Vehicle Speed 
       [0034]    Once the road structure {circumflex over (δ)} is learnt, accurate vehicle speed {circumflex over (ν)} tk  may be calculated from (step  116  in  FIG. 4 ) 
         [0000]        f   D     k     3   =K   3 ν tk  cos({circumflex over (α)} k −{circumflex over (δ)} k ).  (14)
 
       10. Continuously Vehicle Pointing 
       [0035]    Once the pointing direction angle, {circumflex over (α)} k , is known, the target tracking device ( 2  in  FIG. 2  and step  117  in  FIG. 4 ) may continuously point to the vehicle to maintain continuous surveillance. 
       11. Data Recording 
       [0036]    Each vehicle&#39;s speed {circumflex over (ν)} tk  and direction angle {circumflex over (α)} k  are continuously recorded in the data recording device ( 11  in  FIG. 3  and step  118  in  FIG. 4 ). 
         [0037]      FIG. 5  shows the motion pattern of two moving Doppler radars whose motion vectors,  ν   r1k  and  ν   r2k , are defined by angles, θ 1  and θ 2 , along the radar motion rays,  3  and  4 .  FIG. 5  also shows the relationship between the scaled motion vector,    ν     r2k  and its perpendicular vector,  ν   r12k , and the relationship between the perpendicular vector and the moving vehicle direction vector,  n   k , and the vehicle LOS angle, α k . 
         [0038]    Note: This patent application is in reference to the following patent applications of both inventors: Application Numbers 12255081 and 12266227. Patent Applications 12255081 is for 3D imaging where it uses three radars and one video camera and requires the sensor suite to move with a known motion. Patent Applications 12266227 requires also three radars and a video camera and precise registration between the radars and the camera is needed. This patent application is also in reference to the following patent application of the first inventor: Application Number 12333735, where it also requires three radars and one video camera and a fusion algorithm of radar and video signals is presented. This invention uses only three radars and no cameras, but recovers the same information as a camera.