# Real-Time Hybrid Multi-Sensor Fusion Framework for Perception in Autonomous Vehicles
## 4 Hybrid Sensor Fusion Algorithm Overview
### Testing Obstacle Detection and Tracking using Kalman Filtering
#### 4.2.1 Radar Beat Signal Data Processing  
The radar sensor can detect obstacles and their states in a four step process as shown in Figure 9. The first step is to process the noisy digitized mixed or beat signal we receive from the radar. The beat signal is sent through an internal radar ADC to get converted to a digital signal. The digital signal is in  
Figure 8: Overview of the object detection and road segmentation architecture via a Fully Convolutional Neural Network. The tiles and colors indicate different layers of the FCNx and their type respectively. Encoder consists of: convolution (teal) and max pooling (green) layers. Decoder consists of: skip convolution (purple) and transposed convolution (yellow). The size and number of feature maps are shown at the top and bottom of each layer. the time domain, which comprises of multiple frequency components. In the second step, we use a one-dimensional (1D) Fast Fourier Transform (FFT), also known as 1 st stage or Range FFT, to transform the signal from the time domain to frequency domain, and separate all its frequency components. The output of the FFT is a frequency response represented by signal power or amplitude in dBm unit versus beat frequency in MHz unit. Each peak in the frequency response represents a detected target. The x-axis can be converted from beat frequency of the targets to range of the targets by:  
\[R=\frac{c\cdot T_{swcep}\cdot f_{b}}{2\cdot B_{swcep}} \tag{1}\]  
where \(R\) is the range of the target, \(c\) is the speed of light and \(f_{b}\) is the beat frequency of the target. The \(T_{swcep}\) and \(B_{swcep}\) are the chirp/sweep time and bandwidth respectively, which can be calculated by:  
\[T_{swcep}=5.5\cdot\frac{2\cdot R_{max}}{c} \tag{2}\]  
\[B_{swcep}=\frac{c}{2\cdot d_{res}} \tag{3}\]  
where \(R_{max}\) is the radar maximum range and the \(d_{res}\) is the range resolution of the radar. The constant 5.5 comes from assumption that the sweep time is often between 5-6 times the delay corresponding to the maximum radar range. As discussed, the range FFT output gives us the beat frequency, amplitude, phase, and range (from the equations above) of the targets. To measure the velocity (or Doppler velocity) of the targets, we need to find the Doppler frequency shift, which is the rate of change of phase across radar chirps. The target phase changes from one chirp to another. Therefore, after acquiring the range FFT on the radar chirps, in the third step, we run another FFT (Doppler FFT) to measure the rate of change of phase i.e., the Doppler frequency shift. The output of the Doppler FFT is a 3D map represented by signal power, range, and Doppler velocity. This 3D map is also referred to as Range Doppler Map (RDM). RDM gives us an overview of the targets range and velocity. Figure 9: Radar data processing pipeline. Plots from left to right: Noisy radar beat signal, output of the 1D-FFT showing a detection with its range, output of the 2D-FFT showing a detection with its range and Doppler velocity, and output of the 2-Dimensional Constant False Alarm Rate (2D-CFAR) filtering showing the detection with noises filtered out. RDM can be quite noisy since reflected radar signals received can be from unwanted sources like the ground, and buildings, which can create false alarms in our object detection task. In the fourth and final step, these noises or clutters are filtered out to avoid such false positives. One filtering method most used in automotive applications is Cell Averaging CFAR (CA-CFAR) [38] which is a dynamic thresholding method i.e., it varies thresholds based on the local noise level of the signal, see Figure 10. CA-CFAR slides a window across the 2D FFT output cells we generate.