# Making Automotive Radar Sensor Validation Measurements Comparable
## 2 Evaluation Methodology  
To answer the research questions, a methodology based on the so-called DVM introduced by [PERSON] [17] (pp. 118,119) is used. The corresponding process is visually illustrated in Figure 1. In the first step, the measurement data for the different experiments is collected. We gather multiple measurements per experiment setup. After pre-processing and position-based filtering of the measurement data, the number of resulting measurement data points is compared between the respective measurements or experiments. We name this filtered area region of interest (ROI). The comparison of the overall number is due to the condition that this deviation must be small for the DVM to be used [17] (p. 103). The DVM is based on a comparison of the EDFs of the measured data and, therefore, the underlying distribution. [PERSON]'s approach uses the area validation metric of [PERSON] et al. [18] (p. 307) and extends it by a second factor to evaluate the EDFs. In contrast to [PERSON], we do not summarize the individual distribution functions of the measurements, but evaluate the differences of the EDFs individually. Therefore, the area between all the resulting distribution functions is determined, which is a measure for the deviation of the mean value denoted as \(d_{\text{bias}}\) of the measurements and experiments. Therefore, the mathematical description for \(d_{\text{bias}}\) results with \(\mathbf{\mathcal{F}}\) denoted as the first EDF, \(\widetilde{\mathbf{\mathcal{F}}}\) denoted as the second EDF, \(d^{+}\) denoted as the area between \(\widetilde{\mathbf{\mathcal{F}}}\) and \(\mathbf{\mathcal{F}}\), where \(\widetilde{\mathbf{\mathcal{F}}}\geq\mathbf{\mathcal{F}}\) and \(d^{-}\) denoted as the area between \(\widetilde{\mathbf{\mathcal{F}}}\) and \(\mathbf{\mathcal{F}}\), where \(\widetilde{\mathbf{\mathcal{F}}}<\mathbf{\mathcal{F}}\), in  
\[d_{\text{bias}}(\mathbf{\mathcal{F}},\widetilde{\mathbf{\mathcal{F}}})=d^{-}-d^{+}. \tag{1}\]If subsequently the EDFs are shifted by \(d_{\text{bias}}\) and the resulting area between the corrected EDFs \(\widehat{\mathbf{\mathcal{F}}}_{\text{c}}\) and the first EDF \(\mathbf{\mathcal{F}}\) is calculated, a metric \(d_{\text{CAVM}}\) for the shape deviation of the distribution functions is obtained, giving information about the scattering difference [17] (pp. 116-119). Therefore, the following mathematical expression is obtained as  
\[d_{\text{CAVM}}(\mathbf{\mathcal{F}},\widetilde{\mathbf{\mathcal{F}}})=d_{\text{AVM}}( \mathbf{\mathcal{F}},\widetilde{\mathbf{\mathcal{F}}}_{\text{c}})=d_{\text{c}}^{-}+d_ {\text{c}}^{+}. \tag{2}\]  
These two parameters give information about the similarity of the different distribution functions in the unit of the analyzed measurand. In a final step, the distribution of \(d_{\text{bias}}\) and \(d_{\text{CAVM}}\) is visualized and analyzed via box plots. Five different evaluation levels are considered input data for the process:  
1. All range azimuth bins with the Doppler component 0 of the radar cuboid \(\rightarrow\) whole radar cuboid (WRC);
2. All detections in the dimensions distance, azimuth, and Radar Cross Section (RCS) \(\rightarrow\) all detections (WD);
3. A region of interest, which only includes the bins where the object is present in the measurement, on radar cuboid level with the Doppler component 0 \(\rightarrow\) ROI of the radar cuboid (ROI RC);
4. A region of interest, which only includes detections where the object is present in the measurement, on detection level \(\rightarrow\) ROI of the detecions (ROI D);
5. Each range azimuth bin combination separately on the radar cuboid level \(\rightarrow\) each bin of the radar cuboid (EB RC);  
The different evaluation levels are selected according to their relative deviation of the number of data points or detections of all measurements for the respective experiment. Furthermore, the additional information content plays a role in whether the evaluation is performed.