# Radio Frequency Fingerprinting for Frequency Hopping Emitter Identification
## 5 Experimental Results and Discussion
### Outlier Detection Performance  
We evaluated the outlier detection performance of the proposed algorithm and baseline 3. The experimental dataset was prepared using the test dataset for trained emitters and the outlier dataset for outlier emitters. Before executing the experiment, the detector-related parameters, that is, the temporal scaling factor \(T_{s}\) and detection threshold \(\lambda\), were optimized. For \(T_{s}\in[1,2,3,4,5,10,15,20]\) and \(0\leq\lambda\leq 0.3\), the parameters were set to \(T_{s}=2\) and \(\lambda=0.07\) for the proposed algorithm and \(T_{s}=1\) and \(\lambda=0.05\) for proposed baseline 3. These values were selected by finding the minimum FPR when the TRP was higher than 95%.  
As discussed in Section 3.4, the key idea of the outlier detector is based on the fact that the maximum probability of the output vectors from the outlier samples has a smaller value than the maximum probability of the output vectors from the trained samples. To verify this idea, we plotted a histogram in Figure 14 showing the maximum probabilities of the output vectors obtained from the proposed method, i.e., the output vectors of the ensemble approach in the DIN. Evidently, the maximum probability values of the outlier samples occur at positions \(<0.1\). Conversely, the values of trained samples mostly exist at position \(\geq 0.2\). These results demonstrate that the differences between the characteristics of the outliers and trained samples are easily identified and can be utilized to detect the outlier samples.  
We present the confusion matrices of the outlier detectors based on the proposed method and baseline 3 in Tables 6 and 7. As we optimized our parameters based on the FPR values when the TPR was higher than 95.0%, both TPRs yielded similar rates in the detection of the actual trained samples. However, in the case of the true negative ratio, which represents the actual outlier sample detection ability, the proposed method can achieve a rate of 95.6%, which is 6.6% higher than that of baseline 3 (89.0%). In other words, the proposed method can reduce the FPR from 11.0% to 4.4%. These results indicate that  
Figure 14: Histogram of the output vectors.  
Figure 13: PLCP frame format for FHSS networks in the 802.11 standard [10].  
the DIN classifier-based approach is useful for training SF features in FH signals and can effectively detect outlier samples by using these trained features.  
Figure 15 plots the ROC curve and compares the AUROCs. As was done for the previously presented results in Section 5, the values were averaged over 10 experiments. The ROC metric describes the relationship between the probability of detection (i.e., TPR) and the probability of a false alarm (i.e., FPR). This result can be achieved by plotting the FPR together with the TPR at different detector thresholds \(\lambda\). Additionally, this ROC metric is known as the cost-benefit relationship in decision theory. Thus, when a high benefit is obtained at a low cost, i.e., when the probability of false alarms is low, high detection rates should be obtained. In other words, if the curve moves toward the upper left with a high AUROC, the model possesses strong detection ability. The results confirm that the proposed method can clearly improve the ROC curve compared with baseline 3. The AUROC also improves from 0.97 to 0.99. These results provide clear evidence that the proposed DIN-based ensemble method is more effective than the residual block-based method.