# Detecting Out-of-Distribution Samples in Complex IoT Traffic Based on Distance Loss
## 4 Experiments
### Results  
We evaluate our method on the open-source CIC IoT Dataset 2022, which is characterized by a large number of classes and highly imbalanced class distributions. The results, as shown in Table 3, demonstrate that our model significantly outperforms baseline methods in terms of OOD detection performance. Through repartitioning the feature space, designing an OOD-specific learning objective, and introducing a dedicated scoring function, our method substantially improves detection accuracy. It achieves AUROC and AUPR values above 0.95 and up to 0.99, while reducing FPR95 to as low as 0.03. We further evaluate our method using IoT Sentinel as the in-distribution dataset and the Shenzhen Grid traffic as the OOD dataset. The results, averaged over 10 runs, are presented in Table 4. Our method consistently outperforms the baseline methods, achieving an average AUROC of 0.9315--outperforming the second-best method by 0.24--and an FPR95 of 0.1770, representing a reduction of 0.5806 compared to the second-best method. These results demonstrate the effectiveness of our approach in real-world scenarios. However, we observe that the performance on the IoT Sentinel dataset is slightly lower compared to the CIC IoT Dataset 2022. This is mainly due to the high similarity of feature distributions among classes in the IoT Sentinel dataset, which leads to lower overall classification accuracy, as illustrated in Figure 5.  
As presented in Table 3, our method achieves significantly better performance compared to existing baselines across all OOD detection metrics. This improvement is closely related to both the characteristics of the dataset and the structure of the proposed framework. The CIC IoT 2022 dataset contains classes with relatively clear boundaries and distinguishable traffic patterns. This property allows the model to learn feature representations that are naturally more separable, which benefits OOD detection methods that rely on distributional differences rather than classifier outputs. The grouping mechanism contributes to this effect by aggregating classes with similar feature distributions into the same group. This reduces inter-class interference and transforms the original fine-grained classification task into a more robust group-level modeling process. The group-wise distance loss further refines this structure by encouraging tighter intra-group clustering and enlarging the margins between groups in the feature space. Consequently, OOD samples are more likely to fall into low-density regions, making them easier to distinguish from in-distribution instances. Recent studies support that feature-based methods often outperform classifier-based ones in complex or imbalanced settings, which aligns with our findings [31]. \begin{table}
\begin{tabular}{l c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{4}{c}{**CIC IoT Dataset 2022**} \\ \cline{2-5}  & **AUROC\(\uparrow\)** & **AUPR\(\uparrow\)** & **FPR95\(\downarrow\)** & **Inference Time (s)** \\ \hline MSP & 0.4918 & 0.8460 & 0.9798 & 3.3157 \\ AE & 0.3626 & 0.8536 & 0.9899 & 0.9031 \\ ODIN & 0.2889 & 0.8442 & 0.9603 & 2.0252 \\ Energy & 0.3317 & 0.8187 & 0.9477 & 0.7935 \\ Mahalanobis & 0.6027 & 0.8895 & 0.8005 & 2.9780 \\
**Ours** & \(\mathbf{0.9905\pm 0.0151}\) & \(\mathbf{0.9997\pm 0.0002}\) & \(\mathbf{0.03047\pm 0.0482}\) & 0.04324 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of OOD detection performance between our method and baseline methods on the CIC IoT Dataset 2022. All values represent the average performance when each class is individually considered as an OOD sample, as detailed in Section 4.1. For our method, we additionally report the standard deviation across multiple runs to assess performance stability. \(\uparrow\) indicates that higher values are better, while \(\downarrow\) indicates that lower values are preferred. The best-performing method is highlighted in bold. In the two experimental setups described above, we calculate the OOD detection F1 scores for various threshold selection strategies.