# Detecting Out-of-Distribution Samples in Complex IoT Traffic Based on Distance Loss
## 3 Proposed Approach
### Distance Loss Optimization Design  
Existing classification-based OOD detection methods primarily focus on classification objectives, typically using cross-entropy loss. However, such objectives tend to encourage overconfidence in ID data, leading to suboptimal OOD detection. For instance, as shown in Figure 4a, methods like MSP initially perform well but degrade with continued training due to overfitting [6; 7]. These issues are exacerbated in complex IoT scenarios with overlapping class features. Figure 4: OOD detection performance. (**a**) Variation in AUROC and FPR95 as training epochs increase. Performance first improves and then degrades, indicating overfitting in later training stages. (**b**) Comparison of different threshold selection strategies within our framework. The performance disparity highlights the need for an adaptive thresholding mechanism to ensure consistent OOD detection. To address this, we propose a group-level distance loss optimization strategy, which explicitly introduces a distance-based objective during training. Unlike conventional approaches that rely on feature spaces not specifically designed for OOD detection [13], our method enhances the structure of the learned feature space by guiding the model to increase inter-group separation and reduce intra-group variance. This promotes better feature discrimination and robustness to OOD samples [22]. Specifically, we partition the feature space at the penultimate layer based on the grouping results derived from class mean similarity. This partitioning allows each group to capture class-specific characteristics, minimizing feature overlap. Our loss function incorporates the following components:  
* Intra-group feature aggregation: Encourages features within a group to cluster around their group center, improving the intra-class compactness and ensuring that similar samples are grouped together. This reduces the variance within the group, leading to better classification performance for ID samples. * Inter-group feature separation: Maximizes the minimum distance between different group centers to reduce confusion between groups, ensuring that the feature representations of distinct classes are well-separated. This enhances the discriminability of in-distribution samples and increases the margin between ID and OOD instances, making it easier to identify OOD samples. We retain the standard cross-entropy loss to preserve ID classification performance while integrating the distance-based loss to strengthen OOD detection. The optimization objective is formulated as:  
\[\mathcal{L}=\mathcal{L}_{\text{CE}}+\mathcal{L}_{\text{in\_group}}-\mathcal{ L}_{\text{inter\_group}} \tag{1}\]  
where  
\[\mathcal{L}_{distance}=\mathcal{L}_{\text{in\_group}}-\mathcal{L}_{group} \tag{2}\]  
where \(\mathcal{L}_{\text{CE}}\) represents the cross-entropy loss for classification and \(\mathcal{L}_{\text{in\_group}}\) minimizes the distance between features \(\vec{f}_{j}\) and their group center \(\hat{\mu}_{i}\):  
\[\mathcal{L}_{\text{in\_group}}=\frac{1}{|G|}\sum_{i=1}^{|G|}\frac{1}{|G_{i}|} \sum_{j\in G_{i}}\left\|\vec{f}_{j}-\hat{\mu}_{i}\right\|^{2} \tag{3}\]  
where  
* \(|G|\) is the total number of groups;
* \(G_{i}\) is the set of samples in group \(i\);
* \(\vec{f}_{j}\) is the feature vector of sample \(j\) in group \(i\);
* \(\hat{\mu}_{i}\) is the mean feature vector of group \(i\). \(\mathcal{L}_{group}\) ensures that the minimum distance between different groups is maximized:  
\[\mathcal{L}_{group}=\min_{i}\|\hat{\mu}_{m_{i}}-\hat{\mu}_{i}\|^{2} \tag{4}\]  
where  
* \(\hat{\mu}_{m_{i}}\) is the mean feature vector of the out-of-group samples;
* \(\hat{\mu}_{i}\) is the mean feature vector of group \(i\). Note that although features across groups may be embedded in different semantic subspaces, we calculate the inter-group separation based on their representations within the shared feature space of the current layer.