# Detecting Out-of-Distribution Samples in Complex IoT Traffic Based on Distance Loss
## 3 Proposed Approach
### OOD Detection with Feature Distance  
Our method improves feature space representation by applying class mean clustering and optimizing feature distance loss, which increases the separation between ID and OOD samples. Since our approach relies on feature distances [23], we use the Mahalanobis distance as the OOD detection metric. The Mahalanobis distance measures the distance of a sample from the center of the distribution, considering feature covariance. Given a test sample \(x\), its Mahalanobis distance to a group center \(\mu_{G_{i}}\) is defined as:  
\[D_{M}(x,G_{i})=\sqrt{(f-\mu_{G_{i}})^{T}\Sigma_{G_{i}}^{-1}(f-\mu_{G_{i}})} \tag{5}\]  
where \(f\) represents the feature vector of the test sample \(x\) extracted by the model, \(\mu_{G_{i}}\) is the mean feature vector of group \(G_{i}\), and \(\Sigma_{G_{i}}^{-1}\) is the inverse covariance matrix of the group. The OOD detection score is defined as the negative minimum Mahalanobis distance across all groups:  
\[\text{OOD Score}(x)=-\min_{i}D_{M}(x,G_{i}) \tag{6}\]  
A sample is classified as OOD if its score falls below a threshold \(\gamma\), which is selected to ensure at least 95% in-distribution classification accuracy. To ensure the stability of Mahalanobis distance computation in high-dimensional settings, we adopt a dual strategy: (1) grouping similar classes to reduce the effective dimensionality of the feature space, and (2) applying Tikhonov regularization to the covariance matrices by adding a small multiple of the identity matrix. Specifically, the regularization parameter \(\lambda\) is empirically set to a small positive value (e.g., \(10^{-3}\)) after tuning on a validation set, balancing numerical stability and fidelity to the original covariance structure. This approach not only improves numerical robustness but also aligns with standard practices in high-dimensional statistical modeling, ensuring reliable inversion of covariance matrices without significantly distorting data distribution. Threshold selection plays a crucial role in OOD detection. However, traditional OOD methods often lack adaptability. Figure 4b shows that different threshold selection strategies yield significantly different performance, even when applied to the same model. This inconsistency highlights the need for a model-specific threshold selection approach. To address this, we propose an adaptive threshold selection strategy using KDE to determine an optimal threshold dynamically. KDE estimates the probability density function of OOD scores:  
\[\hat{p}(x)=\frac{1}{n\hbar}\sum_{i=1}^{n}K\left(\frac{x-x_{i}}{h}\right) \tag{7}\]  
where \(K(\cdot)\) is a Gaussian kernel function, \(h\) is the bandwidth parameter controlling the smoothness, and \(x_{i}\) are the OOD scores obtained from the test set. Specifically, \(x\) represents a specific OOD score for which the density function is being estimated, and \(x_{i}\) denotes the set of all OOD scores used to estimate the probability density function \(\hat{p}(x)\). This non-parametric method estimates the density by averaging contributions from each score, providing a smooth approximation of the score distribution. The bandwidth \(h\) plays a crucial role in balancing bias and variance in the density estimate: a smaller bandwidth captures more local detail but may overfit noise, while a larger bandwidth results in a smoother but potentially oversimplified density. To adaptively select an appropriate bandwidth for each dataset, we employ [PERSON]'s rule, which calculates \(h\) based on the number of samples \(n\), the data dimensionality \(d\), and the standard deviation \(\sigma\) of the scores, according to  
\[h=\sigma\cdot n^{-\frac{1}{2\gamma+4}}.\]  
This automatic bandwidth selection ensures that the KDE thresholding remains stable and reliable across different test sets with varying score distributions, eliminating the need for manual tuning. The threshold is selected at the first significant inflection point in the density curve, where it exhibits a sharp decline, marking the transition between ID and OOD samples.