# Unsupervised Feature Selection for Outlier Detection on Streaming Data to Enhance Network Security
## 4 Unsupervised Feature Selection for Streaming Outlier Detection
### Operation Modes  
The conceptualization for an online unsupervised FS in an application for streaming OD is depicted in Figure 2 showing the sequential operation mode for UFSSOD. Independent of the applied operation mode, in the starting point \(t_{0}\), where no FS has been  
Figure 1: Block diagram and operation principle of UFSSOD. performed, or the initialization of each algorithm takes place, \(\gamma\) is set to \(d\) in order to work with all features. The module _Unsupervised Online OD_ operates an online capable unsupervised OD algorithm, e.g., xStream [14], Loda [52] or iForest_ASD [53], and uses the proposed subset \(\mathcal{F}_{S}\) to boost its performance, yielding a more precise outlier score \(y_{i}\) based on \(\mathbf{x}_{i}^{*}\). One might also apply multiple classifiers in parallel in the Unsupervised Online OD module to exploit the power of a combination of \(m\) individual classifiers depending on the available resources. For this, one might define a certain resource budget \(r\) comprising of computational resources such as CPU or wallclock time as well as memory usage and let \(m\) be a function of \(r\). Thus, even during runtime, some classifiers might be turned off for the sake of resource preservation. The outlier scores \(y_{0}\) and \(y_{i}\) are normalized using _Gaussian Scaling_ proposed by [54] according to Equation (3) in which \(erf()\) is the Gaussian Error Function, which is monotone and ranking stable. \[\tilde{y}=max\{0,erf(\frac{y-\mu_{t}}{\sigma_{t}\sqrt{2}})\} \tag{3}\]  
The moving average \(\mu_{t}\) and standard deviation \(\sigma_{t}\) of the outlier scores until time \(t\) are obtained by applying, e.g., the well-known [PERSON]'s algorithm [55] or one of the methods proposed in [13]. Scaling is necessary when combining algorithms of different nature with different characteristics. For instance, operating \(\text{Lodat}_{\text{Two Hist. }}\) with different window sizes will result in different averaged score values due to the different state of knowledge on normal data. With increasing window sizes, the model becomes more accurate while seeing larger amounts of normal data, which are then scored less compared to models obtained from smaller window sizes. With the mean and standard deviation proportion of the _Gaussian Scaling_ this difference will be eradicated and with the Error Function the scores are tailored to the interval between 0 and 1. With the normalized outlier scores, a final score can be computed, which might also be weighted. The confidence level of the Unsupervised Online OD module is typically higher than those of UFSSOD, since it operates on the reduced representation of a sample \(\mathbf{x}_{i}^{*}\) and thus one might give a higher weight to its score. For this, Equation (4) can be used depending on the weights \(g_{0}\) for UFSSOD and \(g_{i}\) for each classifier (typically \(g_{i}>g_{0}\)) where \(g_{0}+\sum_{i}g_{i}=1\) and \(\tilde{y}\in[0,1]\) applies. Using domain expertise, a reasonable threshold can be determined over runtime, yielding a decent classification performance that assigns a binary value from the set \(\{normal, abnormal\}\) for each \(\tilde{y}\). \[\tilde{y}=g_{0}\tilde{y}_{0}+\sum_{i=1}^{m}g_{i}\tilde{y}_{i} \tag{4}\]  
Since the UFSSOD module is able to provide different feature scores at each time step \(t\), resulting in possible other top-\(\gamma\)-features than at time \(t-i\), the downstream applied Unsupervised Online OD module might be capable of dealing with those changing features. If this is the case, the setup can truly work online, meaning that UFSSOD provides an updated version of the feature subset \(F_{S}\) with a potentially changing cardinality that is used by the Unsupervised Online OD module to detect outliers in each \(t\). To the best of our knowledge, only xStream is able to handle this situation. Moreover, xStream not only handles a varying cardinality of the feature subsets but is also able to deal with completely newly occurring features.