# Adaptive Quick Reduct for Feature Drift Detection
## 1 Introduction
### Feature Drift  
Analogously, and relatedly, _feature drift_ is defined as the change of relevance of a feature over time. It comes into play when a feature selection algorithm is used on a data stream, where the non-stationarity of the phenomenon produces a variable subset of best features for classification as times passes. Compared to concept drift, feature drift has emerged only recently and few techniques have been developed so far (please see [4] for a review). It must be stressed that feature drift can be captured with dynamic feature selection methods, as proposed in this paper, but that it is different from _Online Feature Selection_ or _Streaming Feature Selection_ methods: the latter involve learning on a stream of features, not instances (please see [5; 6] for further details and an example of techniques based on Rough Sets).  
The _relevance_ of a feature \(f\) in a Feature Selection problem can be expressed by a function, that in the simplest cases has a binary codomain:  
\[R(f):X\to Y,\ Y\in\{0,1\} \tag{2}\]  
the relevance function, can also have a continuous codomain in more sophisticated cases:  
\[S(f):X\to Y,\ Y\in[0,1] \tag{3}\]  
where ultimately a threshold function \(T\) is often used to establish which feature should be finally included in the reduced set.  
\[T=\begin{cases}0,&if\qquad x\leq\theta\\ 1,&if\qquad x\ >\theta\end{cases} \tag{4}\]  
\[R(f)=T\circ S(f) \tag{5}\]  
So without loss of generality \(R(f)\) can be used for both cases. The relevance of a feature is related to its prediction power with respect to the target classes.  
The _redundancy_ of a feature \(f\) in a Feature Selection problem can be expressed by a function, that in the simplest cases may be a simple linear correlation coefficient with other features. A feature is considered redundant if its prediction power is absorbed by the feature already in the best set, in other terms if the prediction power of the best set of feature does not change removing this feature from it.  
Again, four types of drift can be observed, but using windows instead of single instances as references for time and in the context of a classification problem:  
* Sudden drift (also called feature _shift_), it is the case when in in two consecutive time windows a feature becomes irrelevant or redundant (and should be excluded from the best set of features) or relevant (and should be added to the best set of features) for classification, remaining in that state for a certain number of consecutive windows;
* Gradual drift, that happens when considering a number of consecutive time windows the frequency of a feature resulting relevant (irrelevant) increases (decreases) over time: it is a situation when a feature at first appears and disappears intermittently from the set of relevant features--it is on the edge of relevance--and then stabilizes itself;
* Incremental drift (or morphing), that happens when the relevance of a feature between a number of consecutive time windows is expressed by a continuous number in \([0,1]\) that increases (or decreases) monotonically;
* Reoccurring drift, that happens when a feature that has become irrelevant turns back to be relevant and vice versa, with unknown periodicity. It may become a reoccurring shift if the transition is abrupt.  
Furthermore, for feature drift, as for concept drift, it holds the stability-plasticity dilemma: the shorter the time windows are, the more flexible the model is, the most sensitive to noise and the fastest adaptation to drift--eventually spurious--it shows; the longer the time windows are, the more stable the model is, the less sensitive to noise and the slowest adaptation to the--eventually actual--drift it shows. Reoccurring drift/shift can be included in the other three cases given enough plasticity of the model, while gradual and incremental drift may easily coexist in longer time windows.  
An adaptation of QuickReduct to streaming data, able to recognize feature shift and to remove redundant features at the same time is proposed in the following.