## 4 Design
### Attribute Filtering Based on Feature Importance  
The basic idea for the attribute filter is to find the attributes that have a negligible or negative influence on the prediction results. Inspired by the idea of decision tree models that use entropy or the Gini index to decide the split nodes for classification, in this work, both entropy and Gini index are employed to decide the features' importance, where the Gini index is chosen by default, because it had a relatively better performance than the other approach in our experiments, as shown in Section 5.  
Entropy is a quantitative indicator of the uncertainty of random variables. When the entropy of a dataset is higher, the dataset is more chaotic. As shown in Equation (2), when the task is binary classification, the entropy of the dataset \(D\) is represented.  
\[Entropy(D)=-(p\log_{2}^{p}+(1-p)\log_{2}^{1-p}), \tag{2}\]  
where \(p\) is the possibility of one class, and \(1-p\) is the possibility of the other class. Based on an attribute \(A\), \(D\) can be divided into several subsets; in this case, the entropy of \(D\) based on \(A\) is calculated as in Equation (3):  
\[Entropy(D|A)=\sum_{i=1}^{k}q_{i}\times Entropy(D|a_{i}), \tag{3}\]  
where \(A\) has k values, and \(q_{i}\) is the possibility when the value of \(A\) is \(a_{i}\). Then, \(Entropy(D)-Entropy(D|A)\) is the information gain when classifying \(D\) using \(A\), when this value is larger, \(A\) is more likely to be selected to be a spite node for the classifier; thus, \(A\) is more important.  
The Gini index is similar to the idea of entropy and employed in classification and regression trees (CART). Equation (4) shows the Gini index for the dataset \(D\) when the task is binary classification, where \(p\) is the possibility of one class, and \(1-p\) is the possibility of the other class.  
\[Gini(D)=2p(1-p), \tag{4}\]  
Based on the attribute \(A\), the dataset may be divided into several subsets; thus, the Gini index for the dataset based on \(A\) can be calculated based on the Gini index of every subset, as shown in Equation (5). If the Gini index is low, that means the uncertainty of the dataset is low, and the related attribute would be selected as a spite node for the classifier; then, based on this attribute, it would find the next spite node with the same calculation, until the reduction in the Gini index is too small to split.  
\[Gini(D,A)=\sum_{i}^{k}\frac{|D_{i}|}{|D|}Gini(D_{i}). \tag{5}\]  
Then, we can gain the importance of each feature based on the reduction in entropy or the Gini index for the dataset with attributes. If the entropy or the Gini index of an attribute can further reduce the uncertainty of the dataset, it would be more important. Thus, the importance value of a feature \(A\) is \(\dfrac{R_{A}}{R}\), where \(R_{A}\) is the uncertainty reduction in \(A\), and \(R\) is the total reduction in the classifier. Finally, the attributes whose importance values are lower than \(T_{i}\) are removed, in which \(T_{i}\) is the importance threshold. To choose reasonable \(T_{i}\), we experimented with different threshold values from 0.005 to 0.02, and when this value was around 0.01, the prediction models performed best. Thus, in this work, \(T_{i}\) was set to 0.01. In this step, the attributes that have less influence on dividing the negative and positive samples are filtered.