## 3 Research Methodology
### Variable of Sample Information for One Feature  
A sample \(E\) is represented by a set of feature values \(X(x_{1},x_{2},\cdots,x_{m})\) and a class label \(Y\), the value of \(Y\) can only be 1 or 0. According to Bayes probability, posterior probability of \(Y\) can be calculated as  
\[p(Y|X)=\frac{p(X|Y)p(Y)}{p(X)} \tag{1}\]  
Then posterior probability of a sample \(E_{i}\) with \(m\) features being class \(y\) can be calculated as  
\[p(Y=y|X_{i})=\frac{p(X_{i1}=(X_{i1}=x_{i1},X_{i2}=x_{i2},\cdots,X_{im}=x_{im}) |Y=y)p(Y=y)}{p(X_{i}=(X_{i1}=x_{i1},X_{i2}=x_{i2},\cdots,X_{im}=x_{im}))} \tag{2}\]  
Because of the assumption that all features are independent for the given class label, the conditional probability \(p(X_{i}\mid Y=y)\) can be calculated as  
\[p(X_{i}=(X_{i1}=x_{i1},X_{i2}=x_{i2},\cdots,X_{im}=x_{im})|Y=y)=\prod_{j=1}^{m }p(x_{ij}\mid y) \tag{3}\]  
Figure 1: Overall structure of the proposed method LIMCR. The Naive Bayes classifier is expressed as  
\[f_{b}(X_{i})=\frac{p(Y=1)}{p(Y=0)}\prod_{i=1}^{n}\frac{p(x_{ij}\mid Y=1)}{p(x_{ ij}\mid Y=0)}=IR\prod_{i=1}^{n}\frac{p(x_{ij}\mid Y=1)}{p(x_{ij}\mid Y=0)} \tag{4}\]  
where \(n\) is the number of samples and \(IR\) represents the imbalance radio. Then the class label \(Y\) of a sample with feature \(X_{i}\) is predicted according to \(f_{b}(X_{i})\)  
\[ \begin{cases}Y=1,f_{b}(X_{i})\geq 1\\ Y=0,f_{b}(X_{i})<1\end{cases}  \tag{5}\]  
For the single feature, the bigger the gap between \(p(x_{ij}\mid Y=1)\) and \(p(x_{ij}\mid Y=0)\), the larger the value of \(\frac{p(x_{ij}\mid Y=1)}{p(x_{ij}\mid Y=0)}\) is, simultaneously the easier the sample \(X_{i}\) can be correctly classified by Naive Bayes classifier. Correspondingly, the easier a sample is misclassified the more informative it is. Generally, the value of conditional probabilities \(p(x_{ij}\mid Y=0)\) and \(p(x_{ij}\mid Y=1)\) are calculated based on samples in dataset, and the likelihood of the features is assumed to be Gaussian. Then when there is only one feature, the conditional probabilities are calculated as  
\[p(x_{i}\mid y)=\frac{1}{\sqrt{2\pi\sigma_{y}^{2}}}\text{exp}(-\frac{(x_{i}-\mu _{y})^{2}}{2\sigma_{y}^{2}}) \tag{6}\]  
If a sample with one feature can be precisely classified into corresponding class (for instance \(y=0\)), the conditional probability \(p(x_{i}\mid y=0)\) should be close to 1 and \(p(x_{i}\mid y=1)\) should be close to 0, then the difference between them should be close to 1. This kind of samples cannot provide much effective information for classifier except making sample variance larger. Figure 2 presents two curves in each figure which mean the probability density functions of two probabilities in one dimension, respectively. It is known that samples in overlapping area are hard to be classified correctly and may disturb model training. The data distribution of original dataset in one dimension is like curves in Figure 2a, the distribution of majorities is dispersive and creates a large overlapping area with minorities. After removing less informative samples in majorities, the variance of majorities turns small and the overlapping area turns smaller as well in Figure 2b. The two figures illustrate that the increase of sample variance of one class would lead sample overlapping area larger and make the learning phase harder to classify. Considering the amount of majority samples in datasets with imbalanced distribution is larger than minority samples, we define an informative variable \(D\) for evaluating how informative a majority sample is for one feature. **Definition 1**. : _In affirmative variable \(D\). In imbalanced datasets, the variable for evaluating information of a majority sample for one feature is defined as its difference between conditional probabilities \(D_{ik}=p(x_{ik}\mid y_{i}=0)-p(x_{ik}\mid y_{i}=1)\), where \(i\) represents the \(i\)th sample and \(k\) represents the \(k\)th feature._  
Figure 2: Distribution of overlapping area.