# Enhanced Label Noise Filtering with Multiple Voting
## 5 Experimental Work  
In this section, a set of experiments is conducted to verify the effectiveness of our proposed approach. To test its performance, several representative single-voting and multiple-voting based filters are used. SVFilters include the majority filter (MF) and consensus filter (CF) [27; 28]. MF based MVFilters include MF\({}_{1}\), MF\({}_{MF}\), and MF\({}_{CF}\)[40]. CF based MVFilters include CF\({}_{1}\), CF\({}_{MF}\), and CF\({}_{CF}\)[40]. Suppose the number of SVFilters in an MVFilter is t. In MF\({}_{1}\) and CF\({}_{1}\), the decision point is 1. In MF\({}_{MF}\) and CF\({}_{MF}\), the decision point is t/2, while in MF\({}_{CF}\) and CF\({}_{CF}\), the decision point is t. When the decision point is determined by our approach, the MF based MVFilter is denoted by MF\({}_{ODP}\) and the CF based MVFilter is denoted by CF\({}_{ODP}\). When filtering noises, the costs incurred by MF\({}_{ODP}\) and CF\({}_{ODP}\) will be compared to other methods. If our approach is effective, MF\({}_{ODP}\) and CF\({}_{ODP}\) should incur less cost compared to other methods.  
Six bioinformatics datasets from the UCI repository were used in this experiment. Information on these datasets is tabulated in Table 2, where pos/neg presents the percentage of the number of positive examples against that of negative examples.  
An SVFilter (referring to Algorithm 1) is configured as follows: the number of subsets is 3 (n = 3); three learning algorithms are used (y = 3) including naive Bayes, decision tree, and 3-nearest neighbor. The configurations of an MVFilter (referring to Algorithm 2) are basically identical to the SVFilter configurations. One additional parameter in MVFilter is the number of SVFilters, which equals nine in the experiments (t = 9). Our proposed algorithm (referring to Algorithm 3) is based on MVFilter. Its additional parameter is the number of iterations to search for ODP. Here, it equals ten (numIter = 10).  
The experiments were performed on each benchmark dataset by dividing it into a training set and test set. The filter algorithms were applied to each training set to remove the mislabeled data. Test data were only used by our algorithm, which is represented as E\({}_{nf}\) in Algorithm 3. It is important to clarify here that domain experts were involved to establish the noise-free benchmark dataset, which included the desired labels finalized after coming to a common consensus.  
Making the cost value as a baseline computation, the performance of each filter algorithm was evaluated against each dataset D using the following steps:  
1. Evaluating the performance of each filter using three trials derived from the threefold cross-validation of D. For each trail, 2/3 of D or Tr were used for the training set. We purposely changed some correct labels in the Tr using the predefined mislabeled ratio to generate the mislabeled data. For this purpose, three different mislabeled ratios were used: 10%, 20%, and 30%. As an example, for a 10% mislabeled ratio, 10% of the samples from Tr were randomly selected and then the correct labels changed.
2. The average cost of each algorithm was calculated by taking the mean cost of errors for each filter of the three trails.
3. In order to avoid the influence of the partitioning of D on the generated mislabeled data, we considered ten cost values retrieved from each experiment conducted ten times (i.e., repeating the previous two steps ten times).  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **No. of Features** & **No. of Instances** & **pos/neg** \\ \hline Heart & 14 & 270 & 55.6\%/44.4\% \\ \hline Wdbc & 30 & 569 & 62.7\%/37.3\% \\ \hline Wpbc & 33 & 198 & 76.3\%/23.7\% \\ \hline Spect & 22 & 267 & 79.4\%/20.6\% \\ \hline Spect1 & 44 & 267 & 79.4\%/20.6\% \\ \hline Promoter & 57 & 106 & 50\%/50\% \\ \hline \hline \end{tabular}
\end{table}
Table 2: Datasets used in the experiment. pos, positive; neg, negative.  
Finally, the reported cost value was obtained as the mean of these ten values.