## 4 Experimental Results and Analysis  
### Experimental Data and Research Method  
To verify the effectiveness of the optimized BT-MTS, we selected four multi-classification data sets from the KEEL database for the experiment. The data sets are selected as follows: (1) imbalance ratio: the imbalance ratio of the dataset varies widely from 5.88 to 439.6, which better reflects the imbalance data processing ability of different methods; (2) divisibility: the data sets have different degrees of difficulty in classification, which is conducive to compare the classification performance of different methods comprehensively, and the advantages and shortcomings are more obvious; (3) sample size: considering that small sample data tend to undertrain a machine learning algorithm and render it ineffective [37], the total sample size of the data sets selected in this paper greatly varies, and the number of instances of each category within the data sets also greatly varies, which tests the performance of the algorithm more. The data sets used are listed in Table 1.  
In this study, BT-MTS is first used as one of the comparison methods to verify the effectiveness of the optimization algorithm. The BT-MTS method uses the following processes to determine the classification sequence: in sequence, a category of the training set is used as the MS. The rest of the categories are regarded as abnormal samples. In \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Data Set** & **Number of Samples** & **Number of Feature Variables** & **Number of Categories** & **Number of Samples** & **Imbalance Ratio** \\ \hline Balance & 625 & 4 & 3 & 49/288/288 & 5.9 \\ Wine quality-white & 4898 & 11 & 7 & 20/163/1457/2198/880/175/5 & 439.6 \\ Glass & 214 & 8 & 6 & 70/76/17/13/9/29 & 8.4 \\ Page-blocks & 5437 & 10 & 5 & 4913/329/28/88/115 & 175.5 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Data set information. Figure 3: The schematic of the optimized BT-MTS. addition, the classification accuracy rate of the testing set can be calculated. The category with the highest classification accuracy is used as the root node of the MS. In the training and testing sets, the sample data of the root node are removed. The aforementioned steps are repeated for the remaining categories to calculate the classification accuracy rate. The category with the highest classification rate is used as the MS of the Level 2 node. This process is repeated until the MS of all non-leaf nodes has been identified. Finally, the conventional MTS implementation steps are used to train the classifiers of the inner nodes. Furthermore, this paper selects four algorithms, RF, FCBF-SVM, SMOTE-KNN, and BPNN, as the comparison algorithms from four levels: integrated learning, feature method, data level, and neural network method. The parameter set of the algorithm is crucial for the performance of the algorithm, and each critical parameter should be optimized by using training data [38]. For the four machine learning algorithms, this study adopted the grid search method to optimize their critical parameters using MATLAB R2019a software. Regarding RF, this study optimized two critical parameters, namely the number of decision trees (Ntree) and the number of variables to select at random for each decision split (NumPredictorsToSample). The search scope of Ntree is 50-500, and the length of the step is 50. The search scope of NumPredictorsToSample is from one to the number of feature variables; the length of the step is one. For FCBF-SVM, the experiment adopts the Gaussian radial basis function kernel and optimizes the kernel parameter (KernelScale) and penalty factor (BoxConstraint). The search scope of both parameters is from \(2^{-8}\) to \(2^{8}\), and the length of their step is one. Regarding KNN and BPNN, the experiment separately optimizes the k value of KNN and the number of neurons in the hidden layer (Hiddensize).