## 4 Experiment
### Experimental Setup  
In terms of datasets, we consider six binary data commonly found in UCI and two datasets, magazine and o-beauty, used for the detection of naval reviews. For the two text datasets, magazine and All-Beauty, we used the Bert model to extract feature words and convert them into numerical data. Then principal component analysis was adopted to reduce dimension to a certain extent, and all datasets were normalized. We chose 50% as the training set and 50% as the test set. Detailed information is shown in Table 3.  
In order to analyze the effectiveness of the Confidence Truth Serum method in the binary classification task, we selected three common ensemble learning methods (majority voting, random forest, and AdaBoost), and HMTS as the comparison group. These algorithms have greatly improved the prediction effect of the classifier. Majority vote is a simple ensemble learning method that combines the prediction results of multiple classifiers and determines the final classification results through voting. It is suitable for multiple classifiers with similar performance. By synthesizing the advantages \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Datasets** & **\# of Instances** & **\# of Features** & **Numeric Type** & **Description of Content** \\ \hline Australian & 690 & 14 & Integer, Float & Credit card applicant information. \\ German & 1000 & 24 & Integer & Credit history characteristics. \\ spam & 4601 & 57 & Integer, Float & Content characteristics of the emails. \\ biodeg & 1055 & 41 & Integer, Float & Chemical characteristics of the substances. \\ magazine & 2374 & 10 & Double & Comments of magazine in Amazon. \\ all-beauty & 508 & 768 & Double & Characters of Beauty products. \\ musk & 476 & 166 & Integer & Structural characteristics of the molecules. \\ Hilly & 1212 & 100 & Double & Characteristics of the terrain. \\ \hline \hline \end{tabular}
\end{table} 
Table 3: The information of datasets. of multiple models, better results can be obtained than a single model. Its classifier settings are consistent with CTS. Random forest is an ensemble learning method based on decision trees, which makes classification or regression by constructing multiple decision trees and combining their prediction results. It has the characteristics of sample randomization and feature randomization, and has strong anti-overfitting ability; it is especially suitable for processing high-dimensional data. In order to ensure the same number of classifiers, a random forest composed of 15 decision trees is selected. AdaBoost (adaptive boosting) is an ensemble learning method based on weighted combined weak classifiers. It improves the accuracy of the model by gradually adjusting the weights in the dataset, focusing on correcting misclassified data. In the experiment, 15 weak decision trees are used as the base classifier. HMTS (Heuristic Machine Truth Serum): The idea is to train a regression model for each classifier to predict the percentage of agreement on predictions from other classifiers for each specific data point. After obtaining the classifier's prediction label and the predicted peer prediction information, the prior is again approximated using the predicted peer prediction information for each classifier, and the average is calculated and compared to the posterior. Its base classifier setup is consistent with CTS. In terms of evaluation indicators, we mainly choose the traditional accuracy and \(F1\) score. \(TP\) means that the actual label is positive and the classification is positive; \(TN\) means that the actual is negative and the classification is negative, \(FP\) means that the actual is positive and the classification is negative.