## 3 Results
### Experimental Results of the Machine Learning Based Approach  
This section presents the results and analysis of machine learning methods' performance with the meta-learner concept based on feature extraction of meta vectorization. We carried out experiments using the processing scenario, as shown in Table 1. The grouping was based on the percentage of training data, namely 20%, 10%, and 5%. The experiment aimed to determine meta-learners' performance in automatic annotations using the least amount of labeled training data to optimize the annotation process. Thus, the annotation process with a small amount of labeled training data can annotate itself using the application of self-learning. Furthermore, we employ threshold parameters in self-learning to identify and seek similarities within the training dataset. This approach ensures that annotations align closely with the training and testing phases. The results obtained use accuracy to determine the correctness of the annotation process. Figure 3 presents the results of calculating the accuracy of each combination of machine learning algorithms (SVM, DT, KNN, and NB) with feature extraction method (TF-IDF and Word2 Vec). The evaluation uses manual annotations of 20% for 13,169 data. This automatically generated annotation will be used for the initial training. Figure 3 shows that the machine learning model with 20% initially labeled data scenario is capable of performing automatic annotations with varying accuracy, with the most considerable value being 90% at the threshold of 0.9 obtained by the SVM-Word2 Vec method. The accuracy of each individual method is visible on the chart presented in Figure 3. The evaluation showed that the worst machine learning method of all the different scenarios was KNN combined with feature extraction algorithms (TF-IDF and Word2 Vec). The other methods have an accuracy of \(\approx\)80% and above. The accuracy of the KNN-TF-IDF method has increased as compared to [PERSON] et al. [5] from 59.68% to 61.3% (+1.62 p.p.). This enhancement reflects advancements in the preprocessing procedures from prior research and has been validated through various experimental scenarios, as shown in Table 1. In the stemming process, the Indonesian language preprocessing was carried out using the Sastrawi library for Python. In addition, we have also added several stop-words to calculate the average vector features. However, it is worth noting that these stop-words  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**No** & **Training Data (\%)** & **Unlabeled Data (\%)** & **Threshold** \\ \hline
1 & 20 & 80 & 0.6 \\
2 & 20 & 80 & 0.7 \\
3 & 20 & 80 & 0.8 \\
4 & 20 & 80 & 0.9 \\
5 & 10 & 90 & 0.6 \\
6 & 10 & 90 & 0.7 \\
7 & 10 & 90 & 0.8 \\
8 & 10 & 90 & 0  
are not employed in the Word2 Vec method due to the fundamental differences in the calculation process it entails. In addition, the SVM-TF-IDF and SVM-Word2 Vec methods, as implemented by [PERSON] et al. [56], have been assessed under similar conditions. These methods achieved consecutive accuracies of 63.3% and 89%. As a result, feature extraction using TF-IDF has increased accuracy in performing automatic annotations by 19.5%. This also influences the application of the ensemble concept and the addition of stop-word data in the preprocessing. The Word2 Vec method has achieved optimal results, and when a threshold value of 0.9 is added, it leads to a 90% yield, representing a slight 1% increase. Based on the scenario results, the amount of 20% of the labeled data has not yet obtained optimal accuracy because several methods for detecting hate speech annotations have an accuracy of \(\approx\)90% [57] and even more than 95% [47]. So, we conducted a trial to reduce the amount of labeled data to 10% (Figure 4) and 5% (Figure 5) of the total YouTube comment data. We aim to optimize the automatic annotation with a minimal amount of data. The text annotation results improved in the case of several methods, such as DT-TF-IDF and KNN-Word2 Vec.