## 3 Results
### Experimental Results of the Machine Learning Based Approach
These methods obtained an accuracy of 90% and more for all threshold experiments (for DT-TF-IDF) and 0.7, 0.8, and 0.9 threshold experiments (for KNN-Word2 Vec). In addition, the accuracy of SVM-Word2 Vec also increased to 91.9% in the 10%, 90%, and 0.7 parameters of the same scenario. So, the notable accuracy improvement when reducing the labeled data from 20% to 10% data is based on Figures 3 and 4. This increase in accuracy is based on calculating the weight of the available data, where the smaller the amount of data, the higher the accuracy of the process. In addition, the self-learning performance of the meta-learner and meta-vectorizer relies on an ensemble approach to optimize the calculated weights, resulting in enhanced performance. Figure 3: Accuracy of the proposed automatic annotation process in detecting hate speech using a semi-supervised and self-learning approach based on scenarios 1, 2, 3, and 4 (Table 1). The improvement of selected proposed methods from previous studies [5; 56] became a reference for conducting this research. Building upon some proposed methodologies from the previous studies [5], our future work encompasses exploring various scenarios. These scenarios involve reducing labeled data to 5%, 10%, and 20%, in conjunction with adjusting the threshold values to 0.6, 0.7, 0.8, and 0.9. This study has achieved this by employing meta-vectorization and meta-learning methods to enhance the annotation process. Furthermore, applying threshold variations consistently resulted in the highest accuracy at the most significant value, specifically 0.9. This pattern was observed across all variations in the percentage of labeled data, except for the 10% labeled data scenario. This observation is attributed to the robust performance of SVM in various contexts, as documented in references [58; 59], including the present research, as evidenced by Figures 3-5. These comparisons are readily evident in Figure A1 and Table A1. Notably, SVM consistently exhibits increasing accuracy. However, while the overall accuracy results demonstrate an increase, it is worth noting that certain methods experience a reduction of up to 8% in specific scenarios. The detailed comparison of the increase and decrease in method accuracy based on the scenario can be seen in Table A2. Based on the results of the 10% labeled data scenario, the final experiment is to apply a 5% labeled data scenario for training with each existing threshold. Remarkably, each scenario demonstrates a systematic progression, encompassing thresholds ranging from minor to the most significant. The accuracy of this scenario is best, with a threshold value of 0.9. The highest accuracy was obtained for DT-TF-IDF with the value of 97.1%, followed by KNN-Word2 Vec (96.9%), SVM-Word2 Vec (96.8%), SVM-TF-IDF, DT-Word2 Vec (93.4%), and others below 90%. These findings indicate that varying the quantity of data and adjusting the threshold value had a discernible impact on the methods' accuracy, leading to improved performance. In Figure 5, we can observe that DT-TF-IDF consistently  
Figure 4: Accuracy of the proposed automatic annotation process for detecting hate speech using a semi-supervised and self-learning approach based on scenarios 5, 6, 7, and 8 (Table 1). achieves a commendable accuracy level exceeding 94%, surpassing both KNN-Word2 Vec and SVM-Word2 Vec (exact accuracy values are shown in Table A1). Based on the presented scenarios and outcomes, this study adopts a semi-supervised learning approach, as illustrated in Figure 1. It involves annotating unlabeled data and substantially refining the sample dataset, ultimately leading to the automatic hate speech annotation. This process implements a self-learning mechanism, which leverages prior learning experiences to address the limitations associated with manual annotations. Regarding algorithm design, our proposed approach is shown in Listing 1. This annotation delineates an approach that employs minimal training data to maximize accuracy.