## 4 Methodology and Experimental Results
### Results for Classification Algorithms
Figure 4: Visualized results of the KNN algorithm run on different feature sets and numbers of neighbors. The figure shows the percentage of accurately classified instances based on the number number of neighbors per run. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multicolumn{5}{c}{**Flesch–Kincaid + Word Count**} \\ \hline
**Classifier** & **CCI** & **ICI** & **Pre** & **Re** & **F1** \\ \hline NB & 56.675 & 43.325 & 0.574 & 0.567 & 0.556 \\ \hline NB-K & 58.6875 & 41.3125 & 0.587 & 0.587 & 0.586 \\ \hline NB-D & 59.3875 & 40.6125 & 0.594 & 0.594 & 0.593 \\ \hline J48 & 59.2125 & 40.7875 & 0.593 & 0.592 & 0.591 \\ \hline J48-C 0.05 & 59.3625 & 40.6375 & 0.595 & 0.594 & 0.592 \\ \hline J48-U & 59.15 & 40.85 & 0.593 & 0.592 & 0.590 \\ \hline Iilk & 54.3 & 45.7 & 0.548 & 0.593 & 0.592 \\ \hline IBlk-K 13 & 55.95 & 44.05 & 0.560 & 0.560 & 0.559 \\ \hline IBlk-K 71 & 58.275 & 41.725 & 0.584 & 0.583 & 0.581 \\ \hline IBlk-K 551 & 59.4125 & 40.5875 & 0.598 & 0.594 & 0.590 \\ \hline IBlk-K 3501 & 59.3875 & 40.6125 & 0.595 & 0.594 & 0.593 \\ \hline SMO & 58.6375 & 41.3625 & 0.588 & 0.586 & 0.585 \\ \hline Random Forest & 54.8 & 45.2 & 0.549 & 0.548 & 0.547 \\ \hline Random Forest-P 25 & 55.7375 & 44.2625 & 0.558 & 0.557 & 0.556 \\ \hline Random Forest-P 1 & 58.65 & 41.35 & 0.589 & 0.587 & 0.584 \\ \hline Random Forest-P 1 -1 10,000 & 59.2625 & 40.7375 & 0.595 & 0.593 & 0.590 \\ \hline Decision Table & 59.725 & 40.275 & 0.598 & 0.597 & 0.597 \\ \hline Simple Logistic & 58.7125 & 41.2875 & 0.588 & 0.587 & 0.587 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Results from WEKA using 10-fold cross-validation and Flesch–Kincaid + Word Count. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multicolumn{5}{c}{McAlpine EFLAW} \\ \hline
**Classifier** & **CCI** & **ICI** & **Pre** & **Re** & **F1** \\ \hline NB & 5.9625 & 41.075 & 1.371 & 1.560 & 1.541 \\ \hline NB -K & 57.525 & 42.475 & 0.577 & 0.575 & 0.572 \\ \hline NB-D & 57.925 & 42.075 & 0.586 & 0.579 & 0.571 \\ \hline J48 &  
In all of our results tables, it can be seen that the Random Forest (RF) algorithm achieved the best results when we increased the number of trees and significantly reduced the bag size. RF used all possible features in every tree. In RF, if the observations (the trees created based on the examples) exceed the limited number of trees, certain observations will be predicted just once or not at all, usually leading to poor results. It is obvious that increasing the number of iterations (random trees) will increase the performance of the algorithm, as trees with good predictions will clearly outnumber those with bad predictions. The best results in Random Forest are seen with a higher tree count and lower bag size. We highlight the importance of the bag size percentage in the Random Forest algorithm. It is evident that in all of our Random Forest test runs, decreasing the bag size yielded better accuracy. Most of the time in our tests, when reducing the bag size to 1, meaning that each bag contained only 1% of the training set, RF performed the best. This is an extreme case with unconventional results, but similar to the extremely low confidence interval in J48, the same seems to work for RF with the bag size percentage in this case. However, these results might arise due to overfitting despite the 10-fold cross-validation, so we ran further tests later on. To ensure that this parametrization significantly impacts most cases and to attempt to learn what affects the performance of the classifier the most, we decided to run further tests, tuning it even more. In the tests presented in Table 8, we used the exact same dataset and altered the parameters concerning the bag size, the iterations, and the number of features. From the information in Table 8, we can see that increasing the number of iterations (to at least 1000) yielded significantly better results in most cases. Moreover, as seen in the previous tables, reducing the bag size percentage to as low as 1% yielded the best results in our experimental setup with this dataset.