## 4 Methodology and Experimental Results
### Results for Classification Algorithms
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multicolumn{5}{c}{**Flesch–Kincaid + Gunning Fog + Automated Read. + Spache Read. + School Grade**} \\ \hline
**Classifier** & **CCI** & **ICI** & **Pre** & **Re** & **F1** \\ \hline NB & 59.3625 & 40.6375 & 0.594 & 0.594 & 0.593 \\ \hline NB-K & 59.7 & 40.3 & 0.597 & 0.597 & 0.597 \\ \hline
**NB-D** & 59.95 & 40.05 & 0.601 & 0.600 & 0.598 \\ \hline J48 & 58.75 & 41.25 & 0.589 & 0.588 & 0.586 \\ \hline J48-C 0.05 & 58.8375 & 41.1625 & 0.590 & 0.588 & 0.587 \\ \hline J48-U & 57.2625 & 42.7375 & 0.573 & 0.573 & 0.572 \\ \hline J18- & 41.4 & **38.4** & **41.1** & **41.1** & **41.1** \\ \hline IBR-K 13 & 55.1125 & 44.8875 & 0.551 & 0.551 & 0.551 \\ \hline IBR-K 71 & 58.45 & 41.55 & 0.585 & 0.585 & 0.584 \\ \hline IBR-K 551 & 58.8625 & 41.1375 & 0.589 & 0.589 & 0.588 \\ \hline IBR-K 3501 & 59.725 & **40.275** & 0.598 & 0.597 & 0.596 \\ \hline SMO & 58.5625 & 41.4375 & 0.586 & 0.586 & 0.585 \\ \hline Random Forest & 43.4125 & 56.5875 & 0.434 & 0.434 & 0.434 \\ \hline Random Forest-P 25 & 49.7875 & 50.2125 & 0.498 & 0.498 & 0.498 \\ \hline Random Forest -P 1 & 58.2125 & 41.7875 & 0.583 & 0.582 & 0.581 \\ \hline Random Forest-P 1 -110,000 & 59.1625 & 40.8375 & 0.592 & 0.592 & 0.591 \\ \hline Decision Table & 59.6375 & 40.3625 & 0.598 & 0.596 & 0.595 \\ \hline Simple Logistic & 59.4375 & 40.5625 & 0.594 & 0.594 & 0.594 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Results from WEKA using 10-fold cross-validation and Flesch–Kincaid + Gunning Fog + Automated Readability Index + Spache Readability Formula + School Grade. two groups based on complexity. Using the -K flag, we increased the number of neighbors used for the final decision. We observed a consistent pattern with this algorithm across all our tests. We ran the KNN algorithm with different feature sets using 1, 13, 71, 551, and 3501 neighbors (we ensured all the numbers were odd to avoid confusion). Having a single neighbor usually did not perform well. In our case, increasing the number of neighbors significantly increased accuracy, since a limited number of neighbors could not cover the different example cases provided. We found that the sweet spot in most cases was setting the number of neighbors to 1.8/4 of the total examples. Examining our results, as shown in Figure 4, we observe that the lines representing test runs with sets including five features are almost overlapping and form a similar curve. They are also relatively close to the test runs using all the features. The peak performance of these lines lies in the middle to the end of Figure 4. On the other hand, tests with a single feature or pair of features tend to have a much smoother line, especially the line representing the results using McAlpine EFLAW as a single feature, which is almost straight regardless of the number of neighbors. After considering the literature review, we decided to also run tests based on the Flesch-Kincaid grade level and word count as a feature set, since the literature suggests that this combination can offer great accuracy in certain cases [5]. Evaluating the obtained results, it appears that the combination of these two features is indeed relatively good at assessing readability. The results can be seen in Table 6.  
Finally, we examined the predictability of a special feature we selected for this study that focuses specifically on L2 readers. We ran tests using McAlpine EFLAW as a single feature, which yielded interesting results, as shown in Table 7. It should be noted that even though shallow features are usually considered unstable or unreliable, McAlpine EFLAW was very consistent throughout the tests, regardless of the classifier and parameterization. Although the accuracy was not as high as in other tests, it was consistently in the 57 th percentile. McAlpine EFLAW is likely a good feature for reliable readability assessment of text focusing on L2 readers since its predictions in our tests, while not as high as those from our other feature sets, were at least very consistent.