## 7 Experiments
### Analysis of the Experimental Results for the Random Forest Model  
The best classification accuracy (0.741) was obtained by the random forest (RF) model executed on a smaller set of features as described in Section 7.2.  
Here, we provide a further analysis of this result by showing, in Table 6, the confusion matrix of the experiment. In this table, each entry \(X,Y\) indicates the average number--over the 25 repetitions of the 10-folds cross-validation process--of texts which are known to belong to the CEFR level \(X\), but have been classified to the CEFR level \(Y\) by the RF model.  
The correctly classified texts are those accounted in the diagonal of the confusion matrix. In average, 512.9 out of 692 texts were correctly predicted, thus confirming the average accuracy of 74.12%. The confusion matrix also allows us to derive precision and recall measures [39] for all the considered target classes. Observing the data, it is possible to see that the B1 class exhibits the highest precision and recall--respectively, 89.33% and 89.47%--while the weakest predictions are those involving the C1 class, with 56.26% precision and 42.37% recall.  
Furthermore, let observe that most of the incorrectly classified texts are only one level away from their actual classes, i.e., the non-diagonal entries of the confusion matrix showing larger values are those adjacent to the main diagonal. In average, only one text out of 692 was classified to a CEFR class distant two or three levels from the actual class of the text. Therefore, the errors produced by our  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**ActualPred.** & **B1** & **B2** & **C1** & **C2** \\ \hline
**B1** & **222.7** & 26.1 & 0.1 & 0.0 \\ \hline
**B2** & 26.3 & **140.9** & 17.6 & 0.1 \\ \hline
**C1** & 0.3 & 45.8 & **58.9** & 34.0 \\ \hline
**C2** & 0.0 & 0.5 & 28.1 & **90.4** \\ \hline \hline \end{tabular}
\end{table}
Table 6: Confusion matrix for the random forest (RF) model with features selection.  
Figure 4: Impact of the different categories of features. (**a**) Features category distribution in the random forest (RF) model. (**b**) Features importance in the RF model.  
approach are, in some sense, small errors. This is an interesting aspect, especially in light of the fact that, in order to use the most common prediction models in the machine learning literature, we are not considering at all the intrinsic ordering among the four CEFR classes.  
Another important observation is that, often in the linguistic field, the pairs of CEFR levels B1, B2 and C1, C2 are aggregated into the macro-levels B and C, respectively. By considering this aggregation and the data provided in Table 6, it is possible to see that the average accuracy of our system increases up to the remarkable percentage of 90.66%.  
Finally, note that the results discussed are also in line with the 2D visualizations of the dataset depicted in Figure 5, which provides the results of two different executions of the well known stochastic dimensionality reduction technique t-SNE [74] executed on the 139-dimensional representation of the dataset. Every point in the visualizations is the two-dimensional representation of a text in the dataset, while its color indicates the CEFR class of the text. According to t-SNE working principles [74], the spatial distances among the points indicate how distant are the corresponding text in terms of numeric features, thus how easy or difficult is to discriminate the CEFR classes. Clearly, both visualizations show that B1 is the easier class to discriminate, while C1 and C2 are the more difficult to discern.