## 3 Experiments
### Results
#### 3.2.1 Analysis of Single-Graph Metrics  
In this first scenario, the analysis has been performed using the metrics calculated for the graph involved in each exercise and for the three above mentioned poolings of exercises. The fixed effect in the GLMM-BLR model is the interaction between the single-graph metrics and the difficulty of the exercise (\(Metric\times Difficulty\)). Table 4 summarizes the results of the metrics whose significance level was below the threshold value (\(\rho<0.05\)) in the statistical test and, thus, showing a statistically significant correlation in, at least, one of the three above mentioned poolings. Henceforth, when we use the term \"statistically significant correlation\", we mean that the metric is relevant and its values can be regarded as an indicator of the difficulty on solving exercises. The observed outcomes are summarized below. First, the subset of metrics under the categories size and density have a significant correlation for the first pooling of exercises (all exercises). They also have a significant correlation for the second pooling (analysis exercises), except for density (\(\Delta\)) metric, whose significance value is close but slightly above the threshold of 0.05. However, they do not usually show a significant correlation for the third pooling (modifiability exercises). In fact, the metrics that indicate a ratio between the number of arcs and the number of nodes, namely, density (\(\Delta\)) and coefficient of connectivity (\(CNC\)), are the only ones showing a significant correlation for that case. Second, the subset of metrics under the remaining categories (partitionability, connector interplay, and concurrency) show a less clear pattern in the three poolings of exercises. The only metric showing a significant correlation for all exercises is sequentiality (\(\Xi\)), which somehow equates the simplicity of the graph to the presence of sequences of nodes. The metrics showing a significant correlation for the analysis exercises are depth (\(\Lambda\)), control flow complexity (\(CFC\)), number of decisions (\(S_{D}\)), and token split (\(TS\)), which are all related to connectors and possible execution paths. The only metric showing a significant correlation for the modifiability exercises is separability (\(\Pi\)), which might indicate the presence of non-trivial structures in the graph. In general, for the analysis exercises, it makes sense that errors occur in those graphs with a large number of elements, specially those including many multiple-path decisions. \begin{table}
\begin{tabular}{l l c c c} \hline \hline
**Category** & **Metric \& Fixed Effect** & **All** & **Ana.** & **Mod.** \\ \hline \multirow{4}{*}{Size} & Size: \(S_{N}\times Difficulty\) & **0.013** & **0.011** & 0.195 \\ \cline{2-5}  & Number of arcs: \(|A|\times Difficulty\) & **0.031** & **0.001** & 0.205 \\ \cline{2-5}  & Diameter: \(diam\times Difficulty\) & **0.033** & **0.014** & 0.133 \\ \hline \multirow{4}{*}{Density} & Density: \(\Delta\times Difficulty\) & **0.041** & 0.054 & **0.027** \\ \cline{2-5}  & Coefficient of connectivity: \(CNC\times Difficulty\) & **0.020** & **0.018** & **0.041** \\ \cline{2-5}  & Average connector degree \(\widehat{d_{C}}\times Difficulty\) & **0.014** & **0.011** & 0.082 \\ \cline{2-5}  & Maximum connector degree \(\widehat{d_{C}}\times Difficulty\) & **0.021** & **0.007** & 0.138 \\ \hline \multirow{4}{*}{Partitionability} & Separability: \(\Pi\times Difficulty\) & 0.055 & 0.238 & **0.015** \\ \cline{2-5}  & Sequentiality: \(\Xi\times Difficulty\) & **0.044** & 0.204 & 0.148 \\ \cline{2-5}  & Depth: \(\Lambda\times Difficulty\) & 0.090 & **0.043** & 0.111 \\ \hline \multirow{2}{*}{Conn. interplay} & Control flow complexity: \(CFC\times Difficulty\) & 0.08 & **0.041** & 0.570 \\ \cline{2-5}  & Number of decisions: \(S_{D}\times Difficulty\) & 0.059 & **0.033** & 0.398 \\ \hline Concurrency & Token split: \(TS\times Difficulty\) & 0.086 & **0.022** & 0.513 \\ \hline \hline \end{tabular}
\end{table}