## 3 Methodology
### Modeling and Model Evaluation
#### 3.4.3 Clustering Analysis of Automatically Evaluated Programming Code Assignments  
The dataset analyzed in this section represents grades from automatically evaluated programming code assignments in individual topics. The dataset contained eight features. The Elbow method curve suggested setting the number of clusters to five. The results of K-Means clustering with performance metrics for different values of \(k\) for these features confirmed again that the students' performance results could create two clusters (Table 5). However, the highest Silhouette coefficient value was lower than in the dataset with micro-learning units. The comparison of Silhouette plots of K-Means clustering for different values of \(k\) showed the dominancy of one cluster (Figure 10) and uneven distribution of data between other small clusters. The suggestion to divide the students' grades into two clusters aligns with the previous results. The reasoning for the existence of two clusters was also confirmed in teachers' feedback. They repeatedly noticed that involving a micro-learning platform did not cause a normal distribution of the grades between students. However, it highlighted the differences between students with a positive attitude toward learning tools and students who do not prefer intensive use of tools in learning programming. More subjectively, students could be divided into more than two clusters, as suggested in Figure 11. A visualization of inter-cluster distance maps for different \(k\) values indicates that the dataset contains more groups of students with similar performances. This statement aligns with the survey results among the teachers who implemented the course. They observed a similar number of student groups, as identified using clustering methods. For example, the students who made continual efforts during the semester represented the first group. The second group of students focused mainly on automated programming assignments. The third group is characterized by losing interest as the assignments become harder. Finally, the last group or cluster represents students with minimal effort or self-motivation. However, these observations and subjective conclusions should be further analyzed and confirmed using other learning analytics methods or process mining. The reason is that inter-cluster distance maps use PCA with two components to map multidimensional space to 2D space. This means the clusters with visible intra-cluster distance can look different in higher-dimensional space. The comparison of how students moved between clusters identified using micro-learning units with quizzes and programming assignments could add an interesting contribution to this discourse about students' behavior. The Rand Index computes a similarity measure between two clustering results by considering all pairs of samples and counting pairs assigned in the same or different clusters. The Adjusted Rand Index represents its normalized value. It has a value close to 0.0 for random labeling independently of the number of clusters and samples and exactly 1.0 when two clustering results are identical. The order of the samples must be the same in both clustering results. The adjusted Rand Index was 0.574, indicating that several students moved between clusters. A contingency matrix can visualize this movement, which shows the relationship between labels. As a \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Clusters** & **Silhouette** & **Calinski–Harabasz** & **Davies–Bouldin** \\ \hline
2 & 0.5482 & 162.3892 & 0.7030 \\ \hline
3 & 0.4275 & 121.7160 & 1.1015 \\ \hline
4 & 0.3960 & 100.6032 & 1.0784 \\ \hline
5 & 0.4148 & 91.6369 & 1.1800 \\ \hline
6 & 0.4228 & 81.6195 & 1.1520 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: The K-Means clustering performance metric values for different \(k\) values for features, which represent grades from the programming assignments, automatically evaluated using the micro-learning platform.