## 4 Materials and Methods  
### Experimental Courses and Subjects  
This study was conducted during the final exam of a university course titled \"Introduction to Computer\". The participants included 39 students enrolled in the course, comprising 29 males and 10 females, all with a background in electrical engineering and information technology. Notably, the cohort included one senior repeating student (female) and two junior repeating students (male). Since one of the junior students (male) never attended classes and did not participate in the experiment, he was eventually excluded from the experimental data. Another freshman (male) stopped taking classes after the mid-term exam, so he was also excluded from the experimental data. Therefore, the final sample consisted of 35 freshmen, 1 junior, and 1 senior, with an average age of 18. To further validate the adequacy of the sample size, a post hoc power analysis was conducted using the G*Power software ([[https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower](https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower)]([https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower](https://www.psychologie.hhu.de/arbeitsgruppen/allgemeine-psychologie-und-arbeitspsychologie/gpower)) accessed on 16 December 2024). Given the study's design, an effect size of 0.5, and a significance level of 0.05, the power of detecting meaningful differences in system usage and learning outcomes was calculated to be 54.07%. While a larger sample would be ideal for broader generalization, the analysis suggests that the current sample size provides sufficient statistical power to support the study's conclusions. While this is a relatively small sample, it represents a complete cohort of students enrolled in the \"Introduction to Computer Science\" course at the department, ensuring consistency in the learning environment. Given that this study focuses on a controlled educational setting rather than a large-scale generalization, the findings provide valuable insights into ITS-CAL's impact on student learning within this context. Additionally, prior studies in intelligent tutoring systems and AI-driven educational tools  
Figure 4: User-defined Question function. have utilized similar or smaller sample sizes to investigate learning behaviors and system effectiveness [19; 35]. To explore the behavioral differences among students with varying knowledge levels when using the three auxiliary functions of the ITS-CAL system (Hint, Debug, and User-defined Question), a cluster analysis was performed based on the students' course grades prior to the experiment. These grades were derived from two computer-based tests and one mid-term computer-based examination, all conducted in a closed-book format and lasting two hours each. Prior to applying K-means clustering, we conducted Hierarchical Clustering to determine the optimal number of clusters. Based on the dendrogram and agglomeration schedule, we observed distinct separation at three clusters, supporting our decision to set K = 3 in subsequent K-means clustering. To ensure that the clusters were significantly different, we conducted ANOVA (Analysis of Variance) tests and post hoc analyses. The results demonstrated statistically significant differences (\(p<0.05\)) among clusters across all three selected assessments. These results confirm that the clusters represent meaningfully distinct groups based on programming performance--high, medium, and low knowledge levels--as shown in Tables 1 and 2. To assess the validity of the clustering model, we computed the silhouette score and explained variance ratio. The silhouette score, which measures how well students fit within their assigned clusters, was calculated as 0.3994. A score indicates moderate cluster separability. Additionally, the explained variance ratio of 100% confirms that the selected features fully capture student performance differences.