## 4 Materials and Methods
### Experimental Procedure  
The experiment was conducted in three main stages: the system familiarization stage, the examination and experimentation stage, and the post-test survey stage, spanning a total duration of three weeks.  
1. System Familiarization Stage  
One week prior to the experiment, the system familiarization phase was mandatory for all students, ensuring that every participant had an equal opportunity to learn how to use ITS-CAL, including its three primary functions: Hint, Debug, and User-defined Question. This session also covered the system's limitations and usage rules. The training included both demonstration examples and hands-on exercises, allowing students to explore the features of the three functions through multiple interactions. The goal was to ensure that students could effectively use the system during the examination. Additionally, even after the mandatory training session, the ITS-CAL system remained accessible until the exam, giving students the flexibility to further familiarize themselves with the interface. No students reported any difficulties in using the system before or during the exam, nor did they request additional training time.  
2. Examination and Experimentation Stage  
One week after the familiarization phase, a two-hour final examination was conducted as the formal experimental phase. During the exam, students were allowed to use the ITS-CAL system but had to adhere to the pre-established usage restrictions and the \"use deduction\" scoring mechanism. Students were free to decide whether to utilize any of the three auxiliary functions, and the system recorded detailed usage behavior for each operation to support subsequent analysis.  
The examination consisted of six questions. The first five questions focused on implementing specific functions, while the sixth question required students to integrate the functions from the previous questions and adjust parameters to produce the correct results. Each question included test data pre-defined by the instructor, divided into public data and hidden data. Public data helped students understand the problem requirements and expected execution results, whereas hidden data prevented students from tailoring their program logic to specific datasets, ensuring program versatility.  
To ensure the content validity of the final exam, the test questions were collaboratively designed by three instructors with backgrounds in computer science and programming education. The exam has been administered for several years, with similar exam structures appearing in previous iterations of the course, ensuring its alignment with the learning objectives. The test covers key programming concepts that are commonly evaluated in prior programming education research, further reinforcing its validity as a learning assessment tool.  
3. Post-Test Survey Stage  
After completing the test, students were asked to fill out a questionnaire designed to assess their experience with the ITS-CAL system. The questionnaire covered aspects such as the practicality of the system's functions, its ease of use, and its overall helpfulness during the learning process. In addition, test scores and system usage data were collected and combined with the students' feedback for a comprehensive analysis.