## 4 Evaluation
### Results Analysis
#### 4.3.4 Section 4: Analysis
Higher Topic Difficulty Reveals Greater Variability Among Students: The average accuracy of the first submission drops from about 97% for the leading students to approximately 70% for those toward the end, indicating a clear gap in students' comprehension of operations, bit manipulations, and signed versus unsigned calculations. For some students, even the final accuracy remained quite low, suggesting that certain problems may have been excessively difficult or overly complex, hindering effective problem-solving. 3. Summary: Overall, Section 4 represents a significant increase in learning difficulty within the system's curriculum. The trends in average submission count and first-attempt accuracy suggest that additional instructional supports are warranted for advanced operations, such as more detailed worked examples, targeted error correction strategies, and the pre-class categorization of problem types to help students gradually build the necessary computational skills. Figure 20: Solution results for Group 1 in Section 4. Group 2 Test Results  
As shown in Figure 22, both the average submission times and the average accuracy of the first submission for Group 2 were slightly lower than those observed in Group 1. Consistent with the results from Group 1, Questions 4, 6, 8, and 12 exhibited lower accuracy rates. In addition, for Group 2, Question 10 presented a significant challenge; this topic involved signed division, one of the more complex problem types in the section, which resulted in a notably high number of attempts. Beyond the inherent complexity of these questions, potential ambiguities or shortcomings in the problem statements or design may have contributed to these outcomes. In terms of the final accuracy, several of the later questions did not reach the expected target (ideally 97%), likely due to the incomplete problem design or insufficiently clear conceptual explanations. Addressing these issues through refined question design and improved instructional clarity will be a focus for future revisions. Individual Student Performance in Group 2  
Figure 23 illustrates the learning performances of Group 2 students in Section 4, high-lighting pronounced individual differences and challenges. The chart reveals that some students had average submission times exceeding seven attempts, indicating a need for extensive trial-and-error during problem-solving. The average accuracy of the first submission exhibited a clear downward trend, dropping from 100% among the leading students to approximately 30-50% for those in the latter portion of the group, reflecting a substantial gap in initial comprehension. While most students were ultimately able to  
Figure 21: Individual student performance for Group 1 in Section 4. Figure 22: Solution results for Group 2 in Section 4.  
achieve 100% final accuracy, a few did not attain complete mastery, resulting in lower final scores. Compared to Group 1 students, Group 2 students displayed higher average submission times, lower average accuracy in their first submission, and greater disparities among the lower-performing students in this section. This difference is likely attributable to the inclusion of students without relevant backgrounds in Group 2; lacking foundational computer science knowledge, they may have found certain concepts more challenging, which manifested as a steeper learning curve. To address this, future instructions should incorporate more detailed examples and tiered supports to help students from diverse backgrounds gradually develop both a computational understanding and practical skills. Additionally, there was one student (ID 41) with an average submission count of 0.74. Upon investigation, this was due to their incomplete responses to several questions, resulting in anomalously low data. Going forward, data from students with incomplete responses should be treated accordingly to ensure more accurate analysis.