## 4 Discussion  
This study assessed the potential of the GPT-4 model as a tool for grading clinical reports in the OSCE, highlighting its strengths and limitations compared to human evaluators. The findings revealed several key points, starting with the significant correlation between AI and human grading, which supports AI's capacity to adhere to predefined evaluation standards. However, the AI's grading was more stringent, assigning lower scores than experts and completing tasks significantly faster. These results align with previous studies indicating that while AI systems have shown promising levels of accuracy and consistency in educational assessments [15; 29], including evaluations of medical reasoning or exam performance [21; 23; 24], their outcomes are not yet exceptional. For instance, ChatGPT's performance in USMLE-style questions achieved results comparable to medical students [26], which supports its potential in this environment. The tendency of the AI to assign lower grades than human graders, especially experts, was one of the study's most interesting findings. This discrepancy can be attributed to the inherent rigidity of the AI model, which strictly adheres to the explicit criteria established in the checklists. Although this assures consistent evaluation, it also seems to restrict the AI's capacity to adjust to appropriate variances in medical language or interpret implicit inferences, a skill that human graders are better equipped to manage because of their clinical backgrounds. For instance, in the item \"Make a diagnostic judgment: coronary syndrome/osteochondritis\", human graders awarded the point if either of the two conditions was mentioned, whereas the AI required explicit mention of both, leading to significant discrepancies. It is crucial to emphasize that these variations represent opportunities for better AI configuration and design rather than necessarily indicating that its use is invalid. According to published research, AI models like GPT-4 can be made more capable of identifying implicit descriptions or better adjusting to changes in medical language by modifying their prompts and algorithms [20; 23; 24; 30]. According to recent research, AI can perform on clinical reasoning exams at levels comparable to those of medical students, indicating that it might be used in combination with other methods to evaluate clinical competencies [17; 20; 21; 24]. Another relevant finding of this study is that the AI assigned substantially lower scores than human graders, including experts. This result should be viewed as a sign of intrinsic \begin{table}
\begin{tabular}{c c c} \hline \hline  & **F** & \(\mathbf{p}\) \\ \hline EXP1 vs. AI & 206.246 & \textless{}0.001 \\ EXP2 vs. AI & 211.246 & \textless{}0.001 \\ INEXP vs. AI & 86.486 & \textless{}0.001 \\ MEDIA EXPE vs. AI & 235.862 & \textless{}0.001 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Repeated measures ANOVA displaying the comparison of scores between the different evaluators and the AI. variations in the evaluation process that indicate the need for more research rather than a weakness in the AI. The method by which the items on the checklist are designed may be one important component. Prior research has emphasized the necessity of optimizing item design to ensure clarity, remove ambiguity, and preserve uniqueness [4; 9; 10], facilitating a review by automated systems as well as humans. In this way, the AI may classify these ambiguities as errors if it applies the criteria strictly and literally, while human graders are more flexible in their interpretation. Additionally, there are other elements that might affect how human graders score [6; 8]. One such factor is fatigue, which results a decline in accuracy and an increase in leniency as evaluation sessions go on, as demonstrated in in earlier studies [6; 7]. This issue is linked to the requirement to maintain effort and concentration over extended periods of time, which could lead to less strict grading.