A Cluster Randomized Controlled Trial Study on How AI Critical Appraisal Training Enhances Critical Thinking in Nursing Students
Phase: NA
Conditions: Healthy
Sponsor: Xi Cao

 
Study Protocol and Statistical Analysis Plan 
 
 
 
Effect of AI Critical Appraisal Training on Critical Thinking 
in Nursing Students 
 
 
 
April 2nd, 2026 
 
 
 
 
 
 
 
 
 
 

1. Study Synopsis 
• 
Design: Multi-cycle, longitudinal, cluster randomized controlled trial. 
• 
Population: Undergraduate nursing students enrolled in consecutive semesters. 
• 
Groups: For each study cycle, intact class clusters will be randomly allocated 
to one of two groups: 
o Experimental Group: Receives “AI Critical Appraisal Training”, 
where students use AI tools to analyze cases and provide structured 
critiques of the AI-generated output. 
o Control Group: Receives “Traditional Case Analysis Training”, 
completing the same cases without the use of AI tools. 
• 
Intervention: An 8-week program integrated into a surgical nursing course. 
One structured case assignment is completed every two weeks (4 assignments 
total). 
• 
Primary Objective: To evaluate the effect of AI critical appraisal training 
compared to traditional teaching on improving nursing students' disposition 
toward critical thinking. 
• 
Primary Outcome: Change in total score from baseline to week 8 on the 
Chinese Version of the Critical Thinking Disposition Inventory (CTDI-CV). 
• 
Secondary Outcomes: Changes in AI literacy and clinical reasoning ability, 
end-of-semester theoretical exam scores, and qualitative analysis of students' 
critical appraisal texts. 
2. Study Design 
This is a prospective, parallel-group, assessor-blinded, cluster randomized controlled 
trial. The study will be rolled out over multiple academic semesters, with each semester 
constituting an independent research cycle to accumulate evidence. 
3. Study Population 

• 
Inclusion Criteria (Applicable per cycle): 
① Full-time undergraduate nursing students. 
② Enrolled in the target surgical nursing course for that cycle. 
③ Aged 18 years or older. 
④ Provide voluntary written informed consent. 
• 
Exclusion Criteria (Applicable per cycle): 
① No-degree students or auditors. 
② Unable to participate in the full study process. 
③ Students who do not take the final theoretical examination. 
④ Students who do not complete the pre- and post-intervention 
questionnaires. 
4. Interventions 
• 
Experimental Group (AI Critical Appraisal Training): 
o Format: Completion of 4 structured case assignments over 8 weeks. 
o Core Task: For each case, students use an AI tool and submit a report 
containing: (1) the complete dialogue log with the AI, (2) critical 
annotations on the AI's responses (identifying at least 3 debatable points 
or errors with corrections and justifications), and (3) a personal 
reflection. 
• 
Control Group (Traditional Case Analysis Training): 
o Format: Completion of the same 4 case assignments over the same 8-
week period. 
o Core Task: Without using AI tools, students analyze cases by consulting 
traditional resources (e.g., textbooks, academic databases) and submit a 

report containing: (1) their self-generated answers, (2) a personal 
reflection, and (3) a declaration confirming no AI use. 
5. Outcome Measures 
• 
Primary Outcome Measure: 
o Instrument: Chinese Version of the Critical Thinking Disposition 
Inventory (CTDI-CV). 
o Timepoints: Baseline (Week 1, pre-intervention) and Post-intervention 
(Week 8). 
• 
Secondary Outcome Measures: 
① AI Literacy: Score on the Artificial Intelligence Literacy Scale for 
Chinese College Students (AILS-CCS), measured at baseline and week 8. 
② Clinical Reasoning Ability: Score on the Chinese Version of the 
Clinical Reasoning and Reflection Self-Assessment Scale, measured at baseline and 
week 8. 
③ Theoretical Knowledge: Score on a unified, end-of-semester written 
examination. 
④Process Evaluation: Qualitative content analysis of the critical 
annotation texts submitted by students in the experimental group. 
6. Statistical Analysis Plan 
• 
Principles: Intent-to-treat analysis will be applied. The clustered nature of the 
design will be accounted for in all analyses. 
• 
Primary Analysis: A linear mixed-effects model will be used to compare the 
change in the primary outcome between groups, with class cluster included as a 
random effect. 
• 
Secondary Analyses: Similar models will be used for secondary quantitative 

outcomes. Thematic analysis will be conducted for qualitative data. 
7. Ethics and Data Management 
• 
This study has been submitted for review and approval to the Biomedical Ethics 
Committee of Chengdu Medical College. 
• 
Participation is entirely voluntary. The informed consent process is separated 
from course grading. 
• 
All collected data will be de-identified and stored confidentially. 
8. Study Timeline 
• 
Overall Duration: March 2026 to August 2028. 
• 
Phase I (Preparation & Registration): January - February 2026. 
• 
Phase II (Pilot Survey): March 2026. 
• 
Phase III (Longitudinal Implementation): April 2026 to December 2027 
(multi-cycle intervention and data collection). 
• 
Final Analysis Completion: August 2028 (anticipated).