Clinical trial NCT05963802: Evaluation of the Efficacy and Usability of Artificial Intelligence (ChatGPT) for Health Sciences Students
Official title: Crossover Randomized Controlled Trial to Evaluate the Efficacy and Usability of Artificial Intelligence (ChatGPT) for Health Sciences Students (AIHSS)
Study type: INTERVENTIONAL
Overall status: COMPLETED
Phase: NA
Conditions: Educational Activities
AI (Artificial Intelligence)
Keywords: Artificial intelligence
Education
Health Science Students
Lead sponsor: Carleton University
Sponsor class: OTHER
Collaborators: name: Université de Montréal
First posted: 2023-07-27
Enrollment: 26
Sex: ALL
Minimum age (years): 18
Standard ages: ADULT
OLDER_ADULT
Accepts healthy volunteers: yes
Allocation: RANDOMIZED
Intervention model: CROSSOVER
Intervention model description: We employed a crossover design to ensure equity in the learning experience for all students. This involved assigning students either the ChatGPD (A) or the regular online tools (B) for accessing resources and assisting them with their assignments. During the initial phase of the intervention, participants in group A will utilize ChatGPT to complete their assignments, while participants in Group B will serve as the control group. In the subsequent period, the interventions will switch between the two groups.
Primary purpose: OTHER

Brief summary: Crossover Randomized Control trial, in which subjects are randomly assigned to one of two groups: one (ChatGPT) receiving the intervention that is being tested, and the control group receiving usual online resources.

Detailed description: A two-stage randomization strategy will be adopted. The allocation of participants into event groups A and B will be determined through computer-generated randomization in an allocation ratio: 1:1. To address ethical concerns regarding power dynamics between professors and students; the investigators have opted to involve an individual who is independent of the research team. This decision aims to minimize any potential bias or influence that could arise from direct professor involvement in the intervention. This approach helps ensure that the study is conducted fairly and unbiasedly, fostering a research environment that prioritizes equal opportunities for all participants. Both groups will simultaneously work on an assignment (A and B).

The investigators employed a cross-over design to ensure equity in the learning experience for all students. This involved assigning students either the ChatGPD (A) or the regular online tools (B) for accessing resources and assisting them with their assignments. During the initial phase of the intervention, participants in group A will utilize ChatGPT to complete their assignments, while participants in Group B will serve as the control group. In the subsequent period, the interventions will switch between the two groups. To assess the participants' technology usability, both groups will complete pre-and post-questionnaires. Additionally, participants will be requested to fill out a post-intervention questionnaire to evaluate their perceptions of the assistance methods. After a period of a wash-out of 13 days/week, participants who were previously allocated to Group A will be allocated to Group B and the same with participants previously allocated to Group B will be to Group A. This crossover randomized control trial will include 50 health sciences students who are registered to the course Chronic Health Conditions and Disability ( HLTH 3503), Carleton University. During the first part of the intervention, A group will use ChatGPT to complete their assignment, while B participants will act as a control group. In the second week, the interventions will cross over. Both groups' participants will complete a pre-and post-questionnaire to assess their knowledge. Participants will also complete a post-intervention questionnaire to evaluate their perceptions. The System Usability Scale (SUS) will be administered at the end of the interventions and when participants cross over to the other assistance tool. This allows for the assessment of outcomes and comparisons within and between the two intervention arms. By collecting data at multiple points in the study, the investigators can evaluate the effects of each intervention and any potential carryover effects during the crossover period. Additionally, it provides an opportunity to measure changes in outcomes within individual participants as they switch from one intervention to another. The questionnaire about socioeconomic, literacy questions, Benefits of AI and risks and negative experiences associated with AI usage.

Eligibility criteria: Inclusion Criteria:

* Health Science students
* Must be registered for the course Chronic Health Conditions and Disability (HLTH 3503), Fall 2023.

Exclusion Criteria:

• Unable to provide informed consent

Primary outcomes:
measure: System Usability Scale (SUS) scores are 0-100,The average SUS score is 68.
timeFrame: within 24 hours
Secondary outcomes:
measure: Student's perception of artificial intelligence, questionnaire reviwed by experts, No recognised gold standard measurement for outcome.
timeFrame: within 24 hours
Other outcomes:
Locations:
city: Ottawa
country: Canada
facility: Carleton University
Study officials:
affiliation: Carleton University
name: Mirella Veras, PhD
role: PRINCIPAL_INVESTIGATOR
References:
citation: Dwan K, Li T, Altman DG, Elbourne D. CONSORT 2010 statement: extension to randomised crossover trials. BMJ. 2019 Jul 31;366:l4378. doi: 10.1136/bmj.l4378.
citation: Luna A, Casertano L, Timmerberg J, O'Neil M, Machowsky J, Leu CS, Lin J, Fang Z, Douglas W, Agrawal S. Artificial intelligence application versus physical therapist for squat evaluation: a randomized controlled trial. Sci Rep. 2021 Sep 13;11(1):18109. doi: 10.1038/s41598-021-97343-y.
citation: Liu X, Rivera SC, Moher D, Calvert MJ, Denniston AK; SPIRIT-AI and CONSORT-AI Working Group. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI Extension. BMJ. 2020 Sep 9;370:m3164. doi: 10.1136/bmj.m3164.
citation: Zhou L, Bao J, Setiawan IMA, Saptono A, Parmanto B. The mHealth App Usability Questionnaire (MAUQ): Development and Validation Study. JMIR Mhealth Uhealth. 2019 Apr 11;7(4):e11500. doi: 10.2196/11500.