Clinical trial NCT05816473: Artificial Intelligent Clinical Decision Support System Simulation Center Study for Technology Acceptance
Official title: Artificial Intelligent Clinical Decision Support System Simulation Center Study: Trust and Usefulness of Machine Learning Risk Stratification Tool for Acute Gastrointestinal Bleeding Using the Technology Acceptance Model
Study type: INTERVENTIONAL
Overall status: COMPLETED
Phase: NA
Conditions: Gastrointestinal Hemorrhage
Keywords: Implementation science
Artificial intelligence
Decision Support Systems, Clinical
Simulation Training
Lead sponsor: Yale University
Sponsor class: OTHER
Collaborators: name: National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK)
First posted: 2023-04-18
Enrollment: 106
Sex: ALL
Standard ages: CHILD
ADULT
OLDER_ADULT
Accepts healthy volunteers: yes
Allocation: NA
Intervention model: PARALLEL
Primary purpose: HEALTH_SERVICES_RESEARCH

Brief summary: The purpose of this research study is to measure the effect on of a large language model interface on the usability, attitudes, and provider trust when using a machine learning algorithm-based clinical decision support system in the setting of bleeding from the upper gastrointestinal tract (upper GIB). Specifically, the investigators are looking to assess the optimal implementation of such machine learning algorithms in simulation scenarios to best engender trust and improve usability. Participants will be randomized to either machine learning algorithm alone or algorithm with a large language model interface and exposed to simulation cases of upper GIB.

Detailed description: The experiment will deploy a previously validated machine learning algorithm trained on existing clinical datasets within simulation scenarios in which a patient with acute gastrointestinal bleeding (at low, moderate, and high risk for poor outcome) is evaluated.

Prior to the simulation, a baseline educational module about artificial intelligence, machine learning, and clinical decision support will be provided to all participants. The investigators will establish psychological safety by detailing what is available in the room, the opportunity to call a consultant, and availability of laboratory and radiographic studies. Each clinical scenario will run for approximately 10 minutes based on real patient cases where vital signs change over time and laboratory values are made available at specific points in the assessment. The study will evaluate the effect of a large language model-based interaction with the machine learning algorithm with interpretability dashboard compared to the machine learning algorithm with interpretability dashboard alone. Each participant will receive three scenarios in randomized order of risk.

For the large language model interaction arm, participants will be provided the computer workstation a LLM chatbot interface of the algorithm and interpretability dashboard For the machine learning dashboard arm, participants will be provided the computer workstation with the algorithm and interpretability dashboard.

Eligibility criteria: Inclusion Criteria:

* Internal Medicine residency trainees at study institution
* Emergency Medicine residency trainees at study institution

Exclusion Criteria:

* N/A

Primary outcomes:
measure: Change in Attitudes Towards Machine Learning Algorithms in Clinical Care using UTAUT
timeFrame: Approximately 60 minutes
measure: Clinician Decision Making of Triage of GI bleeding
timeFrame: Approximately 60 minutes
Secondary outcomes:
Other outcomes:
Locations:
city: New Haven
country: United States
facility: Yale New Haven Hospital
state: Connecticut
Study officials:
affiliation: Yale School of Medicine Section of Digestive Diseases
name: Dennis Shung, MD
role: PRINCIPAL_INVESTIGATOR
References:
citation: Laine L. Risk Assessment Tools for Gastrointestinal Bleeding. Clin Gastroenterol Hepatol. 2016 Nov;14(11):1571-1573. doi: 10.1016/j.cgh.2016.08.003. Epub 2016 Aug 10. No abstract available.
citation: Laine L, Jensen DM. Management of patients with ulcer bleeding. Am J Gastroenterol. 2012 Mar;107(3):345-60; quiz 361. doi: 10.1038/ajg.2011.480. Epub 2012 Feb 7.
citation: Leonardi, P. M. 2009. Why do people reject new technologies and stymie organizational changes of which they are in favor? Exploring misalignments between social interactions and materiality. Human Communication Research, 35(3): 407-441.
citation: Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27(3), 425-478