(096) REAL WORLD UTILITY OF CHATGPT IN PRE-VASECTOMY COUNSELLING IN AN OFFICE-BASED SETTING: A PILOT STUDY
Authors: Chung D , Dhillon H , Sidhom K , Bal D , Patel P , Jawanda G 
Published: December 2024

Introduction

There is increasing interest among the medical community on to developing new ways to utilize ChatGPT. New applications have included writing scientific manuscripts, drafting correspondents, as well as testing its ability on standardized medical licensing examinations. Despite ChatGPT demonstrating good medical knowledge, it is still unclear if this translates to real world clinical practice. A potential application of this technology is to provide patient counseling and to improve patient flow in one’s practice.

Objective

Our goal is to assess if pre-vasectomy counseling with ChatGPT safely streamlines the consultation process by reducing visit times and increasing patient satisfaction through the consultation process.

Methods

A single institutional prospective, randomized single-blinded pilot study was conducted to evaluate the safety and efficacy for the use of ChatGPT (v4.0) for pre-vasectomy counselling. All eligible patients who are interested in undergoing vasectomy were included in the study. Patients were excluded if unwilling to participate in the study or did not have a home computer/internet connection to interact with the ChatGPT software. Patient were randomized 1:1 to pre-appointment counselling with ChatGPT + standard in-person consultation to in-person consultation alone. Baseline demographic information was collected including age, level of education. Outcomes collected at that time were length of visit, number of questions, as well as a Likert scale (/10) questionnaire assessing participant satisfaction in the ChatGPT counselling. Chat logs were reviewed by 2 men’s health specialists to assess the accuracy and safety of ChatGPT responses. Descriptive statistics were performed as well as comparative analysis with the independent sample t-test method.

Results

Total of 18 patients were analyzed with a mean age of 35.8 (±) 5.4 (n = 9) in the intervention arm and 36.9 (±) 7.4 (n = 9) in the control arm. Overall pre-vasectomy counselling with ChatGPT was associated with provider perceived improved understanding of the procedures (8.8 ± 1.0 vs 6.7 ± 2.8; p = 0.047) and decreased length of in-person consultation (7.7 ± 2.3 min vs 10.6 ± 3.4 min; p = 0.05). Patients however reported no difference in how they rated the quality of their clinical encounter, with excellent scores seen in all domains. Patient experiences with ChatGPT in respect to pre-vasectomy counseling was also surveyed. Quality of information provided by ChatGPT, ease of access/use, and overall experience were rated highly at 8.3 (±) 1.9, 9.1 (±) 1.5, and 8.6 (±) 1.7 respectively. Some participants did express concern regarding the confidentiality of information at 4.9 (±) 2.9. Of the 19 total questions and responses provided by ChatGPT, 95% (n = 18/19) were considered to be accurate responses.

Conclusions

ChatGPT for pre-vasectomy counselling did show to improve the efficiency of encounters and provider perceived patient understanding of the procedure. ChatGPT was also able to provide accurate responses to questions regarding the vasectomy in 95% of cases.

Disclosure

Any of the authors act as a consultant, employee or shareholder of an industry for: Boston Scientific.