Accuracy, satisfaction, and impact of custom GPT in acquiring clinical knowledge: Potential for AI-assisted medical education
Creators: name: Pu, Jiaxi
name: Hong, Jie
name: Yu, Qiao
name: Yu, Pan
name: Tian, Jiaqi
name: He, Yuehua
name: Huang, Hanwei
name: Yuan, Qiongjing
name: Tao, Lijian
name: Peng, Zhangzhe
Publisher: name: Taylor & Francis
Year: 2025
Subjects: subject: Space Science
subject: Medicine
subject: Sociology
subject: FOS: Sociology
subject: Biological Sciences not elsewhere classified
subject: Developmental Biology
subject: Cancer
subject: Science Policy

description: Recent advancements in artificial intelligence (AI) have enabled the customization of large language models to address specific domains such as medical education. This study investigates the practical performance of a custom GPT model in enhancing clinical knowledge acquisition for medical students and physicians. A custom GPT was developed by incorporating the latest readily available teaching resources. Its accuracy in providing clinical knowledge was evaluated using a set of clinical questions, and responses were compared against established medical guidelines. Satisfaction was assessed through surveys involving medical students and physicians at different stages and from various types of hospitals. The impact of the custom GPT was further evaluated by comparing its role in facilitating clinical knowledge acquisition with traditional learning methods. The custom GPT demonstrated higher accuracy (83.6%) compared to general AI models (65.5%, 69.1%) and was comparable to a professionally developed AI (Glass Health, 83.6%). Residents reported the highest satisfaction compared to clerks and physicians, citing improved learning independence, motivation, and confidence (<i>p</i> &lt; 0.05). Physicians, especially those from teaching hospitals, showed greater eagerness to develop a custom GPT compared to clerks and residents (<i>p</i> &lt; 0.05). The impact analysis revealed that residents using the custom GPT achieved better test scores compared to those using traditional resources (<i>p</i> &lt; 0.05), though fewer perfect scores were obtained. The custom GPT demonstrates significant promise as an innovative tool for advancing medical education, particularly for residents. Its capability to deliver accurate, tailored information complements traditional teaching methods, aiding educators in promoting personalized and consistent training. However, it is essential for both learners and educators to remain critical in evaluating AI-generated information. With continued development and thoughtful integration, AI tools like custom GPTs have the potential to significantly enhance the quality and accessibility of medical education.Practice pointsAI-assisted tools, particularly custom GPTs, reform medical education by involving educators in content development, thereby achieving individualized educational goals.Custom GPTs benefit residents by enhancing their experience with clinical knowledge acquisition.Integrating custom GPTs with traditional methods promotes educational equity.Over-reliance on custom GPTs may diminish critical thinking.Future innovations in medical education will combine custom GPTs with other teaching technologies. Practice points AI-assisted tools, particularly custom GPTs, reform medical education by involving educators in content development, thereby achieving individualized educational goals. Custom GPTs benefit residents by enhancing their experience with clinical knowledge acquisition. Integrating custom GPTs with traditional methods promotes educational equity. Over-reliance on custom GPTs may diminish critical thinking. Future innovations in medical education will combine custom GPTs with other teaching technologies.