Clinical trial NCT06997107: AI to Create Accessible & Reliable Patient Education Materials
Official title: AI-CARE: AI to Create Accessible & Reliable Patient Education Materials
Study type: INTERVENTIONAL
Overall status: ACTIVE_NOT_RECRUITING
Phase: NA
Conditions: Primary Care Patients
Keywords: primary care
generative artificial intelligence
health promotion messages
patient education
primary care providers
primary care patients
Lead sponsor: Hopital Montfort
Sponsor class: OTHER
Collaborators: name: Cheo Research Institute
name: Eastern Ontario Health Unit
First posted: 2025-05-30
Enrollment: 50
Sex: ALL
Minimum age (years): 18
Standard ages: ADULT
OLDER_ADULT
Accepts healthy volunteers: yes
Allocation: RANDOMIZED
Intervention model: CROSSOVER
Intervention model description: Each month, patients and providers will receive a series of health promotion messages generated by AI and humans.
Primary purpose: HEALTH_SERVICES_RESEARCH

Brief summary: This study investigates the use of Generative AI (GAI) to support primary care practices in delivering accurate, accessible patient education. With the rise of health misinformation, increasingly complex patient needs, and a strained healthcare workforce, primary care must find new ways to communicate trusted health information effectively. Leveraging the Canadian Primary Care Information Network (CPIN), this study will generate patient education messages on key health topics using both GAI and human content experts.

Diverse review panels of patients and providers will assess the messages on quality of information, adaptability, and relevance and usefulness, with special attention to socioeconomic factors that may impact message accessibility. CPIN will recruit a diverse sample of participants to evaluate both GAI- and human-generated messages. Review panels will provide structured feedback via surveys, aiming to identify differences in content quality and effectiveness.

The study's goal is to determine whether GAI can produce high-quality health information that meets primary care standards. Results will reveal how GAI tools can support primary care in reducing misinformation and administrative burdens, fostering patient-provider relationships, and improving health equity. Findings will inform best practices for integrating GAI in primary care to ensure accessible, timely patient education across Canada.

Detailed description: Background. The increasing prevalence of health misinformation, complex patient needs, and a strained healthcare workforce necessitate innovative approaches to patient education in primary care. Generative AI (GAI) offers the potential to deliver accurate, accessible health information while reducing administrative burdens. This study explores the use of GAI to support primary care practices in producing trusted, high-quality patient education materials.

Objective. The investigators propose to leverage advances in GAI and our experience with CPIN to provide timely and accurate health information for primary care practices across Canada. Our goal is to determine whether GAI can produce education material for primary care that is non-inferior compared to experts in primary care and public health.

Methods. The content team for this study will consist of experts specializing in primary care, public health, or health communication. Team members will create digital health messages in two formats: a short, text-messaging format (850 characters or less), and a one-page handout for patients. On the other hand, a generative AI system will also generate messages. Topics and prompts for message writing will be provided to both the content team and the GAI. Messages in English and French will be available.

To evaluate the generated content, two review panels (a review panel of 25 providers and one of 25 patients) will assess messages created by both human experts and generative AI over the course of 12 months. Each month, using an evaluation grid provided to assess the quality of information, adaptability, and relevance and usefulness of the message, panelists will review a total of 16 messages (four topics x 4 messages).

Panelists will be blinded to the message generation source (AI or human). Short messages will be shown first to minimize potential bias from the detailed information in longer messages and ensure their clarity and completeness are effectively assessed.

Both providers and patients on the review panels will complete assessments via REDCap surveys. The evaluation grid will be the same for providers and patients and will use a Likert scale from 1 to 4 (1: Strongly disagree; 4: Strongly agree). Specifically, there will be statements on Adaptability (subcategories: Clarity and understandability, Appropriate emotional appeal, Appropriate rational appeal, Tone, and Inclusivity) and Relevance and Usefulness. Statements on Quality of Information (subcategories: Accuracy, Reliability, and Completeness) will only be asked to providers. Patients, in contrast, will be asked at the end whether they noticed any inaccuracies in the message.

Eligibility criteria: Inclusion Criteria:

* Content team members: Content team members must have an expertise in primary care, public health, or health communication. The investigators aim to recruit at least two or three members whose mother tongue is French.
* Providers' review panel: The investigators will recruit a diverse group of primary care providers. All providers who provide comprehensive care in Canada and will be eligible to participate. They must be proficient in either written English or French. They must be able to consent to participate in this study.
* Patients' review panel: The investigators will recruit a diverse group of patients. All must be aged 18 years or older and be proficient in either written English or French. They must be able to consent to participate in this study.

Exclusion Criteria:

* Content team members, primary care providers, or patients who cannot write (content team members) or read (review panels) in either French or English will be excluded. Primary care providers or patients who do not have an email address, a computer or a cellphone to complete the evaluations on REDCap will also be excluded. The investigators will not include minors or patients who cannot provide informed consent themselves, such as those with advanced dementia.

Primary outcomes:
measure: Clarity and Understandability score
timeFrame: 12 months
Secondary outcomes:
measure: Overall message score
timeFrame: 12 months
measure: Message category and subcategory scores
timeFrame: 12 months
Other outcomes:
Locations:
city: Ottawa
country: Canada
facility: Institut du Savoir Montfort
state: Ontario
Study officials:
affiliation: Institut du Savoir Montfort
name: Sharon Johnston, MD, LLM
role: PRINCIPAL_INVESTIGATOR
affiliation: Institut du Savoir Montfort
name: William Hogg, MD, MSc
role: PRINCIPAL_INVESTIGATOR
References:
citation: Tam TYC, Sivarajkumar S, Kapoor S, Stolyar AV, Polanska K, McCarthy KR, Osterhoudt H, Wu X, Visweswaran S, Fu S, Mathur P, Cacciamani GE, Sun C, Peng Y, Wang Y. A framework for human evaluation of large language models in healthcare derived from literature review. NPJ Digit Med. 2024 Sep 28;7(1):258. doi: 10.1038/s41746-024-01258-7.
citation: Bedi S, Liu Y, Orr-Ewing L, Dash D, Koyejo S, Callahan A, Fries JA, Wornow M, Swaminathan A, Lehmann LS, Hong HJ, Kashyap M, Chaurasia AR, Shah NR, Singh K, Tazbaz T, Milstein A, Pfeffer MA, Shah NH. Testing and Evaluation of Health Care Applications of Large Language Models: A Systematic Review. JAMA. 2025 Jan 28;333(4):319-328. doi: 10.1001/jama.2024.21700.