DHL Survey on Generative AI for MyChart Messaging

Study Overview
=================
Brief Summary
-----------------
The purpose of this study is to understand how patients feel about the use of computer programs to create responses when they send electronic messages to their doctors.

Detailed Description
-----------------
The investigators will create short surveys online to ask patients how they feel about using computer programs that create messages in their medical records. The surveys will show fictional situations where patients ask questions and get answers from either real people or computer programs, with or without a disclosure about how the response was written. The investigators will ask the people taking the survey to share what they think about these situations using tools like rating scales, comparison scales, or written responses. If patients want to, they can provide their contact information to be part of future discussion groups. Participants do not have to give any personal information to complete the survey.

Official Title
-----------------
Duke Health Listens Survey on Generative Artificial Intelligence (AI) for MyChart Messaging

Conditions
-----------------
Communication

Intervention / Treatment
-----------------
* Behavioral: Generative AI for electronic communication and disclosure


Participation Criteria
=================
Eligibility Criteria
-----------------
Inclusion Criteria: Member of the Duke Health Listens patient advocacy community Exclusion Criteria: Age < 18

Ages Eligible for Study
-----------------
Minimum Age: 18 Years

Sexes Eligible for Study
-----------------
All

Accepts Healthy Volunteers
-----------------
Accepts Healthy Volunteers

Study Plan
=================
How is the study designed?
-----------------

Design Details

Primary Purpose: Other
Allocation: Randomized
Intervention Model: Parallel Assignment
Interventional Model Description: Participants will be randomly allocated to one of six arms. Over the course of the study period, there will be multiple rounds of surveys, which will consist of a clinical scenario, a response (human or AI generated), and a disclosure statement (none, human, or AI). All respondents within an arm will be assigned to a sequence of response-disclosure pairs prior to the first round of surveys.
Masking: Single


Arms and Interventions

| Participant Group/Arm | Intervention/Treatment |
| --- | --- |
| Other: Arm A<br>Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm A receives AHN in Send 1, BAIC in Send 2, and CHH in Send 3 | Behavioral: Generative AI for electronic communication and disclosure<br>* We will use a large language model such as GPT 3.5 to automatically generate responses to fictional messages to a physician. We will disclose whether the message was generated using this technology or not. There are 3 clinical scenarios and 6 pairs of human/AI response and human disclosure/AI disclosure/not disclosed that will test patient attitudes toward this technology.<br>|
| Other: Arm B<br>Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm B receives BHC in Send 1, CAIH in Send 2, and AAIN in Send 3 | Behavioral: Generative AI for electronic communication and disclosure<br>* We will use a large language model such as GPT 3.5 to automatically generate responses to fictional messages to a physician. We will disclose whether the message was generated using this technology or not. There are 3 clinical scenarios and 6 pairs of human/AI response and human disclosure/AI disclosure/not disclosed that will test patient attitudes toward this technology.<br>|
| Other: Arm C<br>Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm C receives CHC in Send 1, AHH in Send 2, and BAIN in Send 3 | Behavioral: Generative AI for electronic communication and disclosure<br>* We will use a large language model such as GPT 3.5 to automatically generate responses to fictional messages to a physician. We will disclose whether the message was generated using this technology or not. There are 3 clinical scenarios and 6 pairs of human/AI response and human disclosure/AI disclosure/not disclosed that will test patient attitudes toward this technology.<br>|
| Other: Arm D<br>Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm D receives AAIH in Send 1, BHN in Send 2, and CAIC in Send 3 | Behavioral: Generative AI for electronic communication and disclosure<br>* We will use a large language model such as GPT 3.5 to automatically generate responses to fictional messages to a physician. We will disclose whether the message was generated using this technology or not. There are 3 clinical scenarios and 6 pairs of human/AI response and human disclosure/AI disclosure/not disclosed that will test patient attitudes toward this technology.<br>|
| Other: Arm E<br>Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm E receives BAIH in Send 1, CHN in Send 2, and AHC in Send 3 | Behavioral: Generative AI for electronic communication and disclosure<br>* We will use a large language model such as GPT 3.5 to automatically generate responses to fictional messages to a physician. We will disclose whether the message was generated using this technology or not. There are 3 clinical scenarios and 6 pairs of human/AI response and human disclosure/AI disclosure/not disclosed that will test patient attitudes toward this technology.<br>|
| Other: Arm F<br>Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm F receives CAIN in Send 1, AAIC in Send 2, and BHH in Send 3 | Behavioral: Generative AI for electronic communication and disclosure<br>* We will use a large language model such as GPT 3.5 to automatically generate responses to fictional messages to a physician. We will disclose whether the message was generated using this technology or not. There are 3 clinical scenarios and 6 pairs of human/AI response and human disclosure/AI disclosure/not disclosed that will test patient attitudes toward this technology.<br>|

What is the study measuring?
-----------------
Primary Outcome Measures

| Outcome Measure | Measure Description | Time Frame |
| --- | --- | --- |
| Patient satisfaction, as measured by survey | Likert-scale responses to satisfaction question: I am satisfied with this interaction, on a scale from 1-5 with answer options of Strongly Disagree (1), Disagree (2), Neither agree nor disagree (3), Agree (4), and Strongly agree (5). | Up to 2 weeks | 
| Patient attitudes towards utility, as measured by survey | Likert-scale responses to utility question: The information is useful, on a scale from 1-5 with answer options of Strongly Disagree (1), Disagree (2), Neither agree nor disagree (3), Agree (4), and Strongly agree (5). | Up to 2 weeks | 
| Patient empathy, as measured by survey | Likert-scale responses to empathy question: I feel cared for during this interaction, on a scale from 1-5 with answer options of Strongly Disagree (1), Disagree (2), Neither agree nor disagree (3), Agree (4), and Strongly agree (5). | Up to 2 weeks | 

 Terms related to the study
=================
Keywords Provided by Centre Hospitalier Valida
-----------------
Artificial Intelligence, Generative Artificial Intelligence