A Patient Portal Intervention to Address Diabetes Care Gaps: A Usability Study
Phase: NA
Conditions: Diabetes Mellitus
Sponsor: Vanderbilt University Medical Center

 
 
 
 
 
 
 
 
A Patient Portal Intervention to Address Diabetes Care Gaps: A Usability Study 
 
NCT04728620 
 
Protocol and Statistical Analysis Plan  
 
February 8, 2022 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Table of Contents: 
 
Study Schema 
1.0 
Background 
2.0 
Rationale and Specific Aims 
3.0 
Inclusion/Exclusion Criteria 
4.0 
Enrollment/Randomization 
5.0 
Study Procedures 
6.0 
Statistical Analysis Plan 
 

1.0 
Background 
Evidence-based diabetes monitoring and preventative services can prevent or delay many costly and highly morbid 
disease related complications, yet many patients do not receive all clinically-meaningful, evidence-based services. For 
example, while detecting and treating early diabetic eye disease can reduce the development of severe vision loss by an 
estimated 60%, approximately 40% of Americans with diabetes do not receive an annual diabetes eye exam. Prior 
research has demonstrated numerous barriers to the completion of evidence-based diabetes monitoring and 
preventative services including patient factors (e.g., lack of awareness and limited health literacy) and clinician/system 
factors (e.g., limited physician time and patient support between visits).  Attempts to increase diabetes monitoring and 
preventative services have had only modest results. To achieve optimal rates of these services, an intervention is 
urgently needed that improves clinical efficiency by decreasing clinician workload, is appropriate for patients with 
varying levels of health literacy, and is highly scalable and sustainable. By providing an engaging and convenient means 
to track and visualize health data, obtain education and guidance, receive notifications, and connect patients and 
doctors, patient portals offer a promising platform to enhance access to health services while overcoming the limitations 
of costly and difficult to scale face-to-face interventions. 
 
2.0 
Rationale and Specific Aims 
Thus, in this study, we will assess the usability and acceptability of a novel patient portal intervention designed in a prior 
study to: (a) notify patients when selected, clinically meaningful, evidence-based diabetes monitoring & preventative 
services (e.g., annual diabetes eye exam) become due and provide reminders for timely completion, (b) promote 
understanding of the importance of these services through literacy sensitive content, and (c) allow patients, when due 
by evidence-based guidelines, to initiate orders for the care (e.g., pneumococcal vaccination).  In addition, we will assess 
pre-post change on secondary psychosocial outcomes including patient activation. 
 
3.0 
Inclusion/Exclusion Criteria 
Participants will be eligible if they receive care at a participating VUMC clinic site and they have type 1 or 2 diabetes 
mellitus (DM), are able to speak and read in English, are ages 18 to 75, have a current MHAV account and reliable access 
to a mobile device with iOS or android operating system with internet access, and are due for any of the five guideline-
based, diabetes monitoring and preventative services (i.e., hemoglobin A1C, urine microalbumin, diabetes eye exam, 
and/or pneumococcal vaccination).  
 
We will exclude patients with medical conditions that affects their memory or ability to think which may preventing the 
use of a mobile device, severe difficulty seeing, severe difficulty hearing, a medical condition that make it hard for 
people to understand what they are saying, as well as pregnant women or women who plan to become pregnant during 
the study period.  
   
4.0 
Enrollment/Randomization 
Participants will be able to complete an electronic consent form and enroll online via Research Electronic Data Capture 
(REDCap™) version 5.0.8. 
 
5.0 
Study Procedures 
Setting. The study will be performed at Vanderbilt Primary Care One Hundred Oaks – North and South Clinics located 
within a large ambulatory care facility in Nashville, TN. An EHR (Epic Systems Corp.) stores all clinical data and patients 
receive access to their clinical data via an integrated and highly-adopted patient web portal, My Health at Vanderbilt 
(MHAV), that is accessible on desktops and via a native mobile app for iOS and Android mobile operating systems. 
 
Participants and Recruitment. Potential participants will be identified automatically using VUMC’s Subject Locator to 
query the EHR for patients with upcoming clinic appointments who meet the discrete inclusion and exclusion criteria. In 
addition, VUMC population health and quality improvement initiatives have created a report that identifies established 
patients with diabetes who receive care at the primary care clinics participating in the study and have current diabetes 
care gaps (e.g., no diabetes eye exam in the last year). These patients are the target population for study (i.e., Vanderbilt 
primary care patients with diabetes) and as such these patients are potentially eligible to participate in the study. As 
with Subject Locator, with the permission of the clinic director, these patients would be mailed a recruitment letter 
describing the study and providing contact information for IRB approved study personnel. With approval of the clinic 

director, we will send potentially eligible patients a recruitment letter describing the study and providing them with the 
contact information for IRB-approved study personnel.  If after receiving the letter, a patient is interested in the study, 
they may contact study personnel to learn more and consider participating in the study. The recruitment letter will 
include a QR code and URL link to a REDCap site where interested patients can check their eligibility for study, review 
the consent form, and enroll.  
 
Participants will complete an electronic consent form and enroll online via Research Electronic Data Capture (REDCap™) 
version 5.0.8. Our target enrollment is 60 participants. In accordance with best practices and to reflect groups with 
distinct usability needs, we will purposively sample to achieve representation of each of the following characteristics: (a) 
limited health literacy and (b) age over 65. Some participants may have one or more of these characteristics. 
 
Intervention. The intervention involves the addition of a new feature to the Vanderbilt patient portal that will allow 
patients to use the My Health at Vanderbilt (MHAV) app on their mobile device (smartphone or tablet) to: (1) receive 
notifications when they are due for certain types of diabetes monitoring and preventative care and (2) initiate an order 
for the care.  Automated notifications will be sent to patients if, according to the Health Maintenance section of their 
electronic health record and medical guidelines, they are due for a hemoglobin A1C blood test, microalbumin (kidney) 
urine test, diabetes eye exam, and/or pneumonia vaccine. If a patient receives a notification, the patient can initiate an 
order for the care using the MHAV app.  After the patient initiates the order, the study team will process the order. 
Daily, the study team will use Epic’s reporting workbench to identify patients that initiated orders and use the EHR’s 
batch order functionality to generate the actual order for the patient’s care (e.g., HgbA1c lab test).  The patient will 
receive confirmation when the order has been processed and instructions to proceed to a Vanderbilt clinic or lab to 
receive the care.  The patient’s primary care physician will be the authorizing provider and once the patient receives the 
care, their primary care doctor will receive any results. If the patient has already received the care outside of Vanderbilt, 
the patient will be able to indicate this.  For example, if according to the patient’s Vanderbilt medical record they are 
due for a diabetes eye exam, they will receive a notification; however if they recently had a diabetes eye exam with an 
eye doctor outside of Vanderbilt, they will be able to report this using the same new MHAV feature.   
 
Patients reporting an outside eye exam will be contacted by study staff and asked to complete an Authorization for 
Release of Medical Information to VUMC via REDCap.  If the patient agrees and signs the release, trained study staff will 
request the records accordingly.  Once the eye exam records are received, trained study staff will review the records to 
determine if the records document screening for diabetic eye disease (i.e., was the participant’s self-report of a diabetes 
eye exam accurate).  If the eye exam records do not indicate screening for diabetes related eye disease (i.e., a diabetes 
eye exam), KSP will contact the participant by phone with this information and encourage the participant to call their 
eye doctor to schedule a diabetes eye exam. If the participant prefers, the participant will be invited to schedule a 
diabetes eye exam at Vanderbilt Eye Institute. If the participant can’t be reached by phone, they will be sent a letter 
with this information. Any outside eye exam records received will be scanned into the participant’s VUMC medical 
record and their primary care physician will be notified.  
 
Data Collection and Outcome Measures. Study participants will complete questionnaires electronically via email using 
REDCap™ at three time points:  enrollment (T0), after first use of the intervention (T1), and end of study (T2). Participants 
will complete an enrollment questionnaire (T0) including basic demographic questions, items about computer and 
internet use, and eHealth literacy.  Based on prior experience, we estimate time to completion for each questionnaire to 
be about 20 minutes. Participants will be compensated $30 each for completing questionnaires ($90 total if they 
complete all three).  In addition, participants can receive $10 for using the intervention for the first time.   
 
The primary outcome measures (Table 1) will be: (a) ease of use and satisfaction as assessed by the System Usability 
Scale at T1, (b) system usage data as assessed by user analytics at T2, (c) user experience as assessed by ‘after first use’ 
and final questionnaires and semi-structured interviews at T2. In addition, we will assess pre/post change in the 
following secondary psychosocial outcomes (Table 1) to estimate effect size and standard deviations for power analyses 
necessary to plan a subsequent randomized controlled trial to be proposed in a subsequent IRB application. 
 
Table 1. Outcome Measures 

Primary Outcomes 
Measures 
Variable 
Type 
How 
Collected 
Time 
Point 
Usability  
System Usability Scale (SUS)  
Continuous 
Questionna
ire 
T1 
User experience 
Unique study specific items to assess participants’ 
perspectives on content, acceptance, and features and 
functionality 
Categorical 
Questionna
ire & 
Interviews 
T1 and 
T2 
Secondary 
Outcomes 
Measures 
Variable 
Type 
How 
Collected 
Time 
Point 
Diabetes self-
efficacy 
Manage Disease in General Scale of the Chronic Disease 
Self-Efficacy Scales 
Continuous 
Questionna
ire 
T0 – T2 
Understanding of 
Diabetes 
Monitoring and 
Preventative Care 
Unique study specific items to assess participants' 
understanding of recommended diabetes monitoring 
and preventative care 
Continuous  Questionna
ire 
T0 – T2 
Diabetes distress 
Problem Areas in Diabetes Scale (PAID-5)  
Continuous 
Questionna
ire 
T0 – T2 
Patient-initiated 
orders 
# of patient-initiated orders for evidence-based 
diabetes monitoring & preventative services (e.g., A1c) 
Continuous 
EHR 
Abstraction
, Tableau & 
Clarity 
Servers 
T2 
Service 
completion 
# of completed (i.e., care received) patient-initiated 
orders for evidence-based diabetes monitoring & 
preventative services (e.g., A1c) 
Continuous 
EHR 
Abstraction
, Tableau & 
Clarity 
Servers 
T2 
T0 = baseline, T1 = after first use of the intervention, T2 = 3-month follow up 
 
At the conclusion of their 3-months of access to the intervention, we will conduct one-on-one, semi-structured 
interviews with a subsample of at least ten participants including at least two participants from each of the purposively 
sampled groups (see Recruitment above).  In addition, we will conduct semi structured interviews with up to five 
primary care physicians (PCPs) that cosigned a patient-initiated order to under their perceptions of the intervention and 
impacts on workflows. This methodology is most appropriate for in-depth assessment of individuals’ perceptions and 
reactions regarding the intervention. Interviews are preferred over focus groups for understanding usability because 
focus groups can amplify bias and individual opinions. Participant interviews will take place by phone or via Zoom within 
two weeks of participants concluding their 3 months of access to the intervention and will last approximately 30 
minutes. PCP interviews will take place after all participants have completed the study. A trained interviewer will use a 
semi-structured interview guide to facilitate the interviews and elicit in-depth understanding of individuals’ perceptions 
and experiences with the intervention. Additional participants will be interviewed until saturation is reached. Saturation 
will be defined as no new usability concerns raised in the preceding two interviews and typically occurs between 10 and 
30 interviews. Interviewees will be compensated $40 for their time.  
 
6.0 
Statistical Analysis Plan 
We will use descriptive statistics to characterize the study participants and survey data. We will use a one sample t-test 
to compare the mean SUS score at T1 to the threshold score of 71 indicative of “good” usability. To assess whether there 
was a significant improvement in the continuous secondary cognitive/behavioral outcomes from baseline to end of 
study (T0 to T1), we will perform two-sided paired t-tests on the pairwise differences. If any of the distributions of 
pairwise differences suggest asymmetry or nonnormality, the non-parametric Wilcoxon signed-rank test will be 
performed in lieu of paired t-tests. We will use the McNemar’s test for to compare paired proportions. Our primary 
outcome will be System Usability Scale (SUS) score at first use (T1).