The My Diabetes Care Patient Portal Intervention for Mobile Devices: A Usability Study
Phase: NA
Conditions: Diabetes Mellitus, Type 2
Sponsor: Vanderbilt University Medical Center

 
 
 
 
 
 
 
 
 
 
The My Diabetes Care Patient Portal Intervention for Mobile Devices: A Usability Study 
 
NCT04894916  
 
Protocol and Statistical Analysis Plan  
 
May 22, 2021 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Table of Contents: 
 
Study Schema 
1.0 
Background 
2.0 
Rationale and Specific Aims 
3.0 
Inclusion/Exclusion Criteria 
4.0 
Enrollment/Randomization 
5.0 
Study Procedures 
6.0 
Statistical Analysis Plan 
 
 

1.0 
Background 
Diabetes is a leading cause of cardiovascular disease, renal failure, vision loss and non-traumatic lower limb 
amputations. Diabetes self-management can prevent or delay diabetes-related complications, yet patients struggle to 
consistently engage in recommended self-care behaviors. Patients with diabetes may lack the understanding and 
motivation necessary to successfully manage their disease. Among patients with diabetes, studies have found that 
higher patient activation (i.e., knowledge, skills and confidence to manage their own health and care) is associated with 
healthy eating, blood sugars monitoring, eye exams and immunizations. Many of these findings were reported in diverse 
and disadvantaged populations. 
 
Patient portals offer a vehicle for interventions aimed at increasing patient activation, improving understanding, and 
promoting self-management, while overcoming the limitations of costly and difficult to scale face-to-face programs. We 
recently applied user-centered design sprint methodology and key strategies for patient engagement to develop a multi-
faceted patient portal intervention, My Diabetes Care (MDC), designed to help patients better understand their diabetes 
health data as well as support self-management. MDC was embedded within an established patient portal, My Health at 
Vanderbilt (MHAV), on Epic’s MyChart platform and best viewed on desktop devices.  MDC uses infographics to visualize 
and summarize patients' diabetes health data, incorporates motivational strategies (e.g., social comparisons), and 
provides literacy-level appropriate educational resources. MDC is grounded in a well-established Chronic Care Model 
(CCM) adapted for eHealth (i.e., healthcare practices supported by electronic processes and communication). By 
leveraging elements within the Model’s five domains (self-management support, delivery system design, decision 
support, clinical information systems, and eHealth education), MDC has the potential to create more informed and 
activated patients leading to improved outcomes.   
 
In a prior longitudinal study of the MDC, participants, including those with limited HL, highly rated its usability, 
anticipated continued use, and reported that it improved their understanding of their personal diabetes health data. In 
addition, participants’ patient activation scores significantly increased over the study period. When asked to identify 
enhancement that would increase their satisfaction with MDC participants indicated a desire use MDC on mobile 
devices. Moreover, most patient portal users now access the portal via mobile devices. Thus, we designed and assessed 
task-based usability of a mobile-friendly version of MDC. We found the final prototype of MDC Mobile (MDC-m) was 
very satisfying for participants to use.  
 
2.0 
Rationale and Specific Aims 
To assess usage patterns, user experience, and to uncover errors in functionality prior to larger interventional trials, we 
will conduct a longitudinal usability study MDC-m. The study will utilize questionnaires to quantify participants’ 
perceptions and responses to the MDC-m and semi-structured interviews to comprehensively assess users’ experience, 
barriers to use, and reasons for non-use or discontinued use. Combining qualitative and quantitative assessments of 
usability identifies more usability concerns than quantitative (e.g., questionnaires) assessments alone. Each participant 
will have access to the MDC-m for 1 month from the time of enrollment. This duration allows sufficient time for 
participants to become familiar with MDC-m and determine if and how to use MDC-m in the management of their 
diabetes. 
 
In the current study, we aim to: (1) assess the feasibility of the intervention and (2) the usability of the MDC-m over time 
among a diverse group of patients with type 2 diabetes mellitus (T2DM). 
 
3.0 
Inclusion/Exclusion Criteria 
Participants will be eligible for the study if they have T2DM, currently taking at least one medication for diabetes, are 
able to speak and read in English, are age 21 or over, have reliable access to a smartphone or tablet with internet access 
and have a MHAV account. 
 
We will exclude patients living in long term care facilities, patients with known cognitive deficits, patients with severe 
visual or hearing impairment, patients with unintelligible speech (e.g., dysarthria), and patients currently participating in 
other diabetes related studies. 
 
4.0 
Enrollment/Randomization 

Participants will be able to complete an electronic consent form and enroll online via Research Electronic Data Capture 
(REDCap™) version 5.0.8. 
 
5.0 
Study Procedures 
Setting. The Vanderbilt Adult Primary Care (VAPC) clinics are located throughout the greater Nashville, TN area. An EHR 
(Epic Systems Corp.) stores all clinical data and patients receive access to their clinical data via an integrated and highly-
adopted patient web portal, My Health at Vanderbilt (MHAV), that is accessible on desktops and via a native mobile app 
for iOS and Android mobile operating systems. 
 
Participants and Recruitment. Potential participants will be identified automatically using VUMC’s Subject Locator to 
query the EHR for patients with upcoming clinic appointments who meet the discrete inclusion and exclusion criteria. In 
addition, VUMC population health and quality improvement initiatives have created a report that identifies established 
patients with diabetes who receive care at the primary care clinics participating in the study. These patients are the 
target population for study (i.e., Vanderbilt primary care patients with diabetes) and as such these patients are 
potentially eligible to participate in the study. With permission of the clinic director, potentially eligible patients, will be 
mailed a recruitment letter describing the study and providing contact information for IRB approved study personnel. If 
after a receiving the letter, a patient is interested in the study, the patient may contact study personnel to learn more 
and consider participating in the study. The recruitment letter will include a QR code and URL link to the eligibility 
screener in Research Electronic Data Capture (REDCap). Participants whose REDCap screener indicates they are eligible 
can advance to the REDCap ICD. Participants will complete an electronic consent form and enroll online via REDCap. Our 
target enrollment is 65 participants. In accordance with best practices and to reflect a range of patient experience with 
diabetes and groups with distinct usability needs, we will aim to recruit a study sample with at least 25% representation 
of each of the following characteristics: (a) limited health literacy and (b) age 65 or over. Some participants may have 
one or more of these characteristics. 
 
Data Collection and Outcome Measures. Study participants will complete questionnaires electronically via email using 
REDCap at two time points:  enrollment (T0) and end of study (T1). Participants will complete an enrollment 
questionnaire (T0) including basic sociodemographic questions and validated measures of health literacy and eHealth 
literacy.   
 
The primary outcome measures (Table 1) will be: (a) ease of use and satisfaction as assessed by the System Usability 
Scale at T1, (b) system usage data of MDC-m at T1, (c) user experience as assessed by end of study questionnaire and 
semi-structured interviews at T1. In addition, we will assess pre/post change in the following secondary cognitive and 
behavioral outcomes (Table 1) to estimate effect size and standard deviations for power analyses necessary to plan 
subsequent randomized controlled trials. 
 
Table 1. Outcome Measures 
Primary Outcomes 
Measures 
Variable Type 
How Collected 
Time Point 
Usability  
System Usability Scale  
Continuous 
Questionnaire 
T1 
System usage data 
MDC-m visits 
Duration of MDC-m visits 
Continuous and 
categorical 
Self-report 
T1 
User experience 
Assess participants’ perspectives on content, layout, acceptance, and particular 
features and functionality 
Categorical, 
Qualitative 
Questionnaire, 
Interviews 
T1 
Secondary Cognitive/Behavioral Outcomes 
Diabetes knowledge 
Short Diabetes Knowledge Instrument (SDKI) 
Continuous 
Questionnaire 
T0 and T1 
Diet adherence 
Summary of Diabetes Self-Care Activities (SDSCA) – General Diet Subscale 
Summary of Diabetes Self-Care Activities (SDSCA) – Specific Diet Subscale 
Continuous 
Questionnaire 
T0 and T1 
Exercise adherence 
SDSCA – Exercise Subscale 
Continuous 
Questionnaire 
T0 and T1 
Medication adherence 
Adherence to Refills and Medications Scale 
Continuous 
Questionnaire 
T0 and T1 
Glucose self-monitoring 
adherence 
SDSCA – Self-monitoring of Blood Glucose Subscale 
Continuous 
Questionnaire 
T0 and T1 
Medication adherence 
Adherence to Refills and Medications Scale (ARMS) 
Continuous 
Questionnaire 
T0 and T1 
Diabetes self-efficacy 
Perceived Diabetes Self-Management Scale (PDSMS)  
Continuous 
Questionnaire 
T0 and T1 
Diabetes distress 
Problem Areas in Diabetes Scale (PAID)-5 
Continuous 
Questionnaire 
T0 and T1 
Knowledge of Diabetes 
Measures 
Unique study specific items to assess participants’ knowledge of measures of 
diabetes health status 
Categorical 
Questionnaire 
T0 and T1 
Diabetes Attitudes 
Unique study specific items to assess participants’ attitudes toward social and 
goal-based comparison information regarding their diabetes health status 
Categorical 
Questionnaire 
T0 and T1 

Diabetes Readiness for 
Change 
Four items assessment resources to assess stage of change based on the 
Transtheoretical Model (TTM) of behavior change. It includes questions on 
physical activity, medication management, glucose self-monitoring, and diet. 
Categorical 
Questionnaire 
T0 and T1 
 
Based on pilot testing, we estimate time to completion for questionnaires to be 20 minutes at T0 and 25 minutes at T1. 
Participants will be compensated $40 for completing the enrollment questionnaire, $35 for completing the study end 
questionnaire, and $5 for first 10 minutes of MDC-m use during which participants will be encouraged to familiarize 
themselves with the full functionality of the MDC-m.  
 
At the conclusion of their one month of MDC-m access, we will conduct one-on-one, semi-structured interviews with a 
subsample of at least 10 participants including at least two participants from each of the purposively sampled groups 
(i.e., participants with limited health literacy and age 65 or over). This methodology is most appropriate for in-depth 
assessment of patients’ perceptions and reactions regarding a proposed intervention. Interviews are preferred over 
focus groups for understanding usability because focus groups can amplify bias and individual opinions. Interviews will 
take place by phone or via Zoom within two weeks of participants concluding their one month of MDC-m access and will 
last approximately 30-40 minutes. Participant MDC-m access will be available during the interview to allow participants 
to recall and/or reference particular concerns or perceptions of usability. A trained interviewer will use a semi-
structured interview guide to facilitate the interview and elicit in-depth understanding of participants’ perceptions and 
experiences with specific MDC-m functionality as well as barriers to use, and reasons for non-use or discontinued use. 
Additional participants will be enrolled until saturation is reached. Saturation will be defined as no new usability 
concerns raised in the preceding two interviews and typically occurs between 10 and 30 interviews. Participants will be 
compensated an additional $40 for the interview.  
 
6.0 
Statistical Analysis Plan 
We will use descriptive statistics to characterize the study participants and survey data. We will use a one sample t-test 
to compare the mean SUS score at T1 to the threshold score of 71 indicative of “good” usability. To assess whether there 
was a significant improvement in the continuous secondary cognitive/behavioral outcomes from baseline to end of 
study (T0 to T1), we will perform two-sided paired t-tests on the pairwise differences. If any of the distributions of 
pairwise differences suggest asymmetry or nonnormality, the non-parametric Wilcoxon signed-rank test will be 
performed in lieu of paired t-tests. We will use the McNemar’s test for to compare paired proportions. Our primary 
outcome will be System Usability Scale (SUS) score at the end of the study period (T1).