Optimizing Message Framing for Healthy Habits for Patients With Type 2 Diabetes - Phase II (Pragmatic Trial)
Phase: NA
Conditions: Diabetes Mellitus, Type 2, Medication Adherence
Sponsor: Brigham and Women's Hospital

Detailed Protocol – Reinforcement Learning in Diabetes Mellitus Trial 
PI: Lauffenburger 
Version 5. April 1 2021 
1 
 
 
 
CLINICAL TRIAL PROTOCOL 
 
Reinforcement learning to personalize message framing for healthy habits (REINFORCE) 
 
National Clinical Trial (NCT) Identified Number: NCT04473326 
 
 
Principal Investigators: 
Julie C. Lauffenburger, PharmD, PhD 
Niteesh K. Choudhry, MD, PhD 
 
Center for Healthcare Delivery Sciences and Division of Pharmacoepidemiology and 
Pharmacoeconomics, Brigham and Women’s Hospital and Harvard Medical School 
 
Funded by the National Institute on Aging 
Grant Number: P30AG064199 
 
Version: No. 4, April 1, 2021 
 
 

Detailed Protocol – Reinforcement Learning in Diabetes Mellitus Trial 
PI: Lauffenburger 
Version 5. April 1 2021 
2 
 
Background and Significance 
Highly influential in motivating behavior change is the prominence of information or how 
information is framed. For example, providing information to patients in terms of risks rather 
than benefits makes them two times more willing to fill a prescribed medication. A potential 
way to optimize behavior is through problem solving, specifically trial and error. While manual 
trial and error is infeasible at scale, reinforcement learning offers a path forward. In specific, 
reinforcement learning is an advanced analytic method that discovers each individual’s pattern 
of responsiveness by observing their actions and then implements a personalized strategy to 
optimize individuals’ behaviors using trial and error. This method was developed based on 
principles of behavioral science to help models make better decisions based on human 
behavior. In contrast to other approaches that use previously-collected data, reinforcement 
learning algorithms learn in real-time and can personalize for specific patients. Given its 
theoretical foundation and ability to adapt in real-time, reinforcement learning holds wide 
promise for changing – and sustaining – health behaviors.  
The goal of the proposed study is to develop and test a novel reinforcement learning-enhanced 
text messaging program to support medication adherence in patients with type 2 diabetes. 
Type 2 diabetes is an optimal condition in which to test this program, as it is one of the most 
prevalent chronic conditions in the US adult population and requires most patients to be on 
daily or twice daily doses of medications.  
Specific Aims 
This pilot seeks to utilize two principle-driven interventions, message framing and 
reinforcement learning, to motivate adherence to evidence-based medications for adults with 
type 2 diabetes. To accomplish this aim, this project consists of two phases. This present 
protocol focuses only on the second phase, which is a small pragmatic trial. 
This phase has the following aims:  
(A) To determine whether a novel reinforcement-learning text messaging program improves 
adherence to medications and clinical outcomes. We will conduct a small pragmatic trial 
to determine the feasibility and efficacy of using reinforcement learning on patient 
adherence to diabetes medications. 
(B) To classify patient responsiveness to reinforcement learning. We will cluster intervention 
patients by their response to different types of messages (e.g., negative or positive 
framing) and evaluate the ability to predict these cluster phenotypes using information 
prior to randomization. This approach will optimize the design of future interventions by 
eliminating the need for electronic pill monitoring to determine clusters. 
 
Subject Selection  
Phase 2: Pragmatic Trial 
• We will enroll and randomize 60 eligible male and female patients (30 each to intervention 
and usual care arms) receiving their care at a BWH-affiliated diabetes practice who are 
between 18-84 years of age with type 2 diabetes mellitus (T2DM) and are prescribed 
between 1-3 daily oral medications for this disease. Patients must have a smartphone with a 

Detailed Protocol – Reinforcement Learning in Diabetes Mellitus Trial 
PI: Lauffenburger 
Version 5. April 1 2021 
3 
 
data plan or WiFi at home, an HbA1c level ≥7.5%, basic working knowledge of English, no 
active enrollment in another diabetes trial within Mass General Brigham, and ability to set 
up the platform and adhere to study procedures. To be included, patients must also either 
not currently use a pillbox or be willing to use electronic pill bottles (EDMs) for their 
diabetes medications for the duration of the study. This criterion is primarily because if 
patients were to concomitantly use pillboxes for these medications, it would undermine our 
ability to measure their adherence patterns.  
• Patients will be enrolled in an experimental study that involves a Pillsy electronic pill bottle. 
These electronic pill bottles have been used by other research studies at MGB. While the 
Pillsy bottle offers a set of “smart” functionalities, we will only use the bottle as a container 
for oral medication and as a measurement of adherence. Participants will be asked to use 
the Pillsy bottle for 6 months. Throughout the 6-month study period, the electronic pill 
bottles will record daily medication use.  
• Patients in the intervention arm will receive up to daily text messages based on their EDM-
measured adherence. Based on patient’s baseline characteristics and time-varying 
responses to the messages, a reinforcement learning algorithm will determine which type 
of messaging works best for each individual patient. A text messaging platform will deliver 
different text messages and adapt over time to the messages that individual patients most 
respond to. 
• Patients in the usual care arm may receive up to daily, untailored messages. 
• At the end of the study period, participants’ medication adherence and diabetes-related 
outcomes will be evaluated.  
At this time, we are requesting IRB approval for the trial portion of our study (Phase 2, 
described above).  
Subject Enrollment  
• Patients for the study will be recruited from the Brigham and Women’s Hospital outpatient 
diabetes clinics at 221 Longwood Ave, 45 Francis Street, and Brigham and Women’s 
Faulkner Hospital.  
• Patients will be recruited weekly through two mechanisms used in prior studies:  
o First, we will ask endocrinologists to refer potentially eligible patients since a similar 
referral system has been successful for several studies in the department. We will 
include all patients receiving care at a BWH-affiliated endocrinology practice. We 
will ask them to introduce the study to their patients with a fact sheet. (See 
Attachments). 
o Second, we will use both a weekly EPIC screen for patients with upcoming 
appointments and RPDR to identify patients with type 2 diabetes who are receiving 
care from a BWH endocrinologist, and who are taking 1-3 oral medications for their 
disease. We will reach out to the identified patients’ endocrinologist to ask them to 
opt out the patient if they do not consider them appropriate for the study. We will 
include a Fact Sheet for them to understand the study and its procedures. If within a 
week the provider does not opt out of the study for their patient, we will send a 
letter to the patient on their behalf inviting them to participate. We will then 

Detailed Protocol – Reinforcement Learning in Diabetes Mellitus Trial 
PI: Lauffenburger 
Version 5. April 1 2021 
4 
 
contact the patient over the phone a week later to inquire about participation. (See 
Attachments). 
• Patient Gateway will also be used intermittently throughout the recruitment period to 
identify potentially eligible patients who have an assigned endocrinologist at one of these 
BWH outpatient diabetes clinics. In specific, following the same process as outlined above, 
we will reach out to the identified patients’ endocrinologist to ask them to opt out the 
patient if they do not consider them appropriate for the study. We will include the Fact 
Sheet for them to understand the study and its procedures. If within a week the provider 
does not opt out of the study for their patient, we will send a letter via Patient Gateway to 
the patient on their behalf inviting them to participate. We will then contact the patient 
over the phone a week later to inquire about participation. (See Attachments).  
 
During the patient’s clinic visit (virtual or in-person), a research assistant will screen the patient 
for eligibility and obtain consent from the participant to participate in the study. All procedures 
for this study can be done completely virtually. 
Patients will be screened for eligibility using a screening form (See Attachments). This form will 
be completed directly in REDCap for all patients screened. Patients will consent to participate in 
the study by signing the Written Consent form (see Attachments). If this visit is completed 
virtually via one of MGB’s approved video platforms (e.g., Zoom), the study staff will be virtually 
present for the consent process and will obtain electronic informed consent from the 
participant using MGB-approved REDCap e-consent/paperless consent process. 
We will include the below text in secure emails or via Partners Patient Gateway, so that 
patients will have the ability to opt-out: 
“The Mass General Brigham (MGB) standard is to send email securely. This requires you to 
initially set up and activate an account with a password. You can then use the password to 
access secure emails sent to you from MGB.  If you prefer, we can send you “unencrypted” 
email that is not secure and could result in the unauthorized use or disclosure of your 
information. If you want to receive communications by unencrypted email despite these risks, 
MGB will not be held responsible. Your preference to receive unencrypted email will apply to: 
for research: [emails sent to you from research staff in this study. If you wish to 
communicate with other research staff at MGB regarding additional studies, your 
preference will have to be documented with each research group.] 
for clinicians: [emails sent from clinicians and staff at this practice or department.]    
for finance: [emails sent to you from Patient Billing Solutions and the Patient Service 
Center.”] 
Please confirm receipt of this email and indicate if your preference is to NOT receive secure 
emails from me at this time. If I do not hear back from you, then all future emails will be sent 
either via patient gateway or via secure messaging as is MGB policy.” 

Detailed Protocol – Reinforcement Learning in Diabetes Mellitus Trial 
PI: Lauffenburger 
Version 5. April 1 2021 
5 
 
Patients who prefer to correspond via email and to complete surveys electronically, will have 
the option to do so securely via the above channels for email and via REDCap, with all links sent 
securely as well.  
 
Study Procedures 
Participants have two study-related visits. These study visits can be completed virtually or in-
person during clinic visits (if/when COVID-19 precautions are lifted) and both sets of procedures 
are described in further detail below. 
Using a random number generator using stratification by baseline self-reported adherence and 
HbA1c control, the study research assistant will randomize 30 patients to the reinforcement 
learning arm and 30 patients to usual care. Other clinic staff and analysts will be blinded to arm 
assignment. 
Virtual Study Procedures 
1st Virtual Study Visit 
Subjects will provide written informed consent for all study procedures, including the use of 
text messages, and be enrolled at Brigham and Women’s Hospital. The study staff will conduct 
the consent discussion via video and obtain paperless consent or e-consent from the 
participant through the MGB approved REDCap e-consent platform. After this visit, they will be 
mailed an electronic pill bottle manufactured by Pillsy along with written instructions for setup 
(see Attachments), which participants will be asked to use for the duration of the study. Based 
on data from prior trials, poorly-controlled patients with type 2 diabetes take an average of two 
oral glucose-lowering medications, and we expect to distribute approximately two EDMs per 
patient but no more than three. 
At the baseline visit, we will also collect some basic information from subjects to 
administratively manage the study: name, address, email address, telephone number and the 
best times and days to contact, in addition to sociodemographic information (see demographics 
baseline questionnaire).  For the patient to complete the surveys in the REDCap database, we 
will provide them with a secure link to the online database so they can complete the surveys at 
their convenience. The survey invites will be sent via secure email or SMS using the Twilio 
platform, a third-party web service integrated into REDCap, which will provide the participant 
with the survey URL, depending on patient preference. We will review contact information with 
subjects at each study visit to make sure that we have the most accurate information. Contact 
information will be stored in the password-protected REDCap database.  
Additional information on demographics, diagnoses, medications, comorbidities, laboratory 
tests and disease activity will be abstracted from participants’ medical records. Data will be 
collected through chart review as well as RPDR and entered into our REDCap database.  
Before beginning follow-up, subjects in both arms will be contacted within 2 business days to 
review procedures, confirm the functionality of their bottles, and verify cellphone numbers for 
delivery of the intervention.  

Detailed Protocol – Reinforcement Learning in Diabetes Mellitus Trial 
PI: Lauffenburger 
Version 5. April 1 2021 
6 
 
Optional Check-in 
Participants will receive their Pillsy electronic pill bottle in the mail along with written 
instructions on how to setup and use of the Pillsy bottle. They will also be provided with contact 
information for assistance from the study staff to properly setup their electronic pill bottle. The 
Pillsy bottle works as a standard pill bottle in which participants will store their medications. To 
ensure that real-time data is transmitted to the Pillsy server, participants will also download the 
Pillsy mobile application on their phone that will be connected by Bluetooth to their electronic 
pill bottle. Participants will not otherwise be asked to engage with the mobile application. 
Before beginning follow-up, subjects in both arms will be contacted to review procedures, 
confirm the functionality of their bottles, and verify cellphone numbers for delivery of the 
intervention.  
Trial Period 
Participants will be asked to use the electronic pill bottle for 6 months. As participants receive 
medication refills, they will transfer their pills to the Pillsy electronic pill bottle. Throughout the 
study, participant medication use data will be captured by the electronic pill bottles. The Pillsy 
electronic pill bottles employ electronic date/time stamp technology that is triggered by 
opening the pill bottle. Dates and times of bottle opening (and therefore adherence) are 
transmitted by Bluetooth to the patient’s phone through the Pillsy mobile application.  
 
The mobile application data will be automatically transferred to a secure, password-protected 
Pillsy portal hosted on a secure server located outside of BWH. These uploaded data will be de-
identified, except for bottle opening dates and times. Participants will be assigned a study-
specific ID, and the electronic bottles will only be linked to this participant by the study-specific 
ID. These data will only be accessible to the MGB researchers with password-protected access 
to the Pillsy portal. The only potentially identifiable data in the Pillsy portal are bottle opening 
dates, times, and study-specific ID.  
 
Patients in the reinforcement learning arm will receive text messages based on their EDM-
measured adherence. Based on patient’s fixed characteristics and time-varying responses to the 
messages, the algorithm will determine which text message to send and adapt over time to the 
messages that individual patients most respond to. The learning algorithm will be re-run daily 
based on recent adherence patterns and will deliver text messages based on those patterns. 
Patients in the usual care arm may receive up to daily, untailored text messages.  
 
Patients in both arms may receive disconnectedness text messages, spaced three days apart, if 
their pill bottles become disconnected from their Pillsy mobile application for longer than a 
week. Study staff may call patients up to two times, spaced two weeks apart, if their pill bottles 
become disconnected from their Pillsy mobile application for longer than two weeks. If patients 
remain disconnected for the rest of the study period, study staff may call patients to reconnect 
their pill bottles at the end of the study. 
 

Detailed Protocol – Reinforcement Learning in Diabetes Mellitus Trial 
PI: Lauffenburger 
Version 5. April 1 2021 
7 
 
The reinforcement learning algorithm will be periodically updated by sending completely de-
identified modeling data to a Microsoft server. No patient identifiers will be included in any 
data on the Microsoft Azure server, only study-specific ID. This server will only be accessible by 
study staff. To update the reinforcement learning prediction model, study staff will manually 
upload and download data from the algorithm from the server. To do so, first, study staff will 
create a spreadsheet with 7-day adherence data from the Pillsy portal onto a MGB computer. 
This spreadsheet will include EDM-measured adherence as binary indicators (yes/no) for the 
past 7 days (i.e., dates/times will be removed). Then, study staff will upload this spreadsheet 
into the reinforcement learning algorithm model. The algorithm will then send fully de-
identified data back to the MGB server with its updated prediction information about which 
type of text to send for each intervention subject. Study staff will then log onto the secure text 
messaging vendor system already used by MGB to schedule the correct text for each participant 
for the next day.   
 
The texts will be sent by a secure messaging system vendor, SMS D365 Solution. The Health 
Innovation Platform team, part of MGB Personalized Medicine and the Digital Care 
Transformation/Enterprise Data and Digital Health Initiative, built this platform that enables 
sending and receiving of SMS messages to patients. The application is built on Microsoft’s 
Power Platform using Dynamics 365.  The HIPAA-compliant platform can store all incoming and 
outgoing messages in relational databases. If the patient replies to the text message that does 
not prompt a reply, they will receive an automated message telling them that the text message 
system is not monitored by a live individual and instructing them, in the case of the need for 
urgent medical attention, to call 911, go to their nearest emergency department or calling the 
24-hour nurse helpline for their BWH diabetes care practice. For those text messages that elicit 
a reply from the patient, automatic action will be programmed with messages as follows: 
Patient Response  
Automated action or text back to patient 
Stop 
<Stop text messages> 
Quit 
<Stop text messages> 
Thank you  
<No response> 
Thanks 
<No response> 
Ok  
<No response> 
<Any other response> 
The message you just replied to was automated, 
so if you need medical help, please contact your 
24-hour nurse helpline, call 911, or go to your 
nearest emergency department.  
 
2nd Virtual Study Visit 
 

Detailed Protocol – Reinforcement Learning in Diabetes Mellitus Trial 
PI: Lauffenburger 
Version 5. April 1 2021 
8 
 
At the 2nd study visit, participants will be offered a $50 Amazon gift card that will be mailed to 
them upon study completion, and they will be able to keep their Pillsy EDM device(s) for 
personal use. Participants will also be asked to complete a questionnaire about their self-
reported medication use and adherence and perspectives on the intervention (if applicable) or 
EDM monitors. In order for the patient to complete the surveys in the REDCap database, we will 
provide them with a secure link to the online database via SMS sent through the Twilio 
platform or via secure e-mail at the patient’s discretion. 
 
In-Person Study Procedures (if/when COVID-19 precautions are lifted – this entire study can 
be done virtually): 
 
1st Study Visit 
At the first study visit, subjects will provide written informed consent for all study procedures, 
including the use of text messages, and be enrolled at Brigham and Women’s Hospital. In this 
visit, they will be given an electronic pill bottle manufactured by Pillsy, which participants will 
be asked to use for the duration of the study. Based on data from prior trials, poorly-controlled 
patients with type 2 diabetes take an average of two oral glucose-lowering medications, and we 
expect to distribute approximately two EDMs per patient but no more than three.  
 
Participants will then be instructed by a trained study staff member on how to use the Pillsy 
bottle. The Pillsy bottle works as a standard pill bottle in which participants will store their 
medications. To ensure that real-time data is transmitted to the Pillsy server, participants will 
also download the Pillsy mobile application on their phone that will be connected by Bluetooth 
to their electronic pill bottle. Participants will not otherwise be asked to engage with the mobile 
application. 
 
At the baseline visit, we will also collect some basic information from subjects to 
administratively manage the study: name, address, email address, telephone number and the 
best times and days to contact, in addition to sociodemographic information (see demographics 
baseline questionnaire). These questionnaires will be completed on paper copies then 
uploaded to the REDCap databases. If a patient wants to complete the surveys at a later time, 
we will provide them with a secure link to the online database so they can complete the surveys 
at their convenience. If so, the survey invites will be sent through secure email via Twilio, a 
third-party web service integrated into REDCap which will provide the participant with the 
survey URL through SMS. We will review contact information with subjects at each follow up 
visit to make sure that we have the most accurate information. Contact information will be 
stored in the password-protected REDCap database.  
 
Additional information on demographics, diagnoses, medications, comorbidities, laboratory 
tests and disease activity will be abstracted from participants’ medical records. Data will be 
collected through chart review as well as RPDR and entered into our REDCap database.  

Detailed Protocol – Reinforcement Learning in Diabetes Mellitus Trial 
PI: Lauffenburger 
Version 5. April 1 2021 
9 
 
Before beginning follow-up, subjects in both arms will be contacted within 2 business days to 
review procedures, confirm the functionality of their bottles, and verify cellphone numbers for 
delivery of the intervention.  
 
Trial Period 
The trial period protocol for the in-person study participants is exactly the same as the protocol 
for the virtual study participants described above.  
2nd Study Visit 
At the 2nd study visit, participants will be offered a $50 Amazon gift card, and they will be able 
to keep their Pillsy EDM device(s) for personal use. Participants will also be asked to complete a 
questionnaire about their self-reported medication use and adherence and perspectives on the 
intervention (if applicable) or EDM monitors.  
Study Schema 
 
 
Outcomes 
The trial’s primary outcome is medication adherence assessed in the 6 months after 
randomization. Medication adherence will be measured by averaging daily adherence (as 
described above) for each medication across each patient beginning the day after 
randomization until 180 days after randomization. Secondary outcomes include change in 
glycemic control as assessed using HbA1c, and self-reported adherence at the end of follow-up. 
HbA1c values will be collected from routine measurements recorded in the EHR system; we will 
use the value closest to each patient’s 6-month end of follow-up, up to 1 month after 

Detailed Protocol – Reinforcement Learning in Diabetes Mellitus Trial 
PI: Lauffenburger 
Version 5. April 1 2021 
10 
 
randomization. In routine care, HbA1cs are measured approximately every 3-6 months, so we 
expect only modest missingness, as we have observed also in prior work. Self-reported 
adherence will be assessed as the proportion of patients who are adherent according to a 
validated 3-item self-report measure and prior literature from the follow-up questionnaire.31 
We will also descriptively measure implementation outcomes informed by the RE-AIM 
framework53, including representativeness of patients in the study, text messaging opt-out 
rates, any feedback from patients, and rates of pill bottle disconnectedness, which will inform 
considerations for how to scale the intervention to other settings.  
 
Statistical Analysis 
 
We will report means and frequencies of pre-randomization variables separately by 
intervention and control arm, comparing these values using absolute standardized differences. 
The outcomes will be evaluated using intention-to-treat principles among all randomized 
participants. In the primary analysis, we will evaluate adherence and glycemic control using 
generalized estimating equations with an identity link function and normally distributed errors. 
We will also adjust for the block randomized design. We do not expect any missing data for the 
primary outcome, but may have up to 25% missingness for the glycemic control outcome. If 
>10% of participants have missing HbA1c data, we will repeat our analyses using multiple 
imputation. A similar approach will be taken for self-reported adherence, except using a log link 
function and Poisson distributed errors to generate relative risks of the proportion of adherent 
patients in the intervention versus control arms. In secondary analyses, we will control for any 
differences in baseline variables between the arms despite randomization.  
 
As a sensitivity analysis, we will censor patients in the analysis when they have stopped using 
the electronic pill bottle for >30 days. We will also evaluate the change in HbA1c from baseline 
until the end of follow-up and differences in self-reported adherence separately for the 3 items 
that make up the self-reported scale we are using. For glycemic control (HbA1c) and self-
reported adherence, we will also conduct complete case analyses. Similarly, subgroup analyses 
will include stratification by age, sex, race/ethnicity, baseline HbA1c, baseline self-reported 
adherence, and number of study medications.  
 
Our study should be sufficiently powered to detect clinically meaningful differences in the 
primary outcome. With 60 subjects, we estimated that we would have the power to detect a 
10% difference in average adherence over the 6-month follow-up period between the 2 arms, 
assuming a standard deviation (SD)=12.5%, power=0.8, and alpha=0.05. With this sample size, 
we would also be able to detect an HbA1c difference of 1.0% between arms (assuming SD=1.3) 
and 50% relative difference in self-reported adherence.  
 
Risk and Discomforts 
We believe that the risks to participation for subjects are no more than minimal. We do not 
anticipate the occurrence of any incremental adverse events as a result of patients receiving 

Detailed Protocol – Reinforcement Learning in Diabetes Mellitus Trial 
PI: Lauffenburger 
Version 5. April 1 2021 
11 
 
adherence support for medications that they were already prescribed and for which providers 
have ultimate oversight. The study team will not be providing any direct care to patients, and all 
treatment decisions will ultimately be made by the patients’ medical teams at MGB. Any 
adverse events will be handled in the course of regular clinical care. We will also obtain their 
written informed consent for participation for the trial and for the use of their data in 
secondary analyses of patient clusters of responsiveness. Finally, we will also safeguard any 
identifiable information in accordance with IRB practices, limit access to any information in 
accordance with IRB practices, limit access to the information to study team members actively 
involved in the research who have all undergone human subjects research training and destroy 
any information upon completion of the research. 
There is a risk that participant medication use data or phone number could be made public if 
Pillsy’s MGB’s SMS D365 Solution, or the virtual video visit platform’s servers are compromised. 
However, since the data are not linked to another identifier, the risks of identification remain 
low. Such an event could lead to embarrassment or other forms of discomfort related to public 
exposure of health data.  
If patients currently use a pillbox for their medications, there is a hypothetical and small risk 
that using electronic pill bottles during this study could change their daily medication-taking 
routines, but this has not been verified in any peer-reviewed study.   
Potential Benefits 
This study is designed to improve medication adherence for patients with diabetes. The patient 
subjects may benefit from improved medication adherence through improved clinical 
outcomes. The patient subjects will also receive compensation for their participation. 
The patient subjects and society may also benefit in the future from accumulated knowledge 
that originates from this research. The potential societal benefits outweigh the minimal risk, 
especially in light of multiple measures in place to protect confidentiality. We will also produce 
several reinforcement learning tool deliverables for this work for the public, researchers, and 
policymakers, which will be shared as generalized knowledge. These deliverables include the 
results from the study, text messages, and a potential reinforcement learning algorithm that 
can be used in a clinical setting to promote medication adherence. 
Monitoring and Quality Assurance  
The research subjects in this study will be subject to no more than minimal risk and we do not 
anticipate the occurrence of any adverse events. Safety will be assessed during enrollment and 
the study period. The data obtained from the electronic pill bottles will be monitored weekly by 
research staff to ensure safety of the subjects. Use of an electronic pill bottle is non-invasive 
and carries little potential for harm, so it is unlikely the study will need to be stopped. 
Relevant clinical data on patients will be retrieved from the electronic medical records at BWH. 
The data extracts obtained from the electronic medical record are continuously used by clinical 
operations staff for quality assessment and improvement, and undergo routine, rigorous peer-
review by experienced data analysts to ensure accuracy and completeness. The principal 

Detailed Protocol – Reinforcement Learning in Diabetes Mellitus Trial 
PI: Lauffenburger 
Version 5. April 1 2021 
12 
 
investigator will work with the research project staff to ensure the accuracy of these data 
throughout the study period. 
General oversight of this study is by the principal investigator. There will be regular meetings 
and contact with research staff to ensure appropriate oversight of all aspects of the study. De-
identified study data will always be accessible for the principal investigator and co-investigators 
to review, if applicable. The principal investigator and co-investigators will also ensure that all 
protocol deviations are reported to the NIH and the IRB according to the applicable regulatory 
requirements. Compliance of regulatory documents and study data accuracy and completeness 
will be maintained through an internal study team quality assurance process. 
Data quality will be assessed by study staff continually throughout the course of the study. The 
electronic pill bottles will automatically upload data, in real time, to Pillsy’s servers. This will 
allow study staff to receive real-time updates on data quality during the study.  
Risks incurred by partnering with Pillsy and using MGB’s SMS Dynamics 365 platform will be 
evaluated by the Research Information Security Office. We will ensure compliance with all MGB 
security requirements through the signing of Master Service Agreements that ensure patient 
privacy is protected. 
We will be in routine contact with the medical leadership at each diabetes clinic to obtain any 
feedback from clinicians regarding the study (Dr. McDonnell). Our plan for data and safety 
monitoring also includes oversight by the project principal investigator (Dr. Lauffenburger) 
throughout the study period.  
Per previous correspondence with RISO for other studies, we will do the following to protect 
the security of our data:  
• 
MGB REDCap will be used  
• 
Should any emails be sent to participants, the blind copy function (BCC) must be used 
when sending to more than one patient/research subject, in order to protect the 
confidentiality of recipients 
• 
Access to web portal by study staff requires a username and password: 
o Passwords must be a minimum of 8 characters 
o Passwords must be alphanumeric, containing at least one of each. 
o Cannot reuse 4 previous passwords. 
o Passwords must be changed immediately if either the password or the system is 
or may be compromised 
o Passwords must not be displayed in clear text when they are being input into an 
application. 
o Passwords must be changed every 90 days 
o Users must be uniquely identified; no shared or group accounts without security 
authorization 
o Passwords may not be shared 
o User access must be terminated immediately upon termination or change of 
responsibilities 

Detailed Protocol – Reinforcement Learning in Diabetes Mellitus Trial 
PI: Lauffenburger 
Version 5. April 1 2021 
13 
 
o Passwords should not be the same used as regular MGB credentials for 
workstations 
• 
Only MGB Workstations / Laptops in use for the research  
o Password requirements from above 
o Encryption at rest is in place 
o Up-to-date malware protection including antivirus, spyware detection and 
removal tools 
o Personal firewall is enabled 
o Manufacturer supported operating system with current updates, if available for 
the device 
o Crowdstrike End Point protection installed