Implementation Strategies for Monitoring Adherence in Real Time
Phase: NA
Conditions: Lung Cancer, Medication Adherence, Symptoms and Signs
Sponsor: Abramson Cancer Center at Penn Medicine

 
 
 
Implementation Strategies for Monitoring Adherence in Real Time (iSMART) 
Study Protocol for Single Center Behavioral Trial (Phase II) 
 
 
 
Principal Investigators 
 
Katharine A. Rendle, PhD, MSW, MPH 
 
 
 
 
 
Assistant Professor, Family Medicine & Community Health 
 
 
 
 
 
Assistant Professor, Epidemiology 
 
 
 
 
 
Perelman School of Medicine, University of Pennsylvania 
 
 
 
 
 
Katharine.rendle@pennmedicine.upenn.edu 
 
 
 
 
 
(215) 662-9147 
 
 
 
 
 
 
Samuel Takvorian, MD, MSHP 
 
 
 
 
 
Instructor, Division of Hematology and Oncology  
 
 
 
 
 
Perelman School of Medicine, University of Pennslyvania 
 
 
 
 
 
Samuel.takvorian@pennmedicine.upenn.edu 
 
(267) 438-8269 
 
Funding Agency: 
 
 
Lung Cancer Research Foundation (with support from Pfizer) 
 
IRB Protocol Number:  
 
834713 
 
Clinical Trials Number: 
 
NCT04347161 
 
UPCC Protocol Number: 
 
UPCC 20520 
 
Version Date: 
 
 
February 17, 2021 
 
 
 
 
 
 
 
 
 
 
 

 
2
Table of Contents 
Page 
1. PROTOCOL SUMMARY 
3 
2. STUDY ABSTRACT 
4 
3. BACKGROUND AND RATIONALE 
4 
4. OBJECTIVES 
5 
4.1. Specific Aims 
5 
4.2. Primary Outcome 
6 
4.3. Secondary Outcome 
6 
4.4. Exploratory Outcome 
6 
5. STUDY DESIGN 
6 
5.1. Overview 
6 
5.2. Study Setting 
7 
5.3. Study Duration 
7 
5.4. Target Population 
7 
5.5. Sample Size and Accrual 
7 
5.6. Key Inclusion and Exclusion Criteria 
8 
5.7. Participant Remuneration 
9 
6. RANDOMIZATION 
9 
6.1. Groups 
9 
6.2. Assignment 
10 
6.3. Blinding 
10 
7. STUDY PROCEDURES 
10 
7.1. Eligibility Screening and Recruitment 
10 
7.2. Consent and Enrollment 
11 
8. DATA MANAGMENT 
11 
8.1. Data Collection 
11 
8.2. Assessment Procedures 
12 
8.3. Measures 
13 
9. ANALYSIS PLAN 
13 
9.1. Primary Analyses 
13 
9.2. Exploratory Analyses 
14 
9.3. Qualitative Analyses 
14 
10. DATA MONITORING AND SAFETY 
14 
10.1. Investigative Team 
14 
10.2. Regulatory Approvals 
15 
10.3. Data Confidentiality  
15 
10.4. Subject Confidentiality  
15 
10.5. Subject Privacy 
16 
10.6. Data Disclosure 
16 
10.7. Risk/Benefit Assessment 
17 
10.8. Data Safety and Monitoring 
18 
11. REFERENCES 
20 
12. APPENDICES 
22 

 
3
1. PROTOCOL SUMMARY 
 
Title 
Implementation Strategies for Monitoring Adherence in Real Time (iSMART) 
Short Title 
iSMART Trial 
Co-Principal 
Investigators 
Katharine Rendle, PhD, MSW, MPH 
Samuel Takvorian, MD, MSHP 
Co- 
Investigators 
Justin Bekelman, MD 
Lawrence Shulman, MD 
E. Paul Wileyto, PhD 
Steven Kimmel, MD, MSCE 
Design 
Prospective, randomized controlled trial 
Objectives 
 
Aim 1: To test the effects of a mobile phone-based chatbot (“Penny”) to 
improve adherence to oral targeted therapies in approximately 170 patients 
with non-small cell lung cancer.  
 
Aim 2: To use mixed methods approaches with 30 clinicians and 30 patients 
to explore factors shaping the acceptability, effectiveness, and future 
implementation of Penny into routine cancer care. 
Trial Duration 
Estimated 12 months (December 2020-November 2021) 
Study Sites 
The University of Pennsylvania Health System 
Sample Size 
Estimated 170 patients and 30 clinicians 
Patient Trial 
Eligibility 
 
Adult patient (age > 18 years) with advanced NSCLC at recruiting site at 
UPHS who is receiving one of the following oral therapies: afatinib, erlotinib, 
dacomitinib, gefitinib, osimertinib, alectinib, brigatinib, crizotinib, or lorlatinib 
 
Patient possession of a mobile device that can send/receive SMS texts 
 
Ability to respond to questions and engage with “Penny” in English  
 
Ability to provide informed consent to participate in the study 
 
Blanket approval from the patient’s medical oncologist to be approached  
Interventions 
 
Treatment Arm: Patients in the treatment arm will receive access to the 
mobile phone-based intervention (“Penny”). Penny is a conversational agent 
(“chatbot”) that engages patients in real time via text messaging, allowing for 
bidirectional communication between clinicians and patients, longitudinal 
symptom monitoring with self-management support, and motivational cues to 
promote adherence. The chatbot will remind patients how and when to take 
their oral targeted therapies (based on prescribed regimen) and provide self-
management and monitoring of mild side effects guided by clinical pathways. 
 
Usual Care: Patients in the usual care arm will receive standard of care for 
symptom monitoring and management of side effects, which includes patient 
education at the time of prescribing and direction to call providers or triage 
nurses if anything arises between appointments. 
Outcomes 
 
Primary Outcome: Adherence (measured using MEMS caps) defined by the 
overall proportion of adherent days and classified dichotomously as adherent if 
total proportion of days is >95% 

 
4
 
Secondary Outcomes: Persistence defined as the number of consecutive 
days on the regimen and conversely gaps in overall treatment  
 
Exploratory Outcomes: Patient-reported outcomes including health-related 
quality of life (HRQOL), symptom burden, patient satisfaction, financial toxicity, 
social and behavioral determinants of health (SBDH), medication self-efficacy, 
and health literacy 
Primary 
Analysis 
Intent-to-treat (ITT) analyses using repeated measures logistic regression to 
compare the overall effect of the intervention on patient adherence 
Secondary 
Analyses 
Baseline survey and clinical data will be used to explore potential mediators and 
moderators of treatment effects. Standard mediation analysis will be used to 
explore separately whether symptom management or self-efficacy mediates 
treatment effects. Multivariable regression analysis with intervention, moderator, 
and interaction terms will be used to assess effect modification by patient 
characteristics based on significance of the interaction term.  
Study Oversight Trial oversight will be conducted by the University of Pennsylvania Institutional 
Review Board. Safety will be monitored on an ongoing basis by the co-PIs, study 
team, and institutional experts at Penn Medicine not directly involved in the study. 
 
2. STUDY ABSTRACT 
 
The objective of this project is to identify effective strategies to help patients with lung cancer manage 
side effects and achieve optimal adherence to oral targeted therapies. To achieve this objective, we will 
evaluate the effect of a mobile phone-based intervention (“Penny”), compared to usual care, on 
adherence to oral targeted therapies using a two-arm randomized controlled trial (Aim 1), and explore 
how multilevel factors impact the acceptability and effectiveness of this strategy by collecting qualitative 
and quantitative data from clinicians and patients (Aim 2). Penny is a conversational agent (“chatbot”) 
that engages patients in real time via text messaging, allowing for bidirectional communication between 
clinicians and patients, longitudinal symptom monitoring with self-management support, and 
motivational cues to promote adherence. The chatbot will walk patients through how and when to take 
their oral targeted therapies (based on prescribed regimen) as well as provide self-management and 
monitoring of mild-to-moderate side effects guided by clinical pathways. Severe or greater side effects 
will be escalated to the clinical research team in real-time and then escalated to wider clinical care team 
using existing clinical pathways. Guided by established theory of behavior change (Information-
Motivation-Behavioral Skills model), we hypothesize Penny will improve adherence to oral targeted 
therapies by improving patient-level symptom management and other determinants of behavior change. 
Primary outcomes (adherence and persistence) will be measured using microelectronic monitoring 
system (MEMS) caps. Secondary outcomes will be assessed using longitudinal surveys and medical 
record data. Semi-structured interview data will also be collected from a subsample of 60 patients and 
clinicians following the intervention period. By combining innovations from behavioral economics and 
machine learning with a highly accessible text-messaging platform, our project has the potential not 
only to identify scalable, patient-targeted strategies for improving symptom management and 
adherence to targeted therapies, but also to transform the way cancer care is delivered. 
 
3. BACKGROUND AND RATIONALE 
 
The availability and use of oral anticancer agents have grown substantially over the past decade, 
especially for patients with lung cancer. There are currently more than 50 approved oral targeted 
therapies, fourteen of which are approved for use in patients with advanced non-small cell lung cancer. 

 
5
Most patients with cancer prefer oral to infusion-based therapies, provided equivalent efficacy and 
toxicity,1 as oral agents reduce the burden of care with convenient dosing outside of the clinic.2 
However, home administration shifts the responsibility of symptom monitoring and adherence to 
patients and their caregivers. Because treatment efficacy depends on proper use, identifying effective 
strategies to ensure adherence to oral therapies is vital.  
Even among cancer patients, non-adherence remains a significant problem.3 Studies have 
shown widely variable rates of adherence to oral therapies, from 16%-105% depending on malignancy, 
medication, assessment, and study follow-up.4,5 In the context of lung cancer, suboptimal adherence 
affects a considerable number of patients: up to 14.3 percent of patients fall below a 80% adherence 
rate and 21% do not take their medication as prescribed, according to recent studies.1,6,7  Suboptimal 
adherence is associated with worse outcomes,8 including cancer progression,9 inferior survival,10 higher 
toxicities,11 and higher resource utilization and spending.12 Reasons for non-adherence include patient, 
clinician, disease, treatment, and healthcare system factors.13 Patient-level factors including inadequate 
understanding of complex dosing instructions,14 limited ability to manage side effects,4 financial costs,15 
and poor patient-clinician communication have been particularly hard to address. There is limited 
research on patient-level strategies to improve medication adherence in patients with advanced lung 
cancer. Small studies have evaluated the use of patient- and clinician-targeted strategies including 
enhanced patient education and treatment monitoring.16,17 However, these were limited by small 
sample size and non-randomized design.  
Patient-directed intervention to improve symptom management and adherence should be 
theoretically based and scalable. Behavioral economic theory holds that to affect a frequent behavior 
such as taking daily medication, the implementation strategy must engage at nearly the same or 
greater frequency.18 For complex regimens or those regimens with significant side effects, this degree 
of engagement would have been impossible before the era of connected devices. However, modern 
technology allows for ‘automated hovering’—patient surveillance outside of clinical settings—which can 
deliver implementation strategies in real time.19 A small but growing literature supports digital 
interventions across a variety of chronic illnesses,20,21 including cancer.22 Today, technology-based 
approaches to aid adherence are feasible and safe, but their efficacy has not been rigorously studied.23 
Across the digital health landscape, conversational agents—or technologies that draw upon augmented 
or artificial intelligence and natural language processing to mimic human conversation and are 
promising to support behavior change.  
 
Low-cost, technology-based approaches such as text messaging have shown promise as 
sustainable and scalable patient-directed strategies to improve adherence, particularly in low-resource 
settings.24 Yet for oral targeted therapies, these strategies have not been extensively studied.25 For 
patient populations outside of oncology, text-based and other digital strategies guided by insights from 
behavioral economics have been effective in improving symptom monitoring and adherence.26 In this 
project, we will test a novel, technology-based intervention (“Penny”), engineered using principles from 
behavioral economics to improve symptom management and medication adherence among patients 
with advanced lung cancer taking oral targeted therapies. Guided by an established theory of behavior 
change, we hypothesize Penny will improve adherence to oral therapies by improving patient-level 
symptom management and other determinants of behavior change. 
 
4. OBJECTIVES 
 
The overall goal of our project is to identify effective strategies for improving lung cancer outcomes by 
helping patients to better manage symptoms and adhere to oral therapies.  
 
4.1 Specific Aims 
The specific aims of this single center pilot randomized project are to:  
 

 
6
Aim 1. To evaluate the effect of a mobile phone-based intervention (“Penny”), compared to usual care, 
on patient adherence to oral targeted therapies using a two-arm randomized-controlled trial.  Patients 
with lung cancer receiving oral targeted therapies at UPHS will be randomized (1:1) to Penny or usual 
care.  
 
Aim 2. To use mixed methods approaches with clinicians and patients to explore factors shaping the 
acceptability, effectiveness, and future implementation of Penny into routine cancer care. Following 
active intervention, semi-structured interview and questionnaire data will be collected from a stratified 
subsample of 30 patients enrolled in the trial and 30 clinicians and staff who provide oncology care at 
Penn Medicine. 
 
In addition primary aims, we will also explore: 1) the impact of the intervention on patient-reported 
outcomes including patient satisfaction, medication self-efficacy, and other outcomes; 2) if symptom 
management or self-efficacy mediates the relationship between arm assignment and adherence; and 3) 
any potential effect modification across different patient subgroups based on patient demographics, 
health literacy, social determinants of health, clinical characteristics, and other factors. 
 
4.2 Primary Outcome 
The primary outcome is adherence to prescribed oral targeted therapy across the 12-week trial period. 
Adherence is defined by the overall proportion of adherent days and classified dichotomously as 
adherent if total proportion of days is >95%. Longitudinal adherence outcomes for all participants will be 
captured using microelectronic monitoring system (MEMS) caps, surveys, and electronic medical 
record (EMR) data.  
 
4.3 Secondary Outcome  
The secondary outcome is persistence defined as the number of consecutive days on the prescribed 
medication regimen and conversely gaps in overall treatment. 
 
4.4 Exploratory Outcomes 
Exploratory patient-reported outcomes including health-related quality of life (HRQOL), symptom 
burden, patient satisfaction, financial toxicity, social and behavioral determinants of health (SBDH), 
medication self-efficacy, health literacy, and healthcare utilization will be evaluated using validated 
survey measures. 
 
5. STUDY DESIGN 
 
5.1 Overview 
We will evaluate the effect of a bidirectional conversational agent (“Penny”), compared to usual care, on 
patient adherence to oral targeted therapies using a two-arm randomized-controlled trial (Aim 1), and 
explore how multilevel factors impact the effectiveness and implementation of the strategy by collecting 
qualitative and quantitative data from clinicians and patients (Aim 2). The intervention will leverage a 
third-party company, Memora Health, who have developed a HIPAA-compliant, SMS text-based 
platform and worked previously with Penn Medicine. Consented patients with advanced lung cancer 
receiving oral targeted therapies within UPHS will be randomized (1:1) to Penny or usual care. 
Longitudinal adherence outcomes for all participants will be captured using microelectronic monitoring 
system (MEMS) caps, surveys, and EMR data. Following active intervention, semi-structured interview 
data will be collected from a subsample of 30 patients and 30 clinicians to further assess mechanisms 
shaping adherence, and the acceptability, effectiveness, and future implementation of Penny into 
routine cancer care. 
 
 

 
7
5.2 Study Setting 
The study will be conducted at two hospitals affiliated with the University of Pennsylvania Health 
System: Perelman Center for Advanced Medicine/Hospital of University of Pennsylvania (PCAM) and 
Penn Presbyterian Medical Center (PPMC).  
 
5.3 Study Duration 
Aim 1: Study enrollment for each participant will last approximately 15-weeks including consent and 
post-assessment. Observation will begin no earlier than one week after therapy initiation (to allow for a 
brief prescription run-in period for new starts) and end either 12-weeks later or at therapy 
discontinuation, whichever is shorter. Post-assessment will occur within 2 weeks after each patient’s 
active observation period. We anticipate the active trial will last approximately nine months, with 
anticipated enrollment beginning in November-December 2020 and ending in October 2021. Data 
ascertainment and analyses will continue through December 2021 or later as needed. 
 
Aim 2: Study enrollment for each interview participant will last approximately 60-90 minutes including 
consent, semi-structured interview, and completion of brief written questionnaire. We anticipate that 
interviews will begin in December 2020 and continue through November 2021. Analyses will continue 
through December 2021 or later as needed. 
 
5.4 Target Population 
Aim 1. The study aims to enroll 170 adult patients (18 years or older) with advanced NSCLC who have 
initiated oral targeted therapies at UPHS. These include afatinib, erlotinib, dacomitinib, gefitinib and 
osimertinib for patients harboring sensitizing EGFR mutations; and alectinib, brigatinib,, crizotinib, and 
lorlatinib for patients with anaplastic lymphoma kinase (ALK) or ROS1 rearrangements. To be eligible, 
participants must be receiving ongoing cancer care at a participating site; have access to a mobile 
phone with SMS text messaging capabilities; be able to read and respond to questions in English; and 
must have started taking oral targeted therapies no earlier than January 1, 2020 and no later than the 
end of study recruitment (estimated October 2021). 
 
Aim 2. The study aims to interview 30-45 patients and 15-30 clinicians. Patients will be eligible if they 
are enrolled in the intervention trial (Aim 1). Clinicians will be eligible if they actively provide care for 
cancer patients on targeted oral therapies at one of the study sites (including but not limited to 
oncologists, advanced practice providers, triage nurses, and pharmacists).  
 
5.5 Sample Size and Accrual 
Aim 1. The study sample will be drawn from adults seeking cancer care at the University of 
Pennsylvania’s Abramson Cancer Center (ACC).  Across recruitment sites, approximately 215 patients 
with lung cancer had orders for one of the included oral targeted therapies from January-September 
2020. For this study, we will include both patients who have new prescriptions (after study launch) and 
patients previously taking these medications in 2020 (and who continue to take these medications at 
the time of eligibility assessment). Based on preliminary studies with Penny, we conservatively estimate 
that >40% of patients approached will be eligible and consent to participate in this study. Of the 215 
patients who started taking these medications in 2020 and estimated number of new starts during the 
study enrollment period (estimated 25 new starts per month), we anticipate that we will reach our target 
sample size of 170 in approximately 9-12 months (enrolling an estimated 85 patients who are currently 
taking these medications and 10 patients monthly who are new medication starts across the enrollment 
period). Allowing for a 15% loss to follow up, 72 participants will remain in each arm (n=144). Our full 
sample will provide 80% power to detect minimum effect of 20.9% or greater between the intervention 
and control group (assuming previously observed 65.5% adherence rate in controls and using fisher’s 
exact test7), an effect size that is clinically significant and appropriate for this pilot stage of research. 
 

 
8
Aim 2. Patients will be eligible if they are enrolled in the intervention trial, and purposively sampled by 
race, age, drug type, study arm, and adherence outcome to participate in the interview (in batches of 
15) to increase variation of perspectives.27 Recruitment will continue until we reach our target sample of 
30-45 patients. This number is based on the estimated sample needed to reach data saturation on 
exploratory themes; however, interviews will continue until data saturation is achieved. Clinicians will be 
eligible if they actively provide care for cancer patients on targeted oral therapies at one of the study 
sites (including but not limited to oncologists, advanced practice providers, triage nurses, and 
pharmacists). As with the patient sample, our target of 15-30 clinicians is based on the estimated 
sample needed to reach qualitative data saturation on exploratory themes, but interviews will continue 
until data saturation is achieved. 
 
5.6 Key Inclusion and Exclusion Criteria 
 
Aim 1: Trial Participants 
Inclusion Criteria 
 
Adult patient (age > 18 years) with advanced NSCLC at UPHS who is receiving one of the following 
nine oral therapies: afatinib, erlotinib, dacomitinib, gefitinib, osimertinib, alectinib, brigatinib, 
crizotinib, or lorlatinib. 
 
Patient possession of a mobile device that can send/receive SMS texts 
 
Ability to respond to questions and engage with “Penny” in English  
 
Ability to provide informed consent to participate in the study 
 
Blanket approval from the patient’s medical oncology to be approached 
 
Exclusion Criteria 
 
Inability to respond to questions and engage with “Penny” in English 
 
Inability or unwillingness to provide informed consent to participate in the study 
 
Inability to engage with SMS text-messaging platform 
 
Concurrent enrollment in a therapeutic clinical trial 
 
Lack of blanket approval from the patient’s oncologist  
 
Aim 2: Interview Participants: Patients 
Inclusion Criteria 
 
Adult patient who consented and enrolled in the intervention trial (Aim 1)  
 
Ability to respond to interview questions in English  
 
Ability to provide informed consent to participate in the study 
 
Exclusion Criteria 
 
Did not meet eligibility criteria to enroll in the intervention trial (Aim 1)  
 
Inability to respond to interview questions in English  
 
Inability to provide informed consent to participate in the study 
 
Aim 2: Interview Participants: Clinicians 
Inclusion Criteria 
 
Clinician employed by UPHS who routinely provides care for cancer patients on targeted oral 
therapies at one of the recruiting study sites including but not limited to oncologists, advanced 
practice providers, triage nurses, and pharmacists. 
 
Ability to respond to interview questions in English  
 
Ability to provide informed consent to participate in the study 
 
 

 
9
Exclusion Criteria 
 
Does not provide routine care for cancer patients on oral targeted therapies at UPHS  
 
Inability to respond to interview questions in English  
 
Inability to provide informed consent to participate in the study 
 
5.7 Participant Remuneration 
In recognition of time needed to complete data collection surveys and the importance of retaining the 
MEMS caps, incremental remuneration will be provided to patients ($25 at each time point) for total of 
$75. All enrolled patients will be asked to complete a brief baseline survey (T0) and will receive $25 
upon completion of the baseline survey. Additionally, all patients will be re-contacted approximately 6 
weeks after enrollment (T1) and after study completion (12 weeks from study start date; T2) to conduct 
two follow-up surveys. Patients who complete the midpoint (T1) survey will receive an additional $25 
upon survey completion. Patients will receive the final $25 upon return of the electronic monitoring cap 
to the study team. If patients do not complete any additional surveys beyond baseline and do not return 
the electronic monitoring cap, they will not receive any additional remuneration beyond the initial $25. 
Patients who do not complete the baseline survey will not be enrolled and not provided with any 
remuneration. 
 
Patients will receive remuneration through issuance of GreenPhire ClinCards that will be mailed to 
participants within 10 days of enrollment and completion of baseline survey. These ClinCards can be 
used in the same manner as a credit card. ClinCards can be re-loaded remotely with additional funds 
by study team members after completion of study activities. Given the amount of participant 
remuneration, we are requesting a waiver of collection of social security numbers for the ClinCards. 
 
6. RANDOMIZATION 
 
6.1 Groups 
 
Arm 1: Intervention Group 
The intervention is an automated, text-messaging program (chatbot) named “Penny” that is guided by 
artificial intelligence and natural language process. Penny engages patients in real time via text 
messaging and continues to learn from interactions. Patients can also directly communicate with Penny 
at any time. Core functionalities include: (1) real-time dosing instructions, (2) motivational reminders, 
and (3) symptom monitoring with self-management support. Penny transmits data from text interactions 
into the secure online dashboard in real time, allowing for enhanced communication and longitudinal 
monitoring between clinicians and patients. Penny is essentially a virtual care team member, always 
available to support patients receiving complex oral anticancer therapies. Preliminary data from 10 
cancer patients indicate that Penny helped patients to manage side effects and adherence, and can be 
integrated into care safely and effectively.28 This project will expand upon these findings to evaluate the 
efficacy of Penny as a patient-directed intervention, and to explore how multilevel mechanisms may 
impact its acceptability and effectiveness in routine care.  
 
Over an approximate 12-week treatment period, Penny will guide patients randomized to the treatment 
arm through when and how to take their oral therapies. Penny will additionally monitor patient-reported 
side effects based on the PRO version of Common Terminology Criteria for Adverse Events (PRO-
CTCAE) grading and established clinical pathways used currently at Abramson Cancer Center to 
escalate and manage patient-reported symptoms (See Appendices).29,30 If the side effect is determined 
to be Grade 1-2 (mild to moderate), patients will be walked through self-care by the chatbot utilizing 
clinically approved and pre-determined symptom management pathways. If the side effect is deemed 
Grade 3 (severe) or higher or if the chatbot does not recognize the symptom to triage, the patient will 
be advised to call his/her care team and an real-time escalation message will be sent to the clinical 

 
10 
research team’s inbox  to follow-up directly with the patient. While the chatbot will be responding 
independently based on embedded algorithms, clinical research team members will monitor and audit 
conversations daily between the patient and Penny to validate its safety, track for any information that 
may be relayed to the patient incorrectly and intervene upon these events to guarantee patient safety. 
Clinical research team members include Samuel Takvorian (Physician, Medical Oncology), Beth 
Mooney (Nurse Practitioner, Medical Oncology), Deirdre Yarosh (Clinical Pharmacy Specialist, 
Thoracic Oncology), and Christine Cambareri (Outpatient Clinical Oncology Pharmacy Specialist). 
Additionally, any Grade 3-4 toxicities or side effects that are severe will be triaged directly to the 
patient’s care team. Coding corrections that are not related to symptom monitoring (e.g. inability to 
recognize new words or commands from patients) will also be reviewed in real-time within the chatbot 
by Memora Health team members to improve the quality of communication. All issues will be tracked 
and any adverse events or deviations from protocol will be reported to the IRB as outlined below. To 
provide additional safety, the research team will communicate clearly (in writing and verbally at several 
time points) that the intervention is experimental, and that patients should always reach out to their care 
team directly if they are uncertain or if they are facing an emergency.  
 
Arm 2: Usual Care 
Patients in the usual care arm will receive standard of care for symptom monitoring and management of 
side effects for patients with cancer at Abramson Cancer Center. This includes patient education at the 
time of prescribing and guidance to call providers or triage nurses if anything issues or questions arises 
between scheduled appointments. 
 
6.2 Assignment 
Participants will be randomized individually to 1 of the 2 arms using REDCap randomization module, 
stratified by three conditions: 1) medication status (new or prevalent starts), 2) drug class (ALK vs 
ROS1), and 3) study hospital (PCAM vs PPMC). Enrolled patients will learn their assignment (open 
label) following completion of the baseline survey. 
 
6.3 Blinding 
Co-investigators and the primary statistician and analyst will be blinded to the randomization 
assignment. The clinical research team, PIs, and patients will be unblinded given the nature of study 
monitoring and intervention. The clinical research coordinator will record the randomization 
assignments on a master list and enter into REDCap using the randomization module. Assignments will 
be maintained by the research coordinator on a password protected computer in a locked office. The 
blind may be broken in the case of an emergency. 
 
7. STUDY PROCEDURES 
 
7.1 Eligibility Screening and Recruitment 
Aim 1. Potentially eligible participants will be identified in one of three ways: (1) direct referral from 
inpatient or outpatient medical oncology teams, (2) manual chart review by research staff of new 
medication orders; and (3) extraction of eligible patients from Clarity based on the inclusion criteria 
listed above. Due to COVID-19, potentially eligible patients will be approached by a member of the 
clinical research team by phone; study introduction will only be conducted in person if allowable by 
research resumption guidelines. A member of the study team will attempt to contact the identified 
patient up to three times by phone to assess interest (recruitment script included in the protocol). 
During the screening phone call, if the patient expresses interest in the study, the study team member 
will ask the patient to provide contact information and preferred day and time to schedule a follow-up 
phone call to discuss informed consent. The study team member will also add a note to the patient’s 
medical record noting the patient has been approached and if the patient agreed or not to be contacted 

 
11 
further about the study. Prior to study launch, all medical oncologists in the lung cancer program at 
Penn will be notified of the study and asked for blanket approval to contact their patients if eligible. 
 
Aim 2. For patient interviews, enrolled patients in the intervention trial will be purposively sampled by 
study arm, race and age, and adherence outcome (in batches of 15) to participate in a one-time 
interview approximately 2 weeks after the completion of active intervention. As part of the trial consent 
process, patients who enroll in the trial will be notified that a subsample will be approached for this 
secondary component of the study. At the time of trial completion, a research coordinator will contact 
the sampled patients via phone to assess interest in participating in the interview. If patients are 
interested, they will be scheduled for an in-person or phone interview at their convenience. Research 
coordinators will attempt to contact patients no more than three times via phone to assess interest, 
following which patients will not be contacted further. For clinician interviews, the PIs will reach out to 
clinicians in ACC via blanket email following completion of the trial. The email will describe the general 
scope of the study and eligibility criteria and ask providers to respond if they are interested in 
participating. We will also directly reach out to individual providers we believe are eligible based on 
clinical relationships to assess interest, and also use snowball sampling to identify additional providers 
following enrollment of first providers.  
 
7.2 Consent and Enrollment  
Aim 1: Consent Discussion (Trial participants): If a patient expresses interest in participating following 
the study introduction, the patient will be contacted by phone to have a consent discussion in 
accordance with Penn Medicine remote consent procedures. During the consent discussion, we will 
discuss the voluntary nature of the study, risks and benefits of participation, randomization, and answer 
any questions that the participant may have. We will discuss the potential accessibility of patients’ 
health information by a third party, Memora Health, for withdrawal of specific data points necessary for 
the development and programming of “Penny” (name, drug type, prescription, contact information). We 
will ask patients to verbally confirm willingness to participate or decline participation.  
 
Aim 1: Documentation of Consent (Trial participants): For those participants who verbally confirm 
willingness to participate during the consent discussion, we will use REDCap to document consent as 
recommended by Penn Medicine remote consent procedures. The individualize REDCap link will 
include the full informed consent form and ask participants to document consent and understanding 
prior to proceeding to the baseline survey. We will also mail and/or email participants a copy of the 
informed consent form (included in this protocol) that will include information on how to contact the 
research team via email or phone if they have study related questions, concerns or choose to 
withdrawal from the study at any time. Only participants who document consent and complete the 
baseline survey will be randomized and included in this study. 
 
Aim 2: Verbal Consent & Waiver of Written Consent (Interview participants only): For patient and 
clinician interviews, the consent process will be separate from the consent process for the trial. 
Interview participants will be asked to provide verbal consent prior to commencement of the interview. 
The verbal consent discussion will describe the potential benefits and risk of participating in the 
interview and provide information on referral resources should they experience any discomfort or 
concern from participation. Due to COVID restrictions, we anticipate the vast majority if not all 
interviews will be conducted via telephone. Therefore, we are requesting a waiver of written 
documentation of informed consent. Interviews present no more than minimal risk of harm and involve 
no procedure for which written consent is normally required. The written consent form would be the only 
record linking the participant with the interview data. Consent statements will be provided via email to 
participants prior to the interview and also reviewed verbally at the time of the interview. Verbal consent 
forms and recruitment scripts for patient and clinician interviews will be submitted for IRB review prior to 
commencement of interviews. 

 
12 
8. DATA MANAGEMENT 
 
8.1 Data Collection  
This study is staffed by a research support team that includes a data manager and statistical analyst 
who will have direct access to study data. Additional study staff (project manager, research 
coordinators, investigators) will have access to patients’ contact information in order to coordinate study 
activities. Access to all study data will be limited to specifically designated researchers.  
 
8.2 Assessment Procedures 
MEMS Caps. Adherence data will be assessed via MEMS caps, which capture a date and time stamp 
each time the pill bottle is opened. MEMS caps are considered the gold standard of objective 
adherence measurement over other methods of adherence monitoring,31,32 and have been used 
successfully in previous studies of oral anticancer agents.5,25 MEMS data will be scanned to a secure 
web portal by research staff at the time of clinic appointments. All participants will receive MEMS cap(s) 
regardless of study assignment. Participants will be instructed to use this system for storage throughout 
the study, and to log any bottle openings for purposes other than pill ingestion (e.g., to check remaining 
supply) in a daily journal. After patients complete the baseline survey, a study team member will mail 
the MEMS Caps directly to the patients with instructions on how to install the MEMS cap. Patients will 
be contacted by phone approximately 7-10 days after the MEMS cap is mailed to them to confirm 
receipt and answer any questions. Patients will be instructed to begin using the MEMS cap the day 
immediately following the phone call as this will be their study start date. 
 
Online Surveys. Trial participants (Aim 1) will be asked to complete online surveys at three timepoints: 
baseline assessment (T0) will occur at time of enrollment, followed by mid-point assessment 
approximately 6 weeks later (T1), and final assessment occurring within 2 weeks of trial completion 
(T2). Data will be captured via REDCap, a secure, web-based application for collecting and managing 
survey data33. For REDCap data, patients will receive individualized links to each survey and will be 
contacted up to three times by email, text message, and/or phone to complete each survey. To 
increase data completion and study retention incremental compensation will be provided to patients. 
Specific measures to be assessed in the survey are described below. 
 
Medical Record Data: Medical record data from enrolled patients will be abstracted across the study 
period to evaluate diagnostic and treatment changes, clinical characteristics, and healthcare utilization. 
We will also use medical record data to identify and recruit potentially eligible patients as outline in the 
HIPAA waiver. 
 
Interviews (Aim 2). Patient Interviews: A semi-structured guide including open- and closed-questions 
informed by the Information-Motivation-Behavioral Skills (IMB) model will be used to a) examine patient 
experiences including barriers and facilitators to medication adherence; b) assess usability and 
acceptability of Penny as part of care; c) explore the potential impact of Penny or other strategies on 
medication adherence, self-efficacy, and symptom management; and d) gather additional feedback on 
how Penny could be improved to help support care. Clinician Interviews. At the beginning of all clinician 
interviews, clinicians will be given a synopsis of Penny (how it works, what it aims to do, and 
preliminary outcomes from pilot testing), and shown an informational video to serve as a stimulus for 
the interview. A semi-structured guide including open- and closed-ended questions will be used to 
explore a) feasibility, acceptability, and potential impact of using Penny to improve medication 
adherence, symptom management, and other cancer outcomes; b) how various factors may shape 
integration of Penny into routine cancer care, drawing from the five domains in the Consolidated 
Framework for Implementation Research (CFIR); c) standard of care provided to patients on targeted 
therapies; and d) clinician characteristics. Both patient and clinician interview participants will also 
complete a structured questionnaire prior to the interview to assess closed-ended questions related to 

 
13 
medication beliefs and practices and perceived usefulness and ease of use of Penny. A copy of the 
final patient and clinician interview guide will be submitted for IRB review prior to interview 
commencement. 
 
Interviews will last 30-45 minutes and will be supervised by Dr. Rendle, who has extensive experience 
conducting interviews with clinicians and patients. Interviews will take place in a private setting or via 
phone (based on participant preference and research resumption guidelines). Structured 
questionnaires will be administered verbally or online (using REDCap) prior to the interview. Interviews 
will be audio-recorded and transcribed verbatim using a HIPAA-compliant transcription company.  
 
8.3 Measures 
Adherence (Primary Outcome). For each participant, adherence will be calculated based on the overall 
proportion of adherent days (defined dichotomously based on whether a patient took all prescribed 
doses for the day within 2 hours of recommended time). Participants who are 95% adherent or above 
will be classified as adherent (1) and those below will be considered not adherence (0). For therapies 
that have more than one dose per day, we will also calculate the proportion of doses taken among all 
prescribed doses. Additionally, we will evaluate persistence as a secondary adherence outcome 
(defined as the number of consecutive days on the regimen and conversely gaps in overall treatment) 
to assess how Penny may impact different dimensions of adherence. Adherence will also be assessed 
via self-report (using a validated visual analog rating scale)34,35 and prescription refill orders from the 
EMR to assess accuracy of self-report and provide contingency measures to support potential, yet 
unlikely, failures in MEMS caps.36  
 
Exploratory Outcomes. Across the intervention study, we will quantitatively assess patient-reported 
outcomes at various time-points to reduce burden and capture longitudinal changes. Specifically, we 
will measure medication beliefs and practices,39, health-related quality of life (HRQOL),37 symptom 
burden,38,39 patient satisfaction,40  financial toxicity,41 social and behavioral determinants of health 
(SBDH),42 medication self-efficacy,47 patient-provider communication, and health literacy.40 HRQOL will 
be measured using the EuroQol EQ-5D, a 5-item index that assesses mobility, self-care, pain, usual 
activities, and anxiety/depression, and provides a composite score of general HRQOL.37 Symptom 
Burden will be measured using the Edmonton Symptom Assessment Scale (ESAS).42, 43 Patient 
Satisfaction will be measured by a single item measuring overall patient satisfaction from FACIT-TS-
PS, a validated survey.40 Financial Toxicity will be measured using the 11-item Comprehensive Score 
for financial Toxicity (COST) instrument, developed to assess financial toxicity in patients with cancer.41 
SBDH will be assessed by a validated panel of measures recommended by the Institute of Medicine.42 
Medication self-efficacy will be measured by the 13-item Self-Efficacy for Appropriate Medication Use 
Scale (SEAMS), which has been validated for this purpose.43,44 Patients in the intervention arm will also 
assess sability using the Health Information Technology Usability Evaluation Scale (Health-ITUES). 
Lastly, we will capture impact of COVID on patient experiences and care. Clinical characteristics (tumor 
and treatment specifics), acute care utilization (emergency room and hospital admission), and mortality 
will be measured as count variables across the study period using EMR data and also patient self-
report to capture any visits outside our healthcare system. Prior to study launch, we pilot tested the 
survey measures to assess participant burden and reduced. All final survey items are included in the 
Appendices. 
 
9. ANALYSIS PLAN 
 
9.1 Primary Analyses 
Our primary analyses will use an intent-to-treat (ITT) approach using repeated measures logistic 
regression to compare the overall effect of the intervention on patient adherence. We will also explore 
potential differences in secondary outcomes across arms. Baseline clinical and survey data will be used 

 
14 
to assess potential differences in treatment groups (i.e., socio-demographics and clinical risk factors) 
using parametric or nonparametric tests as appropriate. Data quality, normality, and other assumptions 
will be assessed prior to analysis. Using repeated measures logistic regression fitted using generalized 
estimating equations (GEE), we will compare the proportion of adherent days between Penny and 
usual care. Data will be treated as binary outcome (0/1) over the study period.  Models will contain a 
predictor term for treatment arm, will be adjusted for time (discrete week), and will contain subject 
specific random effects and fixed effects for site.  Patients will be categorized as adherent (outcome=1) 
across the study period if the total percentage of adherent days is >95% similar to previous studies7; 
other percentages (80%, 90%, 100%) will be explored in sensitivity analyses. Persistence will be 
treated as repeated measure survival data and will be summarized as median survival time using 
Kaplan-Meier methods. We will also generate summary statistics (including mean adherence rate) to 
help describe the study population and inform the design of further studies.  
 
9.2 Exploratory Analyses 
In exploratory analyses, we will apply standard mediation analysis to explore separately whether 
symptom management or self-efficacy mediates the relationship between arm assignment and 
adherence. We will conduct multivariable regression to explore the following: (1) if arm assignment has 
an effect on symptom management or self-efficacy, (2) if either of these intermediate outcomes has an 
effect on daily adherence; (3) if arm assignment has an effect on daily adherence, or (4) if the effect of 
arm assignment on daily adherence shifts when either of these intermediate outcomes is added to the 
model. Rejecting the null for all four hypotheses will be necessary to support either symptom 
management or medication self-efficacy as a mediator. We will use a modified Sobel test for 
significance testing. Additionally, we will explore for potential effect modification across different patient 
subgroups based on patient demographics, health literacy, social determinants of health, clinical 
characteristics, and other factors. We will fit multivariable regression models with intervention, 
moderator, and interaction terms to assess potential effect modification based on significance of the 
interaction term. As planned analysis, we will also explore mortality differences between study arms at 
6- and 12-month follow-up using data collected passively from the EHR (as outlined in the consent 
form).  
 
9.3 Qualitative Analyses (Aim 2) 
We will use convergent mixed-methods analysis to explore multilevel factors shaping acceptability, 
effectiveness, and implementation of Penny. The constant comparative method, guided by grounded 
theory, will be used to inductively explore emergent themes and deductively identify a priori domains of 
interest (guided by the five CFIR domains and IMB model) within and across interviews. Two trained 
coders will first independently read each transcript to identify themes within each domain. We then will 
use this list to develop a coding dictionary and apply it to subset of the data. We will measure inter-rater 
reliability to document and improve coding consistency. Once high reliability is achieved (kappa > 0.7), 
we will apply the full coding dictionary to the interview data using NVivo (computer-assisted qualitative 
analytic software), and produce thematic reports summarizing our findings by each domain and sub-
theme. Survey data will be analyzed descriptively. Qualitative and quantitative data will be analyzed 
and triangulated using a concurrent mixed-methods approach.45  
 
10. DATA MONITORING AND SAFETY 
 
10.1 
Investigative Team 
Our interdisciplinary team at the University of Pennsylvania includes investigators with distinguished 
records in cancer care delivery, implementation science, behavioral economics, pragmatic trials, EMR-
based strategies, adherence, and mixed-methods research. The project will be co-led by Katharine 
Rendle, PhD, MSW, MPH, an interdisciplinary behavioral scientist specializing in cancer care delivery 
and implementation science research, and Samuel U. Takvorian, MD, MSHP, a medical oncologist with 

 
15 
clinical and research expertise in patient-reported outcomes and symptom management in oncology 
care. Other key investigators include Justin Bekelman, MD, the Director of PC3I and leader of cancer 
care reform and innovation; Lawrence N. Shulman, MD, the Deputy Director for Clinical Services at 
ACC and Chair of the Commission on Cancer; E. Paul Wileyto, PhD, lead biostatistician with expertise 
in clinical trials, longitudinal methods, and categorical data; and Stephen E. Kimmel, MD, MSCE, an 
international expert in the assessment and evaluation of medication adherence. Our research team 
also includes clinical experts embedded within Abramson Cancer Center who will help to ensure quality 
and safety of the trial including: Beth Mooney (Nurse Practitioner, Medical Oncology), Deirdre Yarosh 
(Clinical Pharmacy Specialist, Thoracic Oncology), Christine Cambareri (Outpatient Clinical Oncology 
Pharmacy Specialist), and Linda Jacobs, PhD, CRNP (Nurse Practitioner).  
 
10.2 
Regulatory Approvals 
The University of Pennsylvania IRB will serve as the IRB of record for this trial. The study has a 
dedicated project manager who will ensure that the most current version of the study protocol and 
supplementary materials are added into Penn’s Human Subjects Electronic Research Application (HS-
ERA) system and the Abramson Cancer Center’s Clinical Trials Scientific Review and Monitoring 
Committee. The project manager will additionally be responsible for submitting protocol-wide 
modifications and for reporting any deviations, exceptions, and reportable events within required 
timeframes to the co-PIs and IRB. A formal closure request will be submitted once study activity has 
been completed and there is no further access to identifiable subject data for research purposes.  
 
10.3 
Data Confidentiality  
Paper-based records will be stored in a secure location and only be accessible to personnel involved in 
the study. Computer-based records will be stored on a secure UPHS server, within PennChart, or on 
the third party development company, Memora Health’s, secure and HIPAA compliant platform.  The 
technical components of this platform have been reviewed by Penn Medicine IS Security and was 
approved for PHI. Select staff from Memora Health will have access to the Penny platform during the 
study, along with investigators that have been listed from the Penn Medicine. All Penn Medicine staff 
have completed their HIPAA security training and all co-PIs and key investigators have completed CITI 
Protection of Human Subjects Research Training. 
 
Since this study will be done over text messaging, neither Penn Medicine nor Memora Health can 
guarantee the confidentiality of the information sent to a patient once it arrives on their phone.  We will 
include this language in the consent form and it will be reviewed orally with patients at the time of 
consent.  
 
10.4 
Subject Confidentiality  
 
How will confidentiality of data be maintained?  Check all that apply. 
 Paper-based records will be kept in a secure location and only be accessible to personnel 
involved in the study. 
 Computer-based files will only be made available to personnel involved in the study through the 
use of access privileges and passwords. 
 Prior to access to any study-related information, personnel will be required to sign statements 
agreeing to protect the security and confidentiality of identifiable information. 
 Whenever feasible, identifiers will be removed from study-related information.  
 A Certificate of Confidentiality will be obtained, because the research could place the subject at 
risk of criminal or civil liability or cause damage to the subject’s financial standing, employability, or 
liability. 

 
16 
 A waiver of documentation of consent is being requested (for interview participants only), 
because the only link between the subject and the study would be the consent document and the 
primary risk is a breach of confidentiality.  (This is not an option for FDA-regulated research.)  
 Precautions are in place to ensure the data is secure by using passwords and encryption, 
because the research involves web-based surveys.   
 Audio and/or video recordings will be transcribed and then destroyed to eliminate audible 
identification of subjects.  
 Other (specify):    
 
 
Aim 1. All patients participating within the study will be made aware of all risks related to subject 
confidentiality, including the minimum amount necessary of personal data being sent to a third-party 
(Memora Health, with whom we do have a Business Associate Agreement and has previously been 
approved by Penn Medicine IS Security) and the security limitations around receiving text messages. 
This will be stated in the consent form and reviewed verbally with each patient during consent 
discussion.  All other data that contains subject information will be either physically securely stored or 
be protected on secure UHPS servers, REDCap, or within the patient’s medical record.   
 
Aim 2. The most substantial risk to all participants includes potential violation of confidentiality. To help 
protect again this risk, we will use the following safeguards: 1) all data including but not limited to 
consent forms, questionnaire data, audio files from interviews, notes from medical record review, and 
qualitative transcripts will be kept in a password-protected file or locked cabinet; 2) participant identity 
will be masked using unique participant IDs and stored on a password-protected master list to which 
only the PI and approved research staff will have access; 4) any paper questionnaire data will be 
collected and entered into secure REDCap and then destroyed after study completion; 6) any protected 
health information received from the participants will be housed on secure UPHS servers and only 
accessible by the PIs and approved research members; and 7) all statistical or qualitative analytic files 
will be identified only with participant IDs and not contain any protected health information (all 
identifiable data will be recoded (e.g. true dates will be turned into days from index date) prior to 
including it into the analytic datasets). For interviews, all identifying information will be redacted from 
the transcription. Audio files will be kept on the UPHS secure server and only shared with HIPAA-
compliant transcriptionists. Upon completion of the study, audio files will be destroyed in accordance 
with IRB policy. In manuscripts, analytic datasets, or summation or reporting of data, no PHI will be 
used. 
 
10.5 
Subject Privacy 
For trial participants, enrollment will include a detailed description of the voluntary nature of 
participation, study procedures, risks and potential benefits. The enrollment procedures will allow the 
opportunity for potential participants to ask questions and review with family and friends prior to making 
a decision to participate. Participants will be informed that they may drop out at any time, without 
affecting their medical care or cost of their care.  They will be told they may or may not benefit directly 
from the study and that all information will be kept strictly confidential, except as required by law. 
Subjects will be given a copy of the consent form.  All efforts will be made by study staff to ensure 
subject privacy. 
 
For patient and provider interviews, to protect subject privacy, the consent process and interviews will take 
place in a private location or by phone. During the consent process, we will inform participants that they 
have to the right to skip any question that they prefer not to answer and can withdraw from participation 
at any time. 
 
10.6 
Data Disclosure 

 
17 
The following entities, besides the members of the research team, may receive protected health 
information (PHI) for this study: 
 
Memora Health, the company we are working with to develop Penny and who possesses the AI 
chatbot technology. Memora Health has a Master Services Agreement contract in place with 
Penn Medicine, have previously signed a BAA and has been reviewed and approved to receive 
PHI by Penn Medicine’s IS Security Team.   
 
The Office of Human Research Protections at the University of Pennsylvania 
 
Federal and state agencies (examples – Department of Health and Human Services, the 
National Institutes of Health, and/or the Office for Human Research Protections), or other 
domestic or foreign government bodies if required by law and/or necessary for oversight 
purposes.  
 
10.7 
Risk/Benefit Assessment 
 
Potential Study Risks 
 
For trial participants (Aim 1), there are potential physical risks to the patients.  These include: 
 
Incorrect medication adherence information given to the patient through text messages 
 
Incorrect symptom management information given to the patient through text messages 
 
Incorrect alerts to the patient’s care team based on text message information 
 
HIPAA concerns related to text messaging 
 
Non-study personnel having access to the PHI of patients, specifically through the texting 
interactions on a patient’s mobile device or through a security breach on the program platform 
 
Discomfort with disclosing information in the online surveys 
 
Patients will be made aware of these risks during the informed consent process, will be informed that 
study investigators will be monitoring all interactions between the patient and Penny, reminded that 
Penny should not be used for emergency situations, and will be encouraged to ask questions. 
 
For interview participants (Aim 2), potential risks to participating in this study include: potential violation 
of confidentiality or privacy; possible discomfort in disclosing information on screening knowledge, 
beliefs or practices; and possible concern among providers or employees that participation will 
negatively impact job or clinic performance or evaluation. These risks will be outlined in the interview 
consent form (to be submitted for review prior to commencement of interviews). 
 
Potential Study Benefits 
 
For trial participants (Aim 1), there are potential study benefits to the patients and society.  These 
include: 
 
Easing the process of adherence to a patient’s oral therapy regimen 
 
Real time feedback on side effects and symptoms for the patient 
 
Improved communication for the patient with their care team 
 
Potential to improve the quality of life for the patients 
 
Potential to improve the outcome of an oncology diagnosis and its treatment 
 
Potential reduction in costs associated with a patient’s oncology-related medical care if ED 
visits, admissions and side effects are prevented and/or managed sooner 
 
If the study is successful, expanding to use in more disease states within oncology to provide 
benefit to future patients 
 

 
18 
For interview participants (Aim 2), there are no direct benefits who participate in the interview 
component of this study. However, they may indirectly benefit by discussing experience and practices.  
For the health of society, the benefits of the study are substantial and include potential positive impact 
on cancer outcomes through improved symptom management, better adherence, and better 
communication between care teams and patients.  
 
Risk/Benefit Assessment 
 
Although there are potential risks associated with this study, current care and support provided to 
patients on oral therapies is limited. The lack of medical oversight at home can lead to incorrect dosing 
and administration of these cytotoxic drugs and in turn potentially hazardous toxicities, underscoring the 
need for interventions such as the one proposed in this study to help patients. Additionally, we have 
incorporated protocols to help mitigate potential risks including monitoring of interactions between 
patients with Penny by clinical research team members and escalation of side effects to the patient’s 
care team. 
 
The potential benefits of this study are great and include the potential to improve adherence and 
symptom management, improve monitoring of self-based care and decrease unplanned care for cancer 
patients. If proven to be successful, this study could have a significant positive impact on oncology care 
across healthcare systems. 
    
10.8 
Data Safety and Monitoring  
As a phase II behavioral pilot trial, overall data and safety will be monitored on an ongoing basis by the 
PIs and the study team. As needed, the investigators may decide to appoint an independent medical 
monitor to evaluate adverse events and make recommendations for continuing or stopping a trial. No 
interim analysis will be conducted. 
 
Monitoring Text Interactions: The overall framework for monitoring patient-reported data includes 
notifications to all study team members when a patient sends in a message, with 24-hour check-ins on 
all new patient interactions from the clinical research team and from Memora Health team members. 
Given the patient volume, nurse and physician researchers are included in this study for this specific 
reason. A patient’s enrollment in the study will be terminated if they experience an emergency medical 
situation requiring intervention or violate study protocol.  
 
Adverse Event Reporting: If any breaches in confidentiality or identifying information are identified, the 
PI will be responsible for jointly notifying the IRB using the eIRB adverse event reporting system within 
24 hours. The IC and NIH Office for Biotechnology Activities will additionally be notified within 24 hours 
via email. Furthermore, all protected health information will be destroyed within five years after study 
publication. All HIPAA-mandated federal regulations will be followed accordingly. 
 
The probability of adverse events remains extremely low given the nature of this intervention but 
several safeguards have been built into the protocol to mitigate and monitory any risk due to errors in 
the text messaging or symptom pathways. First, patients can report all events directly to the study staff 
or via SMS. Second, all interactions by the chatbot will additionally be monitored by the PI and nurse 
researchers at a minimum frequency of every 24 hours to ensure escalations are properly routed and 
any adverse events addressed and recorded. Depending on the nature of the event, the individual’s 
participation in the study may be terminated. In the case of a technical error that results in patients 
receiving inaccurate information via SMS, the product development team at Memora Health will 
immediately address the error but enrollment will not be terminated. If the issue cannot be immediately 
resolved without affecting other patients, the study will be temporarily put on hold until the issue is 
resolved. In the case of a data leak or HIPAA violation, the study staff and Memora Health team will 

 
19 
jointly address the nature of the leak and determine if any PHI has been leaked. Beyond this, standard 
HIPAA reporting guidelines will be followed. 
 
Integrity of Study Data: The primary step to maintaining integrity of the data will involve de-
anonymization of data prior to any analysis for statistical significance. This is aimed towards reducing 
any potential risk associated with skewing data based on demographic or personal health information. 
The care team at the clinical site and the Memora Health team, of which both fall under approved study 
personnel, will store separate, secure versions of the raw text data in a password- protected file and a 
secure database, respectively. This will allow for cross-checking between data sets to assure that no 
data has been modified and no results skewed. Specifically, the clinical PI will oversee the data set on 
behalf of the clinical site and the care team, while Manav Sevak at Memora Health, will oversee the 
data set on behalf of the Memora Health team. 
 
Adherence to the IRB-approved protocol will be assured by restricting participant enrollment to Penn 
Medicine research staff who will have conducted the appropriate training, including CITI. Research 
assistants and staff trained in IRB protocol for study enrollment will participate in enrollment. 
Additionally, study staff and corresponding research assistants will be briefed on study protocol prior to 
the beginning of the study to assure that the correct procedure and guidelines are followed. 
 
Mobile Security: Patients will not need to download any mobile app to participate in this study. All study 
content and expected responses will take place via secure web data form. All of the data both in transit 
to and from the Memora Health servers, and at rest, will be encrypted. A two-factor authorization 
approach will be taken at the time of enrollment to maintain confidentiality of patient information. When 
a patient is first enrolled in the study, they will receive an introductory SMS asking them to verify their 
identity to assure that messages are being sent to the right individual. Patients are required to opt-in to 
sending and receiving personal health information over SMS immediately after enrollment in the 
platform and are instructed on the risks of using SMS as a means of communicating health information. 
This process complies with both patient privacy and telecommunications laws. Patients will receive 
regular SMS prompts and personally identifiable health information will be omitted from all text 
messages, outside of the patient name, to maximize confidentiality on the patient end. Text reminders 
have patient names in them to assure that the correct person is completing the survey. 
 
Platform Security: Two sets of information, provider login information and patient data, will be 
confidentially stored to assure that data storage abides by HIPAA regulations. Provider login 
information is all 192-bit encrypted using bcrypt, a cryptographic algorithm used for file and data 
encryption purposes. Additionally, provider login passwords, which must be at least eight characters 
long, are hashed as a second layer of security to protect the identity of providers using the software. 
Patient data is 256-bit encrypted using Advanced Encryption Standard (AES) while it is being 
transmitted to the database for storage. Full-disk encryption in a HIPAA-compliant database sponsored 
by Google Cloud Platform (GCP), a verified cloud data storage location, is used for storage of all 
information. 
 
Data will be encrypted and stored on GCP as two different sets, personally identifiable (PI) information, 
such as names and phone numbers, and non-personally identifiable (NPI) information. While PI and 
NPI information are inherently linked to each other to allow for providers to monitor patients during the 
course of the study, data can be de-identified by removing PI information. Additionally, this data 
mechanism storage allows for mitigation of losing any personal health information in the improbable 
case of a storage leak. When it is necessary, the personally identifiable information will automatically be 
stripped before being presented to authorized personnel. 
 

 
20 
All patient data stored by the study staff will be maintained in a password-protected file that contains 
codes corresponding to a patient’s PHI. The code will correlate to a patient’s data stored in a separate 
password-protected file. Ultimately, only authorized study staff will be given login credentials for the 
platform and therefore controlling the individuals who have access to patient data. 
 
No personally identifiable (PI) information will be shared with anyone outside of the research team, 
members of the Memora Health team, or patient’s care team. Any data shared following completion of 
the study will be de- identified prior to use, protecting the patients’ PHI and abiding by the HIPAA 
Privacy Rules. The only foreseeable disclosure of data beyond the study is in the case of publication of 
data. 
 
11. REFERENCES 
 
1. Jacobs JM, Pensak NA, Sporn NJ, et al. Treatment Satisfaction and Adherence to Oral 
Chemotherapy in Patients With Cancer. J Oncol Pract. 2017;13(5):e474-e485. 
2. Eek D, Krohe M, Mazar I, et al. Patient-reported preferences for oral versus intravenous 
administration for the treatment of cancer: a review of the literature. Patient Prefer Adherence. 
2016;10:1609-1621. 
3. DiMatteo MR. Variations in patients' adherence to medical recommendations: a quantitative review 
of 50 years of research. Med Care. 2004;42(3):200-209. 
4. Ruddy K, Mayer E, Partridge A. Patient adherence and persistence with oral anticancer treatment. 
CA Cancer J Clin. 2009;59(1):56-66. 
5. Greer JA, Amoyal N, Nisotel L, et al. A Systematic Review of Adherence to Oral Antineoplastic 
Therapies. Oncologist. 2016;21(3):354-376. 
6. Hess LM, Louder A, Winfree K, Zhu YE, Oton AB, Nair R. Factors Associated with Adherence to 
and Treatment Duration of Erlotinib Among Patients with Non-Small Cell Lung Cancer. Journal of 
managed care & specialty pharmacy. 2017;23(6):643-652. 
7. Timmers L, Boons CC, Moes-Ten Hove J, et al. Adherence, exposure and patients' experiences 
with the use of erlotinib in non-small cell lung cancer. Journal of cancer research and clinical 
oncology. 2015;141(8):1481-1491. 
8. Borghaei H, Yim YM, Guerin A, et al. Severe adverse events impact overall survival and costs in 
elderly patients with advanced non-small cell lung cancer on second-line therapy. Lung cancer 
(Amsterdam, Netherlands). 2018;119:112-119. 
9. Ganesan P, Sagar TG, Dubashi B, et al. Nonadherence to imatinib adversely affects event free 
survival in chronic phase chronic myeloid leukemia. Am J Hematol. 2011;86(6):471-474. 
10. Makubate B, Donnan PT, Dewar JA, Thompson AM, McCowan C. Cohort study of adherence to 
adjuvant endocrine therapy, breast cancer recurrence and mortality. Br J Cancer. 
2013;108(7):1515-1524. 
11. Le Saux O, Bourmaud A, Rioufol C, et al. Over-adherence to capecitabine: a potential safety issue 
in breast and colorectal cancer patients. Cancer Chemother Pharmacol. 2018;82(2):319-327. 
12. McCowan C, Wang S, Thompson AM, Makubate B, Petrie DJ. The value of high adherence to 
tamoxifen in women with breast cancer: a community-based cohort study. Br J Cancer. 
2013;109(5):1172-1180. 
13. Osterberg L, Blaschke T. Adherence to medication. N Engl J Med. 2005;353(5):487-497. 
14. Muluneh B, Deal A, Alexander MD, et al. Patient perspectives on the barriers associated with 
medication adherence to oral chemotherapy. J Oncol Pharm Pract. 2018;24(2):98-109. 
15. Shih YT, Chien CR. A review of cost communication in oncology: Patient attitude, provider 
acceptance, and outcome assessment. Cancer. 2017;123(6):928-939. 
16. Gebbia V, Bellavia M, Banna GL, et al. Treatment monitoring program for implementation of 
adherence to second-line erlotinib for advanced non-small-cell lung cancer. Clinical lung cancer. 
2013;14(4):390-398. 

 
21 
17. Lucca JV, Hooper CL, Boucher J, Pedulla LV, Berry DL, Marcoux JP. A pilot study of direct care 
nurse (DCN) education to improve adherence and knowledge of erlotinib in patients with non-small 
cell lung cancer. Journal of Clinical Oncology. 2012;30(15_suppl):e19637-e19637. 
18. Loewenstein G, Asch DA, Volpp KG. Behavioral economics holds potential to deliver better results 
for patients, insurers, and employers. Health Aff (Millwood). 2013;32(7):1244-1250. 
19. DeKoekkoek T, Given B, Given CW, Ridenour K, Schueller M, Spoelstra SL. mHealth SMS text 
messaging interventions and to promote medication adherence: an integrative review. J Clin Nurs. 
2015;24(19-20):2722-2735. 
20. Patel S, Jacobus-Kantor L, Marshall L, et al. Mobilizing your medications: an automated medication 
reminder application for mobile phones and hypertension medication adherence in a high-risk urban 
population. J Diabetes Sci Technol. 2013;7(3):630-639. 
21. Reese PP, Bloom RD, Trofe-Clark J, et al. Automated Reminders and Physician Notification to 
Promote Immunosuppression Adherence Among Kidney Transplant Recipients: A Randomized 
Trial. Am J Kidney Dis. 2017;69(3):400-409. 
22. Darlow S, Wen KY. Development testing of mobile health interventions for cancer patient self-
management: A review. Health Informatics J. 2016;22(3):633-650. 
23. Fishbein JN, Nisotel LE, MacDonald JJ, et al. Mobile Application to Promote Adherence to Oral 
Chemotherapy and Symptom Management: A Protocol for Design and Development. JMIR Res 
Protoc. 2017;6(4):e62. 
24. Ben-Zeev D. Mobile Health for Mental Health in West Africa: The Case for Ghana. Psychiatr Serv. 
2018;69(7):741-743. 
25. Zerillo JA, Goldenberg BA, Kotecha RR, Tewari AK, Jacobson JO, Krzyzanowska MK. Interventions 
to Improve Oral Chemotherapy Safety and Quality: A Systematic Review. JAMA Oncol. 
2018;4(1):105-117. 
26. Kessler JB, Troxel AB, Asch DA, et al. Partners and Alerts in Medication Adherence: A Randomized 
Clinical Trial. J Gen Intern Med. 2018;33(9):1536-1542. 
27. Rendle KA, Abramson CM, Garrett SB, Halley MC, Dohan D. Beyond exploratory: a tailored 
framework for designing and assessing qualitative health research. BMJ open. 2019;9(8):e030123. 
28. Berges BM, Cambareri C, Takvorian SU, et al. Leveraging a conversational agent to support 
adherence to oral anticancer agents: A usability study. Journal of Clinical Oncology. 
2019;37(15_suppl):6534-6534. 
29. Basch E, Reeve BB, Mitchell SA, et al. Development of the National Cancer Institute's patient-
reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). 
Journal of the National Cancer Institute. 2014;106(9). 
30. Dueck AC, Mendoza TR, Mitchell SA, et al. Validity and Reliability of the US National Cancer 
Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse 
Events (PRO-CTCAE). JAMA Oncology. 2015;1(8):1051-1059. 
31. El Alili M, Vrijens B, Demonceau J, Evers SM, Hiligsmann M. A scoping review of studies 
comparing the medication event monitoring system (MEMS) with alternative methods for measuring 
medication adherence. Br J Clin Pharmacol. 2016;82(1):268-279. 
32. Vrijens B, Urquhart J. Methods for measuring, enhancing, and accounting for medication adherence 
in clinical trials. Clin Pharmacol Ther. 2014;95(6):617-626. 
33. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture 
(REDCap)--a metadata-driven methodology and workflow process for providing translational 
research informatics support. J Biomed Inform. 2009;42(2):377-381. 
34. Sevilla-Cazes J, Finkleman BS, Chen J, et al. Association Between Patient-Reported Medication 
Adherence and Anticoagulation Control. Am J Med. 2017;130(9):1092-1098 e1092. 
35. Kalichman SC, Amaral CM, Swetzes C, et al. A simple single-item rating scale to measure 
medication adherence: further evidence for convergent validity. Journal of the International 
Association of Physicians in AIDS Care. 2009;8(6):367-374. 

 
22 
36. De Geest S, Zullig LL, Dunbar-Jacob J, et al. ESPACOMP Medication Adherence Reporting 
Guideline (EMERGE). Ann Intern Med. 2018;169(1):30-35. 
37. Group E. EuroQol--a new facility for the measurement of health-related quality of life. Health policy 
(Amsterdam, Netherlands). 1990;16(3):199-208. 
38. Basch E, Artz D, Dulko D, et al. Patient online self-reporting of toxicity symptoms during 
chemotherapy. J Clin Oncol. 2005;23(15):3552-3561. 
39. Basch E, Deal AM, Kris MG, et al. Symptom Monitoring With Patient-Reported Outcomes During 
Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol. 2016;34(6):557-565. 
40. Peipert JD, Beaumont JL, Bode R, Cella D, Garcia SF, Hahn EA. Development and validation of the 
functional assessment of chronic illness therapy treatment satisfaction (FACIT TS) measures. 
Quality of life research : an international journal of quality of life aspects of treatment, care and 
rehabilitation. 2014;23(3):815-824. 
41. de Souza JA, Yap BJ, Wroblewski K, et al. Measuring financial toxicity as a clinically relevant 
patient-reported outcome: The validation of the COmprehensive Score for financial Toxicity 
(COST). Cancer. 2017;123(3):476-484. 
42. Giuse NB, Koonce TY, Kusnoor SV, et al. Institute of Medicine Measures of Social and Behavioral 
Determinants of Health: A Feasibility Study. American journal of preventive medicine. 
2017;52(2):199-206. 
43. Lamarche L, Tejpal A, Mangin D. Self-efficacy for medication management: a systematic review of 
instruments. Patient Prefer Adherence. 2018;12:1279-1287. 
44. Risser J, Jacobson TA, Kripalani S. Development and psychometric evaluation of the Self-efficacy 
for Appropriate Medication Use Scale (SEAMS) in low-literacy patients with chronic disease. J Nurs 
Meas. 2007;15(3):203-219. 
45. Creswell JW PCV. Designing and Conducting Mixed Methods Research. 3rd ed. Los Angeles: 
SAGE Publications; 2017. 
 
12. APPENDICES