Text-based Intervention to Minimize the Time Burden of Routine Cancer Care
Phase: NA
Conditions: Cancer, Solid Tumor, Patient Empowerment, Telemedicine
Sponsor: Abramson Cancer Center at Penn Medicine

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
 
Text-based intervention to minimize the time burden of routine cancer care 
 
 
 
Principal Investigator (s) 
Ronac Mamtani MD, MSCE 
Erin Bange, MD, MSCE candidate 
Hematology-Oncology 
Perelman Center South Pavilion 
215 605 4567 
ronac.mamtani@pennmedicine.upenn.edu 
Funding Sponsor 
ASCO Conquer Cancer Young Investigator Award 
2318 Mill Road Suite 800  
Alexandria, VA 223314 
grants@conquer.org 
 
 
Protocol Number 
 
 
IRB Number 
UPCC 16921 
 
 
 
 
 
 
 
Initial version 
6/8/2021 
Amended 
7/12/2021 
Amended 
10/26/2021 
Amended 
03/01/2022 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
 
Table of Contents 
BACKGROUND AND STUDY RATIONALE ................................................................................................ 7 
1 
INTRODUCTION .................................................................................................................................. 7 
1.1 
BACKGROUND AND RELEVANT LITERATURE ..................................................................................... 7 
2 
STUDY OBJECTIVES ......................................................................................................................... 7 
2.1 
SPECIFIC AIMS ............................................................................................................................... 7 
3 
INVESTIGATIONAL PLAN .................................................................................................................. 8 
3.1 
GENERAL DESIGN .......................................................................................................................... 8 
3.2 
ALLOCATION TO INTERVENTIONAL GROUP [IF APPLICABLE] ............................................................... 8 
3.3 
STUDY MEASURES ......................................................................................................................... 8 
3.4 
STUDY ENDPOINTS......................................................................................................................... 9 
3.4.1 
Primary Study Endpoint ........................................................................................................... 9 
3.4.2 
Secondary Study Endpoints .................................................................................................... 9 
4 
STUDY POPULATION AND DURATION OF PARTICIPATION ........................................................ 9 
4.1 
DURATION OF STUDY PARTICIPATION .............................................................................................. 9 
4.2 
TOTAL NUMBER OF SUBJECTS AND SITES........................................................................................ 9 
4.3 
INCLUSION CRITERIA ...................................................................................................................... 9 
4.4 
EXCLUSION CRITERIA ................................................................................................................... 10 
4.5 
SUBJECT RECRUITMENT ............................................................................................................... 10 
4.6 
VULNERABLE POPULATIONS: ........................................................................................................ 10 
5 
STUDY PROCEDURES ..................................................................................................................... 10 
5.1 
SCREENING ................................................................................................................................. 10 
5.2 
STUDY TIMELINE ........................................................................................................................... 10 
5.2.1 
Pilot ........................................................................................................................................ 10 
5.2.2 
Randomized control trial ........................................................................................................ 10 
5.2.3 
End of Study Visit .................................................................................................................. 11 
5.3 
UNSCHEDULED VISITS .................................................................................................................. 11 
5.4 
SUBJECT WITHDRAWAL ................................................................................................................ 11 
5.4.1 
Data Collection and Follow-up for Withdrawn Subjects ........................................................ 11 
6 
STATISTICAL PLAN ......................................................................................................................... 12 
6.1 
SAMPLE SIZE AND POWER DETERMINATION ................................................................................... 12 
6.2 
STATISTICAL METHODS ................................................................................................................ 12 
6.3 
CONTROL OF BIAS AND CONFOUNDING ......................................................................................... 13 
6.3.1 
Baseline Data ........................................................................................................................ 13 
6.3.2 
Analysis of Primary Outcome of Interest ............................................................................... 13 
7 
SAFETY AND ADVERSE EVENTS ................................................................................................... 13 
7.1 
DEFINITIONS ................................................................................................................................ 13 
7.1.1 
Adverse Event ....................................................................................................................... 13 
7.1.2 
Serious Adverse Event .......................................................................................................... 14 
7.2 
RECORDING OF ADVERSE EVENTS ................................................................................................ 14 
7.3 
RELATIONSHIP OF AE TO STUDY ................................................................................................... 14 
7.4 
REPORTING OF ADVERSE EVENTS AND UNANTICIPATED PROBLEMS ............................................... 14 
7.4.1 
Follow-up Report ................................................................................................................... 14 
7.4.2 
Data and Safety Monitoring Plan........................................................................................... 15 
8 
STUDY ADMINISTRATION, DATA HANDLING AND RECORD KEEPING .................................... 15 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
8.1 
CONFIDENTIALITY ......................................................................................................................... 15 
8.2 
DATA COLLECTION AND MANAGEMENT .......................................................................................... 16 
8.3 
RECORDS RETENTION .................................................................................................................. 16 
9 
STUDY MONITORING, AUDITING, AND INSPECTING .................................................................. 16 
9.1 
STUDY MONITORING PLAN ............................................................................................................ 16 
9.2 
AUDITING AND INSPECTING ........................................................................................................... 16 
10 
ETHICAL CONSIDERATIONS .......................................................................................................... 17 
10.1 
RISKS .......................................................................................................................................... 17 
10.2 
BENEFITS .................................................................................................................................... 17 
10.3 
RISK BENEFIT ASSESSMENT ......................................................................................................... 17 
10.4 
INFORMED CONSENT PROCESS / HIPAA AUTHORIZATION .............................................................. 18 
10.4.1 
Alterations to Typical Consent Process ............................................................................ 18 
11 
STUDY FINANCES ............................................................................................................................ 19 
11.1 
FUNDING SOURCE ........................................................................................................................ 19 
11.2 
CONFLICT OF INTEREST ................................................................................................................ 19 
12 
PUBLICATION PLAN ........................................................................................................................ 19 
13 
REFERENCES ................................................................................................................................... 19 
 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
Study Summary 
 
Title 
Text-based intervention to minimize the time burden of routine cancer care 
Short Title 
TIME 
IRB Number 
 
Protocol Number 
UPCC 16921  
Methodology 
This is a randomized control trial of a text-based mobile intervention to triage 
patients prior to treatment with single agent checkpoint blockade. Our 
hypothesis is that the use of a text-based e-triage prior to treatment will lower 
total healthcare time relative to standard office visit.  
Study Duration 
1 year  
Study Center(s) 
Single center 
• 
Perelman Center for Advanced Medicine at the Hospital of the 
University of Pennsylvania 
Objectives 
Primary: 
• 
To measure and refine the performance of a text-based instrument to 
assess patient-reported immunotherapy toxicity by determining 
sensitivity and specificity against the gold standard-in person provider 
assessment as documented in the electronic medical record (EMR). 
• 
To compare total care times, defined as time spent commuting to, 
waiting for, and receiving care, associated with a text-based e-triage 
system versus standard office visit among patients with advanced 
cancer receiving immunotherapy over a three-month period.  
Secondary: 
• 
To determine whether a text-based e-triage affects healthcare quality 
of life as well as treatment and provider satisfaction. 
• 
To determine whether a text-based e-triage affects total number of 
emergency department visits and hospitalizations.   
Number of Subjects 
Up to 100 patients will be enrolled for the pilot study. 176 will be enrolled for 
the randomized control trial.  
Main Inclusion and 
Exclusion Criteria 
Inclusion criteria 
• 
Age 18 or older and able to give informed consent 
• 
English speaking 
• 
Receiving single-agent PDL-1/PD-1 checkpoint blockade  
• 
Diagnosed with a solid tumor malignancy  
• 
Access and ability to use a mobile phone with text-messaging 
capabilities 
• 
 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
Intervention  
 
Pilot study to optimize instrument performance: 
• 
Prior to their scheduled immunotherapy infusion, consented patients 
will receive a text-based e-triage questionnaire. Following completion 
of the questionnaire, patients will proceed to their in-person 
assessment and providers will screen patients for potential immune 
related adverse events per routine care. Performance of the e-triage 
system will then be compared to the provider progress note 
documented in the EMR as the gold standard. 
Randomized control trial: 
• 
Treatment arm: For patients in the intervention arm, symptoms and 
laboratory results will be assessed using the text-based e-triage 96 
hours prior to their intended infusion date. The e-triage will consist of a 
standardized questionnaire and algorithm to evaluate symptoms and 
laboratory values. Patients with acceptable labs and minimal or no 
symptoms can opt to proceed directly to their immunotherapy infusion 
without an in-person office assessment.  
• 
Usual Care: Patients in the usual care arm will receive standard of 
care symptom monitoring including an in-person office assessment 
prior to their scheduled immunotherapy infusion.  
Statistical 
Methodology 
Pilot: 
• 
Sensitivity and specificity will be calculated along with exact 95% 
confidence intervals (CIs). Sensitivity will be calculated as the 
proportion of patients with toxicity captured by the text-based e-triage 
among patients with toxicity as documented by the provider in the 
EMR. Specificity will be calculated as the proportion of negative e-
triage screens among patients without symptoms as documented by 
the provider in the EMR. 
Randomized control trial: 
• 
The primary analyses will be Intent-to-Treat, with the primary outcome 
(total care time measured over three months) compared using a two-
sample t test, or Wilcoxon test if normal distribution assumption is 
violated. A multivariate linear regression model will adjust for potential 
confounders. As a secondary analysis, “as treated” analyses will be 
performed to account for permissive non-adherence to the triage 
assignment. Standard descriptive statistics will be used to describe 
patient factors associated with more frequent triage non-adherence.  
• 
Secondary endpoints include: total wait time measured over three 
months or at least 3 office visits, total number of 
hospitalization/emergency department encounters, patient satisfaction 
[The Patient Satisfaction Questionnaire Short Form (PSQ-18)]1, 
provider satisfaction (Net-Promoter Score)2,3, and health related 
quality of life [Functional Assessment of Cancer Therapy-General 
(FACT-G)4 ]. Total wait time and number of hospitalizations will be 
compared between groups using Wilcoxon rank sum test. The rest of 
the secondary endpoints will be measured at multiple time points and 
longitudinal linear mixed-effects model will be used to compare the 
group difference over time. 
Data and Safety 
Monitoring Plan  
Trial oversight will be conducted by the University of Pennsylvania Institutional 
Review Board. Safety will be monitored on an ongoing basis by the PI, study 
team, and institutional experts not involved in the study.   
 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
Background and Study Rationale 
This study will be conducted in full accordance with all applicable University of Pennsylvania Research 
Policies and Procedures and all applicable Federal and state laws and regulations including the following 
regulations as they apply: 45 CFR 46, 21 CFR Part 50. All episodes of noncompliance will be 
documented. 
1 
Introduction 
1.1 
Background and Relevant Literature  
Patients with advanced cancer spend a substantial portion of their time receiving healthcare, often 
despite limited survival benefit5–11. We have previously demonstrated that patients with advanced 
pancreas cancer spend 10% of their remaining survival time receiving healthcare, with greater than 50% 
spent commuting or in waiting rooms8. Additionally, inefficiencies in care delivery reduce access to both 
clinicians and infusions suites. This problem is only projected to get worse as the number of patients with 
cancer grows12–14. Within the Perelman clinics we have seen a rise in visits from 78,525 to 89,117 in three 
years with a corresponding increase in new patient wait times from 17.5 to 38.1 days. Furthermore, 
patients currently wait an average of 30 minutes to receive their cancer treatment in part due to high 
volume of care and need for same day labs, orders, and drug preparation. Given that access to care, 
including patient wait time, is critical determinant of patient satisfaction10,15–21, there is need for innovative 
strategies that mitigate time toxicity while delivering high quality care.  
 
During the COVID-19 pandemic, oncologists have had to rapidly transform cancer care delivery22–25. 
Telemedicine has emerged as the primary strategy to ensure patient and provider safety26. eHealth, a 
subset of telemedicine, supports traditional healthcare practice through email, portals or text27. The use of 
digital technologies to monitor patient reported outcomes (PRO) in oncology is well established, having 
been shown to improve patient provider communication, satisfaction, and symptom distress28–37. While 
digital technologies have primarily focused on capturing PROs, none have yet been applied as a tool to 
reduce the time-burden associated with cancer care. eHealth initiatives can streamline care, thereby 
providing more patient-centered cancer treatment.  
 
In this research, we propose a randomized controlled trial (RCT) of a text-based mobile intervention to 
triage patients prior to cancer treatment. We will leverage WayToHealth©, an established technology 
platform shown to successfully facilitate patient and provider communication through texting for both 
research and clinical care38–41. The overall objective is to determine whether this intervention can safely 
lessen the need for in-person office visits, thereby lowering care time, while also preserving patient 
satisfaction and treatment outcomes. Results from this study will shift the care delivery paradigm toward 
more patient-centered cancer care, improving accessibility and the patient experience.  
2 
Study Objectives 
The primary objective is to test whether a text-based e-triage can safely identify patients who can proceed 
directly to their immunotherapy infusion without a preceding in-person office assessment. 
2.1 
Specific aims  
The specific aims of this single center randomized project are to: 
 
AIM 1: To measure and refine the performance of a text-based instrument to assess patient-reported 
immunotherapy toxicity by determining sensitivity and specificity against the gold standard in-person 
provider assessment as documented in the electronic medical record. 
H1: In this cross-sectional study, a text-based e-triage can successfully identify patients experiencing 
toxicity with a sensitivity of 100%.  
 
AIM 2: To compare total care times, defined as time spent commuting to, waiting for, and receiving care, 
associated with a text-based e-triage system versus standard office visit among patients with advanced 
cancer receiving immunotherapy over  three months or at least three infusion visits.  

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
H2: In this randomized clinical trial, the use of a text-based e-triage prior to treatment will lower total care 
time relative to standard office visit. 
 
In addition to the primary aims, we will also explore: 1) the impact of the intervention on healthcare quality 
of life and patient/provider satisfaction; 2) the effect of the intervention on total number of emergency 
department and hospitalizations across both study arms; 3) total wait times over a three-month period 
associated with the intervention versus standard of care.  
3 
Investigational Plan  
3.1 
General Design 
This study consists of a rapid pilot validation (aim 1) and a randomized control trial (aim 2) to assess the 
performance and efficacy of a text-based e-triage to minimize the time toxicity of cancer care. The rapid 
pilot validation will consist of a cross-sectional study to measure and refine the performance of a text-
based instrument to assess patient-reported immunotherapy toxicity by determining sensitivity and 
specificity against the gold standard in-person provider assessment as documented in the EMR. The 
randomized control trial will involve randomizing patients 1:1 to either the e-triage arm or standard of 
care. Total care times, defined as time spent commuting to, waiting for, and receiving cancer care, will be 
measured, and compared across both study arms over a three-month period or at least three infusion 
visits. See appendix A.  
3.2 
Allocation to Interventional Group  
Patients will be randomized 1:1 to either the intervention (e-triage) or control (standard of care) arms 
using a randomization sequence created using the WaytoHealth© platform. 
3.3 
Study Measures 
AIM 1 Pilot study: 
• 
E-triage: The e-triage will consist of 16 questions, modified from the validated NCI Pro-CTCAETM, 
which will be sent to patients via WaytoHealth©’s two-way texting system 96 hours prior to their 
scheduled immunotherapy infusion (see appendix B). Questions will pertain to common or 
emergent immune related adverse events as defined by the NCCN guidelines and two senior 
disease experts42–44.Patients will be prompted via text to measure the presence and severity of 
symptoms over the week prior. A final question will be included to capture any additional 
symptoms patients wish to disclose. 
• 
NCI Pro-CTCAETM: A subset of patients may be asked to fill out a subset of the NCI PRO-
CTCAETM, a set of validated questions for the collection of patient-reported outcomes to capture 
symptomatic adverse events in patients on cancer clinical trials, in order assess construct validity.  
• 
Definition of gold standard: Patient reported toxicity captured by the text-based instrument will 
be compared to provider reported toxicity as documented in the EMR as the gold standard. To 
assess provider reported toxicity, standardized abstraction protocols will be developed by the PI 
and implemented by trained research staff.   
 
AIM 2 Randomized control study: 
• 
Care time: Care time will be defined as total time spent commuting to and receiving cancer care 
(including time spent obtaining labs). This will be captured using a combination of time-stamp 
information abstracted from AirFinder©, a real time patient located system deployed in the 
Perelman Cancer Clinics, and self-reported time for commuting and time spent obtaining local 
labs. Commute times will also be estimated using the patient’s home address geocoded using 
ArcGIS software. 
• 
Emergency department and hospitalization rates: Total number of emergency department 
visits and hospitalizations will be collected using data abstracted from the Penn electronic 
medical record during a 3 month follow-up window as well as through patient check-ins with study 
staff during routine infusion visits to account for care received outside the Penn system. 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
• 
Health related quality of life : The Functional Assessment of Cancer Therapy-General (FACT-
G) 27-item questionnaire will be used to measure four main domains of health related quality of 
life in cancer patients including physical, social, emotional and functional well-being4. The FACT-
G takes approximately 5-10 minutes to complete. This will be completed at the time of enrollment 
and during each infusion visit.  
• 
Patient satisfaction: The Patient Satisfaction Questionnaire-18 will be used to assess global 
satisfaction with medical care including six aspects of care: technical quality, interpersonal 
manner, communication, financial aspects of care, time spent with doctor, accessibility of care1. 
The PS8-18 takes approximately 3-4 minutes to complete. This will be completed at the time of 
enrollment and during each infusion visit. 
• 
Provider satisfaction: The net-promoter score is a gauge of participant satisfaction taken by 
asking how likely they would be to recommend your service or program on a scale of 0-102,3. 
These will be completed at the time of study completion.  
3.4 
Study Endpoints  
3.4.1 Primary Study Endpoint 
AIM 1 Pilot: To assess the performance of the text-based e-triage system sensitivity and specificity with a 
gold standard will be determined.  
 
AIM 2 Randomized control trial: The primary endpoint will be cumulative care time associated with 
cancer treatment, measured over a over three months or at least 3 infusion visits. We will define care time 
as total time spent commuting to and receiving care (including time spent obtaining labs).  
3.4.2 Secondary Study Endpoints 
AIM 2 Randomized control trial: Secondary endpoints will include total wait time (see attached study 
flowchart), total number of ED visits and/or hospitalizations, health related quality of life utilizing the 
FACT-G4, treatment satisfaction with the PSQ-181, and providers satisfaction using the net-promoter 
score2,3.  
4 
Study Population and Duration of Participation  
4.1 
Duration of Study Participation  
AIM 1 Pilot study: The duration of study participation will be 2 months from the date of enrollment.  
 
AIM 2 Randomized control study: The duration of study participation will be the time it takes to 
complete 3 infusion visits, approximately 3 months..  
4.2 
Total Number of Subjects and Sites  
Penn (Perelman Center for Advanced Medicine floors 2,3,4) will be the only site recruiting patients. For 
the pilot study we will enroll up to 100 patients over a 2-months. For the randomized control trial, to have 
160 evaluable subjects, with an expected 10% drop out rate, we plan to enroll 176 patients.  
4.3 
Inclusion Criteria 
AIM 1: Pilot participants: 
• 
Over 18 years of age 
• 
Access to mobile device with text messaging capabilities 
• 
Receiving single agent PDL-1/PD-1 targeted immune checkpoint blockade for any solid 
malignancy  
• 
Treating oncologist is at Penn’s Abramson Cancer Center 
 
AIM 2: Randomized control trial participants: 
• 
Over 18 years of age 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
• 
Receiving single agent PDL-1/PD-1 targeted immune checkpoint blockade for any solid 
malignancy  
• 
Access to and ability to use a mobile phone with texting capabilities 
4.4 
Exclusion Criteria 
• 
Non-English speaking  
• 
Unable to perform informed consent 
4.5 
Subject Recruitment  
A weekly screening list of potentially eligible subjects scheduled to received PD-L1/PD-1 pathway 
inhibitors at one of the Perelman infusion suites will be generated using EPIC. Trained research staff will 
then screen for eligible patients using strict inclusion and exclusion criteria (see 4.3) and contact treating 
physicians via email to request for permission to approach the patient at the time of their first 
immunotherapy infusion. Trained members of the research team will then approach the patient in-person 
during their infusion visit to assess interest and complete informed consent. If patients consent, they will 
be enrolled onto the study, but not randomized until the patients respond to an additional screening 
question sent via text 5 days following consent to ensure patient’s engagement with the texting platform. 
Our recruitment strategy will be tailored as necessary under the advisement of Dr. Carmen Guerra, the 
Chair of Diversity and Inclusion, to minimize barriers to accrual for underrepresented groups.  
4.6 
Vulnerable Populations:  
Children, pregnant women, fetuses, neonates, or prisoners are not included in this research study.  
5 
Study Procedures  
5.1 
Screening  
Patients will be offered the option to complete the initial educational visit either in-person or over a video-
communication platform. At this visit patients will be enrolled onto the WaytoHealth© text messaging 
system and given instruction about the interphase. Trial protocol will be discussed in detail and patients 
will be provided with educational materials and study contact information. For patients on the randomized 
control trial baseline questionnaires (FACT-G and PSQ-18) will be completed.   
5.2 
Study timeline 
5.2.1 Pilot 
During the pilot, consented patients will be enrolled onto the Waytohealth© texting system and sent the 
text-based e-triage 4 days prior to their infusion. They will otherwise participate per their usual oncology 
care, which includes an in-person office visit. Following this visit, trained research staff will abstract 
relevant clinical information related to treatment toxicity as documented in the progress note from the 
electronic medical record.  
5.2.2 Randomized control trial  
For patients in the intervention arm, symptoms (using the text-based e-triage) and laboratory results (see 
appendix B) will be assessed 96 hours prior to the intended infusion date. A standardized algorithm to 
assess each laboratory value and clinical question will be used to triage patients. Those with normal 
laboratory tests and no flagged symptoms will proceed to their infusion without an office visit, those with 
either abnormal laboratory tests or any flagged symptoms will proceed to an in-person provider evaluation 
prior to their infusion (see appendix A).  Providers and patients will be offered the option to override the 
system and request in-person evaluation. Patients in the control group will be assessed with standard in-
person assessment. Patient time will be collected using an in-house real time tracking system via 
Bluetooth technology. There will be no additional visits required outside of those necessary for routine 
cancer care.  

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
 
Patients will be monitored for effects on health-related quality of life and treatment satisfaction at the time 
of consent and during their infusion for each cycle for a total of 3 cycles of therapy. Data will be collected 
during this time given that majority of immune mediated toxicity occurs prior to this timepoint44,46,47. 
5.2.3 End of Study Visit 
The last visit will be during their final infusion during the follow-up window, at which time they will 
complete the FACT-G and PSQ-18 questionnaires as well as the opportunity to write an open-ended 
response regarding their experience with their program. At the same time, their physician will be asked to 
answer how likely they would be to recommend this program to a colleague on a scale of 1-10 through 
either email or EPIC inbox message according to physician preference.  
5.3 
Unscheduled Visits 
All unscheduled visits will be included in care-time although no study questionnaires will be administered 
during visits not associated with immunotherapy infusions.  
5.4 
Subject Withdrawal  
Subjects may withdraw from the study at any time without impact to their care. The investigator may 
withdraw subjects who violate the study plan, to protect the subject for reasons related to safety i.e. 
patients ability to safely and adequately execute the e-triage. It will be documented whether each subject 
completes the study. Subjects who withdraw early will have a final visit to collect final questionnaires and 
assess adverse events.  
5.4.1 Data Collection and Follow-up for Withdrawn Subjects 
Table 1. Interim pilot results  
Question 
Prevalence  Sensitivity 
Fever 
0.021 
1.00 (0.03-1.00) 
Fatigue 
0.426 
1.00 (0.83-1.00) 
Pain 
0.163 
0.43 (0.10-0.82) 
Rash 
0.071 
0.33 (0.01-0.91) 
Itching 
0.071 
1.00 (0.30-1.00) 
Dizziness 
0.024 
1.00 (0.03-1.00) 
Neuropathy 
0.00 
 
Weakness 
0.024 
1.00 (0.03-1.00) 
Vision change 0.049 
1.00 (1.16-1.00) 
Headache 
0.024 
1.00 (0.03-1.00) 
Shortness of 
breath 
0.024 
1.00 (0.03-1.00) 
Cough 
0.049 
0.50 (0.01-0.99) 
Nausea 
0.125 
1.00 (0.48-1.00) 
Vomiting 
0 
 
Diarrhea 
0.125 
1.00 (0.48-1.00) 
Constipation 
0.025 
1.00 (0.30-1.00) 
Loss of 
appetite 
0.075 
1.00 (0.30-1.00) 
Summary 
0.532 
1.00 (0.863-

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
Even though subjects may be withdrawn prematurely 
from the study, it is critical that we understand potential 
reasons for subject withdrawal which could indicate issues regarding the intervention’s feasibility or 
acceptability for future work. Therefore, participants who have withdrawn consent will be seen for one 
final study visit during which time they will be asked to complete final study questionnaires as well as an 
open-ended question regarding reasons for study discontinuation.  
6 
Statistical Plan 
6.1 
Sample Size and Power Determination 
AIM 1 Pilot study: This protocol describes a cross-sectional study designed to investigate the operating 
characteristics of a text-based e-triage to capture patients experiencing immune-related adverse events 
against the gold standard of documented toxicity assessments in the EMR. Due to the granular nature of 
our e-triage questionnaire we believe a sensitivity of 95-100% is very reasonable to anticipate.   
 
Results from the interim analysis including 48 patients are shown in Table 1. Median age was 68.5 ( IQR 
58-72), 58% were female, 88% white, and most patients either had thoracic (25%), genitourinary (27%), 
head/neck (15%), or skin (15%) cancers. 53.2% of patients had at least 1 toxicity related adverse event 
documented in the EMR. Sensitivity to identify at least one toxicity related adverse event was 100% (95% 
CI 0.863-1.00) aligning with our primary endpoint.  Despite being at our target sensitivity we identified two 
underperforming questions (pain and rash) and therefore optimized the instrument creating a version 2 
(see appendix). Assuming the sensitivity and prevalence are same as those in the preliminary data, a 
sample of 47 patients will produce adequate precision for the estimate for sensitivity, with the half width of 
the 95% confidence interval no larger than 7%. 
 
AIM 2 Randomized control trial: We plan to enroll 176 patients and expect to have 160 patients left 
assuming a 10% drop out rate (e.g., due to disease progression and need to change therapy). Based on 
the preliminary data recently published by this group measuring total care time of patients treated at the 
Penn Abramson Cancer Center8, we expect the total care time in the control group to have a mean of 130 
hours with a SD of 118. A sample size of 80 per group will provide 79% power to detect a 40% reduction 
in the total care time (130 vs. 78) assuming a type I error rate of 0.05.  
 
6.2 
Statistical Methods 
Aim 1 Pilot study: Sensitivity and specificity will be calculated along with 
exact 95% confidence intervals (CIs). Sensitivity will be calculated as the 
proportion of patients with toxicity captured by the text-based e-triage 
among patients with toxicity as documented by the provider in the EMR. 
Specificity will be calculated as the proportion of negative e-triage 
screens among patients without symptoms as documented by the 
provider in the EMR. We will additionally be performing a series of 
psychometric analyses including response distribution to assure 
responses are not skewed, item-scale correlation, internal consistency, 
floor and ceiling effects and item reduction as well as response rates.  
 
AIM 2 Randomized control study: Standard descriptive statistics will be 
calculated to compare patient characteristics for the intervention arm versus the control arm, such as 
medians and interquartile ranges for continuous variables and frequencies and proportions for categorical 
variables. These characteristics will be compared between the two groups using the Wilcoxon rank sum 
test for continuous variables and the chi squared or Fisher’s exact test for categorical variables. We 
expect these characteristics will be similarly distributed between the intervention groups due to 
randomization, but in the case of imbalance, they may be adjusted in the multivariable model of the 
outcome.  
 
The primary analyses will be Intent-to-Treat, with the primary outcome (total care time) compared using a 
score 
1.00) 
Table 2. Power to detect 
reduction in total care 
time (N=160) 
Reduction in 
care time 
Power 
35% 
68% 
40% 
79% 
45% 
88% 
50% 
93% 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
two-sample t test, or Wilcoxon test if normal distribution assumption is violated. A multivariate linear 
regression will be used to adjust potential confounders including demographics (age, gender, race, SES), 
comorbidities (Elixhauser comorbidity index), treatment intent (curative versus palliative), and 
performance status. The final model will be established using stepwise forward regression based on 10% 
change in effect estimate for intervention. As a secondary analysis, “as treated” analyses will be 
performed to account for permissive non-adherence to the triage assignment. Standard descriptive 
statistics will be used to describe patient factors associated with more frequent triage non-adherence.  
 
Secondary endpoints include: total wait time measured, total number of hospitalization/emergency 
department encounters, patient satisfaction [The Patient Satisfaction Questionnaire Short Form (PSQ-
18)]1, provider satisfaction (Net-Promoter Score)2,3, and health related quality of life [Functional 
Assessment of Cancer Therapy-General (FACT-G)4 ]. Total wait time and number of hospitalizations will 
be compared between groups using Wilcoxon rank sum test. The rest of the secondary endpoints will be 
measured at multiple time points and longitudinal linear mixed-effects model will be used to compare the 
group difference over time. Additional secondary analyses exploring changes in visit time across the 
duration of follow-up will also be performed.  
6.3 
Control of Bias and Confounding  
Patients will be randomized 1:1 to control or intervention arm of the study. Despite the benefits of 
randomization to minimize the risks of confounding we will additionally perform a multivariate linear 
regression to adjust potential confounders including demographics as describe in section 6.2.  
6.3.1 Baseline Data 
Standard descriptive statistics will be calculated to compare patient characteristics for the intervention 
arm versus the control arm, such as medians and interquartile ranges for continuous variables and 
frequencies and proportions for categorical variables. These characteristics will be compared between the 
two groups using the Wilcoxon rank sum test for continuous variables and the chi squared or Fisher’s 
exact test for categorical variables. We expect these characteristics will be similarly distributed between 
the intervention groups due to randomization, but in the case of imbalance, they may be adjusted in the 
multivariable model of the outcome. 
6.3.2 Analysis of Primary Outcome of Interest 
The primary analyses will be Intent-to-Treat, with the primary outcome (total care time measured over 
three months) compared using a two-sample t test, or Wilcoxon test if normal distribution assumption is 
violated (see section 6.2).  
7 
Safety and Adverse Events 
7.1 
Definitions 
7.1.1 Adverse Event 
An adverse event (AE) is any symptom, sign, illness or experience that develops or worsens in severity 
during the course of the study.  Intercurrent illnesses or injuries should be regarded as adverse events.  
Abnormal results of diagnostic procedures are considered to be adverse events if the abnormality: 
• 
results in study withdrawal 
• 
is associated with a serious adverse event 
• 
is associated with identification clinical signs or symptoms      
o 
This will be defined as immune related adverse events, per the NCCN guidelines 
regarding management of immune related adverse events44, that result in 
treatment holds or steroid initiation. 
o 
Delayed identification of immune related adverse events, defined by symptoms 
or lab abnormalities that preceded a treatment infusion that was missed by e-
triage or an office visit, will be documented separately. 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
• 
leads to additional treatment or to further diagnostic tests 
• 
is considered by the investigator to be of clinical significance 
 
The objective of the pilot study is to optimize the questionnaire to quickly identify cancer related 
symptoms and immune related adverse events.  
7.1.2 Serious Adverse Event 
Serious Adverse Event 
Adverse events are classified as serious or non-serious.  A serious adverse event is any AE that is:  
• 
fatal 
• 
life-threatening 
• 
requires or prolongs hospital stay 
• 
results in persistent or significant disability or incapacity 
• 
required intervention to prevent permanent impairment or damage    
• 
an important medical event 
 
Important medical events are those that may not be immediately life threatening, but are clearly of major 
clinical significance.   They may jeopardize the subject, and may require intervention to prevent one of the 
other serious outcomes noted above.  All adverse events that do not meet any of the criteria for serious 
will be regarded as non-serious adverse events.  
 
7.2 
Recording of Adverse Events 
Trained research staff will be meeting with patients during each scheduled infusion visit. At each contact, 
they will seek information on adverse events by specific questioning and, as appropriate, by examination.  
Information on all adverse events will be recorded immediately in the source document, and also in the 
appropriate adverse event module of the case report form (CRF).  All clearly related signs, symptoms, 
and abnormal diagnostic procedures results should be recorded in the source document, though should 
be grouped under one diagnosis. 
 
All adverse events occurring during the study period will be recorded.  The clinical course of each event 
will be followed until resolution, stabilization, or until it has been determined that the study intervention or 
participation is not the cause.  Serious adverse events that are still ongoing at the end of the study period 
will be followed up to determine the final outcome.  Any serious adverse event that occurs after the study 
period and is considered to be possibly related to the study intervention or study participation will be 
recorded and reported. 
 
7.3 
Relationship of AE to Study  
Each adverse event to the study procedures will be characterized by the PI and classified as either 
definitely related, probably related, possibly related, unlikely, or unrelated based on documentation from 
the electronic medical record and as necessary discussions with the patient’s medical provider.  
7.4 
Reporting of Adverse Events and Unanticipated Problems 
The Investigator will promptly notify the Penn IRB of all on-site unanticipated, Adverse Events that are 
related to the research activity. Other unanticipated problems related to the research involving risk to 
subjects or others will also be reported promptly. Written reports will be filed using the HS-ERA and in 
accordance with the Penn IRB timeline of 10 working days.  
7.4.1 Follow-up Report 
If an AE has not resolved at the time of the initial report and new information arises that changes the 
investigator’s assessment of the event, a follow-up report including all relevant new or reassessed 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
information (e.g., concomitant medication, medical history) should be submitted to the IRB. The 
investigator is responsible for ensuring that all SAEs are followed until either resolved or stable.  
 
7.4.2 Data and Safety Monitoring Plan 
Overall data and safety will be monitored on an ongoing basis by the PIs and the study team. As needed, 
the investigators may decide to appoint an independent medical monitor to evaluate adverse events and 
make recommendations for continuing or stopping a trial. No interim analysis will be conducted.  
 
Monitoring e-triage: All Waytohealth© interactions will be input into a flowsheet embedded into the 
patient’s EMR and available to all study and clinical staff. All relevant persons will be notified via an EMR 
alert when any new information is entered. Dedicated research staff will be monitoring for inputs daily and 
will interphase with the research PI as well as clinical team as needed for unanticipated responses or 
symptoms. Any identified symptom will require the patient to have an in-person assessment with their  
routine clinical team. Clinical staff will be notified of the patient’s triage status and offered the opportunity 
to mandate an in-person assessment regardless of the patient’s triage status. A patient’s enrollment in 
the study will be terminated if they experience an emergency medical situation requiring intervention or 
violate study protocol. 
 
Monitoring clinical events: Research staff will see patients at each of their infusion visits. During this time, 
they will assess for any unidentified immunotherapy adverse events as well as screen for any emergency 
department visits or hospitalizations which occurred outside the Penn system. Additionally, research staff 
will receive EPIC alerts corresponding to any emergency department visits or hospitalizations.  
8 
Study Administration, Data Handling and Record Keeping 
8.1 
Confidentiality 
Information about study subjects will be kept confidential and managed according to the requirements of 
the Health Insurance Portability and Accountability Act of 1996 (HIPAA).  Name, date of birth, and 
medical record number will be the only protected health information collected during this study. All PHI 
within the secure research database will be encrypted. All study participants will be assigned a unique 
study ID.  All study information will be stored in locked file cabinets and in password-protected computer 
files.  Only authorized study personnel will have access to these files.   
 
The database (RedCap) will used to track patient information will be encrypted to hide patient identifiers 
(such as name, address, birth date and phone number), allowing for the conducting of research without 
endangering subjects personal privacy.  
 
All subjects will be enrolled on WaytoHealth©. The e-triage will be executed using WaytoHealth©’s 
platform which uses a role-based access control (RBAC) approach to assure that participant 
confidentially and study integrity is preserved. Access and visibility is primary governed by the role of the 
individual accessing the system. Access is granted by invitation only and can be revoked at any time. All 
WaytoHealth© servers are managed by Penn Medicine Academic Compute Services (PMACs). All data 
at-rest is stored on encrypted disks using encryption keys managed by WaytoHealth©. Encrypted disks 
use AES encryption with a minimum of 256-bit keys, or keys and ciphers of equivalent or higher 
cryptographic strength. User passwords are never stored in clear text; they are “salted” and “hashed” to 
eliminate data leakage. All data transmission is encrypted end to end using encryption keys managed by 
WaytoHealth©. Transmission encryption keys use a minimum of 2048-bit RSA keys, or keys and ciphers 
of equivalent or higher cryptographic strength (e.g., 256-bit AES session keys in the case of IPsec 
encryption). Data downloads are generally prohibited by policy. Where appropriate, most datasets are 
blinded of all personally identifiable information when exported for analysis. A limited number of exports 
including identifiers exist to assist research staff with recruitment tracking and study management efforts. 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
These datasets are only accessible to certain user roles. These user roles are required to sign and 
adhere to a W2H Security Agreement as described in appendix C.   
In the event that a patient revokes authorization to collect or use PHI (which will be encrypted from the 
specimen(s) and other health data), the Investigator, by regulation, retains the ability to use all information 
collected prior to the revocation of patient authorization.  For patients that have revoked authorization to 
collect or use PHI, attempts should be made to obtain permission to collect at least vital status (i.e. that 
the patient is alive). This data will not become part of the study subjects’ permanent record, such as 
employment or academic record.  
 
8.2 
Data Collection and Management  
All study participants will be assigned a unique study ID. The Research Electronic Data Capture 
(REDCap) system will be used to autogenerate the participant identifier and track patient information. 
Data is encrypted to hide patient identifies (such as name, address, birth date, and phone number), 
allowing for the conducting of research without endangering subject’s personal privacy. All clinical data is 
entered onto a web-enabled case report form using the REDCAP platform. All patient data are “de-
identified” such that each subject record will be labeled with only a study ID (no MRN, names, account 
numbers will be recorded). All clinical data will subsequently be entered into a REDCap clinical database.  
 
All study participants will be enrolled on the Waytohealth platform where the e-triage instrument will be 
executed. All data at-rest is stored on encrypted disks using encryption keys managed by WaytoHealth©. 
Encrypted disks use AES encryption with a minimum of 256-bit keys, or keys and ciphers of equivalent or 
higher cryptographic strength. User passwords are never stored in clear text; they are “salted” and 
“hashed” to eliminate data leakage (See appendix C and section 8.1 for more on the data protections). 
 
Patients will be tracked using blue-tooth technology while in the cancer clinic. The real-time tracking 
system will be executed using Link Lab’s Air finder system. Only the patients deidentified research study 
ID will be used to link the patient and their tracked health-care time. Access to this information is visible 
only by the research team and will be stored on encrypted disk.  
 
 
8.3 
Records Retention 
Records and documents pertaining to the conduct of this study, including signed consent forms, must be 
retained by the Investigator for the maximum period required by applicable regulations of relevant 
national or local health authorities.  
9 
Study Monitoring, Auditing, and Inspecting 
9.1 
Study Monitoring Plan 
Overall data and safety will be monitored on an ongoing basis by the PIs and the study team. As needed, 
the investigators may decide to appoint an independent medical monitor to evaluate adverse events and 
make recommendations for continuing or stopping a trial. No interim analysis will be conducted.  
9.2 
Auditing and Inspecting 
Participation as an investigator in this study implies acceptance of potential inspection by government 
regulatory authorities and applicable University compliance and quality assurance offices. 
 
The investigator will permit study-related monitoring, audits, and inspections by the EC/IRB, the sponsor, 
government regulatory bodies, and University compliance and quality assurance groups of all study 
related documents (e.g. source documents, regulatory documents, data collection instruments, study data 
etc.).  The investigator will ensure the capability for inspections of applicable study-related facilities (e.g. 
pharmacy, diagnostic laboratory, etc.). 
 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
Participation as an investigator in this study implies acceptance of potential inspection by government 
regulatory authorities and applicable University compliance and quality assurance offices. 
 
10 Ethical Considerations 
10.1 
Risks 
 
AIM 1 Pilot study: Potential study risks are expected to be minimal. There is a minor risk of loss of 
confidentiality and privacy. The research team will take the necessary precautions to ensure that 
confidentiality and privacy are maintained. We will use commercial-grade encryption to protect 
participating information similar to that which is used to protect electronic medical records. Personal 
information will be used only by study team members who have been trained to use secure protocols to 
maintain the privacy of your data. Whenever possible, data will be de-identified to protect privacy. We will 
link individual identifying information with participant ID numbers in one secure file that will only be 
accessed by the study team in the case of an adverse medical event, participant dropout, or if otherwise 
deemed necessary by the Principal Investigator. All other identifying information will be discarded after 
initial contact with the research coordinator. Due to the fact, all patients will be seeing their provider with 
standard laboratory testing within 4 days of the questionnaire we feel that there is minimal physical risk to 
patients.  
 
AIM 2 Randomized control trial: As above, there is a minimal risk of loss of confidentiality and privacy. 
The second potential risk is that participants misinterpret the e-triage as a means of quick communication 
with their care team. We will take great care to emphasize that the e-triage system is investigational, and 
not a replacement for usual means of communication with one’s care team. Participants will be reminded 
repeatedly  both verbally and via text after each questionnaire that they should contact their care team 
directly if they are having any symptoms for which they think urgent medical attention is warranted. There 
is also a risk of participants inappropriately, or incompletely, filing out the e-triage questionnaire. We hope 
to minimize this risk via the execution of the pilot study. Patients who do not fill out the questionnaire 
completely will therefore not be eligible to be fast-tracked to avoid an incomplete symptom assessment. 
Patients will be screened, per routine standard of care, by laboratory tests and vital sign assessments. 
Research staff will see patients at each infusion as well for toxicity assessments. Finally, patients or 
providers will be offered the opportunity to override the system and request in-person assessment.    
10.2 
Benefits 
AIM 1 Pilot study: The benefit to participating in the pilot study is the advancement of knowledge 
regarding safe health-care delivery strategies that optimize the patient experience receiving cancer care.  
 
AIM 2 Randomized control trial: The benefit of participating includes the potential to minimize the time 
burden associated with routine cancer treatment. For patients who are triaged to receive treatment earlier 
drug can be made in advance and patients will have the ability to optimize their infusion appointment 
timing to increase efficiency. The opportunity for participants to have more options for how they interact 
with the healthcare system, i.e., the choice whether to see a provider if they have normal laboratory 
testing and no treatment or cancer related symptomatology. The ability to identify treatment or cancer 
related adverse events earlier which could allow for earlier cancelations of unsafe infusion visits or the 
option to intervene and optimize patients for treatment earlier. Participants will be participating in a study 
designed to innovate cancer care delivery, ideally improving patient-centered care.   
10.3 
Risk Benefit Assessment 
Although there are potential risks associated with this study, the current model of cancer care delivery is 
inefficient resulting in long wait times and care delays, all of which is likely to worsen as the cancer 
population continues to grow. We have incorporated protocols to mitigate potential risks including 
integrating the e-triage into the EMR for easy access by clinician and research staff, the execution of a 
dedicated pilot study to ensure safety, and the fact the instrument is designed to prompt a formal clinical 
evaluation for any level of report toxicity. Additionally, the potential benefits of this study are great and 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
include the ability to streamline care, improve access, and increase the patient-centeredness of cancer 
care delivery by allowing patients the option whether or not to see their provider if they are clinically well. 
If proven successful, this study could have a significant impact on oncology care across health systems.  
10.4 
Informed Consent Process / HIPAA Authorization  
All subjects for this study will be provided a consent form describing this study providing sufficient 
information for subjects to make an informed decision about their participation in this study.  See 
appendix D for a copy of the Subject Informed Consent Form.  This consent form will be submitted with 
the protocol for review and approval by the IRB for the study.  The formal consent of a subject, using the 
IRB-approved consent form, must be obtained before that subject undergoes any study procedure.  The 
subject must sign the consent form, and the investigator-designated research professional obtaining the 
consent. Subjects will be consented by the study Principal Investigator, or appropriate designee, in a 
room we have selected in which to perform consent, which is located outside of the clinic. Potential 
subjects will review the consent form in detail with the person designated to consent (either PI or CRC) 
and have the ability to take the consent home for further review.  
 
Due to the COVID-19 pandemic, when in-person consent is deemed impossible consent will be 
completed remotely via telephone or video, by a member of our research team. They will explain the 
nature of the study to the participant or his/her legally authorized representative and answer all questions 
regarding the study. Participants will be informed that their participation is voluntary. A designated 
member of the research team will explain to each participant the objectives, methods, and potential risks 
associated with the study. Participants will be notified that they can withdraw from the study at any time 
without penalty. Patients or their legally authorized representative will be required to remotely sign or 
verbally affirm a statement of informed consent that meets the requirements of 21 CFR 50, local 
regulations, ICH guidelines, Health Insurance Portability and Accountability Act (HIPAA) requirements, 
and the IRB requirements. A copy of the informed consent will be emailed to the participant, or their 
legally authorized representative and one will be maintained in the participant’s research file.  
  
In cases where e-consent and a paper or electronic signature is not possible, verbal consent will be 
obtained. Documentation of the method used for communication, means by which agreement was 
communicated, and documentation that no imaging technology was available to capture a signed 
informed consent form will be maintained in the participant’s research file. A signed copy will be obtained 
at the earliest convenience when the participant presents to the infusion clinic. See appendix E for the 
telephone script for the telephone consent.  
 
10.4.1 Alterations to Typical Consent Process  
10.4.1.1 Waiver of Written Documentation of Consent  
A waiver of written documentation of consent is requested for participation in the pilot study. Patients will 
be provided language regarding the research objectives as well as the risk of breach of confidentiality 
(see Appendix F for the waiver, G for patient information, and H for script). This study’s primary aim is to 
assess the safety of the e-triage instrument and participants will otherwise receive standard care per their 
clinician. The research involves no more than minimal risk and will not adversely affect the rights and 
welfare to the subjects since it will only entail completing a questionnaire. See appendix F for the waiver 
request.  
10.4.1.2 Waiver of Written Documentation of HIPAA Authorization  
A waiver of written documentation of HIPPA Authorization is requested for participation in the pilot study. 
Patients will be provided language regarding the research objectives as well as the risk of breach of 
confidentiality (see Appendix F for the waiver, G for patient information, and H for script). This study’s 
primary aim is to assess the safety of the e-triage instrument and participants will otherwise receive 
standard care per their clinician. The research involves no more than minimal risk and will not adversely 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
affect the rights and welfare to the subjects since it will only entail completing a questionnaire. For the 
pilot study we are trying to recruit a large number of subjects in a small period of time therefore 
administering formal informed consent and HIPPA authorization will not be logistically feasible. See 
appendix F for waiver request.  
 
11 Study Finances 
11.1 
Funding Source 
This study will be funded through an American Society of Clinical Oncology Conquer Cancer Young 
Investigator award as well as a Marjorie and Bryan Weingarten Fellowship Grant.  
11.2 
Conflict of Interest 
All University of Pennsylvania Investigators will follow the University of Pennsylvania Policy on Conflicts of 
Interest Related to Research.  
12 Publication Plan 
The results of this study may be reported to the public, in the form of a publication or presentation at 
scientific congresses, before completion of the study. 
 
13 References 
1. 
Thayaparan, A. J. & Mahdi, E. The Patient Satisfaction Questionnaire Short Form (PSQ-18) as an 
adaptable, reliable, and validated tool for use in various settings. Med. Educ. Online 18, 21747 
(2013). 
2. 
M, W., S, P., H, L., A, W. & D, C. Measuring the patient experience in community mental health 
services for older people: A study of the Net Promoter Score using the Friends and Family Test in 
England. Int J Geriatr Psychiatry 34, 31–37 (2019). 
3. 
R, K., G, F., K, G. & H, M. The Net Promoter Score (NPS) for Insight Into Client Experiences in 
Sexual and Reproductive Health Clinics. Glob Health Sci Pr. 6, 413–424 (2018). 
4. 
Cella, D. F. et al. The Functional Assessment of Cancer Therapy scale: development and validation 
of the general measure. J. Clin. Oncol. 11, 570–579 (1993). 
5. 
Cheng, A. C. & Levy, M. A. Data Driven Approach to Burden of Treatment Measurement: A Study of 
Patients with Breast Cancer. AMIA Annu. Symp. Proc. AMIA Symp. 2016, 1756–1763 (2017). 
6. 
Cheng, A. C. & Levy, M. A. Measures of Treatment Workload for Patients With Breast Cancer. JCO 
Clin. Cancer Inform. 1–10 (2019) doi:10.1200/CCI.18.00122. 
7. 
Presley, C. J. et al. Treatment Burden of Medicare Beneficiaries With Stage I Non–Small-Cell Lung 
Cancer. J. Oncol. Pract. 13, e98–e107 (2016). 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
8. 
Bange, E. M. et al. Opportunity Costs of Receiving Palliative Chemotherapy for Metastatic Pancreatic 
Ductal Adenocarcinoma. JCO Oncol. Pract. 16, e678–e687 (2020). 
9. 
Yabroff, K. R. et al. Patient Time Costs Associated With Cancer Care. JNCI J. Natl. Cancer Inst. 99, 
14–23 (2007). 
10. Hamel, L. M. et al. Measuring the Use of Examination Room Time in Oncology Clinics: A Novel 
Approach to Assessing Clinic Efficiency and Patient Flow. J. Oncol. Pract. 10, e385–e389 (2014). 
11. Rocque, G. B. et al. Health care‐related time costs in patients with metastatic breast cancer. Cancer 
Med. 9, 8423–8431 (2020). 
12. Shulman, L. N., Sheldon, L. K. & Benz, E. J. The Future of Cancer Care in the United States—
Overcoming Workforce Capacity Limitations. JAMA Oncol. 6, 327 (2020). 
13. Kirkwood, M. K. et al. The State of Oncology Practice in America, 2018: Results of the ASCO 
Practice Census Survey. J. Oncol. Pract. 14, e412–e420 (2018). 
14. Yang, W. et al. Projected Supply of and Demand for Oncologists and Radiation Oncologists Through 
2025: An Aging, Better-Insured Population Will Result in Shortage. J. Oncol. Pract. 10, 39–45 (2014). 
15. Gupta, D. Distribution and determinants of patient satisfaction in oncology: A review of the literature. 
Patient Prefer. Adherence 287 (2009) doi:10.2147/PPA.S6351. 
16. Nguyen, T. V. F. et al. A longitudinal analysis of patient satisfaction with care and quality of life in 
ambulatory oncology based on the OUT-PATSAT35 questionnaire. BMC Cancer 14, (2014). 
17. Azuero, A. et al. An examination of the relationship between patient satisfaction with healthcare and 
quality of life in a geriatric population with cancer in the Southeastern United States. J. Geriatr. Oncol. 
10, 787–791 (2019). 
18. Silver, N. et al. Reducing patient wait times in a head and neck cancer outpatient clinic: A pilot study. 
The Laryngoscope 130, (2020). 
19. Bleustein, C. et al. Wait times, patient satisfaction scores, and the perception of care. Am. J. Manag. 
Care 20, 393–400 (2014). 
20. Crossing the Quality Chasm: A New Health System for the 21st Century. (National Academies Press, 
2001). doi:10.17226/10027. 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
21. Thomas, S., Glynne-Jones, R. & Chait, I. Is it worth the wait? A survey of patients’ satisfaction with 
an oncology outpatient clinic. Eur. J. Cancer Care (Engl.) 6, 50–58 (1997). 
22. Giuseppe, C. The Treatment of Patients With Cancer and Containment of COVID-19: Experiences 
From Italy. ASCO Dly. News (2020). 
23. C, F., A, P., B, M., P, F. & T, H. Managing COVID-19 in the oncology clinic and avoiding the 
distraction effect. Ann Oncol 7534, (2020). 
24. M, U. et al. Managing Cancer Care During the COVID-19 Pandemic: Agility and Collaboration 
Toward a Common Goal. J Natl Compr Canc Netw 1–4 (2020). 
25. Royce, T. J., Sanoff, H. K. & Rewari, A. Telemedicine for Cancer Care in the Time of COVID-19. 
JAMA Oncol. (2020) doi:10.1001/jamaoncol.2020.2684. 
26. Leung, M. S. T., Lin, S. G., Chow, J. & Harky, A. COVID‐19 and Oncology: Service transformation 
during pandemic. Cancer Med. (2020) doi:10.1002/cam4.3384. 
27. Waller, M. & Stotler, C. Telemedicine: a Primer. Curr. Allergy Asthma Rep. 18, (2018). 
28. Jaenicke, C. et al. Use of Mobile Messaging System for Self-Management of Chemotherapy 
Symptoms in Patients with Advanced Cancer. Fed. Pract. Health Care Prof. VA DoD PHS 36, S54–
S57 (2019). 
29. Basch, E. et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer 
Treatment: A Randomized Controlled Trial. J. Clin. Oncol. 34, 557–565 (2016). 
30. Berry, D. L., Blonquist, T. M., Patel, R. A., Halpenny, B. & McReynolds, J. Exposure to a Patient-
Centered, Web-Based Intervention for Managing Cancer Symptom and Quality of Life Issues: Impact 
on Symptom Distress. J. Med. Internet Res. 17, e136 (2015). 
31. Berry, D. L. et al. Enhancing Patient-Provider Communication With the Electronic Self-Report 
Assessment for Cancer: A Randomized Trial. J. Clin. Oncol. 29, 1029–1035 (2011). 
32. Velikova, G. et al. Measuring Quality of Life in Routine Oncology Practice Improves Communication 
and Patient Well-Being: A Randomized Controlled Trial. J. Clin. Oncol. 22, 714–724 (2004). 
33. Basch, E. et al. Long-Term Toxicity Monitoring via Electronic Patient-Reported Outcomes in Patients 
Receiving Chemotherapy. J. Clin. Oncol. 25, 5374–5380 (2007). 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
34. Berry, D. L. et al. Electronic Self-Report Assessment for Cancer and Self-Care Support: Results of a 
Multicenter Randomized Trial. J. Clin. Oncol. 32, 199–205 (2014). 
35. Ebel, M. Perception of cancer patients of their disease, self-efficacy and locus of control and usage of 
complementary and alternative medicine. J. Cancer Res. Clin. Oncol. 141, 1449–1455 (2015). 
36. Kearney, N. et al. Evaluation of a mobile phone-based, advanced symptom management system 
(ASyMS©) in the management of chemotherapy-related toxicity. Support. Care Cancer 17, 437–444 
(2009). 
37. Kotronoulas, G. et al. What Is the Value of the Routine Use of Patient-Reported Outcome Measures 
Toward Improvement of Patient Outcomes, Processes of Care, and Health Service Outcomes in 
Cancer Care? A Systematic Review of Controlled Trials. J. Clin. Oncol. 32, 1480–1501 (2014). 
38. Patel, M. S. et al. Effectiveness of Behaviorally Designed Gamification Interventions With Social 
Incentives for Increasing Physical Activity Among Overweight and Obese Adults Across the United 
States: The STEP UP Randomized Clinical Trial. JAMA Intern. Med. 179, 1624 (2019). 
39. Mehta, S. J. et al. Electronic Pill Bottles or Bidirectional Text Messaging to Improve Hypertension 
Medication Adherence (Way 2 Text): a Randomized Clinical Trial. J. Gen. Intern. Med. 34, 2397–
2404 (2019). 
40. Putt, M. E. et al. The Habit Formation trial of behavioral economic interventions to improve statin use 
and reduce the risk of cardiovascular disease: Rationale, design and methodologies. Clin. Trials 16, 
399–409 (2019). 
41. Mahmud, N. et al. An Automated Text Message Navigation Program Improves the Show Rate for 
Outpatient Colonoscopy. Health Educ. Behav. 46, 942–946 (2019). 
42. Kumar, V. et al. Current Diagnosis and Management of Immune Related Adverse Events (irAEs) 
Induced by Immune Checkpoint Inhibitor Therapy. Front. Pharmacol. 8, (2017). 
43. Gangadhar, T. C. & Vonderheide, R. H. Mitigating the toxic effects of anticancer immunotherapy. Nat. 
Rev. Clin. Oncol. 11, 91–99 (2014). 
44. Management of Immunotherapy-Related Toxicities. NCCN Clin. Pract. Guidel. Oncol. 1.2020, (2020). 
45. Jing, Y. et al. Multi-omics prediction of immune-related adverse events during checkpoint 
immunotherapy. Nat. Commun. 11, 4946 (2020). 

TIME 
Version: 4 
 
CONFIDENTIAL 
 
This material is the property of the University of Pennsylvania.   
 
46. Marrone, K., Ying, W. & Naidoo, J. Immune-Related Adverse Events From Immune Checkpoint 
Inhibitors. Clin. Pharmacol. Ther. 100, 242–251 (2016). 
47. Delanoy, N. et al. Haematological immune-related adverse events induced by anti-PD-1 or anti-PD-
L1 immunotherapy: a descriptive observational study. Lancet Haematol. 6, e48–e57 (2019). 
48. Wang, P.-F. et al. Immune-Related Adverse Events Associated with Anti-PD-1/PD-L1 Treatment for 
Malignancies: A Meta-Analysis. Front. Pharmacol. 8, (2017).