ActivityLink: Clinic-to-Community Physical Activity Referrals for Cardiovascular Disease Risk Reduction in Cancer Survivors.
Phase: NA
Conditions: Cancer, Physical Inactivity
Sponsor: University of Massachusetts, Worcester

Clinic-to-Community Physical Activity Referrals for Cardiovascular Disease Risk Reduction in Cancer 
Survivors. 
May 22nd, 2022 
NCT05216380 
 

3. RESEARCH STRATEGY 
3.1. Significance 
3.1a.  Cardiovascular Disease (CVD) -  the leading cause of death among women.28 CVD affects 47.8 US 
million women annually.28 Though mortality from CVD has been declining overall, the decline has been 
significantly greater in men compared to women.29  Studies show women are less aware of CVD risk than men.30  
Gender disparities are propagated by physicians as well. Physicians are more likely to assign a lower-risk 
category despite a similar calculated risk as compared to men.31 Women are less likely to receive preventive 
treatment or guidance for CVD compared to men with a similar risk.32  
3.1b.   Breast cancer survivors (survivors) are at a higher risk of CVD,33 as much as 30% more than their 
age-matched controls.10 Over 3.8 million U.S. 
women have been diagnosed with breast cancer34 
and an estimated 280,000 new cases will be 
diagnosed in 2020.35 CVD is a leading cause of non-
relapse related mortality in women diagnosed with 
breast cancer.36,37 This has been attributed to two 
main factors: 1) The coinciding risk factors (e.g. 
advanced age, smoking, obesity and physical 
inactivity) associated with CVD and cancer (Figure 
1),8,12 and 2) Survivors undergo toxic treatment that 
places them at an increased risk.9 Racial disparities 
exist in those affected by CVD38 and mortality from 
breast cancer.39 To address these issues, the 
American Heart Association issued the first Scientific 
Statement in 2018 discussing prevention and treatment of CVD and breast cancer.11 The Statement also 
suggests the fields of cardiology and oncology have long been working in silos.11 This project addresses a major 
public health concern in a newly emerging cardio-oncology field receiving little attention to date. 
3.1c. Physical Activity reduces CVD Risk. Over 75% of deaths due to CVD could be prevented with adequate 
lifestyle changes.38 Decades of research show physical activity has a cardioprotective effect on lowering CVD 
risk.40 In addition to treating risk factors,12 physical activity also mediates cardiotoxic effects of cancer 
treatments13,14 and is associated with improved health outcomes after breast cancer diagnosis.41,42   
3.1d. Though safe and effective, few survivors are physically active. Physical activity has been deemed 
safe and efficacious for survivors by the American College of Sports Medicine43 and supported by the American 
Cancer Society.44 Though the direct effect of exercise on CVD prevention in patients with breast cancer has yet 
to be evaluated in long-term randomized trials,11 achieving physical activity guidelines is highly recommended.  
Due to the increased number of survivors, researchers have placed an emphasis on lifestyle interventions, 
including physical activity promotion to improve quality of life and counter disease progression10 and other 
comorbidities.11 However, less than 10% of patients will be physically active during treatment and only 20-30% 
physically active after treatment.15,45  Survivor-reported barriers include program accessibility, affordability12 and 
parking46, while facilitators included convenient and survivor friendly environments.47-49   
3.1e.  The Institute of Medicine has recommended that cancer survivorship care plans be provided to all 
patients and include information on physical activity prescriptions and referrals.16 Receiving a cancer 
diagnosis often presents a teachable time for behavior change, as patients are often motivated to improve their 
prognosis and survival.50,51 Previous evidence50,52 and my prior work (Section 3.3a.iii) shows providers are rarely 
giving survivors physical activity guidance and referrals. Providers often cite difficulty in referring patients to 
available behavior change programs.53 Often, there is much uncertainty around referral pathways, including 
issues surrounding who refers the patient, where they refer them and who performs the follow-up.54 There is a 
critical need for innovative implementation programs  to increase physical activity referrals for these survivors. 
3.1f. Our implementation program will focus on facilitating referrals to LIVESTRONG at the Y – an 
evidence-based community program. LIVESTRONG at the Y is an evidence-based, 12-week community 
physical activity program free of cost for survivors.2-5 It is offered in over 10,000 communities and 42 US states 
at YMCA facilities.55 It has proven efficacious at increasing survivor’s physical activity levels (71% exercising 
at ≥ 150 minutes/week vs 26% of controls; P<.05), improvements in aerobic function (6-minute walk test, group 
difference: 28.9 meters [95% CI, 0.3‐49.0; P = .004])5 and physical function (balance, flexibility and strength).4 
Increases in both activity levels and fitness are related to decreases in one’s CVD risk, thus showing promising 
results for health-related outcomes. Despite this potential for high reach and impact, my preliminary studies show 
that LIVESTRONG programs have trouble enrolling participants, especially from minority and underserved 
groups (Section 3.3a.ii). In order to offer LIVESTRONG, YMCA’s must apply and undergo a 6-month needs 
assessment and training process to ensure they can meet the survivor’s needs.  Community-clinic integrated 
models can address health disparities in chronic disease treatment and care.56  Drawing on components of the 
Chronic Care Model,57 connecting patients to community resources improves management of chronic disease.  

3.1g.  Patient navigators are well-suited to referring survivors to LIVESTRONG but need to be supported. 
Patient Navigators, care coordinators linking patients to resources,58 are well-suited to link patients to community 
physical activity  programs. Navigators may be nurses, social workers or lay health workers who have received 
training in referring patients to programs that fit their logistical, geographic, cultural and economic preferences.7  
Navigators are essential both during and after cancer treatment and play a critical role in overcoming reported 
barriers of fragmented care, poor communication and lack of awareness of late effects and how to maximize 
health outcomes.20 In my preliminary work with navigators (Section 3.3a.iii), they reported wanting more 
information on local lifestyle intervention programs. They also reported there was no formal system for referrals, 
though if they did refer, it was mainly by word of mouth.   
3.1h. Adapting an electronic referral (e-referral) program for referring survivors to LIVESTRONG.  
Evidence has shown that oncologists providing a recommendation + referral compared with a recommendation 
only was associated with increased physical activity program participation in survivors.59-61 However, providers 
lack time and program knowledge for referrals.19 My proposal addresses these barriers through an electronic 
referral.  My mentors’ prior work found that an electronic referral compared to a paper referral was three times 
more effective in patient registration in a web-assisted tobacco intervention (Section 3.3a.iv).22,23 E-referrals allow 
providers to refer and enroll patients in automated programs which can then send messages to the patient while 
providing real-time feedback to the provider.  Using texting will allow for more traditional referral methods (e.g. 
paper, telephone) to be adapted to meet the rapidly evolving everyday use of patient digital technology and 
navigator’s clinical workflow. Since the use of texting is almost ubiquitous including among minority and 
disadvantaged groups,62 it will also increase reach of the referral.  
3.2 Innovation 
1. ActivityLink will be the first comprehensive implementation program developed to support patient 
navigator e-referrals of survivors with CVD risk factors of physical inactivity to a community-based 
program.  Physical activity screening and subsequent referrals as a means of reducing CVD risk factors in 
cancer survivorship is vastly underdeveloped. As noted above, although survivors would greatly benefit from 
physical activity, few are physically active. Providers face many barriers to referring these survivors. I will adapt 
a novel, previously tested e-referral implementation program that was created for referring smokers from 
community medical and dental practices to a web-assisted tobacco intervention. Adaptation is needed to 
increase the efficiency and effectiveness of the program to the targeted population.24 As I adapt the program, I 
anticipate creating additional innovations in each component of the e-referral program (such as the sample 
progress report in Figure 7). ActivityLink uses technology un-tethered to the electronic health record (EHR) 
system, allowing adaptations and increasing potential for dissemination. While the tethered (to the EHR) 
approach has some advantages, it may not be always amenable to adaptations in different settings. Further, it 
is not always accessible by community partners to communicate back to navigators.63 Bi-directional feedback 
with community-based programs can be achieved with an untethered system.   
2. I will use a participatory-stakeholder co-design and iterative approach to develop and beta-test 
ActivityLink, including the use of the PDSA cycle that was originally used by business for the control 
and continuous improvement of processes and products. Designing programs linking clinics and 
communities using technology requires both user and community feedback. As noted in Section 3.3c,  the Clinic 
and Community Advisory Board (CAB) will be an integral part of each aim. Further in Aim 2, we will use the 
PDSA cycle to beta-test ActivityLink in two waves. In the healthcare arena, the PDSA cycle has been used to 
develop an interspecialty inpatient referral system,64 test ideas on a small scale prior to full implementation65 and 
used to implement projects into clinical workflow.66 The conduct of the PDSA cycle while using a participatory-
stakeholder co-design approach67 is another innovation of our project.  Co-design approaches take into account 
the viewpoints of patients, clinicians and stakeholders and actively involve them in all aspects of the design. 
3.3 Approach 
3.3a. Preliminary Data: As a T32 Implementation Science Fellow, I have conducted a series of research 
projects that have informed the proposed K12 project.  In addition, this proposal builds on lessons learned 
from my mentor’s research in e-referrals to smoking cessation programs.  
i. LIVESTRONG has capacity to reach more survivors, though barriers exist. In my first study I 
examined data collected by YMCA-USA for all LIVESTRONG programs between 2010-2018 (manuscript under 
review). I used the RE-AIM framework68 to assess reach, adoption, implementation and variables associated 
with participant-level maintenance, or membership conversion rate. Since 2010, LIVESTRONG has reached 
62,044 survivors and 245 YMCA Associations (29.17%) adopted the program. However, 60,000 is just a fraction 
of the 16.9 million cancer survivors, with that number projected to increase to 20.3 million in the year 2026 69, 
suggesting the need to reach additional survivors. I also found disparities exist. The mean membership 
conversion rate was significantly greater in higher household income areas as compared to lower household 
income areas (48.5 vs. 44.3; p=0.01). Similar disparities in accessing physical activity programs have been 
documented by other researchers.70 These data suggest LIVESTRONG has the capacity to increase its reach 
to more survivors, though disparities exist amongst socioeconomic status levels. 

ii. LIVESTRONG at the Y has difficulty enrolling survivors. In my second study, I performed semi-
structured interviews with LIVESTRONG Program Directors (n=13), Instructors (n=7) and graduates (n=8) from 
9 U.S. states to identify barriers and facilitators to participant enrollment. Barriers: 1) Limited resources to market 
and recruit, 2) Lack of physician knowledge/awareness and lack of formal referral system, 3) Lack of patient 
knowledge about the program, and 4) Lack of patient follow-up on program attendance or activity. Facilitators: 
1) Most programs had the capacity to serve additional survivors, 2) Strong relationships with cancer clinic staff 
and providers, 3) Referrals embedded in electronic medical records, 4) Provider feedback on patient progress 
in the program, and 5) Patient referral and follow-up from cancer clinic staff. These data suggest the need to 
integrate an e-referral system into the clinic to increase patient referrals to LIVESTRONG programs.  
iii. Providers may have limitations to making referrals to physical activity programs, including 
LIVESTRONG. In my 3rd study, I performed interviews with UMass Memorial Health Care (UMMHC) Cancer 
Clinic administrators (n=2), oncologists (n=2), nurses (n=3) and patient navigators (n=8) to understand barriers 
to physical activity education and referrals. They reported lacking knowledge of available physical activity 
programs and/or a lack of standard operating procedures for physical activity referrals (timing in care, type of 
cancer, pre- or post-surgery etc.), having no place to document referrals and no follow-up with patients. All 
feedback suggested patient navigators may be best-suited to educate and refer to programs during their care 
coordination meetings. When discussing integrating a system into the EHR, navigators did not feel strongly about 
this, as EHR presents its own challenges, particularly when working with community-partners. These data 
suggest integration of referrals into standard practice may be best delivered by patient navigators.  
iv. Integrating e-referrals into clinical practice. I will build off my mentors’ prior work testing an e-
referral process (implementation strategy) in community-based medical practices to increase patient access to 
an evidence-based tobacco cessation intervention.22,23,71 Using the Chronic Care Model57, they connected 
medical and dental practices to a community-based tobacco cessation intervention using a secure e-referral 
system. The system used motivational messaging, email reminders, cues and follow-up to increase participation 
and adherence in the intervention (Section 3.3g.). Providers and staff were supported with training, toolkits and 
strategies to deliver referrals, monitor enrollment and provide follow-up. 93% of practices used the e-referral 
system and providers who viewed the practice feedback reports were significantly more likely to refer patients 
compared to those who did not.  The e-referral process resulted in greater smoker referrals (33%) compared to 
the comparison paper referral (11%).22,23 
In conclusion, my preliminary data demonstrates: 1) Survivors face clinic barriers to physical activity 
guidance and referrals, 2) LIVESTRONG programs that are successful in recruiting have referral and feedback 
methods established with providers, and 3) My 
ability to establish relationships and work with 
clinic, community and patient stakeholders, 
making this project feasible.  
3.3b. 
Study 
Overview: 
I 
will 
develop 
ActivityLink, 
an 
adapted 
e-referral 
implementation 
program 
to 
increase 
participation in an evidence-based physical 
activity program (LIVESTRONG at the Y).  
ActivityLink will be adapted to a new setting 
and population from a previously used e-
referral implementation program (Aim 1) and 
then beta tested (Aim 2) to set the stage for a 
future large feasibility pilot trial (Figure 2). In 
Aim 1, I will conduct formative work (1a), 
usability testing with navigators (1b) and 
refinement of ActivityLink to be integrated into 
clinical workflow (1c). Lastly, I will use 2 
process cycles to beta test and refine 
ActivityLink for navigators to deliver to 
survivors with a CVD risk factor of physical 
inactivity (Aim 2). My formative work with  
clinical partners supports the navigator’s 
delivery of the ActivityLink.   
3.3c. The Clinic and Community Advisory 
Board (CAB): First I will finalize the partially 
established CAB. The CAB will bring together 
constituents from the clinic and community to 
provide feedback on the overall project, 

including interview questions, workflow mapping, usability and refinement of the implementation program and 
adaptations during beta testing. The CAB will meet bi-monthly in year 1.  In year 2, we will frontload the 6 
meetings to match to the timing of our PDSA cycles. The CAB will consist of key stakeholders with whom I have 
already established relationships: UMMHC patient navigators, cardiology and oncology providers, YMCA-USA, 
LIVESTRONG staff and participants.  I will include 1 member of the UMMS Center for Clinical and Translational 
Science, Science Participation Recourse Center’s Community Advisory Board will (increase engagement of 
special populations in translational research through tailored, culturally responsive strategies). 
3.3d. Setting: I will conduct the study in the Breast Clinic 
at UMMHC University Campus. The clinic is situated in 
Worcester, MA, with a population of 57.1% White, 20.9% 
Hispanic or Latino, and 11.8% Black or African American. 
The median household income is $45,869; 35.8% of the 
people speak a non-English language. The  clinic employs 
10 patient navigators who we will work with throughout the 
entirety of the study. In 2016 the clinic treated 461 women, 
with ~40% (n=184) of women living <9 miles from the Clinic 
(Figure 3). Breast Clinic data from 2016 showed that ~89% 
of the patients were White, though data on ethnicity and 
socioeconomic status are lacking. I will work with three 
YMCA branches within a 6-mile radius of the clinic: 1) Boroughs Family Branch YMCA (Westborough, MA), 2) 
Central Community Branch YMCA (Worcester, MA), and 3) Greendale YMCA (Worcester, MA). Though the 
Breast Clinic and YMCA sites have lower proportions of minority and  underserved survivors, I chose them for 
convenience for this K12.  I plan to work with more diverse sites in future. 
3.3e. LIVESTRONG at the Y – an evidence-based community-based physical activity program designed 
for survivors: The evidence-based intervention to which navigators will refer patients is LIVESTRONG at the 
Y.72 As previously stated, LIVESTRONG is an evidence-based 12-week physical activity program free of cost to 
survivors. The program has shown increases in physical activity levels, aerobic fitness and physical function.4,5  
Survivors are able to participate at any point following a cancer diagnoses with physicians’ clearance.  Enrolled 
participants will attend 2 sessions/week for 12 weeks. Sessions focus on aerobic, strength, balance and flexibility 
exercises and participants are tested at baseline and again at the end of the 12-week program.  LIVESTRONG 
is offered in over 800 communities across the U.S.  
3.3f. Theoretical Framework: After discussing with 
my 
mentors, 
I 
chose 
the 
Practical, 
Robust 
Implementation and Sustainability Model (PRISM) as 
an overarching framework to guide future larger, multi-
site trials.1 PRISM is an ideal fit because it spans the 
development spectrum – from the planning to the 
implementation 
and 
evaluation 
stages 
of 
the 
program.73 PRISM is an explanatory-prescriptive mode 
and translates research to practice using concepts 
from quality improvement, chronic care, the diffusions 
of innovations and measures of the population-based 
effectiveness of translation.1 Elements include: 1) 
program or intervention, 2) external environment, 3) 
sustainability and infrastructure, and 4) recipients.   It 
focuses on examining key contextual factors at multiple 
levels to determine their effect on RE-AIM68 outcome 
measures.  The PRISM domains will guide my pre-
implementation work in Aim 1 and examination of 
qualitative RE- AIM measures74 in Aim 2.  These 
assessments will guide the refinement of ActivityLink 
for future feasibility testing of the program in my K01 
proposal. 
3.3g. Developing and Beta-testing ActivityLink: As noted, I will use a co-design approach to adapt an existing 
e-referral program for ActivityLink. Below, I will first describe the existing e-referral program followed the 
anticipated adaptations. I will then describe my approach for Aims 1 and 2.  
The Existing e-referral Implementation Program (ActivityLink):  To develop ActivityLink, I will adapt 
an existing e-referral implementation program developed for facilitating referrals of smokers from community 
medical and dental practices to an web-assisted tobacco intervention.22,23 Below, I describe the technology and 
supporting implementation components of the existing e-referral program. I will then discuss anticipated 
adaptations, followed by describing my steps to adapt this program for ActivityLink in the Aims.   
Figure 3. Distance traveled to UMMHC Breast Clinic 
at UMMHC University Campus 
*Data provided from UMMHC from 2016. 
Figure 4. PRISM Framework.1 

Technology-assisted components of the existing referral program included: 
A. The secure Web-form for e-referral where providers could securely refer patients by entering their emails or 
mobile phone numbers into the system if they agree to be referred (Figure 5). Once entered, the system 
sent motivational emails/texts to encourage participation in the intervention.  
B. Messaging templates that providers could customize to send messages to their patients reinforcing their 
participation in the web-assisted tobacco intervention and progress towards their cessation goals.  
C. Feedback: The system provided real-time feedback to providers about smokers use of the web-assisted 
tobacco intervention. This was displayed as part of the practice report summary on the website.  
Additional supporting components included: 
A. Paper prescription form:  In my mentor’s beta testing of the e-referral system,71 they discovered that practices 
simply forgot to refer patients in the Web-based form. Thus, for the main study, they included printed 
information prescription pads as part of the program. Practices used the pad to write the name and email of 
the smokers who wished to be referred. The champions used the pad to refer the smokers on the secure e-
referral Web-form at a convenient time. As noted above, this process was successful in the main study.27  
B. Training: Individualized telephone/Internet 
trainings using an academic detailing approach 
were provided to implementation champions 
(physicians, nurses, dentist, hygienist, or other 
staff) at each practice. Training included hands-
on demonstrations of the technology-assisted 
components described above, including 
practicing e-referring a “test” smoker. As a guide, 
a detailed training script was developed. 
C. Booster sessions: To support implementation, a 
series of 4 motivational emails were sent and 15-
30-minute calls made over a 6-week period.  
D. Visual cues: Workflow support materials to aid 
referrals, such as fliers, were provided as cues to 
remind providers to refer smokers to the study.   
Anticipated Adaptations of the e-referral and Implementation Program for ActivityLink: The design of 
the original trial was based on the “5A’s” of counseling for behavior change: ASK (Current smoker), ADVISE 
(Recommendations on cessation), ASSESS (Readiness to begin program), ASSIST (Program referral, additional 
resources needed) and ARRANGE (Schedule follow-up call with patient).  I will use a similar design adapted to 
a Breast cancer clinic setting and referral to an in-person community program (LIVESTRONG). Figure 6 shows 
my initial concept of the workflow for referrals in ActivityLink. Anticipated adaptations to the technology-assisted 
components, include: a) the e-referral system to include zip code entering, triggering an automated notification 
to LIVESTRONG program staff at the nearest site, b) motivational messaging templates to prompt patients to 
complete their program enrollment, and c) feedback on participants’ attendance to and performance in 
LIVESTRONG (proposed in Figure 7).  Anticipated adaptations to additional supporting components may 
include: a) training of navigators to include motivational interviewing on overcoming barriers to program 
attendance and, b) systematic protocol for screening patients, providing medical clearance and transmitting 
information to navigators for subsequent referral to LIVESTRONG.  
 
Figure 5. Existing e-referral provider dashboard  
Figure 6. Proposed steps of ActivityLink 

3.3h. Specific Aim 1 (Months 1-12): Develop ActivityLink implementation program to increase e-referrals 
to an evidence-based physical activity program (LIVESTRONG at the Y).   
Sub-Aim 1a (Months 1-6): Understand the clinical workflow, barriers and preferences to referrals of 
survivors to LIVESTRONG  (protocol, screening, 
triaging, and follow-up) by conducting qualitative 
interviews with  key stakeholders (UMMHC patient 
navigators, oncology and cardiology providers, clinic 
staff, patients and LIVESTRONG staff).  Participants 
will provide informed consent prior to interviews.  
1. Develop interview guides informed by PRISM 
domains and implementation prototypes (Table 
1).  Using PRISM domains, interview guides will 
be developed with iterative feedback from the 
CAB.  I will examine topics including those 
affecting implementation of the program and 
participation in LIVESTRONG and 
implementation strategy prototype screenshots for feedback. 
a. I will create screenshot prototypes of implementation strategies to be adapted, including the referral 
webform (Figure 5) and a patient progress report (Figure 7). 
Table 1. Examples of interview questions mapped on to PRISM1 domains. 
PRISM Domain 
Example of Key Questions Relevant to ActivityLink 
Program (Intervention) 
Organizational 
perspective 
Patient perspective 
• 
What steps are needed to prepare key staff to identify and screen patients? To 
conduct the program? How can the system best be embedded into workflow?   
• 
How can the program be more patient-centered and less complex for patients?  Does it 
address important patient barriers to response? 
External Environment 
• 
Are there community/outside resources that can enhance intervention? 
Implementation & 
Sustainability 
Infrastructure 
• 
How can existing infrastructure be used to handle key implementation/sustainability 
tasks?   
• 
Can implementation/sustainability tasks be part of key staff/Navigator job descriptions? 
Recipients 
Organizational 
characteristics 
Patient characteristics 
• 
Which available systems can best support implementing the program?  Do key staff 
expect the program to be sustainable? 
• 
What are common knowledge, beliefs and perceived risk patterns among patient 
recipients? (modifiable factors regarding patient perceived net benefit) 
2. Obtain feedback from key stakeholders. I will conduct semi-structured phone interviews (30-minutes) with 
UMMHC breast patient navigators (n=2), oncologists/radiologists (n=2), cardiologists (n=2), and patients 
(n=2) who are not part of the CAB.  I will ask clinical stakeholders about current workflow for physical activity 
counseling/referrals or anticipated changes in workflow that need to be made using use-cases, often used 
by software engineers, to elicit workflow depiction.75 Use cases describe how a user completed a task in a 
sequence of simple steps, beginning with a user's goal and ending when that goal is fulfilled. Thus, it is easier 
for the user to understand the workflow and highlight issues/provide additional details about the use-case. 
Under Dr. Sadasivam’s guidance (a software engineer), I will use my clinical shadowing experience and field 
notes (see Training Plan) to develop initial use-cases as prompts for the discussion. Recruitment: I will use 
established partnerships and snowball sampling to recruit clinic staff and physicians and information 
extracted from EHR to identify patients for recruitment (HIPAA waiver). Navigators will aid in this effort. 
Sub-Aim 1a Data Analysis:  Under the direction of the qualitative advisor, Dr. Kristin Mattocks, I will 
transcribe interviews verbatim and use a coding start-list based on protocol content (deductive) followed by open 
coding driven by immersion in the data (inductive).76  Myself and one other research team member with 
experience analyzing qualitative data will each code  >10% of transcripts and coding checks will be completed 
to ensure inter-rater reliability. Disputes in coding decisions will be discussed within the research and mentor 
team until resolved and a kappa > 0.75 is attained, at which point the entire dataset will be coded. Based on the 
use-cases, I will also create a workflow map using swimlanes (Lucid software) to document what the actual 
workflow of the patient navigator is surrounding physical activity counseling/referrals. This will include their 
communication and interactions with oncologists, cardiologists and other clinic staff.66  I will then present these 
results to the CAB to refine the program as discussed in Aim 1c. 
Sub-Aim 1b (Months 6-8): Refine the e-referral Web form using think aloud usability testing with 
Navigators. I will perform usability testing with our users, present feedback to the research team and the CAB 
and make iterative refinements to the protocol as needed.   
1. Conduct usability testing.  I will use "Think Aloud"77 protocols as follows: While users are reviewing the 
content, they will be asked to vocalize thoughts, feelings, and opinions while interacting with the system. 
Think Aloud allows you to understand how the user approaches the interface and what considerations they 
keep in mind while using. Participants will complete a rating instrument including questions addressing the 
Figure 7. Sample patient progress report used by a 
successful LIVESTRONG Program. 

acceptability, accuracy, ease of use and satisfaction with the system.  Since the recommendation indicates 
that the majority of issues can be identified with 4±1 users,78 I will test the system with up-to 5 navigators.  
a. Analysis: Testing sessions will be  transcribed, data segmented and codes applied to identify themes 
in the data.77  I will combine these themes with field notes to design usability reports.   
2. Systematically compare users’ feedback.79  To determine the need for refinements and additional testing, we 
will present reports to the CAB. If 50% or more of participants have difficulty achieving a task within the 
system, I will consider this problematic and revise that feature with iterative feedback from users’.80 I will use 
this iterative process until less than 50% of participants are having issues with completing the tasks.   
Sub-Aim 1c (Months 8-12): Refine the e-referral additional components to support integration into the 
clinical workflow (paper prescription form, reminders, training and booster program and feedback) for navigators 
using feedback from aims 1a and 1b.  I will work iteratively with the CAB.  Specifically, I will: 
1. Create feedback summary reports based on aims 1a and 1b. 
a. Barriers and facilitators, implementation strategies, clinical workflow and usability reports. 
2. Create new clinical workflow swimlane mapping and protocol including strategies and e-referral.  
3. Meet with the CAB, present feedback reports, swimlane mapping and protocol. 
4. Based on feedback, refine clinical workflow and protocol.  Re-distribute to CAB for approval. 
5. Develop final version of protocol and swimlane workflow mapping to be beta tested in Aim 2. 
3.3i. Specific Aim 2 (Months 12-22): Beta test in UMMHC Breast Clinic and refine based on user feedback.  
This Aim will provide insight into feasibility and acceptability of ActivityLink delivered by navigators to survivors 
at rick of CVD. I will beta test the program in 1 clinic and at 3 YMCAs (3.3d. Setting).  Beta testing is commonly 
conducted in iterative cycles and used to verify program readiness and user satisfaction prior to a full-scale 
launch.81 Results from beta testing will inform a larger, multi-site feasibility pilot trial resulting in randomization at 
the level of the clinic mirroring my mentors’ prior work.82  
Design: I will beta test ActivityLink in two waves using Plan, 
Do, Study, Act (PDSA) Cycles.25 PDSA cycles are best used to test 
ideas on a small scale prior to full implementation65 and are 
commonly used for implementation of projects into clinical 
workflow.66 As detailed below,  the initial wave  will be 8-weeks in 
length and used to refine ActivityLink for the second wave (8 
weeks). Waves will be timed with the LIVESTRONG program 
calendar (new sessions begin in January, April, July and October).  
Thus, I plan to conduct the first wave for the October 2021 session 
and the second wave for the January 2022 (Figure 8).    
Recruitment, Screening, Informed Consent and Data 
Collection: The Breast clinic employs 10 patient navigators.  
Inclusion Criteria: Navigators in the UMMHC Breast Clinic will be 
eligible to participate if they: 1) are employed as a navigator of the 
clinic and 2) provide written informed consent.  Navigator 
Recruitment: I have already developed relationships with clinic 
navigators and will provide information on the study enrollment at 
their monthly in-service meeting. Interested navigators will complete an informed consent prior to enrolling in the 
study. Patient Recruitment: I will use an opt-out strategy for recruiting patients for the follow-up interview. Based 
on the workflow designed, eligible patients will be screened and referred to the program by the navigator.  For 
those who agree to be referred, the navigator will enter their information to the e-referral system. For these 
patients and those who do not want to be referred, navigators will also document their contact information and 
reasons for not joining the program/refusing referrals as part of their clinical workflow. Navigators will also provide 
a brochure that will include information about the research study and that someone may contact them for a 
follow-up interview.  The brochure will include contact information (email, text, and mailing information) for them 
to use to opt out of the follow-up interview if they wish to do so. I, or my research staff will then contact patients 
who have not opted out to schedule an interview. Data Collection: Follow-up phone calls with navigators, a sub-
sample of patients and LIVESTRONG staff will be done to conduct qualitative assessments (verbal informed 
consent prior to the interview). I will also collect quantitative measures to assess number of referrals.   
PDSA Cycle 1: Plan -  Using feedback from Aim 1, ActivityLink wave 1 will go live 8 weeks prior to the 
start of the 3rd quarterly LIVESTRONG session (October 2021). I will use the plan designed and informed by the 
CAB in Aim1c for the program implementation. Navigators will undergo training on implementing ActivityLink and 
receive detailed protocol and workflow mapping prior to the start.  
Do – During the implementation of ActivityLink navigators will receive booster phone calls and emails 
weekly (8 each) to encourage implementation and troubleshoot any issues. Measures: Throughout the 
implementation period,  I will interview navigators using ActivityLink, patients who have been referred (accepted 
and declined) and LIVESTRONG site Program Directors. Navigator interviews will examine RE-AIM measures 
DO
- Beta test in 
Breast clinic
- Interviews, 
reach data 
STUDY
- Transcribe 
and analyze 
- Assess 
results 
ACT
- Present to 
CAB
  - Determine              
changes 
PLAN
- Plan for 
changes
- Action plan 
to implement 
Figure 8. PDSA Cycle (2 cycles). 

(see Table 2) associated with the ActivityLink and any additional questions identified from the CAB. I will conduct 
at least 1 navigator interview every 2 weeks.  Patient and LIVESTRONG staff interviews will examine barriers, 
facilitators and preferences of the ActivityLink referral process. I will interview patients (n=2-3) who were referred 
to the program (identified by our e-referral system), patients who declined (n=2-3) and LIVESTRONG Program 
Directors (n=3) starting 2 weeks after each wave launches. I will collect data navigators recorded from patients 
who refused the referral. Number of referred patients (from e-referral system) and number refused (from 
navigator documentation) will be collected during the 8 weeks.  
 Study -  For qualitative analysis, I will use a similar protocol to Aim 1a (deductive followed by inductive 
approach.76  Transcripts will be coded by myself and 1 other research team member using coding checks (10% 
transcripts) to ensure inter-rater reliability. We will resolve disputes in coding (> 0.75 Kappa) and then the entire 
dataset will be coded. Quantitative data will not be analyzed due to the small sample size, though I will present 
descriptive statistics of number of referrals, number of declined referrals and proportion of eligible patients 
referred (based on number of survivors treated in the clinic during the wave) and present to the CAB.  
Act – I will organize results and present them to the CAB to help determine the need for adaptations to 
the system. Using the CABs feedback, we will develop action-items for changes to be made in Cycle 2. 
PDSA Cycle 2:  The CAB will help to determine adaptations needed to the new implementation plan.  
The second ActivityLink wave will go live 8 weeks prior to the January 2022 session and navigators will be trained 
on the newly adapted system.  I will collect the same qualitative and quantitative measures as identified in PDSA 
Cycle 1. At the conclusion of wave 2 (Act), I will work with the CAB to refine and package ActivityLink to be ready 
for testing in pilot feasibility  trial (see section 3.4). 
3.4 Impact and Future Directions 
If funded, this project will help to advance my career as an implementation scientist and inform the development 
of a sustainable integrated community-clinical program that can impact the heart health and longevity of breast 
cancer survivors. I will end this award period with a refined version of ActivityLink to be further tested in a multi-
clinic feasibility pilot trial. I have devoted time to manuscript preparation and grant writing which I will take place 
throughout the duration of the award. I will write-up and submit 3 manuscripts involving the project 
planning/design, protocol and final outcomes to Implementation Science and Public Health journals. During the 
second year of this K-award, I will write and submit a K01 (NHLBI) to test the feasibility of ActivityLink. The NHBLI 
K01 mechanism has specifically called for innovative research in the area of: “Implementation (T4 translation) 
research that addresses strategies for sustained adoption of proven, effective interventions in communities, 
health 
systems, 
clinicians, patients and 
families for heart, lung, 
and blood  diseases and 
sleep disorders”. Thus, 
my project will be an apt 
fit for this mechanism. 
To further disseminate 
my work, I will present at 
annual conferences for 
Dissemination 
and 
Implementation 
Science, 
Society 
of 
Behavioral Medicine and UMMCCTS Community Engagement and Research Symposium.  Quarterly reports will 
be distributed to the UMMHC Cancer Center, Cardiology Department and YMCA partners. 
Table 2. Qualitative sample navigator interview questions mapped on to RE-AIM measures.74 
RE-AIM Measures 
Sample Questions 
Reach 
• 
What factors contribute to the participation/non-participation of the participants? 
• 
What might have been done to get more of the target audience to participate? 
Adoption 
• 
What factors contributed to the organization and its individuals taking up ActivityLink?  
• 
What barriers interacted with the ActivityLink to prevent adoption?  
• 
Was there partial or complete adoption?  
• 
Why did some staff members in these organizations participate and others did not? 
Implementation 
• 
How was the ActivityLink implemented? By whom and when?  
• 
What influenced implementation or lack of implementation?  
• 
What combination of implementation effects affected the outcome results?  
• 
How and why was the program adapted or modified over time? 
Maintenance 
• 
Do you foresee ActivityLink being implemented after the program core period?  
• 
What is needed to maintain this program? 
Table 3. Research timeline. 
  
Months of Project 
Task 
2 
4 
6 
8 
10 
12 
14 
16 
18 
20 
22 
24 
Aim 1:   Develop ActivityLink implementation program 
Protocol finalization/IRB approval 
  
  
  
  
  
  
  
  
  
  
  
  
Stakeholder interviews (1a) 
  
  
  
  
  
  
  
  
  
  
  
  
Usability testing/refinement (1b) 
  
  
  
  
  
  
  
  
  
  
  
  
Refine implementation/workflow (1c) 
  
  
  
  
  
  
  
  
  
  
  
  
Finalize ActivityLink 
  
  
  
  
  
  
  
  
  
  
  
  
Aim 2: Beta test implementation program 
Patient referrals and interviews 
  
  
  
  
  
  
  
  
  
  
  
  
Program completion and interviews 
  
  
  
  
  
  
  
  
  
  
  
  

Literature Cited 
 
1 
Feldstein, A. C. & Glasgow, R. E. A Practical, Robust Implementation and Sustainability Model 
(PRISM) for Integrating Research Findings into Practice. The Joint Commission Journal on Quality and 
Patient Safety 34, 228-243, doi:https://doi.org/10.1016/S1553-7250(08)34030-6 (2008). 
2 
Courneya, K. S. Exercise in cancer survivors: an overview of research. Medicine and science in sports 
and exercise 35, 1846-1852, doi:10.1249/01.mss.0000093622.41587.b6 (2003). 
3 
Irwin, M. L. & Ainsworth, B. E. Physical activity interventions following cancer diagnosis: methodologic 
challenges to delivery and assessment. Cancer investigation 22, 30-50 (2004). 
4 
Schumacher, M. M. & McNiel, P. The Impact of Livestrong(R) at the YMCA for Cancer Survivors. Oncol 
Nurs Forum 45, 717-725, doi:10.1188/18.ONF.717-725 (2018). 
5 
Irwin, M. L. et al. Effect of the LIVESTRONG at the YMCA exercise program on physical activity, 
fitness, quality of life, and fatigue in cancer survivors. Cancer 123, 1249-1258, doi:10.1002/cncr.30456 
(2017). 
6 
Schairer, C., Mink, P. J., Carroll, L. & Devesa, S. S. Probabilities of death from breast cancer and other 
causes among female breast cancer patients. J Natl Cancer Inst 96, 1311-1321 (2004). 
7 
Lopez, D. et al. Establishing effective patient navigation programs in oncology. Supportive Care in 
Cancer 27, 1985-1996, doi:10.1007/s00520-019-04739-8 (2019). 
8 
Koene, R. J., Prizment, A. E., Blaes, A. & Konety, S. H. Shared risk factors in cardiovascular disease 
and cancer. Circulation 133, 1104-1114 (2016). 
9 
O’Hare, M., Sharma, A., Murphy, K., Mookadam, F. & Lee, H. Cardio-oncology Part I: chemotherapy 
and cardiovascular toxicity. Expert review of cardiovascular therapy 13, 511-518 (2015). 
10 
Keats, M. R., Cui, Y., Grandy, S. A. & Parker, L. Cardiovascular disease and physical activity in adult 
cancer survivors: a nested, retrospective study from the Atlantic PATH cohort. Journal of Cancer 
Survivorship 11, 264-273, doi:10.1007/s11764-016-0584-x (2017). 
11 
Mehta, L. S. et al. Cardiovascular disease and breast cancer: where these entities intersect: a scientific 
statement from the American Heart Association. Circulation 137, e30-e66 (2018). 
12 
Warburton, D. E., Nicol, C. W. & Bredin, S. S. Health benefits of physical activity: the evidence. Cmaj 
174, 801-809 (2006). 
13 
Scott, J. M. et al. Cardiovascular disease following hematopoietic stem cell transplantation: 
pathogenesis, detection, and the cardioprotective role of aerobic training. Critical reviews in 
oncology/hematology 98, 222-234 (2016). 
14 
Kirkham, A. A. & Davis, M. K. Exercise prevention of cardiovascular disease in breast cancer survivors. 
Journal of oncology 2015 (2015). 
15 
Garcia, D. O. & Thomson, C. A. Physical activity and cancer survivorship. Nutrition in Clinical Practice 
29, 768-779 (2014). 
16 
Hewitt, M. E., American Society of Clinical, O., Ganz, P. & Institute of, M. From Cancer Patient to 
Cancer Survivor: Lost in Transition. 
17 
Bélanger, L. J., Plotnikoff, R. C., Clark, A. & Courneya, K. S. A survey of physical activity programming 
and counseling preferences in young-adult cancer survivors. Cancer nursing 35, 48-54 (2012). 
18 
Weaver, K. E. et al. Cardiovascular risk factors among long-term survivors of breast, prostate, 
colorectal, and gynecologic cancers: a gap in survivorship care? Journal of Cancer Survivorship 7, 253-
261, doi:10.1007/s11764-013-0267-9 (2013). 
19 
Fong, A. J. A qualitative analysis of oncology clinicians’ perceptions and barriers for physical activity 
counseling in breast cancer survivors. Supportive care in cancer 26, 3117-3126, doi:10.1007/s00520-
018-4163-8. 
20 
Pratt‐Chapman, M., Simon, M. A., Patterson, A. K., Risendal, B. C. & Patierno, S. Survivorship 
navigation outcome measures: A report from the ACS patient navigation working group on survivorship 
navigation. Cancer 117, 3573-3582 (2011). 
21 
Braun, K. L. et al. Cancer patient navigator tasks across the cancer care continuum. Journal of health 
care for the poor and underserved 23, 398 (2012). 
22 
Houston, T. K. et al. Evaluating the QUIT-PRIMO clinical practice ePortal to increase smoker 
engagement with online cessation interventions: a national hybrid type 2 implementation study. 
Implementation Science 10, 154, doi:10.1186/s13012-015-0336-8 (2015). 

23 
Sadasivam, R. S. et al. Implementing point of care "e-referrals" in 137 clinics to increase access to a 
quit smoking internet system: the Quit-Primo and National Dental PBRN HI-QUIT Studies. Translational 
behavioral medicine 3, 370-378, doi:10.1007/s13142-013-0230-3 (2013). 
24 
Escoffery, C. et al. A systematic review of adaptations of evidence-based public health interventions 
globally. Implementation Science 13, 125, doi:10.1186/s13012-018-0815-9 (2018). 
25 
Langley, G. J. et al. The improvement guide: a practical approach to enhancing organizational 
performance.  (John Wiley & Sons, 2009). 
26 
NIH. Research Portfolio Online Reporting Tools (RePORT), 
<https://projectreporter.nih.gov/reporter_SearchResults.cfm?icde=49041517> (2020). 
27 
Sadasivam, R. S. et al. Who participates in Web-assisted tobacco interventions? The QUIT-PRIMO and 
National Dental Practice-Based Research Network Hi-Quit studies. J Med Internet Res 15 (2013). 
28 
Centers for Disease Control and Prevention, N. C. f. H. S. Underlying Cause of Death 1999-2017 on 
CDC WONDER Online Database, released December, 2018. Data are from the Multiple Cause of 
Death Files, 1999-2017, as compiled from data provided by the 57 vital statistics jurisdictions through 
the Vital Statistics Cooperative Program., < Accessed at http://wonder.cdc.gov/ucd-icd10.html > ( 
29 
Ford, E. S. & Capewell, S. Coronary heart disease mortality among young adults in the US from 1980 
through 2002: concealed leveling of mortality rates. Journal of the American College of Cardiology 50, 
2128-2132 (2007). 
30 
Mosca, L. et al. Awareness, perception, and knowledge of heart disease risk and prevention among 
women in the United States. Archives of Family Medicine 9, 506 (2000). 
31 
Mosca, L. et al. National study of physician awareness and adherence to cardiovascular disease 
prevention guidelines. Circulation 111, 499-510 (2005). 
32 
Abuful, A., Gidron, Y. & Henkin, Y. Physicians' attitudes toward preventive therapy for coronary artery 
disease: is there a gender bias? Clinical Cardiology: An International Indexed and Peer‐Reviewed 
Journal for Advances in the Treatment of Cardiovascular Disease 28, 389-393 (2005). 
33 
Moey, M. & Moslehi, J. Cardiovascular disease and breast cancer. Nature Reviews Cardiology 15, 200-
202 (2018). 
34 
American Cancer Society. Cancer Treatment & Survivorship Facts & Figures 2019-2021, 
<https://www.cancer.org/content/dam/cancer-org/research/cancer-facts-and-statistics/cancer-
treatment-and-survivorship-facts-and-figures/cancer-treatment-and-survivorship-facts-and-figures-
2019-2021.pdf> (2019). 
35 
American Cancer Society. Cancer Statistics Center, 
<https://cancerstatisticscenter.cancer.org/?_ga=2.140342327.509434287.1563284237-
1041260707.1558116752#!/> (2020). 
36 
Ganz, P. A. Monitoring the physical health of cancer survivors: a survivorship-focused medical history. 
Journal of Clinical Oncology 24, 5105-5111 (2006). 
37 
Lipshultz, S. E., Cochran, T. R., Franco, V. I. & Miller, T. L. Treatment-related cardiotoxicity in survivors 
of childhood cancer. Nature reviews Clinical oncology 10, 697 (2013). 
38 
Benjamin, E. J. et al. Heart disease and stroke statistics-2017 update: a report from the American Heart 
Association. circulation 135, e146-e603 (2017). 
39 
Darby, S. C. et al. Risk of ischemic heart disease in women after radiotherapy for breast cancer. New 
England Journal of Medicine 368, 987-998 (2013). 
40 
Blair, S. N. et al. Influences of cardiorespiratory fitness and other precursors on cardiovascular disease 
and all-cause mortality in men and women. Jama 276, 205-210 (1996). 
41 
Ibrahim, E. M. & Al-Homaidh, A. Physical activity and survival after breast cancer diagnosis: meta-
analysis of published studies. Medical oncology (Northwood, London, England) 28, 753-765, 
doi:10.1007/s12032-010-9536-x (2011). 
42 
Chlebowski, R. T. Nutrition and physical activity influence on breast cancer incidence and outcome. 
Breast (Edinburgh, Scotland) 22 Suppl 2, S30-37, doi:10.1016/j.breast.2013.07.006 (2013). 
43 
Campbell, K. L. et al. Exercise guidelines for cancer survivors: consensus statement from International 
Multidisciplinary Roundtable. Medicine & Science in Sports & Exercise 51, 2375-2390 (2019). 
44 
Rock, C. L. et al. Nutrition and physical activity guidelines for cancer survivors. CA: A Cancer Journal 
for Clinicians 62, 242-274, doi:10.3322/caac.21142 (2012). 
45 
Ballard-Barbash, R. et al. Physical activity, biomarkers, and disease outcomes in cancer survivors: a 
systematic review. J Natl Cancer Inst 104, 815-840, doi:10.1093/jnci/djs207 (2012). 

46 
Santa Mina, D. et al. Enablers and barriers in delivery of a cancer exercise program: the Canadian 
experience. Current oncology (Toronto, Ont.) 22, 374-384, doi:10.3747/co.22.2650 (2015). 
47 
Jones, L. W. & Courneya, K. S. Exercise counseling and programming preferences of cancer survivors. 
Cancer practice 10, 208-215 (2002). 
48 
Vallance, J. K., Courneya, K. S., Jones, L. W. & Reiman, T. Exercise preferences among a population-
based sample of non-Hodgkin's lymphoma survivors. Eur J Cancer Care (Engl) 15, 34-43, 
doi:10.1111/j.1365-2354.2005.00617.x (2006). 
49 
Courneya, K. S. & Friedenreich, C. M. Physical exercise and quality of life following cancer diagnosis: a 
literature review. Ann Behav Med 21, 171-179, doi:10.1007/bf02908298 (1999). 
50 
Jones, L. W., Courneya, K. S., Fairey, A. S. & Mackey, J. R. Effects of an oncologist’s recommendation 
to exercise on self-reported exercise behavior in newly diagnosed breast cancer survivors: a single-
blind, randomized controlled trial. Annals of Behavioral Medicine 28, 105-113 (2004). 
51 
Rabin, C. Promoting lifestyle change among cancer survivors: when is the teachable moment? 
American Journal of Lifestyle Medicine 3, 369-378 (2009). 
52 
Nadler, M. et al. Oncology care provider perspectives on exercise promotion in people with cancer: an 
examination of knowledge, practices, barriers, and facilitators. Supportive Care in Cancer 25, 2297-
2304 (2017). 
53 
van Dillen, S. M., van Binsbergen, J. J., Koelen, M. A. & Hiddink, G. J. Nutrition and physical activity 
guidance practices in general practice: a critical review. Patient education and counseling 90, 155-169 
(2013). 
54 
Stout, N. L., Sleight, A., Pfeiffer, D., Galantino, M. L. & deSouza, B. Promoting assessment and 
management of function through navigation: opportunities to bridge oncology and rehabilitation systems 
of care. Supportive Care in Cancer 27, 4497-4505 (2019). 
55 
Mina, D. S. et al. Connecting people with cancer to physical activity and exercise programs: a pathway 
to create accessibility and engagement. Current oncology (Toronto, Ont.) 25, 149-162, 
doi:10.3747/co.25.3977 (2018). 
56 
Agency for Healthcare Research and Quality. Clinical-Community Linkages, 
<https://www.ahrq.gov/professionals/prevention-chronic-care/improve/community/index.html> (2016). 
57 
Wagner, E. H., Austin, B. T. & Von Korff, M. Organizing care for patients with chronic illness. Milbank Q 
74, 511-544 (1996). 
58 
Natale-Pereira, A., Enard, K. R., Nevarez, L. & Jones, L. A. The role of patient navigators in eliminating 
health disparities. Cancer 117, 3541-3550, doi:10.1002/cncr.26264 (2011). 
59 
Winters-Stone, K. M. et al. Enhancing an oncologist's recommendation to exercise to manage fatigue 
levels in breast cancer patients: a randomized controlled trial. Support Care Cancer 26, 905-912, 
doi:10.1007/s00520-017-3909-z (2018). 
60 
Kirkham, A. A. et al. Effectiveness of Oncologist-Referred Exercise and Healthy Eating Programming 
as a Part of Supportive Adjuvant Care for Early Breast Cancer. Oncologist 23, 105-115, 
doi:10.1634/theoncologist.2017-0141 (2018). 
61 
Park, J. H. et al. The effect of oncologists' exercise recommendations on the level of exercise and 
quality of life in survivors of breast and colorectal cancer: A randomized controlled trial. Cancer 121, 
2740-2748, doi:10.1002/cncr.29400 (2015). 
62 
Pew Research Center. Mobile Technology Fact Sheet., <http://www.pewinternet.org/fact-
sheets/mobile-technology-fact-sheet/> (2015). 
63 
Haque, S., Ebron, S., Bailey, B. & Blumenfeld, B. Use of Health Information Technology among Patient 
Navigators in Community Health Interventions. Perspect Health Inf Manag 16, 1a-1a (2019). 
64 
Shephard, E. et al. E-referrals: improving the routine interspecialty inpatient referral system. BMJ Open 
Quality 7, e000249, doi:10.1136/bmjoq-2017-000249 (2018). 
65 
Byrne, J., Xu, G. & Carr, S. DEVELOPING AN INTERVENTION TO PREVENT ACUTE KIDNEY 
INJURY: USING THE PLAN, DO, STUDY, ACT (PDSA) SERVICE IMPROVEMENT APPROACH. 
Journal of Renal Care 41, 3-8, doi:10.1111/jorc.12090 (2015). 
66 
Knox, L., Brach, C. Mapping and Redesigning Workflow (Agency for Healthcare Research and Quality, 
2015). 
67 
Kildea, J. et al. Design and Development of a Person-Centered Patient Portal Using Participatory 
Stakeholder Co-Design. J Med Internet Res 21, e11371, doi:10.2196/11371 (2019). 

68 
Glasgow, R. E., Vogt, T. M. & Boles, S. M. Evaluating the public health impact of health promotion 
interventions: the RE-AIM framework. American Journal of Public Health 89, 1322-1327, 
doi:10.2105/ajph.89.9.1322 (1999). 
69 
Bluethmann, S. M., Mariotto, A. B. & Rowland, J. H. Anticipating the "Silver Tsunami": Prevalence 
Trajectories and Comorbidity Burden among Older Cancer Survivors in the United States. Cancer 
epidemiology, biomarkers & prevention : a publication of the American Association for Cancer 
Research, cosponsored by the American Society of Preventive Oncology 25, 1029-1036, 
doi:10.1158/1055-9965.EPI-16-0133 (2016). 
70 
O’Keefe, E. B., Meltzer, J. P. & Bethea, T. N. Health Disparities and Cancer: Racial Disparities in 
Cancer Mortality in the United States, 2000–2010. Frontiers in Public Health 3, 
doi:10.3389/fpubh.2015.00051 (2015). 
71 
Sadasivam, R. S. et al. Development of an interactive, Web-delivered system to increase provider-
patient engagement in smoking cessation. J Med Internet Res 13, e87, doi:10.2196/jmir.1721 (2011). 
72 
Heston, A. H., Schwartz, A. L., Justice-Gardiner, H. & Hohman, K. H. Addressing physical activity 
needs of survivors by developing a community-based exercise program: LIVESTRONG(R) at the 
YMCA. Clin J Oncol Nurs 19, 213-217, doi:10.1188/15.CJON.213-217 (2015). 
73 
McCreight, M. S. et al. Using the Practical, Robust Implementation and Sustainability Model (PRISM) to 
qualitatively assess multilevel contextual factors to help plan, implement, evaluate, and disseminate 
health services programs. Translational behavioral medicine 9, 1002-1011 (2019). 
74 
Holtrop, J. S., Rabin, B. A. & Glasgow, R. E. Qualitative approaches to use of the RE-AIM framework: 
rationale and methods. BMC Health Services Research 18, 177, doi:10.1186/s12913-018-2938-8 
(2018). 
75 
Services, U. S. D. o. H. H. usability.gov, <https://www.usability.gov/how-to-and-tools/methods/use-
cases.html> (2017). 
76 
Bradley, E. H., Curry, L. A. & Devers, K. J. Qualitative data analysis for health services research: 
developing taxonomy, themes, and theory. Health Serv Res 42, 1758-1772, doi:10.1111/j.1475-
6773.2006.00684.x (2007). 
77 
Van Someren, M., Barnard, Y. & Sandberg, J. The think aloud method: a practical approach to 
modelling cognitive.  (Citeseer, 1994). 
78 
Faulkner, L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. 
Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc 35, 
379-383, doi:10.3758/bf03195514 (2003). 
79 
Dabbs, A. D. V. et al. User-centered design and interactive health technologies for patients. Computers, 
informatics, nursing: CIN 27, 175 (2009). 
80 
Rubin, J. & Chisnell, D. Handbook of usability testing: how to plan, design and conduct effective tests.  
(John Wiley & Sons, 2008). 
81 
Dolan, R. J. & Matthews, J. M. Maximizing the utility of customer product testing: beta test design and 
management. Journal of Product Innovation Management: AN INTERNATIONAL PUBLICATION OF 
THE PRODUCT DEVELOPMENT & MANAGEMENT ASSOCIATION 10, 318-330 (1993). 
82 
for the National Dental, P. a. Q. C. G. et al. Implementing point of care “e-referrals” in 137 clinics to 
increase access to a quit smoking internet system: the Quit-Primo and National Dental PBRN HI-QUIT 
Studies. Translational Behavioral Medicine 3, 370-378, doi:10.1007/s13142-013-0230-3 (2013).