JMIR FORMATIVE RESEARCH

Sohl et al

Original Paper

Adaptation of a Personalized Electronic Care Planning Tool for
Cancer Follow-up Care: Formative Study
Stephanie J Sohl1,2, PhD; Pamela W Duncan1, PhD; Elyse Thakur3, PhD; Nicole Puccinelli-Ortega1, MS, NBC-HWC;
John M Salsman1,2, PhD; Greg Russell1, MS; Boris C Pasche1,2, MD, PhD; Stacy Wentworth2, MD; David P Miller
Jr1,2, MS, MD; Lynne I Wagner1,2, PhD; Umit Topaloglu1,2, PhD
1

Wake Forest University School of Medicine, Winston-Salem, NC, United States

2

Atrium Health Wake Forest Baptist Comprehensive Cancer Center, Winston-Salem, NC, United States

3

Atrium Health, Charlotte, NC, United States

Corresponding Author:
Stephanie J Sohl, PhD
Wake Forest University School of Medicine
Medical Center Blvd.
Winston-Salem, NC, 27157
United States
Phone: 1 3367135093
Email: ssohl@wakehealth.edu

Abstract
Background: Most patients diagnosed with colorectal cancer will survive for at least 5 years; thus, engaging patients to optimize
their health will likely improve outcomes. Clinical guidelines recommend patients receive a comprehensive care plan (CP) whentransitioning from active treatment to survivorship, which includes support for ongoing symptoms and recommended healthy
behaviors. Yet, cancer care providers find this guideline difficult to implement. Future directions for survivorship care planninginclude enhancing information technology support for developing personalized CPs, using CPs to facilitate self-management,
and assessing CPs in clinical settings.
Objective: We aimed to develop an electronic tool for colorectal cancer follow-up care (CFC) planning.
Methods: Incorporating inputs from health care professionals and patient stakeholders is fundamental to the successful integrationof any tool into the clinical workflow. Thus, we followed the Integrate, Design, Assess, and Share (IDEAS) framework to adapt
an existing application for stroke care planning (COMPASS-CP) to meet the needs of colorectal cancer survivors (COMPASS-CP
CFC). Constructs from the Consolidated Framework for Implementation Research (CFIR) guided our approach. We completed
this work in 3 phases: (1) gathering qualitative feedback from stakeholders about the follow-up CP generation design and workflow;(2) adapting algorithms and resource data sources needed to generate a follow-up CP; and (3) optimizing the usability of theadapted prototype of COMPASS-CP CFC. We also quantitatively measured usability (target average score ≥70; range 0-100),
acceptability, appropriateness, and feasibility.
Results: In the first phase, health care professionals (n=7), and patients and caregivers (n=7) provided qualitative feedback on
COMPASS-CP CFC that informed design elements such as selection, interpretation, and clinical usefulness of patient-reported
measures. In phase 2, we built a minimal viable product of COMPASS-CP CFC. This tool generated CPs based on the needs
identified by patient-completed measures (including validated patient-reported outcomes) and electronic health record data, whichwere then matched with resources by zip code and preference to support patients’ self-management. Elements of the CFIR assessed
revealed that most health care professionals believed the tool would serve patients’ needs and had advantages. In phase 3, theaverage System Usability Scale score was above our target score for health care professionals (n=5; mean 71.0, SD 15.2) andpatients (n=5; mean 95.5, SD 2.1). Participants also reported high levels of acceptability, appropriateness, and feasibility. Additional
CFIR-informed feedback, such as desired format for training, will inform future studies.
Conclusions: The data collected in this study support the initial usability of COMPASS-CP CFC and will inform the next steps
for implementation in clinical care. COMPASS-CP CFC has the potential to streamline the implementation of personalized CFC
planning to enable systematic access to resources that will support self-management. Future research is needed to test the impact
of COMPASS-CP CFC on patient health outcomes.

https://formative.jmir.org/2023/1/e41354

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e41354 | p. 1
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Sohl et al(JMIR Form Res 2023;7:e41354) doi: 10.2196/41354
KEYWORDS
cancer survivorship; cancer survivor; colorectal cancer; patient engagement; shared decision-making; follow-up care; patient careplanning; electronic tool; patient-reported outcomes; behavior change

Introduction
Colorectal cancers are among the most prevalent cancers, withover 1.4 million men and women living with a prior diagnosisof colorectal cancer [1]. Approximately 66% of colorectal cancer
patients will survive for at least 5 years after the diagnosis [2];however, many cancer patients get lost in the transition from
active cancer treatment to survivorship and do not receive theguidance they need to manage lasting effects (eg, fatigue,
cognitive difficulties, and pain) and increased health risks (eg,cardiac problems and secondary malignancies [3,4]). Thus, there
is an opportunity to improve cancer follow-up care (CFC),
including providing support for how survivors can take a more
active role to optimize their own health outcomes (ie,self-management) [3].
One way in which CFC can be optimized is throughcomprehensive care plans (CPs), which are recommended inclinical guidelines [3,5]. These plans outline follow-up careneeds, including support for lasting effects (eg, symptom
control) and recommendations for lifestyle changes or healthbehaviors to reduce health risks [3,5,6]. Historically, such plans
have focused on information delivery, which had minimal
benefit [5,7]. Other barriers to effective care planning included
time and resources to develop CPs, lack of routine assessmentof patient-reported outcome (PRO) measures to guide symptom
management, and complexity of shifting the paradigm of carefrom diagnosis and treatment to minimizing long-term risks[4,7]. To improve clinical outcomes, recommendations include
using technology-based solutions to support follow-up careplanning as a personalized and ongoing process [4], in addition
to providing resources to promote self-management. To
empirically assess these benefits, it was recommended that careplanning interventions be evaluated with an implementation
science approach in real-world clinical settings [4,5,7-9].
To address this gap in the literature, we customized an electroniccare planning tool we previously implemented for a large
pragmatic trial of an intervention (COMPASS-CP) designed tosupport stroke care in the real-world setting [10-12].
COMPASS-CP captures and integrates patient-completed data(including validated PROs, social determinants of health, andassessments of health behaviors) with clinical data via astandard-based integration of electronic health record (EHR)
data to enable provider-facilitated decision-making to
immediately generate personalized CPs [10]. Its questionnaires
are designed to yield a comprehensive overview of factors that
can impair a patient’s ability to manage his or her health andrecovery. COMPASS-CP also houses a directory of medical,
rehabilitation, and community resources to support identified
concerns, which can be used to address identified needs, thus
making CPs actionable to improve the patient’s function andquality of life. Moreover, integration with the EHR (Epic) in
an interoperable manner using the Substitutable Medical
https://formative.jmir.org/2023/1/e41354

XSL• FO
RenderX

Applications and Reusable Technologies (SMART) on Fast
Health Interoperability Resources (FHIR) standard promotes
scalability. This unique combination of features is not currently
present in most other electronic survivorship care planning tools
or the EHR itself [13-15]. For example, a similar tool links
patient data to create a CP with guideline-concordant guidance
and not community resources [16]. Participants who successfullyreceived an intervention that included this electronic careplanning tool (COMPASS-CP) had better functional status andsatisfaction with care than those who did not [17]. Stroke healthcare providers (n=44) indicated high user satisfaction (eg, 74%
agreed that the electronic tool identified important factors
impacting the patient’s recovery and ability to self-manage)[10].
The success of the COMPASS-CP tool prompted us to consider
other populations that may benefit from a similar resource, suchas those diagnosed with cancer. This tool has the potential tocompile the core components of CFC planning. Facilitating
personalized links to community resources as a component ofsurvivorship care planning is aligned with the need forcommunity support during this process [18]. The identificationof such community resources is especially important for rural
populations as they may not have access to resources available
in larger metropolitan areas. With limited access to resources,
patients in such settings have higher rates of healthbehavior–related risk factors (ie, age-adjusted smoking andobesity), cancer incidence, and mortality, as compared with therest of the population [19-22]. These patients may havedecreased compliance with recommended follow-up. Thus, it
is important to consider how to best develop comprehensive
and personalized CPs that consider pertinent risk factors forthis patient population.
This study adapted the COMPASS-CP tool to colorectal CFC
planning (COMPASS-CP CFC) with a focus on generating thefollow-up CP component (ie, integrating and not changing thetreatment summary already used in clinical care). We adopted
a user-centered design process to adapt COMPASS-CP and
evaluated elements in the Consolidated Framework for
Implementation Research (CFIR) [23] to plan for successful
implementation in clinical care. In order to prepare for
implementing this tool into a clinical workflow to facilitate
comprehensive care, we first gathered input from key
stakeholders. Patient input on survivorship care needs was also
gathered through our previous work [24]. We then adapted
assessments, algorithms, and resource data sources needed togenerate a cancer follow-up CP. Lastly, we tested the usabilityof the adapted prototype version of COMPASS-CP CFC.

JMIR Form Res 2023 | vol. 7 | e41354 | p. 2
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Sohl et al

Methods

first in focus groups and then after building a minimum viable
product as part of usability testing.

Study Design

Ethical Considerations

To adapt COMPASS-CP to colorectal cancer survivors whocompleted treatment, we adopted the Integrate, Design, Assess,and Share (IDEAS) framework, which consists of the following
10 stages: (1) empathize with target users; (2) specify the target
behavior; (3) ground in behavioral theory (self-determination
theory to inform future work [25]); (4) ideate implementation
strategies; (5) prototype potential products; (6) gather user
feedback; (7) build a minimum viable product, defined as afully functional initial intervention with only the most essential
features; (8) pilot test to assess potential efficacy and usability(current and future studies); (9) evaluate efficacy (futurestudies); and (10) share the intervention and findings (futurestudies) [26]. Stages 1-3 are part of the Integrate aspect, stages
4-7 are part of the Design aspect, stages 8 and 9 are part of theAssess aspect, and stage 10 is part of the Share aspect of theframework. This framework is implemented iteratively until arefined viable product is developed [27,28]. We also used amixed methods approach to assess implementation outcomes

Study procedures involving the participation of human subjects
received institutional review board (IRB) approval (phase 1:
IRB00059620; phase 3: IRB00064181). Participants were
provided an information sheet that described study involvement.
This sheet stated, “By continuing, I agree to take part in thisstudy,” with a waiver for signed consent. Study data were
deidentified. Patients and caregivers received US $50 forparticipation in phase 1.

Description of the COMPASS-CP CFC Process
Patients who completed treatment of colorectal cancer and healthcare professionals involved with colorectal cancer care weretargeted for this application development. The study workflow
is shown in Figure 1. The first component involved patients
filling out measures before a clinic visit to inform provider
sessions. Completion of the provider decision support tool(component 2) then facilitated shared decision-making andresulted in real-time creation of a personalized follow-up CP(component 3).

Figure 1. Summary of COMPASS-CP CFC application components. CFC: cancer follow-up care; CP: care plan.

Phase 1: Learning From Target Users, Ideating
Implementation Strategies, and Gathering Feedback
on Prototype Potential Products
Focus Groups
We elicited feedback to iteratively refine the cancer follow-up
CP generation design and workflow. First, we conducted two
60-minute focus groups (one with patients and caregivers, andone with health professionals) for feedback on content displayedon paper prototypes (enlarged screenshots of application content)with a divergent brainstorming focus (IDEAS stages 1, 4, 5,
and 6).
The first patient and caregiver focus group reviewed application
components 1 and 3. REDCap (Vanderbilt University, TN), an
established web-based application, was adopted as thepatient-facing interface for completing surveys. Questionnaire
data stored in REDCap were retrieved by COMPASS-CP
electronically via application programming interfaces (APIs).
REDCap surveys can be sent to participants through a variety
of methods, including a unique link (ie, token), which will takethem directly to the survey without the need to log in tocomplete the survey. Thus, feedback was mostly requested on
how to make the resulting CP actionable. Patient eligibility
criteria included the following: age ≥18 years; diagnosis of stage
I-III colorectal cancer; cognitive ability to complete interviews;
and ability to understand, read, and write English. Caregiver
eligibility criteria included the following: age ≥18 years andhttps://formative.jmir.org/2023/1/e41354

XSL• FO
RenderX

provision of some capacity of unpaid care for a patient meetingthe patient eligibility criteria.
The second health care professional focus group (providers,administrators, and informatics professionals) reviewed
application components 2 and 3 within the context of an example
patient case. Health care professionals included key
representatives from our study team.

Semistructured Interviews
We then conducted semistructured interviews for another roundof feedback with the health care professional participants (somewere follow-up interviews with stakeholders who participatedin the focus groups and some were initial interviews withadditional stakeholders). These interviews had a convergent
focus and identified anything that was missing or needed to berefined to ensure integration of COMPASS-CP CFC into
standard care. Both rounds of feedback were audio recorded,
and the follow-up interviews were transcribed verbatim by an
external service.
The follow-up interviews included questions guided by the
CFIR [23] to facilitate assessment of potential barriers andfacilitators at multiple levels and help us prepare for future
implementation of the COMPASS-CP CFC tool. Processcomponents of this framework guided our planning andengagement of opinion leaders (The Wake Forest Baptist
Comprehensive Cancer Center leadership, oncologists, andinformation technology professionals), implementation leaders(advanced practice providers and navigators), champions (study
JMIR Form Res 2023 | vol. 7 | e41354 | p. 3
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
team and others), and possible external change agents (clinicalguidelines and requirements) to prepare for future
implementation. Engagement was iterative and incorporated
into the application development to strengthen stakeholders’representation. Interview questions relevant to this stage of
research asked about the inner setting constructs of culture,
implementation climate, learning climate, and leadershipengagement and the outer setting constructs of patient needs
and resources, perceived influence of external policy andincentives for implementation of survivorship care planning,and potential peer pressure to innovate regarding how to addressthese policies [29].

Phase 2: Clarifying the Target Behavior and Buildinga Minimum Viable ProductWe iteratively compiled feedback to clarify the most importantdata elements to incorporate in creating follow-up CPs (IDEAS
stages 2 and 7). The core set of data elements in relevant clinicalguidelines for follow-up care of colorectal cancer included thefollowing [14,30-32]: (1) schedule of clinic visits; (2) cancer
surveillance or other recommended tests (eg, surveillancecolonoscopy, history, and physical screening for other primarycancers); (3) assessment and management of physical andpsychosocial long-term and late effects of colorectal cancer anditstreatment(eg,bowel/gastrointestinal
issues,numbness/tingling, and ostomy/stoma); (4) other late andlong-term effects (depression, anxiety, fatigue, sleep disturbance,
sexual function, fertility, financial toxicity, insurance,
memory/cognitive function, social functioning, physicalfunctioning, and pain); (5) health behaviors (alcohol use,
diet/nutrition, obesity/weight changes, physical activity,
smoking, and sunscreen use); and (6) care coordination andpractice implications (communication among specialists,management of medications, and management of otherillnesses). The development team evaluated the availability of
the identified data elements in Epic.
COMPASS-CP currently uses SMART on FHIR, which is adata exchange standard to enhance interoperability with EHRs
and other systems. The FHIR Patient Resource pushes thepatient medical record number and other demographics into
COMPASS-CP, which creates the patient record. It is also
possible through additional FHIR resources (eg, Encounter) for
COMPASS-CP to be able to communicate back the CPs andprovider reports to Epic.
We identified appropriate patient-completed measures forassessing other essential data elements. To streamline theintegration of the patient-completed data in this study, we
integrated the REDCap electronic data capture system.
Therefore, patients were able to complete assessments
independently (prior work was conducted using a phoneinterview). REDCap has a comprehensive and publicly
accessible library of PRO measures. COMPASS-CP uses
REDCap’s extensive APIs, which enable bidirectional dataexchange in a secure and scalable manner, to incorporate PROdata and other patient-completed data for CP generation.

Sohl et al
personalized CPs. We also updated the existing COMPASS-CP
resource database to be more specific to cancer care [32]. The
minimal viable product of COMPASS-CP CFC generated CPs
based on the needs identified by elevated health concerns raisedin the patient-completed measures and EHR data elements,
which were then matched with relevant resources that could beselected by zip code and preference to support self-management.
Patient zip code from the FHIR drives the filtering andrecommendation of proper local community resources that are
geographically close to the patient.

Phase 3: Assessing the Usability of COMPASS-CP
CFC
We individually implemented the minimal viable version of
COMPASS-CP CFC and elicited qualitative feedback about
the potential efficacy and usability (IDEAS stage 8). We
conducted 2 rounds of usability tests and expected to reachsaturation with 5 participants per round [33,34]. The first roundof testing was with health care professionals (informatics
professionals and health care providers) using mock patient
scenarios. For the second round, a health care professional met
with patients who were colorectal cancer survivors. Both roundsof feedback used a concurrent “think aloud” protocol andretrospective probing questions regarding acceptability [35].
We used the System Usability Scale and aimed to achieve an
average score ≥70 (considered “good”) [34,36,37]. We further
assessed quantitative measures of acceptability, appropriateness,and feasibility (Acceptability of Intervention Measure,Intervention Appropriateness Measure, and Feasibility ofIntervention Measure, respectively) [38].
We also incorporated qualitative assessments in a brief
semistructured interview and questionnaire after the think-aloudtesting in both groups [39]. With the relevant components of
the CFIR Qualitative Interview guide, we systematicallyconsidered select intervention characteristics (design quality
and packaging) and individual characteristics (knowledge andbeliefs about the intervention) to inform the future content, skills
training, and support needed by those who will be using theapplication.

Analyses
Detailed field notes and audio recordings were reviewed tosummarize
qualitative
data
regardingparticipant
recommendations that were iteratively incorporated into theapplication adaptation. For the CFIR analyses to inform future
implementation, we reviewed all deidentified transcripts bylistening to the audio while reading the transcripts to ensure
accuracy. Transcripts were imported into ATLAS.ti (version
7.5; ATLAS.ti Scientific Software Development GmbH) to
manage the data. Two research team members independently
coded each transcript using the CFIR codebook. After the data
were coded, the data within each category were abstracted andsynthesized into themes. Themes were determined by theirprevalence and salience in the data, according to the principlesof thematic analysis [40]. Participant characteristics and usability
data were summarized using descriptive statistics.

We created algorithms in COMPASS-CP CFC informed byclinical guidelines [32] to integrate patient-completed measureswith provider-facilitated decisions and immediately generate
https://formative.jmir.org/2023/1/e41354

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e41354 | p. 4
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Results
Phases 1 and 2: Gathering and Incorporating
Stakeholder Feedback
The characteristics of health care professionals who provided
feedback are described in Table 1, and the characteristics ofSohl et alpatients and caregivers are described in Table 2. The first focus
group included 3 health care professionals, and this wasfollowed up by 7 individual interviews to represent feedback
from 2 nurses, 2 nurse practitioners, 1 physician, 1 physician
assistant, and 1 allied health professional who was in an
administrative role. The second focus group included 5 patientsand 2 caregivers.

Table 1. Demographic characteristics of health care professionals.
Characteristic

Phase 1 (N=7)

Phase 3 (N=5)

Female7 (100)2 (40)

Male

0 (0)

3 (60)

7 (100)5 (100)7 (100)5 (100)44 (14)46 (11)

Sex, n (%)

Ethnicity, n (%)
Not Hispanic or Latino
Race, n (%)
White
Age (years), mean (SD)https://formative.jmir.org/2023/1/e41354

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e41354 | p. 5
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH

Sohl et al

Table 2. Characteristics of colorectal cancer survivors and caregivers.
Characteristic

Phase 1 (N=7)

Phase 3 (N=5)

Female4 (57)

3 (60)

Male

3 (43)

2 (40)

7 (100)5 (100)White/Caucasian

3 (43)

4 (80)

Black/African American

2 (29)

0 (0)

Asian

2 (29)

1 (20)

Married5 (71)

3 (60)

Separated/never married

2 (29)

2 (40)High school or equivalent

2 (29)

1 (20)

Some college/technical

4 (57)0 (0)College graduate/postgraduate

1 (14)

4 (80)

Employed

3 (43)

1 (20)

Self-employed/homemaker

2 (29)

2 (40)

Retired

0 (0)2 (40)Disabled

2 (29)0 (0)0-19,999

3 (43)0 (0)20,000-49,999

2 (29)

2 (40)

50,000-99,000

1 (14)

1 (20)≥100,000

1 (14)

2 (40)

50 (10)60 (14)Sex, n (%)

Ethnicity, n (%)
Not Hispanic or Latino
Race, n (%)

Marital status, n (%)

Education, n (%)

Occupational status, n (%)

Income (US$), n (%)

Age (years), mean (SD)

Selection of Patient-Completed Measures
We iteratively incorporated stakeholder feedback on the relative
importance of recommended data elements and how to best
assess them. For example, we asked health care professionals
to indicate which patient-completed measures would be mostimportant to evaluate with rigor, briefly, or not at all. Thoseprioritized for more rigorous patient assessment were assessedwith Patient-Reported Outcomes Measurement Information
System (PROMIS) measures (ie, anxiety, depression, fatigue,
and physical function), as were other constructs that were also
captured by the PROMIS-29 profile measure already in use
clinically by our local cancer survivorship care providers (ie,social functioning, pain, and sleep disturbance). The PROMIS
measures were also used for brief assessments when the clinicalcutoffs were clear and the number of items was similar to otherassessments (also adopted for assessing cognitive function andsexual function) [41,42]. We adopted PRO-CTCAE(Patient-Reported Outcomes version of the Common
Terminology Criteria for Adverse Events; another widely usedhttps://formative.jmir.org/2023/1/e41354

XSL• FO
RenderX

measurement system specifically for cancer-related symptomatic
toxicity) [43] items for other brief assessments of colorectal
cancer–specific health concerns (numbness and tingling [1 item
for severity and 1 item for interference], and presence of an
ostomy appliance). We grouped together these 3 items specificto colorectal cancer and expected all other items assessed togeneralize more broadly to CFC. We screened for healthbehaviors with items from assessments already used clinicallyfor smoking [44] and alcohol use [45], and with other screening
instruments for diet [46] and physical activity [47,48].
We also broadened the concept of social functioning to include
relevant social determinants of health previously identified in
the initial study of COMPASS-CP. This included a measure ofhealth literacy [49], presence of a caregiver, desire for support
creating a living will, family and community support [48], anditems already included in the EHR record to assess financial,
food, and transportation challenges [48]. Furthermore, theoriginal COMPASS-CP for stroke care included an item tobriefly assess “What really matters to you and is a reason for
JMIR Form Res 2023 | vol. 7 | e41354 | p. 6
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
you to be optimally healthy?” that we adapted from a values
exercise [50]. Examples included in the description of eachvalue option provided were from prior work with cancer
survivors [24].
Other variables were extracted directly from the EHR, including
date of birth, birth sex (incorporated into logic regarding
recommendations for alcohol use), height, weight (used to
calculate BMI to inform recommendations regarding weight
management), and zip code (to inform resources provided).

Additional Stakeholder Feedback
Examples of other feedback that informed adaptations included
offering the patient-completed measures in multiple formats(paper and electronic) and allowing patients to choose the mostacceptable format. Patients said they would expect a provider
to use the information collected from patient-completed
measures to identify what is most important to talk about during
the clinical encounter. This was a change from the original
COMPASS-CP such that we adopted a self-report patient
assessment format (rather than a facilitated interview), which
particularly impacted our choice of cognitive assessment. A
self-reported cognitive assessment was considered more
appropriate for cancer survivors than neuropsychological tests
administered by the study team because stakeholder feedback
indicated that we should include only a brief cognitive
assessment, and cognitive difficulties specific to cancer
treatment are often too subtle for neuropsychological tests. Other
feedback included that provider decision support should clearly
indicate thresholds to help the interpretation of results. Multiple
other specific modifications were suggested and incorporated.
Positive feedback about the CP from health care professionals
included that they liked that it would be personalized and would
not contain extraneous information. Furthermore, stakeholders
suggested that the resource linking tool should include headingsthat explain what each resource will address. They also
appreciated the feature that allowed searching for resources by
zip code, given that help in identifying resources for patients
who are not local is an unmet need for survivorship careproviders.

Summary of CFIR Outer Setting
Patient Needs and Resources
Health care professionals were asked to discuss how well they
believed the tool would meet the needs of individuals servedby the organization. Most said it would serve the needs ofpatients because it will consolidate survivorship resources forthem in the areas where they reside, whereas 1 health care
professional thought it was oversimplified (eg, could do more
to normalize distress).
External Policies and Incentives
Health care providers suggested financial and other incentives
that would influence implementation and affect revenue suchas increased internal referrals, adding personalized care, seeing
more patients who are attracted to the service, conducting joint
services (eg, including life coaches or social workers), andsaving provider time. One provider was concerned about
fulfilling the responsibility to provide adequate and timely
https://formative.jmir.org/2023/1/e41354

XSL• FO
RenderX

Sohl et al
resources for a patient who scores high for depression or otherrelevant issues. One stakeholder commented as follows:
Saving time is really important. I think we can see
more patients and we cannot have to follow up a lot
after the visit, as much, with our own researching.
Trying to find resources for patients and following
up with them later. Doing it right there on the spot
during their visit would be really helpful and would
allow us to see more patients. [Participant ID 003]
Peer Pressure
Health care professionals were asked how implementing thetool would provide an advantage to the local cancer centercompared with other organizations and if there was something
specific that would attract more patients. Most said the tool may
provide an advantage to the organization by being known forproviding wellness resources to assist patients in their own
communities.

Summary of CFIR Inner Setting
CultureWhen asked to what extent new ideas are embraced and used
to make improvements in the organization, most health care
professionals said new ideas are embraced in their immediatearea; however, the organization as a whole is slower to embrace
new ideas.
Implementation Climate: Tension for Change
Health care professionals were asked if they believed there wasa strong need for the tool. Most participants said they saw aneed for the survivorship CP tool because survivorship is
growing, and the tool provides for a “survivorship pathway,”
removes burden from providers, and provides resources that fit
the needs of their patients. There were some concerns that they
are not required to provide a survivorship plan anymore and all
survivors may not need resources. Moreover, there wereconcerns about whether the tool would serve patients seen atother satellite clinics and concerns that working with
COMPASS-CP was motivated because it is already being usedby another part of the organization.
Implementation Climate: Compatibility
Health care professionals were asked how COMPASS-CP CFC
fits with their values and norms, and the values and norms within
the organization. Most participants said the tool fits in with theirvalues to provide holistic wellness, best practices, and quality
patient care. One participant commented as follows:
I definitely think it aligns with our organization’s
mission and personal mission. I think it’s just to beable to fully address their holistic needs, to care forthem beyond their immediate treatment, and tocontinue to take care of them related to side effects
that they may have from long or late-term toxicities,
it fits within the promise that we have to care for ourpatients. [Participant ID 004]
Implementation Climate: Learning
Most health care professionals said the climate is conducive totrying new things to improve work processes, yet they also do
JMIR Form Res 2023 | vol. 7 | e41354 | p. 7
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
not always have the time and energy to think about ways toimprove. Considerations for implementation of COMPASS-CP
CFC included the benefit to the patient, need for physician
buy-in, and context (eg, if a lot of change has happened
recently).
Readiness for Implementation: Leadership Engagement
Health care professionals suggested the best roles to champion
the tool would be nurse navigators, survivorship clinic providers,
and the survivorship clinic director. It would also be importantto have buy-in from other hematology/oncology providers, front
desk schedulers, and administrators. They thought it would help
with implementation to present the tool to the team only when
it is finalized and to follow-up with patients about usefulness
or challenges with community resources.

Sohl et al

Phase 3: Assessing the Usability of COMPASS-CP
CFC
The demographic characteristics of health care professionals(informatics professionals and health care providers; n=5) andpatients (n=5) who engaged in usability testing for
COMPASS-CP CFC are described in Tables 1 and 2. The
average System Usability Scale score was above our target scoreof 70 for both health care professionals and patients (Table 3),indicating an overall high ease of use. Yet, the range of scores
reported by health care professionals included values lower than
the target score, indicating the additional need for improved
usability. Participants also quantitatively reported high levelsof acceptability, appropriateness, and feasibility for
COMPASS-CP CFC (Table 3). The following qualitative
feedback guided by the CFIR elaborates on intervention andindividual characteristics to consider for improving usability
and for other aspects of future implementation planning.

Table 3. Summary of quantitative usability metrics.
Scale and participant group

Score
Mean (SD)MedianMinimum

Maximum

Health care professionals

71.0 (15.2)

70.0

50.0

90.0

Patients

95.5 (2.1)

95.0

92.5

97.5

Overall

83.3 (16.5)

91.3

50.0

97.5

Health care professionals

3.9 (0.3)

4.0

3.5

4.3

Patients

4.6 (0.6)

5.0

3.8

5.0

Overall

4.2 (0.6)

4.0

3.5

5.0

Health care professionals

4.1 (0.1)

4.0

4.0

4.3

Patients

4.7 (0.4)

5.0

4.0

5.0

Overall

4.4 (0.5)

4.1

4.0

5.0

Health care professionals

4.1 (0.3)

4.0

4.3

5.0

Patients

4.9 (0.3)

5.0

3.8

4.5

Overall

4.5 (0.5)

4.4

3.8

5.0

System Usability Scale (range 0-100)a

Acceptability of Intervention Measure (range 1-5)aIntervention Appropriateness Measure (range 1-5)aFeasibility of Intervention Measure (range 1-5)aaHigher scores indicate more agreement with the measured construct.

Intervention Design Quality and Packaging
Health care professionals offered suggestions regarding training
and support that would be needed to deliver COMPASS-CP.
Some explicitly requested a video recording or avideoconference, whereas others preferred written materials to
describe workflow instructions (1 person asked for both). They
also suggested giving context for why the CP is important,including if it contributes to patient satisfaction.
Training and support suggestions for future implementationfrom patients were also provided. It was mentioned that
COMPASS-CP was self-explanatory and would not require
https://formative.jmir.org/2023/1/e41354

XSL• FO
RenderX

training. Additionally, providing a support phone number totalk with someone if needing help would be useful, andproviding instructions from the onset to explain how thepatient-completed measures will be used (to generate a CP)
would be supportive. They also noted that older patients mayneed more support.

Individual Knowledge and Beliefs About the Intervention
Health care professionals indicated preferences to facilitate easeof use of COMPASS-CP in the EHR. For example, participantssuggested saving a summary of the CP in a clinic note with alink for the full plan as part of a clinical encounter. They would
JMIR Form Res 2023 | vol. 7 | e41354 | p. 8
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
like to be able to edit it, and therefore, a PDF file would notwork well. Moreover, providers said they would not open asaved PDF file. They emphasized the importance of embedding
the tool within the provider workflow and emphasized that apatient’s primary care provider would need to be copied on aCP. Another health care professional commented that
COMPASS-CP could be used to prompt research in addition
to the CP and clinical action.
Patients said it would be helpful to refer to the plan regularly(eg, monthly) to track their goals and progress. Putting this planin writing would serve as a helpful reminder of their goals. In
addition, having the plan in the medical chart would provide
motivation not to ignore it.

Additional Usability Feedback
Both health care professionals and patients indicated that it
would be important to limit the number of resources provided,so recipients would not be overwhelmed. To do this, health care
professionals said there would need to be enough information
included in the resource list for a provider to makerecommendations. Patients would also like to have providers
review the CP with them.

Discussion
Principal Findings
The adaptation of COMPASS-CP CFC was rated as usable byhealth care professionals and patients who were colorectal
cancer survivors. COMPASS-CP provides flexibility to compile
assessments from different locations (eg, PROs and EHR data)and provides tools to help with interpretation of the datacollected. COMPASS-CP also links identified concerns withpersonalized community resources. Incorporating validated
PROMIS measures is an important strength of COMPASS-CP
CFC since this measurement system has many advantages,
including its ability to function as both a screening tool and atool for assessing clinical outcomes [51-53]. We also enhanced
the original COMPASS-CP tool by successfully integrating
REDCap as the patient-facing interface, and this is particularly
important because REDCap streamlines our ability to send
surveys to patients, which they can complete independently (asopposed to a facilitated telephone interview), and allows
investigators to easily make modifications. It is also possible
for patients to complete the assessments on paper and for
responses to be entered by clinical staff.
The COMPASS-CP CFC tool facilitates integrating
patient-completed assessments in clinical care, which is relevant
to improving patient engagement and patient-provider
communication [54,55]. Patient feedback in this study and otherstudies indicated that providers are expected to discuss theconcerns raised in measures completed by patients and that thiswould influence the selection of the personalized recommended
actions in the CP. COMPASS-CP CFC has the potential toreduce the time it takes providers to comprehensively assess
and prioritize patient concerns during a clinic visit, thus
facilitating more time for coaching to empower patients’
self-management. Supporting patients to increasingly engage
in self-management and connecting them with relevant resources
https://formative.jmir.org/2023/1/e41354

XSL• FO
RenderX

Sohl et al
are important components of proposed systematic changes thatinvolve risk stratifying the delivery of follow-up cancer care
[56,57]. COMPASS-CP CFC could facilitate access and referral
to recommended supportive interventions that are often notintegrated into standard care.
Health care professionals indicated that the ability of
COMPASS-CP CFC to link patients’ concerns with resources
near their place of residence was particularly valuable since it
was an unmet need that could potentially save them time. Yet,
health care professionals were also concerned about fulfilling
the responsibility to provide adequate and timely resources forthe concerns identified (eg, a patient who scores high on
concerns such as depression). Thus, we considered that it wasparticularly important for each patient-completed measureincluded in COMPASS-CP CFC to be linked to an actionable
resource. The next steps may include integrating COMPASS-CP
CFC with an existing national database of community resourcesthat is increasingly used for addressing social determinants ofhealth to support the feasibility of maintaining current andup-to-date resources [58]. This integration would further
enhance scalability.
Incorporating stakeholder feedback in this adaptation process
is a critical step to ensure that COMPASS-CP CFC will be
successfully implemented in future work [59]. Data collectedusing the CFIR will particularly be applied to inform theintegration of COMPASS-CP CFC into clinical care. As
currently designed, for the use of COMPASS-CP CFC, it will
be important to include a shared decision-making discussion
with a health care professional when selecting and providing
resources to ensure that patients’ concerns are normalized andappropriately prioritized [60]. Future research will explore theappropriate professional for this role. As suggested by astakeholder, this professional may be a health coach or social
worker who is integrated with clinical care. The appropriate
professional may also vary based on the local clinical context.
The next steps for this line of work include pilot testing
COMPASS-CP CFC when integrated within clinical care toassess further usability and effectiveness. It will be important
to assess the appropriate timing to implement COMPASS-CP
CFC in the follow-up care process. Moreover, it will be
important to consider how to follow-up with patients andencourage ongoing adherence to CPs.

Limitations
A limitation of this study includes the range of usability scores
reported by health care professionals, with some scores being
below the target score. This indicates that further work is needed
to improve usability. Future studies may incorporate additional
usability feedback from health care professionals to further
optimize COMPASS-CP CFC. In addition, COMPASS-CP CFC
has not yet been tested in the flow of clinical care, which will
be an imperative next step for tool refinement. Prior researchon electronic care planning support interventions suggests that
adopting COMPASS-CP CFC at multiple time points as partof a dynamic care planning process will result in the largest
impact [15]. Furthermore, although this study was focused onfollow-up care specific to colorectal cancer, we designed
COMPASS-CP CFC to be easily adapted to other cancer types
JMIR Form Res 2023 | vol. 7 | e41354 | p. 9
(page number not for citation purposes)

JMIR FORMATIVE RESEARCH(by modifying the colorectal cancer–specific items). We choseto begin with colorectal cancer because these patients commonly
have a good prognosis with a high need for follow-up care andthis cancer type is prevalent in both men and women. Therefore,
the next steps include broadening stakeholder involvement andconsidering patients diagnosed with other types of cancers.

Conclusion
We gathered stakeholder feedback to inform our iterative
adaptation of COMPASS-CP CFC to facilitate patientengagement in their follow-up cancer care. This electronic tool
is integrated with the EHR and provides flexibility toSohl et alcomprehensively compile assessments from different locations(including PROs), add assistance with interpretation of the datacollected, and link concerns identified with personalized
resources. This study supports the initial usability of
COMPASS-CP CFC and will inform the next steps for
implementation in clinical care. COMPASS-CP has the potential
to be a widely applicable and scalable tool for personalizing
and streamlining comprehensive follow-up cancer care. The
implementation of this tool in coordination with clinical care
may ultimately facilitate sustainable improvements in patient
health outcomes.

Acknowledgments
The project described was supported by the National Cancer Institute’s Cancer Center Support Grant (award number P30CA012197)
issued to the Wake Forest Baptist Comprehensive Cancer Center (including the Bioinformatics, Qualitative and Patient-Reported
Outcomes, and Biostatistics Shared Resources). The content is solely the responsibility of the authors and does not necessarily
represent the official views of the National Institutes of Health.

Data Availability
The data sets generated during or analyzed during this study are available from the corresponding author upon reasonable request.

Conflicts of Interest
ET was on an advisory board for Mahana Therapuetics. The other authors have no conflicts to declare.

References
1.
2.
3.
4.

5.

6.
7.

8.

9.

10.

11.

Miller K, Nogueira L, Devasia T, Mariotto A, Yabroff K, Jemal A, et al. Cancer treatment and survivorship statistics, 2022.
CA Cancer J Clin 2022 Sep;72(5):409-436 [FREE Full text] [doi: 10.3322/caac.21731] [Medline: 35736631]
Cancer Facts and Figures 2019. American Cancer Society. URL: https://www.cancer.org/research/cancer-facts-statistics/
all-cancer-facts-figures/cancer-facts-figures-2019.html [accessed 2022-12-01]
Institute of Medicine and National Research Council. From Cancer Patient to Cancer Survivor: Lost in Transition. Washington,
DC: The National Academies Press; 2006.
Kline R, Arora N, Bradley C, Brauer E, Graves D, Lunsford N, et al. Long-Term Survivorship Care After Cancer Treatment- Summary of a 2017 National Cancer Policy Forum Workshop. J Natl Cancer Inst 2018 Dec 01;110(12):1300-1310 [FREEFull text] [doi: 10.1093/jnci/djy176] [Medline: 30496448]
van de Poll-Franse LV, Nicolaije KAH, Ezendam NPM. The impact of cancer survivorship care plans on patient and healthcare provider outcomes: a current perspective. Acta Oncol 2017 Mar;56(2):134-138. [doi: 10.1080/0284186X.2016.1266080][Medline: 28084140]
Commission on Cancer. Optimal Resources for Cancer Care: 2020 Standards. American College of Surgeons. 2020. URL:
https://www.facs.org/media/xlsey5iv/optimal_resources_for_cancer_care_2020_standards.pdf [accessed 2022-12-01]
Jacobsen PB, DeRosa AP, Henderson TO, Mayer DK, Moskowitz CS, Paskett ED, et al. Systematic Review of the Impact
of Cancer Survivorship Care Plans on Health Outcomes and Health Care Delivery. J Clin Oncol 2018 Jul 10;36(20):2088-2100[FREE Full text] [doi: 10.1200/JCO.2018.77.7482] [Medline: 29775389]
Keesing S, McNamara B, Rosenwax L. Cancer survivors' experiences of using survivorship care plans: a systematic reviewof qualitative studies. J Cancer Surviv 2015 Jun;9(2):260-268 [FREE Full text] [doi: 10.1007/s11764-014-0407-x] [Medline:
25343971]
Kvale EA, Huang CS, Meneses KM, Demark-Wahnefried W, Bae S, Azuero CB, et al. Patient-centered support in thesurvivorship care transition: Outcomes from the Patient-Owned Survivorship Care Plan Intervention. Cancer 2016 Oct
15;122(20):3232-3242 [FREE Full text] [doi: 10.1002/cncr.30136] [Medline: 27387096]
Duncan PW, Abbott RM, Rushing S, Johnson AM, Condon CN, Lycan SL, COMPASS Investigative Team. COMPASS-CP:An Electronic Application to Capture Patient-Reported Outcomes to Develop Actionable Stroke and Transient Ischemic
Attack Care Plans. Circ Cardiovasc Qual Outcomes 2018 Aug;11(8):e004444 [FREE Full text] [doi:10.1161/CIRCOUTCOMES.117.004444] [Medline: 30354371]
Bushnell CD, Duncan PW, Lycan SL, Condon CN, Pastva AM, Lutz BJ, COMPASS Trial.. A Person-Centered Approachto Poststroke Care: The COMprehensive Post-Acute Stroke Services Model. J Am Geriatr Soc 2018 May 23;66(5):1025-1030[FREE Full text] [doi: 10.1111/jgs.15322] [Medline: 29572814]

https://formative.jmir.org/2023/1/e41354

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e41354 | p. 10(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
12.

13.

14.
15.

16.
17.

18.

19.
20.

21.

22.
23.

24.

25.

26.

27.

28.

29.

30.
31.

32.

Duncan PW, Bushnell CD, Jones SB, Psioda MA, Gesell SB, D'Agostino RB, COMPASS Site InvestigatorsTeams..
Randomized Pragmatic Trial of Stroke Transitional Care: The COMPASS Study. Circ Cardiovasc Qual Outcomes 2020
Jun;13(6):e006285 [FREE Full text] [doi: 10.1161/CIRCOUTCOMES.119.006285] [Medline: 32475159]
Salz T, McCabe MS, Oeffinger KC, Corcoran S, Vickers AJ, Salner AL, et al. A head and neck cancer intervention for use
in survivorship clinics: a protocol for a feasibility study. Pilot Feasibility Stud 2016;2:23 [FREE Full text] [doi:10.1186/s40814-016-0061-3] [Medline: 27965842]
Survivorship Care Planning Tools. American Society of Clinical Oncology. URL: https://old-prod.asco.org/node/9931/[accessed 2022-12-01]
Mikles SP, Griffin AC, Chung AE. Health information technology to support cancer survivorship care planning: A systematic
review. J Am Med Inform Assoc 2021 Sep 18;28(10):2277-2286 [FREE Full text] [doi: 10.1093/jamia/ocab134] [Medline:
34333588]
Brant J, Hirschman K, Keckler S, Dudley W, Stricker C. Patient and Provider Use of Electronic Care Plans Generated From
Patient-Reported Outcomes. ONF 2019 Nov 1;46(6):715-726. [doi: 10.1188/19.onf.715-726]
Gesell SB, Bushnell CD, Jones SB, Coleman SW, Levy SM, Xenakis JG, et al. Implementation of a billable transitionalcare model for stroke patients: the COMPASS study. BMC Health Serv Res 2019 Dec 19;19(1):978 [FREE Full text] [doi:10.1186/s12913-019-4771-0] [Medline: 31856808]
Taplin SH, Anhang Price R, Edwards HM, Foster MK, Breslau ES, Chollette V, et al. Introduction: Understanding andinfluencing multilevel factors across the cancer care continuum. J Natl Cancer Inst Monogr 2012 May;2012(44):2-10 [FREEFull text] [doi: 10.1093/jncimonographs/lgs008] [Medline: 22623590]
Weaver KE, Geiger AM, Lu L, Case LD. Rural-urban disparities in health status among US cancer survivors. Cancer 2013
Mar 01;119(5):1050-1057 [FREE Full text] [doi: 10.1002/cncr.27840] [Medline: 23096263]
Identifying underserved populations. Wake Forest Baptist Comprehensive Cancer Center Office of Health Equity Newsletter.
2019. URL: https://myemail.constantcontact.com/Jan-2019-Newsletter---Identifying-Underserved-Populations.
html?soid=1129938688361&aid=imzgeuiJc7A [accessed 2022-12-01]
Singh GK, Williams SD, Siahpush M, Mulhollen A. Socioeconomic, Rural-Urban, and Racial Inequalities in US Cancer
Mortality: Part I-All Cancers and Lung Cancer and Part II-Colorectal, Prostate, Breast, and Cervical Cancers. J Cancer
Epidemiol 2011;2011:107497 [FREE Full text] [doi: 10.1155/2011/107497] [Medline: 22496688]
Van Blarigan EL, Meyerhardt JA. Role of physical activity and diet after colorectal cancer diagnosis. J Clin Oncol 2015
Jun 01;33(16):1825-1834 [FREE Full text] [doi: 10.1200/JCO.2014.59.7799] [Medline: 25918293]
Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services
research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009 Aug
07;4:50 [FREE Full text] [doi: 10.1186/1748-5908-4-50] [Medline: 19664226]
Sohl SJ, Befus D, Tooze JA, Levine B, Golden SL, Puccinelli-Ortega N, et al. Feasibility of Systems Support Mapping to
guide patient-driven health self-management in colorectal cancer survivors. Psychol Health 2021 Sep 27:1-21. [doi:10.1080/08870446.2021.1979549] [Medline: 34570677]
Sohl SJ, Birdee G, Elam R. Complementary Tools to Empower and Sustain Behavior Change: Motivational Interviewing
and Mindfulness. Am J Lifestyle Med 2016 Nov;10(6):429-436 [FREE Full text] [doi: 10.1177/1559827615571524][Medline: 28239308]
Mummah SA, Robinson TN, King AC, Gardner CD, Sutton S. IDEAS (Integrate, Design, Assess, and Share): A Framework
and Toolkit of Strategies for the Development of More Effective Digital Interventions to Change Health Behavior. J MedInternet Res 2016 Dec 16;18(12):e317 [FREE Full text] [doi: 10.2196/jmir.5927] [Medline: 27986647]
Miller DP, Weaver KE, Case LD, Babcock D, Lawler D, Denizard-Thompson N, et al. Usability of a Novel Mobile Health
iPad App by Vulnerable Populations. JMIR Mhealth Uhealth 2017 Apr 11;5(4):e43 [FREE Full text] [doi:10.2196/mhealth.7268] [Medline: 28400354]
Miller DP, Spangler JG, Case LD, Goff DC, Singh S, Pignone MP. Effectiveness of a web-based colorectal cancer screening
patient decision aid: a randomized controlled trial in a mixed-literacy population. Am J Prev Med 2011 Jun;40(6):608-615[FREE Full text] [doi: 10.1016/j.amepre.2011.02.019] [Medline: 21565651]
Fernandez ME, Walker TJ, Weiner BJ, Calo WA, Liang S, Risendal B, et al. Developing measures to assess constructs
from the Inner Setting domain of the Consolidated Framework for Implementation Research. Implement Sci 2018 Mar
27;13(1):52 [FREE Full text] [doi: 10.1186/s13012-018-0736-7] [Medline: 29587804]
Important information regarding CoC Survivorship Care Plan Standard. American College of Surgeons. URL: https://www.
facs.org/quality-programs/cancer/news/survivorship [accessed 2019-01-10]
El-Shami K, Oeffinger KC, Erb NL, Willis A, Bretsch JK, Pratt-Chapman ML, et al. American Cancer Society Colorectal
Cancer Survivorship Care Guidelines. CA Cancer J Clin 2015;65(6):428-455 [FREE Full text] [doi: 10.3322/caac.21286][Medline: 26348643]
Sanft T, Denlinger CS, Armenian S, Baker KS, Broderick G, Demark-Wahnefried W, et al. NCCN Guidelines Insights:
Survivorship, Version 2.2019. J Natl Compr Canc Netw 2019 Jul 01;17(7):784-794 [FREE Full text] [doi:10.6004/jnccn.2019.0034] [Medline: 31319383]

https://formative.jmir.org/2023/1/e41354

XSL• FO
RenderX

Sohl et al

JMIR Form Res 2023 | vol. 7 | e41354 | p. 11(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
33.

34.
35.
36.
37.

38.

39.
40.
41.

42.

43.

44.

45.

46.

47.

48.
49.

50.
51.

52.

53.

Press A, McCullagh L, Khan S, Schachter A, Pardo S, McGinn T. Usability Testing of a Complex Clinical Decision Support
Tool in the Emergency Department: Lessons Learned. JMIR Hum Factors 2015 Sep 10;2(2):e14 [FREE Full text] [doi:10.2196/humanfactors.4537] [Medline: 27025540]
Recruiting Usability Test Participants. Usability.gov. URL: https://www.usability.gov/how-to-and-tools/methods/
recruiting-usability-test-participants.html [accessed 2022-12-01]
Moderating Usability Tests. Usability.gov. 2013. URL: https://www.usability.gov/get-involved/blog/2013/04/
moderating-usability-tests.html [accessed 2022-12-01]
Bangor A, Kortum P, Miller J. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. Journalof Usability Studies 2009;4(3):114-123 [FREE Full text]
Everett E, Kane B, Yoo A, Dobs A, Mathioudakis N. A Novel Approach for Fully Automated, Personalized Health Coachingfor Adults with Prediabetes: Pilot Clinical Trial. J Med Internet Res 2018 Mar 27;20(2):e72 [FREE Full text] [doi:10.2196/jmir.9723] [Medline: 29487046]
Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed
implementation outcome measures. Implement Sci 2017 Aug 29;12(1):108 [FREE Full text] [doi:10.1186/s13012-017-0635-3] [Medline: 28851459]
Qualitative Data. Consolidated Framework for Implementation Research. URL: https://cfirguide.org/evaluation-design/qualitative-data/ [accessed 2022-12-01]
Green J, Thorogood N. Qualitative Methods for Health Research. Thousand Oaks, CA: SAGE Publications Ltd; 2018.
Rothrock NE, Cook KF, O'Connor M, Cella D, Smith AW, Yount SE. Establishing clinically-relevant terms and severity
thresholds for Patient-Reported Outcomes Measurement Information System (PROMIS) measures of physical function,
cognitive function, and sleep disturbance in people with cancer using standard setting. Qual Life Res 2019 Dec
13;28(12):3355-3362 [FREE Full text] [doi: 10.1007/s11136-019-02261-2] [Medline: 31410640]
Flynn KE, Lin L, Cyranowski JM, Reeve BB, Reese JB, Jeffery DD, et al. Development of the NIH PROMIS ® Sexual
Function and Satisfaction measures in patients with cancer. J Sex Med 2013 Mar;10 Suppl 1(0 1):43-52 [FREE Full text][doi: 10.1111/j.1743-6109.2012.02995.x] [Medline: 23387911]
Dueck AC, Mendoza TR, Mitchell SA, Reeve BB, Castro KM, Rogak LJ, National Cancer Institute PRO-CTCAE Study
Group. Validity and Reliability of the US National Cancer Institute's Patient-Reported Outcomes Version of the Common
Terminology Criteria for Adverse Events (PRO-CTCAE). JAMA Oncol 2015 Nov;1(8):1051-1059 [FREE Full text] [doi:10.1001/jamaoncol.2015.2639] [Medline: 26270597]
NCI-AACR Cancer Patient Tobacco Use Assessment Task Force. Cancer Patient Tobacco Use Questionnaire (C-TUQ).
NCI Division of Cancer Control and Population Sciences. URL: https://cancercontrol.cancer.gov/sites/default/files/2021-02/
C-TUQ_User_Manual.pdf [accessed 2022-12-01]
Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA. The AUDIT alcohol consumption questions (AUDIT-C): aneffective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol
Use Disorders Identification Test. Arch Intern Med 1998 Sep 14;158(16):1789-1795. [doi: 10.1001/archinte.158.16.1789][Medline: 9738608]
Yaroch AL, Tooze J, Thompson FE, Blanck HM, Thompson OM, Colón-Ramos U, et al. Evaluation of three short dietary
instruments to assess fruit and vegetable intake: the National Cancer Institute's food attitudes and behaviors survey. J Acad
Nutr Diet 2012 Oct;112(10):1570-1577 [FREE Full text] [doi: 10.1016/j.jand.2012.06.002] [Medline: 23017567]
Coleman KJ, Ngor E, Reynolds K, Quinn VP, Koebnick C, Young DR, et al. Initial validation of an exercise "vital sign"in electronic medical records. Med Sci Sports Exerc 2012 Nov;44(11):2071-2076. [doi: 10.1249/MSS.0b013e3182630ec1][Medline: 22688832]
The Accountable Health Communities Health-Related Social Needs Screening Tool. Center for Medicare and Medicaid
Innovation. URL: https://innovation.cms.gov/files/worksheets/ahcm-screeningtool.pdf [accessed 2022-05-05]
Morris NS, MacLean CD, Chew LD, Littenberg B. The Single Item Literacy Screener: evaluation of a brief instrument to
identify limited reading ability. BMC Fam Pract 2006 Mar 24;7:21 [FREE Full text] [doi: 10.1186/1471-2296-7-21][Medline: 16563164]
Harris R. The Complete Set of Client Handouts and Worksheets from ACT books. The Happiness Trap. URL: https://thehappinesstrap.com/upimages/Complete_Worksheets_2014.pdf [accessed 2022-12-01]
Cella D, Choi S, Garcia S, Cook KF, Rosenbloom S, Lai J, et al. Setting standards for severity of common symptoms inoncology using the PROMIS item banks and expert judgment. Qual Life Res 2014 Dec;23(10):2651-2661 [FREE Full text][doi: 10.1007/s11136-014-0732-6] [Medline: 24938431]
Wagner LI, Schink J, Bass M, Patel S, Diaz MV, Rothrock N, et al. Bringing PROMIS to practice: brief and precise symptom
screening in ambulatory cancer care. Cancer 2015 Mar 15;121(6):927-934 [FREE Full text] [doi: 10.1002/cncr.29104][Medline: 25376427]
Garcia SF, Wortman K, Cella D, Wagner LI, Bass M, Kircher S, et al. Implementing electronic health record-integrated
screening of patient-reported symptoms and supportive care needs in a comprehensive cancer center. Cancer 2019 Nov
15;125(22):4059-4068 [FREE Full text] [doi: 10.1002/cncr.32172] [Medline: 31373682]

https://formative.jmir.org/2023/1/e41354

XSL• FO
RenderX

Sohl et al

JMIR Form Res 2023 | vol. 7 | e41354 | p. 12(page number not for citation purposes)

JMIR FORMATIVE RESEARCH
54.

55.

56.

57.
58.
59.
60.

Sohl et al

Yang LY, Manhas DS, Howard AF, Olson RA. Patient-reported outcome use in oncology: a systematic review of the impact
on patient-clinician communication. Support Care Cancer 2018 Jan;26(1):41-60. [doi: 10.1007/s00520-017-3865-7] [Medline:
28849277]
Lordon RJ, Mikles SP, Kneale L, Evans HL, Munson SA, Backonja U, et al. How patient-generated health data andpatient-reported outcomes affect patient-clinician relationships: A systematic review. Health Informatics J 2020
Dec;26(4):2689-2706 [FREE Full text] [doi: 10.1177/1460458220928184] [Medline: 32567460]
Mayer D, Alfano C. Personalized Risk-Stratified Cancer Follow-Up Care: Its Potential for Healthier Survivors, HappierClinicians, and Lower Costs. J Natl Cancer Inst 2019 May 01;111(5):442-448 [FREE Full text] [doi: 10.1093/jnci/djy232][Medline: 30726949]
Brauer ER, Ganz PA. Moving the Translational Needle in Breast Cancer Survivorship: Connecting Intervention Researchto Clinical Practice. JCO 2022 Jul 01;40(19):2069-2073. [doi: 10.1200/jco.22.00174]
Findhelp. URL: https://company.findhelp.com/ [accessed 2022-12-01]
Chambers DA, Norton WE. The Adaptome: Advancing the Science of Intervention Adaptation. Am J Prev Med 2016
Oct;51(4 Suppl 2):S124-S131 [FREE Full text] [doi: 10.1016/j.amepre.2016.05.011] [Medline: 27371105]
Makoul G, Clayman ML. An integrative model of shared decision making in medical encounters. Patient Educ Couns 2006
Mar;60(3):301-312. [doi: 10.1016/j.pec.2005.06.010] [Medline: 16051459]

Abbreviations
API: application programming interface
CFC: cancer follow-up care
CFIR: Consolidated Framework for Implementation Research
CP: care plan
EHR: electronic health record
FHIR: Fast Health Interoperability Resources
IDEAS: Integrate, Design, Assess, and Share
PRO: patient-reported outcome
PROMIS: Patient-Reported Outcomes Measurement Information System
SMART: Substitutable Medical Applications and Reusable Technologies

Edited by A Mavragani; submitted 22.07.22; peer-reviewed by R Marshall, E Ray Chaudhuri, S Gelcich; comments to author 09.09.22;
revised version received 03.11.22; accepted 29.11.22; published 10.01.23
Please cite as:
Sohl SJ, Duncan PW, Thakur E, Puccinelli-Ortega N, Salsman JM, Russell G, Pasche BC, Wentworth S, Miller Jr DP, Wagner LI,
Topaloglu U
Adaptation of a Personalized Electronic Care Planning Tool for Cancer Follow-up Care: Formative StudyJMIR Form Res 2023;7:e41354
URL: https://formative.jmir.org/2023/1/e41354
doi: 10.2196/41354
PMID: 36626203©Stephanie J Sohl, Pamela W Duncan, Elyse Thakur, Nicole Puccinelli-Ortega, John M Salsman, Greg Russell, Boris C Pasche,
Stacy Wentworth, David P Miller Jr, Lynne I Wagner, Umit Topaloglu. Originally published in JMIR Formative Research(https://formative.jmir.org), 10.01.2023. This is an open-access article distributed under the terms of the Creative Commons
Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction
in any medium, provided the original work, first published in JMIR Formative Research, is properly cited. The complete
bibliographic information, a link to the original publication on https://formative.jmir.org, as well as this copyright and license
information must be included.

https://formative.jmir.org/2023/1/e41354

XSL• FO
RenderX

JMIR Form Res 2023 | vol. 7 | e41354 | p. 13(page number not for citation purposes)