Telehealth Based Synchronous Navigation to Improve Molecularly-Informed Care for Patients With Lung Cancer
Phase: NA
Conditions: Non-small Cell Lung Cancer
Sponsor: Abramson Cancer Center at Penn Medicine

TESTING 
1 
 
 
 
TESTING: TElehealth based Synchronous navigation To Improve 
molecularly-informed care for patients with luNG cancer 
 
 
 
Principal Investigator: 
 
 
Charu Aggarwal, MD, MPH 
 
 
 
 
 
 
Associate Professor of Medicine 
 
 
 
 
 
 
charu.aggarwal@pennmedicine.upenn.edu 
 
 
 
 
 
 
(215) 662-6318 
 
 
Funding Agency: 
 
 
 
National Institutes of Health 
 
NIH Grant Number:  
 
 
P50CA271338 
 
IRB Protocol Number: 
 
 
852857 
 
UPCC Protocol Number: 
 
 
22522 
 
ClinicalTrials.gov Identifier: 
 
NCT05790460 
 
Version Date: 
 
 
 
August 18, 2023 (Version 2) 
 
 
 
 
 

TESTING 
2 
 
Protocol Summary 
Title 
TESTING: TElehealth based Synchronous navigation To 
Improve molecularly-informed care for patients with luNG 
cancer 
Short Title 
TESTING 
Principal 
Investigator 
Charu Aggarwal, MD, MPH 
Co-Investigators 
Jeffrey Thompson, MD, MTR 
Anil Vachani, MD, MS  
Justin Bekelman, MD 
Shivan Mehta, MD, MBA 
Katharine Rendle, PhD, MSW, MPH 
Marilyn Schapira, MD, MPH 
Alisa Stephens-Shields, PhD 
Andy Tan, PhD, MPH, MBA, MBBS 
Megan Roy MSN, RN, OCN 
Christoph Hutchinson, MD 
Kevin Ma, MD 
David DiBardino, MD 
Andrew Haas, MD, PhD 
 
Design 
Randomized, two-arm clinical trial 
Objectives 
1. In a randomized pilot trial of patients with suspected locally 
advanced/metastatic NSCLC, test the effectiveness of enhanced 
synchronous telehealth nurse navigation, compared to usual 
care nurse navigation, to increase timely molecularly-informed 
treatment recommendations through early integration of 
concurrent molecular testing. 
 
2. Evaluate contextual mechanisms contributing to the 
effectiveness of synchronous telehealth nurse navigation. 
Trial Duration 
30 months 
Study Sites 
University of Pennsylvania Health System (UPHS) sites: 
Hospital of the University of Pennsylvania Pavilion (HUP) – East 
Hospital of the University of Pennsylvania (HUP) 
Pennsylvania Hospital (PAH) 
Penn Presbyterian Medical Center (PPMC) 
Sample Size 
Objective 1: 138 patients 
Objective 2: 20-30 patients and 10-15 clinicians/clinical leaders 
Patient Eligibility 
Patients will be eligible for the randomized clinical trial if they: 
a) are aged 18 years or older 

TESTING 
3 
 
b) based on cross-sectional imaging, are suspected to have 
locally advanced/metastatic NSCLC (as determined by the 
evaluating clinician) 
c) are scheduled for an appointment in the lung cancer 
evaluation clinic. 
Clinician Eligibility 
Clinical leaders and clinicians (including physicians, advanced 
practice providers, and other clinicians) who treat lung cancer at 
Penn Medicine. 
Interventions 
Eligible patients will be individually randomized to one of two arms: 
1. Usual care – standard nurse navigation. 
2. Enhanced nurse navigation – synchronous telehealth nurse 
navigation visit prior to biopsy. 
Outcomes 
Objective 1: Receipt of a molecularly-informed treatment 
recommendation for patients with metastatic NSq NSCLC at the time 
of the patient’s initial oncology visit. 
 
Secondary Outcomes: 
1. Rate of telehealth visit completion 
2. Rate of completion of comprehensive molecular testing 
(tissue and/or plasma testing) prior to initiation of first line 
therapy 
3. Identification of one or more targetable mutations 
4. Timeliness of molecularly-informed treatment 
recommendation 
5. Overall survival 
6. Intervention costs 
7. Proportion of patients with diagnosis other than metastatic 
nonsquamous NSCLC (Telehealth arm only) 
8. Time to treatment initiation 
 
Objective 2: Individual and contextual factors (e.g., treatment 
knowledge, medical mistrust, clinical beliefs, organizational climate, 
patient-centered communication, financial toxicity, and knowledge of 
genetic testing) shaping trial effectiveness and patient experience. 
Primary Analysis 
Objective 1: Intention-to-treat (ITT) analyses for effectiveness using 
logistic regression. 
 
Objective 2: Mixed methods evaluation of descriptive survey data 
and thematically coded interview data. 
Study Oversight 
Dr. Aggarwal (PI) will be responsible for monitoring the trial and 
ensuring subject safety, as well as the integrity of the interventions 
and the data collected. Trial oversight will be conducted by the 
University of Pennsylvania Institutional Review Board. 
 
 

TESTING 
4 
 
Contents 
Principal Investigator: .............................................................................................................. 1 
Funding Agency: ....................................................................................................................... 1 
Protocol Summary .................................................................................................................... 2 
1. 
BACKGROUND AND RATIONALE .................................................................................... 5 
2. 
OBJECTIVES ...................................................................................................................... 6 
2.1 
Primary Objectives ..................................................................................................... 6 
2.2 
Primary Outcomes ...................................................................................................... 6 
2.3 
Secondary Outcomes ................................................................................................. 6 
3. 
STUDY POPULATION ........................................................................................................ 6 
3.1 
Target Population ....................................................................................................... 6 
3.2 
Inclusion Criteria ........................................................................................................ 6 
4. 
STUDY DESIGN .................................................................................................................. 7 
4.1 
Preliminary Studies .................................................................................................... 7 
4.2 
Overview ...................................................................................................................... 7 
4.3 
Study Duration and Timeline ................................................................................... 10 
4.4 
Study Setting............................................................................................................. 10 
5. 
STUDY PROCEDURES..................................................................................................... 11 
5.1 
Recruitment and Retention ...................................................................................... 11 
5.2 
Informed Consent ..................................................................................................... 11 
5.3 
Compensation ........................................................................................................... 12 
5.4 
Measures and Endpoints.......................................................................................... 13 
5.5 
Sources of Materials ................................................................................................. 14 
6. 
STATISTICAL DESIGN AND POWER .............................................................................. 14 
6.1 
Sample Size ............................................................................................................... 14 
6.2 
Analysis Plan ............................................................................................................ 14 
6.2.1 
Objective 1 Primary Analyses ........................................................................... 14 
6.2.2 
Objective 1 Secondary Analyses ...................................................................... 15 
6.2.3 
Objective 1 Exploratory Analyses .................................................................... 15 
6.2.4 
Objective 2 Analyses ......................................................................................... 15 
7. 
DATA MANAGEMENT AND RISK-BENEFIT ASSESSMENT .......................................... 16 
7.1 
Data and Safety Monitoring Plan ............................................................................. 16 
7.2 
Potential Risks .......................................................................................................... 17 
7.3 
Protections Against Risk.......................................................................................... 17 
7.4 
Potential Benefits ..................................................................................................... 18 
7.5 
Importance of the Knowledge to be Gained ............................................................ 19 
7.6 
Risk-Benefit Ratio ..................................................................................................... 19 
8. 
RESOURCES NECESSARY FOR HUMAN RESEARCH PROTECTION .......................... 19 
9. 
STUDY TEAM ................................................................................................................... 19 
10. REFERENCES .................................................................................................................. 20 
11. APPENDICES ................................................................................................................... 22 
 
 
 

TESTING 
5 
 
1. BACKGROUND AND RATIONALE 
The present pilot project is an initiative being conducted by the University of Pennsylvania Telehealth 
Research Center of Excellence (TRACE). TRACE strives to advance care delivery research through 
design and testing of innovative telehealth strategies to improve effectiveness and equity across the 
cancer care continuum.  
Precision cancer medicine holds great promise for improving lung cancer outcomes and survival, but its 
promise will only be realized if patients have broad access to a singular and essential precision 
component: molecular testing.1 For non-small cell lung cancer (NSCLC), the past decade has been 
marked by a steady increase in the number of therapeutically actionable molecular targets used to guide 
management.2,3 These therapeutically actionable mutations are present in approximately 30-40% of 
patients with non-squamous (NSq) NSCLC. At present, over a dozen orally administered targeted 
therapy regimens for metastatic NSq NSCLC require molecular testing for specific targetable genomic 
alterations.4 National guidelines currently recommend that comprehensive molecular testing be 
performed for all newly diagnosed metastatic NSq NSCLC patients as standard of care.4,5 Despite the 
importance of molecular testing in patients with metastatic NSq NSCLC, considerable barriers to timely 
completion of comprehensive molecular testing prior to initiation of systemic therapy exist, with 
concerning evidence of disparities by race. In a real-world analysis of 14,768 patients, next-generation 
sequencing (NGS) was performed among 50.1% of white patients and only 39.8% of Black patients (p < 
0.0001), even though prevalence of mutations does not vary by race.6 
Testing of tumor tissue has traditionally been the preferred source of molecular testing, but tumor tissue 
may be insufficient or unavailable for testing due to factors including location of disease or quality of 
biopsy.7,8 Even when feasible, the “turn-around-time” for tumor tissue testing frequently requires 3-4 
weeks.7 These barriers hinder the extent to which patients and clinicians can make guideline-concordant 
treatment decisions at the time of the initial in-person medical oncology visit, delaying initiation of cancer 
therapy and diminishing both patient and clinician satisfaction and quality of care. For metastatic NSq 
NSCLC, treatment initiation without molecular testing or treatment delays of any duration can be 
consequential, degrading the effectiveness of life-extending therapy, and can weigh heavily on patients 
and clinicians.9,10 Our team, and others, have shown that the addition of concurrent plasma-based 
molecular testing may help address the crucial molecular under-testing gap by offering an approach that 
is fast and convenient, especially for patients where tissue is unavailable or inaccessible. Plasma-based 
molecular testing is a complementary mechanism to obtain molecular results upon initial diagnosis, is 
included in national guidelines, and could be particularly meaningful to address the inequities faced by 
patients from communities that experience health disparities.11,12 
The overarching goal of this pilot trial is to design and test a nurse navigation intervention delivered via 
telehealth for patients with suspected locally advanced/metastatic NSCLC to improve timely molecularly-
informed treatment recommendations through early integration of concurrent molecular testing (i.e., 
tumor tissue and plasma-based molecular testing or plasma only when tumor tissue is 
insufficient/unavailable). Our central hypothesis is that providing telehealth nurse navigation to support 
completion of concurrent molecular testing will result in higher rates of comprehensive testing, improved 
timeliness of molecularly-informed treatment recommendations (primary endpoint), earlier initiation of 
molecularly-informed treatment, more meaningful patient-clinician communication, and higher levels of 
overall satisfaction among patients and clinicians. Drawing from systematic evidence on the role of 
navigation for coordination of cancer care and informed by insights from communication science and 
behavioral economics, the specific telehealth strategy to be tested is synchronous telehealth nurse 
navigation in combination with default ordering of plasma-based testing. This trial will allow us to assess 
multilevel determinants shaping the effectiveness and equity of telehealth strategies for cancer treatment 
initiation. 
 

TESTING 
6 
 
2. OBJECTIVES 
2.1 Primary Objectives 
Objective 1: In a randomized pilot trial of patients with suspected locally advanced/metastatic NSCLC, 
test the effectiveness of enhanced synchronous telehealth nurse navigation, compared to usual care 
nurse navigation, to increase timely molecularly-informed treatment recommendations through early 
integration of concurrent molecular testing. 
Objective 2: Evaluate contextual mechanisms contributing to the effectiveness of synchronous telehealth 
nurse navigation. 
2.2 Primary Outcomes 
Objective 1: Receipt of a molecularly-informed treatment recommendation for patients with metastatic 
NSq NSCLC at the time of the patient’s initial oncology visit. 
Objective 2: Individual and contextual factors (e.g., treatment knowledge, medical mistrust, clinical 
beliefs, organizational climate, patient-centered communication, financial toxicity, and knowledge of 
genetic testing) shaping trial effectiveness and patient experience. 
2.3 Secondary Outcomes 
Objective 1: Secondary outcomes include:  
1) Rate of telehealth visit completion 
2) Rate of completion of comprehensive molecular testing (tissue and/or plasma testing) prior to 
initiation of first line therapy 
3) Identification of one or more targetable mutations 
4) Timeliness of molecularly-informed treatment recommendation  
5) Overall survival 
6) Intervention costs 
7) Proportion of patients with diagnosis other than metastatic NSq NSCLC (telehealth arm only) 
8) Time to treatment initiation 
 
3. STUDY POPULATION 
3.1 Target Population 
Objective 1: The target population is 138 patients with suspected locally advanced/metastatic NSCLC.  
Objective 2: The target population is 20-30 patients who participate in the pilot trial and 10-15 clinicians 
or clinical leaders. 
3.2 Inclusion Criteria 
Objective 1:  
Patients will be eligible for the randomized clinical trial if they: 
a) are aged 18 years or older 
b) based on cross-sectional imaging, are suspected to have locally advanced/metastatic NSCLC (as 
determined by the evaluating clinician) 
c) are scheduled for an appointment in the lung cancer evaluation clinic. 
 
 

TESTING 
7 
 
 
Objective 2: 
Patients will be eligible if they were selected for participation in the pilot trial. Clinicians will be eligible if 
their area of expertise relates to the protocol topic (e.g., oncologists, nurses, clinical leads). 
 
3.3 Exclusion Criteria 
Patients will be ineligible for the pilot randomized clinical trial if they: 
a) are not suspected to have locally advanced/metastatic NSCLC 
b) have a concurrent active malignancy. 
 
3.4 Vulnerable Populations 
Children, fetuses, neonates, or prisoners are not included in this research study. 
 
4. STUDY DESIGN 
4.1 Preliminary Studies 
We have conducted two prior studies that inform the design of this trial. In the first, we evaluated the 
impact of plasma-based molecular testing in addition to tissue testing on the detection of actionable 
mutations in patients with metastatic NSCLC. In 229 patients who underwent concurrent plasma and 
tissue molecular testing, tissue alone detected targetable mutations in 21% of patients, whereas addition 
of plasma testing increased targetable mutation detection to 36%.13 Thus, plasma-based testing 
increased the rate of detection of therapeutically targetable alterations in metastatic NSCLC when used 
concurrently with tissue testing. In a second study, initiation of plasma-based testing performed at time of 
biopsy resulted in earlier availability of molecular results that translated into treatment recommendations 
in 75% of patients at initial oncology evaluation, compared to a baseline treatment recommendation rate 
of 47% in a historical control population.14 It also decreased time to guideline-concordant treatment 
initiation. However, implementing this intervention at the time of biopsy also resulted in over-testing of 
some patients (as tissue confirmation and stage were not yet established). The proposed pilot project will 
examine effectiveness of telehealth nurse navigation with an early comprehensive molecular testing 
blood draw during the periprocedural period associated with the patient’s diagnostic biopsy in order to 
reach a larger proportion of patients, reduce disparities in molecular testing, and target testing more 
optimally. 
4.2 Overview 
Prior to the trial launch, we will use a rapid-cycle approach (RCA) to optimize delivery of the enhanced 
synchronous telehealth nurse navigation visit and coordination/default ordering of molecular testing. RCA 
procedures will involve design meetings with the study team, discussions with administrators and 
clinicians who are members of our Implementation Lab, as well as contextual inquiry with clinicians and 
patients to elicit feedback.  
Objective 1: As shown in Figure 1, this two-arm pilot trial will randomize patients with suspected locally 
advanced/metastatic NSCLC who are scheduled for an appointment in the lung cancer evaluation clinic. 
Eligible patients will be identified by screening of clinical schedules by study staff using the EMR. Prior to 
or soon after completing an appointment in the lung cancer evaluation clinic, eligible patients will be 
randomized to either enhanced synchronous telehealth nurse navigation or to usual care nurse 
navigation. 

TESTING 
8 
 
 

TESTING 
9 
 
Usual Care Arm:  
Currently, in the standard care pathway at Penn Medicine, a patient who is being 
evaluated for lung cancer has an appointment within the lung cancer evaluation clinic and is 
then scheduled for a biopsy (e.g., bronchoscopy, transthoracic needle biopsy). After the biopsy 
is complete, the patient’s clinician contacts the trained lung cancer nurse navigator, who then 
calls the patient to 1) review the roles of clinicians on the medical oncology care team; 2) 
provide brief education on lung cancer; and 3) review the patient’s diagnostic history and 
coordinate collection or completion of imaging required for guideline-recommended cancer 
staging. In the standard care pathway at Penn Medicine this phone call between the patient and 
the lung cancer nurse navigator usually takes place after the biopsy is complete, though there 
are some cases in which the clinician asks the lung cancer nurse navigator to reach out to the 
patient sooner. 
For patients randomized to the usual care arm of this study, the researchers will ask the 
clinicians and the lung cancer nurse navigator not to arrange plasma-based testing earlier than 
the date of the initial in-person medical oncology visit. Instead, the lung cancer nurse navigator 
can have the patient complete a blood draw for plasma-based testing on the same day as their 
initial in-person medical oncology visit.  
Intervention (Telehealth) Arm: 
Patients in the intervention arm will be scheduled for an enhanced synchronous 
telehealth visit with a trained lung cancer nurse navigator prior to tissue biopsy. The enhanced 
synchronous telehealth visit will ideally occur between the initial clinical appointment and 
diagnostic biopsy (typically a period between two and seven days). In addition to the activities 
conducted as part of usual care, the nurse navigator will: 1) provide more detailed and 
individualized education on lung cancer and the rationale for comprehensive molecular testing, 
including plasma-based tests; and 2) if the patient agrees to testing, pend a default order for 
plasma-based molecular testing (if not already ordered) for the clinician to sign and arrange for 
phlebotomy to be performed during the periprocedural window associated with the patient’s 
tissue biopsy. 
Objective 2: Using rigorous approaches proven successful in our prior work,15,16 we will recruit 
20-30 patient participants of the primary trial and 10-15 clinicians and clinical leaders to 
complete semi-structured interviews following the active trial period. 
Patients: We will oversample for Black patients to understand effectiveness by race/ethnicity 
and stratify by trial outcome and study arm to understand factors contributing to both success 
and failure (e.g., primary endpoint) in either group (5-10 patient in each stratum). Trial 
participants will be invited in randomly selected batches each month (to enhance capture over 
time) until we reach our target sample of approximately 20-30 patients. Patients will be invited to 
participate in interviews via email and/or letter within 6 weeks of their in-person oncology visit. 
Clinicians: Recruitment for clinicians will be similar to the procedure for patients, except we will 
wait until the completion of the full trial to conduct interviews to avoid potential contamination. 
Clinicians will be purposively sampled by clinical role (e.g., oncologists, nurses, clinical leads) to 
enhance variation and invited via email. Recruitment will continue until we reach our target 
sample of 10-15 clinicians. 

TESTING 
10 
 
4.3 Study Duration and Timeline 
The study duration will be approximately 30 months. In months 1-3 we will conduct rapid-cycle 
approaches (RCAs) to optimize the telehealth navigation intervention.  
Objective 1: In month 3, we will launch the study and randomize patients by study arm. The 
active trial enrollment period will be approximately 18 months in length to reach the estimated 
sample size of 138 patients. Longitudinal data capture to collect secondary outcome information 
for patients will be completed up to 1-year post-enrollment. Collection and verification of all 
study endpoints will be completed by 30 months after the study’s start date. 
Objective 2: Invitations for surveys and qualitative answers for patients will be sent within 6 
weeks of the in-person oncology visit and will be completed within 3 months of invitation. 
Clinician and clinical leader interviews will be completed between months 15 and 21 of funding. 
Mixed methods analysis for Objective 2 will run from months 18-24. 
 
An overview of the project timeline is below. 
Table 1. Trial Timeline 
Project Timeline 
2023 
2024 
2025 
Q1 
Q2 
Q3 
Q4 
Q1 
Q2 
Q3 
Q4 
Q1 
Q2 
Q3 
Q4 
Objective 1: Enroll eligible trial 
participants 
 
• 
• 
• 
• 
 
 
 
 
 
 
 
Objective1: Deliver intervention 
 
  • 
• 
• 
• 
 
 
 
 
 
 
 
Objective 1: Endpoint ascertainment 
& analysis 
 
  • 
• 
• 
• 
• 
• 
 
 
 
 
 
Objective 2: Recruit & conduct 
patient interviews 
 
 
• 
• 
• 
• 
• 
• 
 
 
 
 
Objective 2: Recruit & conduct 
clinician interviews 
 
 
 
 
 
 
• 
• 
 
 
 
 
Objective 3: Mixed methods coding & 
analysis 
 
 
 
 
 
 
 
• 
• 
 
 
 
Submit manuscripts & disseminate 
results 
 
 
 
 
 
 
 
 
• 
 
 
 
 
4.4 Study Setting 
This study will occur within the University of Pennsylvania Health System (UPHS). Research 
activities will be conducted at the following UPHS sites: 
• 
Hospital of the University of Pennsylvania Pavilion (HUP) – East 
• 
Hospital of the University of Pennsylvania (HUP) 
• 
Pennsylvania Hospital (PAH) 
• 
Penn Presbyterian Medical Center (PPMC) 
 
 

TESTING 
11 
 
5. STUDY PROCEDURES 
5.1 Recruitment and Retention 
Objective 1: This study will employ a waiver of consent mechanism (see Section 5.2 Informed 
Consent). Eligible patients will be identified from weekly screening of appointments scheduled at 
the lung cancer evaluation clinic at Penn Medicine. The process will be optimized to identify all 
patients with suspected locally advanced/metastatic NSCLC. After eligibility is confirmed based 
on review of medical records, eligible participants will be randomized to receive an early 
enhanced synchronous telehealth nurse navigation visit (intervention) or standard nurse 
navigation (usual care). After completion of the initial clinical appointment, patients in the 
telehealth arm will be scheduled to receive a synchronous visit (with audio and video 
capabilities) with a lung cancer trained nurse navigator prior to their tissue biopsy. 
Objective 2: This portion of the study will utilize prospective consent with waiver of written 
documentation (see Section 5.2 Informed Consent). An estimated 20-30 patients and 10-15 
clinicians and clinical leaders will be interviewed (sample size dependent upon when data 
saturation is reached). Interview participants will also complete a survey before or at the time of 
the interview. 
Patients: A sample of patients enrolled in the randomized clinical trial will be invited to 
participate in the semi-structured interview and survey via email or letter within 6 weeks 
of in-person visit. Participants will be invited in randomly selected batches (estimate 
approximately 5 per month to enhance capture over time) until we reach our target 
sample of approximately 20-30 patients. We will oversample for Black patients to 
understand effectiveness by race/ethnicity and stratify by trial outcome (treatment 
recommendation or not) and study arm (telehealth versus usual care) to understand 
factors contributing to both success and failure of the telehealth strategy (5-10 patients 
in each stratum).  
Clinicians: Recruitment of clinicians will be similar to that of patients, except we will wait 
until the completion of the full trial to conduct interviews to avoid potential contamination. 
Clinicians will be randomly selected and invited via email. Recruitment will continue until 
we reach our target sample of 10-15 clinicians, purposively sampled by role (oncologists, 
nurses and advanced practice providers, practice managers, and health system 
leaders).  
Interviews will be conducted by the qualitative data analyst and other trained research staff 
supervised by Dr. Rendle, a doctoral-trained qualitative researcher and Co-Investigator on the 
team. Interviews will be conducted in-person, by phone, or using a HIPAA-compliant video 
platform. 
5.2 Informed Consent 
Objective 1: A waiver of informed consent will be used for the randomized pilot trial. A waiver of 
informed consent is advocated for certain pragmatic trials testing methods for comparing and 
improving the delivery of established interventions within health care systems. The present 
study meets criteria for such an alteration of the requirement for informed consent set forth in 
the Common Rule in the following ways: 
• 
The research involves no more than minimal risk to subjects. The risks to subjects 
participating in this study and interventions are aimed at improving patient knowledge 

TESTING 
12 
 
and completion of molecular testing. In Sections 7.1 and 7.3, we outline the safeguards 
we have in place to prevent any breach of confidentiality – the primary risk of study 
participation. 
• 
The waiver or alteration will not adversely affect the rights and welfare of the subjects. 
• 
To the extent possible, the subjects will be provided with pertinent information after 
participating in the trial. 
• 
The research cannot be practicably conducted without a waiver or alteration of the 
requirement for informed consent. This is an explicitly pragmatic approach designed to 
test the real-world effectiveness of an early telehealth nurse navigation visit to provide 
patient education and coordinate molecular testing. Key outcomes include assessment 
of patient centered communication and patient and clinician satisfaction. Requiring 
traditional (opt-in or opt-out) individual consent would introduce important selection 
biases (e.g., trial patients would be those willing to be randomized to receipt of a 
telehealth visit). Knowledge of the trial arms would bias participant opinions on the 
education and care coordination interventions and influence their views on satisfaction 
with care received. This would impact the value of the knowledge to be obtained 
because it would not generalize to the full range of patients diagnosed with lung cancer. 
All participants will retain the right to decline patient education or plasma-based molecular 
testing or other ordered studies, regardless of study assignment. 
 
Objective 2: Potential interview participants will be initially contacted by study team members by 
email or letter and given the option to decline further contact from the team. If the participant 
has not opted out within two weeks, they will be contacted by phone to assess their interest in 
participating in the study. If the participant agrees to participate, they will be scheduled to have 
an interview in-person, by phone, or using a HIPAA-compliant video platform (based on 
preference and study procedures at the time). For this portion of the study, a waiver of 
documentation of written informed consent will be used because the risk to the individual is 
minimal, a signed informed consent form (ICF) could identify that an individual participated in 
the study, and obtaining a signed paper form would significantly decrease the likelihood of 
proceeding towards an interview. Prior to the start of the interview, research staff will review the 
study purpose, procedures, and the rights of the participant. They will also provide an 
information statement to participants via email prior to the scheduled interview. Research staff 
will state that participation is voluntary and ask for the participant’s permission to record their 
interview. They will describe the transcription and de-identification process, and they will ask 
permission to proceed with the interview. All participants will be free to withdraw participation at 
any time, and study enrollment will not impact employment or care at Penn Medicine.  
5.3 Compensation 
Objective 1: Patients who participate in the randomized clinical trial (Objective 1) will not receive 
compensation.  
Objective 2: All patients and clinicians who complete the semi-structured interviews and surveys 
will receive $50 for their participation. Payments will be made through issuance of GreenPhire 
ClinCards, which can be used in the same manner as a debit card. The ClinCards will be mailed 
to participants within 10 days after interview and survey completion.  

TESTING 
13 
 
5.4 Measures and Endpoints 
Objective 1: The specific outcomes align with 5 key endpoints (access, quality, outcomes, 
equity, and efficiency). The primary endpoint is receipt of a molecularly-informed treatment 
recommendation at the time of the patient’s initial in-person oncology visit, measured up to 12 
weeks from randomization. This outcome encompasses successful completion of 
comprehensive molecular testing and the ability of the patient and oncology care team to have 
all necessary information to collaboratively arrive at the optimal treatment approach. We 
anticipate that approximately 10% of patients randomized to usual care will be able to receive a 
molecularly-informed treatment recommendation at initial visit – these include patients in whom 
tissue was successfully tested and identified a targetable mutation prior to referral. We 
anticipate this will be substantially improved with enhanced nurse navigation (See Section 6.1 
Sample Size). The primary endpoint will be assessed by review of clinician documentation (e.g., 
progress notes) within the electronic medical record (EMR) on the day of the initial visit.  
Secondary outcomes include:  
1) Operational endpoints: 
a. Rate of completion of telehealth visit defined as successful completion of a 
nurse navigator telehealth visit prior to biopsy, measured up to 3 weeks from 
randomization. 
b. Rate of completion of comprehensive molecular testing (tissue and/or plasma 
testing) prior to initiation of first line therapy, measured up to 12 weeks from 
randomization. 
2) Clinical endpoints: 
a. Time from randomization to molecularly-informed treatment recommendation, 
measured up to 12 weeks from randomization.  
b. Proportion of participants with one or more targetable mutations, measured 
up to 12 weeks from randomization. 
c. Overall survival, measured as 1-year overall survival from time of 
randomization to death from any cause. 
d. The proportion of enrolled patients with a diagnosis other than metastatic 
NSq NSCLC (telehealth arm only), measured at 12 weeks from 
randomization. 
e. Time from randomization to treatment initiation, measured up to 12 weeks 
from randomization. 
3) Cost outcomes: 
a. Intervention costs (measured using a pragmatic method to capture and 
analyze all system-level resources, such as personnel time and patient 
outreach costs) that are needed to deploy telehealth strategies in routine 
care.19 
 
Objective 2: We will use structured and validated measures and a semi-structured interview 
guide based on our Framework for Integrating Telehealth Equity (FITE) to systematically 
evaluate individual and contextual factors (e.g., treatment knowledge, medical mistrust, clinical 
beliefs, patient-centered communication, financial toxicity, and knowledge of genetic testing) 
that will shape the effectiveness of telehealth strategies. Domains that will be assessed as part 
of the Objective 2 include: 

TESTING 
14 
 
1) Telehealth usability, acceptance, and satisfaction (using the University of Pittsburgh 
Telehealth Usability Questionnaire) 
2) Patient-centered communication (using the University of North Carolina Patient-
Centered Communication in Cancer Care Instrument) 
3) Medical-related trust and mistrust (using the Group-Based Medical Mistrust Scale 
[GBMMS]) 
4) Acceptability, appropriateness, and feasibility of interventions (using the Acceptability 
of Intervention Measure [AIM] Intervention Appropriateness Measure [IAM] and 
Feasibility of Intervention Measure ([FIM]) 
5) Treatment related knowledge and beliefs  
6) Other contextual factors (e.g., organizational climate, barriers to using telehealth) 
that may impact effectiveness of trial 
Objective 2 endpoints for the subset of enrolled participants will be assessed up to 1 year 
from randomization.  
5.5 Sources of Materials 
Objective 1: Electronic health record (EHR) data will be used to collect the primary endpoint as 
well as covariates required for statistical analysis. We will use the EHR and survey data 
collected to assess secondary endpoints. 
Objective 2: Semi-structured interviews will be collected and analyzed in conjunction with 
quantitative measures conducted as part of the RCT. 
 
6. STATISTICAL DESIGN AND POWER 
6.1 Sample Size 
Objective 1: Sample size is based on estimates of receiving a molecularly-informed treatment 
recommendation at the initial oncology visit. Based on our prior studies, we anticipate this will 
occur in 10% of patients in the usual care arm. This pilot study is designed to detect an absolute 
increase of 20% in our primary outcome for patients in the intervention arm. With a sample size 
of 124 participants and a two-sided alpha, we will have >80% power to detect an increase of 
this magnitude. Given the potential for drop-out of patients following randomization, we have 
increased the estimate by 10%, leading to a final sample size of 138 participants (69 in each 
arm). Although a difference <20% in our primary outcome rate may be clinically meaningful, this 
sample size is an optimal balance to allow an assessment of our primary and secondary 
outcomes and assess for mechanisms of action. Study outcomes will be used to preliminarily 
assess effectiveness across all outcome domains to inform the design of future studies, 
including a pragmatic trial, if warranted. 
Objective 2: Proposed sample size is based on the estimated number of interviews needed to 
reach data saturation within each group and by intervention outcome to support mixed methods 
evaluation; however, interviews will continue until saturation is achieved.16,21 
6.2 Analysis Plan 
6.2.1 Objective 1 Primary Analyses 
We will produce data summaries to assess data quality and demographic and clinical 
characteristics across arms. First, our primary analytic approach will be to conduct ITT analyses 

TESTING 
15 
 
using unadjusted logistic regression to compare the overall effectiveness of the intervention to 
result in receipt of a treatment recommendation at the initial in-person visit and time to treatment 
initiation. The ITT analysis provides an unbiased test of the overall effectiveness of the 
interventions. An additional analysis will adjust for imbalanced covariates, where imbalance is 
determined by standardized mean differences >0.1. If the number of imbalanced covariates 
exceeds the number allowed for adjustment based on the number of events, we will prioritize 
covariates with the largest imbalances. 
6.2.2 Objective 1 Secondary Analyses 
We will use regression methods to assess the impact of the telehealth intervention on 
secondary outcomes of telehealth visit completion, rate of comprehensive molecular testing, 
patient satisfaction, clinician acceptability, and intervention costs, with logistic, Cox proportional 
hazards, and linear regression for binary, time-to-event, and continuous outcomes, respectively. 
Additional secondary analyses will adjust for factors that exhibit residual imbalance between 
randomized intervention groups. In addition, secondary as-treated analyses will be conducted. 
6.2.3 Objective 1 Exploratory Analyses 
A difference in effectiveness of the telehealth intervention for Black versus White patients will be 
conducted by adding a main effect of race and the interaction of race and intervention to the 
primary unadjusted logistic regression model. Given that hypothesis tests of interaction terms 
are likely severely underpowered at our pilot study sample size, we will focus statistical 
inference on the point estimate and 95% confidence intervals for the interaction term. While 
these intervals may cross the null value consistent with no difference in effectiveness, 
confidence intervals that largely lie away from the null provide evidence in favor of differential 
intervention effects that could be detected statistically with a larger sample size in a subsequent 
study. Patients who do not identify as Black or White will be excluded from this analysis.  
 
6.2.4 Objective 2 Analyses 
We will descriptively assess survey data and thematically code interview data. These data will 
be used to conduct qualitative comparative analysis (QCA) to identify how contextual factors 
shaped both timeliness and quality of care and the effectiveness of nurse navigation. QCA is a 
multistep analytic method that combines qualitative and quantitative coding and calibration to 
identify conditions shaping the 
effectiveness of telehealth strategies. 
The four-step process will use survey 
and interview data as inputs or 
“conditions.”  
Code Contextual Data (Step 1-2). We 
will use convergent mixed methods 
analysis to code contextual conditions 
(inner setting, outer setting, and 
individual characteristics) and 
implementation conditions 
(characteristics of specific telehealth 
strategy and process).  

TESTING 
16 
 
Qualitative Data. The constant comparative method, guided by modified grounded theory,22 will 
be used to iteratively identify a priori domains of interest (guided by FITE) and to inductively 
explore emergent themes. Two trained coders will first independently read through each 
transcript to identify themes within each domain. We then will use this list to develop a coding 
dictionary and apply it to the data. We will measure inter-rater reliability to document and 
improve coding consistency. Once high reliability is achieved (kappa >0.8), we will apply the full 
coding dictionary to the interview data using NVivo and produce thematic reports summarizing 
our findings. We will then use qualitative data to expand upon and triangulate quantitative 
patterns identified in trial and surveys.23 
Survey Data. Survey data will be analyzed descriptively and coded dichotomously or 
categorically as appropriate. Contextual data will serve as QCA inputs to identify necessary and 
sufficient conditions for telehealth success.  
Qualitative Comparative Analysis (QCA) (Steps 3-4). For QCA, our primary endpoint will be 
receipt of a molecularly-informed treatment recommendation at the time of the patient’s initial in-
person oncology visit, and success or failure will be determined at the patient level. Each case 
will be calibrated as having or not having the primary outcome or condition (described above).24 
Thresholds for coding primary outcomes and the presence or absence of each condition (e.g., 
low quality communication) will be determined based on existing literature or stakeholder 
consensus. Most outcomes will be dichotomous, but continuous values and fuzzy set QCA will 
be used as appropriate.27,28 Data (“truth”) tables will be created for analysis, which list all 
possible configurations of conditions, the number of cases that fall into each configuration, and 
the consistency of the cases—or the proportion of cases in the specific configuration that have 
the desired outcome.28,29 We will conduct QCA analyses using R package QCApro.30–32 Raw 
and unique coverage will be calculated and consistency will be set at 80% for sufficient and 
90% for necessary conditions. This iterative analytic process will identify what conditions—alone 
or in combination with others—are necessary or sufficient to yield the primary endpoint.  
 
7. DATA MANAGEMENT AND RISK-BENEFIT ASSESSMENT 
7.1 Data and Safety Monitoring Plan 
A further safeguard to protect research participants is the development of a plan for ongoing 
data and safety monitoring to anticipate, and protect against, any human subjects research 
concerns that may arise. The data and safety monitoring plan will have two parts.  
1. The study PI (Dr. Aggarwal), biostatistician, and data analyst will develop and implement 
methods of verifying entered data and of quality control. Individual-level data for participants 
will be kept confidential and will only be stored on highly secure servers available for patient-
level data. Only authorized project personnel will have access to the data and the data will 
only be stored on servers and not stand-alone PCs or laptops. All data will be reported as 
units of aggregation which make impossible the identification of individual patients or 
clinicians. 
2. As this is a minimal risk study, monitoring will be performed by Dr. Aggarwal, the principal 
investigator, who will be directly responsible for identifying and reporting protocol 
deviations/violations and unanticipated events to the IRBs and funding agency promptly, as 
appropriate.  

TESTING 
17 
 
A written research protocol will undergo formal institutional scientific and IRB review at the 
University of Pennsylvania (Penn) to ensure protection of the rights and welfare of human 
research subjects. Specifically, the PI and the IRB will be responsible for ensuring risks to 
human subjects are minimized, risks are reasonable, subject selection is equitable, the research 
team has access to adequate resources to conduct the study, the informed consent process 
meets regulatory and ethical requirements, adequate provision is made to protect human 
subjects by monitoring the data collected, and there are adequate provisions to protect subject 
privacy per HIPAA regulations and confidentiality of data.  
All senior/key personnel and research staff who will be involved in the design and conduct of the 
study must receive education in human research subject protection from a training program that 
is approved by a properly constituted independent Ethics Committee or Institutional Review 
Board. The PI will be responsible for ensuring project faculty and staff have the equipment and 
training required to protect privacy and confidentiality and will monitor and document that these 
individuals are properly certified. If new senior/key personnel and staff become involved in the 
research, documentation that they have received the required education will be included in the 
annual progress reports.  
7.2 Potential Risks 
Objective 1: For trial participants, the potential risks to human subjects attributable to 
participating in this trial are minimal. The eligibility processes are designed to identify patients 
with metastatic NSCLC who are eligible for plasma-based molecular testing according to 
existing practice guidelines. The main risk of participation is breach of confidentiality; however, 
we have described the strong data safeguards in place to prevent confidentiality breaches 
above. 
Objective 2: For patient and clinician interviews, potential risks to participating in this study 
include: potential violation of confidentiality or privacy; possible discomfort in disclosing 
information on testing knowledge, beliefs, or practices; and possible concern among providers 
and clinicians that participation will negatively impact job or clinic performance or evaluation. 
Participation in this pilot project presents minimal risks. There is a potential risk of breach of 
confidentiality. We will minimize this risk by maintaining confidentiality for individual-level data, 
storing data on secure servers, and reporting data as units of aggregation. Other risks include 
possible discomfort in answering interview questions, such as those related to medical distrust 
or clinical beliefs. Participants are allowed to skip any interview or survey question as needed. 
7.3 Protections Against Risk 
The primary risk to participants is loss of confidentiality and/or privacy, including of protected 
health information. To minimize the risk of breach of data and confidentiality, we will use secure, 
encrypted servers to host the data and conduct the analysis. The Penn Medicine Academic 
Computing Services (PMACS) will be the hub for the hardware and database infrastructure that 
will support the project. PMACS is a joint effort of the University of Pennsylvania's Abramson 
Cancer Center, the Cardiovascular Institute, the Department of Pathology, and the Leonard 
Davis Institute. PMACS provides a secure computing environment for a large volume of highly 
sensitive data, including clinical, genetic, socioeconomic, and financial information. PMACS 
requires all users of data or applications on PMACS servers to complete a PMACS-hosted 
cybersecurity awareness course annually, which stresses federal data security policies under 
data use agreements with the university. The curriculum includes Health Insurance Portability 

TESTING 
18 
 
and Accountability Act (HIPAA) training and covers secure data transfer, passwords, computer 
security habits and knowledge of what constitutes misuse or inappropriate use of the server. We 
will implement multiple, redundant protective measures to guarantee the privacy and security of 
the participant data. All investigators and research staff with direct access to the identifiable 
data will be required to undergo annual responsible conduct of research, cybersecurity, and 
HIPAA certification in accordance with University of Pennsylvania regulations. Data will be 
stored, managed, and analyzed on a secure, encrypted server behind the University of 
Pennsylvania Health System (UPHS) firewall. All study personnel that will use this data are 
listed on the IRB application and have completed training in HIPAA standards and CITI human 
subjects research. Data access will be password protected. Whenever possible, data will be de-
identified for analysis. 
For Objective 2, all interview and survey data will also be stored on the secure/firewalled 
servers of the PMACS Data Center, in data files that will be protected by multiple password 
layers. In addition, we will use the following safeguards: 1) all paper-based data including audio 
files from interviews, notes from medical record reviews, and qualitative transcripts will be kept 
in a locked filing cabinet; 2) participant identity will be masked using unique participant IDs and 
stored on a password-protected master list to which only the PIs will have access; 3) survey 
data will be collected using a secure, password-protected web-based application (REDCap); 4) 
surveys will use consent mechanisms that do not require signatures; 5) any protected health 
information will be housed on secure PMACS servers and only accessible by the PIs and 
approved research staff; and 6) all statistical or qualitative analytic files will be identified only 
with participant IDs and not contain any protected health information (all identifiable data will be 
made de-identifiable prior to including it into the analytic datasets). 
For survey and interview participants, a second risk includes possible discomfort of disclosing 
information on lung cancer knowledge and practices. If experienced, we anticipate this risk will 
be temporary and minimal.  
For survey and interview participants, a third risk for clinicians and patients is that study 
participation may include possible concern that participation will negatively impact job evaluation 
or clinical care. In conjunction with efforts to protect confidentiality, we will work to ensure that 
no individual information regarding lung cancer treatment practices (from clinical record or 
surveys) is reported to anyone outside the study team, including healthcare management, staff, 
or providers. The disclosure of lung cancer treatment practices will be communicated only to 
individual providers. All information regarding beliefs, perspectives, or practices collected in 
surveys or qualitative activities will be de-identified and reported in aggregate. Lastly, consent 
language will also clearly state that participation is not required based on employment at UPHS 
and will not impact employment or clinical care. 
7.4 Potential Benefits 
There are no direct benefits to patients or clinicians who participate in the interview or survey 
component of this study. In the trial part of the study, patients may indirectly benefit through 
improved and more timely communication regarding their cancer diagnosis and the role of 
molecular testing for treatment planning. For the health of society, the benefits of the study are 
substantial and include potential improvements in cancer care delivery access, quality, 
efficiency, equity, and outcomes. 

TESTING 
19 
 
7.5 Importance of the Knowledge to be Gained 
Findings from this study will help to advance the field of cancer care delivery and 
communication science by testing the effectiveness of an early telehealth care coordination visit 
among patients with advanced lung cancer referred for evaluation and treatment and identifying 
key conditions that may need to be targeted or adapted to ensure success. By identifying 
underlying mechanisms supporting or hindering patient-centered care, this project will provide 
insights into improving lung cancer treatment across the population. This work will also help to 
advance causal theory in behavioral economics and communication science by evaluating 
underlying multilevel mechanisms that contribute to the success or failure of care delivery 
strategies in real-world care. 
7.6 Risk-Benefit Ratio 
Given the limited risk associated with participation, we anticipate that the benefits of this study 
will outweigh the risks involved. 
 
8. RESOURCES NECESSARY FOR HUMAN RESEARCH PROTECTION 
Adequate facilities are available within Penn Medicine’s Clinical Practice Network. Members of 
the research team, listed in HS-ERA, will be overseen by the PI and include appropriate 
personnel to successfully implement this pilot project. All personnel will complete required 
training before being granted access to any identifying information. Training includes information 
on confidentiality through the Collaborative IRB Training Initiative (CITI) courses. All personnel 
will also be trained in procedures for reporting unintentional breaches in confidentiality to the PI. 
All personnel will be aware that violations of participant’s confidentiality, either unintentional or 
deliberate, may result in termination of hire.  
9. STUDY TEAM 
Our team includes investigators with expertise in thoracic medical oncology, pragmatic trials, 
behavioral economics, and communication science. Dr. Charu Aggarwal will lead this pilot 
project. Dr. Aggarwal has led multiple prospective studies on the impact of plasma-based 
molecular testing on clinical outcomes and leads efforts at Penn Medicine and nationally on 
integration of plasma-based genotyping into the diagnostic algorithm for patients with NSCLC. 
Dr. Jeffrey Thompson (co-I) also has considerable expertise in NSCLC and plasma-based 
genotyping. Drs. Justin Bekelman (co-I), Shivan Mehta (co-I) and Anil Vachani (co-I) have 
considerable experience with leading pragmatic trials with waivers of consent. Other co-
investigators include Dr. Alisa Stephens-Shields (biostatistician), Dr. Marilyn Shapiro (decision 
scientist) as well as Drs. Katharine Rendle and Andy Tan, who bring implementation science 
and communication science expertise to the team. 
 
 
 
 
 

TESTING 
20 
 
10. REFERENCES 
1.  
Politi MC, Yen RW, Elwyn G, et al. Women Who Are Young, Non-White, and with Lower 
Socioeconomic Status Report Higher Financial Toxicity up to 1 Year After Breast Cancer 
Surgery: A Mixed-Effects Regression Analysis. Oncologist. 2021;26(1):e142-e152. 
doi:10.1002/onco.13544 
2.  
Carlisle JW, Ramalingam SS. A banner year for immunotherapy and targeted therapy. 
Nat Rev Clin Oncol. 2019;16(2):79-80. doi:10.1038/s41571-018-0138-4 
3.  
Kris MG, Johnson BE, Berry LD, et al. Using Multiplexed Assays of Oncogenic Drivers in 
Lung Cancers to Select Targeted Drugs. JAMA. 2014;311(19):1998-2006. 
doi:10.1001/jama.2014.3741 
4.  
Lindeman NI, Cagle PT, Aisner DL, et al. Updated Molecular Testing Guideline for the 
Selection of Lung Cancer Patients for Treatment With Targeted Tyrosine Kinase 
Inhibitors: Guideline From the College of American Pathologists, the International 
Association for the Study of Lung Cancer, and the Association for Molecular Pathology. 
Arch Pathol Lab Med. 2018;142(3):321-346. doi:10.5858/arpa.2017-0388-CP 
5.  
Kalemkerian GP, Narula N, Kennedy EB, et al. Molecular Testing Guideline for the 
Selection of Patients With Lung Cancer for Treatment With Targeted Tyrosine Kinase 
Inhibitors: American Society of Clinical Oncology Endorsement of the College of 
American Pathologists/International Association for the Study of Lung 
Cancer/Association for Molecular Pathology Clinical Practice Guideline Update. J Clin 
Oncol. 2018;36(9):911-919. doi:10.1200/JCO.2017.76.7293 
6.  
Bruno DS, Hess LM, Li X, Su EW, Zhu YE, Patel M. Racial disparities in biomarker 
testing and clinical trial enrollment in non-small cell lung cancer (NSCLC). JCO. 
2021;39(15_suppl):9005-9005. doi:10.1200/JCO.2021.39.15_suppl.9005 
7.  
Gutierrez ME, Choi K, Lanman RB, et al. Genomic Profiling of Advanced Non-Small Cell 
Lung Cancer in Community Settings: Gaps and Opportunities. Clin Lung Cancer. 
2017;18(6):651-659. doi:10.1016/j.cllc.2017.04.004 
8.  
Bernicker EH, Xiao Y, Abraham A, et al. Adherence to National Comprehensive Cancer 
Network ALK Testing Guidelines for Patients with Advanced Non-Small Cell Lung 
Cancer in U.S. Community Medical Centers. Oncologist. 2021;26(6):e1050-e1057. 
doi:10.1002/onco.13779 
9.  
Singal G, Miller PG, Agarwala V, et al. Association of Patient Characteristics and Tumor 
Genomics With Clinical Outcomes Among Patients With Non-Small Cell Lung Cancer 
Using a Clinicogenomic Database. JAMA. 2019;321(14):1391-1399. 
doi:10.1001/jama.2019.3241 
10.  
Schoenfeld AJ, Rizvi H, Bandlamudi C, et al. Clinical and molecular correlates of PD-L1 
expression in patients with lung adenocarcinomas. Ann Oncol. 2020;31(5):599-608. 
doi:10.1016/j.annonc.2020.01.065 
11.  
Aggarwal C, Rolfo CD, Oxnard GR, Gray JE, Sholl LM, Gandara DR. Strategies for the 
successful implementation of plasma-based NSCLC genotyping in clinical practice. Nat 
Rev Clin Oncol. 2021;18(1):56-62. doi:10.1038/s41571-020-0423-x 

TESTING 
21 
 
12.  
National Comprehensive Cancer Network (NCCN). NCCN Clinical Practice Guidelines in 
Oncology. Non-Small Cell Lung Cancer. Version 5.2021. Published online May 2021. 
https://www.nccn.org/professionals/physician_gls/pdf/nscl.pdf 
13.  
Aggarwal C, Thompson JC, Black TA, et al. Clinical Implications of Plasma-Based 
Genotyping With the Delivery of Personalized Therapy in Metastatic Non-Small Cell 
Lung Cancer. JAMA Oncol. 2019;5(2):173-180. doi:10.1001/jamaoncol.2018.4305 
14.  
Thompson J, Aggarwal C, Wong J, et al. Plasma NGS at time of diagnostic tissue biopsy 
– Impact on time to treatment: results from a pilot prospective study. Presented at: World 
Conference on Lung Cancer 2021; September 8, 2021; Denver, Colorado. 
15.  
Ogden SN, Glanz K, Schapira MM, Bocage C, Eriksen W, Rendle KA. Patient 
Perspectives on Primary HPV Testing for Routine Cervical Cancer Screening. J Gen 
Intern Med. Published online July 16, 2019. doi:10.1007/s11606-019-05175-4 
16.  
Rendle KA, Abramson CM, Garrett SB, Halley MC, Dohan D. Beyond exploratory: a 
tailored framework for designing and assessing qualitative health research. BMJ Open. 
2019;9(8):e030123. doi:10.1136/bmjopen-2019-030123 
17.  
Parmanto B, Lewis AN, Graham KM, Bertolet MH. Development of the Telehealth 
Usability Questionnaire (TUQ). Int J Telerehabil. 2016;8(1):3-10. 
doi:10.5195/ijt.2016.6196 
18.  
Thompson HS, Valdimarsdottir HB, Winkel G, Jandorf L, Redd W. The Group-Based 
Medical Mistrust Scale: psychometric properties and association with breast cancer 
screening. Prev Med. 2004;38(2):209-218. doi:10.1016/j.ypmed.2003.09.041 
19.  
Ritzwoller DP, Sukhanova A, Gaglio B, Glasgow RE. Costing behavioral interventions: a 
practical guide to enhance translation. Ann Behav Med. 2009;37(2):218-227. 
doi:10.1007/s12160-009-9088-5 
20.  
Weiner BJ, Lewis CC, Stanick C, et al. Psychometric assessment of three newly 
developed implementation outcome measures. Implementation Sci. 2017;12(1):1-12. 
doi:10.1186/s13012-017-0635-3 
21.  
Morse JM. Critical Analysis of Strategies for Determining Rigor in Qualitative Inquiry. 
Qualitative Health Research. 2015;25(9):1212-1222. doi:10.1177/1049732315588501 
22.  
Glaser BG, Strauss AL. The Discovery of Grounded Theory: Strategies for Qualitative 
Research. Aldine Pub Co; 1967. 
23.  
Creswell JW, Plano Clark VL. Designing and Conducting Mixed Methods Research. 3rd 
ed. SAGE Publications, Inc; 2018. 
24.  
Rihoux B, Ragin C. Configurational Comparative Methods. SAGE Publications; 2008. 
25.  
Kane H, Lewis MA, Williams PA, Kahwati LC. Using qualitative comparative analysis to 
understand and quantify translation and implementation. Transl Behav Med. 
2014;4(2):201-208. doi:10.1007/s13142-014-0251-6 
26.  
Weiner BJ, Jacobs SR, Minasian LM, Good MJ. Organizational designs for achieving 
high treatment trial enrollment: a fuzzy-set analysis of the community clinical oncology 
program. J Oncol Pract. 2012;8(5):287-291. doi:10.1200/JOP.2011.000507 

TESTING 
22 
 
27.  
Baumgartner M, Ambühl M. Causal modeling with multi-value and fuzzy-set Coincidence 
Analysis. Polit Sci Res Methods. 2020;8(3). doi:10.1017/psrm.2018.45 
28.  
Ragin CC. Redesigning Social Inquiry: Fuzzy Sets and Beyond. University of Chicago; 
2008. 
29.  
Whitaker RG, Sperber N, Baumgartner M, et al. Coincidence analysis: a new method for 
causal inference in implementation science. Implement Sci. 2020;15(1):108-108. 
doi:10.1186/s13012-020-01070-3 
30.  
Thiem A. Using Qualitative Comparative Analysis for Identifying Causal Chains in 
Configurational Data: A Methodological Commentary on Baumgartner and Epple (2014). 
Sociol Methods Res. 2015;44(4):723-736. doi:10.1177/0049124115589032 
31.  
Baumgartner M, Thiem A. Often Trusted but Never (Properly) Tested: Evaluating 
Qualitative Comparative Analysis. Sociol Methods Res. 2020;49(2). 
doi:10.1177/0049124117701487 
32.  
Baumgartner M, Ambühl M. Cna: An R Package for Configurational Causal Inference 
and Modeling. 
 
11. APPENDICES 
1. University of Pittsburgh Telehealth Usability Questionnaire 
2. University of North Carolina Patient-Centered Communication in Cancer Care 
Instrument 
3. Group-Based Medical Mistrust Scale (GBMMS) 
4. Acceptability of Intervention Measure (AIM) Intervention Appropriateness Measure 
(IAM) and Feasibility of Intervention Measure (FIM) 
5. Patient Recruitment Email/Letter Template 
6. Patient Recruitment Telephone Script 
7. Patient Interview Verbal Consent 
8. Patient Survey and Interview Guide