Implementing Guidelines for Shared Decision Making in Lung Cancer Screening (QUE 15-286)
Conditions: Prevention and Control
Sponsor: VA Office of Research and Development

 
 
 
 
QUX 16-003, Implementing Guidelines for Shared Decision Making in Lung Cancer Screening, QUE 16-003 
 
 
Study Protocol and Statistical Analysis Plan 
 
September 15, 2020 
 
 
 

 
This study was conducted as a non-research quality improvement initiative primarily to improve the quality of 
preventive services in the VA as part of an ongoing partnership with the VA National Center for Health Promotion 
and Disease Prevention (see letter from Linda Kinsinger, MD, MPH; Chief Consultant for Preventive Medicine). 
Given the non-research quality improvement designation, this study did not have a protocol reviewed by an 
Institutional Review Board, but the following is an excerpt from the study proposal which was peer reviewed and 
approved for funding in June 2015 as part of a VA Quality Enhancement Research Initiative field program at the VA 
Ann Arbor Healthcare System (aka PROVE QUERI). This document provides an excerpt from the nationally 
reviewed proposal that describes the study aims, procedures, design, and analysis plan for this trial.  
 
 

 
Excerpt from PROVE QUERI Program Proposal:  
  Project #1: 
Implementing Guidelines for Shared Decision Making in Lung Cancer Screening 
1.0  Specific Aims 
The goal of this project is to test two strategies for implementing shared decision making into clinical practice with 
the aid of the Decision Precision lung cancer screening tool. To accomplish this goal we have three aims: 
Aim 1:  To determine effectiveness of LEAP plus Audit and Feedback (A&F) versus A&F alone. Hypothesis:  
VAMCs with LEAP + A&F will have better implementation outcomes (defined by RE-AIM104) than VAMCs with 
A&F alone. 
Aim 2:  To determine the effectiveness of allowing providers to set risk threshold algorithms. 
Hypothesis:  Implementation outcomes will improve when providers are allowed to set risk thresholds. 
Aim 3: To conduct a formative evaluation to determine the factors most important for successful implementation 
of risk-based shared decision making tools. 
2.0  Rationale 
Lung cancer is both prevalent and deadly. It is the third most common cancer and is the leading cause of 
cancer-related deaths in the US. Lung cancer is both more prevalent and associated with worse outcomes 
among Veterans than among the general US population.105,106 
Based on the results of the National Lung Screening Trial (NLST), the United States Preventive Services Task 
Force (USPSTF) recommends low-dose computed tomography (LDCT) screening for patients who meet age and 
smoking criteria.107,108 The NLST found that eligible heavy smokers are significantly less likely to die from lung 
cancer if screened with LDCT vs. chest x-rays. For perspective, the reduction in deaths from lung cancer with 
screening is larger than the reduction in deaths from the target cancers of other common screening tests, such as 
mammograms for breast cancer. In 2013, VHA rolled out a two-year clinical demonstration project in eight VA 
medical centers to determine the resources needed to “provide screening and follow-up…with accuracy, 
efficiency, and safety similar to that achieved in the NLST.”109 
However, the NLST also found substantial harms associated with LDCT screening, including high false- positive 
rates which can lead to unnecessary and harmful invasive procedures.  Because lung cancer screening has 
substantial benefits and harms, and the net benefit can vary dramatically across eligible smokers, the USPSTF 
strongly advocated for shared decision making and asserted that “the decision to begin screening should be the 
result of a thorough discussion of the possible benefits, limitations, and known and uncertain harms.”110 
Furthermore, in a February 5, 2015, decision memo, the Centers for Medicare and Medicaid Services (CMS) 
required a shared decision making consultation between patients and providers for payment of initial lung cancer 
LDCT screening. The role of shared decision making was specified within CMS guidelines as “including the use 
of one or more decision aids, to include benefits and harms of screening, follow-up diagnostic testing, over-
diagnosis, false positive rate, and total radiation exposure.” 
Shared decision making (SDM) is the process wherein the provider communicates medical information about the 
screening options and patients are responsible for sharing their values and screening preferences. 
Following the sharing of information by both parties, patients and providers collaborate to decide on the course 
of screening that reflects both the best medical evidence as well as the patient’s preferences and values.111,112 
The mandate for shared decision making by the USPSTF and CMS is reflective of the strong evidence that it can 
improve patient-centered outcomes. Research has found that patients who participate in medical decisions tend 
to be more satisfied with their care,113 report a better quality of life,114 and are more adherent to 
recommendations115 than those who do not. Furthermore, models of informed consent and SDM emphasize  the 
importance of presenting the pros and cons of all treatment options to patients,116 and shared decision making is 
strongly advocated by many as an optimal method to promote patient-centered care, including in VA117 (e.g., 
2006 priorities and patients’ rights brochures). 
To meet this need for shared decision making, we created “Decision Precision”, a provider facing web-based 
decision support tool. This tool provides a method for providers to implement evidence-based guidelines for 
SDM in lung cancer screening. Our goal is to have the tool facilitate SDM within the confines of a busy clinical 
schedule. This web tool goes beyond current VA patient materials in several ways. It provides: 1) personalized 

quantitative risk assessment of the trade-offs; 2) patient-friendly language; 3) graphics that have been empirically 
demonstrated to help patients understand their personalized risks and benefits; and 4) quick and easy 
documentation of personalized SDM after using the tool. 
“Decision Precision” Tool Development. To determine the best method for communicating the benefits and 
harms of lung cancer screening, we first conducted a randomized survey experiment with 1,612 adults aged 18 
years or older who reported smoking daily or almost daily. We compared comprehension and perception of lung 
cancer screening benefits and harms when the information was presented using four different evidence- based 
formats, including the pictograph currently used vin VHA’s patient lung cancer screening materials. We found 
that participants who viewed the VHA pictograph had better knowledge about the magnitude of the benefit and 
how this benefit compared with important harms. Thus, we used the VHA pictograph in the tool. In addition, we 
iteratively designed four versions of the web tool based on usability testing with decision aid researchers, PCPs, 
and patients. The final version will be developed during Year 1 and will incorporate additional content and 
features suggested by front-line providers and patients, along with a module to enable personalized screening 
recommendations based on provider-determined risk thresholds. 
3.1 Procedures 
3.2 Proposed Implementation Strategies 
We will compare the effect of adding the LEAP implementation strategy (see Implementation Core) to more 
typical and widely studied strategies that we will consider standard (i.e., audit and feedback and introductory 
webinar discussing use and benefits of the tool). We will also slowly roll out risk threshold tailoring to determine if 
that improves implementation outcomes. 
Implementation Strategies:  Control 
1. Webinar, Promotion, and Tool Access. All sites will receive a professionally developed, 15-minute webinar that 
describes the tool’s development (e.g., how the algorithm was designed) and a tutorial on how to use the web-
site. We will promote the webinar and using the tool through key local leaders. A web-link to use the tool will be 
placed within the lung cancer screening clinical reminder. 
2. Audit and Feedback. Due to the demonstrated importance of audit and feedback for changing provider 
behavior, we will deploy an audit and feedback system to all sites.102,118 This system will provide feedback on 
the screening and shared decision making process (e.g., number of provider’s eligible patients screened for 
lung cancer, use of the tool, patient knowledge and satisfaction from patient surveys). When we visit the 
intervention sites to conduct usability testing (see below), our informationist from the Implementation Core will 
accompany us and meet with users to identify their needs and preferences for feedback (see Implementation 
Core). We will use this information to design an audit and feedback system to be implemented in all of the sites. 
Implementation Strategies:  Two Interventions 
1. An additional part of LEAP will be user-centered design. Prior to the implementation of the web-based tool, 
we will also visit the “intensive” implementation sites and conduct usability testing on the tool.119 Usability testing 
will be conducted with providers, nurses, lung cancer screening coordinators, and patients. We will go through 
the tool page by page to get feedback on the tool. We have done this in numerous studies and have specific 
protocols in place to do so. To the extent possible, we will use this feedback to modify the tool for these sites. 
Evidence suggests that when providers feel more engaged in the design of the tool, they will be more likely to 
value and use it.124-126   During this time we will also meet with the local Clinical Applications Coordinators to 
discuss adding a link to the Decision Precision website within clinical reminders. 
2. Risk Threshold Tailoring. Providers at all sites will determine their own risk thresholds for making personalized 
recommendations within the tool. This will be done in a stepped-wedge design and will add a new site every two 
months until providers at all sites have been given the opportunity to tailor their thresholds. After completing a risk 
tailoring tutorial, they will identify two risk thresholds: 1) a low risk/low benefit threshold below which the provider 
would generally suggest that most patients not get screened (e.g., a number needed to screen of >500 to avoid 
one lung cancer death) and 2) a high risk/high benefit threshold above which they would generally suggest that 
most patients get screened (e.g., a number needed to screen of <100 to avoid one lung cancer death). In 
between these two thresholds is a gray zone where screening and not screening are presented as equally good 
options. We will provide significant leeway in allowing providers to make these cut-offs, but will impose certain 
constraints to make sure the thresholds do not violate professional standards. Of course, every screening 
recommendation will be accompanied by a recommendation for shared decision- making. We believe this strategy 
will address several potential barriers to implementation of clinical support systems—including provider’s lack of 

comfort with making decisions based on numeric risk-benefit information, lack of agreement with tool content, or a 
feeling of being taken out of the decision making process.120-123 
 
3.3 Study design and methods 
Aim 1:  To determine effectiveness of LEAP plus Audit and Feedback (A&F) versus A&F alone. 
We will use multi-site, cluster-based randomization to evaluate the effect of the LEAP implementation strategy. 
Currently, there are eight VHA sites involved in the LDCT demonstration project and each site has agreed to 
be involved in our implementation initiative (see attached letters of support). We decided to only use these 
sites because they already have a successful lung cancer screening program in place and thus provide an 
ideal situation to test different implementation strategies for a decision support intervention. NCP will 
randomize which of the eight sites receive the LEAP implementation strategy. 
Aim 2:  To determine the effectiveness of allowing providers to set risk threshold algorithms. 
Once the providers have experience with using the tool in practice, we can then start to roll out the risk 
threshold tailoring. We will roll this out at a rate which is limited by our capacity to do the tailoring and 
accompanying training in only one site at a time, but eventually reaching all of the sites by 18 months into the 
study. We will evaluate this additional part of the implementation strategy by comparing at different time points 
the sites that have received the training and tailoring to those who have not yet in what is effectively a stepped 
wedge analysis. Thus, we will enroll one site every two months into this strategy, stratified by whether the site 
is receiving the LEAP strategy, over 16 consecutive months. 
 
Evaluation Framework. We will use the RE-AIM framework (Reach, Effectiveness, Adoption, Implementation, 
and Maintenance) to evaluate our two implementation strategies. Specific measures for the implementation 
evaluation are detailed in Table 2. Our primary goal is to determine whether these strategies lead to more 
frequent use of the Decision Precision Tool and align screening utilization with the magnitude of the mortality 
benefit from screening. While lung cancer screening is a preference sensitive decision and thus for any given 
patient, we cannot assess whether they made a “good decision”, it is possible to evaluate the strength of the 
association between mortality benefit and screening utilization. If our tool is successful in communicating the 
benefit-harm information to providers and patients, we would expect that those with the largest potential to 
benefit would be more likely to be screened than those patients who have limited potential to benefit (we are 
calling this “precision decision making”). Based on these goals, our primary outcomes will be:  1) the proportion 
of eligible Veterans for whom the Precision Decision tool is used; and 2) ability of intensive implementation to 
promote precision decision making. 
Data Collection. The majority of the data will be collected from Health Factors data within the VA Corporate 
Data Warehouse (CDW) and from ‘paradata’ automatically generated from use of the tool—i.e., data recorded 
by the website, which allows us to determine use of the various components of the Decision Precision tool. We 
should emphasize that paradata do not contain PHI—rather it includes which pages were viewed and for how 
long. Patients will also complete surveys to provide data for the audit and feedback system. We will query the 
Health Factors table on a weekly basis to identify patients who had initial screening conversations with a 
provider that week (i.e., clinical reminder completed) and will mail surveys to these patients. 
Table 2:  Implementation Evaluation Measures using RE-AIM 
RE-AIM Construct 
Study Measures 
Reach 
Participation rate in the target population (patient level). 
Proportion of eligible patients with documented tool use after 
screening discussions (primary outcome). 
Effectiveness (Aim 1) 
The impact of the intervention on targeted outcomes. 
• Precision decision making (primary outcome) 
• High quality decision making 
• Patient satisfaction with decision and process 
Adoption (Aim 2) 
Percent of providers who adopted the guidelines. 
• # of tool assessments for each provider 
• % of provider’s eligible patients with documented tool use 
Implementation (Aim 2) 
The quality and consistency with which the guidelines 
were followed. 
# of tool assessments where: 
• Patient decision aid was printed 
• Shared decision making documentation was copied 
• Dynamic pictograph depicting personalized benefit and 
harm was opened for display 
Maintenance (Aim 2) 
Degree to which the guidelines were followed over time. 
A re-assessment of each measure defined above will be 
completed during the maintenance phase. 

Outcome measures. The outcome measures are specified in Table 2. The proportion of eligible patients with 
screening discussions is defined as those patients without documented exclusions for screening who have had 
initial lung cancer screening clinical reminders resolved.  Documented tool use is defined as provider selection 
of the “use tool” radio-button that will be inserted into the clinical reminder (‘Reach’ construct). “Precision 
decision making” is defined as screening utilization aligning with Veterans’ potential screening benefit. This will 
be estimated by inputting CDW data into a validated model.110,127,128 High quality decision making, obtained 
from patient surveys, will be defined as 1) adequate knowledge about the benefits and harms, and 2) values 
consistent with the screening decision.129-131 Additional outcome measures include decisional conflict,112 patient 
perception of communication with physician,132 participation in shared decision making,133 and satisfaction with 
decision and decision making process.134 
Power Considerations. Our unit of analysis is site. Randomizing interventions by provider or PACT team is 
not feasible and would also likely result in significant contamination between arms. LEAP Strategy: Even with 
our small sample size (n=4 in each arm), we will still be able to detect an absolute difference of 25-35% from a 
control rate of 50% documented tool use, for an ICC of .05 to .10 and 400 total patients undergoing screening 
discussions in each site. We believe it would be necessary to see substantial differences in tool uptake in this 
range with the LEAP strategy relative to the control strategy to justify the resources that would be needed for 
widespread dissemination. We should note that it should be feasible to achieve this sample size as roughly 50 
patients were eligible for the tool per month at each site during January, February, and March of 2015 (based 
on personal communication from NCP). Risk Threshold Tailoring Strategy: For this the stepped-wedge design 
with an average of 400 total patients undergoing screening discussions per site, we can detect a much smaller 
absolute difference in documented tool use of 7.5% from a control rate of 50%. We can also readily detect a 
small effect size (0.15 standard deviations) in any of our continuous variables. 
Analysis. To compare statistical differences in outcomes between LEAP implementation and control sites, we 
will use generalized estimating equations (GEE) to control for the clustering of observations within sites. 
Models will include the specific outcomes from Table 2 as the dependent variables and each of the two 
treatment variables (LEAP and Risk Tailoring) as we effectively have a form of factorial design that allows us to 
separately test the effects of each strategy while conditioning on the presence of the other. We will also include 
the estimated lung cancer risk for each individual patient as a pre-specified, pre-treatment variable in each 
regression. As this variable is likely to be associated with many of our key outcome variables (like screening 
rate), it can substantially increase our power as described by Raudenbush.135 For the models examining one of 
the primary outcomes for the risk tailoring intervention, precision decision making, we will look at whether the 
odds of screening as a function of the patients estimated risk of cancer increase after the risk tailoring 
intervention. This reduces to testing the significance of the interaction between estimated cancer risk and the 
treatment variable indicating the risk tailoring strategy. 
Aim 3: To conduct a formative evaluation to determine the factors most important for successful 
implementation of risk-based shared decision making tools. 
Near the end of the study, our qualitative analysts from the Implementation Core will conduct telephone 
interviews with those providers at each of the sites who identify themselves as participating in the shared 
decision making process with patients. Prior to the meeting, we will conduct a separate telephone interview 
with the clinician from each site who oversees the lung cancer screening program. The interview will consist of 
a survey (Appendix B) that briefly describes each of the CFIR constructs, and the provider will be asked to 
consider how important each factor was in affecting successful implementation of the Decision Precision tool. 
Those items that the lead clinicians across sites deem to be very important in the successful implementation of 
the guidelines will form the basis for developing our semi-structured interview guide to use with the other 
participating providers at the sites. The purpose of these questions will be to gain a better understanding of the 
barriers that occurred, as well as those factors that facilitated implementation. Coding and analysis of the 
responses will be done according to the protocol described on the CFIR website and in Appendices C and D. 
In addition, we will survey all providers for their feedback on tool usability, including:  1) perceived use of the 
tool, 2) satisfaction with the tool, 3) satisfaction with engaging in shared decision making with patients where 
the tool was used, and 4) their preference to use the tool going forward. Finally, we will ask providers whether 
any of the implementation strategies we used (audit and feedback, involvement in user-centered design, 
determination of risk thresholds, LEAP) influenced their use of the tool. 

4.0 Impacts 
Unfortunately, even though there is strong evidence of the benefits of shared decision making, there is 
equally strong evidence of the failure to implement SDM into practice.136 While research studies are able to 
use ample resources (e.g., research assistants, participant incentives) to successfully use decision aids or 
other interventions to facilitate SDM, implementation of these methods often fails without the structure of 
research. A number of barriers to implementing SDM within primary care have been consistently 
documented, including provider perceptions about the time it will take and limited provider comfort with risk 
communication tasks.121,137,138 Within the context of lung cancer screening in VHA, a pilot study conducted 
by Dr. Wiener on our team revealed poor understanding of the pros and cons of LDCT screening among 
both Veterans getting screened and the VA PCPs offering the service. Given the critical importance of 
shared decision making in the context of LDCT screening, we must do better. 
5.0 Partnerships/Management 
Expertise and specific responsibilities of the team are detailed in the budget justification. 
PI:  Angela Fagerlin, PhD, Core Investigator, VA CCMR; Associate Professor of Internal Medicine, 
University of Michigan, and Co-Director of the Center for Bioethics and Social Sciences in Medicine. 
Co-PI:  Tanner Caverly, MD, MPH, HSR&D Advanced Fellow, VA CCMR; Clinical Lecturer, Department of 
Medicine, University of Michigan. 
Co-Investigator:  Renda Wiener, MD, MPH, Physician and Core Investigator, Bedford/Boston Center for 
Healthcare Organization & Implementation Research; Assistant Professor of Medicine, Boston University 
School of Medicine. 
Staff from the University of Michigan Center for Health Communications Research (CHCR) will be 
responsible for making changes to the Decision Precision tool and will work closely with Drs. Caverly and 
Fagerlin on changes in content. 
Support for project management, data management, and data analysis will come from QUERI core staff. 
Partnerships. As the office responsible for overseeing the lung cancer screening demonstration project, NCP 
(Linda Kinsinger) has already informed the sites that NCP supports the use of this tool and encourages 
participation in this project. If the tool proves to be successful, they will include it as part of the lung cancer 
screening guidelines to be disseminated throughout VHA. Primary Care Services (Gordon Schectman) will 
also indicate their support to primary care teams for their participation in LEAP. In addition, as tools from this 
project are vetted by the sites and are viewed positively by providers and patients, Primary Care Services 
will work with us to design ways to promote dissemination throughout VHA. We will schedule annual 
conference calls (at a minimum; more frequently as issues and findings of interest arise) with our partners to 
update them on our progress and obtain their input on next steps. 
Figure 3 depicts the proposed timeframe for the project.  Roll-out to all of the sites will be completed by 
Y3Q4; we plan to continue with audit and feedback through Y4 to allow one year of follow-up to evaluate 
implementation outcomes over the longer term (maintenance). 
Figure 3:  Timeframe for Project Activities 
 
Project Period: 10/1/15–9/30/19 
Activity 
Year 1 
Year 2 
Year 3 
Year 4 
1 
2 
3 
4 
1 
2 
3 
4 
1 
2 
3 
4 
1 
2 
3 
4 
Materials development 
User testing of tool 
Revision of Decision Precision 
RA training 
Implementation 
Qualitative interviews 
Analysis 
Manuscript and report writing 
 

Letter of Support from Linda S. Kinsinger, MD, MPH, Chief Consultant for Preventive Medicine, VA National 
Center for Health Promotion and Disease Prevention, dated May 26, 2015 describing that this project is quality 
improvement and not research: 

providers on the quality of the decision process, including whether patients have an accurate understanding of 
the benefits and harms, and whether the ultimate screening decision is consistent with patients’ values.   
 
Justification for Exemption of this Study under 5 CFR Part 1320.3: 
 
This is a Quality Improvement project designated by the National Center for Health Promotion and Disease 
Prevention (NCP) and the Office of Primary Care Services (PCS). Results from this study will be shared with 
NCP, PCS, and other policymakers and VA stakeholders. NCP supports the use of the Decision Precision tool 
and encourages participation in this project.  If the tool proves to be successful, NCP will include it as part of 
the lung cancer screening guidelines to be disseminated throughout VHA. PCS will also indicate their support 
to primary care teams to participate in the project. In addition, as tools from this project are vetted by the sites 
and are viewed positively by providers and patients, PCS will work with us to design ways to promote 
dissemination throughout VHA. 
 
This research will prevent clinical disorders by: 
Use of the Decision Precision Tool with risk threshold tailoring will help align lung cancer screening utilization 
with the magnitude of the mortality benefit from screening. If our tool is successful in communicating the 
benefit-harm information to providers and patients, we would expect that those with the largest potential to 
benefit would be more likely to be screened than those patients who have limited potential to benefit. 
Justification that the Surveys are Not Duplicative:  
The survey is not duplicative because: 
QUERI has funded this project based on its originality in addressing the specified RFA requesting projects to 
improve the health of Veterans by supporting the more rapid implementation of effective clinical practices into 
routine care. This decision was based upon merit review. This quality improvement project is unique in that it 
will implement and evaluate effectiveness and use of a shared decision making tool for lung cancer screening 
to inform practice in VA nationwide. Patient surveys conducted in this project are not duplicative of any other 
survey with this population.