Impact of Family History and Decision Support on High-risk Cancer Screening
Phase: NA
Conditions: Colorectal Cancer
Sponsor: VA Office of Research and Development

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
1 
 
 
Title: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
Principal Investigator: Corrine Voils, PhD  
Durham Site Principal Investigator: Karen Goldstein, MD 
Co-Investigators: Deborah Fisher, MD; Elizabeth Hauser, PhD; Ryanne Wu, MD; Maren Scheuner, MD 
(Greater LA); Lori Orlando, MD (DUMC); Dawn Provenzale, MD; Nina Sperber, PhD; Virginia Wang, PhD; 
Cynthia Coffman, PhD; Richard Nelson, PhD (Salt Lake City) 
 
Clinicaltrials.gov identifier NCT02247336 
Version 16 
February 11, 2020 
 
 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
2 
 
Introduction. Risk assessment through family health history (FHH) permits tailoring of prevention 
strategies and treatment regimens to an individual’s familial risk level. Although FHH is commonly 
accepted as an important risk factor for common, chronic diseases, it is rarely used in clinical practice as 
part of a structured risk assessment due to time constraints for the office visit, lack of standardization of 
FHH collection, failure to consolidate access into one location, and lack of confidence in identifying high-
risk patients.1-3   
The Genomic Medicine Model (GMM) was developed by our university affiliate colleagues to 
address barriers to providers’ use of structured risk assessment. The GMM comprises three components: 
1) education of primary care providers (PCPs) and patients; 2) an IT-based platform, MeTree, that collects 
FHH from patients and generates a pedigree and decision support reports for patients and PCPs; and 3) 
provides support for patients and PCPs to interpret FHH information and adhere to recommendations. A 
standardized FHH assessment accompanied by decision support facilitates 3 important objectives: 1) 
identification of patients at higher risk, 2) genetic consultation referral for patients at risk for hereditary 
cancer syndromes, and 3) recommendation of guideline-based preventive care for patients of all risk 
levels.  
Currently, the Veterans Health Administration (VA) has no system-wide, comprehensive, validated 
method for assessing FHH or providing decision support. The GMM offers a promising strategy for 
identifying patients at risk and providing risk-appropriate referrals and screening.  
The goal of this study is to evaluate the feasibility and effectiveness of the GMM components (i.e., 
educational materials, FHH collection via MeTree, and decision support) in VA primary care practices to 
improve risk-appropriate CRC referrals and screening. This 3.5-year, mixed methods study involves a 
cluster randomized trial in which PCPs and their patients are randomized to immediate vs. delayed (wait-
list control) patient-entered FHH data collection and decision support. The primary aims are: 
 
Aim 1: Determine whether FHH collection via MeTree improves identification of patients at higher 
familial risk for CRC by comparing rates of high-risk identification in the medical record prior to 
study enrollment to rates of high-risk identification following MeTree completion. 
Aim 2: Evaluate whether providing decision support to patients and PCPs improves risk-
appropriate PCP referrals for, and patient uptake of, CRC screening/surveillance. 
 
The secondary aims are: 
 
Aim 3: Assess experience with decision support and effects on workflow from PCPs, and obtain 
information to inform eventual implementation in the VA healthcare system from administrative 
leaders, via qualitative interviews. 
Aim 4: Conduct cost-consequence and budget-impact analyses of implementing FHH collection 
and GMM decision support in VA. 
 
If MeTree effectively increases identification of high-risk Veterans, and decision support effectively 
increases appropriate CRC screening for high-risk Veterans, then we will be poised to inform and 
evaluate implementation of GMM in VA on a larger scale and for a larger disease set.  
 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
3 
 
Methods for Aims 1 and 2 
Sample selection and enrollment.  We will enroll primary care providers (PCPs) and patients from the 
Madison VAMC, Durham VAMC, Raleigh, Hillandale, and Greenville clinics. To achieve our target sample 
size of 500 patients who complete MeTree, we will enroll all eligible providers. We anticipate enrolling 40 
providers, with 12-13 patients per provider; however, the number of patients enrolled per provider may 
vary if fewer providers enroll. PCP and patient eligibility will be determined from the following study 
inclusion/exclusion criteria.  
PCP inclusion criteria will be determined from clinic administrative staff: 
• 
Primary care physician, physician assistant, or nurse practitioner  
• 
Working in one of the targeted primary care clinics 
• 
At least one half-day of primary care clinic per week 
 
We will email recruitment letters briefly describing the study and informing PCPs that we will be attending 
a staff meeting to discuss study details. Following methods used in other Durham HSR&D studies, Drs. 
Voils, Fisher, and Wu will attend staff meetings to inform providers about the study goals and methods. 
We will emphasize that the study is not meant to increase burden in their already busy clinics, but, rather, 
to facilitate guideline-appropriate referrals. No education regarding family health history (FHH) or 
colorectal cancer (CRC) will be provided during these meetings.  
Providers will be recruited in a manner similar to previous HSRD-funded projects. We request a waiver of 
documentation of informed consent from providers. Study staff will introduce the study to providers in one 
of two ways: 1. during a provider/staff meeting or 2. via a recruitment email.  Providers who attend the 
meeting and would like to consent may sign the consent for at the time of the meeting.  Study staff will 
email all PCPs who did not attend or provide consent at the meeting a message containing a description 
of the study and all required elements of informed consent to participate in the study.  These PCPs may to 
participate in the study via an affirmative email response.   If the study team does not receive a response 
from a provider at the time of the provider meeting or within 1 week from sending the email, the study 
team may re-contact the provider by email, or approach the provider in-person to ask whether they are 
willing to participate. The method of consent and consent date will be documented in the study tracking 
database. The study team will also keep the email responses as documentation of willingness to 
participate. We will use a rolling recruitment strategy to capture any new providers.  
Consented providers will be randomized to the immediate or delayed condition, in which a small subset of 
their patients will complete MeTree and receive decision support at enrollment or 12 months later. PCPs 
randomized to the immediate group will then receive additional with details of the study and educational 
materials throughout the study’s recruitment period (those randomized to the delayed group will receive 
this information 12 months later). This additional detail includes information on the role of FHH in CRC 
and a description of the procedures to ensure that the decision support documents are available for the 
patient appointments. We will ask PCPs in this group not to discuss the decision support documents, 
process, or experience with other clinicians. We will minimize education provided to delayed wait list 
(control) providers by sending separate educational materials prior to initiating family history completion 
and decision support to wait-list control participants 12 months post-enrollment.  
Patient eligibility criteria will be determined by electronic data abstraction:  
• 
Assigned to one of the enrolled PCPs 
• 
At least one primary care appointment in the 18 months prior to enrollment 
• 
Upcoming PCP appointment with assigned PCP 
• 
Aged 40-64 years, confirmed via telephone screening 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
4 
 
• 
No previous history of CRC (V10.05, V10.06, ICD-9 153, 154), inflammatory bowel disease (555.0,  
 
555.1, 555.2, 555.9, 556.0) or adenomatous polyps (indicated in pathology report) 
• 
No endoscopy within the previous 3 years, indicated in inpatient/outpatient encounter data and  
 
confirmed via telephone screening 
 
Additional patient eligibility criteria will be determined by telephone screening: 
• 
English as preferred language  
• 
No plans to relocate or leave the VA healthcare system in the next 12 months 
• 
No more than 2 errors on a validated 6-item screener for cognitive impairment 
• 
Have at least some knowledge of FHH of first- and second-degree predecessor relatives, even if 
adopted  
• 
Willing to have their family history data maintained at Duke for future research use  
• 
No concurrent enrollment in a competing research study (related to colon cancer) 
 
Patients with an upcoming appointment (usually at least 6 weeks out) with their assigned PCP will receive 
a recruitment letter by mail in addition to an informational brochure about MeTree. Patients may opt out of 
the study by calling a toll-free number; otherwise, study staff will contact potential participants by 
telephone to describe the study, confirm knowledge of FHH, and obtain verbal consent. Verbal consent is 
used because participants may elect to complete the family history tool online; in such cases, the only 
purpose in attending an in-person visit would be to provide written informed consent. (HIPAA 
Authorization may be obtained by mail or fax, as explained below, or may be waived.)  
 
After providing verbal consent, participants will complete a telephone survey of self-reported screening 
(Appendix 1). One measure will be a self-report questionnaire that assesses the various recommended 
CRC screening test modalities, which was previously validated in the Veteran population by Drs. Fisher, 
Voils, and Coffman.39 For the other types of cancer screening, we will query receipt of breast magnetic 
resonance imaging, mammogram, , pelvic ultrasound, or genetic testing for screening purposes; these 
data will be examined in exploratory analyses. An advantage of collecting self-reported data is that we can 
capture receipt of any test received outside the VA. At this time, we will also administer a validated tool 
designed to assess computer proficiency, as this is a computer-based intervention.  After completing the 
survey, participants will be informed of their randomization assignment, which is determined by whether 
their PCP was randomized to the immediate vs. delayed condition.  
 
Immediate Condition. Participants in the immediate group will be provided with the choice of 
completing MeTree on their own online (at home, library, or wherever they can access it) or in 
person prior to their PCP visit. Participants in the delayed group will have this choice 12 months 
later. After receiving the signed HIPAA authorization form, or after completing the consent process 
when HIPAA is waived, study staff will log into the Duke software to register the patient user and 
will provide the patient with username and password, both of which will be identical, non-
identifying, and generated by our tracking database. The username and password will be provided 
to the participant by mail, text message, or MyHealtheVet Secure Messaging, depending on 
veteran preference. 
 
Delayed (wait-list control) Condition. We will inform control participants that we will contact them 12 
months post-consent to complete MeTree. We will ask for contact information for a family member 
or friend to facilitate future contact. These participants will be informed that there is little risk 
associated with waiting, as CRC takes 7-10 years to develop. During the 12-month period, neither 
control patients nor their providers will receive further contact from study staff. Ten to 11 months 
following consent of the first participants, we will contact wait-list PCPs to provide the same 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
5 
 
education that was provided to the immediate group. Also at 10 months following patient consent, 
we will mail a reminder letter to control patients informing them about the process to complete 
MeTree. At 11 months following baseline, we will telephone control participants to assess 
preference for mode of completion using the aforementioned procedures. If we are unable to reach 
control participants by telephone by 12 months following consent, we will mail another reminder 
letter. We will continue to try to reach control participants by telephone for up to 6 weeks following 
the 12-month time point. Note that control participants may not have an appointment scheduled at 
the 12-month time point. We will follow the same procedures as in the immediate arm with regard 
to uploading the PCP decision support document requiring PCP signature. PCPs may choose to 
order tests or schedule an appointment with control participants who are identified as high-risk. 
Likewise, patients who are informed that they are high-risk by the patient decision support 
document may call to request an appointment with their PCP following usual care procedures. 
 
Provider Departure. If a patient’s provider leaves before they are contacted by the study team, the 
participant’s randomization will match that of their new provider, or the patient will be ineligible if 
the new provider does not consent to participate. Participants who have already completed MeTree 
prior to their provider leaving the VA will keep their assigned randomization per intent-to-treat 
principles.  
 
Procedures (both groups). Consented participants will be provided with access to a worksheet to assist 
them in collecting FHH from their family members prior to logging onto MeTree. The worksheet is 
available for download from the MeTree website, but can also be mailed to the participants, depending on 
timing and preference. The worksheet queries names of children, parents, siblings, nieces/nephews, 
aunts/uncles, and grandparents; age now or at death of these individuals; diseases these individuals have 
had and age at diagnosis; and cause of death. A list of types of cancers, hereditary cancer syndromes, 
and other diseases is provided, along with definitions of various diseases and causes of death. 
Participants will be asked for their FHH for all of these diseases because MeTree has been designed to 
collect information on these diseases and decision support for additional diseases is ongoing. In this 
study, the primary outcome relates to CRC, with referrals and uptake of recommendations related to these 
other diseases as secondary outcomes.  
 
Information for accessing the MeTree website will not be granted until a signed HIPAA Authorization has 
been received, unless HIPAA is waived during the consent process. Participant login information will be 
generated by Lesa Powell, OIT programmer, to meet all VA security requirements. The list with the login 
information and pin will be kept in a separate file from the database containing participant data (phone 
numbers) on a Durham HSR&D server (\\vhadurhsmcifs01.v06.med.va.gov \Family History), with access only 
available to VA team members. This will not contain PHI, and will be told to participants via a phone call, 
letter or text message.  All text messages will be sent from a study specific phone number used only for 
texting and calling participants. All family history data will be entered on the Duke MeTree site. The family 
history data will be maintained indefinitely by Duke and combined with family history data collected using 
the MeTree software from non-veterans at other sites. These data will be used for future research 
purposes at Duke. All other data, including those abstracted from participants’ CPRS records, will be kept 
in VA files only. Patients will be contacted by study staff with reminders as needed (by text message, 
MyHealtheVet Secure Messaging, or phone) to return their signed HIPAA Authorization form, and to 
complete MeTree before their appointment.  If necessary, patients may meet study staff to complete 
MeTree on a study computer.  Participants who initially elect to complete MeTree in person, and who later 
elect to complete MeTree with a study staff member because they did not do it on their own, can complete 
the HIPAA Authorization in person prior to completing MeTree on a study laptop in the presence of study 
staff. Patients who are screened as eligible but who never log in to complete MeTree either on their own 
or in person will not provide data for the study and thus will not be counted toward the required sample 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
6 
 
size of 500. Thus, we may screen and consent up to 850  patients in order to enroll 500 who complete 
MeTree. 
 
Once patients have completed MeTree, they will be able to download the patient decision support 
document and print it. Study staff will log into a secure Duke website to download the provider decision 
support document, which will be identified by a random study ID number and saved to the project folder 
on the VA server. A study staff member will input the decision support document findings as a CPRS note 
requiring signature (and thus review) from the PCP. Because participants are recruited based on an 
upcoming PCP appointment, this CPRS note should be recent and will be easy for PCPs to find. PCPs 
will be responsible for ordering recommended tests, including consults to GMS as they deem clinically 
appropriate for the patient (given patient preference, comorbidities, etc.). Patients who complete MeTree 
on their own (outside clinic) will have access to download and/or print the decision support documents, 
whereas participants who complete MeTree in person can receive the decision support document from 
study staff. All veteran participants will receive $50 after completion of MeTree. This will be paid in a 
check or by electronic funds transfer via the agent cashier or a payment voucher.  
 
Twelve months following consent, participants in both arms will be contacted by telephone to complete the 
same survey of self-reported cancer screening as completed at baseline. This follow-up survey will also 
ask about several additional screenings, including the fecal occult blood test (FOBT) for colorectal cancer 
and the CA-125 blood test for ovarian cancer.  All participants will complete an exit survey about 
acceptability of using MeTree (Appendix 2). This survey will be administered via telephone call or by mail, 
with data entered into an Illume survey. In the immediate arm, the exit survey will be administered via 
telephone call placed after patients have completed MeTree; in the delayed arm, the exit survey will be 
administered in the same call in which self-reported screening is assessed.  
 
Then CPRS and administrative data will be reviewed to determine the baseline rate of patients identified 
at increased risk for CRC based on FHH. This rate will be compared to the rate obtained subsequent to 
completing MeTree so that the effectiveness for increasing the identification of higher risk patients can be 
determined. CPRS data abstraction will be completed by research staff at either site (Durham or 
Madison). Study staff at both sites (Durham and Madison) will have National Electronic Health Record 
(EHR) access, using VistAWeb or Joint Legacy Viewer (JLV), in order to complete these data 
abstractions. Medical records will be accessed using VistAWeb, JLV, or CPRS and the resulting data will 
be recorded in DatStat Illume, behind the VA firewall.  
Blinding cannot be ensured because the MeTree decision support document for PCPs will be scanned 
into CPRS, and there is no practical way to prevent the RA from seeing it. At baseline, all patients enrolled 
in the study will have their EMR audited to determine whether they have been identified as high risk for 
CRC as observed by either: (1) a diagnosis of high-risk for CRC (diagnosis code: V16.0) or (2) 
documentation in CPRS. The RA will review progress notes from all providers for documentation of FHH 
of CRC, including the relationship and age at diagnosis or explicit statement of no FHH of CRC. This will 
be accomplished with a text search. Initially, we will validate our search terms by reading all progress 
notes in a subset of 25 patients. If necessary, the terms will be adjusted to accommodate those found in 
the progress notes, and validation will be repeated on an additional 25 patients.  
 
 
 
 
 
 
 
 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
7 
 
 
Analyses for Aims 1-2 
 
Sample Size and Power Considerations. The CRC 
screening rate of patients aged 50 years and older in VA is 
high, around 80%.44 However, this is based on the 
assumption that all patients are average-risk; as such, the 
risk-appropriate referral rate is not well-characterized. We 
estimate that the risk-appropriate CRC screening referral 
rate is 60% to 70%. Our final sample size calculation is 
based conservatively on the screening referral rate of 
60%. Based on a type I error rate of 5% and Z test for the 
difference of two proportions adjusted for provider 
clustering, we will need approximately 480 patients (240 in 
each arm) to detect a 13% difference (i.e., 73% in the 
immediate arm) with 80% power. Table 1 provides 
estimated differences that can be detected with this sample size for different CRC screening referral rates. 
Lost to follow-up either due to death or the Veteran leaving the VA system will be minimal as outcome 
data are collected administratively.  We inflated the sample size to 500 patients to account for a 4% loss 
to follow-up. Patients who screen as eligible but who never log in to complete MeTree either on their own 
or in person will not provide data for the study and thus will not be counted toward the required sample 
size of 500. Thus, we may screen and consent >500 patients in order to enroll 500 who complete MeTree. 
 
By design, patients may not be independent because they are clustered within a participating provider.  
The “relatedness” of patients within a PCP is measured by the intraclass correlation coefficient (ICC) and 
must be considered in the sample size estimation.  We have a fixed number of PCPs (20 in each arm). 
The sample size for difference in proportions is adjusted by the intraclass correlation coefficient (ICC); we 
used an ICC of 0.02.45  Although an ICC is difficult to estimate in advance of a study, we believe this is an 
appropriate and conservative estimate based on our own and others’ research findings.  A recent survey 
article reviewing outcomes from 31 studies found a median ICC of 0.01 (IQR 0 to 0.03).46 PASS 2008 was 
used for all sample size calculations.  
 
For the secondary outcome, uptake of risk-appropriate CRC screening, our sample size is smaller as this 
outcome is only available for Veterans that were referred for a CRC screening/surveillance test. We used 
a hypothesized CRC referral rate of 60% in the wait-list control arm and 73% in the immediate arm for the 
power calculation. Based on a hypothesized CRC screening/surveillance uptake rate of 60% in the control 
arm, type I error rate of 5%, and an ICC of 0.02, we can detect a difference of 18% in uptake of CRC 
screening test rates between arms with 80% power. Table 2 provides estimated differences that can be 
detected with this sample size for different CRC screening/surveillance uptake rates. For Aim 1, the 
analysis is a pre-/post- type comparison of high-risk CRC identification rates. With a sample of 500 
Veterans, we will have adequate power to detect clinically reasonable differences in the rates of high-risk 
CRC identification. 
 
 
 
 
 
 
 
 
 
Table 1. Estimated differences in CRC 
screening referral rates that can be 
detected between arms for different 
baseline screen referral rates with 80% 
power 
Sample 
size (per 
arm) 
Baseline 
referral rate 
for control 
arm 
Estimated 
difference 
between 
arms 
240 
60% 
13.2% 
240 
65% 
12.1% 
240 
65% 
12.0% 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
8 
 
The outcome for Aim 1 is rate of high-risk 
categorization. The analysis for this aim does not 
compare the outcome between arms as our aim is 
to determine if using MeTree to collect FHH 
improves the rate of high-risk identification for 
CRC compared to what is currently available 
administratively in VA. In our study design, both 
arms use MeTree to collect FHH, so the design for 
the aim is a pre-/post- comparison within Veterans. 
A binary variable will be created for each 
participant (1=presence of high-risk for CRC, 
0=absence) at baseline (current practice) and 
following MeTree administration. We will test 
whether the rate of high-risk identification is higher 
following the MeTree administration compared to 
what is available in the medical record under current practice using a McNemar’s test adjusted for PCP 
clustering.40,41 In addition to examining identification rates, specification of the source of identification of 
high-risk CRC will be important to formulating a standard identification strategy for clinical follow-up (e.g., 
referral to or receipt of colonoscopy screening). 
 
The primary effectiveness outcome for Aim 2 is risk-appropriate CRC screening/surveillance 
referral for all patients, regardless of risk level. Outcomes will be compared between arms to 
determine the impact of decision support on risk-appropriate referrals and screening/surveillance. The 
risk-appropriate screening/surveillance referral will be determined based on the MeTree algorithm. A 
binary variable will be created for each patient (1=adherent to risk-appropriate referral, 0=non-adherent to 
risk-appropriate referral) at 12-month follow up. The appropriateness of the referral will take into account if 
they were previously (prior to baseline visit) appropriately referred based on the current determination of 
their risk profile using the MeTree FHH data collection for both arms.  
 
Data on whether the referral was made will be extracted in a multi-phase process that maximizes 
efficiency and accuracy. First, we will examine administrative data. FOBT codes are G0328, 82270, 
82274, FOBT, FOBT1, FOBT2, FOBT3. We previously found that for 2,836 of 2,989 patients (95%) FOBT 
use identified by CPT code in the OPC was confirmed in the records of the local VA laboratory performing 
the test. Similarly, for 2,842 of 2,943 patients (97%) with an FOBT documented in the laboratory records, 
a CPT code was found in the OPC database.31 If administrative data reveal a CPT code for colonoscopy 
or FOBT, then patients will be considered to have undergone that test. If not, then we will examine orders. 
If no order exists, then we will examine laboratory reports and VISTA imaging. If no evidence of referral or 
screening is found at this point, then notes will be reviewed to determine whether participants refused, no-
showed, received screening outside VA, or there is medical justification (e.g., life-limiting illness) for not 
receiving screening. The appropriate referral guideline will be based on the risk-categorization of the 
patient from their FHH. Patients will be coded as risk-appropriate referral or not; recommendation of any 
option listed as risk-appropriate will be classified as a risk-appropriate referral. Nonadherence to risk-
appropriate referral could result from a referral being too aggressive based on risk categorization (e.g., a 
40-year old without a FHH of CRC) or that the risk-appropriate referral was not made. 
 
Patient uptake of CRC screening/surveillance recommendation will be evaluated as a secondary 
outcome. The patient uptake of CRC Screening/Surveillance referral will be examined for the subset of 
patients that received risk-appropriate referral by having an order in their record following the baseline 
visit for a CRC screening/surveillance referral. A binary variable will be created for each patient that was 
referred for CRC screening/surveillance (1=uptake of referred CRC screening, 0=no uptake of referred 
Table 2.  Estimated differences in CRC 
screening/ surveillance uptake that can be 
detected between arms for different baseline 
screening/surveillance uptake with 80% power 
as well as different CRC screen referral rates 
Baseline 
referral rate 
for control 
arm  
Sample 
size 
(control/ 
immediate
) 
Baseline 
screening/ 
surveillanc
e uptake 
for control 
arm 
Estimated 
difference 
between 
arms 
60% 
100/120 
60% 
18.0% 
65% 
100/120 
65% 
17.2% 
70% 
120/140 
70% 
15.0% 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
9 
 
CRC screening) at 12-month follow up. Data on whether participants completed the CRC screening test 
will be obtained in a multi-step process. At the time of data abstraction, we will make a determination 
about whether to examine fee basis files for screens performed by non-VA providers according to whether 
the Durham and/or Madison VA is outsourcing CRC screening.42 Patients will be considered adherent if 
their medical record shows any risk-appropriate CRC test(s) performed in the time between completion of 
the computer program in clinic and 12-month follow-up.   
 
Referral for genetic consultation will be examined as a secondary outcome. This outcome is risk-
appropriate referral for genetic consultation or not. Because the Durham VAMC is online with GMS, a 
consult option will be generated to refer patients to GMS for genetic consultation, which may or may not 
include genetic testing. The Madison VAMC is not online with GMS. Therefore, fee basis files will be 
reviewed to determine if genetic counseling or testing was performed outside VA. The RA will review 
PCPs’ consults and progress notes subsequent to the visit associated with completion of MeTree FHH 
regarding recommendations for genetic consultation. GI notes will also be reviewed to determine if a 
consult to GMS was made. Appropriate referral based on risk level will be classified as adherence. We will 
also examine the rate of receipt of a genetic consultation by conferring with GMS (which may be lower 
than the referral rate if the patient declines). As a relatively small proportion of participants are expected to 
receive genetic referral (6.4% in the Cone Health study), we will be underpowered to detect a significant 
effect. However, the descriptive rates will be informative from a policy perspective. 
 
Process data will be collected to inform eventual implementation. During the study, process data will be 
collected, including detailed records on enrollment, field notes from the study coordinator regarding the 
need to assist participants in completing MeTree, modifications to the program or decision support sheets, 
and any communication with PCPs (comments, questions). Enrollment and completion rates (the number 
of patients contacted who agree to have a baseline visit and who complete MeTree) will be calculated 
from information collected in the tracking database. At the end of the study, all enrolled PCPs will 
complete an exit survey about their experience with the intervention and impact on workflow. 
 
Analyses will compare the risk-appropriate referral rate between patients randomly assigned to the 
immediate vs. waitlist control arm. A binary variable will be created for each patient (1=adherent to risk-
appropriate referral, 0=non-adherent to risk-appropriate referral) at 12-month follow up. Patients with the 
same PCP may have more similar outcomes as compared to patients with a different PCP. Therefore, we 
plan to use Generalized Estimating Equations (GEE) as our primary analysis tool.43 The regression 
coefficients from a GEE model have essentially the same interpretation as those from a standard 
regression analysis but are more appropriate because they properly incorporate the within-PCP 
correlation. When using GEE methods, it is only necessary to specify the general structure of the mean 
and correlation between repeated observations. Because the outcome is binary, we will use a GEE with a 
logit link function and an exchangeable correlation structure. In addition, we will use empirical standard 
errors for inference because they are robust to misspecifications of the correlation structure. The 
hypothesis for Aim 1 will be tested using the following model of the mean structure:   logit(πij) =  β0 + 
ARM*β1  where πij represents probability of risk-appropriate testing adherence for patient i seen by 
physician j.  ARM is the dummy-coded intervention arm. A positive, significant estimate for β1 provides 
evidence that patients who receive the intervention immediately have higher risk-appropriate referrals for 
CRC screening/surveillance than those in the wait-list control arm. PROC GENMOD in SAS (Cary, NC) 
will be used to fit the GEE model. 
 
Secondary outcomes for Aim 2 are uptake of referred CRC screening test and risk-appropriate referral for 
genetic counseling. A binary variable will be created for each patient (1=uptake of referred CRC 
screening, 0=no uptake of referred CRC screening). Note that determination of uptake will depend on 
each patient’s risk-appropriate referral. For patients for whom multiple tests are appropriate, uptake will be 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
10 
 
defined by performance of any of them. For the referral for genetic consultation outcome, all patients 
eligible for genetic consultation based on their risk profile from FHH will have their EMR audited to 
determine whether a risk-appropriate genetic consult referral was documented. A binary variable will be 
created for each patient (1=presence of risk-appropriate genetic consult referral, 0=absence) at 12-month 
follow up. A similar variable will be created to indicate whether participants received the recommended 
genetic consultation. As these secondary outcomes are binary, we will follow the same modeling strategy 
using GEE as described for the primary outcome. In exploratory analyses, similar outcomes will be 
created for the other cancer types (breast and ovarian) as described for CRC. We will calculate 
descriptive statistics for these outcomes and, if sample size is adequate, we will fit similar GEE as 
described for CRC.  
 
The primary analyses for this cluster-randomized trial will be conducted on an intent-to-treat basis; 
participants will be analyzed in the group to which they were randomized, regardless of intervention 
adherence. Additional supporting analyses focusing on alternative, more restrictive analytic cohorts (e.g., 
per-protocol analysis) may be considered as exploratory analyses to provide additional information about 
the impact of amount of exposure to the intervention. Given that the outcomes will be determined based 
on EMR review over a 12-month period, there should be very little missing data. The most likely scenarios 
are that a participant dies during the study or leaves the VA system before completion of the 12-month 
follow-up period. However, we do not expect this to occur at a high rate; as needed, we will explore the 
impact with sensitivity analyses. 
 
The main conclusions drawn from this trial will be based on the pre-specified primary and secondary 
hypotheses outlined previously and will be tested with two-sided p-values at the standard 0.05 level. For 
both primary and secondary outcomes, stratification variables (provider type and patient age 
categorization) will be included in GEE models. Statistical analyses will be performed using SAS for 
Windows (Version 9.2: SAS Institute, Cary, NC) and R (http://www.R-project.org).   
 
Aim 3 
 
The evaluation of Aim 3 employs a single, holistic case study design with the participating Durham 
and Madison VA primary clinics as the unit of analysis.66  Although GMM will be used in 3 different 
clinics, our focus is to examine the holistic nature of GMM implementation. Should we find clinic-level 
variation in identification, referral, or screening/surveillance rates (Aims 1-2), we will use a replicated case 
study design to describe implementation experiences for each clinic. Case study design is well-suited for 
describing the implementation of GMM in the VA system, which has not been examined before.  In 
addition to permitting in-depth analysis of individual cases, case study methods offer strategies for 
systematic contextual analysis from multiple sources of information.66,67   
 
To ensure that implementation evaluation in this case study includes all the relevant evidence and 
includes rival or conflicting interpretations, we will employ a mixed methods approach, collecting 
information about GMM implementation from multiple sources of data.66,68 Quantitative data will come 
from EMR and administrative data collected in Aims 1-2.  Qualitative data will be collected via semi-
structured interviews with clinic staff and administrators in Aim 3, conducting at the conclusion of Aims 1 
and 2. These interviews will provide rich contextual data regarding clinic structure and operations that are 
otherwise unobservable in secondary data.   
 
Potential participants will include PCPs and clinic leaders (nurse manager, charge nurse, and clinic 
director) from each site, the Chief of Ambulatory Care, and the Chief of Medicine. A recruitment letter will 
be emailed; this letter will note that leaders/PCPs may opt out by contacting one of the study team 
members. If we do not hear from them, then we will follow up by telephone. Purposeful sampling in 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
11 
 
qualitative research is directed toward the selection of information-rich cases to draw “illustrative 
inferences” regarding “possibility.”69 Accordingly, we will recruit a subset of participating PCPs who are 
available and willing to answer the qualitative questions. We aim to interview 4-5 PCPs per clinic who had 
at least 10 patients complete MeTree. We also aim to interview clinic leaders who were not involved in the 
trial. Thus, we will enroll up to 25 VA employees for this aim. Patients are not key informants for this aim 
because they are not involved in clinic implementation, and their reactions to the FHH collection process 
will be assessed with quantitative measures immediately following completion of MeTree. 
 
Interview protocols will be pilot-tested among team members to ensure that questions draw responses to 
critical data from each respondent. A member of the research team will conduct interviews in-person or by 
phone. Prior to starting the conversations, written or verbal informed consent will be obtained. We will use 
a semi-structured script to conduct the interviews. All interviews will last 15-30 minutes and be digitally 
audio-recorded to facilitate data analysis. All interviews will be audio-recorded using a sparky USB device 
and saved to a folder on the VA network. Audio-recorded interviews will be transcribed verbatim, and the 
information will be systematically coded for analysis by Drs. Sperber, Wang and Voils using a directed 
approach to content analysis.70 We will develop codes based on predetermined categories, such as those 
that will form the interview questions; any text that cannot be categorized with the pre-determined scheme 
will also be identified and categorized. This is a more structured approach to qualitative analysis, and is 
appropriate when there are a priori notions of key concepts or variables that can form initial codes. The 
coding scheme will be developed and finalized through a process of discussion and consensus among the 
team and then applied to all transcripts. Through the constant comparison process, Drs. Sperber, Wang 
and Voils will sort related codes into larger thematic categories and refine the categories by comparing 
them with one another within and between transcripts.71 Reports for all text segments will be generated for 
each code to assess fit between our data and constructs from Weiner’s framework. From this, we will 
identify themes in the coded data and assess whether constructs affected implementation. A checklist 
matrix will visually display construct valences to identify patterns between (and within, if applicable) 
clinics.51,72  Finally, we will generate a conceptually clustered matrix to enable between analysis of 
implementation facilitators and barriers by each construct.   
 
Aim 4 
 
To derive intervention cost, we will take an implementation perspective. Study-related costs, such as 
study staff time to consent and randomize patients will not be included. Development costs, such as 
designing MeTree and decision support materials, will be treated as a one-time “sunk cost” that has 
already been incurred. Intervention costs will be calculated for the computer required for MeTree use in 
clinic, providing the decision support documents, and the labor cost of providing assistance to patients to 
use MeTree and generate notes in CPRS for each patient’s PCP. We will collect a sample of times 
required to perform each of these tasks and apply appropriate wage and fringe benefit rates to calculate 
labor cost. In the follow-up physician survey and qualitative interviews, we will ask PCPs in the immediate 
arm whether this process affected the length of their clinic visits, and, if so, by how much. If PCPs report 
an increase or decrease in time spent with patients, then we will apply a wage-per-minute cost to the time 
difference to derive an incremental visit time cost for the appropriate study arm. 
 
The intervention may differentially impact utilization of health care services. We will derive screening costs 
of FOBT, flexible sigmoidoscopy, or colonoscopy from the VA’s Decision Support System outpatient and 
inpatient extracts datasets. We will collect a sample of times required to provide CRC genetic counseling 
and wage and fringe benefit rate for genetic counselors from GMS to estimate the cost of providing CRC 
genetic counseling. For patients that receive genetic testing, we will apply the prevailing market rate for a 
genetic test at study conclusion to the analysis. Although intervention and screening costs are likely to be 
higher in the immediate arm, at least some of this cost may be mitigated by cost savings from CRC cases 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
12 
 
being averted. As the short time frame of this study will not allow for the appropriate capture of such costs 
savings, we will rely on the literature to project long-term cost savings from averting CRC cases. 
 
Personnel Training and Quality Control. Drs. Voils will train the PC and RA in standard survey and 
outcome assessment methods:93 1) Careful scheduling and timely monitoring of the schedule to ensure 
that complete visits occur; 2) Collection of contact information at each visit; 3) Prompt contact and 
rescheduling of missed visits; 4) Review and documentation of the reasons for missed visits; and 5) 
Review and edit of participant data promptly to identify problems or missing values that can be completed 
by follow-up.  
To decrease the risk of ascertainment bias and to standardize the abstraction, a detailed record 
abstraction protocol with data dictionary will be created, and training will be conducted by Dr. Fisher. An 
electronic database will be developed for this purpose and will include quality measures such as 
mandatory fields and range checks. Quality will be further ensured by independent abstraction and data 
entry of 15% weekly by the study staff throughout the study, with comparison of the two abstractions and 
correction and retraining as needed. 
 
Safety Monitoring Plan. Study staff will report any adverse event or unanticipated problem to the PI, Dr. 
Fisher (gastroenterologist) immediately. These individuals will review all such events and determine the 
likelihood they were due to the study. The PI will report any serious, unexpected, and study-related 
adverse event or unanticipated problem to the local IRB according to the institution’s requirements. The 
IRB will review all adverse events during continuing review, which will occur at least annually. 
Data Management, Storage, and Transfer. Regarding privacy, research personnel will use only those 
parts of the medical record necessary to determine eligibility and follow the research protocol. Measures 
will be taken to maintain confidentiality during in-person sessions, such as scheduling them in a private 
room where other patients cannot overhear the conversation. All information security and privacy 
incidents will be reported to the ISO and PO immediately. 
MeTree is stored on a Duke server, where it is continuously updated and maintained. Moving the software 
to VA would require resources (time and money) well beyond the scope of this research study. Moreover, 
it is our position that moving the software is not justified until we can demonstrate clinical benefit; the goal 
of this study is to determine if there is clinical benefit. If there is, then future efforts will be directed toward 
implementation, which may include moving MeTree to VA.  
Patients may provide HIPAA Authorization (by mail/fax if completing online or in person if completing in 
clinic) indicating that their data will be entered and stored at Duke, or may waive HIPAA authorization 
during the verbal consent process. The data will be entered on a secure Duke website that meets VA 
security standards. Duke will maintain the family history data without identifiers and may combine these 
data with data they obtain from other studies. To prevent the exposure of personal identifying information 
or protected health information, we will use the following procedures. Each potential participant will be 
assigned a study participant number for tracking purposes. An electronic key linking identifiers to study 
numbers will be stored on a Durham HSR&D server (\\vhadurhsmcifs01.v06.med.va.gov \Family History), with 
access only available to VA team members. VA team members in Madison will be granted permission to 
access the study folder. Patients will be encouraged not to enter identifying information into the Duke 
website (e.g., provide age instead of date of birth); they do so at their own risk. The decision support 
documents generated by MeTree will be downloaded by the study staff from a secure Duke server (to 
which study staff will have access) to the Durham VA server (for Durham patients) or to server in Madison 
(for Madison patients). A copy of the documents will be scanned into CPRS, and a copy will be stored in 
the electronic study files.  Throughout the study, Duke staff may provide the study statisticians with 
MeTree data for analysis.  This will be done using the VA-approved Safe Access File Exchange (SAFE), 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
13 
 
as a method to safely send files electronically.  The data files can be emailed to the statisticians using 
SAFE, then saved in the study folder on the Durham HSR&D server. As an alternative to using SAFE, 
Duke staff can also provide data to the VA statisticians by encrypting data (compliant with FIPS 140-2) on 
a VA-issued CD or USB/thumb drive, or using Azure RMS for electronic data transfer.  
For chart abstractions, study staff in Durham and Madison will use VistAWeb, JLV, or CPRS to access 
medical records. The data collected through these chart abstractions will be recorded using the VA-
approved DatStat Illume, and data will be saved behind the VA firewall on a Durham HSR&D server 
(\\vhadurhsmcifs01.v06.med.va.gov or \\vhadurhsmcifs01.v06.med.va.gov).  
All hard copies of documents (e.g., consent forms, case report forms) will be stored in the office or lab 
space of the PI. Electronic data files will be password-protected and maintained on a password-protected 
VA server, with access only available to approved study staff and investigators. This includes the study 
database, which contains the link of study identification numbers to identifying information.  All data will be 
retained for at least 10 years and will be destroyed according to VA guidelines. The removal of access to 
research study data will be accomplished for study personnel when they are no longer part of the 
research team.  
 
 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
14 
 
References 
1. 
Eichler, K., et al., Barriers to apply cardiovascular prediction rules in primary care: A postal survey. 
BMC Fam Pract, 2007. 8(1). 
2. 
Sabatino, S., et al., Breast cancer risk assessment and management in primary care: Provider 
attitudes, practices, and barriers. Cancer Detect Prev, 2007. 31(5): p. 375-383. 
3. 
Suther, S. and P. Goodson, Barriers to the provision of genetic services by primary care 
physicians: A systematic review of the literature. Genetics in Medicine 2003. 5(2): p. 70-76. 
4. 
Levin, B., et al., Screening and surveillance for the early detection of colorectal cancer and 
adenomatous polyps, 2008: a joint guideline from the American Cancer Society, the US Multi-
Society Task Force on Colorectal Cancer, and the American College of Radiology. CA Cancer J 
Clin, 2008. 58(3): p. 130-60. 
5. 
Cancer facts & figures 2013, 2013, American Cancer Society: Atlanta. 
6. 
Mandel, J.S., et al., Reducing mortality from colorectal cancer by screening for fecal occult blood. 
New Engl J Med, 1993. 328(19): p. 1365-1371. 
7. 
Kronborg, O., et al., Randomised study of screening for colorectal cancer with faecal-occult-blood 
test. Lancet, 1996. 348(9040): p. 1467-71. 
8. 
Hardcastle, J.D., et al., Randomised controlled trial of faecal-occult-blood screening for colorectal 
cancer. Lancet, 1996. 348(9040): p. 1472-7. 
9. 
Mandel, J.S., et al., The effect of fecal occult-blood screening on the incidence of colorectal 
cancer. N Engl J Med, 2000. 343(22): p. 1603-7. 
10. 
Atkin, W.S., et al., Once-only flexible sigmoidoscopy screening in prevention of colorectal cancer: 
A multicentre randomised controlled trial. Lancet, 2010. 375(9726): p. 1624-1633. 
11. 
Schoen, R.E., et al., Colorectal-cancer incidence and mortality with screening flexible 
sigmoidoscopy. New Engl J Med, in press. 
12. 
Lansdorp-Vogelaar, I., A.B. Knudsen, and H. Brenner, Cost-effectiveness of colorectal cancer 
screening. Epidemiol Rev, 2011. 33(1): p. 88-100. 
13. 
Hannon, P.A., et al., Colorectal cancer screening practices of primary care physicians in 
Washington State. Cancer Control, 2008. 15(2): p. 174-81. 
14. 
Klabunde, C.N., et al., A national survey of primary care physicians' colorectal cancer screening 
recommendations and practices. Prev Med, 2003. 36(3): p. 352-62. 
15. 
Shah, T.U., et al., Understanding gastroenterologist adherence to polyp surveillance guidelines. 
Am J Gastroenterol, 2012. 107(9): p. 1283-7. 
16. 
Scheuner, M., et al., A cancer genetics toolkit improves access to genetic services through 
documentation and use of the family history by primary-care physicians. Genet Med, 2013. 
Advances online publication June 13. 
17. 
Kerber, R.A., et al., Frequency of familial colon cancer and hereditary nonpolyposis colorectal 
cancer (Lynch syndrome) in a large population database. Fam Cancer, 2005. 4(3): p. 239-44. 
18. 
Hamilton, A., et al., Factors influencing organizational adoption and implementation of clinical 
genetic services. Genet Med, 2013. e-pub ahead of print Aug 15. 
19. 
Fisher, D.A., L. Judd, and N.S. Sanford, Inappropriate colorectal cancer screening: Findings and 
implications. Am J Gastroenterol, 2005. 100(11): p. 2526-30. 
20. 
Orlando, L.A., et al., The genomic medicine model: An integrated approach to implementation of 
family health history in primary care. Per Med, 2013. 10(3): p. 295-306. 
21. 
Orlando, L.A., et al., Protocol for implementation of family health history collection and decision 
support into primary care using computerized family health history system. BMC Health Serv Res, 
2011. 11: p. 264-270. 
22. 
Glanz, K., et al., Underreporting of family history of colon cancer: correlates and implications. 
Cancer Epidemiol Biomarkers Prev, 1999. 8(7): p. 635-9. 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
15 
 
23. 
Elias, P., J. Romagnuolo, and B.B. Hoffman, Poor patient knowledge regarding family history of 
colon polyps: Implications for the feasibility of stratified screening recommendations. Gastrointest 
Endosc, 2012. 75(3): p. 598-603. 
24. 
Orlando, L.A., et al., Development and validation of a primary care-based family health history and 
decision support program (MeTree). N C Med J, 2013. 74(4): p. 287-296. 
25. 
Orlando, L.A., et al., Implementing family health history risk stratification in primary care: 
Estimating resource needs. Am J Med Gen, in press. 
26. 
Orlando, L., et al., What’s the impact? Clinical validity and utility of MeTree, an electronic family 
history collection and decision support tool for primary care, in Society of General Internal 
Medicine2011. 
27. 
Wu, R.R., et al., Quality of family history collection with use of a patient facing family history 
collection tool. Quality of family history collection with use of a patient facing family history 
collection tool, under review. 
28. 
Wu, R.R., et al., Patient and primary care provider experience using a family health history 
collection, risk stratification, and clinical decision support tool: A type 2 hybrid controlled 
implementation-effectiveness trial. BMC Fam Prac, 2013. 14: p. 111. 
29. 
Voils, C.I., et al., Examining the impact of genetic testing for type 2 diabetes on health behaviors: 
Study protocol for a randomized controlled trial. Trials, 2012. 13: p. 121. 
30. 
Fisher, D.A., et al., Determinants of medical system delay in the diagnosis of colorectal cancer 
within the Veteran Affairs Health System. Dig Dis Sci, 2010. 55(5): p. 1434-41. 
31. 
Patwardhan, M., et al., Assessing the quality of colorectal cancer care: do we have appropriate 
quality measures? (A systematic review of literature). J Eval Clin Pract, 2007. 13(6): p. 831-45. 
32. 
McKinney, M., B. Weiner, and V. Wang, Recruiting participants to cancer prevention clinical trials: 
Lessons from successful community oncology networks. Oncol Nurs Forum, 2006. 33(5): p. 951-
959. 
33. 
Wang, V., et al., Economic evaluation of telephone self-management interventions for blood 
pressure control. Am Heart J, in press. 
34. 
Datta, S.K., et al., Economic analysis of a tailored behavioral intervention to improve blood 
pressure control for primary care patients. Am Heart J, 2010. 160(2): p. 257-63. 
35. 
Baer, H., et al., Use of a web-based risk appraisal tool for assessing family history and lifestyle 
factors in primary care. J Gen Intern Med, 2013. 28(6): p. 817-824. 
36. 
Curran, G., et al., Effectiveness-implementation hybrid designs: Combining elements of clinical 
effectiveness and implementation research to enhance public health impact. Med Care, 2012. 
50(3): p. 217-226. 
37. 
Fisher, D.A., et al., Ascertainment of colonoscopy indication using administrative data. Dig Dis Sci, 
2010. 55(6): p. 1721-5. 
38. 
Tsai, J. and R. Rosenheck, Use of the internet and an online personal health record system by US 
veterans: Comparison of Veterans Affairs mental health service users and other veterans 
nationally. J Am Med Inform Assoc, 2012. 19: p. 1089-1094. 
39. 
Fisher, D.A., et al., Validation of a questionnaire to assess self-reported colorectal cancer 
screening status using face-to-face administration. Dig Dis Sci, 2009. 54(6): p. 1297-306. 
40. 
Obuchowski, N., On the comparison of correlated prpoprtions for clustered data. Stat Med, 1998. 
17: p. 1495-1507. 
41. 
Durlaski, V., et al., Analysis of clustered matched-pair data. Stat Med, 2003. 22: p. 2417-2428. 
42. 
Ko, C., et al., Determination of colonoscopy indication from administrative claims data. Med Care, 
2012. Epub ahead of print. 
43. 
Zeger, K. and S. Liang, An overview of methods for the analysis of longitudinal data. Stat Med, 
1992. 11: p. 1825-39. 
44. 
Long, M.D., et al., Colorectal cancer testing in the national Veterans Health Administration. Dig Dis 
Sci, 2012. 57(2): p. 288-93. 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
16 
 
45. 
Donner, A., Klar, N., Design and Analysis of Cluster Randomization Trials in Health 
Research.2000, New York: Oxford University Press. 
46. 
Adams, G., et al., Patterns of intra-cluster correlation from primary care research to inform study 
design and analysis. J Clin Epidemiol, 2004. 57(8): p. 785-794. 
47. 
Rogers, E., Diffusion of innovations. 5th ed2003, New York: Free Press. 
48. 
Klein, K., . Sorra, JS., The challenge of implementation. Acad Manage J, 1996. 21(4): p. 1055-
1080. 
49. 
Emmons, K.M., et al., The relationship between organizational characteristics and the adoption of 
workplace smoking policies. Health Educ Behav, 2000. 27(4): p. 483-501. 
50. 
Weiner, B.J., M.A. Lewis, and L.A. Linnan, Using organization theory to understand the 
determinants of effective implementation of worksite health promotion programs. Using 
organization theory to understand the determinants of effective implementation of worksite health 
promotion programs, 2009. 24(2): p. 292-305. 
51. 
Weiner, B., et al., Implementing the MOVE! weight management program in the Veterans Health 
Administration, 2007-2010: A qualitative study. Prev Chronic Dis, 2012. 9: p. 110127. 
52. 
Goodman, R.M., et al., Development of level of institutionalization scales for health promotion 
programs. Health Educ Q, 1993. 20(2): p. 161-78. 
53. 
Klein, K.J., A.B. Conn, and J.S. Sorra, Implementing computerized technology. J Appl Psychol, 
2001. 86(5): p. 811-824. 
54. 
Leonard-Barton, D., Implementation as mutual adaptation of technology and organization. Res 
Policy, 1988. 17: p. 251-267. 
55. 
Nord, W.R. and S. Tucker, Implementing routine and radical innovations1987, Lexington, MA: 
Heath and Company. 
56. 
Klein, K.J. and J.S. Sorra, The challenge of implementation. Acad Manage Rev, 1996. 21(4): p. 
1055-1080. 
57. 
Holahan, P.J., et al., Implementing computer technology: A multi-organizational test of Klein and 
Sorra's model. Journal of Engineering and Technology Management, 2004. 21: p. 31-50. 
58. 
Cooper, R.B. and R.W. Zmud, Information Technology Implementation Research - a Technological 
Diffusion Approach. Manag Sci, 1990. 36(2): p. 123-139. 
59. 
Fichman, R.G. and C.F. Kemerer, The assimilation of software process innovations: An 
organizational learning perspective. Manag Sci, 1997. 43(10): p. 1345-1363. 
60. 
Fichman, R.G. and C.F. Kemerer, The Illusory Diffusion of Innovations: An Examination of 
Assimilation Gaps. Information Systems Research, 1999. 10: p. 255-275. 
61. 
Griffith, T.L. and G.B. Northcraft, Cognitive elements in the implementation of new technology: Can 
less information provide more benefits? Mis Quarterly, 1996. 20(1): p. 99-110. 
62. 
Leonard-Barton, D., Implementation characteristics of organizational innovations. Commun Res 
1988. 15: p. 603-631. 
63. 
Guth, W.D. and I.C. Macmillan, Strategy Implementation Versus Middle Management Self-Interest. 
Strategic Management Journal, 1986. 7(4): p. 313-327. 
64. 
Goodman, R.M. and A. Steckler, A model for the institutionalization of health promotion programs. 
Fam Comm Health, 1989. 11(4): p. 63-78. 
65. 
Yin, R.K., Changing urban bureaucracies1979, Lexington, MA: DC Heath and Company. 
66. 
Yin, R., Case study research: Design and methods. 2nd ed1994, Thousand Oaks, CA: Sage  
67. 
Creswell, J., Qualitative inquiry and research design: Choose among five approaches. 2nd ed2007, 
Thousand Oaks: Sage. 
68. 
Yin, R., Enhancing the quality of case studies in health services research. Health Serv Res, 1999. 
34(5 Pt 2): p. 1209-1224. 
69. 
Wood, M. and R. Christy, Sampling for possibilities. Qual Quant, 1999. 33(22): p. 185-202. 
70. 
Hsieh, H.-F. and S.E. Shannon, Three approaches to qualitative content analysis. Qual Health 
Res, 2005. 15(9): p. 1277-1288. 

Protocol: The Impact of Family History and Decision Support on High-Risk Cancer Screening 
 
V16   2/11/20 
17 
 
71. 
Charmaz, K., Constructing Grounded Theory: A Practical Guide Through Qualitative Analysis2006, 
London: Sage Publications, Ltd. 
72. 
Miles, M. and A. Huberman, Qualitative data analysis: An expanded sourcebook. end ed1994, 
Thousand Oaks: Sage Publications.