mFOCUS (Multilevel FOllow-up of Cancer Screening)
Phase: NA
Conditions: Cancer
Sponsor: Massachusetts General Hospital

PARTNERS HUMAN RESEARCH COMMITTEE 
PROTOCOL SUMMARY 
Partners Human Subjects Research Application Form  
 
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
1 
 
Answer all questions accurately and completely in order to provide the PHRC with the relevant 
information to assess the risk-benefit ratio for the study.  Do not leave sections blank. 
 
PRINCIPAL/OVERALL INVESTIGATOR 
 
Jennifer Haas, MD, MSc 
 
PROTOCOL TITLE 
 
mFOCUS (multilevel FOllow-up of Cancer Screening), RCT 
 
 
FUNDING 
 
NCI (U01CA225451) 
 
VERSION DATE 
 
January 6, 2023 
 
SPECIFIC AIMS 
Concisely state the objectives of the study and the hypothesis being tested. 
Specific Aim 1: To evaluate the effectiveness of the system, team 
and individual components of mFOCUS vs. standard care by 
conducting a 4-arm cluster randomized controlled trial (RCT) of 
individuals who are due for follow-up of an abnormal cancer 
screening test. Standard care consists of well-characterized existing 
decision support and systems for follow-up in these three participating 
primary care networks and their affiliated integrated delivery systems 
(Brigham and Women’s Hospital and Massachusetts General Hospital, both 
members of Partners HealthCare System, Massachusetts’s largest integrated 
health care delivery system, and Dartmouth Hitchcock Health, the largest 
health care provider in New Hampshire). The primary outcome will be 
whether an individual receives follow-up, defined based on the type of 
screening abnormality and organ type, within 120 days of becoming eligible 
for mFOCUS. Secondary comparisons will assess multi- and cross-level 
(individual, team, system) outcomes. Because the Coronavirus pandemic 
impacted access to health care, we will also assess whether an individual 
receives follow-up, defined based on the type of screening abnormality and 
organ type, within 240 days of becoming eligible for mFOCUS. The study 
design will allow us to examine the marginal effectiveness of system, team 
and individual-level enhancements, and exploratory analyses will address 
subgroups defined by race/ ethnicity, socioeconomic status and cancer type. 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
2 
Hypothesis: mFOCUS will significantly increase the proportion of individuals 
who receive follow-up testing, thereby reducing time to follow-up vs. 
standard care.  
Specific Aim 2: To evaluate facilitators and barriers to the Reach, 
Adoption, Implementation and Maintenance of mFOCUS at 
participating primary care practice sites affiliated with three primary 
care networks The impact of any intervention depends on its ability to be 
implemented in clinical practice. 
Hypothesis (Reach): Individuals who are due for follow-up can be reached 
by mFOCUS irrespective of patient sociodemographic characteristics and 
“severity” of the screening result. Patient surveys will be used to examine 
barriers and facilitators. 
Hypothesis (Adoption, Implementation, and Maintenance): mFOCUS 
will be adopted across providers/ practices and consistently implemented. 
Provider and system-level surveys will be used to examine barriers and 
facilitators to adoption, implementation and maintenance. 
 
NOTE: This protocol involves patients who receive care at Partners (MGH 
and BWH) as well as Dartmouth.  Dartmouth is the relying site on the 
Partners IRB review a SmartIRB reliance agreement has been put in place 
 
BACKGROUND AND SIGNIFICANCE 
Provide a brief paragraph summarizing prior experience important for understanding the 
proposed study and procedures. 
 
In the United States (US), most cancer screening is initiated in primary care 
settings but requires care transitions with other providers who either 
perform the test or evaluate the result. Screening benefits are only realized 
through timely follow-up of abnormal breast, cervical, colorectal and lung 
screening results. Incomplete follow-up represents an ongoing challenge as 
the initial screening event transitions to diagnostic evaluations involving 
complex interactions among patients, primary care providers (PCPs), and 
specialists. 
Our recent study within the NCI-funded Population-Based Research 
Optimizing Screening through Personalized Regimens (PROSPR) consortium 
reported significant rates of screening abnormalities that required additional 
evaluation, including; 11% of breast cancer, 8% of cervical and 5% of fecal 
occult blood test (FOBT)/ fecal immunochemical test (FIT) screens, 
consistent with other studies. The abnormality rates for colonoscopy 
screening (~20% adenomas) and lung cancer screening with low dose 
computed tomography (LDCT)’s are also high (~25%). Our work also 
demonstrates wide variation in follow-up rates across primary care clinics, 
that differ by cancer type as well as the “severity” of the screening 
abnormality (5 – 95% with an average of ~ 50% overall). Apart from the 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
3 
legislated requirement for radiologists to follow-up abnormal mammograms, 
responsibility for comprehensive screening follow-up falls to the ordering 
provider, typically a primary care provider (PCP). Unfortunately, few PCPs/ 
primary care practices have systems to track abnormalities and promote 
follow-up. PCPs face the challenge of responsibility for managing the 
diagnostic evaluation for populations of patients for each of these cancers, 
each with differing requirements and timeframes for completion. 
Barriers to follow-up of “abnormal screens” exist at individual patient and 
provider levels as well as at patient’s care team and health system levels. 
Providing high-quality care for abnormal screens requires establishing new 
processes to systematically identify abnormal results, notify patients and 
providers, address barriers, and track completion of the diagnostic work-up. 
We propose to develop, test and disseminate a health information 
technology (IT)-enabled, multilevel, stepped care intervention grounded in 
primary care, mFOCUS (multilevel FOllowup of Cancer Screening) for 
abnormal breast, cervical, colorectal and lung screens. Comprehensive 
intervention to promote follow-up needs to be based in primary care as PCPs 
holistically approach each patient’s needs and coordinate care among 
specialists. Key mFOCUS components include: system design to promote 
identification/tracking using an electronic health record (EHR)-integrated 
population management platform with education to promote culture change 
around the management of abnormal screens; use of individual patient and 
PCP reminders and tools; and a stepped-care team-level enhancement with 
increasing intensity of contact (i.e., administrative support and patient 
navigation).  
 
 
RESEARCH DESIGN AND METHODS 
Briefly describe study design and anticipated enrollment, i.e., number of subjects to be enrolled 
by researchers study-wide and by Partners researchers.  Provide a brief summary of the 
eligibility criteria (for example, age range, gender, medical condition).  Include any local site 
restrictions, for example, “Enrollment at Partners will be limited to adults although the sponsor’s 
protocol is open to both children and adults.”
 
 
Specific Aim 1: To evaluate the effectiveness of the system, team 
and individual components of mFOCUS vs. standard care by 
conducting a 4-arm cluster randomized controlled trial (RCT) of 
individuals who are due for follow-up of an abnormal screen.  
Inclusion criteria: Individuals who have an abnormal screen that is due for 
follow-up including: 
 Breast: women 40-80 years with an incident (i.e., newly detected) 
abnormal screening mammogram or digital breast tomosynthesis (DBT) 
exam.  
 Cervical: women 21-65 years with an incident abnormal screening Pap.  

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
4 
 Colorectal: adults 40-80 years with an abnormal screen, including 
incident FOBT/ FIT, or prevalent colonoscopy. Because of the long periods 
of time required for follow-up of colonoscopies, we will look back over a 
5-year period and will therefore find prevalent abnormalities that become 
due for follow-up. 
 Lung: adults 55-80 years, current and former smokers, with an incident 
abnormal LDCT result.  
Definition of the specific screening abnormalities and timing of diagnostic 
evaluation are based on our prior work, expert opinion, and the literature. 
Need for follow-up will be determined by the lack of an appropriate clinical 
test within the specified time as documented in the EHR.  
Exclusion criteria: We will exclude patients who: 1) are not English or 
Spanish-speaking, or 2) have had prior cancer of the organ for each 
screening test (i.e., women with prior breast cancer will not be tracked for 
breast cancer screening abnormalities) as these individuals may have non-
standard follow-up care recommendations. 
 
Intervention Design and Randomization Scheme. We will perform a 4-
arm cluster RCT with randomization done at the practice-level, across the 
participating practices. We have chosen this level of randomization as it 
would be logistically complicated for the practices to have different patients 
randomized to different intervention components (i.e., reduced 
contamination). While we have received the approval of the primary care 
network leadership of the participating networks and have discussed the 
study with practice leadership, practices will be given the opportunity to opt-
out of the study before practices are randomized. Randomization will be 
stratified based on: (1) practice size, (2) percent female, and (3) percent of 
patients on Medicaid as a marker for socioeconomic status.  
 
The 4-arm design will allow us to compare “standard care” (Arm 1) to three 
intervention arms that represent the sequential/ stepped addition of: (Arm 
2) a system-level, visit-based intervention consisting of PCP facing 
reminders deployed when a patient is overdue for follow-up (health 
maintenance reminders; automated problem list additions); (Arm 3) the 
addition of a population-based intervention with study staff administration of 
the patient and PCP facing reminders deployed when a patient is overdue for 
follow-up regardless of whether they come in for a visit (e.g. bulk In-Basket 
messaging to the PCP, Patient Gateway reminders or letters, a brief phone 
call or voice message for the patient); and (Arm 4) the addition of a team-
level intervention including phone calls from an outreach coordinator and a 
patient navigator. Part of the role of the Patient Navigator will be to assess a 
patient’s social determinants of health. If the patient has any needs the 
patient navigator will use “Aunt Bertha” an online platform to help patients 
find social services in their area. Patients in Arm 1 will receive the usual care 
provided by their PCP. Given our stepped care approach, this randomization 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
5 
scheme will allow us to compare the cumulative addition of each level of 
intervention to standard care as well as the marginal effect of each 
additional level to the prior group. This design allows us to evaluate 
changing the responsibility for “opportunistic” follow-up, typically by the PCP 
at the time of a visit (Arm 1), to a systematic, multilevel approach, 
examining the cumulative and marginal effects of each subsequent level of 
intervention. In addition to the multilevel components, the individual- and 
team-level engagement algorithm will take a “stepped care” approach with 
increasing level of intensity of engagement.  
Recruitment and Enrollment. Key organizational components of the study 
will involve:  
(1) Practice and Provider Engagement. We have obtained support of network 
leadership for the study. Investigators have/ will attend practice meetings at 
intervention sites to integrate work flow with practice routines and educate 
staff about the study and its procedures. PCPs will not be required to do 
recruitment; identification of eligible patients will be done centrally using the 
IT platform described above. We therefore anticipate a high level of practice 
and PCP participation since the intervention does not constrain usual 
practice, but adds additional systems beyond those currently available. As 
noted above, practices and providers will be given the opportunity to opt-out 
of the study before practices are randomized.  
(2) Patient Identification and Enrollment. The mFOCUS platform will 
incorporate algorithms to identify eligible patients and patient-practice 
attribution. All patients will remain under the direction of their personal PCP/ 
practice and the intervention components will facilitate the delivery of care in 
addition to the current standard of care.  
 
We plan to enroll 3324 eligible participants (approximately 831/ arm – as 
randomization will be done at the practice level the arms will not be the 
same size) across the 3 participating primary care networks (BWH, MGH, 
and Dartmouth).  We anticipate that recruitment will be somewhat higher at 
BWH and MGH than Dartmouth as the networks are bigger. Because 
randomization will be done at the practice level, we will not be able to recruit 
exactly 3324 individuals (patients will be in process of recruitment since they 
will be recruited in batches) – this is the number suggested by our sample 
size calculation. We anticipate that this calculations will need to be updated 
due to the impact of COVID-19 and that our recruitment number will be 
higher because of the back log of testing that will occur as hospitals reopen 
for screening procedures. We anticipate that we could possibly enroll up to 
9,000 eligible patients at MGHBWH, and 12,000 eligible patients study wide. 
We will not exceed recruitment of 12,000 eligible individuals. Due to the way 
our IT system is configured, we anticipate that we could review up to 20,000 
patient charts study wide. Not all of the patient charts reviewed will be 
deemed eligible. 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
6 
Any patient contact (outreach coordinator, patient navigator) will be done by 
staff at the respective patient care sites (Partners staff will contact Partners’ 
patients and Dartmouth staff will contact Dartmouth patients). Dependent 
upon randomization, patients will receive a patient gateway message (if 
patient has an account) or mailed letter and a phone call from a study 
coordinator. Patients in study arm 4 will receive patient navigation, which 
may involve multiple contacts by phone and/or the by a telehealth Partners 
approved platform.  
(3) Surveys: Some of the outcomes of the study will be measured by PCP 
and patient surveys: 
PCP Surveys: We developed brief, self-administered survey instruments to 
be administered to PCPs just before the trial is launched and 3 months after 
recruitment ends.  
PCPs will be sent an introductory email from the clinical director of each 
primary care network.  This will be followed by an email from the study 
investigators that explains the study purpose and opportunities to opt out/ 
participate of the survey. The second email will include a unique url for the 
PCP to use to connect to a RedCap version of the survey.  PCP surveys will 
be administered using RedCap (up to 5 email contacts with a RedCap link 
over 4 weeks). 
 
Patient surveys: We will randomly sample 15% of participants. Patient 
surveys will be administered 4-12 weeks after their individual follow-up 
period has been completed, so surveys will be conducted throughout the 
enrollment period. Patients will be sent an introductory letter from the 
clinical director of each primary care network paired with a letter from the 
study investigators that explains the study purpose and opportunities to opt 
out/ participate of the survey. The second letter will include a unique url/QR 
code for the patient to use to connect to a RedCap version of the survey.  If 
patients do not opt-out within 2 weeks or complete the survey using the 
RedCap link provided in the letter, they will be contacted by phone and 
asked to complete the survey by phone or RedCap. If the patients opt for 
the later, they will be asked for permission to receive the survey link via un-
encrypted email (and asked for the email where they would like to receive 
it).  
 
Summary of primary and secondary outcome measures 
Measures (data source)
Primary outcome: Completion of 
follow-up within 120 days of eligibility 
for mFOCUS (EHR/ claims) 
Secondary outcomes:  
Completion of follow-up within 240 
days of eligibility for mFOCUS (EHR/ 
claims) 
Patient-Reported Assessment of:
Individual-PCP: Satisfaction with care 
provided by PCP  
Care Team: Satisfaction with care provided 
by care team  
System: Receipt of reminder letters, calls, or 
emails  

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
7 
# of days to completion of required 
diagnostic evaluation (EHR/ claims) 
Satisfaction with follow-up care 
(patient survey) 
Knowledge of test result and need for 
follow-up (patient survey) 
Satisfaction with ability of patients to 
get timely follow-up (PCP survey) 
Time required for management (PCP 
survey) 
PCP-Reported Assessment of:
Individual-Patient: Patient understanding of 
need for follow-up 
Care Team: Satisfaction with coordination of 
care 
System: Satisfaction with IT functionality to 
support follow-up  
 
 
Analysis Plan/ Sample Size. Our primary analysis will be intention-to-
treat (ITT). All eligible, enrolled patients will be part of the ITT cohort. We 
expect that a small number of patients may change primary care clinics 
within our systems. These individuals will be evaluated according to their 
initial intervention status. Prior to analysis, all data will be examined for 
accuracy, logical consistency and missing data. We expect complete data for 
intervention status and our outcomes. We will use multiple imputation, using 
the MI procedure in SAS, to account for any missing values of covariates. 
Through the stratified randomization we will attempt to balance patient, 
provider and practice characteristics between study groups. However, since 
randomization occurs at the practice level, we will be cautious and still 
compare the patient, provider and practice characteristics using Fisher exact 
tests, t-tests and Wilcoxon tests as appropriate to the covariate distribution. 
Any characteristics that show substantial clinical or statistical difference 
(p<.05) will be entered into the regression model and retained as covariates 
if they alter the effect estimate of the intervention by >20%.  
The primary analysis model will be a random effects logistic regression, 
implemented through the SAS Glimmix procedure. Timely follow-up (yes/no) 
will be the patient-level outcome and random effects for practice and 
physician will allow for exchangeable correlation between patients seen 
within the same practice and by the same physician. The primary fixed 
predictors will be 3 indicator variables representing the 3 intervention arms 
and we will use a global likelihood ratio test to compare the 4 study arms. If 
the global test is significant (p<0.05), we will compare the intervention arms 
to the control group (our primary comparisons with a Bonferroni-adjusted 
significance level of 0.0167) and to each other (secondary comparisons). 
Comparisons to the control arm will capture the cumulative effects of the 
interventions, while comparisons between the intervention arms will capture 
the marginal effects of each level of intervention. We will include covariates 
for the type of cancer, level of initial screening abnormality, and any patient, 
provider or practice characteristics which are identified as confounders. 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
8 
Results will be presented as adjusted follow-up rates, with 95% confidence 
intervals, calculated using marginal standardization. 
Secondary analyses will model time-to-follow-up, using a clustered 
proportional hazards regression to examine whether abnormal screens are 
followed-up as quickly as possible. Patient-level time-to-follow-up will be 
recorded with censoring at the end of study for patients who never received 
follow-up. Patients will be clustered within providers using the generalized 
estimating equation approach, implemented as a “frailty” analysis in the SAS 
Phreg procedure. An additional correlation component for patients within 
practices cannot be included, but we will evaluate the robustness of our 
findings by using an alternative model, clustering by practice rather than 
provider. Predictors and covariates in this model will be identified in the 
same way as described above. The proportional hazards assumption for the 
intervention effects will be verified by entering a time-varying version of 
those predictors. Other secondary outcomes based on patient, provider and 
team surveys will use clustered linear regression models to compare 
satisfaction scores between study arms.  Our secondary system outcome: 
the number of patient contacts; will be compared between arms using 
clustered Poisson regression. The model building will be analogous to the 
approach detailed above. 
We will also consider several exploratory analyses. In particular, we will 
perform our primary and secondary analyses within each of the four types of 
cancer and by risk of cancer. Prior to any of these subgroup analyses, we 
will put the appropriate interaction terms into the primary models above. 
However, it was not practical to design this study to have sufficient power to 
pursue all of these possibilities. Instead, any “findings” in these secondary 
analyses will be considered as clues to be pursued in future studies and in 
alternative databases. 
Finally, while it would be clinically important to pursue cancer detection 
rates, we will not do so here because we lack statistical power and because 
of the possibility that more cancers may be diagnosed in the intervention 
arms than the control arm because of higher follow-up rates with more 
diagnostic evaluations. We will not have long enough follow-up times to look 
at differences in cancer incidence related to inadequate follow-up. Instead 
we will perform descriptive analyses within each study arm to look at the 
number of incident cancers identified.  
 
Our sample size calculation was based on an increase in the follow-up test 
completion rate of 11-14% with the intervention– this would give us 80% power 
to detect an 11% difference in the follow-up rate. We have an adequate number of 
potential participants across our sites to achieve this sample size. 
 
Specific Aim 2: To evaluate facilitators and barriers to the Reach, Adoption, 
Implementation and Maintenance of mFOCUS. 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
9 
The RE-AIM framework (Reach, Effectiveness, Adoption, Implementation and 
Maintenance with Effectiveness assessed in Aim 2) will allow us to assess the 
multilevel implementation of our intervention. The primary goal is to inform and 
facilitate dissemination of the program in the 3 networks rather than to test specific 
hypotheses. We will calculate summary statistics and conduct bivariate comparisons 
across sites using Fisher’s exact test, t-tests, and the Wilcoxon rank sum test where 
applicable for survey outcomes.  
 
RE-AIM 
Metric
Measure 
Data Source 
Reach 
• % of eligible individuals in each intervention group who 
engage in relevant step (open email reminder, speak with 
outreach coordinator or PN) 
• Population management system 
 
• Patient barriers and facilitators of follow-up  
• Patient survey 
Adoption 
• Barriers and facilitators to follow-up  
• Systems & processes for follow-up 
• Knowledge of and satisfaction with components of 
mFOCUS 
• PCP survey 
 
Implementation 
• Technical and organizational barriers to implementation 
• Fidelity of delivery of the intervention 
• PCP survey 
• Population management system 
Maintenance 
• Technical and organizational barriers to maintenance 
• PCP surveys 
 
Patient, and provider surveys are attached. Once the final version of the 
patient survey is approved it will be translated by an IRB approved vendor 
and submitted to the IRB. 
 
 
Briefly describe study procedures.  Include any local site restrictions, for example, “Subjects 
enrolled at Partners will not participate in the pharmacokinetic portion of the study.”  Describe 
study endpoints.
 
See study procedures described above. No local site restrictions are planned. 
 
For studies involving treatment or diagnosis, provide information about standard of care at 
Partners (e.g., BWH, MGH) and indicate how the study procedures differ from standard care.  
Provide information on available alternative treatments, procedures, or methods of diagnosis.
 
All individuals in this study will receive the current standard of care at 
Partners for the follow-up of abnormal cancer screening tests (Arms 1-4).  In 
addition to this standard of care, we will test 3 additional arms of increasing 
intensity of intervention: Arm 2-access to an IT platform to facilitate updates 
to HM modifiers and problem lists for visit-based reminders; Arm 3- the IT 
platform in addition to automated reminders to patients (Patient Gateway 
messages or letters, one brief phone call or voice message) and the care 
team (bulk In-Basket messages); Arm 4-the IT platform, automated 
reminders, and administrative outreach and navigation to help with 
scheduling and to address social barriers to care. 
 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
10 
Patients will only be recommended to receive care that is recommended by 
national guidelines/ local expert opinion. Our study is focused on ensuring 
that people get timely appropriate care.  Since launching the study, we have 
found that there is a small number of patients who get care hat is not 
supported by national guidelines.  An example of this would be a woman 
with a high-grade cervical abnormality who gets a repeat pap smear and not 
a colposcopy.  In some of these instances it is documented that the patient 
and their personal care team had a discussion of the harms and benefits of 
making an alternative choice, but in others there is no documentation of why 
alternative care was received. 
 
If we find that a patient has received an alternative evaluation to the 
diagnostic evaluation recommended by national care guidelines and there is 
no documentation of informed decision making, the PIs will contact the 
patient’s PCP and/or clinical director of the primary care practice and/or the 
institutional ambulatory team.  This will be done for patients in any of the 
study arms. While these patients remain eligible for our study as they will 
not have achieved our primary outcome, we will not contact these patients 
in arms 3 and 4 to minimize confusion or anxiety for the patient.  Because of 
the small number of patients we believe that ethically this is the right choice 
even though it may marginally decrease our power to find an effect across 
the study arms. 
 
Describe how risks to subjects are minimized, for example, by using procedures which are 
consistent with sound research design and which do not unnecessarily expose subjects to risk or 
by using procedures already being performed on the subject for diagnostic or treatment purposes.
 
We will protect against risks of psychological discomfort by using neutral 
language in patient surveys. To prevent loss of data, all information is stored 
within the Partners Healthcare System, Inc. firewall, password protected, 
and anti-virus software enabled (or Dartmouth for Dartmouth patients). Only 
study staff will have access to the study data on Shared File Areas. We do 
not anticipate any adverse effects from this study, but we will promptly 
report any problem to the IRB and address any problems accordingly. 
 
Our study is focused on ensuring that people get timely appropriate care.  
Since launching the study, we have found that there is a small number of 
patients who get care hat is not supported by national guidelines.  An 
example of this would be a woman with a high-grade cervical abnormality 
who gets a repeat pap smear and not a colposcopy.  In some of these 
instances it is documented that the patient and their personal care team had 
a discussion of the harms and benefits of making an alternative choice, but 
in others there is no documentation of why alternative care was received. 
 
If we find that a patient has received an alternative evaluation to the 
diagnostic evaluation recommended by national care guidelines and there is 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
11 
no documentation of informed decision making, the PIs will contact the 
patient’s PCP and/or clinical director of the primary care practice and/or the 
institutional ambulatory team.  This will be done for patients in any of the 
study arms. While these patients remain eligible for our study as they will 
not have achieved our primary outcome, we will not contact these patients 
in arms 3 and 4 to minimize confusion or anxiety for the patient.  Because of 
the small number of patients we believe that ethically this is the right choice 
even though it may marginally decrease our power to find an effect across 
the study arms. 
 
After our study, we found that the Health Maintenance Modifiers we created 
were placed on 259 patients’ charts by their clinicians (outside of our 
research). There is no functionality built in Epic to remove these modifiers 
when an appropriate follow-up test was received by a patient unless they 
were part of our study. Our study has now ended, and we have removed the 
health maintenance modifiers that we created.  Because we didn’t create the 
modifiers for these 259 patients, we are concerned that they will linger even 
if a patient has received appropriate follow-up. We will contact anyone in the 
Epic HMM Audit User list and ask them to review the modifier and either 
replace it with other functionality available in Epic or remove it. We will send 
2-3 reminders over the course of 4-6 weeks. The last reminder will inform 
them that the modifier will be removed by our study staff.  
 
Describe explicitly the methods for ensuring the safety of subjects.  Provide objective criteria for 
removing a subject from the study, for example, objective criteria for worsening disease/lack of 
improvement and/or unacceptable adverse events.  The inclusion of objective drop criteria is 
especially important in studies designed with placebo control groups.
 
This protocol is minimal risk, as it is designed to improve the quality of care 
for patients with an abnormal cancer screening test result that is overdue for 
follow-up. Automated outreach (by Patient Gateway or letters) will provide 
information for patients to contact their PCP’s office to arrange for needed 
follow-up.  Patients in Arm 4 who are contacted by the study outreach 
administrator or navigator who and say that they do not want further 
evaluation will not be contacted further by study staff. Participants selected 
for the surveys will be informed that they can refuse participation at any 
point.  
 
 
FORESEEABLE RISKS AND DISCOMFORTS 
Provide a brief description of any foreseeable risks and discomforts to subjects.  Include those 
related to drugs/devices/procedures being studied and/or administered/performed solely for 
research purposes.  In addition, include psychosocial risks, and risks related to privacy and 
confidentiality.  When applicable, describe risks to a developing fetus or nursing infant.
 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
12 
The potential risks to subjects include loss of confidentiality of healthcare 
data. Study staff will follow careful protocols to minimize these risks. The co-
investigators will emphasize the importance of maintaining confidentiality in 
the training of all study staff. All study data and survey questionnaires will 
be coded with unique study identification numbers. Electronic data will be 
stored within the Partners and Dartmouth firewalls, will be password 
protected, and will be protected by anti-virus software. Only study staff will 
have access to study data on shared file areas.  
 
Surveys may potentially cause psychological stress (topics will include 
barriers and facilitators to follow-up of screening abnormalities). Potential 
survey subjects will be informed about potential risks as part of the 
informational letter that they will receive before deciding whether to 
participate. Patients will be encouraged to discuss any concerns with their 
provider, prior to participating (all participants will have a PCP in this study). 
While unlikely, some patients may be contacted during the trial based upon 
inaccurate or incomplete information in her/his electronic health record. 
There may be psychological stress associated with such contact, but 
information provided by patients will be used to update the patient’s record/ 
notify the patient’s care team, thus ultimately resulting in better quality 
care. Study staff will specifically be trained to help patients cope with these 
issues. Study staff, particularly the outreach coordinators and the 
navigators, will be trained to specifically address any personal stressors that 
a patient may have that is interfering with their ability to get needed care. 
During the trial phase, patients are only indirectly affected through process 
of care modifications at the practices they attend. We do not anticipate 
physical risks to patients as a result of participation. Patients in the “control” 
arm will receive standard care as provided by their PCP and practice. 
Potential risks also include the time associated with survey participation for 
those selected. This time commitment will be minimal. Potential participants 
will be informed of the potential harms of participation when they decide 
whether to participate in a survey; participants will also be informed that 
they can decide to discontinue their survey participation at any time.  
 
 
EXPECTED BENEFITS 
Describe both the expected benefits to individual subjects participating in the research and the 
importance of the knowledge that may reasonably be expected to result from the study.  Provide 
a brief, realistic summary of potential benefits to subjects, for example, “It is hoped that the 
treatment will result in a partial reduction in tumor size in at least 25% of the enrolled subjects.”  
Indicate how the results of the study will benefit future patients with the disease/condition being 
studied and/or society, e.g., through increased knowledge of human physiology or behavior, 
improved safety, or technological advances. 
 
Participants in practices randomly assigned to the intervention arms may 
benefit by receiving more timely follow-up of their abnormal cancer 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
13 
screening test results. Patients in control practices will receive usual care 
under the direction of their primary care provider. If the intervention is 
effective, more timely follow-up of abnormal cancer screening test results 
could lead to earlier detection, treatment, and cure of the cancers studied in 
this proposal. In the future, all patients could benefit from the knowledge 
produced by this study through the dissemination of similar care systems. 
 
 
EQUITABLE SELECTION OF SUBJECTS 
The risks and benefits of the research must be fairly distributed among the populations that stand 
to benefit from it.  No group of persons, for example, men, women, pregnant women, children, 
and minorities, should be categorically excluded from the research without a good scientific or 
ethical reason to do so.  Please provide the basis for concluding that the study population is 
representative of the population that stands to potentially benefit from this research.
 
We will be recruiting all individuals during the study period at the 
participating clinics who are eligible for screening for breast, cervical, 
colorectal and lung cancer screening based on current guidelines as specified 
above, and who receive an abnormal screening result and are overdue to 
receive clinically indicated diagnostic evaluation. 
 
We only have resources to deliver this study in English and Spanish. We are 
aiming to recruit an ethnically and racially diverse group of patients who are 
representative of the eligible population. 
 
 
When people who do not speak English are excluded from participation in the research, provide 
the scientific rationale for doing so.  Individuals who do not speak English should not be denied 
participation in research simply because it is inconvenient to translate the consent form in 
different languages and to have an interpreter present.
 
Individuals who speak English or Spanish will be eligible, as these two 
languages reflect the significant majority of the Partners and Dartmouth 
patient populations and sufficiently allow us to meet our study goals. Once 
final study materials are developed in English they will be professionally 
translated by an IRB-approved vendor and submitted for review before they 
are used. 
 
 
For guidance, refer to the following Partners policy: 
          Obtaining and Documenting Informed Consent of Subjects who do not Speak English
          https://partnershealthcare-public.sharepoint.com/ClinicalResearch/Non-
English_Speaking_Subjects.1.10.pdf
 
 
 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
14 
RECRUITMENT PROCEDURES 
Explain in detail the specific methodology that will be used to recruit subjects.  Specifically 
address how, when, where and by whom subjects will be identified and approached about 
participation.  Include any specific recruitment methods used to enhance recruitment of women 
and minorities.
 
Trial: We plan to recruit 3324 participants in the RCT who fit the eligibility 
criteria defined above (are overdue for follow-up of an abnormal cancer 
screening test result). Because the trial is testing a “care enhancement” we 
are requesting a waiver of informed consent given the large number of 
subjects distributed over a maximum of 44 primary care practices. 
Randomization will be done at the practice level and individual patients will 
not be approached for recruitment. Because randomization will be done at 
the practice level, we will not be able to recruit exactly 3324 individuals 
(patients may be in process of recruitment since they will be recruited in 
batches) – this is the number suggested by our sample size calculation.  We 
will not exceed recruitment of 12,000 eligible individuals. The outcomes for 
the main trial will be assessed using data from the electronic health records 
at BWH, Faulkner, MGH, NWH, NSMC, and DFCI. NWH, NSHM, and DFCI are 
included because patients from MGH and BWH may have tests that are 
performed at these sites, thus they will be included in our review for 
diagnostic testing. Because of the large number of study participants and the 
minimal risk, we request a waiver of informed consent for this portion of the 
trial. It would not be feasible to contact this number of individuals and as 
this is a pragmatic trial it would adversely affect the validity and 
interpretation of our results. 
 
Surveys: Some of the outcomes of the study will be measured by PCP and 
patient surveys: 
PCP Surveys PCPs, excluding trainees, at participating sites using the 
electronic health record will be surveyed prior to launch of the RCT and 3 
months after recruitment ends. Prior to fielding the survey, we will request 
that the primary care network director send an email to PCPs in his/ her 
network informing them about the purpose to the survey will then receive an 
email from one of the study investigators with a link to a Redcap survey. 
Each PCP will receive up to 5 email contacts over 4 weeks. Remuneration is 
described in the appropriate section below. 
PCP surveys will be sent by email via a RedCap Survey link, that will include 
a gift card code. The pre-trial surveys will be sent electronically during the 3 
months before the trial.  Post-trial surveys will be sent electronically during 
the 3 months following the trial. Because the trial will run for 2 years it may 
be different doctors who are asked to complete the survey. The consent 
process for the 2 surveys will be done independently because the long period 
of time in between. 
 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
15 
Patient surveys: As noted above, we will randomly sample 15% of 
participants. Patient surveys will be administered 4-12 weeks after their 
individual follow-up period has been completed, so surveys will be conducted 
throughout the enrollment period. Survey procedures will include: 1) an 
introductory letter from the clinical director of each primary care network 
paired with a letter from the PI that explains the study purpose and 
opportunities to opt out/ participate of the survey (see attached). Patients 
will be given the opportunity to go to a unique Redcap url/QR code from the 
letter or will receive an the un-secured email. The letter will include contact 
directions to email/ phone study staff if they do not wish to opt-out of the 
survey or do not wish to receive un-secured email.  Those who do not opt-
out within 2 weeks will be contacted by phone and asked to complete the 
survey by phone or RedCap. If the patients opt for the later, they will receive 
the survey link via un-encrypted email (and asked for the email where they 
would like to receive it). Patients will receive up to 5 email/mail attempts 
over 4 weeks followed by up to by one phone call.  All mail surveys will be 
sent with a pre-addressed envelope with pre-paid postage.
Remuneration is described in the appropriate section below. 
 
Provide details of remuneration, when applicable.  Even when subjects may derive medical 
benefit from participation, it is often the case that extra hospital visits, meals at the hospital, 
parking fees or other inconveniences will result in additional out-of-pocket expenses related to 
study participation.  Investigators may wish to consider providing reimbursement for such 
expenses when funding is available
 
Subjects in the main trial will not receive remuneration. 
Patients who are sampled for the survey and complete the survey will 
receive a $50 gift card from their choice of Amazon or a regional grocery 
store. 
PCPs who are eligible for the survey will receive a $50 gift card code for 
Amazon with the initial delivery of the survey as the survey literature 
supports the use of “up front” incentives for health care providers.   
No additional remuneration will be provided as this study does not incur any 
out-of-pocket expenses for participating. 
 
 
For guidance, refer to the following Partners policies: 
          Recruitment of Research Subjects 
          https://partnershealthcare-
public.sharepoint.com/ClinicalResearch/Recruitment_Of_Research_Subjects.pdf
 
          Guidelines for Advertisements for Recruiting Subjects

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
16 
          https://partnershealthcare-
public.sharepoint.com/ClinicalResearch/Guidelines_For_Advertisements.1.11.pdf
 
          Remuneration for Research Subjects
          https://partnershealthcare-
public.sharepoint.com/ClinicalResearch/Remuneration_for_Research_Subjects.pdf
 
 
CONSENT PROCEDURES 
Explain in detail how, when, where, and by whom consent is obtained, and the timing of consent 
(i.e., how long subjects will be given to consider participation).  For most studies involving more 
than minimal risk and all studies involving investigational drugs/devices, a licensed physician 
investigator must obtain informed consent.  When subjects are to be enrolled from among the 
investigators’ own patients, describe how the potential for coercion will be avoided.
 
We are requesting a waiver of consent for the trial given the large number of 
patients and practices and the minimal risk from participation.  All patients 
will remain under the care of their primary care team and the intervention 
components will be delivered in addition to those individuals who are 
overdue for follow-up. 
 
The introduction to the surveys will include the elements of informed 
consent.  As these surveys are minimal risk, we request a waiver of 
informed consent. Consent will be implied if the subject choses to complete 
the survey. 
 
 
NOTE: When subjects are unable to give consent due to age (minors) or impaired decision-
making capacity, complete the forms for Research Involving Children as Subjects of Research 
and/or Research Involving Individuals with Impaired Decision-making Capacity, available on 
the New Submissions page on the PHRC website: 
      https://partnershealthcare.sharepoint.com/sites/phrmApply/aieipa/irb 
 
For guidance, refer to the following Partners policy: 
     Informed Consent of Research Subjects:
     https://partnershealthcare-
public.sharepoint.com/ClinicalResearch/Informed_Consent_of_Research_Subjects.pdf
 
 
 
DATA AND SAFETY MONITORING 
Describe the plan for monitoring the data to ensure the safety of subjects.  The plan should 
include a brief description of (1) the safety and/or efficacy data that will be reviewed; (2) the 
planned frequency of review; and (3) who will be responsible for this review and for determining 
whether the research should be altered or stopped.  Include a brief description of any stopping 
rules for the study, when appropriate.  Depending upon the risk, size and complexity of the 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
17 
study, the investigator, an expert group, an independent Data and Safety Monitoring Board 
(DSMB) or others might be assigned primary responsibility for this monitoring activity.        
 
NOTE: Regardless of data and safety monitoring plans by the sponsor or others, the principal 
investigator is ultimately responsible for protecting the rights, safety, and welfare of subjects 
under his/her care. 
Data monitoring plan 
All data collection/ storage systems will be piloted before the study begins. 
Any patient data collected as part of the study itself will be stored 
electronically on secure, Partners or Dartmouth servers behind the 
institutional firewall with password protection and anti-virus software. Only 
study staff will have access to the study data on Shared File Areas. The PIs 
will be responsible for monitoring and assuring the validity and integrity of 
the data and adherence to the IRB-approved protocol. 
 
Safety monitoring plan 
The main safety risks for the study include the potential for psychological 
discomfort associated with the intervention. Though this should not differ 
among patients in intervention or control practices, those in intervention 
practices may receive more timely notification of an overdue abnormal test 
result and information about why it is important to complete recommended 
follow-up. The benefits of receiving timely follow-up of an abnormal cancer 
screening tests substantially outweigh this risk. We will monitor complaints 
received from patients and/or providers and notify the local IRB as required 
by local governance. The main risk of the study is unintended release of 
patient health information collected and maintained by the study 
investigators. As noted, we will apply rigorous data safety and monitoring 
standards to ensure that this does not occur. As noted above, if we find that 
a patient has received an alternative evaluation to the diagnostic evaluation 
recommended by national care guidelines and there is no documentation of 
informed decision making, the PIs will contact the patient’s PCP and/or 
clinical director of the primary care practice and/or the institutional 
ambulatory team.  This will be done for patients in any of the study arms 
 
Describe the plan to be followed by the Principal Investigator/study staff for review of adverse 
events experienced by subjects under his/her care, and when applicable, for review of sponsor 
safety reports and DSMB reports.  Describe the plan for reporting adverse events to the sponsor 
and the Partners’ IRB and, when applicable, for submitting sponsor safety reports and DSMB 
reports to the Partners’ IRBs.  When the investigator is also the sponsor of the IND/IDE, include 
the plan for reporting of adverse events to the FDA and, when applicable, to investigators at 
other sites.   
 
NOTE: In addition to the adverse event reporting requirements of the sponsor, the principal 
investigator must follow the Partners Human Research Committee guidelines for Adverse Event 
Reporting

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
18 
 
Although we do not anticipate any direct adverse events from this study, we 
will promptly report any such adverse events to the IRB. Currently, the 
Partners IRB requires that serious adverse events are to be reported to the 
IRB as soon as possible, within 5 working days/7 calendar days of the date 
the investigator first becomes aware of the problem. Non-serious adverse 
events are to be reported within 20 working days.   
 
If patients report significant complaints about our recruitment approach, 
study staff will document all patient concerns and provide a detailed report 
to the IRB about the nature of the complaint. Study staff will discuss with 
the PIs on how to adapt recruitment approaches to prevent future 
complaints, and include any changes required by the IRB.  
 
 
MONITORING AND QUALITY ASSURANCE 
Describe the plan to be followed by the principal investigator/study staff to monitor and assure 
the validity and integrity of the data and adherence to the IRB-approved protocol.  Specify who 
will be responsible for monitoring, and the planned frequency of monitoring.  For example, 
specify who will review the accuracy and completeness of case report form entries, source 
documents, and informed consent.   
 
NOTE: Regardless of monitoring plans by the sponsor or others, the principal investigator is 
ultimately responsible for ensuring that the study is conducted at his/her investigative site in 
accordance with the IRB-approved protocol, and applicable regulations and requirements of the 
IRB.
 
 
 
The PIs will be responsible for monitoring and assuring the validity and 
integrity of the data and adherence to the IRB-approved protocol. 
 
For guidance, refer to the following Partners policies: 
          Data and Safety Monitoring Plans and Quality Assurance
            https://partnershealthcare-
public.sharepoint.com/ClinicalResearch/DSMP_in_Human_Subjects_Research.pdf  
          
          Reporting Unanticipated Problems (including Adverse Events)
          https://partnershealthcare-
public.sharepoint.com/ClinicalResearch/Reporting_Unanticipated_Problems_including_A
dverse_Events.pdf
 
 
 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
19 
PRIVACY AND CONFIDENTIALITY 
Describe methods used to protect the privacy of subjects and maintain confidentiality of data 
collected.  This typically includes such practices as substituting codes for names and/or medical 
record numbers; removing face sheets or other identifiers from completed 
surveys/questionnaires; proper disposal of printed computer data; limited access to study data; 
use of password-protected computer databases; training for research staff on the importance of 
confidentiality of data, and storing research records in a secure location.   
 
NOTE: Additional measures, such as obtaining a Certificate of Confidentiality, should be 
considered and are strongly encouraged when the research involves the collection of sensitive 
data, such as sexual, criminal or illegal behaviors.
 
To ensure patient privacy and confidentiality, all information will be stored 
within the Partners Healthcare or Dartmouth firewall, password protected, 
and anti-virus software enabled. Only study staff will have access to the 
study data on Shared File Areas. A limited dataset will be sent to Dartmouth 
using secure file transfer for data analysis.  A Data Use Agreement will be in 
place before any data transfer occurs. Only de-identified data will be used 
for the purposes of publication or presentation. 
 
 
SENDING SPECIMENS/DATA TO RESEARCH COLLABORATORS OUTSIDE 
PARTNERS 
Specimens or data collected by Partners investigators will be sent to research collaborators 
outside Partners, indicate to whom specimens/data will be sent, what information will be sent, 
and whether the specimens/data will contain identifiers that could be used by the outside 
collaborators to link the specimens/data to individual subjects.
 
A limited dataset will be sent to Dartmouth using secure file transfer for data 
analysis.  A Data Use Agreement will be in place before any data transfer 
occurs. Data elements will include a unique study ID, demographic 
characteristics (age, gender, race/ ethnicity, language, marital status), 
insurance, comorbidity, # of PCP visits in prior 12 months, # of no show 
visits in prior 12 months, date of abnormal cancer screening test, screening 
test abnormality (e.g., BIRAD 3 mammogram result), dates of any follow-up 
diagnostic evaluation, type of any subsequent diagnostic evaluation (e.g., 
subsequent mammogram, biopsy), and survey responses. 
 
Specifically address whether specimens/data will be stored at collaborating sites outside 
Partners for future use not described in the protocol.  Include whether subjects can withdraw 
their specimens/data, and how they would do so.  When appropriate, submit documentation of 
IRB approval from the recipient institution.
 
Data will be stored at Dartmouth for a period of up to 7 years from the date 
of the last publication that results from the data. 

Partners Human Subjects Research Application Form 
  
Filename: Protocol Summary 
Version Date:  October 15, 2014 
 
  
20 
 
As noted above, Dartmouth will be a relying site on the Partners IRB using a 
SIRB review that will be added in an amendment once the Partners protocol 
is approved. 
 
 
 
RECEIVING SPECIMENS/DATA FROM RESEARCH COLLABORATORS OUTSIDE 
PARTNERS 
When specimens or data collected by research collaborators outside Partners will be sent to 
Partners investigators, indicate from where the specimens/data will be obtained and whether the 
specimens/data will contain identifiers that could be used by Partners investigators to link the 
specimens/data to individual subjects.  When appropriate, submit documentation of IRB 
approval and a copy of the IRB-approved consent form from the institution where the 
specimens/data were collected.
 
We will receive a copy of the final limited data set from Dartmouth with pooled 
Partners and Dartmouth data once data collection is completed.  
 
 
_____________________________________________________________ 
 
 
mFOCUS Primary Paper Statistical Analysis Plan 5.19.2022 
 
The goal of the primary paper is to report the primary results of the trial 
1. Study cohort 
a. Primary analysis:  Intent-to-treat  
- 
All patients who met protocol eligibility criteria for at least one abnormal cancer screen 
- 
Patients with multiple tests are included for only the first abnormal screen (design paper) 
- 
Includes patients whose ineligibility was discovered through chart review 
 
b. Secondary analysis:  Modified Intent-to-Treat 
- 
All patients in the ITT analysis, but excluding those whose ineligibility was not discovered 
until chart review (e.g., proc cancer diagnosis; non-clinical care; language; received follow-
up prior to eligibility assessment) 
 
 
 
2. Primary efficacy outcome: Completion of follow-up within 120 days of mFOCUS eligibility (in protocol) 
Two secondary outcomes:  Completion of follow-up within 240 days 
Time to completion 
 
 (others in protocol, but not in this manuscript) 
Exploratory subgroup analyses:  Based on the same primary and secondary outcomes above, within: 
 
Types of cancer (breast; cervical; CRC; lung)    
(in protocol) 
 
Severity of abnormality (low; medium; high)   
(in protocol) 

Partners Human Subjects Research Application Form  
 
Filename: Protocol Summary 
Version Date:  October 15, 2014 
   
21 
 
 
 
 
3. Missing data:  If outcomes or randomization groups are missing for more than 1% of subjects, multiple 
imputation will be used.  If model covariates (see below) are missing for more than 5% of subjects, 
multiple imputation will be used. 
 
 
 
Comments:   The protocol also specified secondary outcomes based on patient and provider surveys; 
these will not be presented in this manuscript but will be the focus of a second publication. 
 
 
In discussions (but not in the protocol), there was interest in exploring how COVID affected 
the outcome, in the sense that patients who became eligible when COVID was more prevalent (and elective 
care was less available) may have taken longer to have follow-up.  Because this is a complicated topic and 
will distract from the main randomized results, it was decided to save such analyses for a subsequent 
manuscript.  The current manuscript will simply adjust (through indicator variables) for 3-month time 
windows.  Since COVID peaked in different areas at different times, these indicator variables will be 
included as interactions with study site (Boston versus Dartmouth). 
 
Contents: 
1. Exhibit 1:   Figure 1: Consort Diagram 
 
 
2. Exhibit 2: Table 1:  Baseline Characteristics, by Arm 
a. Patient characteristics  
i. Sex 
ii. Age 
iii. Race/Ethnicity 
iv. Primary insurance 
v. Primary language 
vi. Marital status 
vii. Comorbidities 
viii. Number of PCP visits in prior year 
ix. Number of no show visits in prior year 
x. Type of cancer 
xi. Severity of screening abnormality 
xii. Rurality (distance or population) 
xiii.  
 
b. PCP characteristics (from protocol; none currently in the prototype table): 
i. Sex 
 
ii. Provider type (MD/ DO, NP/ PA, other) 
iii. Specialty 
iv. Panel size (not risk adjusted) 

Partners Human Subjects Research Application Form  
 
Filename: Protocol Summary 
Version Date:  October 15, 2014 
   
22 
 
c. Clinic characteristics (from protocol; none currently in the prototype table): 
i. Location (hospital; community health center) 
ii. Visit volume at time of randomization 
iii. Patient mix (% female; % Medicaid) at time of randomization 
 
 
3. Exhibit 3:   Table 2: Overall Primary (120 day completion) and secondary (240 day) outcome analyses 
Intent-to-Treat and Modified Intent-to-Treat;   
“Unadjusted” and Adjusted 
 
a. Presentation:  Data will be shown by arm, both ITT and mITT, both adjusted and 
“unadjusted”   
i. The overall global p-value (<0.05) for the adjusted ITT analysis will be shown.  If the 
global test is significant, then three pairwise tests (each arm against control/Arm 1) 
looking for p<0.0167. 
ii. All other analyses and comparisons will be based on 95% confidence intervals 
 
b. Primary and secondary analyses:  SAS Proc GLIMMIX with the patient as the unit of analysis 
and a binary outcome (completed/not completed).  All models (including “unadjusted”) 
will include random effects for practices and physicians.  All models (including 
“unadjusted”) will include fixed effects for type of cancer.  All models (including 
“unadjusted”) will include indicator variables for time (calendar month), along with 
interaction terms with site (Boston versus Dartmouth). 
  
i. Unadjusted analysis: The model above will include random effects for practices and 
physicians, fixed effects for cancer types, fixed effects and interactions for calendar 
month and study stie, and indicator variables for Arms 2, 3, and 4. 
 
ii. Adjusted analysis: The covariates from 2 above will be added individually to the 
model.  Any that are significant at p<0.05 and change the coefficient of any of the 3 
indicators for treatment arm by >20% will be retained in the model as confounders.  
(Type of cancer and calendar month will be retained in the model regardless of 
significance or confounding.)  Adjusted completion rates will be calculated by 
marginal standardization and presented in the Table. 
 
 
 
4. Exhibit 4:   Figure 2:  Forest plot with subgroup analyses for the primary (120 day completion) 
outcome 
Intent-to-Treat   
Adjusted only 
 

Partners Human Subjects Research Application Form  
 
Filename: Protocol Summary 
Version Date:  October 15, 2014 
   
23 
a. Presentation:  Data will be shown by subgroup as an odds ratio (versus the control arm 1) with 
95% confidence interval; ITT; adjusted only   
i. P-values for interaction will be presented 
ii. All other comparisons will be based on 95% confidence intervals 
 
b. Subgroup analyses:  SAS Proc GLIMMIX with the patient as the unit of analysis and a binary 
outcome (completed/not completed).  All models will include random effects for practices and 
physicians.  All models will include fixed effects for type of cancer and calendar month. 
 
i. Subgroups (each set analyzed separately): 
    Type of cancer (breast; cervical; CRC; lung) 
Severity of abnormality (low; medium; high)  
  
ii. Adjusted analysis: The model above will include random effects for practices and 
physicians, fixed effects for cancer types and calendar month, indicator variables for 
Arms 2, 3, and 4, indicator variables for the subgroups, and interaction terms for the 
arms-by-subgroups. 
A global test for effect modification will be run and reported.  However, regardless of 
significance, a separate “unadjusted” model will be run for each subgroup.   
Since, some of the subgroup sample sizes will be limited, not all the covariates from the 
primary overall model above in (3) will be added to each stratified models.  Decisions 
will be made based on the degree of confounding and the sample size. 
 
c. Appendix:  A similar figure will be created for the mITT data.  A decision of whether to 
display it in the appendix will be made based on the findings. 
 
5. Exhibit 5:   Figure 3:  Kaplan-Meier Curves 
a. Four curves (4 arms), unadjusted, ITT 
b. Log-rank  p-value 
c. A Cox regression model will be built for the ITT data.  Since only one level of clustering is 
available, the model will use the analogue of random effects for practices and adjust for 
the same variables as in the logistic regression model above. 
i. Significance testing will use a global p-value across all 4 arms and, if significant at 
p<0.05, pairwise comparisons between each arm and the control arm will be 
carried out.  Results for the pairwise comparisons will be presented as 95% 
confidence intervals.   
ii. The results of the Cox regression will be reported in the text.  A table of the model 
results may appear in the Appendix.