Implementing to Sustain: Determining the Minimum Necessary Intervention to Maintain a Postpartum Depression Prevention Program (ROSE) in Clinics Providing Prenatal Services to Low-income Women
Phase: NA
Conditions: Depression, Postpartum
Sponsor: Michigan State University

Protocol Cover Sheet 
 
Principal Investigator (PI): Jennifer Johnson, PhD  
Institution: Michigan State University 
Email: jjohns@msu.edu  
Phone: (810) 600-5669 
Study Title: Implementing to Sustain: Determining the Minimum Necessary Intervention to 
Maintain a Postpartum Depression Prevention Program (ROSE) in Clinics Providing 
Prenatal Services to Low‑Income Women 
Co-Investigators: Caron Zlotnick, PhD; Rebecca Weinberg, PhD; Ellen Poleshuck, PhD; 
Tiffany Moore-Simas, MD, MPH, Med; Laura Carravallah, MD; Shannon Wiltsey-Stirman, PhD 
NCT Number: NCT03267563 
Study Sponsor/Funding Source: National Institute of Health 
IRB Submission Number (if applicable): 17-1052 
Date: 01/30/2023 
Study Summary: 
ROSE is a group-based program delivered during pregnancy to prevent postpartum 
depression, with strong evidence of effectiveness among low‑income women. This study 
will test how best to sustain ROSE across 98 prenatal clinics using a Sequential Multiple 
Assignment Randomized Trial (SMART) design. All clinics begin with enhanced 
implementation as usual, and those at risk of failing to sustain the program are sequentially 
randomized to receive additional low‑ or high‑intensity coaching. Follow‑up continues 
through 30 months, with implementation support ending at 18 months. Key outcomes 
include sustainment of program delivery, clinic‑level postpartum depression rates, and the 
cost‑effectiveness of each sustainment strategy 
 
 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
1 
MICHIGAN STATE UNIVERSITY HUMAN RESEARCH PROTECTION PROGRAM 
x 
Complete this template for new exempt, expedited, or full board studies. 
o Complete Section I for ALL studies (exempt, expedited, full board) 
o Complete Section II ONLY if your study does not qualify for exemption and requires an 
expedited or full board review. Contact the IRB office if you have any questions. 
x 
CLICK™ IRB:  
o Include the template with a New Study Submission. 
o Upload the completed template to the Basic Information SmartForm page, Question 10.  
o When uploading documents to Click (e.g. consent documents, instrument), provide distinct 
file names. 
x 
See the Click Quick Guides and the HRPP Manual for more information, available at hrpp.msu.edu 
 
Study Title: 
Study title.  Implementing to sustain: Determining the minimum 
necessary intervention to maintain a postpartum depression 
prevention program (ROSE) in clinics providing prenatal services to 
low-income women  
 
Click Study ID (if known): 
      
Sponsor (if applicable): 
National Institute of Health 
Sponsor ID (if applicable): 
      
 
Section I. IRB Protocol for All Studies 
Section I is completed for all studies and includes questions to determine whether the study qualifies 
for exemption. Section II is only completed if the study does not qualify for exemption. 
 
1. 
Hypothesis / Objective / Goals / Aims. 
Briefly describe the study’s hypothesis / objectives / goals / aims.  
ROSE is a group intervention to prevent postpartum depression (PPD), delivered during 
pregnancy. ROSE has been found to reduce cases of PPD in multiple randomized trials in 
community prenatal settings with low-income pregnant women. Given the need for return on 
investment studies about sustainment efforts, we propose a Sequential Multiple Assignment 
Randomized (SMART) Trial of the effectiveness and cost-effectiveness of a stepwise approach 
to sustainment of ROSE in 98 outpatient clinics providing prenatal care to pregnant women on 
public assistance in the U.S. states. 
 
In Year 1, all clinics will receive enhanced implementation as usual (EIAU; initial training + 
tools for sustainment). At the first time at which a clinic is determined to be at risk for failure to 
sustain (i.e., at 3, 6, 9, 12, 15 months), that clinic will be randomized to receive either: (1) no 
additional implementation support (i.e., EIAU only), or (2) low intensity coaching and feedback 
(LICF). If clinics receiving LICF are still found to be at risk at subsequent assessments, they 
will be randomized to either (1) EIAU + LICF only, or (2) high intensity coaching and feedback 
(HICF). Additional study follow-up interviews will occur at 18, 24, and 30 months, but no 
implementation intervention will occur after 18 months. Outcomes include: 1. Sustainment of 
core program elements at each time point and total length of time ROSE services were 
provided and were provided with at least moderate fidelity. 2. Health impact (PPD rates over 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
2 
time at each clinic) and reach. 3. ROI (costs, cost-offsets, and cost-effectiveness) of each 
sustainment step. 
2. 
Subject Population. 
2A. Study purposefully includes the following subject population(s) (select all that apply): 
 Cognitively impaired adults 
 Minors (children) (view information about the definition of a child) 
 Minors who are wards of the state 
 Pregnant women, fetuses, or neonates 
 Prisoners 
 Students 
2B. Study involves (select all that apply): 
 Funding, support, or other requirement to comply with U.S. Department of Justice 
regulations 
 Incomplete disclosure or attempted deception of subjects 
3. 
Estimated Study Duration. 
Provide the time estimated to complete all human subject research, including analysis of the 
subjects’ identifiable private information. 
5 years 
4. 
Reasonably Foreseeable Risks. 
4A. There are (select one of the following): 
 
 No reasonably foreseeable risks to subjects 
 Reasonably foreseeable risks to subjects 
4B. Explain the selection. If you selected that there are reasonably foreseeable risks to subjects, 
describe the risks, considering physical, psychological, social, legal and economic risks. 
There are two related areas of low to moderate reputational risk associated with participation 
in the proposal research study: (1) a breach of confidentiality by researchers, or (2) 
discomfort at having their in-session behaviors self-monitored to assess their ability to deliver 
ROSE and overall clinic performance discussed. In addition, it is possible that a breach of 
confidentiality (e.g., of answers to surveys about clinic climate and leadership) could affect 
employment, although every effort will be made to minimize this risk.. 
4C. If you selected that there are reasonably foreseeable risks, describe the procedures for 
protecting against or minimizing potential risks and provide an assessment of their likely 
effectiveness. 
Confidentiality. Materials will be identified by an assigned provider ID number (which will not 
involve personal identifying features such as date of birth, etc.). No individual-level data will 
be shared with employers, and any published reports or presentations will only include data 
that is aggregated in a manner that does not allow the identities of providers, administrators, 
or agencies. Feedback about individual provider performance will be shared with that provider 
only, or with the provider’s written permission, in a group. Feedback about administrative or 
operational challenges will be given to the operational/administrative study respondent. 
Feedback about clinic-asa- whole performance (e.g., PPD rates, etc.) will be provided only to 
staff from that clinic. No individual-level data will be shared with employers, and any 
published reports or presentations will only include data that is aggregated in a manner that 
does not allow the identities of providers or agencies. 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
3 
Confidentiality will be scrupulously maintained by standard procedures employed in clinical 
trials, including the use of participant numbers/codes instead of names, the storage of all data 
in locked cabinets or rooms, the use of secure electronic data management systems, and the 
withholding of all participant information from release without the express written consent of 
the individual. Electronic data will be stored on a secure research server at Michigan State 
University and will be available only to authorized personnel. Paper data will be kept in locked 
file cabinets at Michigan State University (MSU) and/or Butler Hospital. 
Clinics will manage clinical risks of their patients in accordance with their usual policies and 
procedures. Participating providers and administrators will be reminded to address any high-
risk situations that occur with participating clients according to their employers’ crisis 
management procedures and policies. If these procedures or policies are affected by 
implementation of ROSE, as part of EIAU, we will offer to work with clinics to consider options 
for how they can extend or expand their procedures/policies to manage clinical risk in a way 
that fits their own clinical responsibilities to patients. 
Recruitment and Informed Consent.  
There will be 2 sets/kinds of consents for participating clinics: 
(1) Clinical and organizational representatives who will respond to study surveys and 
qualitative interviews. 
(2) ROSE providers who will provide self-reported fidelity ratings of ROSE. 
To prevent individuals in categories 2-4 above from feeling pressure to participate, the 
following measures will be taken: Respondents will only be recruited directly by the research 
team, from agencies or clinics that are willing to allow their clinicians to participate in all study 
procedures. It will be made clear that data that is gathered during the study will not be shared 
with the administration of agencies (other than clinic-level feedback, such as quarterly PPD 
rates at the clinic, clearly specified in advance in the consent form). Staff who participate in 
the study will receive a thorough description of study-related procedures, will have the 
opportunity to ask questions, and will provide informed consent. 
Some people may be in multiple categories above and will be asked to consent to each 
aspect of the study that is relevant to them. Given that participating clinics will be from many 
different U.S. states, consent will take place electronically, with the study also described by 
telephone or at staff meetings when feasible. Each person will be emailed a link to a 
confidential study electronic consent form explaining study participation, risks and benefits, 
etc. and be given an opportunity to enroll or decline. The consent form will outline the nature 
of the study, the procedures they will be asked to follow, time commitment, how to withdraw if 
they choose to do so, compensation, who to contact with questions or concerns during the 
study. The consent form will be clear that the clinic will not be told whether or not they choose 
to participate and that agency directors have agreed that participation or non-participation will 
not be shared and will not affect their employment in any way. They will be told that the 
agency administrative leaders have affirmed that a decision to withdraw participation at any 
phase of the study will not be a part of their employment record, and will not impact their 
employment or their eligibility for other training opportunities, promotions, etc. They will be 
given time to decide whether to participate (at least a day, more often a week or more) before 
study procedures begin. Study staff will also be available by phone to answer questions or 
explain aspects of consent. 
5. 
Conflict of Interest. 
Do any investigators or research staff have a financial interest related to the 
research that has not otherwise been disclosed elsewhere in this submission? 
 No 
 Yes 
6. 
Exemption Criteria.  
 Not Applicable  
A study may qualify for exemption when the only involvement of human subjects will be in one 
or more of the following categories (please view full exemption category / description here: 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
4 
https://hrpp.msu.edu/help/required/exempt-categories.html).  (If the study does not qualify 
for the exemption criteria, do not complete this question and proceed to Section II.) 
6A. Exemption Categories. 
6A1.  Select the category(ies) applicable to the study if the only involvement of human 
subjects in this study will be in one or more of the categories. Studies involving 
prisoners cannot be exempt UNLESS the research is aimed at involving a broader 
subject population that only incidentally includes prisoners If your study is subject to 
U.S. Department of Justice requirements, do not complete this section; complete 6A2 
below. 
 Exempt 1. Research conducted in established or commonly accepted educational settings, 
involving normal educational practices that are not likely to adversely impact students' 
opportunity to learn required educational content or the assessment of educators who provide 
instruction. IF YOU SELECTED THIS CATEGORY, EXPLAIN WHY THE RESEARCH WILL 
NOT LIKELY ADVERSELY IMPACT STUDENTS’ OPPORTUNITY TO LEARN REQUIRED 
EDUCATIONAL CONTENT OR THE ASSESSEMENT OF EDUCATORS WHO PROVIDE 
INSTRUCTION. 
       
 Exempt 2. Research that only includes interactions involving educational tests (cognitive, 
diagnostic, aptitude, achievement), survey procedures, interview procedures, or observation of 
public behavior. IF YOU SELECTED THIS CATEGORY, SELECT THE APPROPRIATE 
OPTION(S) BELOW. 
 
 (i) Information obtained is recorded by investigator in manner that identity of subjects cannot 
readily be ascertained, directly or through identifiers linked to subjects 
 
 (ii) Any disclosure of subjects' responses outside research would not reasonably place 
subjects at risk of criminal or civil liability or be damaging to subjects' financial standing, 
employability, educational advancement, or reputation. 
 
 (iii) LIMITED IRB REVIEW REQUIRED. Information obtained is recorded by investigator in 
manner that identity of subjects can readily be ascertained, directly or through identifiers 
linked to subjects, and responses could reasonable place subjects at risk of criminal or civil 
liability or be damaging to the subjects' financial standing, employability, educational 
advancement, or reputation (LIMITED IRB REVIEW IS REQUIRED; YOU MUST ALSO 
COMPLETE QUESTION 6E TO DESCRIBE PRIVACY AND CONFIDENTIALITY 
SAFEGUARDS.) 
 Exempt 3. Research involving benign behavioral interventions in conjunction with the collection 
of information from an adult subject through verbal or written responses (including data entry) or 
audiovisual recording if the subject prospectively agrees to the intervention and information 
collection. IF YOU SELECTED THIS CATEGORY, SELECT THE APPROPRIATE OPTION(S) 
BELOW.   
 
 (i) Information obtained is recorded by investigator in manner that identity of subjects cannot 
readily be ascertained, directly or through identifiers linked to subjects. 
 
 (ii) Any disclosure of subjects' responses outside research would not reasonably place 
subjects at risk of criminal or civil liability or be damaging to subjects' financial standing, 
employability, educational advancement, or reputation 
 
 (iii) LIMITED IRB REVIEW REQUIRED. Information obtained is recorded by investigator in 
manner that identity of subjects can readily be ascertained, directly or through identifiers 
linked to subjects, and responses could reasonable place subjects at risk of criminal or civil 
liability or be damaging to subjects' financial standing, employability, educational 
advancement, or reputation (LIMITED IRB REVIEW IS REQUIRED; YOU MUST ALSO 
COMPLETE QUESTION 6E TO DESCRIBE PRIVACY AND CONFIDENTIALITY 
SAFEGUARDS.) 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
5 
 Exempt 4. Secondary research uses of identifiable private information or identifiable 
biospecimens. IF YOU SELECTED THIS CATEGORY, SELECT THE APPROPRIATE 
OPTION(S) BELOW.   
 
 Identifiable private information or identifiable biospecimens are publicly available. 
 
 Information, which may include information about biospecimens, is recorded by the 
investigator in such a manner that the identity of the human subjects cannot readily be 
ascertained directly or through identifiers linked to the subjects, the investigator does not 
contact the subjects, and the investigator will not re-identify subjects. IF YOU SELECTED 
THIS CATEGORY, CONFIRM THE FOLLOWING: 
  
 Investigator and research team will not contact the subjects 
 Investigator and research team will not re-identify the subjects 
 
 The research involves only information collection and analysis involving the investigator's 
use of identifiable health information when that use is regulated under the Health Insurance 
Portability and Accountability Act (HIPAA) (45 CFR parts 160 and 164). 
 
 The research is conducted by, or on behalf of, a Federal department or agency using 
government-generated or government-collected information obtained for nonresearch 
activities, if the research generates identifiable private information that is or will be 
maintained on information technology that is subject to and in compliance with specific 
federal privacy standards. 
 Exempt 5. Federal demonstration projects. 
 Exempt 6. Taste and food quality evaluation and consumer acceptance studies. 
 Exempt 97. ONLY applicable to research NOT FUNDED by a federal department or agency: 
Research involving the study of previously collected identifiable data (please view additional 
exclusions before selecting this category). 
By checking the boxes below, you are confirming that the study will not include any of the 
following exclusions for the study’s duration: 
 Federal funding or federal training grants 
 FDA regulated 
 Sponsor or other contractual restrictions 
 Clinical interventions (including clinical behavioral interventions) 
 Receipt of an NIH issued certificate of confidentiality to protect identifiable research data 
 Multi-site collaborative research study where another institution plans to rely or is relying upon 
MSU’s IRB review 
 Exempt 98. ONLY applicable to research NOT FUNDED by a federal department or agency: 
Prospective data collection with adults through verbal or written responses involving a benign 
intervention (please view additional exclusions before selecting this category). 
By checking the boxes below, you are confirming that the study will not include any of the 
following exclusions for the study’s duration: 
 Federal funding or federal training grants 
 FDA regulated 
 Sponsor or other contractual restrictions 
 Clinical interventions (including clinical behavioral interventions) 
 Receipt of an NIH issued certificate of confidentiality to protect identifiable research data 
 Multi-site collaborative research study where another institution plans to rely or is relying upon 
MSU’s IRB review  
 Children as research subjects 
 
6A2. DEPARTMENT OF JUSTICE Exemption Categories. Complete this section ONLY if the 
research is subject to Department of Justice requirements. 
 
6A2i. Select the category(ies) applicable to the study if the only involvement of human subjects in this 
study will be in one or more of the categories. Studies involving prisoners cannot be exempt. 
 
 Exempt 1. Research conducted in established or commonly accepted educational settings, 
involving normal educational practices 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
6 
 
 Exempt 2. Educational tests, survey procedures, interview procedures, observation of public 
behavior unless data is recorded in a manner such that subjects are identifiable and the 
responses could reasonably place the subjects at risk of criminal or civil liability or be damaging 
to the subjects’ financial standing, employability, or reputation (research cannot involve children, 
except for educational tests or observation of public behavior where the investigator does not 
interact with the child). 
 
 Exempt 3. Educational tests, survey procedures, interview procedures, or observation of public 
behavior not otherwise exempt that involves public officials or federal statute. 
 
 Exempt 4. Collection or study of existing data, documents, records, pathological specimens, or 
diagnostic specimens if publicly available or information is recorded by investigator in a manner 
that subjects cannot be identified. 
 
 Exempt 5. Federal demonstration projects. 
 
 Exempt 6. Taste and food quality evaluation and consumer acceptance studies. 
 
6A2ii. Explain why the study presents minimal risk to subjects. 
 
       
6B. By checking the boxes below, you are confirming that the following are true and will remain 
true for the study’s duration: 
  Selection of subjects is equitable (considering the purposes of the research, setting in 
which research will be conducted, any vulnerable populations). 
  If there is recording of identifiable information, there are adequate provisions to maintain 
the confidentiality of the data. 
  There are adequate provisions to maintain the privacy interests of subjects. 
  Safeguards are or will be put in place to protect against any coercion or undue influence if 
you or members of your study team are or may be associated with the subjects at any point 
in the study (e.g. students, employees, colleagues, patients). 
6C. Consent  
6Ci. There will be a consent process for the study’s duration that will disclose information such as 
that the activity involves research, a description of the procedures, that participation is 
voluntary and withdrawal is without penalty, and the name and contact information for the 
researcher (select appropriate option below): 
 For All Subjects  
 For Some Subjects 
 For None of the Subjects (consent will not be obtained) 
6Cii. Please explain your selection.  
There will be 2 kinds of consents for participating clinics: 
(1) Clinical and organizational representatives who will respond to study surveys and 
qualitative interviews. 
(2) ROSE providers who will provide self-reported fidelity ratings of ROSE. 
6D. Please acknowledge that you may not begin the research at non-MSU institutions (regardless 
of engagement), until you receive the appropriate approvals/permissions from the sites (e.g. 
IRB review/exempt determination from non-MSU sites, data use or research agreements, other 
regulatory approvals). An MSU exempt determination does not provide approval/permission for 
a non-MSU site, including sites with reliance agreements with MSU. Please note that non-MSU 
sites may have requirements that differ from MSU for exempt research. Note that this also 
applies to sites added after the MSU exempt determination. 
 Acknowledged 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
7 
6E. LIMITED IRB REVIEW. If the exemption(s) require limited IRB review (if you selected 
Exemption 2(iii) or 3(i)(C) in Question 6A), complete questions 1 and 2 to describe privacy and 
confidentiality.  
6E1. Privacy of Subjects. 
 How will subjects’ privacy be protected? Consider the number of individuals interacting with the 
subject or subject’s records, location of consent process and study, presence of individuals not 
associated with the study, sensitivity of the research. 
 Materials will be identified by an assigned provider ID number (which will not involve personal 
identifying features such as date of birth, etc.). No individual-level data will be shared with 
employers, and any published reports or presentations will only include data that is aggregated 
in a manner that does not allow the identities of providers, administrators, or agencies. 
Feedback about individual provider performance will be shared with that provider only, or with 
the provider’s written permission, in a group. Feedback about administrative or operational 
challenges will be given to the operational/administrative study respondent. Feedback about 
clinic as a whole performance (e.g., PPD rates, etc.) will be provided only to staff from that 
clinic. No individual-level data will be shared with employers, and any published reports or 
presentations will only include data that is aggregated in a manner that does not allow the 
identities of providers or agencies. 
Confidentiality will be scrupulously maintained by standard procedures employed in clinical 
trials, including the use of participant numbers/codes instead of names, the storage of all data 
in locked cabinets or rooms, the use of secure electronic data management systems, and the 
withholding of all participant information from release without the express written consent of the 
individual. Electronic data will be stored on a secure research server at Michigan State 
University and will be available only to authorized personnel. Paper data will be kept in locked 
file cabinets at Michigan State University (MSU) and/or Butler Hospital. 
6E2. Confidentiality of Data. 
6E2i. Select the appropriate option: 
 
 Identifying or coded information will not be stored with the information and/or 
biospecimen(s) 
 Identifying or coded information will be stored with the information and/or biospecimen(s) 
6E2ii. Please explain your selection. If you are storing identifying or coded information with the 
information and/or biospecimen(s), explain why identifiable or coded information and/or 
biospecimen(s) needs to be maintained and how long it will be necessary to maintain it. 
 Electronic data will be stored on a secure research server at Michigan State University and will 
be available only to authorized personnel. Paper data will be kept in locked file cabinets at 
Michigan State University (MSU) and/or Butler Hospital. 
6E2iii. Describe the procedures and safeguards you will use to secure the information and/or 
biospecimen(s), including during transport of information and/or biospecimen(s). 
 
Only trained research staff will have access to the research data. 
 
 
 
 
IF THE STUDY MAY QUALIFY FOR AN EXEMPTION  
(INCLUDING THOSE THAT MAY REQUIRE LIMITED IRB REVIEW),  

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
8 
STOP HERE AND DO NOT COMPLETE SECTION II.  
 
CONTINUE ONLY IF THE STUDY  
DOES NOT QUALIFY FOR AN EXEMPTION.  
 
COMPLETE QUESTIONS 7-23 FOR AN  
EXPEDITED OR FULL BOARD STUDY. 
 
 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
9 
Section II. Additional Questions for an Expedited or Full Board Study 
Not all questions or sections are applicable to every study. If the question or section is not applicable, 
check the “Not Applicable” box. All other questions are required. 
 
7. 
Expedited Categories. 
7A. Please select the Expedited category(ies) and sub-categories as applicable to the study if the 
only involvement of human subjects in this study will be in one or more of the categories. If the 
study involves more than minimal risk or none apply, select “The study involves more than 
minimal risk OR none of the expedited categories apply.” 
 
 The study involves more than minimal risk OR none of the expedited categories apply. IF 
THIS OPTION IS SELECTED, DO NOT SELECT ANY OF THE EXPEDITED CATEGORY(IES). 
 
 Expedited 1. Clinical studies of drugs and medical devices only when condition (a) or (b) is met. 
IF YOU SELECTED THIS CATEGORY, SELECT THE APPROPRIATE OPTION(S) BELOW. 
 
 
 (a) Research on drugs for which an investigational new drug application (21 CFR Part 
312) is not required. (Note: Research on marketed drugs that significantly increases the 
risks or decreases the acceptability of the risks associated with the use of the product is 
not eligible for expedited review.) 
 
 
 (b) Research on medical devices for which (i) an investigational device exemption 
application (21 CFR Part 812) is not required; or (ii) the medical device is 
cleared/approved for marketing and the medical device is being used in accordance with 
its cleared/approved labeling. 
 
 Expedited 2. Collection of blood samples by finger stick, heel stick, ear stick, or venipuncture. IF 
YOU SELECTED THIS CATEGORY, SELECT THE APPROPRIATE OPTION(S) BELOW. 
 
 
 
(a) from healthy, nonpregnant adults who weigh at least 110 pounds. For these subjects, 
the amounts drawn may not exceed 550 ml in an 8 week period and collection may not 
occurmore frequently than 2 times per week; or 
 
 
 
(b) from other adults and children, considering the age, weight, and health of the subjects, 
the collection procedure, the amount of blood to be collected, and the frequency with 
which it will be collected. For these subjects, the amount drawn may not exceed the lesser 
of 50 ml or 3 ml per kg in an 8 week period and collection may not occur more frequently 
than 2 times per week 
 
 Expedited 3. Prospective collection of biological specimens for research purposes by 
noninvasive means. 
 
 Expedited 4. Collection of data through noninvasive procedures (not involving general 
anesthesia or sedation) routinely employed in clinical practice, excluding procedures involving x-
rays or microwaves. Where medical devices are employed, they must be cleared/approved for 
marketing. (Studies intended to evaluate the safety and effectiveness of the medical device are 
not generally eligible for expedited review, including studies of cleared medical devices for new 
indications.) 
 
 Expedited 5. Research involving materials (data, documents, records, or specimens) that have 
been collected, or will be collected solely for nonresearch purposes (such as medical treatment or 
diagnosis).  
 
 Expedited 6. Collection of data from voice, video, digital, or image recordings made for research 
purposes. 
 
 Expedited 7. Research on individual or group characteristics or behavior (including, but not 
limited to, research on perception, cognition, motivation, identity, language, communication, 
cultural beliefs or practices, and social behavior) or research employing survey, interview, oral 
history, focus group, program evaluation, human factors evaluation, or quality assurance 
methodologies.  

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
10 
7B. For Studies Regulated by the U.S. Food and Drug Administration or the U.S. Department of 
Justice ONLY. If you selected an expedited category, explain why the study presents minimal risk to 
subjects. 
       
8. 
More than Minimal Risk Research. Complete the following question if you selected “The 
study involves more than minimal risk OR none of the expedited categories apply” in Question 
7A (Expedited Categories). 
8A. Describe the relevant prior experience and gaps in current knowledge, relevant preliminary 
data, if any, and the scholarly background for, and significance of, the research based on 
existing literature and how it will add to existing knowledge. 
Postpartum depression (PPD) is a common and impactful public health problem, especially 
among low-income women. There is little consensus among researchers as to what 
constitutes the postpartum period: timeframes range from one month to one year after 
delivery.4 A PPD meta-analysis obtained an average prevalence rate of 13% within the first 12 
weeks following childbirth.5 Evidence suggests that the postpartum period places a woman at 
increased risk for a major depressive disorder (MDD).6,7  Untreated PPD can have severe 
and lasting consequences for mother and infant, including maternal increased risk of suicide,8 
compromised functional status9-11 and social functioning,8,12-19 less maternal time in 
positive activities with infant,18 and less maternal infant sensitivity.20  For infants, the effect of 
untreated PPD are also severe and lasting, with poorer cognitive and language development 
from infancy to late childhood.21-24 Furthermore, the cost of health and social care is much 
higher in women with PPD compared to those without PPD.25     
The burden and consequences of PPD are greatest among low-income women. Low-income 
women have higher rates of PPD than other income groups,26,27 especially unrecognized 
and untreated PPD.28,29  Up to 50% of low-income women report PPD.30-34  Low-income 
women are often exposed to chronic stress and their high levels of perceived stress are 
strongly associated with increased risk for PPD.35  In addition, the consequences of PPD are 
often more severe for low-income women. The negative effects of untreated PPD on mother-
infant interactions, maternal caregiving, infant development, and child language development 
appear to be potentiated by the presence of socioeconomic adversity.20,21,36  Among low-
income women, the higher prevalence of PPD, the reduced likelihood of treatment, and the 
impact of untreated PPD on impaired development in children37,38 leads to health disparities 
in children of these affected mothers.22   
Care gap or quality gap: prenatal clinics do not routinely do anything to prevent PPD.  In 
addition to the risks associated with PPD among low-income women, they are unlikely to 
receive treatment for their PPD. Therefore, timely and effective interventions to reduce their 
risk of PPD (i.e., to prevent PPD rather than wait to treat it once it appears) are critical.  
Prevention can significantly reduce the human and economic burden associated with mental 
illness.39  Despite this, health professionals have remained focused on identifying and treating 
perinatal depression after its onset.  Current initiatives focus on trying to help clinics identify 
PPD after it occurs (i.e., screening) and provide linkage to PPD treatment, with mixed 
success.40,41  OBGYN clinics and other clinics providing prenatal care do not routinely do 
anything to prevent PPD.  In fact, there is no related information for health professionals on 
the prevention of PPD on the websites of several key  
U.S. national healthcare organizations (e.g., American College of Obstetricians and 
Gynecologists, College of Nurse Midwives; Association of Women’s Health Obstetric and 
Neonatal Nurses).  Likewise, current websites and apps for pregnant or postpartum women 
provide only information on how to identify, track symptoms or treat postpartum depression, 
rather than information on how to prevent PPD.   
Policy trends and increasing investment in addressing PPD support the likelihood of 
successful implementation of a PPD prevention program. Preventing PDD is a priority of the 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
11 
federal government as well as most state and regional health departments. Several states 
have passed legislation that requires state agencies to develop educational programs 
pertinent to PPD for women and their families42.  A bill was recently passed under Section 
2952 of the federal Patient Protection and Affordable Care Act authorizing grants to support 
the establishment, operation, and delivery of effective and cost-efficient systems for providing 
clinical services to women with, or at risk for, PPD. Given that federally qualified health centers 
(FQHCs) and other clinics are now mandated to provide comprehensive perinatal services 
including behavioral health care to low-income women, we believe these Centers have the 
interest in and funds to support PPD prevention. For example, the Philadelphia Department of 
Public Health plans in this year to train interventionists to deliver ROSE across clinics serving 
low-income pregnant women and integrate it into their standard care plan; the U.S. Health 
Resources and Services Administration (HRSA) will cover the costs. Furthermore, under a 
capitation model in which the Centers for Medicare & Medicaid Services (CMS) pays each 
health plan a prospective capitation payment, a prevention model of care could reduce the 
cost of treatment.  When a capitation model has been implemented, primary care physicians 
emphasize preventive care.43  PPD was also an emphasis of a recent US Preventive 
Services Task Force Recommendation.44  As clinics work to put new PPD screening 
guidelines into place, PPD is of interest to many clinics, facilities, and providers.    
The ROSE Program is an evidence-based practice for preventing PPD.45  To date, despite 
over 40 experimental studies evaluating preventative interventions for PPD, there exists only 
one preventative intervention for PPD, the ROSE Program (Reach Out, Stay Strong, 
Essentials for mothers of newborns), that (1) has been found to significantly reduce cases of 
PPD, (2) has replicated findings, (3) has been tested in community settings, (4) has been 
tested using a validated diagnostic measure of PPD, and (5) has been tested in racially and 
ethnically diverse women as well as in heterogeneous samples (e.g., low-income pregnant 
women and teens). ROSE uses principles of interpersonal psychotherapy (IPT), an evidence-
based treatment for MDD46,47 and PPD.48,44,49 However, the goal of ROSE is to prevent 
PPD. Designed to address the especially high risk of PPD among low-income women, ROSE 
is administered during pregnancy to women in small groups, and teaches IPT-based skills for 
improving communication and building social support, identified risk factors for PPD.5,50,51 
ROSE is presented as a course to minimize stigma and emphasize the program as an 
educational experience. ROSE consists of 4 90-minute group sessions and a postdelivery 50-
minute individual booster session, with easy to read handouts and homework for each 
session. At least 5 randomized trials (RCTs) support its efficacy and effectiveness in 
preventing PPD among low-income women. ROSE is flexible, easy to implement, and is easily 
delivered by individuals already working in prenatal clinics. The manual is highly scripted.  
Interventionists with varying qualifications, such as health educators and paraprofessionals 
(i.e., individuals with a bachelor’s degree), have delivered ROSE with good adherence and 
competence.57 Nurses or medical assistants may deliver ROSE.  Several rounds of 
adaptation to improve fit with the target population have already been integrated: ROSE 
overcomes barriers to attendance for low-income women by coordinating sessions with 
women’s prenatal clinic appointments and having a flexible delivery structure. Despite multiple 
life demands and stressors, very few of the low-income women in these studies used baby-
sitting or transportation services that were offered and ROSE attendance and PPD outcomes 
were still good (a mean of 3.5 of 5 sessions attended).57 Our implementation interventions will 
work with clinics to determine which staff, scheduling, and support options work best for them. 
Furthermore, there is interest in picking up ROSE outside of a research context.  In sum, 
ROSE is an effective program that low-income women like and engage in, and its logistics 
(staff, location, etc.)  
There are very few studies on the implementation of interventions to prevent mental health 
disorder among adults. However, this study capitalizes on what is known about barriers and 
facilitators in similar situations (i.e., providing mental health treatments in primary care 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
13 
chosen predictors (organizational and policy contexts) are clearly articulated by all these 
authors.3,69 70   
Timing.  Strict definitions of sustainment specify that core elements, etc. remain after external 
supports are withdrawn.3,69  All clinics in this study will be assessed for at least 12 months 
after external supports are withdrawn (Months 19-30). Clinics that receive EIAU only (either 
because they implement successfully for 18+ months after initial training, or because they do 
not and are randomized to continue receiving EIAU only) will not receive additional 
implementation supports after initial training/planning, and thus will be assessed for 30 months 
after supports are withdrawn. This is an estimated 37% of clinics. We chose to potentially 
intervene with other clinics for up to 18 months (of 30) after initial training to better evaluate 
what supports are needed and when to achieve optimal sustainment (i.e., to give more clinics 
a chance to succeed).  While some have suggested a rule of thumb timeframe of two years 
post-implementation to measure sustainment3, we conceptualize this study as examining 
when and how much support is necessary to set programs up for sustainment, including  
“rescuing” sites that implement well for a while and then become at risk not to sustain.   
 Choice of implementation interventions is based on the action-oriented Replicating Effective 
Programs (REP) framework,71 developed by the CDC to guide implementation of prevention 
programs (ROSE is a prevention program).  For implementing ROSE in prenatal clinics, all the 
implementation pre-conditions have been met. We have identified a high-burden condition 
(PPD), identified an effective intervention that fits prenatal clinics (ROSE), and packaged the 
intervention (i.e., it has already been packaged for, tested with, and found feasible, 
acceptable, and effective using OBGYN clinic nurses as interventionists52,53,57). 
Implementation and sustainment interventions used in the current study are guided by the next 
three phases.  
Implementation interventions being tested in current study, & rationale.  The implementation 
interventions being tested fit the REP conceptual framework.  Enhanced implementation as 
usual (EIAU; initial training + tools for sustainment) will consist of the Pre-implementation 
steps shown in the framework: initial training will include an explanation of the core elements, 
discussion about how delivery can (and should not) be customized, logistics planning, and 
staff training (including training of both clinical staff and work with office staff to clarify billing, 
scheduling, staffing, and other issues). The two experimental sustainment conditions (i.e., low-
intensity coaching and feedback [LICF] and high-intensity coaching and feedback [HICF]) will 
contain lower (every 3 months) and higher doses (every month) of the steps of Implementation 
in the framework (technical assistance, ongoing support of and conversation with the clinics, 
coaching [including booster training and process evaluation], feedback; figure adapted from 
Kilbourne et al.71).  All three implementation interventions being tested provide clinics with 
opportunity and guidance for re-customizing delivery and making organizational and financial 
changes to sustain the intervention in varying doses, as suggested in the framework’s 
Maintenance and Evolution phase. As suggested by REP, our “Collaborative Board” will 
include staff from clinics who have implemented ROSE previously in clinical practice (i.e., 
before this study, not part of research) who can provide advice on sustainment strategies.   
Relevance to return on investment (ROI) for sustainment efforts. Given that the research 
question focuses on determining the minimum necessary to sustain ROSE, the 
implementation conditions (EIAU, LICF,  
HICF) were also chosen to reflect three different doses/intensities of a relatively standard 
approach72 to allow their intensity/cost aspects to take precedence. A review by Herschell et 
al.72 of more complex psychosocial interventions (e.g., cognitive-behavior therapy) concluded 
that while workshop follow-ups that included  feedback, consultation, and/or coaching have 
improved adoption, skill  and client outcome and allow contextual barriers to be addressed, 
“The challenge is that these methods are resource intensive. The field needs to determine 
how to sequence learning activities to be cost-effective without compromising training and 
treatment outcome.”72  That is the goal of this study and the rationale for our choice of 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
14 
implementation interventions (EIAU, LICF, HICF) that represent increasing steps of cost and 
intensity.  Finally, the implementation interventions chosen utilize the training and coaching 
materials and experiences from previous research and real-world implementations of 
ROSE.57  Therefore, this project is guided by a clear, well-known framework, much of the 
groundwork for successful implementation and sustainment has already been laid, and the 
chosen implementation interventions are a good fit for the research question and settings.      
8B. Sample Size. 
8Bi. Total number of subjects who will be approached (including screen failures, controls and 
subject withdrawals) to reach enrollment numbers for the lifetime of the study at this 
investigator’s sites. 
 193 
8Bii. Total number of subjects who will be enrolled in the study at this investigator’s site. 
 98 
8Biii. Describe the statistical justification or rationale for the proposed sample size. Considerations 
for sample size may include the acceptable level of significance, power of the study, expected 
effect size, underlying event rate in the population, standard deviation in the population, 
saturation of themes, and/or have a theoretical basis. 
Approach: Research Design 
This SMART study evaluates cost-effectiveness of a stepwise approach to sustainment of 
ROSE in 98 outpatient prenatal clinics serving women on public assistance. Rigor and 
reproducibility are ensured by the randomized trial design, clear inclusion criteria for 
participating clinics, manualized protocols and fidelity assessment for EIAU, LICF, and HICF, 
careful characterization of implementation processes, reliable and valid measures, and 
transparent power and statistical analyses with attention to potential biases and missing data.   
E1. Critical Design Decisions   
  
E1a. Rationale for design/Why a SMART design? The goal of the study is to determine 
the minimum intervention needed to sustain ROSE in clinics that provide prenatal services to 
women on public assistance, and the optimal timing of boosters. To achieve this goal, the 
implementation interventions (EIAU, LICF, HICF) were chosen to reflect different intensities of 
a relatively standard approach to allow their intensity/cost aspects to take precedence.  When 
an intervention does not produce a desired outcome, two options are available: to give it more 
time or step up the intervention intensity. It is possible that higher intensity would lead to better 
outcomes. However, it is also possible that higher intensity would prove to be too much to 
implement for the clinic staff or have a cost that is too high. The design of this study allows us 
to rigorously isolate the effect of intensifying an intervention (i.e., stepping up to LICF or HICF) 
versus giving a simpler one (such as EIAU or LICF) more time. The proposed SMART study 
will build an evidence base for precision algorithms for achieving sustainment and optimizing 
the allocation of implementation resources to do so.   
E1b. There are different ways to think about timing in terms of sustainment.3  In this study, 
withdrawal  
of external implementation/sustainment supports occurs at Month 1 for EIAU clinics (leaving 
30 months of remaining assessments) and at Month 18 for LICF and HICF clinics (leaving 12 
months of remaining assessments). In addition, LICF and HICF clinics will not receive post-
initial training sustainment intervention until they need it (at 3-15 months post-baseline), 
providing additional un-intervened assessment periods to determine the optimal time for 
booster sessions for different kinds of clinics.    
E1c. Rationale for sample. Clinics will be 98 outpatient medical clinics providing prenatal care. 
We chose to include any kind of outpatient clinic for which ROSE would be appropriate in 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
15 
order to increase generalizability and speed knowledge acquisition relevant to widespread 
scale-up.  To be included in the study, clinics will be (1) outpatient, (2) provide prenatal 
services, (3) estimate that at least 50% of their pregnant patients receive some kind of public 
assistance (such as federal or state assistance in the form of cash assistance such as 
Temporary Assistance for Needy Families [TANF], food stamps, subsidized housing, and/or 
health care such as Medicaid; median in our pilot survey was 75%), (4) have at least 10 new 
pregnant women per month on average (i.e., enough patient flow to run ROSE), and (5) agree 
to study procedures.  Clinics may include FQHCs, hospital-affiliated clinics, health-system 
affiliated clinics, family practices that provide prenatal care, and/or free-standing clinics, 
visiting nurses (e.g., MI’s MIHP program76) and/or any other kind of clinic or program that 
provides outpatient prenatal services. Given the particular need for prevention of PPD among 
low-income women (higher rates, more severe consequences, less likely to receive PPD 
treatment if needed), we target clinic-wide ROSE implementation to clinics serving mainly low 
income women; however, any woman within the clinic can receive ROSE. We have included 
letters of support from 140+ clinics or agencies covering clinics serving low-income women 
from 6 U.S. states (MI, NY, RI, PA, MA, FL) that have already agreed to participate. We have 
additional clinics we can reach out to if needed. We will document characteristics of enrolled 
clinics. Having study clinics in 6 states allows us to capture a range of health policy and other 
contexts and to examine these as potential predictors/tailoring variables. We will examine 
effects of ecological context on ROSE sustainment using mixed methods. In SMART 
analyses, clinic locations and characteristics will be explored as potential tailoring factors to 
determine optimal sustainment strategies and timing.    
E2. Design  
E2.1. Implementation interventions. Except for one in-person training in HICF, meetings and 
trainings take place by videoconference or telephone (both PIs have successfully provided 
training and supervision this way before). Clinics that don’t have a webcam will be sent one; 
trainings and meetings for each clinic will be recorded and provided to that clinic for optional 
later viewing. Pilot work found that clinics’ views of the role of the most important person to 
facilitate implementation varied, therefore implementation interventions include individuals 
from many clinic roles.    
EIAU consists of initial training and problem-solving plus planning for sustainment, and covers 
the Pre-Implementation steps shown in the REP framework above. Step 1: in a two-hour 
collaborative process the joint PIs will meet with key clinical and operational staff. This 
meeting will include: (1) a brief clinical and operational overview of ROSE; (2) problem-solving 
and discussion around adaptable elements of ROSE, and (3) planning and tools for 
sustainment.  Sustainment planning will be based on the outline provided in the U.K. National 
Health Service’s Sustainability Model and Guide,81 which discusses process, staff, and 
organization needs and engagement in sustainment.  Sustainment planning will include 
identification of leaders and responsibilities for ROSE delivery and sustainment within the 
clinic, and of clinic staff who can deliver ROSE within the clinic’s work flow and billing 
structure. This collaborative process will mesh the clinic’s context, needs (including needs of 
patient population), and resources, with discussion of ROSE core and adaptable elements, 
resulting in a written, tailored implementation plan.  Step 2 will consist of two separate video 
meetings: 60-90 minutes with Dr. Johnson and identified operational staff to discuss 
administrative issues (such as reimbursement, identification and referral procedures, 
identification of suitable providers). Following the format and content of her prior ROSE 
trainings, Dr. Zlotnick will provide a live four-hour training to providers on how to conduct 
ROSE via videoconference.  At the conclusion of the training, each provider will be given a 
manual that will include the highly scripted ROSE Program with patient handouts, a list of 
frequently asked questions and answers, a summary of key components for each session, a 
copy of the PowerPoint training slides, scripts for presenting ROSE to potential participants, a 
script for follow-up calls, and a customized description of the clinic’s logistics (e.g., identifying 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
16 
available rooms in the clinic, identifying and referring participants for ROSE within the clinic, 
medical record templates for ROSE sessions, billing codes). Partnership interventions in EIAU 
will consist of facilitators taking a collaborative, discussion-oriented stance toward all 
interactions and having a general philosophy of learning from the clinics.  Because training 
sessions are recorded and there is a written, clinic-specific implementation and sustainment 
plan, it will in theory be possible for clinics to replenish staff turnover, but turnover may create 
risk of not sustaining, which would be addressed in LICF or HICF.    
LICF will include EIAU plus low-intensity coaching and feedback, consisting of 3 components.  
First, one clinical and one operational telephone “booster” meeting quarterly (i.e., every 3 
months) for additional support (up to 1 hour each). Per the REP model, clinical and operational 
meetings will both have the option to discuss re-customization of delivery as the need arises.  
In the clinical telephone meetings, Dr. Zlotnick and ROSE interventionists will identify 
challenges (e.g., competing demands on time, nonattendance of patients) and successes in 
implementing ROSE and conducting ROSE with fidelity (e.g. insufficient time to present core 
components, introducing material not consistent with ROSE). They will collaboratively problem 
solve solutions and develop an action plan to address barriers. At subsequent meetings, they 
will review implementation progress and review ways in which the action plan has been 
successful, failed, or unused.  Changes to the action plan based on new data, experiences, 
and discussion will be made collaboratively in the meeting.  In the operational support 
meeting, Dr. Johnson will work with administrative staff to identify and smooth organizational 
and financial changes needed to sustain the intervention, identify and reduce the clinic’s 
specific barriers and reinforce assets to program adoption, discuss ways to capitalize on or 
reduce the negative effects of changes to context that could affect implementation (e.g., new 
health care legislation). A collaborative solution plan will be developed and at subsequent 
meetings reviewed and modified accordingly. Second, we will provide feedback to clinical and 
operational staff in the booster meetings to help guide discussion and planning. Clinical 
feedback will include information about their fidelity to core ROSE elements based on the 
ROSE core elements fidelity scale and interview validation described in Section E2.6b.  We 
will also provide information to clinical and operational staff on any changes in the clinic’s rates 
of PPD for the last quarter, and on any challenges or successes we detect from survey 
measures.  Any protocol refinements changes will be distributed and discussed during the 
“booster” sessions.  Third, to promote partnership and  ownership, clinical and operational 
staff will be invited to participate quarterly in “Collaborative Board” phone meetings with the 
study PIs, the clinic-related study Co-Is, and staff from other study clinics to provide feedback 
to the study team about the implementation strategies being used, helpful adaptations to the 
intervention that preserve core elements, the study itself, and to provide problem-
solving/suggestions to challenges that have come up in other clinics.  A few staff members 
from non-study clinics who have previously implemented ROSE will also attend to provide 
experienced clinic perspectives. Therefore LICF components cover the REP framework tasks 
of technical assistance, ongoing support of and partnership with implementing organizations, 
booster training, process evaluation71 (i.e., through feedback from study assessments to 
clinics and from clinics to the study team), organizational and financial changes to sustain 
intervention, and re-customization of delivery as the need arises.    
  
Given that examining intensity and cost is the main goal of the study, clinics in the 
HICF condition will receive everything that the clinics in LICF receive, but at a higher intensity.  
Clinical and operational booster meetings (with feedback), and participation in “Collaborative 
Board” meetings will be monthly, rather than quarterly.  [The Collaborative Board which 
includes the PIs and clinic-related Co-Is will meet monthly regardless, so it is just a matter of 
which clinics will be attending each month.]  In the month after randomization to HICF, Drs. 
Johnson and Zlotnick will travel to the clinic to provide an in-person clinical and administrative 
“booster” meeting (2 hours each; an estimated 19 trips over the grant period).  It is our 
experience than an in-person meeting, though costly, increases engagement (a proposed 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
17 
mechanism). Part of the test of HICF will be to see whether and for whom this extra expense 
is warranted. The remaining monthly meetings will be by telephone or videoconference. Drs. 
Zlotnick and Johnson will also be available to answer questions on an ad-hoc basis via email 
or phone.    
E2.2. Characterizing implementation interventions (EIAU, LICF, HICF).  Every implementation 
encounter (e.g., Collaborative Board meetings) will be documented in an electronic 
implementation case note and audio or video recorded. The case note will include: encounter 
length, time spent on operational vs. clinical support, a checklist of implementation strategies 
used (taken from Powell et al., 2015)82 as recommended in recent guidelines for specifying 
and reporting implementation strategies,83 a checklist discussion topics (e.g., billing options), 
and free response sections to describe clinic staff’s responses and anything not covered by 
the checklists.  We will assess the frequency with which each of the Powell strategies occur.  
We will format the case note to account for the fact that some strategies will occur within the 
calls (e.g., provide clinical supervision, provide feedback) and some will be higher level (e.g., 
the calls themselves—facilitation). This will allow us to characterize which sites received how 
much of which kind of support and which time, in order to examine (1) fidelity to the 
implementation framework, (2) the association of these variables with sustainment outcomes, 
(3) clinic participation or non-participation in offered supports (i.e., % of coaching sessions 
attended). 20% of the recordings will be rated by members of the study team in order to 
verify/augment the notes (i.e., calculate reliability of the notes by computing intraclass 
correlations or kappas between notes and tapes) to better characterize implementation 
strategies.  Recordings are also available for review if needed to get a more fine-grained 
description for dose-response analyses.    
E2.3. Randomization.  After the baseline assessment, all clinics will receive EIAU (initial 
training +  
tools for sustainment).  Clinics that are determined to be at-risk for operational (defined as no 
ROSE intervention in 3 months and none planned) and/or clinical (defined as low fidelity to 
ROSE core elements; failure to sustain at subsequent assessments up to 15 months will be 
randomized to receive additional support. At the first time period at which a clinic is 
determined to be at risk (i.e., at 3, 6, 9, 12, or 15 months), that clinic will be randomized in a 
3.8:1 ratio to receive either: (1) the addition of low-intensity (every 3 months) coaching and 
feedback (LICF), or (2) no additional implementation support (EIAU only). If clinics receiving 
LICF are still found to be at risk at subsequent monitoring periods, they will be randomized in a 
1:1 ratio to either (1) nothing additional (i.e., EIAU + LICF only), or (2) high-intensity (monthly) 
coaching and feedback (HICF).  Additional study follow-up interviews will occur at 18, 24, and 
30 months, but no implementation intervention will occur after 18 months. EIAU only clinics 
are followed for 30 months without post-baseline intervention.  To control for the timing of 
randomization, initial and the subsequent randomization procedures will balance trial arms by 
time (3, 6, 9, 12, or 15 months) and whether or not the clinic is a FQHC; yes or no; FQHCs 
have their own billing structure and mandate), so that these factors will have the same 
distribution in randomized groups.   
E2.4. Power. Power is based on the primary outcome, measured as percent of sustainment of 
core program elements at each time point. We start with powering the comparison created by 
the second randomization, EIAU+LICF vs. EIAU+LICF+HICF. For this comparison, the 
literature reports a range of effect sizes, with 3 out of 4 most relevant ones84,85 exceeding 
Cohen’s d=0.48.72 Aim 1 analyses include adjustment for the 6-month version on the primary 
outcome (baseline for second randomization), and repeated measures at 9, 12, 15, 18, 24, 
and 30 months. If a clinic is randomized at a later than 6 month time point, 6-month version of 
the primary outcome will still be used as a covariate for consistency in time scale, repeated 
measures prior to randomization will be set to missing, but all other available data will be used 
in the mixed effects model. Assuming correlations between pairs of repeated measures of 0.7 
based on past work, the necessary sample size to detect the target effect size with power of 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
18 
RUJUHDWHUDWĮ LQWZR-tailed tests is n=19 per group. Assuming that 2/3 of the 
clinics would still be at risk after EIAU+LICF, n=38 entered into the second randomization are 
2/3 of the clinics determined to be at risk. Therefore 1/3, or n=19 clinics would be deemed low 
risk and continue EIAU+LICF.  Moving to the left in Figure 1 to the first randomization, the 
EIAU+LICF group will have size n=57 as determined above. The size of the other group in the 
first randomization of n=15 will allow to detect the target effect size with power of 0.89, which 
is more than sufficient because the effect size for LICF+EIAU (intervention) vs. EIAU alone 
(standard care) would be larger than HICF vs. LICF (comparison of two interventions), and 
power would be even greater to detect the larger effect size. Finally, based on relevant 
reviews,72,86 after the initial EIAU period, we expect approximately 80% of clinics to be at 
risk. Therefore, 72 (57+15) clinics in the first randomization will be 80% of the sample. Thus 
the initial sample size receiving EIAU will be n=90.  Because the analysis technique of mixed 
effect modelling allows to include all clinics with at least one repeated measure post-baseline, 
this sample size will ensure sufficient power for the intent-to-treat (ITT) analysis for the primary 
aim.  Because comparisons are EIAU vs. LICF and LICF vs. HICF, clinics who complete 
baseline measures but drop out of implementation intervention before receiving EIAU will not 
be included in main ITT analyses. We will, however, characterize them, follow them over time 
(i.e., they receive all study assessments), and evaluate them in separate analyses.  
Therefore, 90 clinics will receive EIAU, but 95 or so may receive baseline measures and be 
followed over time.  
We already have letters of support from individual clinics or agencies covering more than 140 
clinics.    
What if more clinics than expected are able to sustain with just EIAU? The sample size 
determination used the most conservative estimates of the effect sizes (smallest) and 
proportions of clinics that will be determined at risk (lowest) at the second randomization (LICF 
vs. HICF), which drives power analyses. If more clinics are entered into randomization or the 
effect sizes are larger than anticipated, power will be greater.  If more clinics than expected 
sustain successfully with EIAU (initial training/planning) only and fewer than expected are 
entered into randomizations, then we will have found that ROSE is sustainable with lower 
cost/effort than most psychosocial interventions,72 which is a contribution to clinical and 
implementation science knowledge. This would also allow us to consider more factors 
associated with such sustainment and processes leading to it. Either way, we will be able to 
characterize clinics that are more and less likely to require only EIAU and develop a decision 
model to triage sites into EIAU or more intensive support.    
E2.5. Clinic Retention. We will make every effort to collect all assessment data from all clinics 
who begin the study, regardless of their level of participation in EIAU, LICF, or HICF. We will 
work to develop positive relationships with respondents, to value and appreciate their time, 
and keep in close touch with them for the duration of the study. Study staff will call to remind 
respondents of the survey and to offer to help navigate it if needed. We can also administer 
the survey by phone, if helpful. Study staff will provide assistance with other aspects of study 
data collection (e.g., cost, PPD rates) if needed, in order to reduce perceived and actual clinic 
burden.  We will reimburse each respondent (2 per clinic per time) $100 for completing the 
surveys and qualitative interview (if chosen for a qualitative interview at that assessment), plus 
an extra $40 for the person reporting the aggregated ROSE participation data. These 
strategies have been used with success in Dr. Johnson’s (R01 MH095230; K23 DA021159) 
and Stirman’s current and previous studies87,88 which have enrolled multiple clinics and 
surveyed/interviewed providers with good follow-up rates. A few clinics may decline additional 
implementation support at some point. If this occurs, we will document why.    
E2.6. Assessments will be conducted at 0, 3, 6, 9, 12, 15, 18, 24, and 30 months.  Each clinic 
will identify two people: the person most involved in ROSE (1) clinical, and (2) operational 
(e.g., billing, scheduling) functions. We anticipate two respondents per clinic, but power and 
measures are unaffected if they are the same person. Clinic-level measures will be derived 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
19 
from the surveys and analyses occur at the clinic level. Each respondent will complete 
quantitative measures online (with a mailed paper option for respondents who prefer it). 
Qualitative interviews (conducted by the postdoc) will take place by phone.   
E2.6a. We will collect clinic descriptive information including the kind of clinic (FQHC, etc), 
size and age of clinic, # patients seen per year, # pregnant and low income pregnant patients 
seen per year, what kind of providers on staff and which are delivering ROSE, staff turnover at 
each time point, etc.  We will also use descriptive survey sections from the National Survey of 
Physician Organizations89 to characterize clinics.   
E2.6b. We define operational failure to sustain as no ROSE intervention in 3 months and none 
planned.  We define clinical failure to sustain as less than adequate fidelity to ROSE core 
elements (i.e., an average of < 75% of core elements for each session delivered, as measured 
by the ROSE Session-by-Session Adherence Scale). This checklist lists on average 8 items 
(rating of present/absent) that assess whether key tasks of each session were completed. 
75% may seem like a high mark, but given that ROSE is so scripted, item answers are in 
yes/no format, and core elements are fairly basic (e.g., y/n - did interventionist explain PPD?  
Did interventionist have group members practice communication skills through role plays?), 
completing 75% of these for “adequate” fidelity seemed reasonable.    
E2.6c. Primary outcome. As recommended by the Wiltsey-Stirman review,3 our primary 
outcome is a multifaceted assessment of sustainment, specifically, the effectiveness of each 
sustainment step in terms of (a) sustainment of core program elements, and (b) total length of 
time any ROSE services were provided, and length of time they were provided with at least 
moderate fidelity to core elements. Statistically, one of these measures should be the primary 
outcome; we have chosen the first because it has repeated measures over time, improving 
power. ROSE’s core program elements will be assessed through the ROSE Session-by-
Session Adherence Scale, a self-rated intervention fidelity scale completed by ROSE 
interventionists after each session.57 The outcome for each time point (i.e., quarter) will be the 
mean % of core elements delivered that should have been delivered at each ROSE session 
(mean [# of core elements delivered/# core elements should have been delivered] at each 
session; zero if no sessions were completed). Self-reported checklists of mental health 
intervention fidelity have shown excellent validity when compared to observer-rated scales. 
Co-I Dr. Wiltsey-Stirman found excellent (k = .90, k = .85, k = .83) rater (clinician vs. observer) 
agreement for a 5-8 item self-rated mental health intervention adherence checklist used in 
public mental health settings. Other recent studies have similar findings, suggesting that self-
rated fidelity scales work well for psychosocial interventions.90-93  In this study, we will also 
validate checklist responses against expert ratings using qualitative interviews for 3 sessions 
per quarter. Interviews will ask about what the interventionist did with for each core element 
and determine the degree to which the interventionist’s idea of the element and the 
interviewer’s idea of the element correspond. Total length of time any ROSE services were 
offered. Using a weekly calendar method94,95 that begins at initial training and ends at the 
30-month interview, we will also track (1) the total length of time any ROSE services were 
provided (recognizing that clinics may stop and restart the program, so times the program was 
provided may not be contiguous); and (2) total amount of time ROSE services were offered 
with adequate fidelity of core elements (defined as 75% or more on the ROSE Session by-
Session Adherence Scale averaged across interventionists and sessions each month).  We 
will also assess the length of time from training to offering the program, and from training to 
offering it with adequate fidelity.  
E2.6d. Secondary outcome data will be collected quarterly, when we collect the surveys.  
Health impact (PPD rates over time at each clinic). Quarterly, we will ask each clinic to report 
the following overall numbers: (1) # of women who should have come for their 6-week 
postpartum appointment; (2) # who came; (3) # that were screened for PPD; (4) # who 
screened positive for PPD.  We will also collect this information for 4 quarters (12 months) 
prior to baseline. We will use these numbers to calculate PPD rates for each time period. 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
20 
Although though not every woman in the clinic will receive ROSE, we chose to track overall 
PPD rates at each clinic because: (1) the study examines larger-scale sustainment aimed at 
clinic-wide (and eventually population-wide) outcomes; and (2) the clinic-level outcomes are 
primary; not consenting individual patients makes the needed sample size (90 clinics) feasible. 
Number of patients enrolled in and completing ROSE (i.e., reach).1  Clinics will track the 
number of: (1) patients who agree to come to ROSE, (2) patients attending at least 1 session, 
and (3) patients attending at least 3 of the 5 sessions.  Clinic size (# of new pregnant patients 
per quarter) and estimated % patients on public assistance will be considered in analyses.    
Return on Investment. The economic analysis proposed here complies with NIH Notice NOT-
OD-16025 guidelines on appropriate aims of economic analyses in NIH grants. We will 
analyze 4 cost-effectiveness (CE) outcome measures: (1) a primary clinical outcome, number 
of PPD cases averted, estimated as the change in PPD rate at the clinic (post-pre)*(clinic’s 
caseload), (2) another clinical outcome, number of quality adjusted life years (QALYs) saved, 
computed from the primary outcome using Morrell et al.’s88 model, (3) an implementation 
process outcome, number of clients served with fidelity, and (4) a sustainment outcome, 
months of additional service delivery. The latter 2 measures have little measurement error.  
Principles of CE analysis dictate that all costs must be taken into account.96-98  Our grant 
accounting will capture our costs to provide EIAU, LICF, and HICF. We will separately track 
costs for EIAU, LICF, and HICF using a micro-costing approach and a detailed chart of 
accounts. We will exclude our research costs. Clinic costs to receive EIAU, LICF, and HICF 
will be assessed using hours that clinic staff spent on EIAU, LICF, or HICF plus associated 
direct costs (e.g., printing), as well as information from the clinics on staff salaries per hour, 
fringe benefits, and overheads. Clinic costs to deliver ROSE. The only extant cost estimate for 
ROSE45 ignored attrition, no-shows, and reschedules. Therefore, we will track ROSE delivery 
costs at each clinic for one pay period (using a time sheet for staff who spend time on ROSE 
to record their ROSE-related hours, work hours on other programs, residual personal 
overhead hours, and training time). For the pay period, clinics also will track other direct costs 
or resource use such as printing. We will randomize pay periods assigned to individual clinics 
over a 1-year period so that we can provide quality control and supportive technical assistance 
to each and to avoid any bias if delivery costs vary by season. Finally, we will collect clinic 
supervisory, in-service training, and overhead costs and allocate a fair share to ROSE. We will 
analyze the distribution of costs per client served at the clinic level and experiment with 
excluding outlier clinics that never fully launched.           Proposed mechanism #1: Clinical & 
organizational capacity to deliver ROSE. The primary measure will be the Organizational 
Capacity subscale of the Program Assessment Sustainability Tool (PSAT), which has good 
reliability and factor structure.99,100  Secondary measures include # people trained who have 
time to deliver ROSE and perceiving the clinic as able to manage space/scheduling and to 
bill/get reimbursed for ROSE.            Proposed mechanism #2: Ownership/engagement by 
clinic staff will be assessed using the sum of other relevant subscales of the PSAT (primary) 
which assess domains including: Communication, Partner-ships, Political Support, and 
Environmental Support (reflecting buy-In across multiple stakeholders, including leaders and 
champions).99,100  The Staff section of the NHS’s Sustainability Model and Guide101,102 
and investment in addressing PPD [assessed using the “Attitudes toward PPD” scale based 
on the “Awareness and Concern” items from Steckler (1992)75 and used in R01 MH 08668)] 
will be secondary measures.   
E2.6e. We will assess Predictors and processes to provide information about which kinds of 
sites need which level of sustainment support. Organizational context will be assessed using 
Aarons’ Implementation Climate Assessment (ICA), a 57-item scale measuring agency focus 
on and support of evidence-based practices.103,104  State policy context will be assessed 
through two measures: (1) Enacted state legislation about PPD (0 = no state enacted state 
legislation related to PPD, 1 = awareness-related PPD legislation, 2 = legislation mandating 
PPD education and services, 3 = legislation with $ attached for PPD education/services). (2) 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
21 
State-level maternal mortality.  Maternal mortality is a key indicator of health for Healthy 
People 201077 and 2020,105 and reflects the ability of women to secure maternal and other 
health services.77,105 We will also document any relevant national policy changes that occur 
during the course of the study.    
Processes of implementation and sustainment efforts will be documented using: Qualitative 
interviews, implementation strategy checklists, and process notes of initial training and 
coaching/feedback calls. The timing of implementation and sustainment efforts with desired 
outcomes will be tracked using the weekly calendar method described in Section E2.6c. We 
will record the dates of EIAU, LICF, and HICF interventions and dates of any change in ROSE 
status (i.e., ROSE offered, not offered, ROSE offered with fidelity vs. not) to examine the 
temporal relationships among these events (e.g., retraining or technical assistance calls one 
week and improvements in fidelity over subsequent weeks). Respondent perceptions of critical 
incidents to sustainment success or failure will be assessed using qualitative interviews.    
E2.7. Analyses. Clinic is the unit of randomization and analysis. Primary analyses will be 
intent-to-treat. Aim 6 will explore the relationship between dose and processes of each 
intervention and outcomes. All statistical tests will be two-sided and perforPHGDWĮ IRU
sustainment of core program elements (primary) and health impact (secondary) outcomes, 
specified a priori. In the exploratory analyses, false discovery rate due to multiple tests will be 
controlled using the Hochberg adjustment.106,107        
9. 
Minimal Risk Research. Complete the following question if you selected an expedited 
category in Question 7A. 
9A. Briefly describe the background for conducting the research. (1-2 sentences) 
To test the effectiveness and cost-effectiveness of a stepped support approach to sustaining 
ROSE, an evidence-based group intervention that prevents postpartum depression (PPD) in 
low-income pregnant women across 98 outpatient prenatal clinics in the U.S. states. Using a 
Sequential Multiple Assignment Randomized Trial (SMART) design, the study will assess how 
different levels of coaching support affect long-term implementation, health outcomes, and 
return on investment, addressing a key gap in implementation science. 
9B. Sample Size. 
9Bi. Provide an estimated sample size for the lifetime of the study at this investigator’s sites. 
 98 
9Bii. Describe the basis for that estimate. 
We will survey two provider respondents each from 98 clinics (for roughly 196 participants). 
Analyses are at the clinic level. Power is based on the primary outcome, measured as percent 
of sustainment of core program elements at each time point. We start with powering the 
comparison created by the second randomization, EIAU+LICF vs. EIAU+LICF+HICF. For this 
comparison, the literature reports a range of effect sizes, with 3 out of 4 most relevant ones 
84,85 exceeding Cohen’s d=0.48.72 Aim 1 analyses include adjustment for the 6-month 
version on the primary outcome (baseline for second randomization), and repeated measures 
at 9, 12, 15, 18, 24, and 30 months. If a clinic is randomized at a later than 6 month time point, 
6-month version of the primary outcome will still be used as a covariate for consistency in time 
scale, repeated measures prior to randomization will be set to missing, but all other available 
data will be used in the mixed effects model. Assuming correlations between pairs of repeated 
measures of 0.7 based on past work, the necessary sample size to detect the target effect 
size with power of 0.80 or greater at &#61537;&#61472;= 0.05 in two-tailed tests is n=19 per 
group. Assuming that 2/3 of the clinics would still be at risk after EIAU+LICF, n=38 entered 
into the second randomization are 2/3 of the clinics determined to be at risk. Therefore 1/3, or 
n=19 clinics would be deemed low risk and continue EIAU+LICF. Moving to the left in Figure 1 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
22 
to the first randomization, the EIAU+LICF group will have size n=57 as determined above. The 
size of the other group in the first randomization of n=15 will allow to detect the target effect 
size with power of 0.89, which is more than sufficient because the effect size for LICF+EIAU 
(intervention) vs. EIAU alone (standard care) would be larger than HICF vs. LICF (comparison 
of two interventions), and power would be even greater to detect the larger effect size. Finally, 
based on relevant reviews,72,86 after the initial EIAU period, we expect approximately 80% of 
clinics to be at risk. Therefore, 72 (57+15) clinics in the first randomization will be 80% of the 
sample. Thus the initial sample size receiving EIAU will be n=98. Because the analysis 
technique of mixed effect modelling allows to include all clinics with at least one repeated 
measure post-baseline, this sample size will ensure sufficient power for the intent-to-treat (ITT) 
analysis for the primary aim. Because comparisons are EIAU vs. LICF and LICF vs. HICF, 
clinics who complete baseline measures but drop out of implementation intervention before 
receiving EIAU will not be included in main ITT analyses. We will, however, characterize them, 
follow them over time (i.e., they receive all study assessments), and evaluate them in separate 
analyses. Therefore, 98 clinics will receive EIAU, but 100 or so may receive baseline 
measures and be followed over time  
10. 
Benefits.  
Describe any potential direct benefit(s) to subjects in this study, if any and the importance of 
the knowledge that may reasonably be expected to result. Within the description, do not 
include payment to subjects as a benefit. 
The study is specifically designed to examine the effectiveness and cost-effectiveness of a 
stepwise approach to sustainment of an empirically based intervention to prevent postpartum 
depression (ROSE) in 98 outpatient prenatal clinics serving women on public assistance. 
Participating clinics will receive free training in ROSE, free ROSE manuals and supportive 
materials, and free implementation/support in the form of Enhanced Implementation as Usual 
(EIAU), Low-Intensity Coaching and Feedback (LICF), and/or High-Intensity Coaching and 
Feedback (HICF). Clinic patients may have the opportunity to receive an evidence-based 
postpartum depression prevention intervention (ROSE) that will be available at the clinic as a 
result of the study. There is also the more general benefit to the public in learning whether and 
how the implementation of an effective intervention to reduce postpartum depression can be 
sustained in community settings. Untreated postpartum depression has severe and lasting 
consequences for mother and child, including maternal increased risk for suicide, 
compromised functional status, and adverse infant developmental outcomes. Consequently, 
the potential risks of participation in the study appear to be outweighed by the potential benefit 
to clinics’ patients who receive the ROSE intervention. 
11. 
Screening, Recruitment, and Determining Eligibility. 
11A. Describe how subjects will be identified and recruited, including who will perform the 
recruitment.  
We already have letters of support from agencies covering prenatal clinics in 6 states. Study 
investigators approached the clinics about potential participation. We will also recruit more 
outpatient perinatal clinics, along with WIC, Healthy Start, and Doula agencies across the 
country. We will mail invitational letters, call offices as a recruitment effort, and provide 
presentations. Each agency will nominate potential respondents to complete study surveys 
and interviews. To prevent these individuals from feeling pressure to participate, the following 
measures will be taken: Respondents will only be recruited directly by the research team, from 
agencies or clinics that are willing to allow their clinicians to participate in all study procedures. 
It will be made clear that data that is gathered during the study will not be shared with the 
administration of agencies (other than clinic-level feedback, such as quarterly postpartum 
depression rates at the clinic, clearly specified in advance in the consent form). Staff who 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
23 
participate in the study will receive a thorough description of study-related procedures, will 
have the opportunity to ask questions, and will provide informed consent. 
11B. The study team will obtain for the purpose of screening, recruiting, or 
determining the eligibility of prospective subjects (please select the 
appropriate option): 
 Not Applicable 
 
 Information through oral or written communication with the prospective subject or legally 
authorized representative. Before the information is obtained for the purpose of screening, 
recruiting, or determining eligibility, consent: 
 will be obtained. 
 will not be obtained. 
Please describe screening consent procedures in Question 12. 
 Identifiable private information or identifiable biospecimens by accessing records or stored 
identifiable biospecimens. Before the information is obtained for the purpose of screening, 
recruiting, or determining eligibility, consent: 
 will be obtained. 
 will not be obtained. 
Please describe screening consent procedures in Question 12. 
Note: The revised Common Rule permits an exception from informed consent for screening, 
recruiting, or determining eligibility when certain criteria are met; this exception does not 
apply to studies subject to the Pre-2018 Common Rule Requirements and/or studies 
regulated by the U.S. Food and Drug Administration (FDA). 
11B1. Please explain your selection(s). 
 The study team will speak with agency leadership to determine if the agency meets study 
criteria prior to consenting agency representatives to the study.  
12. 
Consent Process. 
12A. If the study involves adults, consent will be obtained from (select 
appropriate option(s)): 
 Not Applicable 
 All subjects  
 Some subjects  
 No subjects (consent will not be obtained) 
12B. If the study involves children, parental permission will be obtained from 
(select appropriate option(s)):   
 Not Applicable 
 Both parents or guardians (unless one parent is deceased, unknown, incompetent, or not 
reasonably available, or when only one parent has legal responsibility for the care and 
custody of the child) 
 One parent or guardian 
 Will not be obtained 
12C. If the study involves children, child assent will be obtained from (select 
appropriate option):  
 Not Applicable 
 All children 
 Some children 
 Will not be obtained 
12D. Describe the consent process, including an explanation of your selection(s) above. If the study 
involves screening activities, please describe whether consent will be obtained and if consent 
will not be obtained, explain how the screening data will be used. If only some subjects will 
provide consent, explain who will or will not provide consent. If only some children will provide 
assent, explain which children will and will not provide assent. 
There will be 2 kinds of consents for participating clinics: 
(1) Clinical and organizational representatives who will respond to study surveys and 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
24 
qualitative interviews. 
(2) ROSE providers who will provide self-reported fidelity ratings of ROSE. 
To prevent individuals from feeling pressure to participate, the following measures will be 
taken: Respondents will only be recruited directly by the research team, from agencies or 
clinics that are willing to allow their clinicians to participate in all study procedures. It will be 
made clear that data that is gathered during the study will not be shared with the 
administration of agencies (other than clinic-level feedback, such as quarterly PPD rates at the 
clinic, clearly specified in advance in the consent form). Staff who participate in the study will 
receive a thorough description of study-related procedures, will have the opportunity to ask 
questions, and will provide informed consent. 
Some people may be in multiple categories above and will be asked to consent to each aspect 
of the study that is relevant to them. We will consent agency directors first. Given that 
participating clinics will be from many different U.S. states, consent will take place 
electronically, with the study also described by telephone or at staff meetings when feasible. 
Each person will be emailed a link to a confidential study electronic consent form explaining 
study participation, risks and benefits, etc. and be given an opportunity to enroll or decline. 
The consent form will outline the nature of the study, the procedures they will be asked to 
follow, time commitment, how to withdraw if they choose to do so, compensation, who to 
contact with questions or concerns during the study. The consent form will be clear that the 
clinic will not be told whether or not they choose to participate and that agency directors have 
agreed that participation or non-participation will not be shared and will not affect their 
employment in any way. They will be told that the agency administrative leaders have affirmed 
that a decision to withdraw participation at any phase of the study will not be a part of their 
employment record, and will not impact their employment or their eligibility for other training 
opportunities, promotions, etc. They will be given time to decide whether to participate (at least 
a day, more often a week or more) before study procedures begin. Study staff will also be 
available by phone to answer questions or explain aspects of consent. 
12E. If consent will not be obtained, explain why. Describe why the research 
could not be practicably carried out if consent was required. If the research 
involves identifiable private information or identifiable biospecimens, 
describe why the research could not practicably be carried out without 
using such information or biospecimens in an identifiable format.  
 Not Applicable 
 We will collect participant demographic information in accordance with NIMH and those, 
consent is required.   
12F. If your study involves use of a consent form, complete i, ii and iii.  
 Not Applicable 
12Fi. Select the appropriate option(s) below for the documentation of consent.  
 Will use a written consent document signed by subjects 
 Will use a short form written consent document signed by subjects 
 Will not obtain a signed consent document for some subjects 
 Will not obtain a signed consent document for all subjects 
12Fii. Describe when and how the subject will receive a copy of the consent form. 
 Participants will receive a copy of their consent form electronically through the confidential 
study electronic consent form system.  
12Fiii. If subjects will not be signing the consent document, please explain why. If 
some subjects will not sign the consent document, explain who will and will 
not sign the consent. 
 Not Applicable 
       

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
25 
12G. If the study involves cognitively impaired adults, explain the process to 
determine whether a subject is capable of consent, use of any legally 
authorized representative(s), and any assent process.  
 Not Applicable 
       
13. 
Coercion or Undue Influence.  
13A. 
 
If some or all of the subjects are likely to be vulnerable to coercion or 
undue influence, such as children, prisoners, pregnant women, mentally 
disabled persons, individuals with impaired decision-making capacity, or 
economically or educationally disadvantaged persons, describe additional 
safeguards that have been included in the study.  
 Not Applicable 
       
13B. If you or your study team are associated with the subjects (e.g. your 
students, employees, colleagues, patients), explain the nature of any 
association and measures taken to protect subjects’ rights, including 
safeguards against any coercion or undue influence (e.g. pressure a 
subject might feel to participate based on the association).  
 Not Applicable 
 The Brown University PI (Caron Zlotnick) will help to train clinic staff in the ROSE intervention 
and provide them with technical assistance. 
Co-I’s Dr. Ellen Poleshuck (University of Rochester), Dr. Tiffany Moore Simas 
(UMMS/UMMHC Memorial), and Dr. Rebecca Weinberg (West Penn Hospital) will serve as 
liaisons to participating clinics in their respective states. 
Dr. Ted Miller (Co-I from PIRE) will perform cost-effectiveness analyses. 
Dr. Shannon Wiltsey-Stirman (Co-I from Stanford) will provide input into implementation and 
sustainment procedures. 
14. 
Privacy.  
 
How will subjects’ privacy be protected? Consider the number of individuals interacting with 
the subject or subject’s records, location of consent process and study, presence of individuals 
not associated with the study, sensitivity of the research.  
Confidentiality. Materials will be identified by an assigned provider ID number (which will not 
involve personal identifying features such as date of birth, etc.). No individual-level data will be 
shared with employers, and any published reports or presentations will only include data that 
is aggregated in a manner that does not allow the identities of providers, administrators, or 
agencies. Feedback about individual provider performance will be shared with that provider 
only, or with the provider’s written permission, in a group. Feedback about administrative or 
operational challenges will be given to the operational/administrative study respondent. 
Feedback about clinic as a whole performance (e.g., PPD rates, etc.) will be provided only to 
staff from that clinic. No individual-level data will be shared with employers, and any published 
reports or presentations will only include data that is aggregated in a manner that does not 
allow the identities of providers or agencies. 
Confidentiality will be scrupulously maintained by standard procedures employed in clinical 
trials, including the use of participant numbers/codes instead of names, the storage of all data 
in locked cabinets or rooms, the use of secure electronic data management systems, and the 
withholding of all participant information from release without the express written consent of 
the individual. Electronic data will be stored on a secure research server at Michigan State 
University and will be available only to authorized personnel. Paper data will be kept in locked 
file cabinets at Michigan State University (MSU) and/or Butler Hospital.  

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
26 
15. 
Withdrawal of Subjects.  
 Not Applicable 
If there are any anticipated circumstances where the researcher will 
withdraw subjects from the study regardless of the subject’s wishes, 
describe the circumstances and the procedures when subjects are 
withdrawn from the study. 
 
       
16. 
Monitoring Plan to Assess Data to Ensure Safety of Subjects. 
 
16A. Is there a monitoring plan to periodically assess the data to ensure the 
safety of subjects or to ensure negative outcomes do not occur? 
 No 
 Yes 
Explain your answer. If you answered Yes, describe the monitoring plan. 
Monitoring Procedures  
The study investigators will be responsible for implementing and maintaining quality assurance 
and quality control systems for this study and are committed to following the guidelines for 
monitoring clinical trials. Written standard operating procedures will be used to guide the 
research visits. The PIs and research staff at their institutions will meet weekly to review study 
progress, including recruitment, consenting procedures, adverse effects on clinics or 
unanticipated problems.  The protocol and subject consent forms will be reviewed and 
approved by the Michigan State University Biomedical IRB (FWA #00004556) prior to the 
recruitment of subjects. Informed written consent will be obtained from all participants prior to 
the start of the study.  
 
Unanticipated Problems 
All unanticipated problems related to participation in the study will be reported to the Principal 
Investigators or delegated research staff for the duration of the study. The Principal 
Investigators will have the front-line responsibility for identifying potential unanticipated 
problems experienced by study participants, adjusting the study procedures accordingly, and 
reporting the experience. Because there are no patient participants in the study (only provider 
participants and aggregated, clinic-level clinic data, such as overall PPD rates at the clinic), 
the study will not monitor and report “adverse events” as typically defined (i.e., patient death, 
hospitalization, etc.).  In fact, we can’t monitor or report these events because we do not track 
individual patients.  Clinics will manage clinical risk of their own patients in accordance with 
their usual policies and procedures.  However, the PIs will ensure that information on any 
adverse effects on clinics or clinic-level PPD rates are systematically collected and evaluated.  
Reports for various categories of unanticipated problems will be submitted to the respective 
IRBs and NIMH within required timeframe. The Principal Investigators will be responsible for 
tracking these reports and relaying them as required to their IRBs. 
 
Data Monitoring 
 The study will compare HICF to LICF and LICF to EIAU for study outcomes including (1) 
sustainment of core program elements at each time point, and total length of time in weeks 
that: (i) any ROSE services were provided, and (ii) were provided with adequate fidelity to core 
elements (primary); (2) health impact (e.g., PPD rates over time at each clinic) and reach 
(number of patients enrolled in and completing the ROSE program); and (3) return on 
Investment (costs and cost-effectiveness of each sustainment step).  Hypothesized 
mechanisms of the effect of LICF and HICF on sustainment of core program elements include 
sustainment of: (a) clinical and organizational capacity to deliver core elements, and (b) a 
sense of engagement/ownership by key clinic staff.  The study will also explore which clinic 
characteristics (e.g., organizational and state policy contexts) and hypothesized mechanisms 
are associated with best sustainment to determine tailoring variables for choosing/sequencing 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
27 
EIAU, LICF, HICF in the future. Finally, we will document implementation and sustainment 
effort processes, their timing relative to desired outcomes, critical incidents to explore factors 
most related to sustainment after accounting for hypothesized mechanisms.  The alpha level 
will be set at 5%. 
 
Data management services will be provided by Michigan State University’s (MSU’s) 
Bioinformatics Research Core (BRIC), part of MSU’s Clinical and Translational Sciences 
Institute (CTSI), BRIC provides post award support to investigators for a variety of data 
capture, data security, and study management services.  BRIC provides strong support for 
observational studies for both internal and external investigators, working closely with data 
analysis collaborators and the investigator’s team members. BRIC provides best-in-class 
informatics support of the operation and administration of the MSU-CTSI.  This includes 
research information databases and web sites to provide information and to foster 
communication among MSU clinical and translational investigators.  BRIC provides a risk-
based security management system focused on the confidentiality, availability and integrity of 
data. It has an active program to insure compliance with HIPAA and other federal and state 
laws and regulations as they pertain to data matters.  BRIC has supported over 150 clinical 
research protocols to date throughout the United States, Africa and in the Caribbean. 
Currently, BRIC serves as the Data Coordinating Center for the largest federally funded (CDC) 
epidemiological study on Autism Spectrum Disorders (now in its third renewal; 200-2016-8983, 
U10 DD000007 and U01DD000498). 
The BRIC facility houses enterprise implementations of major database engines (e.g., Oracle 
Enterprise, MS SQL Server, and MySQL) and employs staff with expertise in constructing, 
managing, and maintaining database solutions for clients. BRIC provides informatics support 
for the Research Electronic Data Capture (REDCap) system of clinical research data 
collection and management. Vanderbilt University (VU), with collaboration from the Clinical & 
Translational Science Award (CTSA) consortium of institutional partners, developed the 
REDCap system for electronic collection and management of clinical research and trial data. 
The REDCap system provides secure, web-based applications that are intuitive, web-based 
interfaces for users to enter data and are flexible enough to be used for various research 
areas.  Because the REDCap system is web-based, users with appropriate permissions can 
access the system from anywhere in the world by using an internet connection.  
 
For this study, most study data will be collected using an online data management 
system.  BRIC will create a REDCap data capture and data storage system for study data. 
The BRIC Developer and Data Manager will work together to create (1) a study database in 
which each clinic is created as a ‘subject’ (each clinic is a record in the database) and email 
addresses are entered for Clinic personnel designated to complete data collection forms (2) a 
schedule for study data collection events for sending email messages and reminder messages 
to drive periodic collection of operations and clinical data from each study clinic, and (3) a 
custom feature to streamline the method by which study personnel will view and manage data 
regarding the sending/reminding/receiving of the emailed survey.  The online system will also 
be used to track study assessments, and enter any paper surveys (from respondents who do 
not want to fill them out online).  Ongoing development will be used, as needed, for 
management and troubleshooting of the email survey system, the email addresses for clinic 
personnel completing electronic surveys, and the custom functionality.  BRIC will also provide: 
3 times daily off-site data backup with 30 days retention, third-party services to monitor 
REDCap availability through the internet, and ongoing troubleshooting and assistance with the 
data collection system.   
Data be identified with the study’s ID of the participant. The codes that link the name of the 
participant and the study ID will be kept confidential by the Principal Investigator using 
established procedures.  Individual survey responses will be inspected by the study research 
assistant to ensure completeness and clarity.  Overall data quality will be monitored 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
28 
approximately monthly by BRIC and/or the study statistician by inspection of study databases 
and data distributions.   
 
The servers used for data entry and analysis are protected by passwords and secure 
logon and data communications procedures to minimize the potential for disclosure of 
research information either inadvertently or as a result of external attack.  Within each 
computer system, only those users authorized to access the data for a given study are able to 
do so.  Backup tapes are stored in a secure location.  Paper research records are stored in 
areas that are locked when staff are not present.   
The study statistician (Co-I: Sikorskii), will analyze the data for most outcomes.  The study 
health economist (Co-I: Miller) and his staff will analyze cost data.  There will be no interim 
analysis.   
 
Educational Training 
All investigators are certified in human subjects education by their respective institutions.  All 
study staff will be certified in human subjects education before having contact with human 
subjects.   
 
16B. If there is a data safety monitoring committee or board, describe the 
composition and frequency of meetings.  
 Not Applicable 
As an NIH-funded clinical trial, the study has a data safety and monitoring plan. 
It does not have or need a data safety and monitoring board. 
17. 
Results and Data Sharing. 
 
17A. Could this research generate any results that could be clinically relevant, including individual 
research results, or general, or aggregate research findings? 
 
 No 
 Yes, clinically relevant individual research results 
 Yes, clinically relevant general or aggregate research findings 
17A1. If yes, explain what clinically relevant research results will be generated, whether they will be 
disclosed to subjects or others (e.g. subject’s primary care physician), and if so, under what 
conditions. Address individual research results and/or general or aggregate research findings, 
as appropriate. This also needs to be explained in the consent document. 
       
17B. For other research results, select all that apply: 
 Not Applicable 
 Overall study results will be shared directly with subjects 
 Individual results or incidental findings of individual subjects will be shared with subjects or 
others 
 Data will be submitted to a repository or database as part of data sharing agreement (e.g. 
genomic data sharing) 
17B1. Explain your selection(s), including how the data or results will be shared and with who (e.g. 
data repository).  
 After data have been collected and study results published, de-identified data will be made 
available to other qualified researchers upon request, on a CD or other electronic means 
compatible with our systems. The request will be evaluated by the PIs to ensure that it meets 
reasonable standards of scientific integrity. We will also place the de-identified dataset, along 
with the data dictionary and documentation of data collected, into the appropriate NIMH Data 
Repository. The published results paper will be emailed to participanting agency leaders and 
consented personnel.  

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
29 
We will use standard measures where possible in order to promote data sharing and 
integration into larger databases and to allow other researchers to analyze the data, including 
conducting meta analyses. We will work on the data dictionary throughout the study. Data 
checking will occur regularly. We will submit primary results for publication by the end of the 
project period, and will have final de-identified datasets and data dictionaries available by CD 
and on the Data Repository within the required time frame.  
18. 
Local Context and Multi-Site Study.  
18A. Describe the locations of where the study team will obtain information or biospecimens 
through intervention or interaction with the subject or obtain the subjects’ private identifiable 
information.  
One of the principal investigators and all of the research support staff are at MSU. The other 
principal investigator is at Brown University. The grant will also have subcontracts to Co-
Investigators at Stanford, PIRE, Rochester, University of Massachusetts, and Allegheny-
Singer Research Institute 
18B. If the study will engage employees or agents of non-MSU organizations 
(e.g. performance sites), explain how the employees or agents will be 
engaged (e.g. will they perform research procedures, will they obtain 
informed consent from subjects). 
 Not Applicable 
 
The Brown University PI (Caron Zlotnick) will help to train clinic staff in the ROSE intervention 
and provide them with technical assistance. 
Co-I’s Dr. Ellen Poleshuck (University of Rochester), Dr. Tiffany Moore Simas 
(UMMS/UMMHC Memorial), and Dr. Rebecca Weinberg (West Penn Hospital) will serve as 
liaisons to participating clinics in their respective states. 
Dr. Ted Miller (Co-I from PIRE) will perform cost-effectiveness analyses. 
Dr. Shannon Wiltsey-Stirman (Co-I from Stanford) will provide input into implementation and 
sustainment procedures. 
The MSU research support team will consent agency participants to the study.  
18C. If the study involves multiple performance sites, describe the methods for 
communicating with engaged sites related to the protection of human 
subjects (e.g. any potential unanticipated problems that may involve risks 
to subjects others). 
 Not Applicable 
 
      
18D. If there are any cultural or local contexts or requirements that may impact 
the protection of human subjects or present additional risks to subjects that 
have not otherwise been described, please describe. If research is 
conducted outside the state of Michigan, this could include additional state 
or international requirements or laws. 
 Not Applicable 
 
      
18E. If translations to a language other than English will be provided to subjects, 
describe the translation process. 
 Not Applicable 
      
19. 
Resources and Financial Compensation and Costs. 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
30 
19A. If someone will receive a payment for recruiting the subjects, explain the 
amount of payment, who pays it, who receives it, and why they are being 
paid. 
 Not Applicable 
      
19B. If subjects will incur additional financial costs as a result of their 
participation in this study, explain the additional costs. 
 Not Applicable 
      
19C. Describe any resources not otherwise described elsewhere in the 
submission (e.g. internal funding) for the protection of human subjects. 
 Not Applicable 
      
19D. If subject’s biospecimens (even if identifiers are removed) may be used for 
commercial profit, describe whether the subject will or will not share in the 
commercial profit. This also needs to be explained in the consent 
document. 
 Not Applicable 
       
20. 
Information and/or Biospecimens(s) Management and Confidentiality. 
20A. Select the appropriate option:  
 Identifying or coded information will not be stored with the information and/or 
biospecimen(s) 
 Identifying or coded information will be stored with the information and/or biospecimen(s) 
20B. Please explain your selection. If you are storing identifying or coded information with the 
information and/or biospecimen(s), explain why identifiable or coded information and/or 
biospecimen(s) needs to be maintained and how long it will be necessary to maintain it.  
Electronic data will be stored on a secure research server at Michigan State University and will 
be available only to authorized personnel. Paper data will be kept in locked file cabinets at 
Michigan State University (MSU) and/or Butler Hospital. 
20C. Describe the procedures and safeguards you will use to secure the information and/or 
biospecimen(s), including during transport of information and/or biospecimen(s). 
Only trained research staff will have access to the research data. 
21. 
Drug and/or Device Storage, Handling, and Administration. 
 Not Applicable 
 Describe the procedure and plan for storage, handling, and administration of the drug and/or 
device so that they will be used only on enrolled subjects and be used only by authorized 
study personnel. 
      
22. 
Future Research 
 
 If the research involves the collection of identifiable private information or 
identifiable biospecimens, select the appropriate option: 
 Not Applicable 
 The subject’s information or biospecimens, even if identifiers are removed, could be used 
for future research studies or distributed to another investigator for future research studies 
 The subject’s information or biospecimens, even if identifiers are removed, will NOT be 
used or distributed for future research studies 

HRP-503 - Template – Protocol – No Instructions 
 
V19-03 (9/27/2019) 
31 
 Please be sure to carefully consider the appropriate option, as this needs to be explained in 
the informed consent and can limit what is done or used for future research.  
23. 
MSU Additional Information. 
 Not Applicable 
Identify if your study involves any of the following: (check all that apply) 
 Use of human stem cells 
 Research with biospecimens will (if known) or might include whole genome sequencing 
(i.e., sequencing of a human germline or somatic specimen with the intent to generate the 
genome or exome sequence of that specimen). If so, this needs to be explained in the 
consent document.