AVANCE-Houston Building Futures Healthy Marriage and Responsible Fatherhood Program Evaluation
Phase: NA
Conditions: Youth
Sponsor: Midwest Evaluation & Research

 
 
1 
 
AVANCE-Houston Building Futures 
Program Evaluation 
Study Protocol 
3/9/2023 
Clinicaltrials.gov ID: NCT05293145 

Evaluation Plan Template for Healthy Marriage and Relationship Education Grantees  
OMB # 0970-0356 
 
Valid Through 06/30/2021 
 
2 
Evaluation of AVANCE-Houston Building Futures Project in Harris County 
I. Research Question(s)  
A. Background 
Early relationships can both positively and negatively influence professional, peer, and 
romantic relationships later in life, and research indicates that adolescents often do not have the 
necessary knowledge and skills to recognize and develop healthy relationships and avoid 
unhealthy relationships. Youth in Houston, located in Harris County, Texas, face additional 
challenges in creating stable relationships and home environments, as they face high rates of 
poverty, homelessness, and teen pregnancy. Homelessness among young adults in Harris County 
schools increased significantly following Hurricane Harvey, and school districts in Harris 
County saw surges of more than 22,000 students facing homelessness even months after Harvey 
hit (Texas Network of Youth Services 2018). Further, homeless youth, youth aging out of foster 
care, and LGBTQ youth are more vulnerable to falling victim to human trafficking, an issue that 
is highly prevalent in Houston. 
Trauma-informed services are needed to improve self-sufficiency, economic stability, and 
healthy relationships among at-risk and homeless youth in Harris County, especially those living 
in areas with high numbers of single-family households, high poverty rates, low educational 
attainment, high teen pregnancy rates, high incidence of domestic violence, and high 
involvement with the (juvenile) justice system. AVANCE-Houston’s Building Futures program 
(A-HBF) aims to facilitate successful transitions to adulthood for cohorts of at-risk and homeless 
youth by offering healthy relationship and marriage education through the Survival Skills for 
Healthy Families (SSHF) curriculum. 
Developed by Family Wellness Associates, SSHF is an evidence-based curriculum delivered 
through a series of workshops that focus on communication skills, conflict resolution, knowledge 
of the benefits of marriage, stress and anger management, parenting skills, financial literacy, job 
and career advancement, and relationship skills to improve family and economic stability. To 
maximize participation, attendance, and convenience for youth and their parents, AVANCE will 
offer two curriculum delivery models: weekly workshops for youth that take place in targeted 
high-need schools and an intensive weekend retreat model co-attended by youth and parents. 
AVANCE seeks to understand if participation in the A-HBF program is associated with 
improved outcomes among the target population of at-risk and homeless youth. Additionally, the 
evaluation will examine how and why participant-level characteristics such as relationship status, 
life status, and demographics are associated with greater or lesser benefit from A-HBF program 
participation. This research will provide insights about the SSHF curriculum and its effectiveness 
for the target population that can be used in the development of future programming efforts. 
 
 
 
 
 
 
 
 

Evaluation Plan Template for Healthy Marriage and Relationship Education Grantees  
OMB # 0970-0356 
 
Valid Through 06/30/2021 
 
3 
Table 1: Research Question Rationale 
Research 
Question 
Topics 
Existing Research 
Contribution to the 
Evidence Base 
Interest to the Program 
and/or Community 
R1 
Little empirical evidence 
exists in the field of 
HMRE to demonstrate 
the extent to which a 
skills-based curriculum 
can help at-risk and 
homeless youth build 
healthy partner 
relationship attitudes. 
Determine whether 
offering a skills-based 
curriculum, in 
combination with family 
and career support 
services, enhances 
partner relationship 
attitudes among at-risk 
and homeless youth. 
Inform practitioners 
about useful methods to 
consider when delivering 
skills-based curriculum 
to build healthy partner 
relationship attitudes 
among at-risk and 
homeless youth. 
R2 
Little empirical evidence 
exists in the field of 
HMRE to demonstrate 
the extent to which a 
skills-based curriculum 
can help at-risk and 
homeless youth build 
healthy sexual attitudes. 
Determine whether 
offering a skills-based 
curriculum, in 
combination with family 
and career support 
services, enhances sexual 
attitudes among at-risk 
and homeless youth. 
Inform practitioners 
about useful methods to 
consider when delivering 
skills-based curriculum 
to build healthy sexual 
attitudes among at-risk 
and homeless youth. 
R3 
Little empirical evidence 
exists in the field of 
HMRE to demonstrate 
the extent to which a 
skills-based curriculum 
can help at-risk and 
homeless youth build 
healthy partner 
relationship behavior. 
Determine whether 
offering a skills-based 
curriculum, in 
combination with family 
and career support 
services, enhances 
partner relationship 
behavior among at-risk 
and homeless youth. 
Inform practitioners 
about useful methods to 
consider when delivering 
skills-based curriculum 
to build healthy partner 
relationship behavior 
among at-risk and 
homeless youth. 
R4 
Little empirical evidence 
exists in the field of 
HMRE to demonstrate 
the extent to which a 
skills-based curriculum 
can help at-risk and 
homeless youth build 
healthy financial 
attitudes. 
Determine whether 
offering a skills-based 
curriculum, in 
combination with family 
and career support 
services, enhances 
financial readiness 
among at-risk and 
homeless youth. 
Inform practitioners 
about useful methods to 
consider when delivering 
skills-based curriculum 
to build healthy 
financial attitudes 
among at-risk and 
homeless youth. 

Evaluation Plan Template for Healthy Marriage and Relationship Education Grantees  
OMB # 0970-0356 
 
Valid Through 06/30/2021 
 
4 
R5 
Little empirical evidence 
exists in the field of 
HMRE to demonstrate 
the extent to which a 
skills-based curriculum 
can help at-risk and 
homeless youth build 
healthy employment 
attitudes. 
Determine whether 
offering a skills-based 
curriculum, in 
combination with family 
and career support 
services, enhances 
employment readiness 
among at-risk and 
homeless youth. 
Inform practitioners 
about useful methods to 
consider when delivering 
skills-based curriculum 
to build healthy 
employment attitudes 
among at-risk and 
homeless youth. 
B. Research question(s) 
Research questions in this study are framed by a descriptive evaluation design to assess 
whether outcomes improve for at-risk and homeless youth after participating in the AVANCE-
Houston Building Futures (A-HBF) Project. Primary outcomes will indicate whether behavior is 
improving over time for healthy relationships. Secondary benefits will indicate whether the 
attitudes that facilitate and reflect behavior for healthy family relationships and economic 
stability (financial and employment) are improving over time. Research questions that focus on 
primary and secondary outcomes for this study are presented below in Table 2. 
 
Table 2: Outcome Research Questions 
No 
Research Question 
Primary or 
Secondary 
Outcome? 
R1 
Do A-HBF participants report significantly healthier partner relationship 
attitudes after completing the SSHF curriculum in either a standard 
(weekly) or compressed (weekend retreat) workshop format and support 
services? 
Secondary 
Outcome 
R2 
Do A-HBF participants report significantly healthier sexual attitudes after 
completing the SSHF curriculum in either a standard (weekly) or 
compressed (weekend retreat) workshop format and support services? 
Secondary 
Outcome 
R3  
Do A-HBF participants report significantly healthier partner relationship 
behavior after completing the SSHF curriculum in either a standard 
(weekly) or compressed (weekend retreat) workshop format and support 
services? 
Primary 
Outcome 
R4 
Do A-HBF participants report significantly healthier financial attitudes 
after completing the SSHF curriculum in either a standard (weekly) or 
compressed (weekend retreat) workshop format and support services? 
Secondary 
Outcome 
R5 
Do A-HBF participants report significantly healthier employment 
attitudes after completing the SSHF curriculum in either a standard 
Secondary 
Outcome 

Evaluation Plan Template for Healthy Marriage and Relationship Education Grantees  
OMB # 0970-0356 
 
Valid Through 06/30/2021 
 
5 
(weekly) or compressed (weekend retreat) workshop format and support 
services? 
C. Relation to program logic model 
Figure 1 in Appendix A presents a logic model to specify a theory of change for 
delivering A-HBF services. Service delivery processes specified in the model are linked to the 
desired outcomes for healthy relationships and economic readiness. Model specification 
incorporates a descriptive study design to conceptualize service delivery and the outcomes 
assessed for the A-HBF Project. 
Service delivery processes: Key aspects of service delivery processes in the theory of 
change—goals, inputs, activities, and outputs—articulate the experiences that are designed to 
solve specific problems for those who agree to participate in the A-HBF Project. As a result, 
three broad service delivery goals are identified to maximize A-HBF Project participation 
benefits as explained below: 
• Goal 1 - Deliver core curriculum as primary services to A-HBF participants: 
Candidates will understand that they receive core curriculum to develop healthy attitudes 
(partner relationship, sexual, financial, and employment) and skills to engage in healthy 
partner relationship behaviors, but only after receiving an orientation about the A-HBF 
Project and giving project staff informed consent to participate in study activities. Then, 
SSHF curriculum will be delivered as specified in the logic model. 
• Goal 2 - Deliver support services to A-HBF participants: Candidates will understand 
that they receive support services based on their needs to further develop their skills to 
engage in healthy relationships and their financial and employment readiness, but only 
after receiving an orientation about the A-HBF Project and giving project staff informed 
consent to participate in study activities. Then, Family Coaches and Career Coaches will 
offer trainings and make referrals to promote further success for participants. 
• Goal 3 - Conduct Continuous Quality Improvement (CQI) to ensure full 
implementation of project services to participants: Reports prepared and presented to 
the CQI Team by evaluators in each of the 5 steps in the CQI Process will use a series of 
performance indicators to track key outputs over time to identify any A-HBF services 
delivered to study groups that might fall short of the intended amounts to be offered (i.e., 
fidelity standards) and received (i.e., dosage thresholds) by them. The CQI Team will 
then work with project staff to develop and implement performance interventions to 
address any outputs that need improvement to ensure the services offered to and received 
by participants meet the intended amounts by the end of each program year. 
 
Desired Outcomes: Outcomes specified in the logic model theorize the primary and 
secondary outcomes that are desired for participants after they complete A-HBF services. 
Secondary outcomes are the improved attitudes that indicate and reflect participant engagement 
in healthy partner relationship behaviors and capacity for future economic stability. Primary 
outcomes are the healthier behaviors exhibited by participants for partner relationships that 
ultimately define the participation benefits for the A-HBF Project. All outcomes specified in the 
logic model are theorized to be more positive for eligible at-risk and homeless youth after they 
complete primary services and support services. 

Evaluation Plan Template for Healthy Marriage and Relationship Education Grantees  
OMB # 0970-0356 
 
Valid Through 06/30/2021 
 
6 
D. Hypotheses 
Table 3 below presents the hypotheses associated with the primary and secondary 
research questions for this descriptive study. Primary and secondary outcomes are reported on 
participant survey measures that are administered shortly after A-HBF enrollment and again one 
year later after study groups complete services (or should have completed them). Confirmatory 
hypotheses focus on the extent to which participants report improved attitudes (partner 
relationship, sexual, financial, and employment) and behaviors (partner relationship) after A-
HBF participation. 
 
Table 3: Hypothesized Results 
Research 
Question 
Hypothesized Result  
R1 
Participants will report healthier partner relationship attitudes after they complete 
primary services and support services in the A-HBF Project. 
R2 
Participants will report healthier sexual attitudes after they complete primary services 
and support services in the A-HBF Project. 
R3  
Participants will report healthier partner relationship behavior after they complete 
primary services and support services in the A-HBF Project. 
R4 
Participants will report healthier financial attitudes after they complete primary 
services and support services in the A-HBF Project. 
R5 
Participants will report healthier employment attitudes after they complete primary 
services and support services in the A-HBF Project. 
II. Research Design and Staff 
A. Research design 
In this study, a descriptive study design will be used to determine if A-HBF participants benefit 
from the Survival Skills for Healthy Families (SSHF) curriculum. At-risk or homeless youth who 
agree to participate in the study will receive the SSHF curriculum through either a series of 7 in-
school weekly workshops (14 hours) or an intensive weekend retreat co-attended by their parents 
(16 hours) followed two weeks later by a Reflections Workshop (2 hours). All program 
participants also receive support services delivered by Family Coaches and Career Coaches as 
needed. 
 
Baseline data will be collected from participants shortly following study enrollment. Post-test 
data will be collected following curriculum completion, approximately 2-7 weeks following 
study enrollment depending on the curriculum delivery format. Follow-up data will be collected 
one year after study enrollment. 

Evaluation Plan Template for Healthy Marriage and Relationship Education Grantees  
OMB # 0970-0356 
 
Valid Through 06/30/2021 
 
7 
B. Methods to develop study groups  
Because this is a descriptive evaluation, this subsection is not relevant. 
C. Sample  
The target population for this study is at-risk and homeless youth from high-need school districts 
in Harris County who are 14-18 years of age and enrolled in 9-12th grades. AVANCE will work 
with school districts, school boards, and individual schools to identify and recruit participants 
from the target population. The unit of analysis for this study is individual youth. See Appendix 
D for anticipated sample flow and attrition rates. 
 
Power analysis: 
The analysis of survey data will address the primary and secondary research questions with 
findings derived from running a linear regression model that quantifies the potential benefits of 
Building Futures Program participation and outcomes as well as the relationship of participant 
characteristics to Building Futures Program benefits. Benefits are quantified by specifying a 
model that compares the relative influences of participant characteristics as predictor that is 
regressed against a series of dependent variables (e.g., outcomes specified in the logic model). 
Given the predicted sample size as described in the appended logic model, the following 
calculations regarding power and effect size can be made. 
 
 
Participant Responses 
Minimum Detectable Effect Size  
Post-test 
1105 
0.017 
Follow-up 
774 
0.023 
 
These power analyses assume an Alpha Level of 0.05 with a two-tailed test and power level set 
at 0.80 (1 – Beta). The estimated effect sizes are all small effects or nearly small effects that 
would be detected, based on best practices outlined by Cohen (1988) of small effect sizes <0.02 
and medium effect sizes <0.15. With these analyses, the proposed research design is 
demonstrated to measure large, medium, and small changes in participants’ responses. 
 
Table 4: Methods to Promote Sufficient Program Participation 
What methods will you use to 
ensure sufficient sample is 
recruited, enrolls, and 
participates in the program? 
AVANCE will work with schools and their community 
partners to recruit program participants, and AVANCE will 
provide incentives for program retention and completion. 
The CQI process will address issues regarding program 
recruitment and enrollment to ensure targets are met. 
Who will be responsible for 
recruiting the evaluation 
sample? 
The evaluation sample will not differ from the program 
population, in that all participants will be invited to participate 
in the evaluation. Enrollment into the evaluation will be 
conducted by the CQI Data Manager who will conduct the 
informed consent process and proctor the baseline data 
collection efforts. 

Evaluation Plan Template for Healthy Marriage and Relationship Education Grantees  
OMB # 0970-0356 
 
Valid Through 06/30/2021 
 
8 
Please describe any incentives 
to be offered for program 
participation and/or 
completion and/or data 
collection and/or participation 
in the evaluation. 
Program Participation – AVANCE will provide a series of 
incentives for program retention (promotional items) and 
completion ($75 gift card). 
Evaluation Participation – No incentives will be used for the 
first two data collections- baseline and post-test, while 
participants are beginning or still engaged in services. A $50 
incentive will be used for the 1-year follow-up survey and will 
be provided in the form of a Walmart gift card. 
 
D. Lead staff 
See Appendix B for CVs of lead evaluation staff. 
 
Table 5: Lead Evaluation Staff 
Name 
Organization 
Role in the Evaluation 
Dr. Matthew Shepherd 
Midwest Evaluation and 
Research 
Principal Investigator 
Dr. Theodore Jurkiewicz Midwest Evaluation and 
Research 
Lead Evaluation Consultant 
McKenna LeClear, MA 
Midwest Evaluation and 
Research 
Evaluation Project Manager 
Jennifer Leveille, MPH 
Midwest Evaluation and 
Research 
CQI Data Manager 
 
Dr. Matthew Shepherd will serve as the Principal Investigator for this grant. As such, he has 
corporate responsibility for all evaluation activities. Dr. Shepherd has over 25 years’ 
experience in program design and implementation, applied research, program evaluation, policy 
analysis, and evaluative technical assistance. 
 
Dr. Theodore Jurkiewicz will serve as the Lead Evaluation Consultant and provide day-to-day 
oversight for the HMRF evaluation activities. Dr. Jurkiewicz has been conducting HMRF 
evaluations since 2006 and has been involved with every cohort of HMRF grantees. 
 
McKenna LeClear will serve as the Evaluation Project Manager. The Evaluation PM leads 
the effort to conduct an impact study and a Continuous Quality Improvement (CQI) process for 
the grant. Prior to joining MER, Ms. LeClear managed evaluation projects as a Senior Research 
Associate with the Child and Family Research Partnership at the University of Texas at Austin. 
 

Evaluation Plan Template for Healthy Marriage and Relationship Education Grantees  
OMB # 0970-0356 
 
Valid Through 06/30/2021 
 
9 
Jennifer Leveille will serve as the CQI Data Manager. The CQI Data Manager will be 
responsible for accurate and timely data collection, report generation, and assistance with 
Continuous Quality Improvement (CQI) throughout the process of the grant. Prior to joining 
MER, Mrs. Leveille worked as a Senior Research Assistant with MD Anderson Cancer Center 
and as a Public Health Consultant with Rice University’s Texas Policy Lab. 
E. Ongoing coordination between grantee and local evaluator  
The basis for ongoing coordination between AVANCE (the grantee) and MER (the local 
evaluator) is regular communication, by way of recurring meetings and daily interactions with 
embedded staff. Throughout the original proposal process, and now during the evaluation 
planning phase, MER worked in consort with AVANCE to design a study with research 
questions that are appropriate to the intervention. MER guides the process, given our experience 
designing and running evaluations, and AVANCE provides expertise on their community, target 
population, and program/curricula specifics. 
Recurring meetings will include a bi-weekly project CQI Team meeting. Under the leadership of 
the CQI Data Manager and Lead MER Evaluator, the CQI Team reviews data from the nFORM 
and local evaluation systems to identify and mitigate implementation or data issues, and closely 
examine trends and accomplishments. This team includes AVANCE organizational and project 
leadership, the MER Evaluation team, and front-line staff representatives (e.g., Program 
Administrators, Case Managers).  
In addition to CQI Team meetings, overall project team meetings occur monthly (at a minimum), 
with project leaders across MER and AVANCE in attendance, to ensure the partnership remains 
strong and that coordination across organizations is on track. This recurring, ongoing meeting 
structure is conducive to close coordination, ensuring that challenges can be quickly addressed, 
and promising strategies can be efficiently maximized. 
One of the key components of this coordination effort is the CQI Data Manager, who is a MER 
employee, embedded with AVANCE. The CQI Data Manager functions to bridge the gap 
between organizations. They will interact with AVANCE staff daily while completing their job 
duties and play a leadership role in the recurring meetings outlined above. See Section II.D 
above for more details about this role and others. Both the meetings and the roles outlined above 
will continue throughout the entire project period, providing opportunities to ensure the rigor and 
relevance of the evaluation and its findings, and to discuss and coordinate dissemination efforts 
(which will also be shared across MER and AVANCE).  
MER has experience operating prior RCT evaluations using this exact process. Clearly outlining 
roles and responsibilities maintains the independence of the evaluation. That is, the evaluation 
team helps identify and illuminate areas of concern or improvement (for the program and the 
evaluation), but the program staff have responsibility for implementing improvements and 
providing direct services to participants. In this way, AVANCE and MER acknowledge our 
shared interest in and responsibility for a well-executed project and evaluation, but that MER is 
also an independent and external organization with a high level of integrity and is not responsible 
for nor invested in the specific outcomes of the program. This allows for close coordination 
without allowing for co-dependence or personal interests to influence evaluation findings. 

Evaluation Plan Template for Healthy Marriage and Relationship Education Grantees  
OMB # 0970-0356 
 
Valid Through 06/30/2021 
 
10 
III. Data Collection 
A. Data collection 
Table 6: Constructs and measures/data collection instruments 
Construct 
Measure  
Instrument 
Reliability and Validity 
(if standardized instrument, you 
provide a citation for the instrument) 
Partner 
Relationship 
Attitudes 
9 items: levels of 
agreement with 
relationship 
beliefs 
(categories, 4-
point); 3 items: 
likelihood of key 
relationship 
beliefs 
(categories, 5-
point scale) 
nFORM Healthy 
Marriage Youth 
Program Survey 
(A1.a-i; A2.a-c) 
nFORM 
Sexual 
Attitudes 
7 items: levels of 
agreement with 
key sexual 
attitudes 
(categories, 4-
point scale) 
nFORM Healthy 
Marriage Youth 
Program Survey 
(B1.a-g) 
nFORM 
Partner 
Relationship 
Behavior 
5 items: levels of 
reported skill 
with key 
relationship 
behaviors 
(categories, 4-
point scale); 6 
items: frequency 
engage in key 
relationship 
behaviors 
(categories, 5-
point scale) 
nFORM Healthy 
Marriage Youth 
Program Survey 
(C1b.a-e; C3.a-f) 
nFORM 
Financial 
Attitudes 
4 items: levels of 
agreement with 
key financial 
OLLE (Online 
Local 
In appendix D of:                           
Jurkiewicz, T. & Friedman, L. (2020). 
Impact Evaluation of The TYRO 

Evaluation Plan Template for Healthy Marriage and Relationship Education Grantees  
OMB # 0970-0356 
 
Valid Through 06/30/2021 
 
11 
attitudes 
(categories, 7-
point scale) 
Evaluation) 
Survey 
Champion Dads Project in Dallas, 
Texas: Final Impact Evaluation Report 
for Anthem Strong Families. Midwest 
Evaluation and Research. 
Employment 
Attitudes 
4 items: levels of 
agreement with 
key employment 
attitudes 
(categories, 7-
point scale) 
OLLE (Online 
Local 
Evaluation) 
Survey 
In appendix D of:                           
Jurkiewicz, T. & Friedman, L. (2020). 
Impact Evaluation of The TYRO 
Champion Dads Project in Dallas, 
Texas: Final Impact Evaluation Report 
for Anthem Strong Families. Midwest 
Evaluation and Research. 
 
Consent: 
Because the planned evaluation involves human subjects, AVANCE understands program 
implementation requires both IRB approval and participant informed consent. MER has an 
established relationship with Solutions IRB, having secured more than 14 IRB approvals 
and renewals for evaluations it has conducted during the past four years. IRB clearance will 
be submitted for approval and obtained during the planning period. 
 
Each school location will have a signed letter of agreement to be included in this project. This 
must be filed with the IRB during the application process. Any school locations that are later 
recruited into the study will result in MER submitting amendments to the IRB to include these 
schools as research sites, and the additional school’s letter will be included in the project. 
 
An opt-out letter will be provided to all parents/guardians of applicants under the age of 18 who 
are interested in the project. The letter will state that because this is a research study, survey data 
will be collected on youth, but will be kept confidential. If a parent or guardian does not want 
their child to participate in the project, that letter will need to be signed and returned to the 
school or facilitator. 
 
During the survey administration process, youth participants will be given a letter along with the 
survey that explains that the survey is optional, any or all questions can be skipped for any 
reason, and not taking the survey will not result in any penalties. 
 
Table 7: Data Collection Timeline 
Wave of Data Collection  
(e.g., baseline, short-term follow-up, 
long-term follow-up) 
Timing of Data Collection 
 
Baseline 
Collected immediately following informed consent 
and enrollment – during orientation or first 
workshop 

Evaluation Plan Template for Healthy Marriage and Relationship Education Grantees  
OMB # 0970-0356 
 
Valid Through 06/30/2021 
 
12 
Post-test 
Collected after the completion of the primary 
services programming – during the last workshop 
or session 
1 year follow-up (post-enrollment) 
Collected approximately one year after program 
enrollment 
 
Table 8: Measures 
Measure 
Timing of Data Collection 
(baseline, wave of data 
collection) 
Method of Data 
Collection 
Who Is Responsible 
for Data Collection? 
On-line Local 
Evaluation 
(OLLE) and 
nFORM 
Baseline 
Survey 
Baseline 
Participant self-enters 
survey using online 
data collection program 
or completes survey 
with program staff 
through Zoom or over 
the phone in the case of 
virtual service delivery 
Data Manager and 
program staff will 
proctor data collection 
and assist participants as 
necessary 
OLLE and 
nFORM Post-
Test Survey 
Post-Test (approx. 7 weeks 
after enrollment for standard 
format and 16 days after 
enrollment for compressed 
format) 
Participant self-enters 
survey using online 
data collection program 
or completes survey 
with program staff 
through Zoom or over 
the phone in the case of 
virtual service delivery 
Data Manager and 
program staff will 
proctor data collection 
and assist participants as 
necessary 
OLLE 1 Year 
Follow-up 
Survey 
1 year after enrollment / 
baseline 
Participant self-enters 
survey using online 
data collection platform 
and link – or – Phone 
interview data 
collection  
MER Research Staff/ 
participant tracking 
team 
 
Ensuring and monitoring high quality data collection: 
This evaluation will utilize both post-program surveys (completed at the completion of core 
programming) and follow-up surveys collected one year after enrollment. The methods for these 
data collections differ. The primary driver for post-program survey completion is high rates of 
program retention. This data point will be collected during the last workshop session – after (but 
during the same session of) the completion of the nFORM post-program data collection. As such, 

Evaluation Plan Template for Healthy Marriage and Relationship Education Grantees  
OMB # 0970-0356 
 
Valid Through 06/30/2021 
 
13 
only those individuals who complete the program and who are at the data collection session will 
participate in the post-program data collection. 
All program staff and evaluation staff will undergo a rigorous set of trainings to prepare for the 
evaluation. All staff receive an overview and introductory training to present the goals and 
objectives of the evaluation effort and its importance to the overall project. Next, all staff receive 
training on human subjects protection and are required to pass a certification test on the subject 
matter. All staff will also receive a detailed training on the evaluation, including the evaluation 
tools, timing and data collection process, and the role and importance of randomization of 
participants. 
In addition, the CQI Data Manager and the primary local evaluation staff will undergo a rigorous 
training process to better understand the context of HMRF research, training on data collection 
procedures they will be responsible for, and training on the nFORM system and use of nFORM 
data in a CQI process. MER is creating networks of CQI data managers and Evaluation Project 
Managers across the 12 projects that we are evaluating so that all staff have access to 
experienced data managers and evaluation staff who have done this work previously. This 
training takes the form of weekly training sessions that are currently underway. 
Members of the CQI team will also receive specific training on the MER CQI process that has 
been developed prior to the launch of data collection or program services. As described 
elsewhere, MER is assisting the program staff in implementing a robust CQI process that will 
focus on retention as one of the primary areas of program improvement, and as such, we are 
anticipating relatively modest levels of attrition for this data collection. 
On a bi-weekly basis, the CQI Data Manager, the local evaluation staff, and MER technical 
specialists will be responsible for downloading data from the nFORM and MER On-Line Local 
Evaluation (OLLE) systems for processing and presentation to the CQI team for tracking and 
monitoring performance measurement outcomes (recruitment, enrollment, dosage, completion, 
referrals, etc.) so that near real-time adjustments can be made to program implementation to 
ensure compliance with program goals and objectives. 
All MER training is currently being recorded, and as new staff come on board with projects or 
project staff turnover (or need refresher training), recorded training material can be shared and 
accessed with follow-up one on one training with the primary local evaluator and the MER LOB 
Lead, who has more than 15 years’ experience in evaluating HMRF projects. 
 
Tracking participants and reducing attrition: 
For each wave of data 
collection listed in Table 7, 
what is your estimated 
response rate?  
Baseline 
100% - done at the time of enrollment 
Wave 1 
90% - estimated A-HFP completion rate 
Wave 2 
70% - estimated 1 year follow-up rate