AVANCE-Houston FRAMEWorks Healthy Marriage and Responsible Fatherhood Program Evaluation
Phase: NA
Conditions: Adults
Sponsor: Midwest Evaluation & Research

 
 
 
 
AVANCE Houston 
FRAMEWorks Program 
Evaluation 
 
Study Protocol 
2/6/2024 
Clinicaltrials.gov ID: 
NCT05261802 
Plan Template for Healthy Marriage and 
Relationship Education Grantees 

 
 
2 
Evaluation of AVANCE-Houston FRAMEWorks Project In Houston, 
Texas 
I. Research Question(s)  
A. Background 
Houston, located in Harris County, Texas, is one of the fastest growing and most diverse 
metropolitan areas in the country, yet 15% of the population of Harris County lives in poverty1. 
Factors such as low educational attainment, financial instability, unemployment, domestic and 
intimate partner violence, and homelessness contribute to the poverty and create barriers that 
make it difficult for adults to meet their obligations as partners, parents, and providers. 
Research has shown that healthy, stable families can prevent or buffer against these barriers2. 
AVANCE-Houston aims to promote healthy family relationships and economic stability among 
the diverse population of Harris County, particularly those in areas with a high number of 
single-family households, high poverty rates, low educational attainment, and high incidence 
of domestic violence. 
AVANCE-Houston’s FRAMEWorks (Family, Relationship, and Marriage Education Works) 
Program seeks to accomplish these goals by providing the Survival Skills for Healthy Families 
(SSHF) curriculum to cohorts of low-income adults. Developed in 1980 by Family Wellness, 
this curriculum has been delivered to over 2,000,000 around the world by a variety of trained 
instructors and through organizations such as schools, social services, and churches. 
Curriculum delivery by AVANCE follows the below schedule: 
Session 1 – All Aboard (Program Intro) 
Session 2 – Talk to Me (Communication & Conflict Resolution) 
Session 3 – Love Doesn’t Hurt (Domestic Violence) 
Session 4 – Family Matters (Parenting / Co-Parenting and Family Wellness) 
Session 5 – Igniting Pathways (Career Readiness / Advancement) 
Session 6 – For the Love of Money (Financial Literacy) 
Session 7 - Me, myself, and I (Self-Care & Family Balance) 
 
Target populations have included the military, refugees, foster/adoptive families, grandparents 
raising grandchildren, fathers, low-income adults, and has included various races/ethnicities3. 
This evidence-based curriculum is delivered through a series of workshops that focus on 
communication skills, conflict resolution, knowledge of the benefits of marriage, stress and 
anger management, parenting skills, financial literacy, job and career advancement, and 
relationship skills to improve family and economic stability. Research conducted on the Family 
Wellness model, which includes the SSHF as its base, shows significant positive changes 
across racial/cultural, socioeconomic, and geographic regions associated with attending 
classes4. One study conducted on the St. Louis Marriage Coalition consisted of 234 
participants, most of whom were at-risk urban African American, Bosnian, and Hispanic 
mothers and fathers5. Researchers concluded that the program is making progress toward 
one of the program objectives of ‘‘decreased number of adversarial relationships between 
parent partners’’. The second study was conducted on the Healthy Marriage Initiative in New 

 
 
3 
Mexico, which consisted primarily of Hispanic participants (73%). Using an instrument that 
measured marital satisfaction, couple communication, and conflict resolution, researchers 
reported a statistically significant difference in communication skills and conflict resolution 
among participants that completed the program6. Lastly, OPRE published their report titled 
Strengthening Relationship Education and Marriage Services in November 2021, detailing the 
Empower Families impact study, the first rigorous evidence of the effects of the Family 
Wellness curriculum. Using one-year follow-up data from almost 800 low-income couples 
randomized into a treatment and control group, researchers found that all aspects of a 
couples’ relationship quality that were examined had improved. This included, but was not 
limited to, increased level of support and affection partners felt toward each other, use of 
constructive conflict behaviors and avoidance of destructive conflict behaviors, the level of 
commitment and happiness they felt toward their relationship, and quality of their co-parenting 
relationship7. While this study did not find that participants had improved labor market 
outcomes, significant reduction in economic hardship was found for those who participated in 
the program, likely due to how the program’s financial literacy services helped participants 
avoid certain economic hardships by managing their limited resources more effectively. 
AVANCE has delivered the SSHF curriculum in part since 2009 and completely since 2015, 
with a target population of low-income adults residing in the Houston area. AVANCE now 
offers the SSHF curriculum in multiple delivery models: in recurring weekly workshops and 
intensive weekend retreats. As well, AVANCE will offer the weekly workshops either virtually 
via Zoom or in-person, while the weekend retreat will only be offered in-person due to the 
importance of activities designed to provide intense connection between participants. Each of 
these models meets the needs of the target population in different ways, and allows 
participants to consider factors such as childcare, transportation, work/school schedules, 
weekend availability, and geographic location when choosing their preferred model. All 
workshop models will be offered to program participants who will have the choice of whether 
the weekly (either in-person or virtually) or the weekend (in-person only) format works best for 
them. Delivering the SSHF curriculum in a virtual setting is a novel method of offering 
participants the information, which AVANCE hopes will add great value to the field.  
The key goal of this descriptive evaluation is to assess the extent to which participation in the 
FRAMEWorks Program is associated with improved attitudes and behaviors among program 
participants. Change will be measured from the time of program enrollment to one-year after 
enrollment, to allow for investigation of long-term outcomes. Research questions will focus on 
whether attitudes and behaviors related to parenting, partner relationships, and financial and 
job readiness change after participation in program services. Additionally, a secondary 
analysis will focus on the ancillary question of whether differences in outcomes exist between 
participants selecting in-person and virtual groups.  
 
 
Table 1: Summary of rationale for investigation 
Research 
Question Topics 
Existing Research 
Contribution to the Evidence Base 
Interest to the Program and/or 
Community 

 
 
4 
R1: Parenting 
Behavior 
Survival Skills for Healthy 
Families, the primary curriculum 
of the FRAMEWorks Program, 
is an evidence-based curriculum 
that has been proven effective at 
building skills needed for healthy 
family relationships and child 
development. 
Determine to what extent participation in the 
FRAMEWorks Program is associated with 
positive changes in parenting behaviors 
among low-income participants in the 
Houston area. 
Healthy parenting behaviors support 
children to achieve more positive 
outcomes, both short- and long-term, 
and promote adult wellbeing. 
R2: Parenting 
Attitudes 
Survival Skills for Healthy 
Families, the primary curriculum 
of the FRAMEWorks Program, 
is an evidence-based curriculum 
that has been proven effective at 
building skills needed for healthy 
family relationships and child 
development. 
Determine to what extent participation in the 
FRAMEWorks Program is associated with 
positive changes in parenting attitudes 
among low-income participants in the 
Houston area. 
Healthy parenting attitudes support 
children to achieve more positive 
outcomes, both short- and long-term, 
and promote adult wellbeing.  
R3: Partner 
Relationship 
Behavior 
Survival Skills for Healthy 
Families, the primary curriculum 
of the FRAMEWorks Program, 
is an evidence-based curriculum 
that has been proven effective at 
building skills needed for healthy 
family relationships and child 
development. 
Determine to what extent participation in the 
FRAMEWorks Program is associated with 
positive changes in partner relationship 
behaviors among low-income participants in 
the Houston area. 
Healthy partner relationship behaviors 
support children to achieve more 
positive outcomes, both short- and 
long-term, and promote adult 
wellbeing. 
R4: Partner 
Relationship 
Attitudes 
Survival Skills for Healthy 
Families, the primary curriculum 
of the FRAMEWorks Program, 
is an evidence-based curriculum 
that has been proven effective at 
building skills needed for healthy 
family relationships and child 
development. 
Determine to what extent participation in the 
FRAMEWorks Program is associated with 
positive changes in partner relationship 
attitudes among low-income participants in 
the Houston area. 
Healthy partner relationship attitudes 
support children to achieve more 
positive outcomes, both short- and 
long-term, and promote adult 
wellbeing.  
R5: Financial 
Readiness 
Survival Skills for Healthy 
Families, the primary curriculum 
of the FRAMEWorks Program, 
is an evidence-based curriculum 
that has been proven effective at 
building skills needed for healthy 
family relationships and child 
development. 
Determine to what extent participation in the 
FRAMEWorks Program is associated with 
positive changes in financial readiness 
among low-income participants in the 
Houston area. 
Healthy financial readiness behaviors 
support children to achieve more 
positive outcomes, both short- and 
long-term, and promote adult 
wellbeing. 
R6: Job 
Readiness 
Survival Skills for Healthy 
Families, the primary curriculum 
of the FRAMEWorks Program, 
is an evidence-based curriculum 
that has been proven effective at 
building skills needed for healthy 
Determine to what extent participation in the 
FRAMEWorks Program is associated with 
positive changes in job readiness among 
low-income participants in the Houston area. 
Healthy job readiness behaviors 
support children to achieve more 
positive outcomes, both short- and 
long-term, and promote adult 
wellbeing. 

 
 
5 
family relationships and child 
development. 
R7: Differences 
by delivery format 
Survival Skills for Healthy 
Families, the primary curriculum 
for the FRAMEWorks Program, 
is an evidence-based curriculum 
that has been proven effective 
when delivered in person at 
building skills needed for healthy 
family relationships and child 
development. 
Determine to what extent changes in 
outcomes differ between participants who 
receive different curriculum delivery formats 
(virtual vs. in-person). 
Participants who receive the Survival 
Skills for Healthy Families 
curriculum virtually may report 
different changes in outcomes than 
those who receive the curriculum in 
person. Assessing potential 
differences will provide AVANCE 
and the field with information about 
delivery formats that can be applied to 
future cohorts. 
 
B. Research question(s) 
Research questions in this study are framed by a descriptive evaluation design that will be 
used to measure the extent to which participation in the FRAMEWorks Program is positively 
associated with improved attitudes and behaviors among low-income adults in the Houston, 
TX area. Research questions will focus on whether attitudes and behaviors related to 
parenting and partner relationships, and financial and job readiness behaviors change after 
participation in the program. Changes will be assessed between outcomes at program 
enrollment before primary workshops begin using entrance surveys and outcomes one year 
after program enrollment using the 1-year follow-up survey. The sample for the primary 
analysis includes all program participants who consent to be in the study, regardless of 
program format (e.g., virtual, in person, weekly workshops, weekend retreat).  Implementation 
analyses seek to determine how many curriculum hours the program is offering versus how 
many curriculum hours participants are completing. Exploratory secondary analyses will 
investigate to what extent changes in outcomes differ between participants who receive the 
curriculum virtually vs. in person.  
Table 2: Research questions by type (implementation or outcome) 
No  
Research Question  
Implementation or Outcome?  
I1  
To what extent is the SSHF curriculum offered to and received by program 
participants?  
Implementation  
R1  
How did parenting behavior outcomes change from program enrollment to 1 
year after enrollment?  
Outcome  
R2  
How did parenting attitude outcomes change from program enrollment to 1 
year after enrollment?  
Outcome  
R3  
How did partner relationship behavior outcomes change from program 
enrollment to 1 year after enrollment?  
Outcome  
R4  
How did partner relationship attitude outcomes change from program 
enrollment to 1 year after enrollment?  
Outcome  

 
 
6 
R5  
How did financial readiness behavior outcomes change from program 
enrollment to 1 year after enrollment?  
Outcome  
R6  
How did job readiness behavior outcomes change from program enrollment to 
1 year after enrollment?  
Outcome  
R7 
Do the change in outcomes differ between those selecting in-person or virtual 
groups? 
Exploratory Outcome 
 
Table 3 below summarizes key aspects of the research questions that are used to measure 
changes in participant outcomes after participation in the FRAMEWorks Program. For each 
outcome research question, there is a summary of the intervention, target population, comparison 
group, changes expected (the outcome), and whether the outcome is confirmatory or exploratory. 
As a secondary exploratory analysis, we will explore whether any differences in outcomes exist 
between those participants who receive the curriculum virtually vs. those who receive the 
curriculum in-person. 
Table 3: Outcome Research Questions 
Research 
Question 
Number  
Should 
correspond to 
the number 
indicated in 
Table 2 
above  
 
Intervention  
Program 
component or set 
of activities that the 
evaluation will 
examine  
Target Population  
Population for 
which the 
association 
between the 
treatment and 
change in the 
outcome will be 
examined  
Comparison   
What the 
intervention will 
be compared to 
(e.g., pre-
intervention for 
descriptive 
designs)  
Outcome  
Changes in key measures 
over time  
Confirmatory or 
Exploratory?  
Confirmatory: those 
upon which conclusions 
will be drawn  
Exploratory: those that 
may provide additional 
suggestive evidence  
R1  
FRAMEWorks 
Program 
participation (the 
combination of the 
Survival Skills for 
Healthy Families 
curriculum, case 
management 
services, and 
secondary support 
services) 
Low-income adults 
in the Houston area 
who are at least 18 
years of age 
Pre-intervention 
outcome 
scores  
Participants will report 
improved parenting 
behaviors after participating 
in the FRAMEWorks Program 
Confirmatory 
R2  
FRAMEWorks 
Program 
participation (the 
combination of the 
Survival Skills for 
Healthy Families 
curriculum, case 
management 
services, and 
Low-income adults 
in the Houston area 
who are at least 18 
years of age 
Pre-intervention 
outcome 
scores  
Participants will report 
improved parenting 
attitudes after participating in 
the FRAMEWorks Program 
Confirmatory 

 
 
7 
secondary support 
services) 
R3  
FRAMEWorks 
Program 
participation (the 
combination of the 
Survival Skills for 
Healthy Families 
curriculum, case 
management 
services, and 
secondary support 
services) 
Low-income adults 
in the Houston area 
who are at least 18 
years of age 
Pre-intervention 
outcome 
scores  
Participants will report 
improved partner 
relationship behaviors after 
participating in the 
FRAMEWorks Program  
Confirmatory 
R4 
FRAMEWorks 
Program 
participation (the 
combination of the 
Survival Skills for 
Healthy Families 
curriculum, case 
management 
services, and 
secondary support 
services) 
Low-income adults 
in the Houston area 
who are at least 18 
years of age 
Pre-intervention 
outcome 
scores  
Participants will report 
improved partner 
relationship attitudes after 
participating in the 
FRAMEWorks Program  
Confirmatory 
R5 
FRAMEWorks 
Program 
participation (the 
combination of the 
Survival Skills for 
Healthy Families 
curriculum, case 
management 
services, and 
secondary support 
services) 
Low-income adults 
in the Houston area 
who are at least 18 
years of age 
Pre-intervention 
outcome 
scores  
Participants will report 
improved financial readiness 
after participating in the 
FRAMEWorks Program  
Confirmatory 
R6 
FRAMEWorks 
Program 
participation (the 
combination of the 
Survival Skills for 
Healthy Families 
curriculum, case 
management 
services, and 
secondary support 
services) 
Low-income adults 
in the Houston area 
who are at least 18 
years of age 
Pre-intervention 
outcome 
scores  
Participants will report 
improved job readiness after 
participating in the 
FRAMEWorks Program  
Confirmatory 
R7 
Primary workshop 
delivery format 
Low-income adults 
in the Houston area 
Pre-intervention 
outcome 
scores  
Participants will report 
improved outcomes after 
participating in the program, 
Exploratory 

 
 
8 
who are at least 18 
years of age 
but differences may exist for 
virtual vs. in-person 
participants  
 
C. Relation to program logic model 
Figure 1 in Appendix A presents a logic model to specify a theory of change for delivering 
FRAMEWorks Program services. Service delivery processes in the model are linked to the 
desired outcomes for healthy family relationships and economic stability. Model specification 
incorporates a descriptive study design to measure change in outcomes from program 
enrollment to one year after enrollment. 
Service delivery processes: Key aspects of service delivery processes in the theory of change—
goals, inputs, activities, and outputs—articulate the experiences that are designed to solve 
specific problems for those who agree to participate in the FRAMEWorks Program. As a 
result, three broad service delivery goals are identified to maximize FRAMEWorks Program 
participation benefits as explained below: 
• Goal 1 - Deliver primary curricula workshops to AVANCE-Houston FRAMEWorks 
Program participants: Candidates will understand that they will receive core curricula 
to develop their skills to engage in healthy behaviors for parenting, partner relations, 
employment, and financial management, but only after receiving an orientation about the 
FRAMEWorks Program and giving project staff informed consent to participate in study 
activities. Then, SSHF curricula will be delivered as specified in the logic model. 
• Goal 2 - Deliver support services to FRAMEWorks Program participants: 
Candidates will understand that they will receive support services based on their interest 
to further develop their skills to engage in healthy behaviors for parenting, partner 
relations, employment, and financial management, but only after receiving an orientation 
about the FRAMEWorks Program and giving project staff informed consent to 
participate in study activities. Then, staff from AVANCE-Houston will offer classes and 
referrals to promote further development for participants.  
• Goal 3 - Conduct Continuous Quality Improvement (CQI) to ensure full 
implementation of FRAMEWorks Program services to participants: Reports 
prepared and presented to the CQI Team by evaluators as part of the CQI process will use 
a series of performance indicators to track key outputs over time to identify if the 
curriculum delivered to study groups could be falling short of the intended amounts to be 
offered (i.e., fidelity standards) and received (i.e., dosage thresholds) by them. The CQI 
Team will then work with project staff to develop and implement performance 
interventions to address any outputs that need improvement.  
 
Desired Outcomes: Outcomes are those that are desired for program participants after they 
participate in the program. Those outcomes are the improved attitudes that facilitate and reflect 
participant engagement in healthy parenting, partnership, financial, and employment behaviors 
after participating in the FRAMEWorks Program. 

 
 
9 
D. Hypotheses 
Table 4 below presents hypotheses to address the research questions for this descriptive study. 
Participants are surveyed shortly after FRAMEWorks Program enrollment before primary 
workshops begin and again one year after program enrollment.  
Table 4: Hypotheses by research question 
Research 
Question  
Hypothesized Result   
R1  
Participants will report healthier parenting behaviors after participating in the FRAMEWorks Program.  
R2  
Participants will report healthier parenting attitudes after participating in the FRAMEWorks Program. 
R3   
Participants will report healthier partner relationship behaviors after participating in the FRAMEWorks 
Program. 
R4 
Participants will report healthier partner relationship attitudes after participating in the FRAMEWorks 
Program. 
R5 
Participants will report healthier financial readiness behaviors after participating in the FRAMEWorks 
Program.  
R6 
Participants will report healthier job readiness behavior after participating in the FRAMEWorks Program. 
R7 
All participants will report improved outcomes after FRAMEWorks Program participation, but differences may 
exist between virtual and in-person participants.  
II. Research Design and Staff 
A. Research design 
In this descriptive evaluation design, parenting and partner relationship behaviors and attitudes, 
as well as financial and job readiness behaviors, will be measured at the time of program 
enrollment and again one year after program enrollment. Low-income adults who agree to 
participate in the study will choose which workshop model works best for them to receive the 
SSHF curriculum (that is, the recurring weekday or weekend format, and virtual or in-person). 
All participants are also offered support services, delivered by Family Coaches and Career 
Coaches. 
Table 5 below presents the role played by descriptive design to produce findings that inform 
research questions addressed in the evaluation.  
Table 5: Justification of Descriptive Study Designs 
Research 
Question  
Design  
Justification  
R1  
For this descriptive evaluation, healthy parenting 
behaviors will be measured at the time of program 
enrollment and again one year after program 
enrollment. Analyses will determine if changes in 
parenting behavior outcomes are statistically 
significant.  
This study is interested in whether participation in the 
FRAMEWorks Program is associated with improved 
outcomes for participants, so a descriptive study of within 
subject change is sufficient for measuring differences in 
participant outcomes over time.  

 
 
10 
R2  
For this descriptive evaluation, healthy parenting 
attitudes will be measured at the time of program 
enrollment and again one year after program 
enrollment, and analyses will determine if changes in 
parenting behavior outcomes are statistically 
significant.  
This study is interested in whether participation in the 
FRAMEWorks Program is associated with improved 
outcomes for participants, so a descriptive study of within 
subject change is sufficient for measuring differences in 
participant outcomes over time.  
R3  
For this descriptive evaluation, healthy partner 
relationship behavior will be measured at the time 
of program enrollment and again one year after 
program enrollment, and analyses will determine if 
changes in parenting behavior outcomes are 
statistically significant.  
This study is interested in whether participation in the 
FRAMEWorks Program is associated with improved 
outcomes for participants, so a descriptive study of within 
subject change is sufficient for measuring differences in 
participant outcomes over time.  
R4 
For this descriptive evaluation, healthy partner 
relationship attitudes will be measured at the time 
of program enrollment and again one year after 
program enrollment, and analyses will determine if 
changes in parenting behavior outcomes are 
statistically significant.  
This study is interested in whether participation in the 
FRAMEWorks Program is associated with improved 
outcomes for participants, so a descriptive study of within 
subject change is sufficient for measuring differences in 
participant outcomes over time.  
R5 
For this descriptive evaluation, financial readiness 
behaviors will be measured at the time of program 
enrollment and again one year after program 
enrollment, and analyses will determine if changes in 
parenting behavior outcomes are statistically 
significant.  
This study is interested in whether participation in the 
FRAMEWorks Program is associated with improved 
outcomes for participants, so a descriptive study of within 
subject change is sufficient for measuring differences in 
participant outcomes over time.  
R6 
For this descriptive evaluation, job readiness 
behaviors will be measured at the time of program 
enrollment and again one year after program 
enrollment, and analyses will determine if changes in 
parenting behavior outcomes are statistically 
significant.  
This study is interested in whether participation in the 
FRAMEWorks Program is associated with improved 
outcomes for participants, so a descriptive study of within 
subject change is sufficient for measuring differences in 
participant outcomes over time.  
R7 
For this descriptive evaluation, the study is 
interested in exploring how service format may 
moderate participant outcomes by looking at 
differences between virtual vs. in-person format 
groups. 
AVANCE-Houston has delivered the SSHF curriculum in-
person to participants in the past but virtually for the first 
time this grant cycle. They are interested in exploring 
differences in outcomes between participants who receive 
the curriculum virtually vs. in-person. 
B. Methods to develop study groups  
Because this is a descriptive evaluation, this subsection is not relevant. 
C. Sample 
Recruitment into the FRAMEWorks Program and the study rely heavily on referrals from past 
participants and community partners who serve eligible participants in their respective target 
populations. Referrals will come from the AVANCE-Houston website that presents available 
programs and services, advertising by AVANCE-Houston staff about the program, and word of 
mouth from current and past participants. Recruitment targets largely low-income parents or 
individuals in the Houston area who are: at least 18 years of age with no open criminal cases 
Commented [DF1]: It might be good to note your sample 
size here. I know it is in your diagram, but it could be noted 
here as well. 

 
 
11 
(can be deferred); have children up to 24 years; and are interested in services (see Table 6 
below). 
Table 6: Target population for analytic sample 
Description of 
Target Population 
How is the population 
different from those 
who will be broadly 
served by the grant? 
How will the target population be identified? 
Unit of Analysis 
Adults who are 
primarily low-
income and 18+ 
years of age 
No difference, all 
program participants 
will be offered a 
chance to enroll in the 
evaluation study. 
Those who do not are 
still eligible for the 
program. 
AVANCE and community partners will recruit 
potential participants through informational flyers and 
targeted recruitment at community events (e.g., open 
houses, career fairs, Head Start centers, etc.). 
AVANCE program staff will determine eligibility for 
program participation. 
Individual 
 
Staff present the purpose and benefits of program services at orientations held at partner sites 
to recruit eligible parents. Orientations also discuss the descriptive study and explain informed 
consent before soliciting study participation. Participant responsibilities are clarified at the 
orientation, such as providing contact information and responding to surveys. 
 
 
 
Table 7: Methods to Promote Sufficient Program Participation 
What methods will you use to 
ensure sufficient sample is 
recruited, enrolls, and 
participates in the program? 
AVANCE and their community partners will recruit program participants, and AVANCE 
will provide incentives for program retention and completion. The CQI process will 
address issues regarding program recruitment and enrollment to ensure targets are met. 
Who will be responsible for 
recruiting the evaluation 
sample? 
The evaluation sample may differ from the program population, in that all participants will be 
invited to participate in the evaluation. Any participant who declines to enroll in the 
evaluation will still be eligible for the program. Enrollment into the evaluation will be 
conducted by the CQI Data Manager who will conduct the informed consent process and 
proctor the baseline data collection efforts. 
Please describe any incentives 
to be offered for program 
Program Participation – AVANCE will provide a series of incentives for program retention 
(promotional items) and completion ($100 gift card). 

 
 
12 
participation and/or completion 
and/or data collection and/or 
participation in the evaluation. 
Evaluation Participation – No incentives will be used for baseline data collection. A $50 
incentive will be used for the 1-year follow-up survey and will be provided in the form of a 
Walmart or PayPal gift card. 
 
D. Lead staff 
Evaluators for this study comprise an experienced group with many years of experience 
investigating HMRF Projects. Each staff member has a specific role in the evaluation project as 
summarized below in Table 8 and subsequent discussion (see CVs in Appendix B for more 
details on the qualifications of evaluation staff for this project). 
Table 8: MER Evaluation Staff 
Name 
Organization 
Role in the Evaluation 
Dr. Matthew Shepherd 
Midwest Evaluation and Research 
Principal Investigator 
McKenna LeClear 
Midwest Evaluation and Research 
Lead Evaluation Consultant 
Jennifer Leveille, MPH 
Midwest Evaluation and Research 
Evaluation Project Manager 
Deepika Bhatt, MPH 
Midwest Evaluation and Research 
CQI Data Manager 
 
Dr. Matthew Shepherd will serve as the Principal Investigator for this grant. As such, he has 
corporate responsibility for all evaluation activities. Dr. Shepherd has over 25 years’ 
experience in program design and implementation, applied research, program evaluation, 
policy analysis, and evaluative technical assistance. 
McKenna LeClear will serve as the Lead Evaluation Consultant and provide day-to-day 
oversight for the HMRF evaluation activities. Prior to joining MER, Ms. LeClear managed 
evaluation projects as a Senior Research Associate with the Child and Family Research 
Partnership at the University of Texas at Austin. 
Jennifer Leveille will serve as the Evaluation Project Manager. The Evaluation PM leads 
the effort to conduct a descriptive study and a Continuous Quality Improvement (CQI) process 
for the grant. Prior to joining MER, Mrs. Leveille worked as a Senior Research Assistant with 
MD Anderson Cancer Center and as a Public Health Consultant with Rice University’s Texas 
Policy Lab. 
Deepika Bhatt will serve as the CQI Data Manager. The CQI Data Manager will be responsible 
for accurate and timely data collection, report generation, and assistance with Continuous 
Quality Improvement (CQI) throughout the process of the grant. Prior to working with MER, 
Deepika worked as Research, Monitoring, and Evaluation (RME) Officer at Sunaulo Parivar 
Nepal/ Marie Stopes International Nepal. 

 
 
13 
E. Ongoing coordination between grantee and local evaluator 
The basis for ongoing coordination between AVANCE (the grantee) and MER (the local 
evaluator) is regular communication, by way of recurring meetings and daily interactions with 
embedded staff. Throughout the original proposal process, and now during the evaluation 
planning phase, MER worked in consort with AVANCE to design a study with research 
questions that are appropriate to the intervention. MER guides the process, given our 
experience designing and running evaluations, and AVANCE provides expertise on their 
community, target population, and program/curricula specifics. 
Recurring meetings will include a bi-weekly project CQI team meeting. Under the leadership 
of the Data Manager and Lead MER Evaluator, the CQI team reviews data from the nFORM 
and local evaluation systems to identify and mitigate implementation or data issues, and 
closely examine trends and accomplishments. This team includes AVANCE organizational and 
project leadership, the MER Evaluation team, and front-line staff representatives (e.g., 
Program Administrators, Case Managers).  
In addition to CQI team meetings, overall project team meetings occur monthly (at a 
minimum), with project leaders across MER and AVANCE in attendance, to ensure the 
partnership remains strong and that coordination across organizations is on track. This 
recurring, ongoing meeting structure is conducive to close coordination, ensuring that 
challenges can be quickly addressed, and promising strategies can be efficiently maximized. 
One of the key components of this coordination effort is the CQI Data Manager, who is a MER 
employee, embedded with AVANCE. The CQI Data Manager functions to bridge the gap 
between organizations. They will interact with AVANCE staff daily while completing their job 
duties and play a leadership role in the recurring meetings outlined above. See Section II.D 
above for more details about this role and others. Both the meetings and the roles outlined 
above will continue throughout the entire project period, providing opportunities to ensure the 
rigor and relevance of the evaluation and its findings, and to discuss and coordinate 
dissemination efforts (which will also be shared across MER and AVANCE).  
MER has experience operating prior descriptive evaluations using this exact process. Clearly 
outlining roles and responsibilities maintains the independence of the evaluation. That is, the 
evaluation team helps identify and illuminate areas of concern or improvement (for the 
program and the evaluation), but the program staff have responsibility for implementing 
improvements and providing direct services to participants. In this way, AVANCE and MER 
acknowledge our shared interest in and responsibility for a well-executed project and 
evaluation, but that MER is also an independent and external organization with a high level of 
integrity and is not responsible for nor invested in the specific outcomes of the program. This 
allows for close coordination without allowing for co-dependence, or for personal interests to 
influence evaluation findings. 

 
 
14 
III. Data Collection 
A. Constructs and Measures 
 
Table 9 below summarizes the constructs of interest, measures to evaluate those constructs, and 
specific data collection instruments.  
Table 9: Constructs and measures/data collection instruments 
Construct 
Measure and description 
Instrument 
Reliability and Validity 
(if standardized 
instrument, you provide a 
citation for the instrument) 
Parenting 
Attitudes 
4 items: levels of agreement for parent 
relationship status best for child 
(categories, 4-point scale) 
OLLE (Q1-4) 
Psychometric analyses of 
parenting attitudes construct 
completed in October 2021 
(a = .862) 
Partner 
Relationship 
Attitudes 
5 items: levels of agreement with 
attitudes about relationship (categories 
on 5-point scale) 
Healthy Marriage Adult 
Program Survey (D10.a-e) 
nFORM 
Parenting 
Behaviors 
4 items: frequency supporting child 
(categories on 4-point scale) 
Healthy Marriage Adult 
Program Survey (A2a.a-d) 
nFORM 
Partner 
Relationship 
Behaviors 
3 items: frequency engaged in key 
behaviors with partner (categories, 4-
point scale) 
Healthy Marriage Adult 
Program Survey (D11.a-c) 
nFORM 
Financial 
Readiness 
Behaviors 
2 items: have checking/savings account 
(yes/no) 
Healthy Marriage Adult 
Program Survey (B3a-b) 
nFORM 
Job Readiness 
Behaviors 
1 item: do you have an update resume 
you can give to employers (yes/no) 
Healthy Marriage Adult 
Program Survey (B2) 
nFORM 
B. Consent 
 
Because the planned evaluation involves human subjects, AVANCE understands program 
implementation requires both IRB approval and participant informed consent. MER has an 
established relationship with Solutions IRB, having secured more than 20 IRB approvals and 
renewals for evaluations it has conducted during the past five years. IRB clearance will be 
submitted for approval and obtained during the planning period.  

 
 
15 
To secure informed consent, participants will attend an orientation/enrollment session where 
the Data Manager will describe the evaluation process and the risks and benefits of the project. 
At this point in time, participants have not yet taken entrance surveys (including the ACS) and 
are therefore not yet enrolled in the program or the study. Enrollment immediately follows 
consent. Those participants that wish to participate in the evaluation will complete an informed 
consent process and affirm their consent on the OLLE Entrance Survey in the form of a survey 
question. All participants will receive a copy of the consent form with contact information for 
evaluation staff and how to contact the IRB if needed. Informed consent will take place 
prior to study enrollment and the collection of evaluation data. Those participants who 
initially indicate they do not wish to participate in the evaluation study will be referred to 
a family coach who will work with that person to explain study benefits. If they still refuse 
to enroll in the evaluation study, they are still eligible to participate in the program and 
receive services. 
C. Data Collection 
Implementation study data from nFORM will be collected bi-weekly and reviewed with regard 
to enrollment, retention, participation, survey completion, etc. This data is updated in real time 
by the evaluator and reviewed with grantee staff during scheduled bi-weekly CQI meetings. 
These meetings are designed to keep the evaluator informed about project updates, progress 
toward program targets, and to discuss any interventions that might need to be developed. 
In addition to the bi-weekly CQI meetings between the evaluator and the grantee staff, MER 
will conduct a yearly site visit either in person or virtually. Evaluation staff will meet with 
program leadership, support staff (facilitators, family coaches, career coaches, etc.), and 
participants (past and present if possible) in an interview style setting. Questions will aim to 
seek information about how the program is progressing, what we have learned so far, if there 
are issues the evaluation staff can identify that might need to be addressed, and to provide 
guidance on how to run the program better. MER will create a report of the information 
discussed in the site visit that will be shared with program staff and the program ETAP. 
Survey administration procedures collect data for outcomes assessment that are carried out by 
case managers under supervision of the CQI Data Manager. Evaluation data collection occurs 
at two time points – nFORM and OLLE entrance surveys at wave 1 before primary workshops 
begin and the 1-year follow-up OLLE survey at wave 2 one year after program enrollment. 
Table 10 below presents the timing of collection for each wave. 
Table 10: Timeline 
Wave of Data Collection  
(e.g., baseline, short-term follow-up, long-
term follow-up) 
Timing of Data Collection 
 
Entrance (nFORM and OLLE) 
Collected immediately after informed consent and enrollment – during 
orientation or first workshop. 

 
 
16 
One-year follow-up (1 year post-enrollment, 
OLLE only) 
Collected approximately one year after project enrollment. MER’s survey 
tracking team will reach out 60-days prior to the one year mark and will stop 
attempting to collect follow-up surveys 30 days after the due date. 
 
Table 11: Measures 
Measure 
Timing of Data 
Collection 
(baseline, wave of 
data collection) 
Method of Data 
Collection 
Who Is Responsible for 
Data Collection? 
Will 
Methods or 
Collection 
Procedures 
Differ by 
Study 
Group? 
Administrative 
Data Only: 
Will data 
access require 
data sharing 
agreement? 
nFORM Applicant 
Characteristics 
(ACS) Survey  
At enrollment 
Participant self-enters 
survey using online data 
collection platform 
(nFORM) 
AVANCE staff will 
proctor data collection 
and assist participants as 
necessary 
No 
N/A 
On-line Local 
Evaluation (OLLE) 
and nFORM 
Entrance Survey 
At enrollment 
Participant self-enters 
survey using online data 
collection platforms 
(Qualtrics and nFORM, 
respectively) 
AVANCE staff will 
proctor data collection 
and assist participants as 
necessary 
No 
N/A 
OLLE One-Year 
Follow-up Survey 
1 year after 
enrollment / 
baseline 
Participant self-enters 
survey using online data 
collection platform and 
link – or – Phone 
interview data collection  
MER Research Staff/ 
participant tracking 
team 
No 
N/A 
 
Ensuring and monitoring high quality data collection 
This evaluation will utilize the nFORM and OLLE entrance surveys at enrollment and the 
OLLE 1-year follow-up surveys collected one year after enrollment. The data collection 
methods for these time points differ, as shown in Table 11 above and as discussed further in 
Table 12 below. 
All program staff and evaluation staff will undergo a rigorous set of trainings to prepare for the 
evaluation. All staff receive an overview and introductory training to present the goals and 
objectives of the evaluation effort and its importance to the overall project. Next, all staff 
receive training on human subject protection and are required to pass a certification test on the 
subject matter. All staff will also receive a detailed training on the evaluation, including the 

 
 
17 
evaluation tools, timing and data collection process, and the role and importance of 
randomization of participants. 
In addition, the data manager and the primary local evaluation staff will undergo a rigorous 
training process to better understand the context of HMRF research, training on data collection 
procedures they will be responsible for, and training on the nFORM system and use of nFORM 
data in a CQI process. MER is creating networks of CQI data managers and Evaluation Project 
Managers across the 12 projects that we are evaluating so that all staff have access to 
experienced data managers and evaluation staff who have done this work previously. This 
training takes the form of weekly training sessions that are currently underway. 
Members of the CQI team will also receive specific training on the MER CQI process that has 
been developed prior to the launch of data collection or program services. As described 
elsewhere, MER is assisting the program staff in implementing a robust CQI process that will 
focus on retention as one of the primary areas of program improvement, and as such, we are 
anticipating relatively modest levels of attrition for this data collection. 
On a bi-weekly basis, the data manager, the local evaluation staff, and MER technical 
specialists will be responsible for downloading data from the nFORM and MER On-Line Local 
Evaluation (OLLE) systems for processing and presentation to the CQI team for tracking and 
monitoring performance measurement outcomes (recruitment, enrollment, dosage, completion, 
referrals, etc.) so that near real-time adjustments can be made to program implementation to 
ensure compliance with program goals and objectives. 
All MER training is currently being recorded, and as new staff come on board with projects or 
project staff turnover (or need refresher training), recorded training material can be shared and 
accessed with follow-up one on one training with the primary local evaluator and the MER 
LOB Lead. 
Table 12: Tracking participants and reducing attrition 
For each wave of data 
collection listed in IIIA.2, what 
is your estimated response 
rate?  
Entrance 
100% - done at the time of enrollment 
Follow-Up 
70% - estimated 1 year follow-up rate