Gentle Warriors Academy Program Evaluation
Phase: NA
Conditions: Fathers
Sponsor: Midwest Evaluation & Research

Gentle Warriors Academy Healthy 
Marriage and Responsible 
Fatherhood Program Evaluation 
 
4/22/2024 
 
Clinicaltrials.gov ID: NCT06899100 
 
Study Protocol 
 
 

Fatherhood FIRE Evaluation Plan  
Page | 2 
 
PROGRAM BACKGROUND 
1. PROGRAM SUMMARY 
Please provide a brief summary of your grant project including the needs to be addressed, 
the services provided, and the population served. 
Gentle Warriors Academy (GWA) is designed to help urban community adult fathers age 18 and 
older with dependent child(ren) up to 24 years of age incorporate practices and behaviors for 
managing the responsibilities of being parents/co-parents and partners through responsible parting, 
healthy marriage/relationships, economic stability activities, and grant-funded participation supports 
through a 40-hr/10-week cohort-based approach. These core areas are reinforced through GWA’s 
coordinated case management process and wrap-around services, which is tailored to help 
address obstacles that fathers and their families face on their journeys to attaining and maintaining 
healthy parental and partner relationships as well as program completion.  
 
 
Fathers Incorporated utilizes a skills-based curricula for its core GWA services: Next Level 
Fatherhood which was created and tailored by subject matter experts and Fathers Incorporated 
practitioners/staff to address the complex challenges and needs of urban community fathers. All 
curricula consist of 10 modules of learning that are designed to develop practical and 
comprehensive skillsets that are transferrable throughout all aspects of life.  
 
 
The objectives of the Next Level Fatherhood curriculum are to: 1. Provide participants with practical 
skills necessary to navigate critical aspects of responsible fatherhood, healthy marriages, and 
personal/family economic stability; 2. Provide participants with tools that will assist them in 
strengthening their marriages, child engagement, and co-parenting relationships, and to provide 
information and resources around viable economic stability for themselves and their families; 3. 
Provide participants with skills necessary to making better and more informed parenting choices, to 
be more pro-active and less reactive, and to strengthen parental relationships; and 4. Encourage 
participants to become more acutely aware of how they present themselves in the lives of their 
children, significant others, an in the “systems” (family court, judicial court, child support, state 
agencies etc.) in which they have to navigate.  
 
 
Participating fathers will work with Case Managers to review the results of all assessments and to 
determine necessary support services through the duration of program participation. Case 
Managers will implement case management best practices, record case notes that document 
interaction/support provided to all participants and will ensure that each participant receives six 
hours of dedicated case management from program enrollment through completion. Certified Life 
Coaches will work in tandem with Case Managers to help participants obtain and reach their goals, 
promote, and develop their strengths, and to overcome difficult challenges, specifically within their 
marriages or other significant relationships, their economic viability and their parental 
responsibilities as identified through co-developed Individual Success Plans (ISPs).  
 
 
2. EVALUATION GOALS 

Fatherhood FIRE Evaluation Plan  
Page | 3 
Please briefly describe key goals of your evaluation and what you hope to learn below. 
In this evaluation, it is anticipated that participants will show an increase in: 
• 
Communications and handling conflict and other related relationship skills 
• 
Understanding in role as a father 
• 
Improved communication with partner/co-partner 
• 
Understanding the quality of father-child engagement 
• 
Various forms of contact with children 
 
 
3. EVALUATION ENROLLMENT  
Please provide the expected start and end dates for program and evaluation enrollment 
using the tables below. For impact studies, please indicate expected start and end dates 
for each study group. 
DESCRIPTIVE EVALUATION 
Please leave blank if not conducting a descriptive outcome evaluation. 
 
Program Enrollment 
Study Enrollment 
Start Date 
4/1/2021 
4/1/2021 
End Date 
7/1/2025 
3/31/2024 
Definition 
All program participants who enroll in 
services 
All program participants who consent to 
being part of the study 
 
 
 
 
 
 
 
 

Fatherhood FIRE Evaluation Plan  
Page | 4 
4. EVALUATION TIMELINE 
Please include a timeline for key activities of the evaluation below. Example of activities 
may include IRB submission, staff training, waves of data collection, analysis period, and 
report writing and submission. 
Evaluation Activity 
Start Date 
End Date 
Additional Evaluation staff hiring and 
onboarding 
1/15/2021 
2/15/2021 
Evaluation staff training 
1/22/2021 
2/20/2021 
Evaluation Kickoff meeting and orientation 
with all program staff 
2/2/2021 
2/3/2021 
IRB training and certification by all program 
and evaluation staff 
10/1/2020 
2/28/2021 
Development and submission of Evaluation 
Plan document 
1/15/2021 
2/19/2021 
Evaluation Tools Development 
10/1/2021 
2/28/2021 
IRB Approval 
3/1/2021 
3/15/2021 
IRB Full Board Approval 
05/11/2023 
06/01/2024 
IRB Approval for Incarcerated Population 
06/02/2023 
06/01/2024 
CQI Team Formed and Meeting bi-weekly 
3/15/2021 
7/1/2025 
Training for CQI Team 
3/15/2021 
3/28/2021 
Training for all Program Staff on Research 
Methods and process 
3/15/2021 
3/28/2021 
Implementation Evaluation Data Collection  
4/1/2021 
8/15/2025 
-Baseline 
4/1/2021 
7/1/2025 
-Post Test 
5/15/2021 
8/15/2025 
-Follow-up 
4/1/2021 
8/15/2025 
1st Manuscript submitted for publication 
7/1/2025 
9/30/2025 

Fatherhood FIRE Evaluation Plan  
Page | 5 
Final Descriptive Report Submitted 
9/1/2025 
9/29/2025 
* you may add rows by hitting the tab button, or right click and select insert row below 
 
EVALUATION PLAN 
1. RESEARCH QUESTIONS 
1.1. OVERVIEW OF RESEARCH QUESTIONS 
Please state the research questions(s) that the evaluation intends to answer and for each 
research question indicate the type: implementation or outcome.  
o Implementation Questions: Identifying whether a program has been successful 
in attaining desired implementation goals (e.g., reaching intended target 
population, enrolling intended number of participants, delivering training and 
services in manner intended, etc.)  
o Outcome Questions: Identifying whether program is associated with intended 
outcomes for participants (e.g., do participants’ knowledge, attitudes, behaviors, 
or awareness change?)   
Research questions in this study are framed by a Descriptive Evaluation Design to 
assess whether outcomes improve for low-income fathers who participate in the Gentle Warriors 
Academy (GWA) Project. Primary outcomes are assessed before and after participants 
complete core curricula—Next Level Fatherhood —and Case Management and Certified Life 
Coaching as support services (pre to post). Primary outcomes will indicate whether behavior 
improved for healthy family relationships (parent, co-parent, and partner). See Table 1 below. 
No 
Research Question 
Implementation or 
Outcome? 
R1 
Will participants report significantly healthier parenting 
behavior after completing primary educational services and 
employment support services? 
Primary Outcome 
R2 
Will participants report significantly healthier parenting 
attitudes after completing primary educational services and 
employment support services? 
Primary Outcome 

Fatherhood FIRE Evaluation Plan  
Page | 6 
R3 
Will participants report significantly healthier co-parenting 
behavior after completing primary educational services and 
employment support services? 
Primary Outcome 
R4 
Will participants report significantly healthier partner 
relationship behavior after completing primary educational 
services and employment support services? 
Primary Outcome 
* you may add rows by hitting the tab button, or right click and select insert row below 
 
1.2. OUTCOME RESEARCH QUESTIONS 
For each outcome research question listed above, whether a descriptive or impact design, 
summarize the inputs (e.g., program components, program supports, implementation 
features, etc.), target population (e.g., the population for which the effect will be estimated) 
and the outcomes (e.g., child well-being, father-child engagement, etc.) that will be 
examined to answer the research question(s). Comparisons for descriptive evaluations may 
reflect circumstances before the grant, pre-treatment, or pre-determined benchmark from 
other studies with similar interventions. 
 
Research 
Question 
Number 
Should 
correspond 
to the 
number 
indicated 
in Table 
1.1 above 
 
Intervention 
Program 
component or 
set of 
activities that 
the 
evaluation 
will test or 
examine 
Target 
Population 
Population 
for which the 
effect of the 
treatment will 
be estimated 
Comparison  
What the 
intervention 
will be 
compared to 
(e.g., pre-
intervention 
for descriptive 
designs) 
Outcome 
Changes 
that are 
expected to 
occur as a 
result of the 
intervention  
Confirmatory  
or 
Exploratory? 
Confirmatory: 
those upon 
which 
conclusions 
will be drawn 
Exploratory: 
those that 
may provide 
additional 
suggestive 
evidence 
R1 
Primary 
curricula 
(Next Level 
Fatherhood); 
Support 
Services 
(Case 
Management, 
Life 
Coaching); 
Urban fathers 
who are 18+ 
years with 
children up to 
24 years 
Assessment of 
healthy 
parenting 
behavior at 
GWA Project 
enrollment. 
Participants 
who 
complete 
primary and 
support 
services will 
report 
healthier 
parenting 
behavior 
Confirmatory 

Fatherhood FIRE Evaluation Plan  
Page | 7 
Continuous 
Quality 
Improvement 
(CQI) Process 
 
one year 
after GWA 
Project 
enrollment. 
R2 
Primary 
curricula 
(Next Level 
Fatherhood); 
Support 
Services 
(Case 
Management, 
Life 
Coaching); 
Continuous 
Quality 
Improvement 
(CQI) Process 
Urban fathers 
who are 18+ 
years with 
children up to 
24 years 
Assessment of 
healthy 
parenting 
attitudes at 
GWA Project 
enrollment. 
Participants 
who 
complete 
primary and 
support 
services will 
report 
healthier 
parenting 
attitudes 
one year 
after GWA 
Project 
enrollment. 
Confirmatory 
R3 
Primary 
curricula 
(Next Level 
Fatherhood); 
Support 
Services 
(Case 
Management, 
Life 
Coaching); 
Continuous 
Quality 
Improvement 
(CQI) Process 
Urban fathers 
who are 18+ 
years with 
children up to 
24 years 
Assessment of 
healthy co-
parenting 
behavior at 
GWA Project 
enrollment. 
Participants 
who 
complete 
primary and 
support 
services will 
report 
healthier 
co-
parenting 
behavior 
one year 
after GWA 
Project 
enrollment. 
Confirmatory 
R4 
Primary 
curricula 
(Next Level 
Fatherhood); 
Support 
Services 
(Case 
Management, 
Life 
Coaching); 
Continuous 
Quality 
Urban fathers 
who are 18+ 
years with 
children up to 
24 years 
Assessment of 
healthy 
partner 
relationship 
behavior at 
GWA Project 
enrollment 
Participants 
who 
complete 
primary and 
support 
services will 
report 
healthier 
partner 
relationship 
behavior 
one year 
after GWA 
Confirmatory 

Fatherhood FIRE Evaluation Plan  
Page | 8 
Improvement 
(CQI) Process 
Project 
enrollment. 
* you may add rows by hitting the tab button, or right click and select insert row below 
2. BACKGROUND 
For each outcome research question listed in 1.1, whether descriptive or impact design, briefly 
summarize the previous literature or existing research that informs the stated research question 
and how the evaluation will expand the evidence base. Explain why the research questions are 
of specific interest to the program and/or community. Only a short summary paragraph 
description is needed below. Additional documentation, such as a literature review, may be 
appended to this document. 
Research 
Question 
Topic 
Existing Research 
Contribution to the 
Evidence Base 
Interest to the 
Program and/or 
Community 
R1 and 
R2 
Low-income fathers can 
lack the parenting 
skills to fulfill their 
familial obligations as a 
parent to promote 
healthy family 
relationships. 
Determine the extent to 
which the delivery of 
GWA Project services 
enhance the parenting 
skills of low-income 
fathers. 
Inform practitioners 
about whether GWA 
Project design 
promotes healthy family 
relationships by 
improving parent 
behavior. 
R3 
Low-income fathers can 
lack the co-parenting 
skills to fulfill their 
familial obligations as a 
parent to promote 
healthy family 
relationships. 
Determine the extent to 
which the delivery of 
GWA Project services 
enhance the co-
parenting skills of low-
income fathers. 
Inform practitioners 
about whether GWA 
Project design 
promotes healthy family 
relationships by 
improving co-parent 
behavior. 
R4 
Low-income fathers can 
lack the relationship 
skills to fulfill their 
obligations as a partner 
to promote healthy 
family relationships. 
Determine the extent to 
which the delivery of 
GWA Project services 
enhance the partner 
relationship skills of 
low-income fathers. 
Inform practitioners 
about whether GWA 
Project design 
promotes healthy family 
relationships by 
improving partner 
relationship behavior. 
* you may add rows by hitting the tab button, or right click and select insert row below 
 
 
 

Fatherhood FIRE Evaluation Plan  
Page | 9 
3. HYPOTHESES 
For each specified research question, state the hypothesized result(s) and briefly describe why 
these results are anticipated. 
Research 
Question 
Hypothesized Result  
R1 
Participants will report healthier parenting behavior after the complete primary 
services and employment support services in the GWA Project. 
R2 
Participants will report healthier parenting attitudes after the complete primary 
services and employment support services in the GWA Project. 
R3 
Participants will report healthier co-parenting behavior after the complete 
primary services and employment support services in the GWA Project. 
R4 
Participants will report healthier partner relationship behavior after the 
complete primary services and employment support services in the GWA Project. 
* you may add rows by hitting the tab button, or right click and select insert row below 
4. RESEARCH DESIGN 
For each research question, briefly describe why the research design proposed will answer 
each research question(s). State whether the proposed evaluation is a descriptive or impact 
evaluation and justify why the proposed research design is best suited to answer the research 
question(s).  
Research 
Question 
Design 
Justification 
R1 
Pre to post outcomes 
assessment of GWA 
participation for healthy 
parenting behavior.  
Healthy change reported by participants in 
their parenting behavior from pre to post 
estimates the maximum amount of benefit that 
can be attributed to GWA services in the 
absence of a counterfactual. 
R2 
Pre to post outcomes 
assessment of GWA 
participation for healthy 
parenting attitudes.  
Healthy change reported by participants in 
their parenting attitudes from pre to post 
estimates the maximum amount of benefit that 
can be attributed to GWA services in the 
absence of a counterfactual. 
R3 
Pre to post outcomes 
assessment of GWA 
participation for healthy 
co-parenting behavior.  
Healthy change reported by participants in 
their co-parenting behavior from pre to post 
estimates the maximum amount of benefit that 

Fatherhood FIRE Evaluation Plan  
Page | 10 
can be attributed to GWA services in the 
absence of a counterfactual. 
R4 
Pre to post outcomes 
assessment of GWA 
participation for healthy 
partner relationship 
behavior.  
Healthy change reported by participants in 
their partner relationship behavior from pre 
to post estimates the maximum amount of 
benefit that can be attributed to GWA services 
in the absence of a counterfactual. 
* you may add rows by hitting the tab button, or right click and select insert row below 
5. ONGOING GRANTEE AND LOCAL EVALUATOR COORDINATION 
Describe how the grantee and local evaluator collaboratively worked together to identify the 
research question(s) and research design to ensure its feasibility and relevance. Describe how 
the grantee and local evaluator will continue to work together throughout the evaluation to 
proactively address unforeseen challenges as they arise and ensure the rigor and relevance of 
the evaluation and its findings. Describe how the grantee and local evaluator will coordinate 
dissemination efforts. Describe how these processes will occur while maintaining the 
independence of the evaluation. 
Fathers Incorporated and the local evaluator, Midwest Evaluation and Research (MER) began 
the process of collaboration prior to the release of the Funding Opportunity Announcement 
(FOA) for this project.  This included multiple working meetings to discuss the project and the 
goals that Fathers Incorporated had for the project, with a specific emphasis on what they had 
learned as a funded OFA grantee over the last two cohort cycles and what they would like to 
do differently.  This included a discussion of enhancements to the program they believed 
would enhance participant outcomes, and the resource requirements of these enhancements. 
Several viable and relevant research question options were developed prior to the FOA and 
when the FOA was released the team had additional meetings to discuss the program that 
would be proposed to meet the goals and objectives of Fathers Incorporated and OFA and how 
well the research question alternatives developed in advance worked with this program 
outline.  Using this extended collaborative process, the research questions were designed that 
were most relevant and feasible. 
The basis for ongoing coordination between Fathers Incorporated and MER is regular 
communication.  This will be accomplished by the way of bi-weekly meetings and daily 
interactions between staff.  Fathers Incorporated and MER have established a set of standing 
calls and meetings which begam in the first quarter of funding.  This includes a bi-weekly 
(every other week) meeting of the CQI team.  This is a team that is led by the CQI Data 
Manager and MER Lead Evaluator will discuss program implementation, data from the nFORM 
and local evaluation systems and closely examine trends and accomplishments (see CQI 
section for more detail on these meetings).  This team includes Fathers Incorporated 
organizational and project leadership, the MER Evaluation team, and front-line staff in the form 
of Case Manager and Facilitator representatives.   
In addition to this CQI team, a monthly meeting of program and organization leaders and MER 
staff will occur the ensure the partnership and coordination is on track and running smoothly. 
Regular contact allows close coordination.  Perhaps the greatest asset to this coordination 
effort is the CQI Data Manager which is a shared and co-funded staff person that is embedded 

Fatherhood FIRE Evaluation Plan  
Page | 11 
with Fathers Incorporated staff and acts as staff for both MER and Fathers Incorporated-
bridging the gap between the organizations.  
This person will interact with program staff daily while completing the duties of a CQI Data 
Manager.  These meetings and staffed position will continue during the entire project period 
and will include addressing opportunities for dissemination (shared efforts with project staff) 
and ensuring the rigor and relevance of the evaluation and its findings.    
MER has operated prior RCT evaluations with this exact process, by clearly outlining roles and 
responsibilities – such as the evaluation team helps identify and illuminate areas of concern or 
improvement (for the program and the evaluation), but the program staff have responsibility for 
implementing program improvement and for providing direct services to participants. 
Fathers Incorporated and MER share a responsibility for a well-executed project and 
evaluation, but MER is also an independent and external organization that has high integrity 
and is not responsible in the outcomes of the program.  This allows for a high level of 
coordination without allowing any co-dependence or for personal interests to influence 
evaluation findings.  This was well demonstrated in MER’s last evaluation which found the 
program enhancements being studied ineffective in improving outcomes. 
 
 
6. LEAD STAFF 
Define the roles of lead staff for the evaluation from both organizations below. 
Name 
Organization 
Role in the Evaluation 
Dr. Matthew Shepherd 
Midwest Evaluation and 
Research 
Principal Investigator 
McKenna LeClear 
Midwest Evaluation and 
Research 
Senior Research Consultant 
Dr. Saloni Sharma 
Midwest Evaluation and 
Research 
Evaluation Project Manager 
Darris McKenzie 
Fathers Incorporated 
CQI Data Manager 
* you may add rows by hitting the tab button, or right click and select insert row below 
Articulate the experience, skills, and knowledge of the staff for the evaluation (including whether 
they have conducted similar studies in this field), as well as their ability to coordinate and 
support planning, implementation, and analysis related to a comprehensive evaluation plan. 

Fatherhood FIRE Evaluation Plan  
Page | 12 
Dr. Matthew Shepherd will serve as the Principal Investigator for this grant.  As such, he has 
corporate responsibility for all evaluation activities.  Dr. Shepherd has over 25 years’ 
experience in program design and implementation, applied research, program evaluation, policy 
analysis, and evaluative technical assistance.   
 
McKenna LeClear will serve as the Senior Research Consultant and provide oversight for HMRF 
evaluation activities as needed. Ms. LeClear has 5 years of evaluation research experience and 
oversees numerous other HMRF evaluations for MER.  
 
Dr. Saloni Sharma will serve as the Evaluation Project Manager and will lead the effort to conduct a 
descriptive study and a Continuous Quality Improvement (CQI) process for the grant. Dr. Sharma 
has over 5 years or experience in Public Policy research prior to her time at MER. 
Darris McKenzie will serve as the CQI Data Manager. He possesses the managerial and 
analytic skills to successfully serve in this position. The CQI Data Manager will work closely with 
the grantee and community partners on-site to complete data collection and management activities 
for the descriptive study and a Continuous Quality Improvement (CQI) process. 
 
7. SAMPLE 
7.1. TARGET POPULATION(S) 
For each target population identified in Section 1.2, please describe the target population(s), 
and explicitly state whether the population(s) differs from those who will be broadly served by 
the grant. Describe how the target population will be identified. Explicitly state the unit of 
analysis (e.g., non-residential father, unmarried couple). 
Description of 
Target Population 
How is the 
population different 
from those who will 
be broadly served by 
the grant? 
How will the target 
population be identified? 
Unit of 
Analysis 
Urban fathers age 
18 years and older 
with children who 
are up to 24 years 
of age 
No difference, all 
program participants 
will be study 
participants 
The sample will be 
identified and recruited by 
the program staff.  
Urban fathers 
 
7.2. METHODS TO PROMOTE SUFFICIENT PROGRAM PARTICIPATION 
Please describe methods to promote sufficient program participation in the table below. 
 
8. DATA COLLECTION 

Fatherhood FIRE Evaluation Plan  
Page | 13 
8.1. CONSTRUCTS AND MEASURES 
Clearly articulate the constructs of interest, measures to evaluate those constructs, and specific 
data collection instruments. Provide any information on the reliability and validity of the data 
collection instruments. For standardized instruments, you may provide the citation for the 
instrument. 
Construct 
Measure  
Instrument 
Reliability and Validity 
(if standardized instrument, you 
provide a citation for the instrument) 
Parenting 
Attitudes 
7 items: 
frequency of key 
attitudes 
(categories, 5-
point scale) 
nFORM 
Community-
Based Fathers 
Survey (A3: a-g) 
nFORM 
Parenting 
Behavior 
1 item: hours 
spent w/ children 
in last 30 days 
(interval); 1 item: 
frequency reach 
out to children 
(categories, 5 
point scale); 7 
items: frequency 
engage in key 
behaviors 
(categories, 5-
point scale) 
nFORM 
Community-
Based Fathers 
Survey (A2b, 
A2c, A5b: b-d, f-
i) 
nFORM 
Co-Parenting 
Behavior 
11 items: 
frequency of 
agreement with 
key co-parenting 
behaviors 
(interval, 5-point 
scale) 
nFORM 
Community-
Based Fathers 
Survey (A13: a-
k) 
nFORM, OLLE 
Healthy 
Partner 
Behavior 
1 item: feelings of 
being 
overwhelmed as 
parent 
(categories, 4 
point scale)  
nFORM 
Community-
Based Fathers 
Survey (B3 – 
B3a) 
nFORM  
* you may add rows by hitting the tab button, or right click and select insert row below 
 

Fatherhood FIRE Evaluation Plan  
Page | 14 
8.2. CONSENT 
Describe how and when program applicants will be informed of the study and will have the 
option of agreeing (i.e., consenting to) or declining to participate in the study. 
Fathers Incorporated staff will make initial contact with interested applicants. They will 
describe the study and answer any questions at that time. If the applicant is interested in the 
project, he will be invited to an enrollment session where the consent form will be presented. 
Fathers Incorporated staff will ensure that the applicant understands the consent form, remind 
them that participation is voluntary, and that they can drop out of the project at any time for 
any reason without consequences. Informed consent will take place prior to program 
enrollment and the collection of evaluation data.  
 
8.3. METHODS OF DATA COLLECTION 
If the evaluation will collect multiple waves of data, describe the timing of these waves below. 
When describing follow-up periods, specify whether the follow-up period will be post-baseline, 
post-random assignment, or post-program completion. 
Wave of Data Collection  
(e.g., baseline, short-term follow-up, long-
term follow-up) 
Timing of Data Collection 
 
Baseline 
Collected Immediately Following informed consent 
and enrollment – during and orientation session or 
first workshop. 
Post-Test 
Collected after the completion of core curriculum 
programing -during the last workshop session – 
approximately 13 weeks after enrollment. 
1 year Follow-up (post-baseline) 
Collected approximately one year after program 
enrollment/ baseline 
 
 
 
For each measure, describe how data will be collected detailing which data collection measures 
will be collected by which persons, and at what point in the programming or at what follow-up 
point. 

Fatherhood FIRE Evaluation Plan  
Page | 15 
Measure 
Timing of 
Data 
Collection 
(baseline, 
wave of data 
collection) 
Method of 
Data 
Collection 
Who Is 
Responsible 
for Data 
Collection? 
Administrative 
Data Only: 
Will data 
access 
require data 
sharing 
agreement? 
On Line Local 
Evaluation 
(OLLE) 
Baseline 
Survey 
Baseline 
Participant 
Self-Entered 
survey using 
online data 
collection 
program 
Data Manager 
will proctor 
data collection 
and assist 
participants as 
necessary 
No 
OLLE Post-
Test Survey 
Post-Test 
(approx. 13 
weeks after 
enrollment) 
Participant 
Self-Entered 
survey using 
online data 
collection 
program 
Data Manager 
will proctor 
data collection 
and assist 
participants as 
necessary 
No 
OLLE 1 Year 
Follow-up 
Survey 
1 year after 
enrollment / 
baseline 
Participant 
Self-Entered 
survey using 
online data 
collection 
program and 
link – or – 
Phone 
Interview data 
collection  
MER 
Research 
Staff/ 
participant 
tracking team 
No 
 
 
 
 
 
* you may add rows by hitting the tab button, or right click and select insert row below 
 
8.4.  ENSURING AND MONITORING DATA COLLECTION 
Describe plans for training data collectors and for updating or retraining data collectors about 
procedures. Detail plans to regularly review data that have been submitted and to assess and 
swiftly address problems. 

Fatherhood FIRE Evaluation Plan  
Page | 16 
This evaluation will utilize both post-program surveys (completed at the completion of core 
programing) and follow-up surveys collected one year after enrollment / baseline.  The methods for 
these data collections differ.  The primary driver for post-program survey completion is high rates of 
program retention.  This data point will be collected during the last workshop session – after (but 
during the same session of) the completion of the nFORM post-program data collection.  As such 
only those individuals who complete the program and who are at the data collection session will 
participate in the post-program data collection.   
All program staff and evaluation staff will undergo a rigorous set of trainings to prepare for the 
evaluation.  All staff receive an overview and introductory training to present the goals and 
objectives of the evaluation effort and its importance to the overall project. Next all staff receive 
training on human subject’s protection and are required to pass a certification test on the subject 
matter.  All staff will also receive a detailed training on the details of the evaluation including the 
evaluation tools, timing and data collection process.  
In addition, the data manager and the primary local evaluation staff will undergo a rigorous training 
process to better understand the context of HMRF research, training on data collection procedures 
they will be responsible for, and training on the nFORM system and use of nFORM data in a CQI 
process. MER is creating networks of CQI data managers and Evaluation Project Managers across 
the 12 projects that we are evaluating so that all staff have access to experienced data managers 
and evaluation staff who have done this work previously.  This training takes the form of weekly 
training sessions that are currently under way. 
Members of the CQI team will also receive specific training on the MER CQI process that has been 
developed prior to the launch of data collection or program services. As described elsewhere MER 
is assisting the program staff in implementing a robust CQI process that will focus on retention as 
one of the primary areas of program improvement and as such we are anticipating relatively 
modest levels of attrition for this data collection. 
On a bi-weekly basis the data manager, the local evaluation staff and MER technical specialists will 
be responsible for downloading data from the nFORM and MER On-Line Local Evaluation (OLLE) 
systems for processing and presentation to the CQI team for tracking and monitoring performance 
measurement outcomes (recruitment, enrollment, dosage, completion, referrals, etc.) so that near 
real time adjustments can be made to program implementation to ensure compliance with program 
goals and objectives.  
All MER training is currently being recorded and as new staff come on board with projects or 
project staff turn over (or need refresher training) recorded training material can be shared and 
accessed with follow-up one on one training with the primary local evaluator and the MER LOB 
Lead, who has more than 15 years’ experience in evaluating HMRF projects. 
 
 
 
 

Fatherhood FIRE Evaluation Plan  
Page | 17 
9. IRB/PROTECTION OF HUMAN SUBJECTS 
Please describe the process for protection of human subjects, and IRB review and approval of 
the proposed program and evaluation plans. Name the specific IRB to which you expect to 
apply.  
Because the planned evaluation involves human subjects, FI understands that program 
implementation requires both IRB approval and participant informed consent. MER has an 
established relationship with Solutions IRB MER has had several research studies approved by 
Solutions IRB, and has submitted timely amendments when changes to studies needed to take 
effect. Solutions IRB, a private commercial Association for the Accreditation of Human Research 
Protection Programs Inc. (AAHRPP) fully accredited Institutional Review Board, will ensure that this 
study is approved before any research activities take place.  
 
All submissions are completed online, so turnaround for a new study approval is between 24 to 72 
hours, though the full approval process can take approximately one to two weeks depending on the 
number of questions and requested revisions that the IRB makes. In the IRB application 
submission, we will include descriptions of project staff, locations of study sites, the funding source, 
incentives, summary of activities, participant population, recruitment plans, risks and benefits, 
confidentiality of data, and the informed consent process along with all materials to be used in the 
study such as participant forms and surveys.  
 
This project will be submitted for IRB approval in early March during the planning period in order to 
receive official approval to begin enrollment and data collection beginning in April.  
Program staff and facilitators will also obtain Human Subjects Training Certification through 
the IRB to ensure the protection of participants. 
 
 
10. DATA 
10.1. 
DATABASES 
For each database used to enter data, please describe the database into which data will be 
entered (i.e., nFORM and/or other databases), including both performance measure data you 
plan to use in your local evaluation and any additional local evaluation data. Describe the 
process for data entry (i.e., who will enter the data into the database).  
Database Name 
Data Entered 
Process for Data Entry 
nFORM 
Performance Measurement 
Data  
Entered directly by participants, 
and by program staff 
Qualtrics 
Local evaluation data, 
participant outcomes (OLLE 
survey) 
Entered directly by participants, 
and by MER evaluation staff 

Fatherhood FIRE Evaluation Plan  
Page | 18 
 
 
 
* you may add rows by hitting the tab button, or right click and select insert row below 
10.2. 
DATA REPORTING AND TRANSFER 
For each database provided in the table above, please indicate the ability to export individual-
level reports to an Excel or comma-delimited format and whether identifying information is 
available for linking to data from other sources. 
Database Name 
Ability to Export Individual 
Reports? 
What identifying information is 
available to facilitate linking to 
other data sources? 
nFORM 
Yes 
Client ID, name, DOB 
Qualtrics 
Yes 
Client ID, name, DOB 
 
 
 
* you may add rows by hitting the tab button, or right click and select insert row below 
10.3. 
CURRENT SECURITY AND CONFIDENTIALITY STANDARDS 
For each database provided in Section 11.1, please Indicate the ability to be able to encrypt 
data access during transit (for example, accessed through an HTTPS connection); be able to 
encrypt data at rest (that is, when not in transit), have in place a data backup and recovery plan; 
require all users to have logins and passwords to access the data they are authorized to view; 
and have current anti- virus software installed to detect and address malware, such as viruses 
and worms. 
Database Name 
Ability to 
encrypt 
data during 
transit?  
Ability to 
encrypt at 
rest?  
Data 
Backup and 
Recovery 
Plan? 
Require all 
users to 
have logins 
and 
passwords? 
Current Anti-
Virus Software 
Installed? 
nFORM 
Yes 
Yes 
Yes 
Yes 
Yes 
Qualtrics 
Yes 
Yes 
Yes 
Yes 
Yes 
 
 
 
 
 
 
* you may add rows by hitting the tab button, or right click and select insert row below 
 
 

Fatherhood FIRE Evaluation Plan  
Page | 19 
Please describe any plans for study registration with an appropriate registry (e.g., 
clinicaltrials.gov, socialscienceregistry.org, osf.io, etc.). 
This study will be registered with clinicaltrials.gov.