Abt Associates Report Title Insert Date ▌1

April 23, 2021 

Prepared by:

Judith A. Alamprese

Amanda Parsad

Meghan Shea

I-Fang Cheng

Jill Hamadyk

Abt Associates

Submitted to:

Local Initiatives Support Corporation (LISC)

Submitted by:

Abt Associates

6130 Executive Boulevard

Rockville, MD 20852

Evaluation of Bridges to Career

Opportunities: Final Report
Message from Local Initiatives Support Corporation (LISC)

To support and empower people who have been living on low incomes, helping them stabilize finances 

and take on good, family-sustaining jobs, is an evolving, and critical, science. At LISC, we developed our 

Financial Opportunity Centers® (FOCs) as part of that evolution, and research has shown that FOCs’ 

multi-pronged approach—offering employment services, financial coaching and access to income 

supports (like Supplemental Nutrition Assistance Program, rental assistance and child care subsidies) is 

good science. FOCs have supported 25,000 people each year in stabilizing their finances and joining the 

mainstream economy.

But our experience has also shown us that FOC support, on its own, is not enough in all cases. We learned 

that many clients’ potential for long-term financial stability and wealth-building was stymied by the lowwage jobs available to them and by gaps in their education or the basic skills needed to succeed in the 

kinds of occupational training that can ultimately lead to a sustaining career.

So we developed an additional strategy—Bridges to Career Opportunities (BCO)—which builds on our 

FOC model and which we began piloting seven years ago in six cities. In addition to the bundled services 

provided by FOCs, BCO clients receive instruction in basic skills using curricula tailored to occupations 

in each region’s growth industries—often with the partnership of local employers looking to fill jobs in 

those occupations. This foundational training, which includes soft skills instruction, is a doorway to more 

advanced training and certification that in turn leads to an occupational pathway with potential for upward 

mobility. The initial success of the BCO model inspired funders to support scaling the work, and we now 

have BCO programming through our FOCs in 40 locations across the country. 

Which brings us to this evaluation report. In 2017, Abt Associates began a study of the job and financial 

stability outcomes for BCO participants (involving some 1,100 clients) with follow-up assessment over 

the course of an additional 14 months, comparing their successes with those of people enrolled in FOC 

programming alone. The results were very positive, once again demonstrating that this common sense 

model for supporting people to prepare for and advance in 21st-century jobs is good science. The study 

found that:

• BCO program participants had high completion rates (86%) and high credential attainment rates 

(80% of people who completed the program received credentials). 

• BCO participants showed a gain of one grade level in reading on the Test for Adult Basic 

Education (TABE®). 

• When it came to employment, BCO participants were more likely to have found or advanced in a 

job than their FOC counterparts – there was a difference of 19% between the two (56% vs 37%).

The study also found that that certain financial outcomes, including credit score and net worth, were 

better for FOC participants than for BCO participants. This is likely due to the additional time FOC 

participants had when compared to BCO participants who spent more than three to six months in training. 

Nevertheless, credit outcomes were good for both BCO and FOC participants—80% of FOC comparison 

clients and 68% of BCO participants increased their credit score. We expect that over time, we would see 

similar outcomes for both cohorts. 

As encouraging as these outcomes are, this study, like all gauges of economic well-being in the country, 

were upended by the onset of the COVID-19 pandemic, and the staggering unemployment of upwards of 
30 million people last year. On top of the sweeping loss of jobs, one in three Americans took a cut in pay 

or hours. And it has been widely reported that unemployment and other indicators of the financial fallout 

from the pandemic have disproportionately devastated Black and Brown communities—the very 

communities that are home to the majority of FOC and BCO clients.

It will take some time to fully grasp and quantify the pandemic’s impact on our FOC and BOC clients. 

Nevertheless, we can see, as noted in this report, that our strategies work and that the Bridges to Career 

Opportunities model provides imperative skills and training without which people simply cannot access 

quality jobs—and that will still be true in a post-pandemic work. As we continue to hone and test the very 

human science of promoting greater financial well-being for people who have lived in privation and 

instability, we will need these important results to guide us, and the people and communities we serve, 

along the way.
PREFACE

Abt Associates BCO Evaluation Final Report April 2021 ▌i

About This Report

This report provides findings from Abt Associates’ (Abt) evaluation of the Local Initiatives Support 

Corporation’s (LISC) Bridges to Career Opportunities program. The evaluation was funded by the 

Corporation for National and Community Service’s Social Innovation Fund and supported by LISC.

Acknowledgements

The authors of this report extend sincere thanks to the many individuals who contributed to the Bridges to 

Career Opportunities (BCO) evaluation. We thank Laura D’Alessandro, Seung Kim, and Carolina 

Rendon from LISC who have supported our work in carrying out the evaluation, transferring BCO 

participant data, and preparing this report. Their guidance was instrumental in enabling us to conduct the 

evaluation. We also acknowledge the assistance of David Greenberg, who reviewed this report, and 

former LISC staff, Christopher Walker and Sarah Rankin, who were involved in the design of the 

evaluation.

We are particularly grateful to the BCO staff from the seven evaluation sites who hosted our site visits, 

participated in interviews and class observations, worked with us in clarifying data, and provided 

information on an ongoing basis. Their cooperation and willingness to participate in the evaluation has 

been invaluable. We also thank the staff from the six Financial Opportunity Centers® who were part of the 

evaluation for their time in participating in telephone interviews and working with us in clarifying data.

Their assistance was very much appreciated.

In addition to the report’s authors, several Abt staff participated in the evaluation: Brian Freeman, Peiyi 

Zhang, and Nayara Mowry assisted with the data analysis; Sarah Costelloe conducted site visits; Daniel

Litwok, Cristofer Price, and former Abt staff member Katherine Gan, participated in the design of the 

evaluation; Allan Porowski served as a reviewer of the final report; Bry Pollack provided editing support 

for the final report; and Katheleen Linton and Missy Robinson provided production support for the final 

report.

The views expressed in this report do not necessarily reflect the views or policies of the Corporation for 

National and Community Service or the views of the Local Initiatives Support Corporation.

Permission to reproduce the report is not necessary. Suggested citation: Alamprese, J. A., Parsad, A., 

Shea, M., Cheng, I-F., & Hamadyk, J. (April 2021). Evaluation of Bridges to Career Opportunities: Final 

report. Abt Associates. 
CONTENTS

Abt Associates BCO Evaluation Final Report April 2021 ▌ii

CONTENTS

Executive Summary................................................................................................................vii

Introduction ................................................................................................................... vii

Context for the Evaluation and Related Research ......................................................... vii

BCO Model................................................................................................................... viii

Implementation Study.....................................................................................................ix

Implementation Study Research Question.............................................................ix

Implementation Study Design, Measures, and Data Collection.............................. ix

Implementation Study Sites ...................................................................................ix

Data Analysis.........................................................................................................xi

Findings on Fidelity of BCO Implementation .......................................................... xi

Impact Study ..................................................................................................................xi

Impact Study Questions.........................................................................................xi

Sample ................................................................................................................. xii

Impact Study Data Sources .................................................................................. xii

Analytic Approach................................................................................................. xii

Impact Study Results for Confirmatory Research Questions ......................................... xii

Employment Impacts ............................................................................................ xii

Financial Well-Being Impacts............................................................................... xiii

Descriptive Analyses of Treatment Group’s Education Outcomes ................................ xiii

BCO Participants’ Foundational Literacy Skills .................................................... xiv

BCO Participants Program Completion and Credential Attainment...................... xiv

Changes to SEP........................................................................................................... xiv

Education Research Questions............................................................................ xiv

Employment Research Questions......................................................................... xv

Financial Well-Being Research Questions ............................................................ xv

Other Changes .................................................................................................... xvi

Conclusions.................................................................................................................. xvi

Fidelity of BCO Implementation ........................................................................... xvi

Education Outcomes ...........................................................................................xvii

Employment Outcomes .......................................................................................xvii

Financial Well-Being Outcomes...........................................................................xvii

Lessons Learned and Limitations .................................................................................xvii

Introduction...............................................................................................................................1

Context for the Evaluation and Related Research ...........................................................1

BCO Logic Model ............................................................................................................2

Research Questions........................................................................................................3

Implementation Study Research Questions ............................................................3

Impact Study Research Questions..........................................................................5

Changes to SEP Research Questions.............................................................................6

Education Research Questions...............................................................................6

Employment Research Questions...........................................................................6

Financial Well-Being Research Questions ..............................................................7
CONTENTS

Abt Associates BCO Evaluation Final Report April 2021 ▌iii

Implementation Study...............................................................................................................8

Implementation Study Design, Methods, and Analysis ....................................................8

Overview of Design.................................................................................................8

Measures................................................................................................................8

Data Collection .......................................................................................................9

Data Analysis........................................................................................................13

Fidelity Analysis....................................................................................................13

Findings from Fidelity Analysis ......................................................................................13

Impact Study Findings............................................................................................................17

Impact Study Questions, Data Sources, and Analytic Approach....................................17

Confirmatory Research Questions ........................................................................17

Impact Study Data Sources ..................................................................................18

Analytic Approach.................................................................................................18

Findings from Impact Analysis.......................................................................................19

BCO increased the share of participants who had a job improvement. .................19

There was No Significant Effect of BCO on Improving Participants’ Credit 

Score ........................................................................................................20

Descriptive Analyses of Treatment Group’s Education Outcomes .................................20

BCO Participants’ Foundational Literacy Skills .....................................................20

Analysis of Treatment Group’s Completion of Education or Training ....................21

Conclusion ..............................................................................................................................23

Conclusions...................................................................................................................23

Fidelity of BCO Implementation ............................................................................23

Education Outcomes ............................................................................................23

Employment Outcomes ........................................................................................23

Financial Well-Being Outcomes............................................................................23

Lessons Learned and Limitations ..................................................................................24

Appendix A: BCO Logic Model ..............................................................................................25

Appendix B: Additional Information about the Evaluation Design and Impact Study .......27

Research Questions......................................................................................................27

Exploratory Research Questions Regarding Employment.....................................27

Exploratory Questions Concerning Financial Well-Being ......................................28

Exploratory Question Concerning Education.........................................................28

Feasibility Study to Select Study Sample.......................................................................29

Treatment Sites and Participant Sample...............................................................29

Comparison Sites and Participant Sample............................................................30

Formation of the Treatment and Comparison Group.............................................33

Sample Loss and Attrition.....................................................................................36

Method for Testing Baseline Equivalence......................................................................38

Confirmatory Employment Outcome .....................................................................40

Exploratory Employment Outcomes......................................................................42

Confirmatory Financial Well-Being Outcome.........................................................49

Exploratory Financial Well-Being Outcomes .........................................................51

Data Collection..............................................................................................................53

Analytic Methods ...........................................................................................................53

Study Measures....................................................................................................53
CONTENTS

Abt Associates BCO Evaluation Final Report April 2021 ▌iv

Estimating the Effects of BCO ..............................................................................59

Pre-Post Reading Gain Analysis...........................................................................59

Treatment of Missing Data....................................................................................61

Multiple Outcome Measures .................................................................................61

Appendix C: Supplemental Tables and Information on Study Findings .............................62

Employment ..................................................................................................................62

Exploratory Research Questions for Employment.................................................62

Impacts on Employment .......................................................................................62

Financial Well-Being......................................................................................................64

Exploratory Research Questions for Financial Well-Being ....................................64

Impacts on Financial Well-Being...........................................................................64

Education ......................................................................................................................66

References ..............................................................................................................................68

List of Exhibits

Exhibit 1. Integration of BCO Model’s Components.................................................................. viii

Exhibit 2. Implementation Study Research Questions for BCO Model Components....................x

Exhibit 1. Integration of BCO Model’s Components.....................................................................3

Exhibit 2. Implementation Study Research Questions for BCO Model Components....................4

Exhibit 3. Impact Study Research Questions ..............................................................................5

Exhibit 4. BCO Implementation Sites ..........................................................................................8

Exhibit 5. List of Implementation Study Constructs Assessed, Question Topics Addressed, 

and Data Collection Methods ....................................................................................................10

Exhibit 6. Site Visit Schedule and Number of People Interviewed .............................................12

Exhibit 7. Fidelity Ratings by Site for Each BCO Component ....................................................14

Exhibit 8. Extent of Treatment Sites’ Adherence to BCO Bridge Services Model.......................15

Exhibit 10. Extent of Treatment Sites’ Adherence to BCO Financial Coaching and Education, 

and Income Support Services Model.........................................................................................16

Exhibit 11: Flow from Eligible BOC and FOC Participants to Analytic Study Sample.................18

Exhibit 12. Impact of BCO on Whether Participants had a Job Improvement after Program 

Enrollment.................................................................................................................................19

Exhibit 13. Impact of BCO on Credit Score Increase after Program Enrollment.........................20

Exhibit 14. TABE® Reading Pre-Post Results............................................................................21

Exhibit 15. Education or Training Program Completion Status of Treatment Group ..................21

Exhibit 16. Number of Credentials Earned by Treatment Group................................................21

Exhibit 17. Type of Credentials Earned by Treatment Group.....................................................22

Exhibit A-1. Logic Model for BCO..............................................................................................26

Exhibit B-1. Site Selection Process ...........................................................................................29

Exhibit B-2. Seven BCO Treatment Sites..................................................................................30

Exhibit B-3. Six FOC Comparison Sites ....................................................................................31
CONTENTS

Abt Associates BCO Evaluation Final Report April 2021 ▌v

Exhibit B-4. Characteristics of Eligible BCO and FOC Participants............................................32

Exhibit B-5. Characteristics of Eligible BCO and FOC Participants............................................33

Exhibit B-6. Employment Outcomes: Number of Participants Matched per Site (excludes 

Credit Score from the Match) ....................................................................................................35

Exhibit B-7. Financial Well-Being Outcomes: Number of Participants Matched per Site 

(includes Credit Score in the Match) .........................................................................................35

Exhibit B-8. Flow from Eligible BOC and FOC Participants to Analytic Study Samples .............37

Exhibit B-9. Employment Outcomes Initial Match: Baseline Equivalence of Matched Sample 

(excludes Credit Score).............................................................................................................39

Exhibit B-10. Baseline Equivalence of Matched Sample for Job Status Improvement 

Outcome ...................................................................................................................................40

Exhibit B-11. Baseline Equivalence of Matched Sample Lost for Job Status Improvement 

Outcome ...................................................................................................................................41

Exhibit B-12. Baseline Equivalence of Matched Sample for Hourly Wage Increase Outcome ...42

Exhibit B-13. Baseline Equivalence of Matched Sample for Amount of Wage Increase 

Outcome ...................................................................................................................................43

Exhibit B-14. Baseline Equivalence of Matched Sample for Wage at Follow-Up Outcome ........44

Exhibit B-15. Baseline Equivalence of Matched Sample for Increase in Hours Worked 

Outcome ...................................................................................................................................45

Exhibit B-16. Baseline Equivalence of Matched Sample for Mean Hours Worked Increase 

Outcome ...................................................................................................................................46

Exhibit B-17. Baseline Equivalence of Matched Sample for Hours Worked at Follow-Up 

Outcome ...................................................................................................................................47

Exhibit B-18. Financial Outcomes Initial Match: Baseline Equivalence of Matched Sample 

(includes Credit Score)..............................................................................................................48

Exhibit B-19. Baseline Equivalence of Matched Sample for Maximum Credit Score Outcome ..49

Exhibit B-20. Baseline Equivalence of Matched Sample Lost for Maximum Credit Score..........50

Exhibit B-21. Baseline Equivalence of Matched Sample for Latest Credit Score Outcome........51

Exhibit B-22. Baseline Equivalence of Matched Sample for Financial Net-Worth Outcome.......52

Exhibit B-23. Employment Outcome Measures Examined: Data Source and Measure 

Construction..............................................................................................................................54

Exhibit B-24. Number of Months Between Enrollment and Measurement of Confirmatory 

Outcomes .................................................................................................................................55

Exhibit B-25. Financial Well-Being Outcome Measures Examined: Data Source and 

Measure Construction...............................................................................................................57

Exhibit B-26. Education Outcome Measure Examined: Data Source and Measure 

Construction..............................................................................................................................58

Exhibit B-27. Participant Characteristics at Baseline: Data Source and Measure Construction .58

Exhibit C-1. Impacts on Confirmatory and Exploratory Employment Outcomes.........................63

Exhibit C-2. Sensitivity Analysis: Impact on Confirmatory Employment Outcome Controlling 

for Number of Months Between Enrollment and Measure of Outcome ......................................64
CONTENTS

Abt Associates BCO Evaluation Final Report April 2021 ▌vi

Exhibit C-3. Impacts on Confirmatory and Exploratory Financial Well-Being Outcomes ............65

Exhibit C-4. Sensitivity Analysis: Impact on Confirmatory Financial Well-Being Outcome 

Controlling for Number of Months Between Enrollment and Measurement of Outcome ............66

Exhibit C-5. TABE® Reading Pre-Post Test Results by BCO Site..............................................67

Exhibit C-6. TABE® Reading Pre-Test Results for BCO Participants at Baseline by Site...........67
EXECUTIVE SUMMARY

Abt Associates BCO Evaluation Final Report April 2021 ▌vii

Executive Summary

Introduction

This report presents the findings from Abt Associates’ (Abt) implementation study and impact study of 

the Bridges to Career Opportunities (BCO) model. BCO is a multi-component intervention that assists

adults who are underemployed or unemployed in advancing in a career and obtaining a living wage. 

Created by the Local Initiatives Support Corporation (LISC), BCO has four components of services. The 

bridge component provides contextualized basic skills education, career planning, and occupational skills 

training. The other three components of BCO services compose LISC’s Financial Opportunity Center®

(FOC) model, which bundles financial coaching and education, income support services, and employment 

services. The integrated bridge and FOC services—the BCO model—is intended to assist adults with low 

incomes or low skills earn industry-recognized credentials that can enable them to obtain jobs in career 

pathways leading to financial stability and economic well-being. 

With funding from the Corporation for National and Community Services (CNCS)’s Social Innovation 

Fund (SIF) in 2016, LISC implemented the BCO model in 32 community-based organizations in 11 

metropolitan areas. As part of the SIF project, LISC contracted with Abt to conduct an evaluation of the 

BCO model from 2016 to 2020. The evaluation consisted of an implementation study and a quasiexperimental impact study. The implementation study assessed the extent to which a purposeful sample of 

seven of the BCO treatment group sites carried out the services that compose the BCO model, and the 

factors that facilitated and posed challenges to delivering BCO services. The quasi-experimental impact 

study investigated the effects of BCO services in those seven sites on participants’ employment and 

financial well-being outcomes compared to those of a comparison group of similar clients who enrolled in 

only FOC only services in six comparison sites. 

The evaluation’s treatment and comparison groups included 1,133 BCO participants and 1,217 FOC 

participants, respectively, who enrolled in those services during the period April 1, 2017 through 

December 31, 2018. The evaluation targeted a moderate level of evidence and followed the overall 

procedures that Abt submitted in its BCO SIF evaluation plan (Gan, Alamprese, Litwok, & Price, 2016).

Context for the Evaluation and Related Research

Families with low incomes face significant barriers to achieving financial security and upward mobility. 

Workers in low-wage occupations, particularly workers with a secondary education or less, tend to remain 

in these occupations (e.g., Gabe, Abel, &Florida, 2018; Andersson, Holzer, & Lane, 2005).

To aid families in mitigating those barriers, LISC developed the Financial Opportunities Centers®. Based 

on the Annie E. Casey Foundation’s Center for Working Families model, the FOCs assist adults in 

working toward financial stability by advancing their employment and subsequently their financial wellbeing. LISC provides non-profit, community-based organizations with financial support and technical 

assistance to offer FOC bundled services in three areas: employment services; financial coaching and 

education; and linkages to income support services (e.g., Supplemental Nutrition Assistance Program, 

rental assistance, childcare subsidies). LISC’s goal is to increase the positive net income and, ultimately, 

financial stability for adults with low incomes through their participation in FOC services. 

Research on FOC clients has informed LISC’s ongoing approach to services. Early evidence from a study 

of FOC clients indicated that the combination of financial coaching and employment services increased 

FOC clients’ placement in jobs and job retention, as well as their net income. Financial coaching and 
EXECUTIVE SUMMARY

Abt Associates BCO Evaluation Final Report April 2021 ▌viii

assistance in accessing relevant income support streams enabled FOC clients to stabilize their finances in 

preparing for and attaining placement in a job. However, FOC clients’ potential for long-term financial 

advancement was found to be limited by the low wage rates of jobs available to them and by their gaps in 

the basic skills needed to succeed in sector-based occupational training (Rankin, 2015). 

A quasi-experimental study of FOC client outcomes conducted by the Economic Mobility Corporation 

found that a key barrier to study participants’ financial stability is their ability to acquire “good” jobs—

jobs that meet their expenses, have stable schedules, and have opportunities for long-term career growth 

(Roder, 2016). Though occupational training programs for in-demand jobs are available through the 

publicly funded workforce system, community colleges, and private organizations, many FOC clients 

have gaps in their basic skills, lack educational credentials, or have financial constraints that preclude 

their enrollment or success in those programs. 

To address challenges faced by adults with low skills and low incomes in obtaining family-sustaining jobs 

in career pathways, LISC considered the additional services that could strengthen adults’ basic and 

technical skills and attainment of credentials. In 2015 LISC launched Bridges to Career Opportunities, 

which was built on the FOC model and integrates bridge education and training services. 

BCO Model 

A key assumption underlying BCO is that the model’s components of services are necessary to enable 

adults to develop a career and earn a family-sustaining wage, and that those components are most 

effective when they work together (Exhibit 1). 

Exhibit 1. Integration of BCO Model’s Components

The bridge component increases career awareness and provides instruction in basic skills using 

occupationally contextualized curricula that can prepare adults to succeed in training for an occupational 

pathway. Its contextualized basic skills can be delivered as a first step to prepare BCO clients for 

occupational training or can be integrated into the occupational training. The development of clients’ 

basic skills within the context of work enables them to earn credentials and enter jobs with a potential for 

upward mobility.

Contextualized Bridge Services

Employment 

Services

Income 

Support 

Services

Financial 

Coaching and 

Education
EXECUTIVE SUMMARY

Abt Associates BCO Evaluation Final Report April 2021 ▌ix

BCO clients’ enrollment in public benefits can provide support for daily expenses and reduce barriers to 

training and employment. Financial coaching and education are expected to help clients reduce expenses, 

access credit under better terms, increase savings, and build assets. The assumption is that when clients 

are financially stable, they are more likely to engage in and remain in training. 

Finally, employment services ensure that clients are prepared for the job search and application processes, 

and that they are connected to opportunities for jobs with family-sustaining wages.

Implementation Study

Implementation Study Research Question

The BCO implementation study addressed the following question: 

To what extent did the study sites implement the following four components of the BCO program with 

fidelity: (1) contextualized bridge program services, (2) financial coaching and education services, (3) 

income supports, and (4) employment services?

Exhibit 2 lists the implementation study research question for each BCO component and associated 

activities.

Implementation Study Design, Measures, and Data Collection

Abt used a mixed methods design to assess the extent to which treatment sites’ delivery of the BCO 

model adhered to the characteristics of the model as defined by LISC. This approach involved having a 

well-defined set of BCO components of services and associated activities that include features of the 

structure and processes used in delivering the intervention and collecting data about the fidelity of 

implementation using multiple methods.

The measures for BCO treatment sites’ fidelity of implementation of the BCO model are the activities in 

Exhibit 2 that are associated with each of the four BCO components—contextualized bridge services, 

financial coaching and education, income supports, and employment services.

Data to address the overarching implementation study question To what extent did the treatment sites 

implement the BCO program with fidelity? were collected from the treatment sites using three methods: 

(1) interviews with evaluation sites’ staff as part of two rounds of site visits that Abt conducted; (2) 

observations of services delivered in the BCO treatment sites during the site visits; and (3) conduct of biannual interim telephone conferences during the two-year period of sites’ implementation of BCO 

services.

Implementation Study Sites

Abt selected seven BCO programs as treatment sites. These sites had strong capacity to deliver BCO 

services and sufficient projected baseline enrollment to meet the evaluation’s requirements. The sites 

were:

• Brighton Center, Inc., Greater Cincinnati and Northern Kentucky;

• Chinese Community Center, Inc., Houston, TX;

• District 1199C Training and Upgrading Fund, Philadelphia, PA;

• Instituto del Progreso Latina, Chicago, IL;

• International Institute of Minnesota, St. Paul, MN;
EXECUTIVE SUMMARY

Abt Associates BCO Evaluation Final Report April 2021 ▌x

• Project for Pride in Living, Inc., Minneapolis, MN; and

• Wesley Community Center, Inc., Houston, TX.

Exhibit 2. Implementation Study Research Questions for BCO Model Components 

Bridge Services Component: To what extent did BCO treatment sites implement bridge services and their associated 

activities? 

1. Instruction

a. Addresses career pathway in growth sector or industry;

b. Includes an academic pathway;

c. Teaches basic skills that are contextualized to a specific career path;

d. Is based on industry-specific curriculum;

e. Enables clients to attain industry-recognized credentials; and

f. Has measurable endpoint or clearly defined completion criteria.

2. Assessment

a. Uses criterion levels to determine client participation; and

b. Asses pre- and post-test skills.

3. Career Coaching

a. Provides ongoing career coaching; and 

b. Connects academic and career pathways.

Financial Coaching and Education Component: To what extent did the BCO treatment sites implement financial 

coaching and education services and their associated activities?

a. Assesses clients’ financial health, combining a financial profile, budget, balance, and credit information (Combined 

Financial Assessment);

b. Provides regular one-on-one interactions concerning clients’ financial status;

c. Reviews clients’ credit reports every six months; and

d. Uses financial products during coaching.

Income Support Services Component: To what extent did the BCO treatment sites implement income support services 

and their associated activities?

a. Screens for public benefits eligibility;

b. Identifies income supports to enable participants to remain in training; and

c. Reassesses clients’ needs as their situations change.

Employment Services Component: To what extent did the BCO treatment sites implement employment services and 

their associated activities? 

a. Assists clients in preparing job search materials and conducting job search;

b. Provides clients with coaching about job search, placement, retention, and advancement; and

c. Supports clients in exploring careers.
EXECUTIVE SUMMARY

Abt Associates BCO Evaluation Final Report April 2021 ▌xi

Data Analysis

The data collected through the site visits (interviews, observations) and the interim telephone conferences 

were analyzed to assess the overall fidelity of delivery of the BCO model. A content analysis was 

performed to understand the types of activities that the treatment sites conducted for each of the BCO 

components. 

The results from the content analysis of each treatment site’s BCO activities were rated to determine how 

closely a treatment site had adhered to the BCO model, defined as whether the four components of the 

BCO model were delivered as intended. Abt developed a three-point scale to rate the degree to which 

each of the activities for each component was implemented as specified in the BCO list of components.

Findings on Fidelity of BCO Implementation

The seven treatment sites demonstrated high fidelity in implementing the BCO model’s four components 

of activities “as intended;” that is, the sites carried out the activities associated with each of the BCO 

components as they have been defined by LISC. Overall, the treatment sites implemented 94 percent of 

the activities across the four components “as intended,” with little variation in the percentages of 

adherence for each component. The component with the highest percentage of adherence was 

employment services (95.1%), followed by financial coaching and education services, along with income 

support services (94.1%), and bridge services (93.6%). Those results indicate that BCO sites participating 

in the evaluation were able to follow the guidance and assistance that LISC provided in offering bundled 

services to support adults with low incomes. 

The treatment sites were able to achieve a high degree of adherence to the BCO model while using a 

range of approaches in carrying out the activities associated with each of the BCO components. This 

variation reflected the resources of the organizations implementing BCO and their prior experience as 

Financial Opportunity Centers®, which provided a foundation for their approach to BCO services. 

Another factor influencing the BCO sites’ services were the types of partnerships with external 

organizations that staff formed to enable them to provide comprehensive services to clients with low 

incomes or low skills. Those partnerships were important in enabling BCO sites to leverage their services 

to meet the varying needs of their clients. The background and experience of staff affected the treatment 

sites’ approaches to bridge, financial coaching, and education services since those services require 

specialized knowledge and skills to be able to work effectively with clients. The variation in approaches 

points to the flexibility within the BCO model and how the model is used by different types of 

organizations.

Impact Study

The impact study examined whether the BCO model, which integrates bridge services with FOC services, 

better prepares participants with low incomes to increase their employment prospects and financial 

stability than do FOC services alone. The study addressed research questions on employment and 

financial well-being using a quasi-experimental design and compared the outcomes of BCO participants 

(the treatment group) with a matched group of FOC participants (the comparison group). 

Impact Study Questions

The confirmatory research questions for the impact study were:

• Were BCO participants more likely to obtain or advance in a job after program entry relative to a 

comparison group who received only FOC services? (Employment)
EXECUTIVE SUMMARY

Abt Associates BCO Evaluation Final Report April 2021 ▌xii

• Were BCO participants more likely to have obtained or increased their credit scores after program 

entry relative to a comparison group who received only FOC services? (Financial Well-Being) 

Sample

Seven BCO programs were purposively selected as treatment sites, as described above, and six FOC 

programs were purposively selected as comparison sites. Those FOC sites did not provide educational 

services for clients; were in cities with similar labor markets as the treatment sites and served FOC clients 

with similar demographic characteristics as the BCO treatment sites; and collected the evaluation’s 

required data for FOC participants and delivered FOC services according to the FOC model.

Across the BCO treatment programs and the FOC comparison programs, eligible BCO participants were 

exactly matched to one or more eligible FOC participant on gender, age, race/ethnicity, highest level of 

education completed, employment status, and credit score at intake using a coarsened exact matching 

(CEM) process.

Impact Study Data Sources

The data sources were extant administrative data that LISC maintains on BCO and FOC clients’

demographic and background information, service receipt, and outcome data. BCO and FOC sites collect 

and submit those data to LISC using standardized procedures. The data are stored in LISC’s database that 

uses a Salesforce platform.

The data to address the impact study’s confirmatory research question concerning BCO and FOC-only 

participants’ job advancement were drawn from participants’ intake, employment, and advancement 

records in Salesforce. The data for the confirmatory research question concerning study participants’ 

credit score increase were drawn from the credit score record in Salesforce.

Analytic Approach

Participant outcomes were analyzed in a linear regression model that tested the effect of BCO on job 

advancement and credit score increase.

Impact Study Results for Confirmatory Research Questions

Abt’s quasi-experimental impact study addressed confirmatory research questions concerning impacts on 

study participants’ employment and financial well-being.

Employment Impacts

The evaluation’s confirmatory employment question was: Were BCO participants more likely to have a 

job improvement after program entry relative to a comparison group who received only FOC services? To 

address this question, Abt analyzed treatment and comparison group participants’ job improvement1

during the 14 months after their enrollment in a BCO program or an FOC. The analyses indicated that 

study participants enrolled in BCO programs were 19 percentage points more likely to have a “job 

improvement” than were participants in FOC programs (56 percent versus 37 percent). This difference 

was statistically significant at the .001 level. 

 

1 Participant (a) enrolled in the program unemployed and starts a new job in the 1-14 months after enrollment or (b) was 

employed at enrollment and has one of the following outcomes in the 1-14 months after enrollment: (1) obtains a new job 

and the new wage is greater than the wage at enrollment; or (2) has an increase in wages or obtains a promotion.
EXECUTIVE SUMMARY

Abt Associates BCO Evaluation Final Report April 2021 ▌xiii

The BCO model includes activities that support program participants’ selection of a career pathway and 

reinforce participants’ advancement in a pathway, which might have contributed to the positive impacts 

on the treatment group participants’ job advancement. The BCO bridge services include an ongoing 

review of participants’ career goals and selection of pathways that align with their skills, interests, and 

abilities. When BCO participants earn an initial credential, they are encouraged to consider the steps they 

will take to obtain an advanced credential that will enable them to move forward in a career pathway. As 

participants work with an employment coaches in job search and job placement activities, coaches discuss 

strategies for retaining and advancing in a job. The BCO model is designed to provide program 

participants with ongoing support and encouragement during a sustained period that can prepare them for 

success in the workplace. 

Financial Well-Being Impacts 

The evaluation’s confirmatory financial well-being question was: Were BCO participants more likely to 

have obtained or increased their credit scores after program entry relative to a comparison group who 

received only FOC services? To address this question, Abt analyzed treatment and comparison group 

participants’ maximum credit score at follow-up during the 14 months after their enrollment in a BCO 

program or an FOC. The analyses indicated that 68 percent of participants in the BCO program and 

approximately 80 percent of participants in the FOC program had an increase in their credit score after 

program entry. This negative impact of 18 percentage points was not statistically significant.

One condition that might explain the credit score findings is the difference in timeline between BCO 

participants’ and FOC participants’ job search and placement. BCO participants are enrolled in bridge 

services involving concurrent or sequential occupational training for two to seven months after program 

enrollment and search for jobs in their area of occupational training after completing training. In contrast, 

FOC participants generally are placed in jobs as soon as possible after program entry. The earlier job 

placement for FOC participants could result in better credit scores for them during the 14 months after 

their enrollment in an FOC, because they will have begun working sooner after enrollment than BCO 

participants. 

Another factor that might have affected the credit score analysis is the small sample of matched BCO 

participants (181) and FOC participants (79) who had baseline and follow-up credit scores. Although the 

BCO and FOC programs are strongly encouraged to retrieve program participants’ credit scores every six 

months during their engagement in services, there was a high percentage of missing credit score data, 

particularly for FOC participants. The FOC participants’ missing data might reflect their shorter length of 

time receiving services compared with BCO participants.

Since the BCO model is one of the few career pathways programs that includes a financial well-being 

intervention and data collection activities, we are not able to examine the financial well-being results of 

this study relative to the results from other studies.

Descriptive Analyses of Treatment Group’s Education Outcomes

Abt conducted descriptive analyses of the BCO treatment group participants’ education outcomes to 

understand participants’ progress in developing their foundational skills while enrolled in BCO, in 

completing the BCO program, and their progress in earning educational credentials. The evaluation’s 

confirmatory education question could not be addressed due to lack of data and is discussed below in the 

section on “Changes to SEP.”
EXECUTIVE SUMMARY

Abt Associates BCO Evaluation Final Report April 2021 ▌xiv

BCO Participants’ Foundational Literacy Skills

As part of BCO’s bridge services, BCO treatment sites were to provide contextualized instruction to 

increase the foundational literacy skills of BCO participants who had low basic skills. To examine the 

effects of BCO on literacy skills, we compared the pre- and post-test scores on the Test for Adult Basic 

Education (TABE®) Reading test for all BCO participants in the evaluation, not for just those with low 

basic skills at entry.

At BCO program intake, BCO treatment group participants’ TABE® Reading pre-test scale score was 

532, equivalent to about the 6.6 grade level. At the post-test, BCO participants had increased their reading 

scores by about 9 points or by about one grade-level to the 7.4 grade-level equivalent (CTB McGraw-Hill, 

2004). This increase was statistically significant at p< .001 level.

BCO Participants Program Completion and Credential Attainment

Abt’s evaluation examined the treatment group participants’ rates of completion of a BCO program and 

attainment of educational credentials. The majority (86%) of the treatment group participants completed 

the BCO education or training program in which they enrolled. Of treatment group participants who 

completed a training program, approximately 80% percent earned at least one credential, primarily an 

occupational certificate. Over half (57%) of those credentials were in healthcare occupations, which was 

the occupational focus of over half of the education and training programs offered by BCO treatment 

sites.

BCO treatment group participants’ rates of program completion and credential attainment by month 14 

after enrollment compare favorably with the results from education and training programs funded by the 

U.S. Department of Health and Human Services Health Professional Opportunity Grants (HPOG). HPOG 

grants are intended to assist adults with low incomes earn healthcare credentials and obtain familysustaining jobs in a pathway. The HPOG data provide a context for considering the BCO treatment sites’ 

outcomes in training program completion and credential attainment. An evaluation of Pima Community 

College’s Pathways to Healthcare Program, which includes basic skills bridge programs, found that by 

month 18 after program enrollment, 59 percent of participants had completed any education or training 

and 29 percent had received a credential (Gardiner, Rolston, Fein, & Cho, 2017). An evaluation of San 

Diego County Bridge to Employment in the Healthcare Industry Program found that 64 percent of the 

treatment group members earned a credential (Farrell & Martinson, 2017).

Changes to SEP

Abt made changes to the SEP primarily because of the lack of availability of evaluation participants’ 

administrative data. The changes to the SEP are described below.

Education Research Questions

The confirmatory research question in education, “Are BCO participants more likely to have attained an 

occupational certificate or postsecondary credential 12 months after program entry relative to a 

comparison group who receive only FOC services?” could not be addressed due to lack of reliable 

comparison group data. Abt’s analysis of educational outcomes revealed than only four comparison 

participants had data concerning attainment of occupational certificates or postsecondary credentials at 

follow-up. The response rate might have been due to the structure of the Degree, Certificate, License 

record in LISC’s Salesforce database, in which program staff enter data in this record only if participants 

have attained a credential. There is no variable in the record concerning non-attainment of a credential, 

which meant we could not determine whether the response rate of four participants was accurate.
EXECUTIVE SUMMARY

Abt Associates BCO Evaluation Final Report April 2021 ▌xv

The exploratory research question in education for only treatment group members “Are BCO participants 

likely to increase their math skills (as measured by a standardized test) after participating in the BCO 

program?” could not be addressed because most treatment programs did not collect math post-test data 

from participants at the end of bridge services. Rather, the programs collected post-test data on 

participants’ reading skills since most instruction addressed the reading skills participants needed to 

succeed in their occupational training.

The exploratory research question in education “Are BCO participants likely to complete their BCO 

program?” was added to the evaluation since it addressed an important hypothesized outcome from 

participation in BCO bridge services as indicated in the BCO logic model.

Employment Research Questions

Approach to Research Questions. Abt’s intended approach to address employment outcomes was to 

compare the treatment and comparison groups’ outcomes at 12 months after participants’ enrollment in 

BCO services (treatment group) and FOC services (comparison group). The timeline assumed that 

program staff would conduct follow-up data collection 12 months after participants enrolled in their 

respective programs. However, because the BCO model allows for client-driven interactions, only a 

portion of the follow-up data collection occurred at 12 months after enrollment. Because the BCO and 

FOC program services dictate that clients reach out to staff as needed, this interaction does not follow a 

specific schedule. As a result, it was difficult to gather point-in-time data according to a 12-month followup schedule. Rather, follow-up data were defined as any data collected after baseline, beginning in month 

1 after enrollment and continuing throughout the 14-month period in which the evaluation tracked clients’ 

participation in BCO or FOC services. BCO and FOC staff also reported that part of the variation in the 

timeline for follow-up data collection was due to sites’ difficulty in reaching clients after they ended their 

participation in BCO or FOC services. Thus the 12-month timeframe for the employment research 

questions was changed to the timeframe of month 1 through month 14.

Addition of Employment Confirmatory Question. Prior to data analysis, Abt added the confirmatory 

research question “Were BCO participants more likely to obtain or advance in a job after program entry 

relative to a comparison group who received only FOC services?” based on our re-examination of the 

BCO logic model concerning participants’ outcomes from employment services.

Exploratory Employment Questions Not Addressed. Abt could not address the following two 

employment exploratory questions due to the difficulty in gathering the point-in-time information noted 

above: (1) Are BCO participants more likely to retain the same job (for 180 days) relative to a comparison 

group who receive only FOC services? and (2) Are BCO participants likely to have been employed more 

quickly relative to a comparison group who receive only FOC services?

Financial Well-Being Research Questions

Approach to Research Questions. The intended approach and change in timeframe described for 

employment questions also applies to the financial well-being questions.

Addition of Financial Well-Being Confirmatory Question. Prior to data analysis, Abt added the 

confirmatory research question “Were BCO participants more likely to have obtained or increased their 

credit scores after program entry relative to a comparison group who received only FOC services? based 

on our re-examination of the BCO logic model concerning participants’ outcomes from financial services.

Addition of Financial Well-Being Exploratory Question. The following exploratory research question 

was added because of the emphasis on net-worth in financial coaching services: Were BCO participants 
EXECUTIVE SUMMARY

Abt Associates BCO Evaluation Final Report April 2021 ▌xvi

more likely to have increased their net worth after program entry relative to a comparison group who 

received only FOC services?

Exploratory Financial Well-Being Questions Not Addressed. Abt could not address the following

exploratory financial well-being question due to a lack of data on on-time account payments: Are BCO 

participants more likely to have a greater percentage of on-time account payments 12 months after 

program entry relative to a comparison group who receive only FOC services?

Other Changes

The SEP specified that two site visits would be conducted to each of the treatment programs as part of the 

implementation study. Abt conducted two site visits to six of the seven programs. Due to a scheduling 

difficulty, Abt could not conduct a site visit to the seventh program but conducted telephone interviews 

with program staff to gather the same information that was to be collected during the second site visit.

The SEP specified that the evaluation would use a Propensity Score Analysis (PSA), a matching 

technique wherein participants in a program are matched to non-participants based on their participation 

propensity score. PSA is useful for selecting comparison members on a large number (20 or more) of 

characteristics. Given the small number of variables that were appropriate to use for matching in the BCO 

evaluation, Abt used the CEM approach instead of a PSA matching approach. Because all matching 

variables are combined into a summary score, PSA does not guarantee close or exact matches on 

variables of interest. However, CEM ensures exact matches on variables of interest (because treatment 

and comparison matches within the same block have identical values on coarsened matching variables). 

The SEP specified that the timeline for data collection for the impact study would be January 2017 

through December 2018. The beginning of the data collection period was changed from January 2017 to 

April 2017 because LISC’s training of programs’ implementation of their new Salesforce data system was 

still underway at the beginning of 2017. The change to April 2017 ensured that all BCO and FOC 

programs would be able to enter baseline client data into Salesforce at the beginning of the evaluation’s 

data collection period.

Conclusions

The evaluation found that the BCO model, with well-defined services and training and technical 

assistance support, can be implemented by community-based organizations with strong adherence to the 

model. The evaluation also indicated that BCO’s goal to facilitate clients’ attainment of educational 

credentials and employment on their path to economic well-being can be attained through the delivery of 

the multi-component services that define the BCO model. Although BCO evaluation participants were not 

able to make significant advances in their economic status during the 14 months after they enrolled in the 

BCO program, they were able to earn credentials and advance in their jobs. Those outcomes are thought

to be steps toward economic well-being. The process of increasing individuals’ economic status is 

complex and affected by many factors. Although this evaluation provided preliminary insights about nearterm changes in participants’ economic status, more robust investigations with larger samples of 

participants are needed to understand adults’ attainment of economic well-being.

Fidelity of BCO Implementation

The seven treatment sites implemented the BCO model’s four components of activities with a high degree 

of fidelity. Many factors contributed to this result including the ongoing training and technical assistance 

that LISC staff provided to its grantees and the organizational resources of the seven programs that 

participated as treatment sites. Another factor is that the BCO model is specified with sufficient detail to 
EXECUTIVE SUMMARY

Abt Associates BCO Evaluation Final Report April 2021 ▌xvii

guide programs’ main activities in each of the four BCO components but is not so prescriptive that 

programs cannot adjust services as they encounter challenges. This model design enabled the treatment 

programs to respond to the needs of their client populations while delivering the range of services that 

compose the BCO model. 

Education Outcomes

The majority of BCO treatment participants completed the BCO program (86%) and most of those 

completers earned at least one credential, which was primarily an occupational certificate (80%). The 

treatment sites’ approaches to recruitment and orientation helped clients understand the requirements and 

commitment needed to succeed in the BCO program. The BCO bridge design also provided clients with 

an opportunity to master skills in the bridge component that could contribute to their success in the 

occupational training. The ongoing support provided by BCO coaches and instructors helped clients to 

persevere and keep on track as they worked to earn an educational credential.

Employment Outcomes

The evaluation results indicated that BCO treatment participants were significantly more likely to obtain 

or advance in a job after enrollment than FOC comparison participants. Factors contributing to this 

outcome were the comprehensive job readiness and job placement services that the BCO sites provided. 

As the BCO implementation study indicated, the treatment sites’ employment services were the most 

highly aligned of the four BCO components of services such that BCO sites provided many similar 

services. Another factor likely contributing to the BCO participants’ job placement and advancement was 

the partnerships that BCO sites formed with local employers. BCO staff reported that those partnerships 

not only facilitated participants’ job attainment but also help support participants as they moved along in 

their employment.

Financial Well-Being Outcomes

The evaluation found that FOC comparison group participants were more likely to increase their credit 

scores than BCO treatment group participants. Comparison group participants also were more likely than 

treatment participants to decrease their debt, increase their net worth, and have a higher amount of debt 

decrease. Comparison group participants’ earlier participation in the workforce after program enrollment 

may have enabled those participants to engage in financial activities helped to increase their financial 

well-being.

Lessons Learned and Limitations

One lesson from the evaluation concerns processes to mitigate the loss of administrative data. Although 

LISC and Abt encouraged sites throughout the evaluation to follow established protocols for entering 

client data, more frequent monitoring of the BCO and FOC data in Salesforce might have resulted in less 

loss of data. 

Because of the evaluation’s quasi-experimental design, the BCO treatment programs participating in the 

evaluation were not selected to represent the full set of BCO programs but were purposively selected 

because of their planned services, client population, stability of BCO program implementation, and 

projected number of participants. The FOC comparison programs were purposively selected for the 

evaluation because their clients who entered services during the evaluation’s enrollment period had 

similar background characteristics as the BCO clients in the treatment group. The BCO and FOC 

programs were also in cities with similar labor markets. For those reasons, the evaluation’s results may 

not be applicable to clients with background characteristics different from the backgrounds of evaluation 

participants or because of possible differences in BCO program services. 
EXECUTIVE SUMMARY

Abt Associates BCO Evaluation Final Report April 2021 ▌xviii

The goal of using coarsened exact matching (CEM) in the evaluation was to create treatment and 

comparison groups that were similar, before the start of clients’ participation in the BCO and FOC 

programs, on observed characteristics likely to be related to the education, employment, and financial 

outcomes of interest. Therefore, any differences in education, employment, and financial outcomes 

observed can be attributed to the BCO program (or unobserved differences) rather than to initial 

differences between the groups. The treatment and comparison groups’ analytic samples were shown to 

be equivalent at baseline on observed characteristics, even after the loss of sample due to missing 

outcome data. However, a limitation of the study design is that differences observed may be due to 

unobserved characteristics of participants or the BCO program. Further, because the missing data led to 

small sample sizes for the financial outcome, the evaluation may not be powered to detect small 

differences between the groups.
INTRODUCTION

Abt Associates BCO Evaluation Final Report April 2021 ▌1

Introduction

This report presents the findings from Abt Associates’ (Abt) implementation study and impact study of 

the Bridges to Career Opportunities (BCO) model. BCO is a multi-component intervention that assists 

adults who are underemployed or unemployed adults in advancing in a career and obtaining a living 

wage. Created by the Local Initiatives Support Corporation (LISC), BCO has four components of 

services. The bridge component provides contextualized basic skills education, career planning, and 

occupational skills training. The other three components of BCO services compose LISC’s Financial 

Opportunity Center® (FOC) model, which bundles financial coaching and education, income support 

services, and employment services. The integrated bridge and FOC services—the BCO model—is 

intended to assist adults with low incomes or low skills earn industry-recognized credentials that can 

enable them to obtain jobs in career pathways leading to financial stability and economic well-being. 

With funding from the Corporation for National and Community Services (CNCS)’s Social Innovation 

Fund (SIF) in 2016, LISC implemented the BCO model in 32 community-based organizations in 11 

metropolitan areas. As part of the SIF project, LISC contracted with Abt to conduct an evaluation of the 

BCO model from 2016 to 2020. The evaluation consisted of an implementation study and a quasiexperimental impact study. The implementation study assessed the extent to which a purposeful sample of 

seven of the BCO treatment group sites carried out the core services that compose the BCO model, and 

the factors that facilitated and posed challenges to delivering BCO services. The quasi-experimental 

impact study investigated the effects of BCO core services in those seven sites on participants’ 

employment and financial well-being outcomes compared to those of a comparison group of similar 

clients who enrolled in only FOC services in six comparison sites. 

The evaluation’s treatment and comparison groups included 1,133 BCO participants and 1,217 FOC 

participants, respectively, who enrolled in those services during the period April 1, 2017 through 

December 31, 2018.2 The evaluation targeted a moderate level of evidence and followed the overall 

procedures that Abt submitted in its BCO SIF evaluation plan (Gan, Alamprese, Litwok, & Price, 2016).

Context for the Evaluation and Related Research

Families with low incomes face significant barriers to achieving financial security and upward mobility. 

Workers in low-wage occupations, particularly workers with a secondary education or less, tend to remain 

in these occupations (e.g., Gabe, Abel, &Florida, 2018; Andersson, Holzer, & Lane, 2005). Evidence 

suggests this is partially due to gaps in their technical skills and basic skills. For example, three-quarters 

of workers with low wages have a high school diploma or equivalent but lack the relevant occupational 

skills and employer connections required to enter a career pathway (Loprest, Acs, Ratcliffe, & Vinopal, 

2009). Upward mobility for those who lack technical skills, high school credentials, or basic skills is even 

more limited.

To aid families in mitigating these barriers, LISC developed the Financial Opportunities Centers®. Based 

on the Annie E. Casey Foundation’s Center for Working Families model, the FOCs assist adults in 

 

2 SEP Change: The SEP specified that the timeline for data collection for the impact study would be January 2017 through 

December 2018. The beginning of the data collection period was changed from January 2017 to April 2017 because LISC’s

training of programs’ implementation of their new Salesforce data system was still underway at the beginning of 2017. The 

change to April 2017 ensured that all BCO and FOC programs would be able to enter baseline client data into Salesforce at 

the beginning of the evaluation’s data collection period.
INTRODUCTION

Abt Associates BCO Evaluation Final Report April 2021 ▌2

working toward financial stability by advancing their employment and subsequently their financial wellbeing. LISC provides non-profit, community-based organizations with financial support and technical 

assistance to offer FOC bundled services in three areas: employment services; financial coaching and 

education; and linkages to income support services (e.g., Supplemental Nutrition Assistance Program, 

rental assistance, childcare subsidies). LISC’s goal is to increase the positive net income and, ultimately, 

financial stability for adults with low incomes through their participation in FOC services. 

Research on FOC clients has informed LISC’s ongoing approach to services. Early evidence from a study 

of FOC clients indicated that the combination of financial coaching and employment services increased 

FOC clients’ placement in jobs and job retention, as well as their net income. Financial coaching and 

assistance in accessing relevant income support streams enabled FOC clients to stabilize their finances in 

preparing for and attaining placement in a job. However, FOC clients’ potential for long-term financial 

advancement was found to be limited by the low wage rates of jobs available to them and by their gaps in 

the basic skills needed to succeed in sector-based occupational training (Rankin, 2015). 

A quasi-experimental study of FOC client outcomes conducted by the Economic Mobility Corporation

found that a key barrier to study participants’ financial stability is their ability to acquire “good” jobs—

jobs that meet their expenses, have stable schedules, and have opportunities for long-term career growth 

(Roder, 2016). Though occupational training programs for in-demand jobs are available through the 

publicly funded workforce system, community colleges, and private organizations, many FOC clients 

have gaps in their basic skills, lack educational credentials, or have financial constraints that preclude 

their enrollment or success in these programs. 

To address challenges faced by adults with low skills and low incomes in obtaining family-sustaining jobs 

in career pathways, LISC considered the additional services that could strengthen adults’ basic and 

technical skills and attainment of credentials. In 2015 LISC launched Bridges to Career Opportunities, 

which was built on the FOC model and integrates bridge education and training services. 

BCO Logic Model 

The BCO logic model is presented in Appendix A. The BCO model provides clients with the following:

• Bridge services that include contextualized adult basic education (ABE) or English as a Second 

Language (ESL) instruction to develop clients’ proficiencies in reading, writing, numeracy, and 

English language to enter and succeed in technical skills training; and connections to 

occupational training that lead to industry-recognized credentials and career pathways in locally 

in-demand occupations; and

• Financial coaching and education services, income support services, and employment services 

that are integrated to help clients manage the financial, navigational, and logistical challenges 

associated with participation in education and training programs.

A key assumption underlying BCO is that the model’s components of services are necessary to enable 

adults to develop a career and earn a family-sustaining wage, and that those components are most 

effective when they work together (Exhibit 1).
INTRODUCTION

Abt Associates BCO Evaluation Final Report April 2021 ▌3

Exhibit 1. Integration of BCO Model’s Components

The bridge component increases career awareness and provides instruction in basic skills using 

occupationally contextualized curricula that can prepare adults to succeed in training for an occupational 

pathway. Its contextualized basic skills can be delivered as a first step to prepare BCO clients for 

occupational training or can be integrated into the occupational training. The development of clients’ 

basic skills within the context of work enables them to enter jobs with a potential for upward mobility.

BCO clients’ enrollment in public benefits can provide support for daily expenses and reduce barriers to 

training and employment. Financial coaching and education are expected to help clients reduce expenses, 

access credit under better terms, increase savings, and build assets. The assumption is that when clients 

are financially stable, they are more likely to engage in and remain in training. 

Finally, employment services ensure that clients are prepared for the job search and application processes, 

and they are connected to opportunities for jobs with family-sustaining wages.

Research Questions

Implementation Study Research Questions

The BCO implementation study research question was the following:

To what extent did the study sites implement the following four components of the BCO program with 

fidelity: (1) contextualized bridge program services, (2) financial coaching and education services, (3) 

income supports, and (4) employment services?

The research questions for the implementation study are shown in Exhibit 2.

Contextualized Bridge Services

Employment 

Services

Income 

Support 

Services

Financial 

Coaching and 

Education
INTRODUCTION

Abt Associates BCO Evaluation Final Report April 2021 ▌4

Exhibit 2. Implementation Study Research Questions for BCO Model Components 

Bridge Services Component: To what extent did BCO treatment sites implement bridge services and their associated 

activities as defined by LISC? 

1. Instruction

a. Addresses career pathway in growth sector or industry;

b. Includes an academic pathway;

c. Teaches basic skills that are contextualized to a specific career path;

d. Is based on industry-specific curriculum;

e. Enables clients to attain industry-recognized credentials; and

f. Has measurable endpoint or clearly defined completion criteria.

2. Assessment

a. Uses criterion levels to determine client participation; and

b. Assesses pre- and post-test skills.

3. Career Coaching

a. Provides ongoing career coaching; and 

b. Connects academic and career pathways.

Financial Coaching and Education Component: To what extent did the BCO treatment sites implement financial 

coaching and education services and their associated activities as defined by LISC?

a. Assesses clients’ financial health, combining a financial profile, budget, balance, and credit information 

(Combined Financial Assessment);

b. Provides regular one-on-one interactions concerning clients’ financial status;

c. Reviews clients’ credit reports every six months; and

d. Uses financial products during coaching.

Income Support Services Component: To what extent did the BCO treatment sites implement income support services 

and their associated activities as defined by LISC?

a. Screens for public benefits eligibility;

b. Identifies income supports to enable participants to remain in training; and

c. Reassesses clients’ needs as their situations change.

Employment Services Component: To what extent did the BCO treatment sites implement employment services and 

their associated activities as defined by LISC? 

a. Assists clients in preparing job search materials and conducting job search;

b. Provides clients with coaching about job search, placement, retention, and advancement; and

c. Supports clients in exploring careers.
INTRODUCTION

Abt Associates BCO Evaluation Final Report April 2021 ▌5

Impact Study Research Questions

The BCO quasi-experimental impact study investigated whether the BCO intervention, which integrates

bridge services with FOC services, better prepares adults with low incomes and low skills to increase 

their employment prospects and financial stability than do FOC services alone. The impact study 

addressed confirmatory and exploratory research questions concerning study participants’ employment 

and financial well-being outcomes. The impact study also examined exploratory questions concerning 

BCO treatment group participants’ development of reading skills and occupational training completion. 

The impact study research questions are shown in Exhibit 3.

Exhibit 3. Impact Study Research Questions

Employment

Abt addressed the following confirmatory and exploratory questions related to employment outcomes: 

Confirmatory Question:

1. Were BCO participants more likely to obtain or advance in a job after program entry relative to a comparison 

group who received only FOC services? 

Exploratory Questions:

2. Were BCO participants more likely to increase their hourly wages after program entry relative to a comparison 

group who received only FOC services?

3. Were BCO participants more likely to be placed in jobs with higher wages relative to a comparison group who 

received only FOC services?

4. Were BCO participants more likely to work more hours per week after program entry relative to a comparison 

group who received only FOC services?

Financial Well-Being

Abt addressed the following confirmatory and exploratory questions related to financial well-being outcomes:

Confirmatory Question:

1. Were BCO participants more likely to have obtained or increased their credit scores after program entry relative 

to a comparison group who received only FOC services?

Exploratory Questions:

2. Were BCO participants more likely to have lower overall debt after program entry relative to a comparison group 

who received only FOC services?

3. Were BCO participants more likely to have increased their net worth after program entry relative to a comparison 

group who received only FOC services?

Education

Abt addressed descriptive research questions concerning BCO participants’ education outcomes:

1. Are BCO participants likely to increase their reading skills (as measured by a standardized test) after 

participating in the BCO program?

2. Are BCO participants likely to complete their BCO program?

3. Are BCO participants likely to earn credentials after participating in a BCO program? 
INTRODUCTION

Abt Associates BCO Evaluation Final Report April 2021 ▌6

Changes to SEP Research Questions

Abt made changes to the SEP primarily because of the lack of availability of evaluation participants’ 

administrative data. The changes to the research questions in the SEP are described below.

Education Research Questions

The confirmatory research question in education, “Are BCO participants more likely to have attained an 

occupational certificate or postsecondary credential 12 months after program entry relative to a 

comparison group who receive only FOC services?” could not be addressed due to lack of reliable 

comparison group data. Abt’s analysis of educational outcomes revealed than only four comparison 

participants had data concerning attainment of occupational certificates or postsecondary credentials at 

follow-up. The response rate might have been due to the structure of the Degree, Certificate, License 

record in LISC’s Salesforce database, in which program staff enter data in this record only if participants 

have attained a credential. There is no variable in the record concerning non-attainment of a credential, 

which meant we could not determine whether the response rate of four participants was accurate.

The exploratory research question in education for only treatment group member “Are BCO participants 

likely to increase their math skills (as measured by a standardized test) after participating in the BCO 

program?” could not be addressed because most treatment programs did not collect post-test data from 

participants at the end of bridge services. Rather, the programs collected post-test data on participants’ 

reading skills since most instruction addressed the reading skills participants needed to succeed in their 

occupational training.

The exploratory research question “Are BCO participants likely to complete their BCO program?” was 

added to the evaluation since it addressed an important hypothesized outcome from participation in BCO 

services as indicated in the BCO logic model.

Employment Research Questions

Approach to Research Questions. Abt’s intended approach to address employment outcomes was to 

compare the treatment and comparison groups’ outcomes at 12 months after participants’ enrollment in 

BCO services (treatment group) and FOC services (comparison group). The timeline assumed that 

program staff would conduct follow-up data collection 12 months after participants’ enrollment in their 

respective programs. However, because the BCO model allows for client-driven interactions, only a 

portion of the follow-up data collection occurred at 12 months after enrollment. Because the BCO and 

FOC program services dictate that clients reach out to staff as needed, this interaction does not follow a 

specific schedule. As a result, it was difficult to gather point-in-time data according to a 12-month followup schedule. Rather, follow-up data were defined as any data collected after baseline, beginning in month 

1 after enrollment and continuing throughout the 14-month period in which the evaluation tracked clients’ 

participation in BCO or FOC services. BCO and FOC staff also reported that part of the variation in the 

timeline for follow-up data collection was due to sites’ difficulty in reaching clients after they ended their 

participation in BCO or FOC services. Thus the 12-month timeframe for the employment research 

questions was changed to the timeframe of month 1 through month 14.

Addition of Employment Confirmatory Question. Prior to data analysis, Abt added the confirmatory 

research question “Were BCO participants more likely to obtain or advance in a job after program entry 

relative to a comparison group who received only FOC services?” based on our re-examination of the 

BCO logic model concerning participants’ outcomes from employment services.

Exploratory Employment Questions Not Addressed. Abt could not address the following two 

employment exploratory questions due to the difficulty in gathering point-in-time information noted 
INTRODUCTION

Abt Associates BCO Evaluation Final Report April 2021 ▌7

above: (1) Are BCO participants more likely to retain the same job (for 180 days) relative to a comparison 

group who receive only FOC services? and (2) Are BCO participants likely to have been employed more 

quickly relative to a comparison group who receive only FOC services?

Financial Well-Being Research Questions

Approach to Research Questions. The intended approach and change in timeframe described for 

employment questions also applies to the financial well-being questions.

Addition of Financial Well-Being Confirmatory Question. Prior to data analysis, Abt added the 

confirmatory research question “Were BCO participants more likely to have obtained or increased their 

credit scores after program entry relative to a comparison group who received only FOC services? based 

on our re-examination of the BCO logic model concerning participants’ outcomes from financial services.

Addition of Financial Well-Being Exploratory Question. The following exploratory research question 

was added because of the emphasis on net-worth in financial coaching services: Were BCO participants 

more likely to have increased their net worth after program entry relative to a comparison group who 

received only FOC services?

Exploratory Financial Well-Being Questions Not Addressed. Abt could not address the following

exploratory financial well-being question due to a lack of data on on-time account payments: Are BCO 

participants more likely to have a greater percentage of on-time account payments 12 months after 

program entry relative to a comparison group who receive only FOC services?
IMPLEMENTATION STUDY

Abt Associates BCO Evaluation Final Report April 2021 ▌8

Implementation Study 

The BCO implementation study examined the fidelity of the implementation of the BCO logic model 

(Appendix A, Exhibit A-1). The implementation research questions, shown in Exhibit 2, address the key 

activities associated with each of the four BCO components: contextualized bridge services, financial 

coaching and education, income support services, and employment services.

Abt selected seven BCO programs as treatment sites, which are listed in Exhibit 4 (see Appendix B for a 

detailed description of the site selection process). The implementation study assessed the use of the BCO 

model in those sites.

Exhibit 4. BCO Implementation Sites

Site Location

Designation Used 

in Report

Brighton Center, Inc. Greater Cincinnati and Northern Kentucky A

Chinese Community Center, Inc. Houston, TX B

District 1199C Training and Upgrading Fund Philadelphia, PA C

Instituto del Progreso Latino Chicago, IL D

International Institute of Minnesota St. Paul, MN E

Project for Pride in Living, Inc. Minneapolis, MN F

Wesley Community Center, Inc. Houston, TX G

This section of the report describes the design, measures, and data collection for the implementation study 

and the results from the fidelity study.

Implementation Study Design, Methods, and Analysis

Overview of Design 

Abt’s approach to the implementation study was to assess the extent to which treatment sites’ delivery of 

the BCO model adhered to the characteristics of the model as defined by LISC, and to understand how the 

implementation evolved over time depending on challenges that the sites experienced in implementing the 

model. This approach involved having a well-defined set of BCO components of services and associated 

activities that include features of the structure and processes used in delivering the intervention 

(Mowbray, Holter, Teague, & Bybee, 2003; Bond, Williams, Evans, Salyers, Kim, Sharpe, & Leff, 2000), 

and collecting data about the fidelity of implementation using multiple methods (Century, Rudnick, & 

Freeman, 2010).

Measures

The measures for BCO treatment sites’ fidelity of implementation of the BCO model are activities 

associated with each of the four BCO components—contextualized bridge services, financial coaching

and education, income supports, and employment services (Exhibit 2). 

Abt worked with LISC to develop this model of BCO components and associated activities through a 

multi-step process. Abt staff prepared draft BCO core component activities based on our review of the (1)

LISC/SIF Request for Proposal (September 14, 2015), (2) LISC staff meeting notes from September 23, 

2015, and (3) reporting forms for LISC’s Salesforce data system. Abt staff discussed the BCO component 

activities with LISC evaluation and BCO program staff (May 2016), and BCO program staff provided the 

Service Integration Retreat PowerPoint presentation (March 2016) as additional information for Abt to 
IMPLEMENTATION STUDY

Abt Associates BCO Evaluation Final Report April 2021 ▌9

use in refining the BCO component activities. Based on the results from discussions with LISC staff and 

their review of our draft component activities, Abt prepared a final list of BCO component activities (June 

2016). These activities are included in the BCO logic model (Appendix A).

Data Collection

Data to address the implementation study question To what extent did the treatment sites implement the 

BCO program with fidelity? were collected from the treatment sites using three methods: (1) interviews 

with evaluation sites’ staff as part of two rounds of site visits that Abt conducted; (2) observations of 

services delivered in the BCO treatment sites during the site visits; and (3) conduct of interim telephone 

conferences during the two-year period of sites’ implementation of BCO services. Prior to the beginning 

of data collection Abt sent each treatment site a “letter of participation” that described the data collection 

activities which Abt would conduct and the site’s role in the data collection. The letter discussed the 

treatment site’s role in working with Abt to organize each site visit, including identifying individuals to 

be interviewed and services to be observed, scheduling the interviews, providing private space for the 

conduct of the interviews, and arranging for Abt staff’s observation of services. The letter also provided 

information about the purpose of and topics for the telephone conferences.

Site Visits (Interviews, Observations)

The site visits were a mechanism for collecting data about the treatment sites’ implementation of the four 

BCO components. Data points were actual services delivered, staff’s perceptions of services and their 

utility for BCO participants, methods and materials used in the delivery of services, and treatment sites’ 

partners in BCO services. The site visits involved:

• The conduct of face-to-face interviews with BCO staff and partners using a structured interview 

protocol; 

• Observations of delivery of bridge instruction (basic skills instruction and occupational training), 

and job readiness services held in a group setting; and 

• Review of materials used in the delivery of BCO services. 

Abt’s senior research staff trained in observation and interview methods conducted the site visits, which 

occurred during year 1 (2017-2018) and year 2 (2018-2019) of the treatment sites’ participation in the 

evaluation. The two site visits enabled Abt’s staff to gather information over a sufficient period to identify 

any implementation issues that sites experienced with the BCO model. 

Listed in Exhibit 5 are the implementation study’s constructs that were assessed, the topics for questions 

that defined each construct, and method of data collection. Abt developed an implementation study 

protocol that had questions for each topic listed under each construct in Exhibit 5. The protocol was used 

to guide the two site visits and the interim telephone conferences. Abt adapted a class observation form 

used in its prior adult education studies to document (1) activities that the instructor or site staff member 

conducted during each observation, (2) clients’ participation in the activities, (3) materials used, and (4) 

other related activities that took place. 
IMPLEMENTATION STUDY

Abt Associates BCO Evaluation Final Report April 2021 ▌10

Exhibit 5. List of Implementation Study Constructs Assessed, Question Topics Addressed, and 

Data Collection Methods

Constructs and Topics for Questions Defining Constructs

Method of Data Collection

Review of 

BCO 

Application, 

Screening 

Calls

Interview 

during Site 

Visits3

Program

Observation

Interim

Telephone

Conference4

Goals for BCO Grant 

Goals for number of clients to be served during 2017 and 2018 X Site Visit 1 

only

Progress in numbers of clients served in training programs X X

Target Population for BCO

Population organization currently serves X Site Visit 1 

only

Subpopulation targeted for BCO training programs X X

Partnerships X

Partnerships formed to support BCO services being delivered X X

How the partnerships work in terms of activities, staff involved X

Benefits to partners for participating X

Recruitment of Clients X

How organization’s services are advertised X

How clients say they find out about BCO program X

Within-program recruitment for BCO X

Outcomes advertised for clients X

Intake and Orientation X

Activities conducted during intake and format for activities X

Orientation activities, length of time, format, and next steps X

Financial Services and Financial Coaching X

Types of services offered, topics, and timing of services X X

Completion of Financial Planning forms, review of credit score X

Providers of services and location X

Use of financial products, which products, and how used X

Financial coaching approach, timing, topics discussed, and staff X

Income Supports X

Types and details of income supportive services offered X X

Method used for public benefits screening X

Sequence and timing of services, mode of delivery, role of coach X

Employment Services X

Types and details of job readiness, employment search services X X X

Sequence of services delivered, format, and length of services X

 

3 During Site Visit 1, all topics were addressed. During Site Visit 2, program staff were asked to provide updated information 

on services conducted since Site Visit 1.

4 The interim telephone discussions addressed the status of data collection in Salesforce and updates in services that occurred 

since the last communication with Abt. 
IMPLEMENTATION STUDY

Abt Associates BCO Evaluation Final Report April 2021 ▌11

Constructs and Topics for Questions Defining Constructs

Method of Data Collection

Review of 

BCO 

Application, 

Screening 

Calls

Interview 

during Site 

Visits3

Program

Observation

Interim

Telephone

Conference4

Staff and partners who provide services X

Employment coaching approach, timing, topics, and staff X

Job placement activities conducted, relationship with employers X

Bridge Services: Contextualized Adult Education (AE) X

Basic skills assessments used, testing schedule, and tester X X

Basic skills assessment criteria for program participation 

Types of AE services offered, service providers, and location X X

Instructional content, approach, delivery modality, and schedule X

Approach for integrating occupational information into basic 

skills 

X

Type of curriculum, contextualization to career, completion point X

Bridge Services: Occupational Training X

Occupational training type and industry/growth X X X

Occupation training length, delivery mode, location, and provider

Education and skill requirements for participation X

Credentials earned and requirements for credentials X

Bridge Services: Career Planning X

Career planning activities content, format, and timeline

Relationship between academic activities and career pathways

Follow-up Services X

Services provided to clients after they complete Bridge Program X

How clients access follow-up services X

Data Collection and Entry into Salesforce X

Process and timelines for data collection and data entry X

Prior to each site visit, Abt’s team conducted a planning telephone call with each treatment site’s BCO 

program coordinator to discuss the topics that would be addressed during the interviews, types of 

individuals to interview who could address the topics, the service to observe, and possible dates for the 

site visit. After the planning call, Abt sent a draft site visit agenda that was scheduled over a two-day 

period and involved a senior and a mid-level Abt staff member. The BCO site staff prepared a final 

schedule that included individual, one-hour interviews with the two Abt staff and observation of services 

if an observation was to be conducted during that visit. The categories of individuals for the interviews, 

which represent staff and external people involved in the BCO program, were: 

• Organization’s executive director or senior staff member;

• BCO program coordinator or manager;

• Staff who conduct client recruitment, intake, and orientation activities; 

• Bridge service staff including adult education instructor, occupational training instructors or 

vendors, and career advisor;
IMPLEMENTATION STUDY

Abt Associates BCO Evaluation Final Report April 2021 ▌12

• Financial coach and staff involved in delivering or arranging financial education activities;

• Employment coach and staff who conduct job readiness activities and job placement activities;

• Income support services staff;

• BCO program partners and employers; and

• Staff member who works with LISC’s Salesforce database.

Abt’s site visit team interviewed the individuals listed above during the two site visits except for the 

program partners, who were interviewed during only one visit for most of the sites.

In preparation for Site Visit 1, Abt’s site visit team reviewed each treatment site’s BCO application and 

site screening interview notes to record information in the protocol so that this information could be 

confirmed or updated during the site visit. All topics in Exhibit 5 were addressed during Site Visit 1. 

During Site Visit 2, BCO site staff were asked whether BCO activities in each of the four BCO 

components had changed or whether there were new activities since Site Visit 1. Abt’s team conducted at 

least one observation during at least one site visit. The services observed were career planning activities, 

basic skills instruction, or occupational training. The timing of the observations depended on the schedule 

of site activities. The information collected during the site visits was stored in a qualitative database that 

Abt established for each treatment site. The qualitative database was organized according to the 

constructs listed in Exhibit 5. Listed in Exhibit 6 is the schedule for the site visits and the number of 

individuals interviewed during each site visit.

Telephone Conferences

Abt’s staff conducted interim telephone calls with the BCO coordinator or other key staff at each 

treatment site in between the site visits to collect information about implementation of BCO services, 

modifications to existing services, new services added to the BCO components, difficulties encountered 

in implementing the BCO model, and status of the sites’ participant data entry into Salesforce (See 

Exhibit 5). The information from these telephone calls was entered into each treatment site’s qualitative 

database.

Exhibit 6. Site Visit Schedule and Number of People Interviewed

Site

Site Visit Number and 

Schedule

Number of People 

Interviewed

Brighton Center, Inc. #1 May 16-17, 2017

#2 November 28-29, 2018

9

9

Chinese Community Center, Inc. #1 May 10-11, 2017

#2 October 23-24, 2018

7

9

District 1199C Training and Upgrading Fund #1 May 1-2, 2017

#2 December 4-5, 2018

14

10

Instituto del Progreso Latino #1 November 28-29, 20175 10

 

5 Change to SEP. The SEP specified that two site visits would be conducted to each of the treatment programs as part of the

implementation study. Abt conducted two site visits to six of the seven programs. Due to a scheduling difficulty, Abt could 

not conduct a site visit to the Instituto del Progreso Latino program but conducted telephone interviews with program staff 

to gather the same information that was to be collected during the second site visit.
IMPLEMENTATION STUDY

Abt Associates BCO Evaluation Final Report April 2021 ▌13

Site

Site Visit Number and 

Schedule

Number of People 

Interviewed

International Institute of Minnesota #1 September 21-22, 2017

#2 June 5-6, 2019

11

10

Project for Pride in Living, Inc. #1 July 13-14, 2017

#2 October 29-30, 2018

11

11

Wesley Community Center, Inc. #1 June 21-22, 2017

#2 February 12-13, 2019

9

9

Data Analysis

The data collected through the site visits (interviews, observations) and interim telephone conferences

were analyzed to assess the overall fidelity of delivery of the BCO model. A content analysis was 

performed to understand the types of activities that the treatment sites conducted for each of the BCO 

components. The content analysis involved organizing the information from the site visits and telephone 

conferences that was in each BCO site’s qualitative database in a separate analysis matrix for each site. 

The left column in each site’s matrix had a row for each of the BCO components and activities that are 

shown in Exhibit 2. The top row of the matrix had multiple columns for each of the two site visits and the 

interim telephone conferences. Abt’s site visit team prepared a matrix for each BCO site that listed the 

activities the site conducted for each BCO component by the data collection period in which the 

information was collected (i.e., Site Visit 1, Site Visit, telephone conference). The resulting matrix for 

each site showed all activities that Abt’s team had documented for that site during the period of the 

implementation study.

Fidelity Analysis

The activities listed in each BCO site’s matrix were analyzed to determine whether a treatment site had 

adhered to the BCO model, defined as whether the four components of the BCO model were delivered as 

intended. Abt developed a three-point scale to rate the degree to which each of the activities for each 

component was implemented as specified in the BCO list of components (Exhibit 2).

• Rating of 1: implemented as intended; activity carried out as stated in the BCO model;

• Rating of 0.5: partially implemented; part of the activity carried out; and

• Rating of 0: not implemented; activity not carried out.

Abt’s senior researcher used the adherence scale to rate the activities listed in each BCO site’s matrix in 

terms of whether the activities in each matrix aligned with the required activities for each BCO 

component as shown in Exhibit 2. Each site had to have at least one instance of alignment to each activity 

for each component to receive a rating of “1.” If a site implemented only part of the required activity, a 

rating of “0.5” was made. A second researcher then made an independent rating to validate the scale and 

the ratings.

Findings from Fidelity Analysis 

Presented in Exhibit 7 are the ratings for each of the BCO sites for each activity in each BCO component.
IMPLEMENTATION STUDY

Abt Associates BCO Evaluation Final Report April 2021 ▌14

Exhibit 7. Fidelity Ratings by Site for Each BCO Component

BCO Components and Associated Activities

BCO Treatment Sites’ Fidelity Ratings

Site A Site B Site C Site D Site E Site F Site G

1A. Bridge Services Component-Instruction

Addresses career pathway in growth sector or industry 1 1 1 1 1 1 1

Includes an academic pathway 1 1 1 1 1 1

Teaches basic skills that are contextualized to a specific 

career path

1 0.5 1 1 1 0 1

Is based on industry-specific curriculum 1 1 1 1 1 1 1

Enables clients to attain industry-recognized credentials 1 1 1 1 1 0.5 1

Has measurable endpoint or clearly defined completion 

criteria

1 1 1 1 1 1 1

1B. Bridge Service Component-Assessment

Uses criterion levels to determine client participation 1 1 1 1 1 1 1

Tests pre- and post-test skills 1 1 1 1 1 1 1

1C. Bridge Services Component-Career Coaching

Provides ongoing career coaching 1 0.5 1 1 1 1 1

Connects academic and career pathways 1 0.5 1 0.5 1 1 1

Total Bridge Services Rating 100%

(10)

85%

(8.5)

100%

(10)

95%

(9.5)

100%

(10)

75%

(7.5)

100%

(10)

Financial Coaching and Education Component 

Assesses clients’ financial health, combining a financial 

profile, budget, balance, and credit information 

1 1 1 1 1 1 1

Provides regular one-on-one interactions concerning 

clients’ financial status

1 1 1 0.5 1 1 1

Reviews clients’ credit reports every six months 1 1 1 1 1 1 1

Uses financial products during coaching 1 1 1 1 0.5 1 1

Income Support Services Component

Screens for public benefits eligibility 1 1 1 1 1 1 1

Identifies income supports to enable participants to 

remain in training 

1 1 1 1 1 1 1

Reassesses clients’ needs as their situations change 1 0.5 0.5 0.5 1 1 1

Total Financial Coaching and Income Support 

Services Rating

100%

(7)

92%

(6.5)

92%

(6.5)

85%

(6)

92%

(6.5)

100%

(7)

100%

(7)

Employment Services Component

Assists clients in preparing job search materials and 

conducting job search 

1 1 1 1 1 1 1

Provides clients with coaching about job search, 

placement, retention, and advancement

1 1 1 1 1 1 1

Supports clients in exploring careers 1 0.5 1 1 1 0.5 1

Total Employment Services Rating 100% 

(3)

83%

(2.5)

100%

(3)

100%

(3)

100%

(3)

83%

(2.5)

100%

(3)
IMPLEMENTATION STUDY

Abt Associates BCO Evaluation Final Report April 2021 ▌15

Presented in Exhibits 8, 9, and 10 are the results from the analyses of adherence to the BCO model by 

each BCO treatment site. Exhibit 8 shows the percentage of sites that adhered to implementing each 

activity specified for the BCO bridge services. Four of the sites implemented 100 percent of the activities 

“as intended,” one site implemented 95 percent of the activities, one site implemented 85 percent of the 

activities, and one site implemented 75 percent of the activities. 

Exhibit 8. Extent of Treatment Sites’ Adherence to BCO Bridge Services Model

The activities that were not implemented in Site F concerned the delivery of basic skills instruction. At 

intake, that site assessed its BCO treatment group as not needing basic skills enhancement and thus did 

not provide basic skills instruction.

Exhibit 9 shows the percentage of sites that adhered to implementing each activity specified for the BCO 

employment services. Five of the sites implemented 100 percent of the activities “as intended.” The other 

two sites partially implemented one of the activities.

Exhibit 9. Extent of Treatment Sites’ Adherence to BCO Employment Services Model

0%

20%

40%

60%

80%

100%

Site A Site B Site C Site D Site E Site F Site G

Treatment Sites

0%

20%

40%

60%

80%

100%

Site A Site B Site C Site D Site E Site F Site G

Treatment Sites
IMPLEMENTATION STUDY

Abt Associates BCO Evaluation Final Report April 2021 ▌16

Exhibit 10 shows the percentage of sites that adhered to implementing each activity specified for the BCO 

financial coaching and education, and income support services. Three of the sites implemented 100 

percent of the activities “as intended;” three sites partially implemented one activity; and one site partially 

implemented two activities.

Exhibit 10. Extent of Treatment Sites’ Adherence to BCO Financial Coaching and Education, and 

Income Support Services Model

Overall, the seven BCO treatment sites demonstrated a high degree of adherence to the BCO model with 

two of the sites implementing 100 percent of the BCO activities in the four components of services “as 

intended.” Of the four components, the employment coaching services had the highest degree of 

adherence to the BCO model, with five sites implementing 100 percent of the employment services “as 

intended.”

0%

20%

40%

60%

80%

100%

Site A Site B Site C Site D Site E Site F Site G

Treatment Sites
IMPACT STUDY FINDINGS

Abt Associates BCO Evaluation Final Report April 2021 ▌17

Impact Study Findings

The impact study examined whether the BCO model, which integrates bridge services with FOC services, 

better prepares participants with low incomes to increase their employment prospects and financial 

stability than do FOC services alone. To answer the research questions on employment and financial wellbeing, the Abt team used a quasi-experimental design and compared the outcomes of BCO participants 

(the treatment group) with a matched group of FOC participants (the comparison group). 

Across seven organizations’ BCO programs (Brighton Center, Inc.; Chinese Community Center, Inc.;

District 1199C Training and Upgrading Fund; Instituto del Progreso Latino; International Institute of 

Minnesota; Project for Pride in Living, Inc.; and Wesley Community Center, Inc.) and six organizations’ 

FOC programs (Asociación Puertorriqueños en Marcha, Cara, HumanKind, Prosperity Center for 

Financial Opportunity, Sacred Heart Community Service, and Urban League of Essex County), eligible 

BCO participants were exactly matched to one or more eligible FOC participant on gender, age, 

race/ethnicity, highest level of education completed, employment status, and credit score at intake using a 

coarsened exact matching (CEM) process6

. 

The final analytic sample that examined the confirmatory employment research question included 673 

treatment group participants (who received the BCO bridge services) and 566 comparison group 

participants (who did not receive the BCO bridge services); the sample for the analysis of the 

confirmatory financial well-being research question included 181 treatment group participants and 79 

comparison group participants (Exhibit 11).

A detailed description of this impact evaluation’s design, sample recruitment and retention, baseline 

equivalence, methods, and analysis procedures are in Appendix B.

Impact Study Questions, Data Sources, and Analytic Approach

Confirmatory Research Questions

This section of the report is organized by the confirmatory research questions of interest7

:

• Were BCO participants more likely to obtain or advance in a job after program entry relative to a 

comparison group who received only FOC services? (Employment)

• Were BCO participants more likely to have obtained or increased their credit scores after program 

entry relative to a comparison group who received only FOC services? (Financial Well-Being) 

 

6 Iacus, King, & Porro, 2012; Berta, Bossi, & Verzillo, 2017.

7 The study examined the effects of the BCO model on multiple measures of employment and financial well-being, as well as 

on the pre-post change of an education measure for BCO participants. The confirmatory measures of the study focus on job 

improvement and credit score increase after program entry. The study also examined exploratory measures of employment

(wage increase, hourly wage, and number of hours worked); three exploratory measures of financial well-being (credit score 

increase, overall debt, and net worth); and one exploratory measure of education (reading skills). Appendix C provides the 

results of the exploratory measures.
IMPACT STUDY FINDINGS

Abt Associates BCO Evaluation Final Report April 2021 ▌18

Exhibit 11: Flow from Eligible BOC and FOC Participants to Analytic Study Sample

Impact Study Data Sources

Because study participants were drawn from each site’s existing service population, all analyses were

based on extant administrative data that the sites submitted to LISC. BCO and FOC site staff collected

and stored all client demographic and background information, service receipt, and outcome data in their 

Salesforce data system using the same procedures. Staff collect baseline data when clients enter a BCO or 

an FOC-only program, prior to the delivery of an intervention; document the activities in which clients 

participate as services are provided (within 48 hours of data receipt); and collect follow-up data on 

clients’ employment, financial well-being, and education outcomes (when clients initiative contact with 

staff or staff contact clients).

The impact study job advancement data, for BCO and FOC-only participants were drawn from Salesforce 

advancement and employment records and the credit score data were drawn from the Salesforce credit 

score record. 

Analytic Approach

Participant outcomes were analyzed in a linear regression model that tested the effect of BCO on job 

advancement and credit score increase. To account for the matching block design, we estimated the 

impact of BCO using models that include indicators for the matching blocks used in the design. Those

indicators ensured that the overall estimate of the treatment-comparison group outcome difference was

calculated by comparing the BCO and FOC participant outcomes within the matched blocks and 

calculating a precision-weighted average of the within-block differences to produce the overall estimate. 

To improve precision and control for possible baseline differences between the groups that might be 
IMPACT STUDY FINDINGS

Abt Associates BCO Evaluation Final Report April 2021 ▌19

related to outcomes, the model also included covariates for participants’ gender, age, race/ethnicity and 

baseline level of education, employment, and credit score.

For the reading test exploratory analysis, learners’ raw scores were converted into scale scores. Scores 

that represent the amount of a participant’s skill change from pre- to post-tests were constructed by 

subtracting each participant’s pre-test scale score from their post-test scale scores. The statistical 

significance of the mean gain was assessed using the paired t-test.

Findings from Impact Analysis

This section presents the results from the analyses of the impact study’s confirmatory research questions. 

Detailed information about the results from the confirmatory analyses are presented in Appendix C in 

Exhibit C-1 (Impacts on Employment) and Exhibit C-3 (Impacts on Financial Well-Being).

BCO increased the share of participants who had a job improvement. 

The BCO model assumes that contextualized bridge services which are integrated with financial coaching 

and education, income support services, and employment services will benefit adults with lows skills or 

who have difficulty obtaining employment. BCO services are expected to move adults with low skills and 

low incomes toward family-sustaining employment by first increasing their basic skills and credentials,

which in turn, can enable them to qualify for jobs with family-living wages. Hence, the study examined 

whether BCO participants were more likely to have a job improvement after program entry relative to a 

comparison group who received only FOC services. Overall, participants who were in the BCO programs 

were 19 percentage points more likely to have a “job improvement”8 than were participants in FOC-only

programs (56 percent versus 37 percent). These results are statistically significant at p<.001 (Exhibit 12). 

Exhibit 12. Impact of BCO on Whether Participants had a Job Improvement after Program 

Enrollment

Note: ***Difference is statistically significant at the .001 level. Data include 673 participants in BCO programs and 566 participants in FOC 

programs. Percentage of participants represents those who had a “job improvement” in 1-14 months after program enrollment.

Source: BCO and FOC administrative data in Salesforce: FFTTM Employment Record/Job Advancement Record.

 

8 Job advancement is defined as follows: the participant either (1) enrolls in the program unemployed and starts a new job in 

the 1-14 months after enrollment or (2) enrolls in program employed and has one of the following records in the 1-14 

months after enrollment: (a) obtains a new job and the new wage is greater than the wage at enrollment, or (2) has an 

advancement record for an increase in wages or a promotion.

55.6

36.8

0

10

20

30

40

50

60

70

80

90

100

Percentage of participants

BCO Participants FOC Participants

***
IMPACT STUDY FINDINGS

Abt Associates BCO Evaluation Final Report April 2021 ▌20

There was No Significant Effect of BCO on Improving Participants’ Credit Score

BCO services are intended to provide clients with financial coaching and education that will enable them, 

over time, to increase their assets and financial stability. The impact study examined whether BCO 

participants were more likely to have an increase in their credit score after program entry relative to a 

comparison group who received only FOC services. About 68 percent of participants in the BCO program 

and about 80 percent of participants in the FOC-only program had an increase in their credit score after 

program entry. The results were not statistically significant (Exhibit 13).

Exhibit 13. Impact of BCO on Credit Score Increase after Program Enrollment

Note: Data include 181 participants in BCO programs and 79 participants in FOC-only programs. Percentage of participants represents those 

whose maximum credit score after program enrollment was higher than their credit score at intake.

Source: BCO and FOC administrative data in Salesforce: FFTTM Credit Score Record. 

Descriptive Analyses of Treatment Group’s Education Outcomes

Abt conducted descriptive analyses of the BCO treatment group participants’ education outcomes to 

understand participants’ progress in developing their foundational skills while enrolled in BCO, in 

completing BCO, and in earning educational credentials. The findings are discussed below. 

Information about each BCO treatment site’s TABE® results is presented in Appendix C, Exhibit C-5-

(TABE® Pre-Post Test Results) and Exhibit C-6 (TABE Pre-Test Grade-Equivalent Levels).

BCO Participants’ Foundational Literacy Skills

As part of BCO’s bridge services, BCO treatment sites were to provide contextualized instruction to 

increase the foundational literacy skills of BCO participants who had low basic skills. To examine the 

effects of BCO on literacy skills, we compared the pre- and post-scale score on the Test for Adult Basic 

Education (TABE®) for all BCO participants (not just those in the matched samples). Of the 1,133 

treatment group participants in the evaluation, 594 individuals had TABE® Reading pre-test data, and of 

those with valid pre-test data, 282 (47.5%) had valid post-test data.

Presented in Exhibit 14 are the results of the TABE® Reading pre-posttest analysis. BCO participants’ 

average TABE® pre-test score of 532 is equivalent to approximately the 6.6 grade-level. At the post test, 

68.0

79.8

0.0

10.0

20.0

30.0

40.0

50.0

60.0

70.0

80.0

90.0

100.0

Percentage of participants

BCO Participants FOC Participants
IMPACT STUDY FINDINGS

Abt Associates BCO Evaluation Final Report April 2021 ▌21

BCO participants had increased their reading scores by 9 points or by about one grade-level to the 7.4

grade-level equivalent (CTB/McGraw-Hill, 2004). This increase was statistically significant at p<.001.

Exhibit 14. TABE® Reading Pre-Post Results 

Outcome

Average

Pre-Test 

Score

Average

Post-Test 

Score

Average

Pre- to PostTest Gain Score

Standard 

Deviation of 

Pre-Test

Standardized 

Test Score 

Gain

P-Value of 

Gain from 

Paired t-Test

TABE® Reading 

Scale Score

532 541 9 65.99 0.14 0.0002***

Note: *** Difference is statistically significant at the .001 level. Data include 282 participants in BCO programs that had pre- and post-test 

TABE® Reading scores.

Source: BCO administrative data in Salesforce: FFTTM TABE® record.

Analysis of Treatment Group’s Completion of Education or Training

We analyzed the extent to which the treatment group (1,133 participants) completed their education or 

training program and obtained an educational or industry credential by the time of follow-up. As shown in 

Exhibit 15, at follow-up, the majority (86.1%) of treatment group participants had completed their BCO 

education or training program. As shown in Exhibit 16, of the treatment group participants with data who 

completed training or education, the majority (79.7%) also earned either an education credential or an 

industry credential. In one BCO program, the program’s completion certificate was the endpoint for

success in that program and was recognized by local employers.

Exhibit 15. Education or Training Program Completion Status of Treatment Group

Training Program Completion Status for Treatment Group Frequency Percentage

Completed education/training program 878 86.1%

Did not complete education/ training program 142 13.9%

N Missing: 113

Source: BCO administrative data in Salesforce: FFTTM Education/Training Record.

Exhibit 16. Number of Credentials Earned by Treatment Group

Number Frequency Percentage

1 586 79.7%

2 134 18.2%

3 14 1.9%

4 1 0.1%

N Missing: 398

Source: BCO administrative data in Salesforce: FFTTM Education/Training Record.

We analyzed the last credential that treatment group participants earned by the time of follow-up. As 

shown in Exhibit 17, more than half (56.9%) of treatment group participants earned a credential in a

healthcare occupation, such as Certified Nurse Assistant and Medical Assistant, which reflects the type of 

occupational training offered in BCO programs.
IMPACT STUDY FINDINGS

Abt Associates BCO Evaluation Final Report April 2021 ▌22

Exhibit 17. Type of Credentials Earned by Treatment Group

Credential Type Frequency Percentage

Healthcare 418 56.9%

Child Development 64 8.7%

NIMS -Manufacturing 42 5.7%

Hospitality 35 4.8%

Business Technology 23 3.1%

Environmental 0 0.0%

Educational degree 122 16.6%

Ancillary certificates and licenses 31 4.2%

N Missing: 398

Source: BCO administrative data in Salesforce: FFTTM Education/Training Record.
CONCLUSIONS

Abt Associates BCO Evaluation Final Report April 2021 ▌23

Conclusion

Conclusions

The evaluation found that the BCO model, with well-defined services and training and technical 

assistance support, can be implemented by community-based organizations with strong adherence to the 

model. The evaluation also indicated that BCO’s goal to facilitate clients’ attainment of educational 

credentials and employment on their path to economic well-being can be attained through the delivery of 

the multi-component services that define the BCO model. Although BCO evaluation participants were not 

able to make significant advances in their economic status during the 14 months after they enrolled in the 

BCO program, they were able to earn credentials and advance in their jobs. Those outcomes are thought 

to be steps toward economic well-being. The process of increasing individuals’ economic status is 

complex and affected by many factors. Although this evaluation provided preliminary insights about nearterm changes in participants’ economic status, more robust investigations with larger samples of 

participants are needed to understand adults’ attainment of economic well-being.

Fidelity of BCO Implementation

The seven treatment sites implemented the BCO model’s four components of activities with a high degree 

of fidelity. Many factors contributed to this result including the ongoing training and technical assistance 

that LISC staff provided to its grantees and the organizational resources of the seven programs that 

participated as treatment sites. Another factor is that the BCO model is specified with sufficient detail to 

guide programs’ main activities in each of the four BCO components but is not so prescriptive that 

programs cannot adjust services as they encounter challenges. This model design enabled the treatment 

programs to respond to the needs of their client populations while delivering the range of services that 

compose the BCO model. 

Education Outcomes

The majority of BCO treatment participants completed the BCO program (86%) and most of those 

completers earned at least one credential, which was primarily an occupational certificate (80%). The 

treatment sites’ approaches to recruitment and orientation helped clients understand the requirements and 

commitment needed to succeed in the BCO program. The BCO bridge design also provided clients with 

an opportunity to master skills in the bridge component that could contribute to their success in the 

occupational training. The ongoing support provided by BCO coaches and instructors helped clients to 

persevere and keep on track as they worked to earn an educational credential.

Employment Outcomes

The evaluation results indicated that BCO treatment participants were significantly more likely to obtain 

or advance in a job after enrollment than FOC comparison participants. Factors contributing to this 

outcome were the comprehensive job readiness and job placement services that the BCO sites provided. 

As the BCO implementation study indicated, the treatment sites’ employment services were the most 

highly aligned of the four BCO components of services such that BCO sites provided many similar 

services. Another factor likely contributing to the BCO participants’ job placement and advancement was 

the partnerships that BCO sites formed with local employers. BCO staff reported that those partnerships 

not only facilitated participants’ job attainment but also help support participants as they moved along in 

their employment.

Financial Well-Being Outcomes

The evaluation found that FOC comparison group participants were more likely to increase their credit 

scores than BCO treatment group participants. Comparison group participants also were more likely than 
CONCLUSIONS

Abt Associates BCO Evaluation Final Report April 2021 ▌24

treatment participants to decrease their debt, increase their net worth, and have a higher amount of debt 

decrease. Comparison group participants’ earlier participation in the workforce after program enrollment 

may have enabled those participants to engage in financial activities helped to increase their financial 

well-being.

Lessons Learned and Limitations

One lesson from the evaluation concerns processes to mitigate the loss of administrative data. Although 

LISC and Abt encouraged sites throughout the evaluation to follow established protocols for entering 

client data, more frequent monitoring of the BCO and FOC data in Salesforce might have resulted in less 

loss of data. 

Because of the evaluation’s quasi-experimental design, the BCO treatment programs participating in the 

evaluation were not selected to represent the full set of BCO programs but were purposively selected 

because of their planned services, client population, stability of BCO program implementation, and 

projected number of participants. The FOC comparison programs were purposively selected for the 

evaluation because their clients who entered services during the evaluation’s enrollment period had 

similar background characteristics as the BCO clients in the treatment group. The BCO and FOC 

programs were also in cities with similar labor markets. For those reasons, the evaluation’s results may 

not be applicable to clients with background characteristics different from the backgrounds of evaluation 

participants or because of possible differences in BCO program services. 

The goal of using coarsened exact matching (CEM) in the evaluation was to create treatment and 

comparison groups that were similar, before the start of clients’ participation in the BCO and FOC 

programs, on observed characteristics likely to be related to the education, employment, and financial 

outcomes of interest. Therefore, any differences in education, employment, and financial outcomes 

observed can be attributed to the BCO program (or unobserved differences) rather than to initial 

differences between the groups. The treatment and comparison groups’ analytic samples were shown to 

be equivalent at baseline on observed characteristics, even after the loss of sample due to missing 

outcome data. However, a limitation of the study design is that differences observed may be due to 

unobserved characteristics of participants or the BCO program. Further, because the missing data led to 

small sample sizes for the financial outcome, the evaluation may not be powered to detect small 

differences between the groups.
APPENDIX A: BCO Logic Model

Abt Associates BCO Evaluation Final Report April 2021 ▌25

Appendix A: BCO Logic Model

The Bridges to Career Opportunities (BCO) model assumes that contextualized bridge services which are 

integrated with financial coaching and education, income support services, and employment services will 

benefit adults with low skills and have difficulty finding employment. The BCO logic model is presented 

in Exhibit A-1. Inputs are the resources needed to ensure implementation of the BCO model. They 

include but are not limited to LISC technical assistance to BCO grantee sites to facilitate their 

implementation of the BCO model; funding to BCO grantees to help support the delivery of BCO 

services; and grantees that provide an organizational structure and community partnerships for operating 

BCO services. 

Those inputs are expected to enable organizations to carry out BCO activities. The BCO model has four 

program components: 

• A contextualized bridge program to increase clients’ basic skills within the context of an 

academic and career pathway in a specific growth sector or industry (e.g., healthcare);

• Financial coaching and education to help clients solve specific financial problems; plan for 

financial stability; and connect to financial service providers and financial products for saving 

and building credit; 

• Income support services to help clients access public benefits for which they are eligible that 

can enable them to participate in and complete training; and

• Employment services to provide job readiness training, job search, and job placement.

Together those activities are hypothesized to change clients’ proximal outcomes of educational 

attainment, employment, and financial well-being. The distal outcomes, made possible by the proximal 

outcomes, reflect BCO’s goal of helping adults with low incomes improve their financial stability and 

obtain jobs with living wages.
APPENDIX A: BCO Logic Model

Abt Associates BCO Evaluation Final Report April 2021 ▌26

Exhibit A-1. Logic Model for BCO 

Inputs Activities Proximal Outcomes Distal Outcomes

LISC technical 

assistance

Funding

Community-based 

organizations

Contextualized Bridge Program*

• Addresses career pathway in growth sector or industry

• Includes an academic pathway

• Teaches basic skills needed to access educational programs and 

that are contextualized to a specific career path, which in turn 

prepares clients for occupational skills training and “middle 

skills” jobs

• Based on industry-specific curriculum

• Enables clients to attain “industry-recognized” credentials 

• Has measurable endpoint or clearly defined completion criteria

• Pre- and post-test skills assessments. Pre-test is used as part of 

clearly defined entry criteria

• Ongoing career coaching connects academic and career 

pathways

Financial Coaching and Education

• Combined Financial Assessment

• Regular 1:1 interactions concerning financial status

• Credit reports run every 6 months

• Uses financial products during coaching

• May be integrated into bridge course

Income Support Services

• Public benefits screening

• Supports to enable clients to remain in training

• Needs are reassessed as clients’ situations change

Employment Services

• Assistance with preparing job-search materials and conducting 

job search

• Counseling about job search, placement, retention, and 

advancement

• Career exploration support

Education

At program completion

• Increase basic skills

12 months after program 

entry

• Earn postsecondary or 

occupational credential 

Financial Well-Being

12 months after program 

entry

• Obtain or increase credit 

score

• Increase percentage of 

accounts with on-time 

payments 

 

Employment

12 months after program 

entry

• Obtain job

• Increase hourly wages

• Obtain higher hourly 

wages in first placement

• Increase hours worked

• Increase job retention

• Decrease time to job 

Improve financial stability 

and obtain family-sustaining 

employment

* Addition of the contextualized bridge program is the key 

difference between the FOC model and the BCO model. 
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌27

Appendix B: Additional Information about the Evaluation 

Design and Impact Study

This appendix provides details about the evaluation design and its implementation to address the 

evaluation’s research questions. Discussed in this appendix are the research questions, sample, data 

sources, measures, and analytic methods of the evaluation.

Research Questions

The evaluation examined whether the BCO model, which integrates bridge services with FOC services, 

better prepared participants with low skills or who are hard to employ to increase their employment 

prospects and financial stability than FOC services did alone. The impact evaluation was guided by 

confirmatory and exploratory research questions. 

The confirmatory questions tested a key premise of the BCO model compared to the FOC model—that 

BCO programs, by providing education services to increase low-skilled clients’ foundational literacy and 

numeracy skills, can improve clients’ success in occupational training and credentialing programs and 

enable them to qualify for jobs with family-sustaining wages that lead to financial well-being. The Abt

team examined two confirmatory research questions for the study in two separate domains: 

1. Were BCO participants more likely to obtain or advance in a job after program entry relative to a 

comparison group who received only FOC services? (Employment domain)

2. Were BCO participants more likely to have obtained or increased their credit scores after program 

entry relative to a comparison group who received only FOC services? (Financial Well-Being 

domain)

All other research questions were exploratory and concerned the effects of BCO services on outcomes in 

the domains of employment, financial well-being, and education.

Exploratory Research Questions Regarding Employment

Because BCO services are intended to place clients in better jobs by first increasing their basic skills and 

credentials, employment impacts are expected to be realized after clients complete occupational training. 

For the impact study, this period was 1-14 months after clients’ enrollment in BCO services.9

 

9 Abt’s approach to the employment data analyses, as described in the Abt’s evaluation plan (Gan et al., 2016)9 was to 

compare the treatment and comparison groups’ results for a number of employment outcomes at 12 months after 

participants’ enrollment in BCO services (treatment group) or FOC services (comparison group). The timeline assumed that 

BCO and FOC local staff would conduct follow-up data collection 12 months after enrollment. However, because the BCO 

model allows for client-driven interactions, only a portion of the follow-up data collection occurred at 12 months after 

enrollment. This client-driven approach made it difficult to gather point-in-time information, so the model itself did not 

necessarily lend itself to a 12-month follow-up. Rather, sites collected follow-up data, defined as any data collected after 

baseline, beginning in month 1 after enrollment and continued throughout the 14-month period in which Abt tracked clients’ 

participation in BCO or FOC services. BCO and FOC staff also reported that part of the variation in the timeline of followup data collection was due to sites’ difficulty in reaching clients after they ended their participation in program services. Abt 

analyzed employment outcomes beginning in month one after enrollment through month 14, which was the longest period 

all clients in the evaluation could participate in follow-up data collection.
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌28

The exploratory research questions to assess impacts on employment are:

• Were BCO participants more likely to increase their hourly wages after program entry relative to 

a comparison group who received only FOC services?

• Were BCO participants more likely to be placed in jobs with higher hourly wages relative to a 

comparison group who received only FOC services?

• Were BCO participants more likely to work more hours per week after program entry relative to a 

comparison group who received only FOC services?

Exploratory Questions Concerning Financial Well-Being 

BCO services are intended to provide clients with financial coaching and education that will enable them, 

over time, to increase their assets and financial stability. For the impact study, this period is 1-14 months 

after clients’ enrollment in BCO services.10

The exploratory research questions to assess impacts on financial well-being are:

• Were BCO participants more likely to have lower overall debt after program entry relative to a 

comparison group who received only FOC services?

• Were BCO participants more likely to have increased their net worth after program entry relative 

to a comparison group who received only FOC services?11

Exploratory Question Concerning Education

The exploratory research question12 in the education domain provides context for the evaluation and was

examined for the treatment group only, as the comparison group did not participate in a basic skills 

assessment as part of their FOC participation: 

• Were BCO participants likely to increase their reading skills (as measured by a standardized test) 

after participating in the BCO program? 

To answer the employment and financial well-being research questions, Abt used a quasi-experimental 

design and compared outcomes of BCO participants’ (the treatment group) to a matched group of FOC 

 

10 Abt’s approach to the financial well-being analyses as described in Abt’s evaluation plan (Gan et al., 2016) was to compare 

the treatment and comparison groups’ performance on three financial well-being outcomes at 12 months after participants’ 

enrollment in BCO services (treatment group) and FOC services (comparison group). Similar to the employment analyses, 

only a portion of the follow-up data collection on financial well-being occurred at 12 months; rather, sites collected followup data, defined as any data collected after baseline, beginning in month 1 after enrollment and continued throughout the 14-

month period in which Abt tracked clients’ participation in BCO or FOC services. Because of the timeline of the follow-up 

data collection, Abt revised its approach and analyzed financial well-being outcomes beginning in month 1 after enrollment 

through month 14, which was the longest period all clients in the evaluation could participate in follow-up data collection.

11 The third question specified in Abt’s evaluation plan (Gan et al., 2016) concerned percentage of on-time account payments. 

Abt could not address this question due to lack of data on on-time account payments. Thus we examined increase in net 

worth.

12 The confirmatory research question for the evaluation’s education outcome was whether BCO participants were more likely 

to have attained an occupational certificate or postsecondary credential 12 months after program entry relative to a 

comparison group who received only FOC services. Abt’s analysis of education participation data for the comparison group 

participants revealed than only four participants had records of attaining occupational certificates or postsecondary 

credentials at follow-up. Thus we could not address this question in the evaluation.
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌29

participants (the comparison group). The pre- and post-outcomes of BCO participants (the treatment 

group only) used a paired pre-post analysis to descriptively address the research question on education.

Feasibility Study to Select Study Sample

This section describes how the Abt team identified the BCO and FOC programs for the study, how 

eligible participants from each program were identified, how the comparison group was formed to address 

the research questions, the samples included for each analysis, and the baseline equivalence of the groups.

Treatment Sites and Participant Sample 

Seven organizations’ BCO programs (treatment group) were purposively selected for the study and 1,133 

of their clients were eligible to be included in 

the evaluation. 

Site selection was a multi-step screening process 

illustrated in Exhibit B-1. LISC staff began with

32 BCO subgrantee programs that serve the 

target population for this evaluation—

individuals who are unemployed or 

underemployed (adults with low incomes and 

low skills) and have not experienced success in 

traditional education. Of those, they identified 

20 BCO grantee programs that were deemed to 

be high-capacity sites with prior experience in 

delivering the bridge component and had

sufficient projected baseline enrollment to meet 

the requirements of the evaluation. 

Abt reviewed the 20 programs’ BCO 

applications and eliminated six programs from 

consideration because their timeline for 

implementation of bridge services did not align 

with the evaluation timeline, the structure of 

their bridge services was not conducive to the 

evaluation, and/or their anticipated enrollment

numbers were not sufficient to support the 

evaluation. 

We then conducted two rounds of telephone 

interviews with the 14 remaining candidate sites 

to clarify their planned services, client 

populations (spring 2016), stability of BCO 

program implementation, and projected number 

of participants (summer 2016). The multiple 

rounds of screening were necessary to assess the

programs’ BCO activities compared to the BCO 

model as it was articulated for the purposes of 

the evaluation, and to identify the availability of comparison groups. 

Exhibit B-1. Site Selection Process
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌30

Based on this information and sites’ interest in participating in the evaluation, seven BCO programs were 

selected as treatment sites in fall 2016. The seven treatment sites are shown in Exhibit B-2. Because the 

treatment sites provide a range of occupational training programs aimed at different client populations, 

some of which are not the target population for this evaluation, we worked with each treatment site to 

specify the appropriate BCO bridge training programs (i.e., programs offering bridge and FOC services)

to include in the evaluation. 

Eligible participants from those sites were clients in the appropriate BCO bridge training programs who:

• enrolled in BCO services during the period April 1, 2017 and through December 31, 2018;

• enrolled in an eligible training course; and

• had consented to have their BCO data shared at the time of BCO enrollment. 

Exhibit B-2. Seven BCO Treatment Sites

Site Location Eligible Participants

Brighton Center, Inc. Greater Cincinnati and Northern Kentucky 217

Chinese Community Center, Inc. Houston, TX 117

District 1199C Training and Upgrading Fund Philadelphia, PA 130

Instituto del Progreso Latino Chicago, IL 146

International Institute of Minnesota St. Paul, MN 354

Project for Pride in Living, Inc. Minneapolis, MN 24

Wesley Community Center, Inc. Houston, TX 145

Total 1,133

Comparison Sites and Participant Sample 

Six organizations’ FOC programs (comparison group) were purposively selected for the study and 1,217

of their clients were eligible to be included in the evaluation. The six comparison sites are shown in 

Exhibit B-3.

LISC recommended possible FOC comparison sites, and Abt collected information about seven FOCs to 

assess their appropriateness. We reviewed extant information about the FOCs and conducted telephone 

interviews with the FOC leadership in each site. During the interviews, we gathered information about the 

sites’ delivery of FOC services and confirmed that the FOCs were not providing education services to 

FOC clients. Abt also determined the sites’ willingness to serve as comparison sites. 

After the interviews, we conducted a preliminary analysis of the sites’ Salesforce data for clients who had 

enrolled in FOC services during the period April 1, 2017, through December 31, 2018, the dates for 

enrollment of BCO clients participating in the BCO evaluation. Those data analyses examined the extent 

to which the background characteristics of clients who enrolled in the FOCs during the evaluation 

enrollment period were like the background characteristics of the treatment group. We also examined the 

comparability of the labor markets in the cities where the BCO and FOC sites were located. Based on 
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌31

those analyses and review, Abt determined that the seven FOCs13 were suitable comparison sites because 

they:14

• did not provide educational services for clients;

• were in cities with similar labor markets and served FOC clients with similar demographic 

characteristics as the BCO sites selected for the study; and

• collected the evaluation’s required data for FOC participants and delivered FOC services 

according to the FOC model.

Exhibit B-3. Six FOC Comparison Sites

Site Location Eligible Participants

Asociación Puertorriqueños en Marcha Philadelphia, PA 7

Cara Chicago, IL 503

HumanKind (formerly Southside Community Development 

and Housing Corporation) Richmond, VA 53

Prosperity Center for Financial Opportunity Kansas City, MO 62

Sacred Heart Community Service San Jose, CA 351

Urban League of Essex County Newark, NJ 241

Total 1,217

Eligible participants from these sites are clients in the appropriate FOC programs who:

• enrolled in an FOC program (with no BCO services) during the period April 1, 2017 and 

December 31, 2018; and

• had consented to have their data shared at the time of FOC enrollment.

Exhibit B-4 shows the demographic characteristics of those BCO and FOC participants at program intake. 

 

13 Seven FOCs initially agreed to participate in the evaluation. However, after Abt began to work with the comparison sites 

regarding their follow up on missing client data, one FOC determined it did not have the staff to obtain missing data and 

decided to discontinue its participation in the evaluation. Thus six FOCs participated.

14 Abt considered three options for selecting FOC programs that could serve as the comparison sites for the evaluation. One 

option was to use the FOCs in the same organizations selected as BCO treatment sites. We found that to meet their BCO 

enrollment goals, the treatment sites recruited BCO clients from their organization’s FOC programs, which meant the 

remaining FOC clients were not available for or not interested in participating in the bridge services at that time. They were 

not an appropriate comparison group because their motivation differed from that of the treatment group. Abt also considered 

other FOC programs in the same cities as the BCO treatment sites, which would control for labor market conditions. 

However, our review of client data in those FOC programs indicated that clients were not similar in demographic 

characteristics to the BCO participants eligible for the study.
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌32

Exhibit B-4. Characteristics of Eligible BCO and FOC Participants

BCO Eligible a

Participants

(%)

FOC Eligible b

Participants

(%)

Gender

Female 84.8 56.0

Race/Ethnicity

Black/African American non-Hispanic 51.3 58.5

White non-Hispanic 13.5 9.9

Hispanic 24.1 25.1

Other (American Indian, Asian, Multiracial, Biracial, Other) 11.1 6.6

Age at Intake

17-24 27.8 13.8

25-34 33.9 23.7

35-44 19.4 18.6

45-54 13.1 22.4

55+ 5.9 21.5

Marital Status at Intake

Divorced 7.4 8.9

Married 24.6 12.7

Separated 7.5 6.4

Single 59.6 70.6

Widowed 1.0 1.5

Primary Language Spoken

English 62.0 86.3

Level of Education at Intake

No High School Diploma/Equivalency 10.1 15.9

High School Diploma or High School Equivalency 49.8 48.6

Some College 23.9 14.6

College Certificate or Degree 15.9 20.8

Employed at Intake

Yes 40.9 20.3

FICO Credit Score Range at Intake

Exceptional (800+) 0.3 1.1

Very Good (740-799) 7.5 3.6

Good (670-739) 16.5 8.0

Fair (580-669) 21.1 16.3

Poor (<580) 29.6 25.0

Insufficient Credit Score 23.5 40.8

Non-FICO Credit Score Available 1.5 5.3

Total 1,133 1,217

a Eligible participants are those who enroll in BCO services during the period April 1, 2017 and December 31, 2018, enrolled in an eligible 

training course and consented to participate in this evaluation. b Eligible participants are those who enrolled in a FOC program (with no BCO services) during the period April 1, 2017 and December 31, 2018

and consented to participate in this evaluation.

Sample Sizes:

Gender: BCO Participants = 1,125 clients, FOC Participants = 1,203 clients
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌33

Race/Ethnicity: BCO Participants = 1,120 clients, FOC Participants = 1,207 clients

Age at Intake: BCO Participants = 1,119 clients, FOC Participants =1,204 clients

Marital Status at Intake: BCO Participants =1,099 clients, FOC=1,181 clients

Primary Language Spoken: BCO Participants =1,085 clients, FOC Participants =1,179 clients

Level of Education at Intake: BCO Participants =1,105 clients, FOC Participants =1,196 clients

Employed at Intake: BCO Participants =1,108 clients, FOC Participants =1,160 clients

FICO Credit Score Range at Intake: BCO Participants =720 clients, FOC Participants =816 clients

Sources: BCO and FOC administrative data in Salesforce: FFTTM Intake Record, FFTTMEmployment Record, FFTTMCredit Report Record.

Formation of the Treatment and Comparison Group

To ensure that the BCO and FOC participants were comparable at program entry for the impact analyses,

a coarsened exact matching (CEM)15 16 was implemented. CEM controls for selection bias and other 

biases related to group assignment (although not perfectly) by accounting for multiple characteristics of 

individuals. With this method, the eligible BCO participants are exactly matched to one or more of the 

eligible FOC participants on the variables17 shown in Exhibit B-5 across sites.

Exhibit B-5. Characteristics of Eligible BCO and FOC Participants

Domain Baseline Measures Used for Matching

Demographic characteristic • Gender

• Race/ethnicity

• Age 

Education • Highest level of education completed

Employment • Employment status

Financial well-being • Credit score

The full set of baseline measures is described in more detail in Exhibit B-10. Each of the matching 

variables was measured at program intake prior to the start of clients’ services (except for credit score, 

 

15 Change to SEP: The SEP specified that the impact evaluation would o use Propensity Score Analysis (PSA), a matching 

technique wherein participants in a program are matched to non-participants based on their participation propensity score. 

This technique uses characteristics measured before the start of a program to determine the probability (the propensity score) 

that participants are in the treated group. After assigning propensity scores to individuals, participants are placed into blocks 

(or matching strata) such that the treatment and control individuals within each block have approximately equal propensity 

to be in the treated group. PSA is useful for selecting comparison members on a large number (20 or more) of 

characteristics. Given the small number of variables identified for matching in the BCO evaluation, Abt selected the CEM 

approach instead of a PSA matching approach. Because all matching variables are combined into a summary score, PSA

does not guarantee close or exact matches on variables of interest. However, CEM ensures exact matches on variables of 

interest (because treatment and comparison matches within the same block have identical values on coarsened matching 

variables). A drawback of CEM is that it does not work well with more than a few matching variables.

16 Iacus, King, & Porro (2012); Berta, Bossi, & Verzillo (2017).

17 Some of the matching variables were selected because they are baseline measures of the outcomes for the study—i.e., level 

of education, employment status, and credit score range. We also considered other variables that prior studies have shown to 

be related to or predictive of outcomes—that is, gender, age, race/ethnicity, marital status, type of training program, and 

income. Prior studies show that there is a correlation between types of training (e.g., for traditionally female jobs such as 

certified nursing assistant and child development associate) and income (traditionally female jobs pay less than entry-level 

traditionally male jobs), hence we included gender as a matching variable to address these issues. More than half of the 

participants were White non-Hispanic or Hispanic across treatment sites, so we recruited comparison sites with a majority of 

Hispanic or White non-Hispanic participants and matched on race/ethnicity. Marital status is important in that it is related to 

financial status, but the Abt team already matched on employment status and credit score, hence we did not match on marital 

status. We also could not match on income because there is a very high rate of missing data for all participants, so we used 

baseline employment status as a proxy for income.
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌34

which may be measured up to 5 weeks18 after intake). The variables identified for matching are the 

baseline measures of the outcomes of interest or have been found to be associated with (or predictive of—

depending on the design) the outcomes of interest in previous studies. Matching on these variables should 

significantly account for observable differences between BCO and FOC participants (e.g., Roder 2016, 

Rankin 2015). However, one way in which BCO and FOC participants may vary is in why they chose to 

enroll in the BCO or FOC program, which the evaluation was unable to observe.

The CEM was applied to the data using a SAS macro written by Berta, Bossi, & Verzillo (2017). In this 

process, blocks were created where within blocks’ participants were exactly matched on highest level of 

education, employment status, gender, race/ethnicity, age, and credit score. Blocks were created by 

forming categories for the BCO participants across the matching variables (e.g., an employed, white 

female, 25-34 years old with a high school diploma and a credit score between 500 and 550) and finding 

FOC participants with the exact same combination of characteristics. If a given block did not contain at 

least one BCO and one FOC participant, it was dropped. 

Abt formed separate comparison groups for the employment and financial impact analyses because of the 

high percent of missing values on credit scores for both the BCO and FOC participants. There is no 

correlation between credit score and employment outcomes. Hence, credit score was included in the 

match when looking at financial outcomes but was not included in the match when examining educational 

outcomes. Both matches only included participants with non-missing data on all matching variables.

Exhibits B-6 and B-7 list the number of participants matched across sites for both comparison groups.

 

18 The BCO and FOC sites’ downloading of clients’ baseline credit scores generally occurred during the first meeting between 

clients and their financial coach. For BCO clients whose bridge training program did not begin at time of program 

enrollment because of the schedule, the meeting between clients and their financial coach may not occur until after clients 

began the training, which could be about five weeks after enrollment.
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌35

Exhibit B-6. Employment Outcomes: Number of Participants Matched per Site

(excludes Credit Score from the Match)

Site Number of Matched BCO Participants

BCO Sites

Brighton Center, Inc. 182

Chinese Community Center, Inc. 102

District 1199C Training and Upgrading Fund 112

Instituto del Progreso Latino 131

International Institute of Minnesota 303

Project for Pride in Living, Inc. 22

Wesley Community Center, Inc. 141

Total 993

Site Number of Matched FOC Participants

FOC Sites

Asociación Puertorriqueños en Marcha 7

Cara 448

HumanKind 49

Prosperity Center for Financial Opportunity 51

Sacred Heart Community Service 250

Urban League of Essex County 170

Total 975

Exhibit B-7. Financial Well-Being Outcomes: Number of Participants Matched per Site

(includes Credit Score in the Match)

Site Number of Matched Participants

BCO Sites

Brighton Center, Inc. 26

Chinese Community Center, Inc. 42

Wesley Community Center, Inc. 76

District 1199C Training and Upgrading Fund 33

Instituto del Progreso Latino 29

International Institute of Minnesota 99

Project for Pride in Living, Inc. 2

Total BCO 307

Site Number of Matched Participants

FOC Sites

Asociación Puertorriqueños en Marcha 6

Cara 227

HumanKind 22

Prosperity Center for Financial Opportunity 20

Sacred Heart Community Service 72

Urban League of Essex County 10

Total FOC 357
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌36

Sample Loss and Attrition

Exhibit B-8 provides a detailed consort diagram that shows the number of eligible treatment and 

comparison participants, the number of participants included in the matched samples and the number of 

participants in the analytic sample for each outcome. The difference between the numbers in the matched 

sample and the number of participants included in the analytic sample for each outcome is the number of 

participants “lost to follow-up.” For example, for the Job Status Improvement outcome, the matched 

sample included n=993 T group members and n=975 C group members, while the analytic sample (which 

included only individuals with non-missing outcome data) included n=673 T group members and n=566 

C group members. In the T group 993-673 = 320 were lost because they had a missing outcome measure, 

while in the C group 975-566=409 were lost because they had a missing outcome measure. Because this 

evaluation uses all administrative data and only includes non-missing outcomes in the analysis, there is a 

different amount of missing data for each outcome, so the “lost to follow-up” varies by outcome.

The attrition rate for each outcome can also be calculated from Exhibit B-8. For example, for the Job 

Status Improvement outcome, the matched sample included n=993 T group members and n=975 C group 

members, while the analytic sample (which included only individuals with non-missing outcome data) 

included n=673 T group members and n=566 C group members. In the T group 993-673 = 320, (32%) 

were lost because they had a missing outcome measure. In the C group 975-566=409, (42%) were lost 

because they had a missing outcome measure. The overall attrition rate is 37% (320+409) / (993+975) 

and the differential attrition is 10% (42%-32%). Although this represents considerable sample loss due to 

missing outcome data, the results in Exhibit B-10 show that n=674 T and n=566 C group members that 

remained in the analytic sample were well matched on baseline characteristics. That is, the treatment and 

comparison group individuals in the analytic sample were practically identical to each other at baseline in 

terms of their gender, race/ethnicity, age, level of education, and employment status (as shown by the 

standardized effect size difference of zero between groups for each characteristic). Exhibits B-9 to B-22

show that baseline equivalence is established for each analytic sample, as is required for a quasiexperimental design (QED). Further, Exhibit B-11 and Exhibit B-20 for the job status improvement and 

credit score confirmatory outcomes, respectively, show that the treatment and comparison participants 

lost at follow-up because of missing outcome data are also equivalent. For example, for the Job Status 

Improvement outcome, the 320 treatment participants (993-673=320) and the 409 comparison participants 

(975-566) were practically identical to each other at baseline in terms of their gender, race/ethnicity, age, 

level of education, and employment status (as shown by the standardized effect size difference of zero 

between groups for each characteristic).
APPENDIX B: Additional Information about the Evaluation Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌37

Exhibit B-8. Flow from Eligible BOC and FOC Participants to Analytic Study Samples

Included in analytic samples:

Employment Outcomes

Job Status 

Improvement*

(Treatment 

n=673)

(Comparison 

n=566)

Job 

Advancement

(Treatment 

n=673)

(Comparison 

n=566)

Wage Increase

(Treatment 

n=407)

(Comparison 

n=288)

Value of Wage 

Increase

(Treatment 

n=407)

(Comparison 

n=288)

Wage at FollowUp

(Treatment 

n=523)

(Comparison 

n=310)

Hours Worked 

Increase

(Treatment 

n=367)

(Comparison 

n=269)

Value of Hours 

Worked 

Increase

(Treatment 

n=367)

(Comparison 

n=269)

Hours Worked 

at

Follow-up

(Treatment 

n=484)

(Comparison 

n=291)

Financial Outcomes

Maximum Credit Score at Follow-Up*

(Treatment n=181)

(Comparison n=79)

Latest Credit Score at Follow-Up

(Treatment n=177)

(Comparison n=79)

Financial Worth

(Treatment n=194)

(Comparison n=81)

*Outcome used in confirmatory analysis.
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌38

Method for Testing Baseline Equivalence

Abt estimated the standardized differences between the matched treatment and comparison groups at 

baseline to determine their level of similarity on all characteristics used in the matching. By construction, 

the treatment and comparison groups will be exactly matched on the CEM matching variables (highest 

level of education, employment status, gender, age, race/ethnicity and credit score range). 

To estimate the differences between BCO treatment and FOC comparison group participants at baseline, a 

modified version of the impact model (described below) that similarly accounts for the matching blocks 

design by including a set of indicators for the CEM matching blocks was fitted. The result is the average 

difference between treatment and comparison participants within each matching block. The model has the 

structure of Equation 1.

Eq (1) XXkkkk = αα0 + αα1TT + ∑ δδ0mmDDmm MM

mm=1 + εε ,

where:

XXkkkk, TT , and DDmmare as described in the impact model.

αα1 represents the difference between BCO and comparison group learners on the kth baseline 

characteristic. 

Separate models are fit to each of the k baseline characteristics, and the reported difference at baseline 

is the coefficient αα�1 from the model.

Standardized differences less than or equal to 0.05 standard deviations do not require a statistical 

adjustment to the analysis models; regression adjustments are made for differences that are between 0.05 

and 0.25 standard deviations; and baseline equivalence is not met for any standardized difference greater 

than 0.25 standard deviations. The Abt team verified that the matched groups were similar on the baseline 

characteristics used for matching; that is, all standardized differences less than or equal to 0.05 standard 

deviations. 

Exhibits B-9 to B-22 show tests for similarity between the treatment and comparison groups for the initial 

match and each analytic sample used in the evaluation. By construction, the treatment and comparison 

groups are exactly matched on the CEM matching variables. Further, Exhibits B-11 and B-20 show the 

similarity between the treatment and comparison groups’ members that were lost for the two confirmatory 

outcomes.
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌39

Exhibit B-9. Employment Outcomes Initial Match: Baseline Equivalence of Matched Sample

(excludes Credit Score)

Baseline Characteristic

BCO Participants

Mean

FOC Participants 

Mean

Standardized Effect 

Size Difference

Gender

Female 85.9 85.9 0.0

Male 14.1 14.1 0.0

Race/Ethnicity

Black/African American non-Hispanic 53.3 53.3 0.0

White non-Hispanic 12.0 12.0 0.0

Hispanic 25.1 25.1 0.0

Other (American Indian, Asian, Multiracial, 

Biracial, Other)

9.7 9.7 0.0

Age at Intake

17-24 27.2 27.2 0.0

25-34 33.7 33.7 0.0

35-44 19.3 19.3 0.0

45-54 13.5 13.5 0.0

55+ 6.2 6.2 0.0

Level of Education at Intake

No High School Diploma/Equivalency 6.7 6.7 0.0

High School Diploma or High School 

Equivalency

51.4 51.4 0.0

Any College 42.0 42.0 0.0

Employed at Intake

No 59.9 59.9 0.0

Yes 40.1 40.1 0.0

Total N 993 975
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌40

Confirmatory Employment Outcome

Exhibit B-10. Baseline Equivalence of Matched Sample for Job Status Improvement Outcome

Baseline Characteristic

BCO Participants

Mean

FOC Participants 

Mean

Standardized Effect 

Size Difference

Gender

Female 84.8 84.8 0.0

Male 15.2 15.2 0.0

Race/Ethnicity

Black/African American non-Hispanic 53.9 53.9 0.0

White non-Hispanic 10.9 10.9 0.0

Hispanic 27.3 27.3 0.0

Other (American Indian, Asian, Multiracial, 

Biracial, Other)

7.9 7.9 0.0

Age at Intake

17-24 27.8 27.8 0.0

25-34 30.9 30.9 0.0

35-44 20.8 20.8 0.0

45-54 14.1 14.1 0.0

55+ 6.4 6.4 0.0

Level of Education at Intake

No High School Diploma/Equivalency 5.2 5.2 0.0

High School Diploma or High School 

Equivalency

52.9 52.9 0.0

Any College 41.9 41.9 0.0

Employed at Intake

No 59.6 59.6 0.0

Yes 40.4 40.4 0.0

Total N 673 566
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌41

Exhibit B-11. Baseline Equivalence of Matched Sample Lost for Job Status Improvement Outcome

Baseline Characteristic

BCO Participants 

Excluded from 

Analysis

Mean

(n=320)

FOC Participants 

Excluded from 

Analysis

Mean

(n=409)

Standardized Effect 

Size Difference

Gender

Female 88.1 88.1 0.0

Male 11.9 11.9 0.0

Race/Ethnicity

Black/African American non-Hispanic 51.9 51.9 0.0

White non-Hispanic 14.4 14.4 0.0

Hispanic 20.3 20.3 0.0

Other (American Indian, Asian, Multiracial, 

Biracial, Other)

13.4 13.4 0.0

Age at Intake

17-24 25.9 25.9 0.0

25-34 39.7 39.7 0.0

35-44 16.3 16.3 0.0

45-54 12.2 12.2 0.0

55+ 5.9 5.9 0.0

Level of Education at Intake

No High School Diploma/Equivalency 9.7 9.7 0.0

High School Diploma or High School Equivalency 48.1 48.1 0.0

Any College 42.2 42.2 0.0

Employed at Intake

No 60.6 60.6 0.0

Yes 39.4 39.4 0.0
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌42

Exploratory Employment Outcomes

Exhibit B-12. Baseline Equivalence of Matched Sample for Hourly Wage Increase Outcome

Baseline Characteristic

BCO Participants

Mean

FOC Participants 

Mean

Standardized Effect 

Size Difference

Gender

Female 88.7 88.7 0.0

Male 11.3 11.3 0.0

Race/Ethnicity

Black/African-America non-Hispanic 55.5 55.5 0.0

White non-Hispanic 10.8 10.8 0.0

Hispanic 24.8 24.8 0.0

Other (American Indian, Asian, Multiracial, Biracial, 

Other)

8.9 8.9 0.0

Age at Intake

17-24 26.8 26.8 0.0

25-34 31.0 31.0 0.0

35-44 22.9 22.9 0.0

45-54 14.7 14.7 0.0

55+ 4.7 4.7 0.0

Level of Education at Intake

No High School Diploma/Equivalency 5.4 5.4 0.0

High School Diploma or High School Equivalency 52.6 52.6 0.0

Any College 42.0 42.0 0.0

Employed at Intake

No 75.9 75.9 0.0

Yes 24.1 24.1 0.0

Total N 407 288
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌43

Exhibit B-13. Baseline Equivalence of Matched Sample for Amount of Wage Increase Outcome

Baseline Characteristic

BCO Participants

Mean

FOC Participants 

Mean

Standardized Effect 

Size Difference

Gender

Female 88.7 88.7 0.0

Male 11.3 11.3 0.0

Race/Ethnicity

Black/African American non-Hispanic 55.5 55.5 0.0

White non-Hispanic 10.8 10.8 0.0

Hispanic 24.8 24.8 0.0

Other (American Indian, Asian, Multiracial, 

Biracial, Other)

8.9 8.9 0.0

Age at Intake

17-24 26.8 26.8 0.0

25-34 31.0 31.0 0.0

35-44 22.9 22.9 0.0

45-54 14.7 14.7 0.0

55+ 4.7 4.7 0.0

Level of Education at Intake

No High School Diploma/Equivalency 5.4 5.4 0.0

High School Diploma or High School 

Equivalency

52.6 52.6 0.0

Any College 42.0 42.0 0.0

Employed at Intake

No 75.9 75.9 0.0

Yes 24.1 24.1 0.0

Total N 407 288
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌44

Exhibit B-14. Baseline Equivalence of Matched Sample for Wage at Follow-Up Outcome

Baseline Characteristic

BCO Participants

Mean

FOC Participants 

Mean

Standardized Effect 

Size Difference

Gender

Female 85.9 85.9 0.0

Male 14.2 14.2 0.0

Race/Ethnicity

Black/African American non-Hispanic 57.4 57.4 0.0

White non-Hispanic 9.9 9.9 0.0

Hispanic 24.1 24.1 0.0

Other (American Indian, Asian, Multiracial, 

Biracial, Other)

8.6 8.6 0.0

Age at Intake

17-24 27.7 27.7 0.0

25-34 31.9 31.9 0.0

35-44 20.8 20.8 0.0

45-54 14.5 14.5 0.0

55+ 5.0 5.0 0.0

Level of Education at Intake

No High School Diploma/Equivalency 5.2 5.2 0.0

High School Diploma or High School 

Equivalency

50.7 50.7 0.0

Any College 44.2 44.2 0.0

Employed at Intake

No 59.1 59.1 0.0

Yes 40.9 40.9 0.0

Total N 523 310
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌45

Exhibit B-15. Baseline Equivalence of Matched Sample for Increase in Hours Worked Outcome

Baseline Characteristic

BCO Participants

Mean

FOC Participants 

Mean

Standardized Effect 

Size Difference

Gender

Female 88.0 88.0 0.0

Male 12.0 12.0 0.0

Race/Ethnicity

Black/African American non-Hispanic 53.7 53.7 0.0

White non-Hispanic 11.4 11.4 0.0

Hispanic 25.3 25.3 0.0

Other (American Indian, Asian, Multiracial, 

Biracial, Other)

9.5 9.5 0.0

Age at Intake

17-24 27.8 27.8 0.0

25-34 28.9 28.9 0.0

35-44 22.6 22.6 0.0

45-54 15.5 15.5 0.0

55+ 5.2 5.2 0.0

Level of Education at Intake

No High School Diploma/Equivalency 5.7 5.7 0.0

High School Diploma or High School 

Equivalency

50.7 50.7 0.0

Any College 43.6 43.6 0.0

Employed at Intake

No 83.9 83.9 0.0

Yes 16.1 16.1 0.0

Total N 367 269
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌46

Exhibit B-16. Baseline Equivalence of Matched Sample for Mean Hours Worked Increase Outcome

Baseline Characteristic

BCO Participants

Mean

FOC Participants 

Mean

Standardized Effect 

Size Difference

Gender

Female 88.0 88.0 0.0

Male 12.0 12.0 0.0

Race/Ethnicity

Black/African American non-Hispanic 53.7 53.7 0.0

White Hispanic 11.4 11.4 0.0

Hispanic 25.3 25.3 0.0

Other (American Indian, Asian, Multiracial, 

Biracial, Other)

9.5 9.5 0.0

Age at Intake

17-24 27.8 27.8 0.0

25-34 28.9 28.9 0.0

35-44 22.6 22.6 0.0

45-54 15.5 15.5 0.0

55+ 5.2 5.2 0.0

Level of Education at Intake

No High School Diploma/Equivalency 5.7 5.7 0.0

High School Diploma or High School Equivalency 50.7 50.7 0.0

Any College 43.6 43.6 0.0

Employed at Intake

No 83.9 83.9 0.0

Yes 16.1 16.1 0.0

Total N 367 269
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌47

Exhibit B-17. Baseline Equivalence of Matched Sample for Hours Worked at Follow-Up Outcome

Baseline Characteristic

BCO Participants

Mean

FOC Participants 

Mean

Standardized Effect 

Size Difference

Gender

Female 85.1 85.1 0.0

Male 14.9 14.9 0.0

Race/Ethnicity

Black/African American non-Hispanic 56.0 56.0 0.0

White non-Hispanic 10.3 10.3 0.0

Hispanic 24.6 24.6 0.0

Other (American Indian, Asian, Multiracial, 

Biracial, Other)

9.1 9.1 0.0

Age at Intake

17-24 28.5 28.5 0.0

25-34 30.4 30.4 0.0

35-44 20.7 20.7 0.0

45-54 15.1 15.1 0.0

55+ 5.4 5.4 0.0

Level of Education at Intake

No High School Diploma/Equivalency 5.4 5.4 0.0

High School Diploma or High School Equivalency 49.0 49.0 0.0

Any College 45.7 45.7 0.0

Employed at Intake

No 63.6 63.6 0.0

Yes 36.4 36.4 0.0

Total N 484 291
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌48

Exhibit B-18. Financial Outcomes Initial Match: Baseline Equivalence of Matched Sample (includes

Credit Score)

Baseline Characteristic

BCO Participants

Mean

FOC Participants 

Mean

Standardized Effect 

Size Difference

Gender

Female 89.3 89.3 0.0

Male 10.8 10.8 0.0

Race/Ethnicity

Black/African American non-Hispanic 67.4 67.4 0.0

White non-Hispanic 5.2 5.2 0.0

Hispanic 22.8 22.8 0.0

Other (American Indian, Asian, Multiracial, 

Biracial, Other)

4.6 4.6 0.0

Age at Intake

17-24 17.3 17.3 0.0

25-34 34.2 34.2 0.0

35-44 23.8 23.8 0.0

45-54 15.6 15.6 0.0

55+ 9.1 9.1 0.0

Level of Education at Intake

No High School Diploma/Equivalency 7.8 7.8 0.0

High School Diploma or High School Equivalency 52.4 52.4 0.0

Any College 39.7 39.7 0.0

Employed at Intake

No 75.2 75.2 0.0

Yes 24.8 24.8 0.0

Credit Score

A2 (FICO: 800-825) 0.0 0.0 0.0

B1 (FICO: 770-799) 1.3 1.3 0.0

B2 (FICO: 740-769) 1.0 1.0 0.0

C1 (FICO: 705-739) 3.3 3.3 0.0

C2 (FICO: 670-704) 2.3 2.3 0.0

D1 (FICO: 625-669) 10.1 10.1 0.0

D2 (FICO: 580-624) 10.8 10.8 0.0

E1 (FICO: 440-579) 37.1 37.1 0.0

XE1 (non-FICO: 451-600) 0.3 0.3 0.0

Z (Zero/Insufficient Credit History) 33.9 33.9 0.0

Total N 307 357
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌49

Confirmatory Financial Well-Being Outcome

Exhibit B-19. Baseline Equivalence of Matched Sample for Maximum Credit Score Outcome

Baseline Characteristic

BCO Participants

Mean

FOC Participants 

Mean

Standardized Effect 

Size Difference

Gender

Female 91.7 91.7 0.0

Male 8.3 8.3 0.0

Race/Ethnicity

Black/African American non-Hispanic 68.5 68.5 0.0

White non-Hispanic 5.5 5.5 0.0

Hispanic 23.8 23.8 0.0

Other (American Indian, Asian, Multiracial, Biracial, 

Other)

2.2 2.2 0.0

Age at Intake

17-24 9.4 9.4 0.0

25-34 35.9 35.9 0.0

35-44 28.7 28.7 0.0

45-54 16.6 16.6 0.0

55+ 9.4 9.4 0.0

Level of Education at Intake

No High School Diploma/Equivalency 2.8 2.8 0.0

High School Diploma or High School Equivalency 50.8 50.8 0.0

Any College 46.4 46.4 0.0

Employed at Intake

No 66.9 66.9 0.0

Yes 33.2 33.2 0.0

Credit Score

A2 (FICO: 800-825) 0.0 0.0 0.0

B1 (FICO: 770-799) 1.7 1.7 0.0

B2 (FICO: 740-769) 1.1 1.1 0.0

C1 (FICO: 705-739) 5.5 5.5 0.0

C2 (FICO: 670-704) 3.3 3.3 0.0

D1 (FICO: 625-669) 14.9 14.9 0.0

D2 (FICO: 580-624) 17.1 17.1 0.0

E1 (FICO: 440-579) 55.8 55.8 0.0

XE1 (non-FICO: 451-600) 0.6 0.6 0.0

Z (Zero/Insufficient Credit History) 0.0 0.0 0.0

Total N 181 79
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌50

Exhibit B-20. Baseline Equivalence of Matched Sample Lost for Maximum Credit Score

Baseline Characteristic

BCO Participants 

Excluded from 

Analysis

Mean

(n=126)

FOC Participants 

Excluded from 

Analysis

Mean

(n=278)

Standardized Effect 

Size Difference

Gender

Female 85.7 85.7 0.0

Male 14.3 14.3 0.0

Race/Ethnicity

Black/African American non-Hispanic 65.9 65.9 0.0

White non-Hispanic 4.8 4.8 0.0

Hispanic 21.4 21.4 0.0

Other (American Indian, Asian, Multiracial, 

Biracial, Other)

7.9 7.9 0.0

Age at Intake

17-24 28.6 28.6 0.0

25-34 31.8 31.8 0.0

35-44 16.7 16.7 0.0

45-54 14.3 14.3 0.0

55+ 8.7 8.7 0.0

Level of Education at Intake

No High School Diploma/Equivalency 15.1 15.1 0.0

High School Diploma or High School Equivalency 54.8 54.8 0.0

Any College 30.2 30.2 0.0

Employed at Intake

No 87.3 87.3 0.0

Yes 12.7 12.7 0.0

Credit Score

A2 (FICO: 800-825) 0.0 0.0 0.0

B1 (FICO: 770-799) 0.8 0.8 0.0

B2 (FICO: 740-769) 0.8 0.8 0.0

C1 (FICO: 705-739) 0.0 0.0 0.0

C2 (FICO: 670-704) 0.8 0.8 0.0

D1 (FICO: 625-669) 3.2 3.2 0.0

D2 (FICO: 580-624) 1.6 1.6 0.0

E1 (FICO: 440-579) 10.3 10.3 0.0

XE1 (non-FICO: 451-600) 0.0 0.0 0.0

Z (Zero/Insufficient Credit History) 82.5 82.5 0.0
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌51

Exploratory Financial Well-Being Outcomes

Exhibit B-21. Baseline Equivalence of Matched Sample for Latest Credit Score Outcome

Baseline Characteristic

BCO Participants

Mean

FOC Participants 

Mean

Standardized Effect 

Size Difference

Gender

Female 91.5 91.5 0.0

Male 8.5 8.5 0.0

Race/Ethnicity

Black/African American non-Hispanic 68.4 68.4 0.0

White non-Hispanic 5.7 5.7 0.0

Hispanic 23.7 23.7 0.0

Other (American Indian, Asian, Multiracial, 

Biracial, Other)

2.3 2.3 0.0

Age at Intake

17-24 9.6 9.6 0.0

25-34 36.2 36.2 0.0

35-44 28.8 28.8 0.0

45-54 16.4 16.4 0.0

55+ 9.0 9.0 0.0

Level of Education at Intake

No High School Diploma/Equivalency 2.8 2.8 0.0

High School Diploma or High School 

Equivalency

50.9 50.9 0.0

Any College 46.3 46.3 0.0

Employed at Intake

No 67.2 67.2 0.0

Yes 32.8 32.8 0.0

Credit Score

A2 (FICO: 800-825) 0.0 0.0 0.0

B1 (FICO: 770-799) 1.7 1.7 0.0

B2 (FICO: 740-769) 1.1 1.1 0.0

C1 (FICO: 705-739) 5.7 5.7 0.0

C2 (FICO: 670-704) 3.4 3.4 0.0

D1 (FICO: 625-669) 15.3 15.3 0.0

D2 (FICO: 580-624) 17.0 17.0 0.0

E1 (FICO: 440-579) 55.4 55.4 0.0

XE1 (non-FICO: 451-600) 0.6 0.6 0.0

Z (Zero/Insufficient Credit History) 0.0 0.0 0.0

Total N 177 79
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌52

Exhibit B-22. Baseline Equivalence of Matched Sample for Financial Net-Worth Outcome

Baseline Characteristic

BCO Participants

Mean

FOC Participants 

Mean

Standardized Effect 

Size Difference

Gender

Female 88.1 88.1 0.0

Male 11.9 11.9 0.0

Race/Ethnicity

Black/African American non-Hispanic 70.6 70.6 0.0

White non-Hispanic 6.2 6.2 0.0

Hispanic 20.1 20.1 0.0

Other (American Indian, Asian, Multiracial, 

Biracial, Other)

3.1 3.1 0.0

Age at Intake

17-24 13.4 13.4 0.0

25-34 35.1 35.1 0.0

35-44 25.8 25.8 0.0

45-54 15.0 15.0 0.0

55+ 10.8 10.8 0.0

Level of Education at Intake

No High School Diploma/Equivalency 5.2 5.2 0.0

High School Diploma or High School Equivalency 50.0 50.0 0.0

Any College 44.9 44.9 0.0

Employed at Intake

No 76.8 76.8 0.0

Yes 23.2 23.2 0.0

Credit Score

A2 (FICO: 800-825) 0.0 0.0 0.0

B1 (FICO: 770-799) 1.6 1.6 0.0

B2 (FICO: 740-769) 1.0 1.0 0.0

C1 (FICO: 705-739) 4.1 4.1 0.0

C2 (FICO: 670-704) 2.6 2.6 0.0

D1 (FICO: 625-669) 13.4 13.4 0.0

D2 (FICO: 580-624) 12.9 12.9 0.0

E1 (FICO: 440-579) 42.3 42.3 0.0

XE1 (non-FICO: 451-600) 0.0 0.0 0.0

Z (Zero/Insufficient Credit History) 22.2 22.2 0.0

Total N 194 81
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌53

Data Collection

Because evaluation participants were drawn from each site’s existing service population, all analyses 

were based on extant administrative data that the sites submit to LISC. The impact study data for BCO 

treatment group participants FOC-only comparison group participants were drawn from Salesforce, the 

commercial platform that LISC uses to store its client data. BCO and FOC site staff collect and store all 

client demographic and background information, service receipt, and outcome data in the system using 

the same procedures. Staff collect baseline data when clients enter a BCO or FOC-only program, prior to 

the delivery of an intervention; document the activities in which clients participate as services are 

provided (within 48 hours of data receipt); and collect follow-up data on clients’ employment, financial 

well-being, and education outcomes (when clients initiate contact with staff or staff contact clients). Abt 

identified the key variables in Salesforce to be used in the evaluation, and LISC staff transferred deidentified client data for those variables to Abt using HuddleTM, a secure web portal. 

Analytic Methods

Abt created measures about participants’ employment, financial well-being, and education, and estimated 

the effectiveness of BCO on the employment, and financial well-being measures for all participants. This 

section describes the approach we used to examine the effectiveness of the BCO. Described first is the 

rationale for selecting the evaluation’s measures that were used to examine the effects of BCO, and how 

those and other evaluation measures were constructed. Measures were constructed to capture (1) 

employment; (2) financial well-being; and (3) education. Discussed next are the evaluation’s analytic 

methods that were used to estimate effects of the BCO and FOC-only services on participants’ 

employment, and financial well-being measures. The final section provides the approach to examine the 

pre-post change for BCO participants’ education measure. 

Study Measures

The evaluation examined the effects of BCO on multiple measures of employment and financial wellbeing for treatment and comparison participants, as well as the pre-post change of an education measure 

for BCO treatment participants. The evaluation’s confirmatory employment measure was job 

advancement after program entry and is discussed in the main body of the report. The evaluation also 

examined two exploratory measures of employment—hourly wages and number of hours worked after 

program entry. The evaluation’s confirmatory measure of financial well-being was credit score increase

and is discussed in the main body of the report. The evaluation also examined two exploratory measures 

of financial well-being—overall debt and net worth. One exploratory measure of education—level of 

reading skills—is discussed in the main body of the report. Presented in Exhibit B-23 through Exhibit B27 are the data sources and construction of the measures for the employment, financial well-being, and 

education measures, and participant characteristics. Because the two confirmatory measures were

observed over a period of months, Exhibit B-24 shows the distribution of the number of months that 

elapsed between enrollment and measurement of the outcome.

Presented in Appendix C are the results of the exploratory analyses. 
APPENDIX B: Additional Information about the Evaluation Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌54

Exhibit B-23. Employment Outcome Measures Examined: Data Source and Measure Construction 

Measure Data Source Definition/Coding

Job Status Improvement BCO and FOC 1= (a) Enrolls in the program unemployed and then starts a new job in the 1-14 months after enrollment or (b) employed at 

enrollment and has one of the following records in the 1-14 months after enrollment: (1) gets a new job and the new wage is 

greater than the wage at enrollment; or (2) has an advancement record for an increase in wages or obtaining a promotion

0= (a) Enrolls in the program unemployed, has record of follow up with the program but no record of new employment in the 

1-14 months after enrollment or (b) employed at enrollment, has a record of follow-up, but has no record of advancement or 

record of a new job with a higher wage

Maximum Wage Increase at 

Follow-Up

BCO and FOC 1 = Maximum wage at follow-up in the 1-14 months after enrollment is greater than the wage at enrollment (wage defined as 

$0 for those unemployed at the time of enrollment)

0 = Maximum wage at follow-up in the 1-14 months after enrollment is less than or equal to the wage at enrollment (wage 

defined as $0 for those unemployed at the time of enrollment)

Value of Maximum Wage Increase 

at Follow-Up

BCO and FOC = The difference between the maximum wage at follow-up in the 1-14 months after enrollment and the wage at enrollment

Maximum Wage at Follow-Up BCO and FOC = The maximum wage at follow-up in the 1-14 months after enrollment

Latest Wage Increase at FollowUpBCO and FOC 1 = Latest wage at follow-up in the 1-14 months after enrollment is greater than the wage at enrollment (wage defined as $0 

for those unemployed at the time of enrollment)

0 = Latest wage at follow-up in the 1-14 months after enrollment is less than or equal to the wage at enrollment (wage defined 

as $0 for those unemployed at the time of enrollment)

Value of Latest Wage Increase at 

Follow-Up

BCO and FOC = The difference between the latest wage at follow-up in the 1-14 months after enrollment and the wage at enrollment

Latest Wage at Follow-Up BCO and FOC = The latest wage at follow-up in the 1-14 months after enrollment

Hours Worked Increase BCO and FOC 1 = Latest number of hours worked at follow-up in the 1-14 months after enrollment are greater than the number of hours 

worked at enrollment (hours worked defined as 0 hours for those unemployed at the time of enrollment)

0 = Latest number of hours worked at follow-up in the 1-14 months after enrollment are less than or equal to the number of 

hours worked at enrollment (hours worked defined as 0 hours for those unemployed at the time of enrollment)

Value of Hours Worked Increase BCO and FOC = The difference between the latest hours worked at follow-up in the 1-14 months after enrollment and the hours worked at 

enrollment

Hours Worked BCO and FOC = The latest hours worked at follow up in the 1-14 months after enrollment

Source: BCO and FOC administrative data in Salesforce.
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌55

Exhibit B-24. Number of Months Between Enrollment and Measurement of Confirmatory Outcomes

Number of Months Between Enrollment and 

Measurement of Outcome

BCO Participants

N (%)

FOC Participants

N (%)

Job Status Improvement (Confirmatory)

1 month 33 (4.9) 130 (23.0)

2 months 48 (7.1) 126 (22.3)

3 months 68 (10.1) 65 (11.5)

4 months 94 (14.0) 44 (7.8)

5 months 95 (14.1) 42 (7.4)

6 months 61 (9.1) 24 (4.2)

7 months 49 (7.3) 22 (3.9)

8 months 31 (4.6) 25 (4.4)

9 months 34 (5.1) 19 (3.4)

10 months 28 (4.2) 15 (2.7)

11 months 30 (4.5) 8 (1.4)

12 months 29 (4.3) 8 (1.4)

13 months 37 (5.5) 18 (3.2)

14 months 36 (5.4) 20 (3.5)

Credit Score (Maximum Credit Score at Follow-up) (Confirmatory)

1 month 0 (0.0) 0 (0.0)

2 months 0 (0.0) 4 (5.1)

3 months 3 (1.7) 6 (7.6)

4 months 1 (0.6) 1 (1.3)

5 months 3 (1.7) 2 (2.5)

6 months 13 (7.2) 4 (5.1)

7 months 14 (7.7) 4 (5.1)

8 months 15 (8.3) 5 (6.3)

9 months 10 (5.5) 2 (2.5)

10 months 9 (5.0) 5 (6.3)

11 months 7 (3.9) 2 (2.5)

12 months 5 (2.8) 5 (6.3)

13 months 2 (1.1) 5 (6.3)

14 months 6 (3.3) 9 (11.4)

15 months 9 (5.0) 2 (2.5)

16 months 12 (6.6) 1 (1.3)

17 months 9 (5.0) 3 (3.8)

18 months 5 (2.8) 4 (5.1)

19 months 3 (1.7) 4 (5.1)

20 months 7 (3.9) 5 (6.3)

21 months 6 (3.3) 1 (1.3)

22 months 5 (2.8) 1 (1.3)

23 months 8 (4.4) 0 (0.0)

24 months 11 (6.1) 1 (1.3)

25 months 2 (1.1) 1 (1.3)

26 months 4 (2.2) 1 (1.3)

27 months 1 (0.6) 0 (0.0)
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌56

Number of Months Between Enrollment and 

Measurement of Outcome

BCO Participants

N (%)

FOC Participants

N (%)

28 months 4 (2.2) 0 (0.0)

29 months 3 (1.7) 0 (0.0)

30 months 3 (1.7) 1 (1.3)

31 months 0 (0.0) 0 (0.0)

32 months 1 (0.6) 0 (0.0)

Note: Data include 673 participants in BCO programs and 566 participants in FOC programs for Job Status Improvement. Percentage of 

participants represents those who had a “job improvement” in 1-14 months after program enrollment.

Data include 181 participants in BCO programs and 79 participants in FOC-only programs. Percentage of participants represents those whose 

maximum credit score after program enrollment was higher than their credit score at intake.

Source: BCO and FOC administrative data in Salesforce: FFTTM Employment Record/Job Advancement Record.
APPENDIX B: Additional Information about the Evaluation Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌57

Exhibit B-25. Financial Well-Being Outcome Measures Examined: Data Source and Measure Construction 

Measure Data Source Definition/Coding

Credit Score Increase at 

Follow-Up

BCO and FOC 1 = Has a credit score at baseline and follow-up, and the credit score at follow-up is greater than the credit score at 

baseline

0 = Has a credit score at baseline and follow-up, and the credit score at follow-up is less than or equal to the credit score at 

baseline

Credit Score Increase at Least 

One Level at Follow-Up

BCO and FOC 1 = Has a credit score at baseline and follow-up, and the credit score level (A/B/C/D/E) at follow-up is greater than the 

credit score level at baseline

0 = Has a credit score at baseline and follow-up, and the credit score level (A/B/C/D/E) at follow-up is less than or equal to 

the credit score level at baseline

For all FICO scores – Transunion, Equifax, Experian: (A= 800-850, B= 740-799, C= 670-739, D= 580-669, E= 300-579)

For all non-FICO scores – Transunion, Vantage: (A=781-850, B=720-780, C=658-719, D=601-657, E=300-600)

Credit Score Reached a Score 

of 580 at Follow-Up

BCO and FOC 1 = Has a credit score at baseline and follow-up, and the credit score at follow-up is greater than or equal to 580 (580 is the 

minimum credit score to receive a loan)

0 Has a credit score at baseline and follow-up, and the credit score at follow-up is less than 580

Credit Score Reached a Score 

of 620 at Follow-Up

BCO and FOC 1 = Has a credit score at baseline and follow-up, and the credit score at follow-up is greater than or equal to 620 (620 is the 

minimum credit score for a conventional home loan approval)

0 Has a credit score at baseline and follow-up, and the credit score at follow-up is less than 620

Credit Score at Follow Up BCO and FOC = Credit score at follow-up, for individuals with a credit score at baseline and a credit score at follow-up

Debt Decrease at Follow-Up BCO and FOC 1 = Has a measure of total debt at baseline and follow-up, and total debt at follow-up is less than total debt at baseline

0 = Has a measure of total debt at baseline and follow-up, and total debt at follow-up is greater than or equal to total debt at 

baseline

*Calculations for debt exclude student loans, mortgages, and transportation loans.

Net Worth Increase at FollowUpBCO and FOC 1 = Has a measure of total net worth at baseline and follow-up, and total net worth at follow-up is greater than total net 

worth at baseline

0 = Has a measure of total net worth at baseline and follow-up, and total net worth at follow-up is less than or equal to total 

net worth at baseline

*Calculations for net worth exclude student loans, mortgages, and transportation loans.

Change in Debt at Follow-Up BCO and FOC = Difference between individual’s total debt at follow-up and total debt at baseline, for individuals with measures of total 

debt at baseline and at follow up

*Calculations for debt exclude student loans, mortgages, and transportation loans.

Change in Net Worth at FollowUpBCO and FOC = Difference between individual’s total net worth at follow-up and total net worth at baseline, for individuals with measures 

of total net worth at baseline and at follow-up

*Calculations for net worth exclude student loans, mortgages, and transportation loans.

Source: BCO and FOC administrative data in Salesforce.
APPENDIX B: Additional Information about the Evaluation Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌58

Exhibit B-26. Education Outcome Measure Examined: Data Source and Measure Construction 

Measure Data Source Definition/Coding

Test Score Gain from Pre- to Post-test BCO = The difference between the final post-test TABE® Reading scale score and pre-test TABE® Reading scale score

Exhibit B-27. Participant Characteristics at Baseline: Data Source and Measure Construction 

Measure Data Source Definition/Coding

Gender BCO and FOC 1=Female

0=Male

Race/Ethnicity BCO and FOC 1=Black/African American non-Hispanic

2=White non-Hispanic

3=Hispanic

4=Other (American Indian, Asian, Multiracial. Biracial, Other)

Age at Intake BCO and FOC 1=17-24 years old

2=25-34 years old

3=35-44 years old

4=45-54 years old

5=55+ years old

Marital Status at Intake BCO and FOC 1=Divorced

2=Married

3=Separated

4=Single

5=Widowed

Primary Language Spoken BCO and FOC 1=English

0=Not English

Level of Education at Intake BCO and FOC 1=No High School Diploma/Equivalency

2=High School Diploma or High School Equivalency

3=Some College

4=College Certificate or Degree

Employed at Intake BCO and FOC 1=Employed

0=Not Employed

FICO Credit Score Range at Intake BCO and FOC 1=Exceptional (800+)

2=Very Good (740-799)

3=Good (670-739)

4=Fair (580-669)

5=Poor (<580)

6=Insufficient Credit Score History

7=Non-FICO Score Available
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌59

Estimating the Effects of BCO

To account for the matching block design, Abt estimated the impact of BCO using regression models that 

include terms for the matching blocks. The inclusion of terms for the matching blocks ensures that the 

overall estimate of the treatment-comparison group outcome difference is calculated by comparing the 

BCO and FOC participant outcomes within the matched blocks and calculating a precision-weighted 

average of the within-block differences to produce the overall estimate. To control for other baseline 

differences between the groups that might be related to outcomes, the model also includes covariates for 

participants’ baseline level of education, employment, credit score range, gender, age, and race/ethnicity. 

For each outcome, we estimated a model that reflects this matched design and has the structure of 

Equation 2.

Eq (2) YY = ββ0 + ββ1TT + ∑ λλkkXXkkkk

KK

kk=1 + ∑ γγ0mmDDmm MM

mm=1 + εε

where:

Yi is the outcome of interest (e.g., receipt of educational credential) for the i

th individual in the mth

matching block.

Ti is an indicator variable equal to 1 if individual i was an BCO participant and 0 if a FOC participant.

XXkkkk is the kth baseline covariate; these include baseline level of education, employment, credit score 

range, gender, age, and race/ethnicity. 

DDmm is the indicator variable representing the mth matching block. 

εε is the usual random error term.

In this model, ββ1 represents the impact of BCO on the study outcomes, where the impact estimate is 

averaged over the matching blocks. The p-values reported for impact estimates are two-tailed to account 

for the possibility that the intervention might adversely affect one or more of the outcomes. The 

coefficients on the covariates (λλkk) reflect the relationship between the outcome measure and each of the 

covariates while controlling for others.

Pre-Post Reading Gain Analysis

Analysis of Changes in BCO Participants’ Reading Skills 

For each reading test, Abt converted BCO participants’ raw scores into two measures: (1) grade 

equivalent scores that are useful for descriptive purposes; and (2) scale scores that are appropriate for 

more complex statistical analyses. Change scores that represent the amount of a BCO participant’s skill 

change from pre- to post-tests were constructed by subtracting each participant’s pre-test scale score from 

their post-test scale scores. The statistical significance of the mean scale score gain was assessed using the 

paired t-test.

Hypothesis Testing of Change in Scores

Abt tested the null hypothesis that participants’ change scores are zero by conducting a paired t-test. The 

null hypothesis for the paired t-test is that the average population gain is zero scale score points. The twotail alternative hypothesis is that the average gain is not equal to zero. The use of the paired t-test is 

predicated on the assumption that, absent BCO, the reading ability of the learners would not be expected 

to measurably improve over the span of time from pre-test to post-test. If the average change is positive 
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌60

and the null hypothesis is rejected, we interpret this as an indication that program participation is 

associated with the shift in the distribution of reading skills among the population of participants.

Effect Sizes

Abt examined the effect size of the pre-post changes in BCO participants’ reading test scores. The effect 

size is a measure of the mean gain expressed in standard deviation units. An advantage of expressing 

mean gain as an effect size is that the mean gains for each of the various standardized reading tests can be 

directly compared to one another. This is possible because, when expressed as an effect size, they are 

each expressed in a common metric. 

Additionally, there is a commonly used rule-of-thumb for evaluating the magnitude of effect sizes 

(Cohen, 1988). For effect sizes calculated as differences in means, effect sizes of .20 or lower are 

generally considered to be "small"; a "medium" effect size is approximately .50; and a “large” effect size 

is .80 or greater. In interpreting skill change data, we used as conservative standard by focusing on skill 

gains that were statistically significant (by paired t-tests at p<.05 level) and that produced effect sizes of 

.20 or higher.

Effect sizes were calculated using the formula shown below:

Where:

is the standard score mean at post-test,

is the standard score mean at pre-test,

is the pooled standard deviation of the pre- and post-test means.

Because the sample sizes are equal for the pre- and post-tests, the pooled standard deviation was

calculated as

where and are the standard deviations for the pre- and post-test data.

Additionally, effect sizes were calculated for pre-follow-up gains and post-follow-up gains. To ensure 

that these effect sizes were directly comparable to the pre-post effect sizes, Abt used the same value for 

the pooled standard deviation in the denominator for the calculations of the pre-follow-up and postfollow-up effect sizes, as was used for the pre-post gains.

pooled

post pre

post pre s

X X

Effect Size −

− =

X post

X pre

pooled s

2

2 2

pre post

pooled

s s

s

+ =

pre s

post

s
APPENDIX B: Additional Information about the Evaluation 

Design and Impact Study

Abt Associates BCO Evaluation Final Report April 2021 ▌61

The formulas are shown below.

,

where Spooled is the same as shown above.

Treatment of Missing Data

Abt did not impute any baseline or outcome data. By design, only participants with non-missing baseline 

data were included in the matched sample. All analyses included participants with non-missing outcomes 

(complete case analysis).

Multiple Outcome Measures 

To address multiple comparisons, Abt defined confirmatory and exploratory analysis components prior to 

data analysis (Schochet, 2008). Because there is a single confirmatory impact estimate for the 

evaluation’s outcome domains of employment and financial well-being, no multiple comparison 

corrections are required to reduce the chance of a Type-I error. Because the purpose of exploratory 

analyses is to examine relationships within the data—for example, to identify hypotheses for more 

rigorous future evaluation, Abt did not conduct multiplicity adjustments. Instead, we included explicit 

statements in Appendix C that these exploratory analyses do not provide rigorous evidence on the 

intervention’s overall effectiveness. 

pooled

followup post

followup post

pooled

followup pre

followup pre s

X X

Effect Size

s

X X

Effect Size − = −

− = − ,
APPENDIX C: Supplemental Tables and Information 

on Study Findings

Abt Associates BCO Evaluation Final Report April 2021 ▌62

Appendix C: Supplemental Tables and Information on Study 

Findings

This appendix provides the results of the exploratory analyses concerning employment and financial wellbeing that Abt conducted in the BCO evaluation’s impact study. Note that these exploratory analyses do 

not provide rigorous evidence on the intervention’s overall effectiveness.

The appendix also includes BCO site-level data for the TABE® pre-post test results for each BCO site, 

which was one of the treatment group’s descriptive education outcomes measured used in the study.

Employment

The impact study’s exploratory research questions related to employment and results from the analyses to 

address those questions are described below.

Exploratory Research Questions for Employment

1. Were BCO participants more likely to increase their hourly wages after program entry relative to a 

comparison group who received only FOC services?

2. Were BCO participants more likely to be placed in jobs with higher hourly wages relative to a 

comparison group who received only FOC services?

3. Were BCO participants more likely to work more hours per week after program entry relative to a 

comparison group who received only FOC services?

Impacts on Employment 

Presented in Exhibit C-1 are the findings from the employment impact analyses. The far-left column in 

the table lists the employment outcomes that are reported. The second column contains the outcomes for 

the treatment group and the third column contains the outcomes for the comparison group. The fourth 

column lists the impact results. The fifth column lists the standard error and the sixth column provides the 

p-Value for each impact. The first row in the exhibit has data for the confirmatory research question, 

which is discussed in the main body of the report. The data in the other rows in the table pertain to the 

exploratory research questions. Because the confirmatory employment measure was observed over a 

period of months, the study also conducted an exploratory analysis that included the number of months 

that elapsed between enrollment and measurement of the confirmatory outcome as a covariate in the 

impact model to investigate the sensitivity of the results to the timing of the measure. Exhibit C-2 shows 

that the finding remains the same.

Wage Increase

Abt analyzed two types of wage increases to address exploratory Research Questions 1 and 2 concerning 

hourly wages: (1) the maximum hourly wage increase for participants in months 1-14 after enrollment, 

and (2) the latest hourly wage increase for participants in months 1-14 after enrollment. The analyses of 

participants’ maximum hourly wage data indicated that a slightly higher percentage of BCO treatment 

group participants (87.5%) increased their hourly wages compared to FOC comparison group participants 

(85.2%), but this difference is not statistically significant. A similar pattern was found in the analyses of 

participants’ latest hourly wage after enrollment. 

Abt analyzed participants’ mean hourly wage increase in months 1-14 after enrollment. Exhibit C-1 

shows that FOC comparison group participants’ maximum mean hourly wage increase ($11.80) was 

significantly higher than the maximum mean hourly wage increase for BCO treatment group participants 
APPENDIX C: Supplemental Tables and Information 

on Study Findings

Abt Associates BCO Evaluation Final Report April 2021 ▌63

($10.38), with an impact of -$1.42 for the treatment group. Analyses of the latest mean hourly wage 

increase indicated the same pattern of results.

We also analyzed participants’ mean hourly wage in months 1-14 after enrollment. Exhibit C-1 shows 

that FOC comparison group participants’ maximum mean hourly wage ($15.00) was higher than the

maximum mean hourly wage for BCO treatment group participants ($13.23), with an impact of -$1.77 for 

the treatment group. Analyses of the latest mean hourly wage showed the same pattern of results.

Hours Worked

Research Question 3 explored whether participants increased the number of hours they worked in a week 

in months 1-14 after enrollment. Exhibit C-1 shows that BCO treatment group participants (92.4%) were 

slightly more likely than comparison group participants (91.3%) to increase the number of hours that they 

worked in a week, but this increase is not statistically significant. We also examined the number of hours 

that participants worked during months 1-14 after enrollment and found that comparison group 

participants increased their number of hours worked in a week by 29.92 hours compared with treatment 

group participants, who increased the number of hours worked in a week by 28.72. However, this increase 

is not statistically significant.

The final analysis concerning hours worked examined the number of hours per week that participants 

worked in months 1-14 after enrollment. Exhibit C-1 shows that FOC comparison group participants 

worked a mean of 34.29 hours per work compared to BCO treatment group participants who worked a 

mean of 33.20 hours per week. This difference is not statistically significant. 

Exhibit C-1. Impacts on Confirmatory and Exploratory Employment Outcomes

BCO 

Participants

FOC 

Participants Impact

Standard 

Error p-Value a

Job Status (Confirmatory)

Job Status Improvement 55.6% 36.8% 18.7% 3.0 .000***

Wage (Maximum Wage at Follow-up) (Exploratory)

Wage Increase 87.5% 85.2% 2.3% 2.0 .251

Mean Hourly Wage Increase 10.38 11.80 -1.42 0.41 .001**

Mean Hourly Wage at Follow-up 13.23 15.00 -1.77 0.39 .000***

Wage (Latest Wage at Follow-up) (Exploratory)

Wage Increase 86.5% 85.3% 1.2% 2.0 .537

Mean Hourly Wage Increase 10.02 11.85 -1.83 0.39 .000***

Mean Hourly Wage at Follow-up 12.89 14.83 -1.93 0.36 .000***

Hours Worked (Latest Hours Worked at Follow-up) (Exploratory)

Hours Worked Increase 92.4% 91.3% 1.1% 1.7 .523

Mean Hours Worked Increase 28.72 29.92 -1.19 0.99 .230

Hours Worked per Week at Follow-up 33.20 34.29 -1.09 0.83 .187

a p-Values shown in this column are for tests of whether there was a statistically significant impact for the outcome in the row.

Notes: 

* Difference is statistically significant at the .05 level.

** Difference is statistically significant at the .01 level.

*** Difference is statistically significant at the .001 level.

Sample Sizes:

Job Status Improvement: BCO Participants=673 clients, FOC Participants=566 clients.

Wage Increase: BCO Participants=407 clients, FOC Participants=288 clients.

Mean Hourly Wage Increase: BCO Participants=407 clients, FOC Participants=288 clients.

Mean Hourly Wage at Follow-up: BCO Participants=523 clients, FOC Participants=310 clients.
APPENDIX C: Supplemental Tables and Information 

on Study Findings

Abt Associates BCO Evaluation Final Report April 2021 ▌64

Hours Worked Increase: BCO Participants=367 clients, FOC Participants=269 clients.

Mean Hours Worked Increase: BCO Participants=367 clients, FOC Participants=269 clients.

Hours Worked per Week at Follow-up: BCO Participants=484 clients, FOC Participants=291 clients.

Source: BCO and FOC administrative records in Salesforce FFTTM Employment /Employment Advancement Record, FFTTM Employment 

Counseling Record.

Exhibit C-2. Sensitivity Analysis: Impact on Confirmatory Employment Outcome Controlling for 

Number of Months Between Enrollment and Measure of Outcome

BCO 

Participants

FOC 

Participants

Impact Standard 

Error

p-Value a

Job Status (Confirmatory)

Job Status Improvement 55.6% 40.7% 14.8% 3.0 .000***

Note: ***Difference is statistically significant at the .001 level. Data include 673 participants in BCO programs and 566 participants in FOC 

programs. Percentage of participants represents those who had a “job improvement” in 1-14 months after program enrollment.

Source: BCO and FOC administrative data in Salesforce: FFTTM Employment Record/Job Advancement Record.

Interpretation of Results

The overall results concerning BCO treatment and FOC comparison groups’ wage increases show that 

most participants in both groups increased their wages from the time of their enrollment in the BCO or 

FOC program to follow-up, and both groups increased the amount of their mean hourly wages. Both 

groups also increased their mean hourly wage at follow-up. The fact that both BCO treatment and FOC 

comparison group participants received employment services may account for the large percentage of 

both groups who increased their overall wages and mean hourly wages. The value of participating in 

occupational training and earning occupational credentials did not have an effect during this short-term 

follow-up period. 

One factor that might explain why the treatment group’s mean hourly wage at follow-up is lower than the 

FOC comparison group’s is that over half (55%) of the BCO treatment group obtained jobs in the 

supportive healthcare sector, where the hourly rate of pay is often at minimum wage. In contrast, over half 

(59%) of FOC comparison group participants obtained jobs in building and grounds, food preparation, 

and office and administrative support sectors where the hourly rate of pay might have been higher than 

jobs in healthcare.

Financial Well-Being 

The impact study’s exploratory research questions related to financial well-being and results from the 

analyses to address those questions are described below.

Exploratory Research Questions for Financial Well-Being

1. Are BCO participants more likely to have lower overall debt after program entry relative to a 

comparison group who received only FOC services?

2. Are BCO participants more likely to have increased their net worth after program entry relative to a 

comparison group who received only FOC services?

Impacts on Financial Well-Being

Presented in Exhibit C-3 are the findings from the financial well-being impact analyses. The far-left

column in the table lists the financial well-being outcomes that are reported. The second column contains 

the outcomes for the treatment group and the third column contains the outcomes for the comparison 

group. The fourth column lists the impact results. The fifth column lists the standard error, and the six 

column provides the p-Value for each impact. Because the confirmatory financial well-being measure was 
APPENDIX C: Supplemental Tables and Information 

on Study Findings

Abt Associates BCO Evaluation Final Report April 2021 ▌65

observed over a period of months, the study also conducted an exploratory analysis that include the 

number of months that elapsed between enrollment and measurement of the confirmatory outcome as a 

covariate in the impact model to investigate the sensitivity of the results to the timing of the measure. 

Exhibit C-4 shows that the finding remains the same.

Total Debt

Abt examined participants who had a measure of total debt at baseline and follow-up and analyzed 

whether their total debt was lower at follow up than at baseline. For the debt analysis, the calculations 

excluded student loans, mortgages, and transportation loans, which has been LISC’s practice in previous 

studies.

Exhibit C-3 shows that comparison group participants (57%) were more likely to decrease their debt than 

treatment group participants (29%), with an impact of -28% for the treatment group. Comparison group 

participants also decreased their debt by $3,057.70 while treatment group participants increased their debt 

by $1,785.17.

Net Worth

Abt also examined participants who had a measure of net worth at baseline and follow-up and analyzed 

whether their total net worth was higher at follow up than at baseline. Exhibit C- 3 shows while both 

treatment and comparison group participants had increased their net worth, a larger percentage of 

comparison group participants (65%) increased their net worth compared with treatment group 

participants (41%). Comparison group participants’ mean change in net worth was $9,954.86 compared 

with treatment group participants’ mean change in net worth, which was $2,636.26.

Exhibit C-3. Impacts on Confirmatory and Exploratory Financial Well-Being Outcomes

BCO 

Participants

FOC 

Participants Impact

Standard 

Error p-Value a

Credit Score (Maximum Credit Score at Follow-up) (Confirmatory)

Credit Score Increase 68.0% 79.8% −11.8% 7.1 .100

Credit Score Level Increase 28.7% 36.3% −7.5% 7.4 .310

Credit Score Reached 580 59.7% 67.7% −8.0% 7.3 .272

Credit Score Reached 620 35.4% 42.7% −7.4% 5.6 .192

Credit Score Value at Follow-Up 604.1 607.9 -3.8 8.1 .637

Credit Score (Latest Credit Score at Follow-Up) (Confirmatory)

Credit Score Increase 54.8% 76.7% −21.9% 7.6 .005**

Credit Score Level Increase 23.7% 32.8% -9.1% 7.1 .202

Credit Score Reached 580 49.7% 63.1% −13.4% 7.5 .076

Credit Score Reached 620 31.1% 36.3% −5.3% 5.4 .333

Credit Score Value at Follow-Up 591.5 605.7 -14.2 9.0 .117

Financial Worth b (Exploratory)

Debt Decrease 29.0% 57.0% −28.0% 7.5 .000***

Net Worth Increase 41.0% 65.0% −24.0% 7.9 .002**

Change in Debt 1,785.17 -3,057.70 4,842.86 1,603.53 .003**

Change in Net Worth 2,636.26 9,954.86 -7,318.60 4,640.87 .117

a p-Values shown in this column are for tests of whether there was a statistically significant impact for the outcome in the row. b Calculations for debt and net worth exclude student loans, mortgages, and transportation loans.

Notes: 

* Difference is statistically significant at the .05 level.

** Difference is statistically significant at the .01 level.
APPENDIX C: Supplemental Tables and Information 

on Study Findings

Abt Associates BCO Evaluation Final Report April 2021 ▌66

*** Difference is statistically significant at the .001 level.

Sample Sizes:

Credit Score (Maximum Credit Score at Follow-Up): BCO Participants=181 clients, FOC Participants=79 clients.

Credit Score (Latest Credit Score at Follow-Up): BCO Participants=177 clients, FOC Participants=79 clients.

Financial Worth: BCO Participants=194 individuals, FOC Participants=81clientss.

Source: BCO and FOC administrative data in Salesforce: FFTTM Credit Report Record, FFTTM Balance Sheet Record.

Exhibit C-4. Sensitivity Analysis: Impact on Confirmatory Financial Well-Being Outcome 

Controlling for Number of Months Between Enrollment and Measurement of Outcome

BCO 

Participants

FOC 

Participants Impact

Standard 

Error p-Value a

Credit Score (Confirmatory)

Credit Score (Maximum Credit Score 

at Follow-Up) Increase 68.0% 84.5% -16.5% 6.9 .018*

Note: Data include 181 participants in BCO programs and 79 participants in FOC-only programs. Percentage of participants represents those 

whose maximum credit score after program enrollment was higher than their credit score at intake.

Source: BCO and FOC administrative data in Salesforce: FFTTM Credit Score Record. 

Interpretation of Results

The FOC comparison group participants’ increases in net worth and decreases in debt might be explained 

by the length of time since they have been in the workforce since enrolling in the FOC compared with 

BCO treatment group participants. The comparison group participants’ length of time in the workforce 

along with their possibly higher hourly wage rates might have enabled them to reduce their debt in the 

near term compared to treatment group participants.

Education 

Abt’s evaluation plan included an exploratory research question for the treatment group concerning 

whether BCO participants were likely to increase their reading skills (as measured by a standardized test) 

after participating in the BCO program. To address this question, Abt conducted a pre-post analysis of all 

treatment group participants’ performance on the Test for Adult Basic Education (TABE®) Reading Test, 

not just individuals included in the matched samples. Exhibit 12 in the main body of the report shows the 

results from the pre-post analysis of the TABE® Reading scores for the whole treatment group. Exhibit C5 shows the results of the pre-post analysis of TABE® Reading test scores by BCO treatment site. The 

sites varied in the amount of complete pre-post data that were entered into the Salesforce database, with 

two sites submitting minimal data. Four of the sites had statistically significant gains in reading test scores 

with small effects.
APPENDIX C: Supplemental Tables and Information 

on Study Findings

Abt Associates BCO Evaluation Final Report April 2021 ▌67

Exhibit C-5. TABE® Reading Pre-Post Test Results by BCO Site

All BCO 

Sites Site A Site B

Site 

C

Site 

D Site E Site F Site G

Number of Learners (n=282) (n=4) (n=36) (n=6) (n=1) (n=155) (n=12) (n=68)

Average Test Score Gain from Preto Final Post Test

9.05 19.00 -12.81 -4.67 13.00 8.98 8.75 21.40

Standard Deviation of Pre-Test 65.99 19.50 54.79 50.42 - 48.75 31.33 60.03

Standardized Test Score Gain 0.14 0.97 -0.23 -0.09 - 0.18 0.28 0.36

p-Value of Gain from Paired T-Test 0.0002*** 0.004** 0.078 0.460 - 0.001** 0.589 0.0004***

Notes: 

* Difference is statistically significant at the .05 level.

** Difference is statistically significant at the .01 level.

*** Difference is statistically significant at the .001 level.

Source: BCO administrative data in Salesforce: FFTTM TABE® Records.

Exhibit C-6 shows the distribution of BCO sites’ comparison group participants’ reading skills by grade 

level-equivalence at baseline. A slightly higher percentage of participants scored below the 6th grade 

equivalent at baseline. This finding is likely due to the large number of participants at Site D and Site E, 

whose native language is not English. The treatment group participants are distributed across the four 

grade-equivalent levels, which reflects the BCO treatment sites’ skill criteria for participating in the 

occupational training that they offered. Sites generally enrolled participants who tested at the 6th grade 

equivalent or higher at program intake.

Exhibit C-6. TABE® Reading Pre-Test Results for BCO Participants at Baseline by Site

All BCO 

Sites 

(n=1,133)

Site A

(n=217)

Site B

(n=117)

Site C 

(n=130)

Site D

(n=146)

Site E

(n=354)

Site F

(n=24)

Site G

(n=145)

Reading (n=594) (n=83) (n=110) (n=73) (n=47) (n=170) (n=19) (n=92)

Grade Equivalent 

<5.9 (%)

29.6 6.0 15.5 19.2 8.5 72.4 0.0 14.1

Grade Equivalent 

6.0-8.9 (%)

24.9 20.5 31.8 32.9 19.2 18.2 26.3 29.4

Grade Equivalent 

9.0-10.9 (%)

19.9 27.7 25.5 26.0 19.2 7.7 26.3 22.8

Grade Equivalent 

11-12.9 (%)

25.6 45.8 27.3 21.9 53.2 1.8 47.4 33.7

Source: BCO administrative data in Salesforce: FFTTM TABE® Record.
REFERENCES

Abt Associates BCO Evaluation Final Report April 2021 ▌68

References

Andersson, F., Holzer, H. J., & Lane, J. I. (2005). Moving up or moving on: Who gets ahead in the lowwage labor market? Russell Sage Foundation.

Berta, P. & Bossi, M. & Verzillo, S. (2017). % CEM: A SAS Macro to perform Coarsened Exact 

Matching, Journal of Statistical Computation and Simulation, 87 (2), 227-238.

Bond, G., Williams, J., Evans, L., Salyers, M., Kim, H., Sharpe, H., & Leff, H. S. (2000). Psychiatric

rehabilitation fidelity toolkit. Human Services Research Institute.

Century, J., Rudnick, M., & Freeman, C. A (2010). Framework for measuring fidelity of implementation: 

A foundation for shared language and accumulation of knowledge. The American Journal of 

Evaluation, 31(2), 198-218.

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. Routledge Academic.

CTB/McGraw-Hill (2004). Test of Adult Basic Education: Norms Book. Author. 

Fontenot, K., Senega, J. & Kollar, M. (September 2018). Income and poverty in the United States: 

2017.Current Population Reports, P60-263. U.S. Census Bureau. 

Gabe, T., Abel, J.R., & Florida, R. (April 2018). Can low-wage workers find better jobs? Federal Reserve 

Bank of New York Staff Reports, no. 846. Federal Reserve Bank.

Farrell, M. & Martinson, K. (2017). The San Diego County bridge to employment in the healthcare 

industry program: Implementation and early impact report, OPRE Report # 2017-41. Office of 

Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of 

Health and Human Services.

Gan, K., Alamprese, J., Litwok, D. & Price, C. (December 2016). SIF evaluation plan: LISC Bridges to

Career Opportunities. Abt Associates. 

Gardiner, K/. Rolston, H. D., Fein, D., & Cho, S-W. (2017). Pima Community College pathways to 

healthcare program: Implementation and early impact report, OPRE Report No. 2017-10. Office of 

Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of 

Health and Human Services.

Iacus, S. M., King, G., & Porro, G. (2012). Causal inference without balance checking: Coarsened exact 

matching. Political Analysis, 20 (1), 1-24. Website Copy at http://j.mp/2nRpUHQ

Loprest, P., Acs, G., Ratcliffe, C., & Vinopal, K. (2009). Who are low-wage workers? ASPE Research 

Brief. U.S. Department of Health and Human Services. 

Mowbray, C. T., Holter, M. T., Teague, G. B., & Bybee, D. (2003). Fidelity criteria: Development, 

measurement, and validation. American Journal of Evaluation, 24 (3), 315-340.

Rankin, S (2015). Bridge and vocational services within Financial Opportunity Centers: Evidence from 

program data. Local Initiatives Support Corporation.
REFERENCES

Abt Associates BCO Evaluation Final Report April 2021 ▌69

Roder, A. (2016). First steps on the road to financial well-being: Final report from the evaluation of

LISC’s Financial Opportunity Centers. Prepared for the Local Initiatives Support Corporation. 

Economic Mobility Corporation.

Schochet, P. Z. (2008). Technical methods report: Guidelines for multiple testing in impact evaluations

(NCEE 2008-4018). National Center for Education Evaluation and Regional Assistance, Institute of 

Education Sciences, U.S. Department of Education.