POSITIVE 

DISRUPTION

The Promise of the Opportunity Reboot Model

Amy K. Syvertsen

Justin Roskopf

Chen-Yu Wu

Ashley Boat

Jenna Sethi

Rachel Chamberlain
Suggested Citation

Syvertsen, A. K., Roskopf, J., Wu, C-Y., Boat, A., Sethi, J., & Chamberlain, R. (2020). Positive 

Disruption: The Promise of the Opportunity Reboot Model (Report to the Corporation for National 

and Community Service Social Innovation Fund). Minneapolis, MN: Search Institute.

Corporation for National and Community Service Social Innovation Fund 

The Social Innovation Fund (SIF), a program of the Corporation for National and Community 

Service (CNCS), was founded to support the growth of effective programs to have greater impact 

and to develop innovative approaches to address the most challenging social problems. In 

collaboration with private-sector partners, SIF helped invest over $1 billion in finding what works 

and making it work for more people. 

SIF was a program that received funding from 2010 to 2016 from the Corporation for National and 

Community Service, a federal agency that engages millions of Americans in service through its 

AmeriCorps, Senior Corps, and Volunteer Generation Fund programs, and leads the nation’s 

volunteer and service efforts. Using public and private resources to find and grow communitybased nonprofits with evidence of results, SIF intermediaries received funding to award subgrants 

that focus on overcoming challenges in economic opportunity, healthy futures, and youth 

development. Although CNCS made its last SIF intermediary awards in fiscal year 2016, SIF 

intermediaries will continue to administer their subgrant programs until their federal funding is 

exhausted.

Funding

Financial support from the Social Innovation Fund was matched with generous philanthropic 

commitments from these funding partners. The collective contributions of these organizations 

made this project and the evaluation possible.

Amherst H. Wilder Foundation

Beacon Interfaith Housing Collaborative

Bernicks Foundation

Bush Foundation

Canadian National Railway

City of Mpls/STEP-UP Discover

DEED Equity Funds

Duluth-Superior Area Community Foundation

Essentia Health Corporate Contributions

F.R. Bigelow Foundation

Faribault Public Schools

Hardenberg Foundation

Hugh J. Anderson Foundation

Katherine B. Andersen Fund, St. Paul Fdn

Lloyd K. Johnson Foundation

Mardag Foundation 

McKnight Foundation

Minnesota State Funds

MN Youth Intervention Program Grant

MN Office of Higher Education

Northfield Healthy Community Initiative

Northfield Public Schools

Northland Foundation

Northwest Area Foundation

Open Your Heart to the Hungry and Homeless

Ordean Foundation

Otto Bremer Trust

Proto Lab

RBC Foundation

Rice County Family Services Collaborative

Sauk Rapids-Rice School District

Securian Foundation

Shavlik Foundation

State of Minnesota

The Saint Paul Foundation

TORCH Initiative

United Way, Head of the Lakes

Workforce Development Inc.

Xcel Energy

YouthFirst

Youthprise

Individual Donors
Youthprise was founded as a nonprofit philanthropic intermediary in 2010 

by the McKnight Foundation. Youthprise’s mission is to increase equity 

with and for Minnesota’s Indigenous, Low-income, and Racially Diverse 

Youth. Today Youthprise is a resource to youth and youth-serving 

organizations and systems throughout Minnesota. Youthprise mobilizes 

and invests resources, advances knowledge and systems change to 

increase equity for Minnesota’s youth. 

3001 Broadway Street Northeast Suite 310 

Minneapolis, MN 55413

si@search-institute.org (800) 888-7828

3001 Broadway Street Northeast Suite 330 

Minneapolis, MN 55413 info@youthprise.org (612) 564-4858

Search Institute is a nonprofit organization that partners with schools, 

youth programs, and other organizations to conduct and apply research 

that promotes positive youth development and equity. We seek to be an 

innovator by listening to young people and providing insight to create 

change in the lives of youth through those that work directly with young 

people.
This program began in 2015, when Youthprise secured funding from state and federal agencies 

to increase employment opportunities for young people across Minnesota. The Minnesota 

Department of Employment and Economic Development (DEED) Office of Youth Development, 

Search Institute, and MENTOR Minnesota are key partners in the initiative.

Primary funding was provided by the Corporation for National and Community Service, who 

awarded Youthprise with a $3 million Social Innovation Fund grant for three years. McKnight 

foundation provided essential dollars to underwrite the administrative capacity of Youthprise to 

carry out this project. Youthprise matched the amount with local funds and selected six 

organizations to implement the Opportunity Reboot model, which aims at increasing career 

pathways for opportunity youth in Minnesota. The six original partner organizations include:

§ Amherst H. Wilder Foundation, (Project name- Prior Crossing)

§ Guadalupe Alternative Programs (GAP), 

§ MIGIZI Communications, 

§ Northfield Healthy Community Initiative, (Project name- Tri-City Bridges to the Future)

§ Sauk Rapids-Rice School District/ Initiative Foundation, (Project name- Compass) and 

§ SOAR Career Solutions (Project name- Opportunity Youth of Duluth).

Opportunity Reboot addresses the economic challenges confronting young people and is one 

solution to the persistent racial disparities in our state. While the Twin Cities ranks #1 among 

the largest 25 metro areas for the proportion of adults working, the Twin Cities ranks 23 out of 

25 for the largest employment gap between White and Black residents.

A tremendous amount of people worked tirelessly, advocated, and showed-up to make this 

program possible. The Core Partners are Minnesota Department of Employment and Economic 

Development’s Larry Eisenstadt and Kay Tracey; MENTOR Minnesota’s Nicki Patnaude, Jess Anna 

Glover and Mai-Anh Kapanke; and our Training and Technical Assistance Consultant Dr. Rose Chu 

who provided technical support, training and consistent leadership through all 5 years of 

program implementation. 

Youthprise’s key staff who were instrumental in developing the compliance and administrative 

support for this federally funded project include: Vice President Marcus Pope, Program Director 

Melissa Mitchell, and Grants and Compliance Officer Maurice Nins, along with our finance and 

administration teams.

We want to highlight special recognition for the McKnight Foundation and staff the primary 

funder for Opportunity Reboot matching funds. These vital funds and partnership enabled the 

leveraging of public dollars and is a great example of how private philanthropy can increase the 

capacity of local intermediaries and communities to increase access to resources to expand 

positive outcomes.

Core to this and all Youthprise work is being able to lead this work with youth voice being 

centered, a critical component for Youthprise and this project’s success. We recognize brilliant 

youth from Irreducible Grace Foundation who participated in the selection process for our 

subgrantees. Their preparation, passion and input were critical in selecting a strong cohort of 

partners ready to co-create Opportunity Reboot and make it into the successful project it is 

today. 

Acknowledgements From Youthprise
Search Institute is grateful to Youthprise for inviting us to be a thought partner and the 

evaluation lead in this groundbreaking and innovative work aimed at enriching the communitybased supports available to young people who are, too often, forgotten. Their vision both for 

this project, and a Minnesota where all young people have equitable futures is inspiring and 

motivating. Joined by the Minnesota Department of Employment and Economic Development 

and MENTOR Minnesota, Youthprise has moved the needle on behalf of opportunity youth 

across the state. It is a rare opportunity to be part of such a dynamic team that leads with a 

mission and clear focus on what really matters: investing in young people to reduce disparities. 

Bearing witness to the impact of our collaboration has been an incredible experience.

The support, flexibility, and patience of the leaders and staff at each of the community partners 

made this project, and the evaluation, possible. The evaluation efforts of this project changed as 

we met practical and federal funding realities. These partners met each request with an 

openness to new ideas, a willingness to engage in collective and creative problem-solving, and a 

commitment to seeing the work through. Our relationship with these community partners was 

defined by transparency, mutual respect, and shared learning, as we negotiated the need for 

evaluation rigor with the realities of the program and the young people they served. 

Unremittingly, these community partners do the really hard and important work on behalf of, 

and in partnership with, opportunity youth. For this, we are deeply indebted to them.

Without a doubt, the best part of this collaboration was the opportunities our team had to 

listen to, and learn side-by-side from, the young people in the community programs. Through 

the evaluation efforts, they entrusted us with their stories, sharing both their hard truths and 

dreams for the future. There is simply no greater gift. The responsibility of sharing their personal 

narratives and experiences was the touchstone for the execution of this high-quality evaluation.

An evaluation of this scope required the assembly of a richly talented and diverse team of 

collaborators. This included: Lacy Allen, Michelle Decker Gerrard, Eliel Gebru, Edith Gozali-Lee, 

Jennifer Griffin-Wiesner, Jill Johnson, Sera Kinoglu, Christine Lindberg, Nicole MartinRogers, 

Stephanie Nelson-Dusek, Christen Pentek, Nicole Perry, Stephanie Peterson, Eugene 

Roehlkepartain, Jessie Saul, Peter Scales, Mackenzie Steinberg, Theresa Sullivan, and Martin Van 

Boekel. The contributions of each of these people strengthened the evaluation and the utility of 

the study findings.

Acknowledgements From Search Institute
TABLE OF CONTENTS

Executive Summary 8

Section 1. Introduction 13

Section 2. Implementation Evaluation 30

Section 3. Preliminary Impact Evaluation 58

Section 4. Conclusions and Contributions 116

References 126

Appendix A. Changes to the Approved Evaluation Design 136

Appendix B. Opportunity Reboot Mapping Tool 138

Appendix D. Human Research Participant Protections 149

Appendix E. Psychometric Properties of Opportunity Reboot Metrics 150

Appendix F. Developmental Relationships Framework 155

Appendix C. Opportunity Reboot Model Alignment Assessments 140

Positive Disruption: Opportunity Reboot Model
The implementation and evaluation of the Opportunity Reboot model took place under very 

different conditions than exist at the time when this report is being submitted.

In the early months of 2020, the global COVID-19 pandemic reached Minnesota. This resulted not 

only in record job losses and an absence of new hires but the stark showcasing of long-standing 

disparities in access to health care and education among indigenous communities, communities 

of color, and low-income communities. All data included in this report were collected prior to the 

COVID-19 pandemic; therefore the data on unemployment and other occupation- and educationrelevant variables are likely to underestimate the current COVID-19-era state of affairs for 

opportunity youth in Minnesota. Early unemployment data and other indicators suggest that 

indigenous communities, communities of color, and opportunity youth have been hit hardest by 

this pandemic, in terms of physical and mental health, education equity and access, and 

unemployment (Centers for Disease Control and Prevention, 2020; Herold, 2020; Hooper et al., 

2020).

These data were also collected prior to the death of George Floyd, an unarmed Black man, who 

was killed during an arrest by a White Minneapolis police officer on May 25, 2020. Floyd’s death 

sparked protests and an uprising to fight racial injustice across the Twin Cities (and, the country). 

In the Twin Cities, some of these protests resulted in violence, looting, and a strong US military 

presence on city streets. The space occupied by MIGIZI, one of the community partners in the 

Opportunity Reboot evaluation, was set on fire and burned down during these protests. Peaceful 

protests and an outpouring of community support were also abundant. The trauma of these 

events — George Floyd’s death, the violence, the systemic racial injustices people of color 

experience daily — will, undoubtedly, impact the mental health and social-emotional needs of 

the youth served by our community partners and the Opportunity Reboot model in the days and 

months ahead.

The work of our community partners, and the promise of the Opportunity Reboot model, has 

never been more important. The challenges ahead of us are big and require a bold and concerted 

approach. We at Youthprise, Search Institute, and all of our partners remain committed to 

adaptive, inclusive, and responsive efforts to meet the needs of youth during this unprecedented 

time in history. Now more than ever we have to ensure that the opportunity youth in Minnesota 

know they are valued and have advocates in their lives. 

PREAMBLE

Positive Disruption: Opportunity Reboot Model
The number of young people living in the margins of society – disconnected from work and 

school, or isolated from family – reflects one of the most pressing social inequities of our time. 

An estimated one in nine youth ages 16 to 24 are out of work and school in the United States 

(Burd-Sharps & Lewis, 2018). These opportunity youth often face a large range of complex 

issues, such as homelessness, chemical dependency, mental health issues, learning disabilities, 

and health disparities perpetrated by a range of systemic issues. Minnesota is not exempt from 

these same realities. More than 40,000 opportunity youth call Minnesota home (Lewis, 2019); 

the state that also bears the largest educational achievement gap in the nation for both racial 

and economic (Grunewald & Nath, 2019; Minnesota Compass, 2020; Nitardy et al., 2015).

These data are cause for alarm, and yet, there is hope. For many opportunity youth, these 

trajectories can be disrupted by surrounding them with a cohesive web of supports that not 

only provide access to important health, education, and employment resources but also spaces 

where these young adults’ strengths are seen and leveraged. The Opportunity Reboot model is 

rooted in this sense of hope. 

Partners: Youthprise, a philanthropic intermediary focused on increasing equity with and for 

Minnesota’s indigenous, low-income, and racially diverse youth, received a grant from the Social 

Innovation Fund in 2015 to implement and evaluate the Opportunity Reboot model over 4 

years. Youthprise contracted with Search Institute, a nonprofit applied research organization, to 

conduct the evaluation. Six Minnesota-based community partner organizations were selected in 

2016 to integrate the Opportunity Reboot model into their program and execute a two-phase 

implementation and preliminary impact evaluation. These partners included: Opportunity Youth 

of Duluth, Compass, Tri-City Bridges to the Future, MIGIZI, Prior Crossing, and the Guadalupe 

Alternative Program. Figure 4 provides additional background information on these partners. 

The Opportunity Reboot model is rooted in this sense of hope. 

Program Summary: Partnering with community-based organizations who provide wraparound 

supports, Youthprise developed a technical assistance and program enhancement model 

designed to leverage the existing capacity and strengths of community programs to more 

effectively create pathways to school, career, and life success for opportunity youth. They did 

this by expanding services and integrating an intentional, systemic focus on four model features 

that have been empirically linked to educational achievement and career development. These 

four core model features are: 

a) Positive mentoring relationships —

including both relationships 

within and outside of

community partner programs; 

b) Individualized goal supports; 

c) Coordinated career pathways 

supports; and, 

d) Impactful cross-sector 

partnerships. 

EXECUTIVE SUMMARY 

Positive Disruption: Opportunity Reboot Model 8
The Opportunity Reboot model was designed to support opportunity youth: youth ages 14 to 24 

who were either in foster care, the juvenile justice system, homeless, or disconnected from school 

and education. The target sample for the evaluation consisted of 418 youth. These are youth who 

participated between April 2017 (within 6 months prior to baseline data collection; i.e., when the 

model was fully rolled out to partners) and April 2018 (beginning of the endline survey data 

collection window). In total, 298 opportunity youth participated in the baseline youth survey, 236 

participated at endline; 194 of these youth participated at both time points. 

Intended Outcomes: The Opportunity Reboot model aimed to enhance program impact on critical 

short-term youth development outcomes (i.e., positive identity, social-emotional competencies, 

skills for systems navigation) and select intermediate outcomes (for participants age 18 or older) 

including securing living wage employment and stable or increasing wages for four quarters postendline.

Prior Research: Youthprise developed the Opportunity Reboot model in 2015, with further 

refinements in 2017 based on feedback from community partners. When funding was granted, the 

full Opportunity Reboot model was untested. However, the extant scientific literature had 

established preliminary or moderate empirical evidence that each of the model features, 

independently, had an impact on important youth outcomes. Each of the model features had also 

been established, in practice, as effective ways to support opportunity youth.

Targeted Level of Evidence: Using the standards set forth by the Corporation for National and 

Community Service, the overarching purpose of the preliminary impact evaluation was to strengthen 

the level of preliminary evidence and establish emerging moderate evidence that the Opportunity 

Reboot model positively impacted the lives of opportunity youth. Extant research had established 

preliminary or moderate evidence of the impact of each of the Opportunity Reboot model features 

on important youth development outcomes. Yet, evidence had not previously been established of 

how these features impact youth when: (a) implemented in concert via an enhancement model to 

existing programs; (b) with a multi-pronged system of technical assistance supports; and, (c) with 

opportunity youth residing in urban and rural regions of Minnesota. The data collection activities in 

the preliminary impact phase of the evaluation sought to attain this foundational level of evidence 

for the Opportunity Reboot model by demonstrating quantitative change in youths’ self-report 

surveys, in the raw and compelling narratives participants openly shared, and in the employment 

and wage records maintained by the state of Minnesota.

Evaluation Designs: The evaluation examined both program implementation and program impact. 

The implementation evaluation was a descriptive, non-experimental design that employed mixed 

methods. It focused on how the Opportunity Reboot model was implemented, contributing to our 

understanding of contextual differences in how the model is realized on the ground across 

geographically diverse community partners who serve young people with complex needs. 

Implementation data were collected through youth and staff focus groups, structured interviews 

focused on fidelity to the Opportunity Reboot model (i.e., Opportunity Reboot Mapping Tool), and 

staff surveys.

The preliminary impact evaluation fused three evaluation designs: 

1) a single group non-experimental outcome design (which included baseline and endline youth surveys, n = 194; staffreported surveys, n = 194); 

2) qualitative impact interviews, n = 29; and, 

3) a quasi-experimental design study of employment and wage attainment that leveraged data collected by the 

Minnesota Department of Employment and Economic Development (DEED) to compare Opportunity Reboot 

participants (n = 209) and non-participants (n = 241). Propensity score matching was used to generate a 

demographically-similar comparison group of opportunity youth from the state administrative database who did not 

participate in programs using the Opportunity Reboot model.

Positive Disruption: Opportunity Reboot Model 9
Metrics and Instruments: The implementation and preliminary impact research questions 

required the use of data from multiple sources using a variety of instruments. This included: (a) 

youth and staff focus groups using a semi-structured qualitative protocol; (b) the Opportunity 

Reboot Mapping Tool which used Likert-type indicators of model fidelity and open-ended 

questions; (c) the Opportunity Reboot Youth Survey which included metrics on youths’ 

experiences of the Opportunity Reboot model features, youth short-term outcomes, program 

quality and satisfaction indicators, and selected youth-focused outputs named in the 

Opportunity Reboot logic model; (d) a staff-report survey on individual youth focused on 

implementation factors; (e) a staff survey on programming outputs aligned with the Opportunity 

Reboot logic model; and, (f) the Common Participant Profile, which captured detailed sociodemographic on Opportunity Reboot participants and was used by DEED to construct the 

comparison group for the quasi-experimental employment and wage attainment study.

Analytic Approach: Implementation data were analyzed using descriptive statistics. Multiple 

regression models were used to analyze data in the single group non-experimental outcome 

evaluation. All qualitative data were analyzed using systematic inductive qualitative coding 

methods. Difference-in-difference statistical models tested for between-group differences in the 

quasi-experimental design study using propensity score matching methods. 

Research Questions: The study was guided by 10 research questions.

Implementation

1. Was the Opportunity Reboot model implemented with fidelity?

2. What does participation and engagement in the Opportunity Reboot model look like for 

opportunity youth?

3. Did Opportunity Reboot program partners achieve output targets (named in the Opportunity 

Reboot logic model)?

4. How satisfied were Opportunity Reboot participants with their experiences and the overall 

quality of their programs?

Preliminary Impact

5. Did youth who participated in programs using the Opportunity Reboot model experience 

measurable and significant gains in short-term outcomes over the program year? Did all 

youth experience the benefits of program participation equally, or were program impacts 

experienced differentially by diverse demographic groups of youth?

6. Controlling for demographic, background, and implementation factors, what are the 

predictive associations between the youths’ experiences of the Opportunity Reboot model 

features and their short-term outcomes? 

7. In their own words, how did participants describe experiencing the features of the 

Opportunity Reboot model at their program sites?

8. In their own words, how did youth describe the impact of Opportunity Reboot on their lives?

9. What percent of youth in the Opportunity Reboot group were able to secure employment? 

How is this similar or different from the comparison group?

10. What percent of youth in the Opportunity Reboot group were able to secure living wage 

employment? How is this similar or different from the comparison group?

Key Findings: The implementation evaluation demonstrated that the Opportunity Reboot model 

was implemented with a high level of fidelity, and that there was a value-add of providing 

community partners with a multipronged system of support to promote model integration, the 

building of a learning community, and increasing fidelity to the model. 

Positive Disruption: Opportunity Reboot Model 10
The preliminary impact evaluation of Opportunity Reboot built directly and iteratively on the 

implementation evaluation. This evaluation was motivated by two goals: to empirically link 

young people’s experiences in the programs using the Opportunity Reboot enhancements to key 

short-term youth development outcomes; and, to test whether participants in Opportunity 

Reboot programs fare better than non-participants on select intermediate outcomes.

§ The strongest and most consistent empirical finding to emerge from this evaluation is that 

relationships matter. Opportunity youth who strengthen their relationships with program 

staff are more likely to show positive gains in their financial literacy, job-seeking skills, and 

resource identification skills. 

§ When opportunity youth build strong relationships with other adults outside of the program 

that support their growth, they are more likely to show positive gains – over and above all of 

the other model features – on eight of the eleven short-term outcomes: future orientation, 

civic efficacy, self-awareness, responsible decision-making, relationship skills, financial 

literacy, job seeking skills, and resource identification skills. This is reinforced in the 

qualitative interview and focus group data where youth reveal through their personal stories 

and experiences the subtleties of what happened in these relationships that make them so 

powerful.

§ Regression models showed an association between the individualized goal-setting supports 

and the coordinated career pathways supports features of the Opportunity Reboot model on 

only one outcome: growth in job-seeking skills. This association was negative for the 

individualized goal-setting supports feature, and positive for career pathways supports. This 

negative association (which, interestingly, is seen across the regression models, although not 

significant) may be an artifact of the phenomenon that youth who require the most support 

setting goals have more urgent short-term needs than those captured by the targeted 

outcomes in this evaluation. Reassuringly, the experience of being in a program that provides 

career pathways supports was a strong positive predictor of job-seeking skills. The absence of 

significant empirical associations between individualized goal-setting supports, career 

pathways supports, and the short-term youth outcomes should not be interpreted as 

evidence that these features of the Opportunity Reboot model are unimportant. In fact, the 

impact narratives give voice to how critically important these features were to helping youth 

see new possibilities for themselves and propel their lives forward. Relationships are the 

entry point for many of these goal and career supports.

§ Opportunity Reboot participants were more likely than similar peers not participating in 

these programs to secure full-time employment over the period of a year, and to avoid the 

seasonal dip in employment often seen during Minnesota’s winter months. This was 

particularly true for youth Youth of Color in the Opportunity Reboot group. While this finding 

holds promise, the data suggest very few opportunity youth (in the Opportunity Reboot and 

comparison groups) are securing living wage employment and – although their wages did 

increase over time – they still fall below the state-defined threshold to support the costs of 

stable housing, food, and other basic necessities in Minnesota.

Collectively, these findings demonstrate that the Opportunity Reboot model has strong 

preliminary evidence of impact on key developmental and employment outcomes for 

opportunity youth residing in Minnesota.

Positive Disruption: Opportunity Reboot Model 11
Evaluation Updates: In March 2017, the Corporation for National and Community Service Social 

Innovation Fund (SIF) approved Youthprise’s SIF Evaluation Plan for an implementation 

evaluation. The original plan included three major phases: (a) developmental evaluation; (b) 

capacity building; and, (c) pilot testing for the forthcoming impact evaluation. This work was due 

to be completed in July 2018. The original plan was to incorporate and use the findings from the 

implementation evaluation to prepare a high-quality, contextually-informed Impact SIF 

Evaluation Plan, which was due to be submitted in July 2017. When SIF was defunded by the 

United States Congress, Youthprise worked closely with their SIF program officer to reimagine 

ways existing funds could be leveraged for maximum benefit. This resulted in three significant 

changes to the evaluation design: (1) Truncating the capacity-building phase of the 

implementation evaluation; (2) Streamlining research questions; and, (3) Adding more robust 

methods to test for preliminary impact. Additional detail on these changes can be found in 

Appendix A.

Next Steps: Youthprise continues to support and make improvements to the Opportunity 

Reboot model and is actively seeking sustainable funding streams to pay for the multi-pronged 

system of support required to implement the full model and expand it to other community 

partners. Additional research is needed with larger sample sizes and experimental evaluation 

designs to confirm preliminary impact findings.

Positive Disruption: Opportunity Reboot Model 12
Section 1

Introduction 
The number of young people living in the margins of society – disconnected from work and 

school, or isolated from family – reflects one of the most pressing social inequities of our time. 

An estimated one in nine youth ages 16 to 24 are out of work and school in the United States 

(Burd-Sharps & Lewis, 2018). This number is higher in rural areas, and among youth who identify 

as Latinx, Black, or male. Bridgeland and Mason-Elder (2012) characterize these youth who are 

at risk of not reaching their full potential as “opportunity youth — both because they are 

seeking opportunity and they present an opportunity to our nation if we invest in them” (p. 5).

These opportunity youth often face a large range of complex issues, such as homelessness, 

chemical dependency, mental health issues, learning disabilities, and health 

disparities (Grunewald & Nath, 2019; Minnesota Compass, 2020; Nitardy et al., 2015).

Between 2000 and 2030, the population of Minnesota’s Youth of Color (ages 10-19) is 

projected to grow from 16 to 27 percent (Minnesota Department of Health, 2012). In a 

national comparison of graduation rates, Minnesota ranks near the bottom for on-time 

graduation of Youth of Color (Minnesota Compass, 2017) and Youth of Color in Minnesota 

are less likely than their White peers to enroll in 4-year undergraduate colleges (Report of 

the Minnesota Advisory Committee to the U.S. Commission on Civil Rights, 2013). 

These inequities in opportunity and access undergird economic and employment 

disparities. This, along with the changing demographics of Minnesota’s workforce, has 

exacerbated the unemployment gap between Whites and people of color. While Minnesota 

touts some of the lowest unemployment rates in the country, Minnesota’s Black population 

has an unemployment rate that is roughly 13 percentage points higher than that of 

Minnesota’s non-Hispanic, White population (Minnesota Compass, 2018).

These data are cause for alarm, and yet, there is hope. For many opportunity youth, these 

trajectories can be disrupted by surrounding them with a cohesive web of supports that not 

only provide access to important health, education, and employment resources but also spaces 

where these young adults’ strengths are seen and leveraged. The motivating force behind 

Opportunity Reboot has been to enhance the odds that opportunity youth in Minnesota would 

have available to them the kind of programs and supports that would help them access, create, 

prepare for, and take advantage of the education and career opportunities that could radically 

shift their life trajectories.

The following section provides a detailed description of the Opportunity Reboot model and its 

four core features. It elaborates on the design used in the evaluation, and provides detailed 

descriptions of the community partners, and the opportunity youth who were the targeted 

beneficiaries of the program. The Introduction concludes by presenting the 10 research 

questions that were the focus of the evaluation, and briefly summarizes the contributions of 

the study to research and practice knowledge around promoting greater educational and 

employment equity for opportunity youth.

Positive Disruption: Opportunity Reboot Model 14
Positive mentoring relationships — including both relationships within and outside of 

our community partner programs;

Individualized goal supports;

Coordinated career pathways supports; and,

Impactful cross-sector partnerships. 

Six community partner organizations joined Youthprise in integrating the Opportunity Reboot 

model into their program (see Figure 2) and evaluating its implementation and preliminary 

impact. These organizations each have long histories of serving opportunity youth across the 

state of Minnesota. 

Despite growing evidence of the critical need to link technical, academic, and social-emotional 

development to prepare youth to enter the workforce effectively (Aspen Institute, 2019), most 

intervention efforts targeting opportunity youth remain largely siloed, reflecting and 

perpetuating the implicit yet ultimately impractical assumption that youth will access different 

programs for different needs (VanDenBerg & Grealish, 1996). In reality, both for developmental 

and practical reasons, young people need access to programs and services that help them 

develop holistically, rather than having to navigate multiple systems. For this to be possible, 

programs in different sectors need shared sets of approaches, principles, and interventions 

through which they engage young people and set them on a path toward economic stability and 

independence. Improving the education and workforce prospects for opportunity youth 

currently disconnected from such systems depends on this type of collaborative approach.

Youthprise’s Opportunity Reboot model was created as part of a larger strategy to build 

collaborative, community-based approaches to address educational and social obstacles that 

opportunity youth face. The model does this by explicitly attending to developing skills, 

mindsets, and opportunities holistically through youth-adult mentoring and integrated career 

pathways supports. The wraparound nature of the Opportunity Reboot model is posited to work 

because it inherently responds to the multiple and complex needs of opportunity youth (e.g., 

The White House Council for Community Solutions, 2012). The four features of the Opportunity 

Reboot model are designed to work in concert to provide opportunity youth with the full range 

of supports that they need to experience improved educational and career attainment. The 

Opportunity Reboot model was specifically designed to support opportunity youth: youth ages 

14 to 24 who were either in foster care, the juvenile justice system, homeless, or disconnected 

from school and education.

Opportunity Reboot Model

The Opportunity Reboot model is rooted in this sense of hope generated by applied positive 

youth development research. Partnering with community-based partners who provide 

wraparound supports, Youthprise – a Minnesota-based non-profit focused on reducing 

disparities with, and for, Minnesota youth by mobilizing and investing resources, advancing 

knowledge, and advocating for change – developed a technical assistance and program 

enhancement model designed to leverage the existing capacity and strengths of community 

programs to more effectively create pathways to school, career, and life success for opportunity 

youth. They did this by expanding services and integrating an intentional, systemic focus on four 

model features that have been empirically linked to educational achievement and career 

development. These four core model features are (see Figure 1)

❶

❷

❸

❹

Positive Disruption: Opportunity Reboot Model 15
Because Opportunity Reboot is a program enhancement model, rather than a standalone 

program, a multipronged system of support was developed and delivered to assist partnering 

programs in integrating the Opportunity Reboot model into their day-to-day programming. This 

included: biweekly technical assistance calls; quarterly in-person convenings for partners to 

learn and share with one another; annual site visits; and access to two mentoring trainings 

provided by MENTOR Minnesota. These trainings were: Maximize Your Opportunity, which 

focused on helping youth identify and build strong mentoring relationships; and Maximize Your 

Impact, which focused on equipping adults with the skills and mindsets needed to be mentors. 

Figure 2 depicts the Opportunity Reboot logic model, with seven major elements:: 

1. Core Principles: Three core principles undergirded all aspects of the Opportunity Reboot 

model. The work was culturally grounded, relationships-focused, and inclusive of youth 

voice.

2. Inputs: Resources needed to implement the model, including the multipronged system of 

support outlined above.

3. Activities: Planned activities, which in a program enhancement model, refer to the 

Opportunity Reboot model features, concepts, and strategies. For space reasons, the 

strategies are not listed in the graphic.

4. Outputs: Tangible and quantifiable services delivered as a result of implementing the 

Opportunity Reboot model.

5. Short-Term Outcomes: The Opportunity Reboot logic model names both youth- and 

organization-level short-term outcomes. The current evaluation focused only on the youthlevel outcomes. There was a reasonable expectation that all identified youth short-term 

outcomes were malleable, and that quantitative change could be captured over the course 

of a 9-12 month period across diverse populations of young people. See Short-Term 

Outcomes in the Preliminary Impact section of the report for additional detail about how 

these outcomes were selected.

6. Intermediate Outcomes: Outcomes expected 12+ months after the start of program 

participation. Select intermediate outcomes are assessed in the employment and wage 

attainment study (see the Preliminary Impact section for additional detail).

7. Longer Term Outcomes: Outcomes expected 18+ months after the start of the program. 

These outcomes are not assessed in this evaluation.

This logic model was used as a guide for generating research questions.

Positive Disruption: Opportunity Reboot Model 16
Figure 1 Opportunity Reboot Model

Positive Disruption: Opportunity Reboot Model 17
Figure 2 Opportunity Reboot Logic Model

Positive Disruption: Opportunity Reboot Model 18
Yet, we know that youth who have stable 

and high-quality social relationships are 

more likely to stay in school (Center for 

Promise, 2015; Sinclair et al., 2005), 

experience academic gains (Scales et al., 

2019; Thompson & Kelly-Vance, 2001), 

exhibit fewer behavior problems (Keating 

et al., 2002; Somers et al., 2008), 

demonstrate higher socioemotional 

competence (Roehlkepartain et al., 2017), 

and be resilient in the face of persistent 

challenges (Konopka, 1973; Werner & 

Smith, 2001). 

Positive Mentoring Relationships

Overview of Prior Research

Youthprise developed the Opportunity Reboot model in 2015, with further refinements in 2017 

based on feedback from community partners. When funding was granted, the full Opportunity 

Reboot model was untested. However, the extant scientific literature had established 

preliminary or moderate empirical evidence that each of the model features, independently, 

had an impact on important youth outcomes. Each of the model features had also been 

established, in practice, as effective ways to support opportunity youth.

Positive mentoring relationships are a foundational element of the Opportunity Reboot model. 

Mentoring is integrated into the roles of case managers, employment specialists, resource 

specialists, educators, professional mentors, apprenticeship supervisors, and employers. In these 

mentoring roles, adults provide a wide range of critical socioemotional, developmental, and 

instrumental supports for opportunity youth (Center for Promise, 2015; Dubois & Silverthorn, 

2005).

This feature was built on the broad research base showing that opportunity youth are more likely 

than their peers to be disconnected from key sources of social support, including parents, 

extended family, and teachers (Putnam, 2015). 

Opportunity youth are less likely to experience the social capital represented by high-quality 

relationships and their associated resources (Scales, Boat, & Pekel, 2020), and research has also 

found that for some, such as youth from lower-income backgrounds, their relationships with key 

potential resources such as teachers worsens over time (Scales et al., 2019). Thus, the relational 

gap for opportunity youth presents a significant threat to their well-being and developmental 

trajectories. 

Mentoring-based programs and interventions across the country have rallied to fill this void, 

including Big Brothers Big Sisters, MENTOR, and Check & Connect. In a recent study, opportunity 

youth who had a mentor were 20% more likely than opportunity youth without a mentor to plan 

to go to college, and 30% more likely to enroll or plan to enroll in postsecondary learning (Bruce & 

Bridgeland, 2014). Mentoring relationships have the potential to increase opportunities for healthy 

development and more equitable outcomes among opportunity youth by providing youth with the 

confidence, access to resources, and ongoing support they need to achieve their potential (e.g., 

DuBois et al., 2011; Duncan-Andrade, 2010; Erickson et al., 2009; Schwartz et al., 2011). 

Positive Disruption: Opportunity Reboot Model 19
For example, in a work-based mentoring program, youth who participated in the formal 

mentoring program were more likely than non-participants to believe school was directly relevant 

to work; mentored youth also exhibited higher levels of self-esteem (Symonds et al., 2011). 

Randomized control trials of the mentoring-based Check & Connect program also show the power 

of strong, supportive youth-adult relationships to re-engage at-risk youth in school (Sinclair et al., 

2005; Sinclair et al., 1998). Studies of juvenile offender reentry programs with strong mentoring 

components have found that youth who participated in these programs had lower rates of 

recidivism (Bouffard & Bergseth, 2008; Drake & Barnoski, 2006). Relatedly, a mentoring and 

vocational development initiative to help transition violent young adult offenders to their 

community, The Boston Reentry Initiative, found that program participants had 30% lower rates 

of recidivism relative to non-participants (Braga et al., 2009).

To build on the demonstrated link between strong relationships and positive developmental 

outcomes, Operation Opportunity Reboot drew on the Developmental Relationships Framework 

constructed by Search Institute from extensive literature reviews, and qualitative and quantitative 

studies in the family, school, peer, and youth-serving organization contexts (Pekel et al., 2018). 

The framework outlines five major elements of relationships—express care, challenge growth, 

provide support, share power, and expand possibilities—and 20 actions adults and youth engage 

in to experience those relational strengths. Subsequent research using this framework has shown 

that youth with higher levels of developmental relationships with parents, teachers, peers, and 

adults in youth-serving organizations do better on a range of academic, psychological, socialemotional, and behavioral outcomes (Pekel et al., 2018; Scales et al., 2019; Scales et al., 2020; 

Syvertsen et al., 2018). Opportunity Reboot partners were introduced to the framework, which 

was included in trainings showing how it could strengthen their focus on positive mentoring, and 

the framework informed both the quantitative items and qualitative protocols used in the 

evaluation study.

A second core feature of the Opportunity Reboot model is described as individualized goal 

supports. Goals and aspirations are assumed to be critical motivators for learning and work. Yet 

just having positive aspirations or fantasies rarely stimulates goal-directed action (Duckworth et 

al., 2013; Oettingen, 2012). Several factors are at work, including young people’s default mental 

focus on short-term concerns or desires (Oyserman, 2013), the possible lack of knowledge about 

institutional systems, and how to access available resources.

Mentoring often links youth to additional culturally relevant resources and social capital that 

meet their needs (Oransky et al., 2013) and help them make progress towards their personal 

goals (Watson et al., 2016). Furthering the evidence for the importance of responsive adults in 

the goal-setting and attaining process, studies have positively associated youths’ discussions of 

school and future plans with supportive adults with positive academic outcomes (Hill et al., 

2004).

Through individualized goal-setting supports and mentoring, Opportunity Reboot participants are 

guided to shift from vague aspirations toward concrete goals based on their intrinsic interests and 

skills, including the long-term relevance of what they are being asked to do to pursue their own 

aspirations. By regularly monitoring their goals and learning, they increase their investment in 

their own progress toward their aspirations (Hulleman & Harackiewicz, 2009).

Positive Disruption: Opportunity Reboot Model 20

Individualized Goal Supports 
The career pathways feature of the Opportunity Reboot model is based on a toolkit created by the 

Department of Labor’s Employment and Training Administration (Elsey et al., 2015). In the 

Opportunity Reboot model, these career pathways characteristics and key elements have been 

adapted as follows:

§ Create opportunities for career exploration that are grounded in connecting youth with 

employers in high growth–high demand industries.

§ Assist youth with career preparation and planning.

§ Provide culturally appropriate wraparound services that include multiple support structures preand post-employment to promote retention and encourage continuing education.

Programs implementing key career pathways elements have documented success producing 

educational and employment-based outcomes (Symonds et al., 2011). 

For example, internal pre-/post-program metrics from the Guadalupe Alternative Programs (GAP) –

a community partner in this project – found that, over the course of the program, 68% of 

YouthBuild participants who were English Language Learners increased one functional level on an 

assessment that measured basic skills and the English language/literacy. Although 97% of GAP 

YouthBuild participants entered the program deficient in high school credits, 98% earned enough 

credits for a high school diploma by the end of the year. Descriptive outcomes data from a recent 

Minnesota YouthBuild evaluation complemented these findings, showing that 91% of participants 

either completed high school or obtained a GED. Furthermore, of the 66% of enrollees with criminal 

offenses, only 5% recidivated within a 2-year period after program enrollment, compared to 27% in 

a similar cohort (Wiegand et al., 2015).

Coordinated Career Pathways Supports

Positive Disruption: Opportunity Reboot Model 21
Opportunity Reboot draws its approach to community collaboration from Jobs for the Future’s 

partnership approach, with Opportunity Reboot Community Partners serving as intermediaries 

to coordinate employers, industry partners, community providers, and education/training 

providers in a “collective, place-based effort...to build high-quality employer-connected 

pathways” (Grobe et al., 2015, p. vii). One of the exemplars of this ‘wraparound services’ 

model is Homeboy Industries, which intentionally include mental health services, case 

management, workforce development, educational services, and more for their program 

participants (Delgado, 2012). Following a 5-year evaluation, it was found that 70% of 

individuals who completed the 18-month Homeboy program successfully stayed out of prison 

and secured employment; this is an inverse of the local recidivism statistics (Leap & Franke, 

2008). Walter and Petr (2011) also found that promising wraparound services models are 

locally focused, context driven and include participant voice, choice, family teams, 

individualized services, natural supports, and collaboration between all the partners in the 

model. 

Cross-sector partnerships have “nested identities” in which organizations and programs share 

information and resources, as they have overlapping goals, but not necessarily missions 

(Stadtler & Van Wassenhove, 2016). Employees play multiple roles, as people who are 

developing organizational capacity, mentoring, and negotiating the organizational landscape 

with the youth (Ginwright & Cammarota, 2007; Shah & Mediratta, 2008). Research based on 

the theory of intersectionality that opportunity youth, especially those who are simultaneously 

in multiple marginalized groups (e.g., by race, gender identity, economics, sexual orientation) 

face a matrix of systemic oppressions that can halt their opportunities and investments toward 

success (Crenshaw, 2013; Mattsson, 2014; McBride & Mazur, 2008). When successful at 

holding the individual most marginalized youth at the center of programming, cross-sector 

partnerships create an equally complex matrix of supports and resources for addressing the 

systems failures that stop many marginalized young people from succeeding.

Numerous models point out that wraparound services allow for staff and professional teaming, 

which opens space for conversation about systems change. This focus on change allows the 

community to identify and take steps toward addressing gaps in services to better meet the 

needs of opportunity youth (Walter & Petr, 2011).

Drawing on such scientific evidence, positive mentoring relationships, individualized goal 

supports, career pathways supports, and impactful cross-sector partnerships were identified as 

the cornerstone features of the Opportunity Reboot model. The innovation of this model is the 

consolidation of these activities into a single model and the provision of support to integrate 

these enhancements into existing programs.

Impactful Cross-Sector Partnerships

Positive Disruption: Opportunity Reboot Model 22
Youthprise partnered with Search Institute to conduct an independent evaluation of the 

Opportunity Reboot model, with funding support from the Corporation for National and 

Community Service (CNCS) Social Innovation Fund and other funders. The Opportunity Reboot 

evaluation design included two phases: an implementation evaluation and a preliminary impact 

evaluation (see Figure 3). 

The implementation evaluation was designed to advance understanding of the processes by 

which programs utilizing the Opportunity Reboot model engage opportunity youth and guide 

them on a path to success. To do this, a descriptive, non-experimental design with a mixedmethods approach was used to understand how our community partners’ wraparound 

Opportunity Reboot models were being applied and the degree of alignment with the model’s 

logic model, as well as key insights on how to operationalize fidelity and assess dosage. 

The preliminary impact evaluation was designed to empirically link young people’s experiences 

in the programs using the Opportunity Reboot model to change in key youth development 

outcomes. The preliminary impact evaluation used three designs: (1) a single-group nonexperimental design (which included baseline and endline youth surveys and staff-reported 

implementation factors); (2) qualitative impact interviews; and, (3) a quasi-experimental design. 

The quasi-experimental design included a study of employment and wage attainment using data 

collected by the Minnesota Department of Employment and Economic Development (DEED) and 

propensity score matching to compare Opportunity Reboot participants with a comparison 

group of non-participants. Using the standards set forth by the Corporation for National and 

Community Service, the overarching purpose of the preliminary impact evaluation was to 

strengthen the level of preliminary evidence and establish emerging moderate evidence that the 

Opportunity Reboot model positively impacted the lives of opportunity youth. 

Tables 1 and 2 summarize the purpose, timing, and source of each element of the evaluation 

design. Additional information regarding methodology is incorporated into Implementation 

Evaluation (Section 2) and Preliminary Impact Evaluation (Section 3) sections of this report. 

This evaluation design reflects modifications made when the Corporation for National and 

Community Service Social Innovation Fund was defunded and Youthprise was asked to 

reimagine how to maximize the level of the evidence achieved with the available resources. This 

led to three significant changes to the evaluation design: (1) truncating the capacity-building 

phase of the implementation evaluation, (2) streamlining research questions (e.g., additional 

preliminary impact-level research questions were added), and (3) adding more robust methods 

to test for preliminary impact (e.g., propensity score matching). Appendix A provides detailed 

information, and a justification, for the three significant modifications that were made.

Evaluation Design

Positive Disruption: Opportunity Reboot Model 23
Community Partners

All evaluation activities were done in partnership with six community partners: Compass, Guadalupe 

Alternative Program (GAP), MIGIZI, Opportunity Youth of Duluth (OYOD), Prior Crossing, and Tri-City 

Bridges to the Future. These six community partner organizations joined Youthprise in integrating 

the Opportunity Reboot model into their program and evaluating its implementation and preliminary 

impact. These organizations each have long histories of serving opportunity youth across the state of 

Minnesota. A description of each of the community partner organizations and the opportunity youth 

they serve is provided in Figure 4.

Target Beneficiaries

Over the course of their partnership with Youthprise, the six partner organizations served a 

cumulative total of 1,507 opportunity youth between the start of their project funding in April 2016 

to the end of their project funding in February 2020 (December 2019 for one partner organization). 

Drawing on data from the SIF Data Supplement, 36% of these youth were men and boys of color. 

Other known demographic characteristics of these youth included Native Americans (15%), 

Hispanic/Latinx (13%), and immigrants (29%). This total number of 1,507 youth reflects all 

participants who received program services. Many of these youth did not meet the requirements for 

participation in the study because: (a) they exited the program before the evaluation started; (b) the 

joined the program after the evaluation ended; or, (c) their participation was highly episodic and, as 

a result, they were not exposed to the Opportunity Reboot model features. For example, the 1,507 

number includes youth who visited the drop-in services provided by the Opportunity Youth of 

Duluth. However, these youth received none or very little substantive content tied to the 

Opportunity Reboot model making these youth ineligible for study participation.

Positive Disruption: Opportunity Reboot Model 24

Figure 3 Opportunity Reboot Evaluation Design

Notes. T1 refers to the baseline, or Time 1, administration of the youth survey. T2 refers to the endline, or Time 2, administration of the 

youth survey. The Endline (T2) Youth Survey included measures of program quality and satisfaction, which were included in the 

implementation evaluation. The staff-reported implementation factors included measures of dosage and engagement, which were 

included both the implementation and preliminary impact evaluation.
Table 1 Implementation Evaluation Activities

Positive Disruption: Opportunity Reboot Model 25

Thus, the target sample for the evaluation consisted of a much smaller sample of 418 youth (see 

Table 3). These 418 youth participated in programs using the Opportunity Reboot model 

between April 2017 (within 6 months prior to baseline data collection; i.e., when the model was 

fully rolled out to partners) and April 2018 (beginning of the endline survey data collection 

window). Although there are some differences between the survey participants and target study 

participants (see Table 3), the survey participants are reasonably representative of the target 

sample. The final sample of survey participants includes fewer opportunity youth who identify 

as Native American and/or Hispanic/Latinx and more opportunity youth who identify as 

immigrant/non-Native. 

Activity Purpose Timing Source

Design and Planning Workshops

Articulate the underlying program theory 

driving the work of program sites

Spring 2017 Program Staff

Focus Groups

Understand how staff and youth participants 

experience the program

Spring 2017 Youth 

Participants

Program Staff

Mapping Tools

Assess program alignment with the 

Opportunity Reboot model

Spring 2017

Winter 2017

Spring 2018

Program Staff

Youthprise Staff

Evaluators

Endline (T2) Youth Survey Collect endline data on program quality and 

satisfaction

Spring 2018 Youth 

Participants

Staff-Reported Outputs 

Questionnaire

Secure data on each of the named outputs in 

the logic model

Spring 2018 Program Staff

Staff-Reported Youth-Level 

Implementation Factors and Youth 

Outputs

Secure data on whether each youth met the 

target for the youth-focused outputs named 

in the logic model

Spring 2018 Program Staff

Notes. The staff-reported youth-level implementation factors and youth outputs were collected as part of the same brief survey. The 

implementation evaluation used a descriptive, non-experimental design with a mixed methods approach. Measures from the endline (Time 2) 

Opportunity Reboot Youth survey and Staff-Reported Implementation Factors and Youth Outputs survey were included in both the 

implementation and preliminary impact evaluation activities (see Table 2). 
Table 2 Preliminary Impact Evaluation Activities

Activity Purpose Timing Source

Baseline (T1) Youth Survey Collect baseline data on youth demographics and 

short term outcomes

Fall 2017 Youth 

Participants

Endline (T2) Youth Survey Collect endline data on youth demographics and 

short term outcomes

Spring 2018 Youth 

Participants

Qualitative Impact Narrative 

Activities

Gather rich open-ended insights from youth about 

their program experiences and short term 

outcomes

Spring 2018 Youth 

Participants

Staff-Reported Youth-Level 

Implementation Factors and 

Youth Outputs 

Secure data on each youth’s level of program 

engagement, and dosage

Spring 2018 Program Staff

Quasi-Experimental Design 

Employment and Wage 

Attainment Study with 

Propensity Score Matching

Compare Opportunity Reboot participants to 

similar youth on select intermediate outcomes

Fall 2019 DEED

Notes. DEED = Minnesota Department of Employment and Economic Development. The preliminary impact evaluation used three research 

designs: (1) single-group, non-experimental design (using the baseline and endline youth survey, staff-reported implementation factors), (2) 

qualitative impact narrative activities, and (3) quasi-experimental design with propensity score matching (employment and wage attainment 

study). Measures from the endline (Time 2) Opportunity Reboot Youth survey and Staff-Reported Implementation Factors and Youth Outputs 

survey were included in both the implementation and preliminary impact evaluation activities (see Table 1). 

Positive Disruption: Opportunity Reboot Model 26

Total Youth Served Target Sample

of Study Participants

Final Sample

of Study Participants

Participation Time Frame: Start of Partner Funding to End of 

Partner Funding

6 Months Prior to Start of Baseline 

Youth Survey to Start of Endline 

Youth Survey

Participants who Provided Baseline 

and Endline Youth Survey Data

Total n 1,507 418 194

Native American 15% 22% 7%

Hispanic/Latinx 13% 16% 15%

Men/Boys of Color 36% 50% 42%

Immigrant/Non-Native 29% 31% 45%

Notes. Percentages do not add up to 100% due to non-mutually exclusive categories. The demographic characteristics used to compare samples 

were drawn from the SIF Data Supplement. 

Table 3 Comparison of Total Youth Served, Target Sample, and Final 

Sample
Figure 4 Description of Opportunity Reboot Community Partners

Positive Disruption: Opportunity Reboot Model 27

Opportunity Youth of Duluth (OYOD) is implemented in partnership 

with SOAR Career Solutions, Life House, Woodland Hills, Duluth 

Workforce Development, Lake Superior College, Duluth Public Schools 

Independent School District #709, and Duluth Adult Basic Education. 

The OYOD project builds upon the strengths of each partner agency 

and coordinates employment and stabilization services, addresses 

barriers to education and employment and provides youth with an onramp toward a life of meaningful employment and social engagement. 

This project incorporates transitional and traditional employment 

readiness models in order to meet the individualized needs of youth. 

OYOD serves opportunity youth from the Duluth-area, many of whom 

are youth of color experiencing homelessness, previously incarcerated, 

and/or chemically dependent. The project is located in Duluth, MN.

Compass is a blended alternative educational program that 

emphasizes career and college readiness, civic engagement, and 

overall well-being for high school-aged students who are disconnected 

or on a path to disconnection from school. These opportunity youth 

are offered appropriate academic accommodations and individualized 

programming, a structured and supported setting, connections to 

mentors and community partners, and a balance between alternative 

and mainstream settings. Compass staff help students discover, learn, 

and navigate their path to success. Compass is located in Sauk Rapids, 

MN.

Tri-City Bridges to the Future is a collaborative in Faribault, Red Wing, 

and Northfield that offers career pathways approaches for 130 

opportunity youth ages 14-24, including large numbers of incarcerated 

youth and youth from immigrant communities. The primary focus is 

high school completion. The collaborative provides dual enrollment 

options through partnerships with area higher education institutions 

and postsecondary training in career clusters that have clear pathways 

to higher degrees. Tri-City Bridges is located in Northfield, Faribault, 

and Red Wing, MN.

MIGIZI’s programming provides a green jobs pathway for 60 

indigenous opportunity youth per year by providing education, 

training, supports, and experiences needed to prepare them to 

become financially independent, self-determining adults. The project 

also helps indigenous youth connect with, and discover, their cultural 

role as caretakers of the Earth. MIGIZI is located in Minneapolis, MN.

Prior Crossing, provides culturally competent employment and educational programs and wraparound services for 44 homeless 

youth, a majority of whom are youth of color, at a youth housing facility in Saint Paul, MN. Prior Crossing operates on a Housing 

First model: a person’s access to housing should not be determined by their income, chemical or mental health, or their current 

motivation to improve it. Their primary outcome is to increase access to wraparound services to reduce homelessness by 

providing age-appropriate trauma-informed services, employment and skills training, and social connections in a manner that can 

lift people out of poverty with longer-term supports.

Guadalupe Alternative Programs (GAP) is a community-based education and social service agency. GAP implements the 

YouthBuild model for over 100 opportunity youth from Saint Paul’s West Side, Dayton’s Bluff, Payne-Phalen, and Greater East Side 

neighborhoods by giving them the tools and credentials needed to achieve school, career, and life success. Its program provides 

comprehensive wraparound services along with secondary and post-secondary education, workforce preparation, and 

credentialing. In addition, participants have access to academic and social-emotional supports in-house, and from partnerships 

with multiple community agencies and educational institutions. The population served includes a majority from immigrant and 

refugee communities. GAP is located in Saint Paul, MN.
Ten research questions guided the implementation and preliminary impact phases of the 

Opportunity Reboot evaluation. 

Implementation

1. Was the Opportunity Reboot model implemented with fidelity?

2. What does participation and engagement in the Opportunity Reboot model look like for 

opportunity youth?

3. Did Opportunity Reboot program partners achieve output targets?

4. How satisfied were Opportunity Reboot participants with their experiences and the overall 

quality of their programs?

Preliminary Impact

5. Did youth who participated in programs using the Opportunity Reboot model experience 

measurable and significant gains in short-term outcomes over the program year? Did all 

youth experience the benefits of program participation equally, or were program impacts 

experienced differentially by diverse demographic groups of youth?

6. Controlling for demographic, background, and implementation factors, what are the 

predictive associations between the youths’ experiences of the Opportunity Reboot model 

features and their short-term outcomes? 

7. In their own words, how did participants describe experiencing the features of the 

Opportunity Reboot model at their program sites?

8. In their own words, how did youth describe the impact of Opportunity Reboot on their lives?

9. What percent of youth in the Opportunity Reboot group were able to secure employment? 

How is this similar or different from the comparison group?

10. What percent of youth in the Opportunity Reboot group were able to secure living wage 

employment? How is this similar or different from the comparison group?

Research Questions

Collectively, the implementation and preliminary impact evaluations strengthen the preliminary 

evidence that program enhancement models, like Opportunity Reboot, that are infused into 

existing wraparound support programs hold promise for positively disrupting the lives of 

opportunity youth. This evidence is reflected in the quantitative change observed in youths’ selfreport surveys, in the raw and compelling narratives they openly shared, and in the employment 

and wage records maintained by the state of Minnesota.

Contribution of this Study

Positive Disruption: Opportunity Reboot Model 28
The study featured a number of strengths that collectively add to the applied youth 

development literature and contribute to both research and practice. The effects seen for this 

wraparound model suggest that, while new programs are often needed, existing programs for 

opportunity youth can be enhanced in targeted ways for youth success. Opportunity youth often 

represent a difficult challenge to retain in longitudinal studies, yet this evaluation had a good 

retention rate, enhancing the quality of the data. The inclusion of significant percentages of 

rural and Native American youth was also an unusual feature of the evaluation study. The mixed 

methods approaches allowed both a 30,000-foot observation of overall effects across youth and 

programs, as well as a more granular understanding of how the effects were experienced by 

individual youth. Finally, the attention to both in-depth study of implementation and of 

preliminary impact in one study enabled greater insights into the program features responsible 

for youth outcome effects than is common.

The remainder of this report is organized around the implementation and preliminary impact 

research questions, with a final section that articulates the contribution of this study. At the 

outset of the Implementation Evaluation (Section 2) and Preliminary Impact Evaluation (Section 

3) sections, information is provided on: 

(a) Approach, 

(b) Instruments, 

(c) Sample, and 

(d) How the data were analyzed. 

This is followed by a point-by-point presentation of the research questions and the related 

findings. 

Organization of this Report

Positive Disruption: Opportunity Reboot Model 29
Section 2

Implementation 

Evaluation
The implementation evaluation employed a descriptive, non-experimental design with a mixedmethods approach. This portion of the study focused on how the Opportunity Reboot model 

was implemented, contributing to understanding of contextual differences in how the 

Opportunity Reboot model is realized on the ground across geographically diverse community 

partners who serve young people with complex needs. In addition to helping the evaluation 

team understand how the four feature enhancements of the Opportunity Reboot model are 

implemented, this phase of work also built the capacity of the community partners to collect 

high-quality and useful data to inform the preliminary impact evaluation. It also identified how 

program participants described their experience and satisfaction with their participation in the 

Opportunity Reboot programs. 

To launch the implementation evaluation, Search Institute facilitated full-day Design and 

Planning workshops and focus groups with each community partner. The highly interactive 

workshops were attended by 2-3 evaluation team members, 4-6 staff from the community 

partner organization, and a member of the Youthprise team.

The Design and Planning workshops were organized around six objectives:

1 Copies of these reports are available by request from the authors.

1. Build rapport between program staff and the evaluation team;

2. Co-create expectations for how the community partner and the evaluation team would 

work together;

3. Articulate the logic model guiding the program at each partner site, explicitly describing 

how they were (or were not) already serving opportunity youth in ways that align the 

Opportunity Reboot model and identifying possible areas where additional technical 

assistance may be needed;

4. Invite feedback and recommendations on the core Opportunity Reboot model based on 

partner’s expertise and experience working with opportunity youth;

5. Share the proposed evaluation design and gather feedback on potential barriers; and

6. Audit current data collection activities and evaluation capacity of each community partner.

These workshops were paired with youth and staff focus groups at each community partner site. These 

semi-structured conversations: (a) built rapport with program staff and participants; (b) deepened the 

evaluation team’s understanding of each program; and, (c) contextualized and grounded the evaluation 

design in the lived experiences of opportunity youth. Focus group data were recorded, transcribed, 

and systematically analyzed using thematic analysis. Themes were summarized in reports made 

available to partners and Youthprise1. 

The insights from these workshops and focus groups profoundly 

shaped the Opportunity Reboot model, technical assistance 

support plan, and the evaluation design. They also set the stage 

for what became strong, collaborative relationships between 

the evaluation team and community partners.

The following pages describe in more detail how these activities 

shaped the refinement of the Opportunity Reboot model, 

identified model-focused technical assistance needs of the 

partner organizations, and informed the evolution of the 

evaluation design.

Positive Disruption: Opportunity Reboot Model 31

Designing and Planning the Implementation Evaluation
In addition to these refinements, Youthprise leveraged both existing evidence on best practices 

for supporting opportunity youth and the collective wisdom of community partners to further 

articulate the model by specifying the core concepts and strategies that operationalize each of 

the four Opportunity Reboot model features. These nine core concepts and their undergirding 

strategies are adhered to by programs implementing the Opportunity Reboot model through 

their program practices and policies. The concepts and strategies are identified in Table 4. The 

specific tactics used by partners to implement these strategies varied naturally from program to 

program, allowing them the flexibility required to meet the specific needs of their youth 

population, available partners, and geographic location.

Box 1

Opportunity Reboot Model Refinements

Opportunity Reboot Model language was evaluated to ensure cultural responsiveness. All language in 

documents describing the model were critically reviewed by program partners to ensure they were culturally 

responsive to the diversity of young people served by their programs.

The Positive Mentoring Relationship feature was broadened to include both formal and informal mentoring 

relationships. Several partners noted the critical need for mentors who understand trauma and street 

culture; training in the areas is rarely provided to the adults who volunteer in formal mentoring programs. 

Program staff, case managers, teachers, and employers were commonly identified as mentors by program 

participants.

Important role staff play in helping youth identify goals that are realistic and achievable was acknowledged 

via the Individualized Goal Supports feature. Staff work directly with each young person to figure out their 

goals, then break them down into manageable and concrete steps. For many youth, this is a new process 

that requires skillful staff facilitation.

The Impactful Cross-Sector Partnerships feature was simplified. The revised language more adequately 

encompasses the many types of formal, informal, long-standing, and episodic partnerships leveraged to 

support program participants.

The Design and Planning activities provided a forum for dialogue about the Opportunity Reboot 

model. While the model was met with overwhelming support, partners also offered 

constructive and insightful feedback for further improvements. As a result of these 

conversations, four noteworthy refinements were made to the model (Box 1).

Refining the 

Opportunity Reboot Model

Positive Disruption: Opportunity Reboot Model 32
Table 4 Opportunity Reboot Model Features, Concepts, and Strategies

All four features of the Opportunity Reboot model are interrelated and grounded in evidence-based practice. Each of the 

features reflects three core principles: cultural responsiveness, a relationship focus, and inclusion of youth voice. 

Feature 1. Positive Mentoring Relationships

Concept 1A. Program staff engaged with youth adopt a mentoring mindset grounded in best practices of culturally responsive 

informal mentoring and relationship building.

Strategy 1Ai. Program staff receive customized, ongoing technical assistance in informal mentoring best 

practices that supports continuous professional development.

Strategy 1Aii. Program staff develop and implement a plan for incorporating relevant informal mentoring best 

practices to maximize positive relationships between youth and adults (includes employers, when 

applicable).

Concept 1B. Youth are prepared for mentoring experiences and develop skills to identify and engage informal mentors.

Strategy 1Bi. Youth understand their rights, responsibilities and have appropriate expectations of a mentoring 

relationship.

Strategy 1Bii. Youth learn how to identify informal mentors.

Strategy 1Biii. Youth receive support from program staff in order to maximize the impact of informal mentors.

Feature 2. Individualized Goal Supports

Concept 2A. Programs assess youth’s needs and strengths.

Strategy 2Ai. Program staff leverage internal and external resources to meet youth’s needs, such as healthcare, 

housing, and transportation.

Strategy 2Aii. Program staff and youth engage in a visioning process about current and future goals that is 

strengths-based.

Strategy 2Aiii. Youth learn to identify their strengths, potential support networks, and resources.

Concept 2B. Youth develop an individualized education and work plan in achieving their goals with assistance from program 

staff.

Strategy 2Bi. Youth learn to set attainable and measurable goals within the plan that have short-term, 

intermediate, and long-term milestones.

Strategy 2Bii. Youth and program staff identify their roles and responsibilities within the plan.

Strategy 2Biii. Youth learn to identify and address barriers, including modifying their plan if necessary.

Concept 2C. Programs support youth in tracking progress, recognizing and learning from setbacks, and reaching their goals.

Strategy 2Ci. Program staff motivate and encourage youth to attain short-term, intermediate and long-term 

milestones.

Strategy 2Cii. Program staff assist youth in revising and expanding goals when appropriate.

Strategy 2Ciii. Program staff provide informal and formal assistance, including helping youth reflect on the overall 

goal-setting process and address barriers to achieving goals.

Strategy 2Civ. Program staff and youth celebrate successes in attaining milestones.

Positive Disruption: Opportunity Reboot Model 33
Feature 3. Coordinated Career Pathways Supports

Concept 3A. Programs create opportunities for career exploration that are grounded in connecting youth with employers in 

high growth-high demand local industries.

Strategy 3Ai. Program staff facilitate exploration of career interest inventories and labor market information 

related to high growth-high demand occupations.

Strategy 3Aii. Program staff create linkages with specific employers in high demand industries to deepen youth 

participants’ understanding of career options including hands-on work experiences with employers.

Strategy 3Aiii. Program staff arrange opportunities for youth to participate in on-site secondary, post-secondary 

and/or on-the-job training leading to industry/employer recognized credentials.

Concept 3B. Programs assist youth with career preparation and planning.

Strategy 3Bi. Program staff engage youth in occupational aptitude and basic skill assessments to develop and 

implement an agreed upon education and work plan that aligns with their career goals.

Strategy 3Bii. Program staff provide youth with programming aimed at improving financial literacy and money 

management.

Strategy 3Biii. Program staff prepare youth to successfully navigate a job search, application process, interviews, 

and other processes associated with their identified career path.

Concept 3C. Programs provide culturally appropriate wraparound services that include multiple support structures pre- and 

post-employment to promote retention and encourage continuing education.

Strategy 3Ci. Program staff provide training aimed at promoting critical job retention and academic success, 

including social-emotional competencies, work readiness and life skills.

Strategy 3Cii. Program staff have regular contact with youth to discuss barriers to maintaining employment and 

academic success and identify specific resources to address those barriers.

Feature 4. Impactful Cross-Sector Partnerships

Concept 4A. Programs regularly interact with diverse stakeholders to implement strategies and increase capacity to serve 

youth. 

Strategy 4Ai. Cross-sector partners are identified and establish a clear vision and goals for collaboration.

Strategy 4Aii. Cross-sector partners agree on specific roles and responsibilities as it relates to serving youth.

Strategy 4Aiii. Cross-sector partners maintain and strengthen collaboration through regular interactions and share 

successes and lessons learned.

Table 4 Continued Opportunity Reboot Model Features, Concepts, and 

Strategies

Positive Disruption: Opportunity Reboot Model 34
The focused conversations about the Opportunity Reboot model and existing programming that 

occurred during the Design and Planning activities helped identify growth areas where 

additional technical assistance and support is needed for community partners to fully integrate 

and realize the potential of the Opportunity Reboot model. The additional technical assistance 

needs that were generated, and a description of how each was met are described below.

Identifying Model-Focused 

Technical Assistance Needs

Interest In Specific Mentoring-Focused Content in Trainings

Community partners requested resources that provided specific guidance on: (a) 

how to mentor older youth, 18-24 years of age; (b) youth-initiated mentoring; 

and, (c) how to support mentees in ways that are developmentally responsive 

and that progress over the tenure of a multi-year deep and sustained mentoring 

relationship (i.e., bring mentorship to the “next level”). While the core of the 

training remained consistent across partners, MENTOR Minnesota worked 

closely with each community partner to adapt some portion of the mentoring 

training content to meet specific site-level needs. 

Offer Mentoring Training for Employers

Staff shared that employers sometimes know little about the realities and 

challenges opportunity youth face, and how to give critical, yet supportive 

feedback. Sites shared that potential employers are sometimes hesitant to 

accept opportunity youth as staff or interns because they: (a) fear these young 

people will be unable to meet their expectations for the positions, and (b) feel 

unable to provide the supervision necessary to train youth so they can be 

successful. Additionally, some employers have a set of cultural, often biased, 

raced, classed, gendered, abled expectations about opportunity youth that 

create barriers for the youth to succeed. In response to this identified need, 

MENTOR Minnesota developed a mentoring training for employers. Four of the 

community partners hosted these trainings for local employers, and the response 

was positive. As another large contribution of this study, MENTOR Minnesota 

now makes this training available to communities across the state.

Create Opportunities for Community Partners to Share Tools and 

Resources

The community partners were eager to learn about the resources, strategies, and 

tools used by other organizations. Partners expressed interest in learning about 

visioning and goal-setting processes and programs used by other partners. They 

also wanted shared online resources through which they could share career 

mapping tools and assessments, educational support tools, and individualized 

career support tools. As part of their multi-pronged system of support, 

Youthprise designed each of the quarterly in-person convenings to include 

dedicated time for partners to share resources focused on a different element of 

the Opportunity Reboot model. An online repository was also created to 

facilitate cross-partner resource sharing.

Positive Disruption: Opportunity Reboot Model 35
The Design and Planning activities also highlighted key considerations related to the evaluation 

approach and design. It became clear early in these conversations that although the projected 

numbers of young people served by each community partner was sufficient, on paper, for an 

evaluation to be conducted at each site, attrition and apprehension from the young people would 

lead to reductions in eligible research participants, and thus pose significant power challenges for 

all quantitative analyses. Our partners highlighted their struggle with programmatic attrition given 

the transient nature of the youth populations served and the general instability in their lives due to 

mental health, chemical dependency, housing insecurity, and system-related issues, thus not 

guaranteeing their presence at two defined time points for a baseline and end line survey 

assessment. Further, staff foreshadowed that programmatic attrition would likely be compounded 

in an evaluation, as research is seen as a historically oppressive activity for many opportunity youth, 

thus leading them to either abstain from research activities altogether or neglect to provide critical 

information. Given these barriers and their potential impact on powering the analyses, our 

community partners collectively advocated for — and, we subsequently shifted to, with support 

from CNCS — a unified evaluation design in which the data was aggregated across sites (rather than 

a multi-site approach). 

Opportunity youths’ apprehension about research and researchers served as a critical insight for 

this project; one that led to significant shifts in how the evaluation team engaged community 

partners and their program participants in testing the survey items, the approaches used to collect 

data that focused foremost on building trust with program participants, and the addition of the 

qualitative impact narratives to the evaluation design. 

Involving opportunity youth and staff to build trust. The need to test survey items in advance of 

administering a survey is standard practice when working with new items and a new population of 

participants. In the Design and Planning workshops and focus groups, program staff highlighted the 

challenges that surveys can pose when used with opportunity youth. This included concerns about 

participants’ literacy and comprehension abilities, their willingness to trust the source and use of 

the resulting data, and the need to ensure items were interpreted by opportunity youth as they 

were intended. 

Sharing these concerns, time was built into the evaluation design to ensure staff and a subset of 

program participants could thoroughly review all survey content. Ultimately, these reviews helped 

the evaluation team put forward more robust and clear survey items.

For example, it was noted that some youth do not always understand who is and is not a staff 

person at a program site. By using the language the program staff and participants understand (and, 

in some cases, specific names of individuals), the unique design of the site-specific surveys could 

address that particular challenge. These reviews also helped identify and refine items that could be 

traumatizing for some youth. Feedback from staff was used to develop an administration guide that 

was prepared to support program staff with data collection. 

Informing the Evaluation Design

Positive Disruption: Opportunity Reboot Model 36
Program staff also discussed at length issues of trust, or the potential lack of trust, in having 

people external to the community partner organization leading data collection activities. There 

was shared agreement that trust was critical to both collecting high-quality data and the ethical 

conduct of applied research. Program staff highlighted the need for any individual coming into 

their space to build rapport with program participants, to be a consistent presence across data 

collection activities, and the importance of having known program staff introduce evaluation 

staff to reinforce that the evaluator was an invited, and

trusted, guest. The Search Institute evaluation team worked 

closely with each community partner to identify and 

implement rapport-building strategies with participants. For 

baseline and end line data collection, program staff were 

asked to administer the survey to ensure youth buy-in into 

the process. In addition to the survey administration guide, 

program staff received remote coaching by the evaluation 

team on survey administration.

Early conversations with program staff and participants 

showcased the importance and value of being able to pair 

quantitative and qualitative data when sharing opportunity 

youth’s experiences in programs using the Opportunity 

Reboot model. Thus, the evaluation design was expanded to 

include a series of qualitative impact narratives to 

complement the youth survey. These qualitative data provide 

a level of rich, thick

description and nuance that quantitative data often lack. For example, achieving stability for some 

opportunity youth meant that they were getting off the streets and on public assistance, even 

though society may not deem that as the ideal and it may not be fully reflected as success in the 

intermediate outcomes of the Opportunity Reboot model.

In addition to informing a series of responsive evaluation modifications, the Design and Planning 

activities helped the community partners cultivate an evaluative mindset and built buy-in for the 

evaluation as a rigorous process intended to elevate the Opportunity Reboot model’s level of 

evidence. 

Positive Disruption: Opportunity Reboot Model 37
Four instruments were used to collect data during the implementation evaluation: (a) the 

Opportunity Reboot Mapping Tool; (b) staff-reported questionnaire on program outputs, (c) 

staff-reported implementation factors and youth outputs; and, Opportunity Reboot youth 

survey.

Opportunity Reboot Mapping Tool

Tracking fidelity to the Opportunity Reboot model presented an interesting challenge as the 

model was intentionally designed: (a) to be integrated into an existing program, and (b) to give 

partners flexibility in selecting site-specific tactics to realize the core model concepts. Moreover, 

most of the strategies reflect micro-actions meant to be seamlessly integrated into the day-today interactions that occur between youth and program staff making them difficult to 

systematically track quantitatively. In addition, if honest, candid input was desired and in order 

to support full integration of the Opportunity Reboot model, the Design and Planning 

workshops showed the critical need to design a fidelity metric that was seen and experienced 

by community partner leadership and staff as a supportive technical assistance tool — and not 

as a strict evaluation assessment. For these reasons, a longitudinal, mixed methods approach to 

measuring fidelity was utilized that aimed at understanding how the Opportunity Reboot model 

was being implemented by each of the community partners. This approach triangulates data 

from community partners, Youthprise, and the evaluation team in assessing an organization’s 

alignment to the Opportunity Reboot model. 

The implementation evaluation was meant to capture the nuances in programmatic tactics 

employed to account for contextual differences among the partners, including population 

factors such as nativity, housing status, English language proficiency; available career pathways; 

region; and, organizational type including school- or community-based settings. This approach, 

which aligned with the fidelity measurement plan outlined in the approved SIF Evaluation Plan, 

was referred to as the Opportunity Reboot Mapping Tool (see Appendix B). The Mapping Tool 

was used at three time points during a 15-month window: approximately March 2017 (Initial, 

Time 1), December 2017 (Time 2), and May 2018 (Final, Time 3). 

Overview of the Tool

The Opportunity Reboot Mapping Tool is organized around the 26 strategies named in the 

Opportunity Reboot model (see Table 4). For each strategy, partners were asked to reflect on a 

series of questions. 

§ First, they were asked, “What are you doing in your program to implement this strategy?” 

Here partners would list the specific activities or tactics being used by their program.

§ Next, they were asked “How would you describe your program’s current alignment with this 

strategy?” Partners responded to this question using a quantitative 4-point alignment scale: 0 

= Not Aligned; 1 = Minimally Aligned; 2 = Mostly Aligned; 3 = Strongly Aligned. If a partner 

rated a strategy less than strongly aligned, they were asked a series of follow-up questions: 

“What plans, if any, do you have to more fully implement this strategy? By when? What 

resources or assistance do you need, if any, from Youthprise to do this? How aligned will the 

program be with this strategy once the aforementioned plans are implemented?” 

§ Lastly, for each model feature, partners were asked: “What additional activities in your 

program support this feature that are beyond what is captured in the Opportunity Reboot 

strategies?”

Implementation Evaluation Instruments

Positive Disruption: Opportunity Reboot Model 38
While responses to the open-ended questions were formally recorded on the Opportunity 

Reboot Mapping Tool, they were not thematically coded, as their purpose was not evaluative 

but rather to structure the conversation, provide insight on additional technical assistance 

needs, and provide context for the quantitative assessments.

Recruitment

Decisions about which program leaders and staff would participate in the Opportunity Reboot 

Mapping Tool conversations were made collaboratively by Youthprise and the lead contact at 

each community partner site. The goal was to recruit program leaders and staff who had both a 

broad and a deep understanding of the organization-specific tactics being employed to 

operationalize the features of the Opportunity Reboot model and who would be available across 

all waves of data collection. Recruitment to participate in the Mapping Tool discussions was easy 

as community partners were eager to talk about the innovative ways they were bringing the 

Opportunity Reboot model to life, and to problem-solve with Youthprise areas of the model that 

needed to be strengthened. 

Time 1 and Time 2 Administration

To bolster the validity of this fidelity approach, the first two administrations were conducted 

jointly by the evaluation team, program leadership and staff, and Youthprise. As part of these 

conversations, the evaluation team and Youthprise probed qualitatively for specific evidence 

and information about the activities (i.e., tactics) being used to implement each strategy. If 

either the evaluation or Youthprise team member disagreed with a community partner’s 

quantitative assessment on the Mapping Tool, the specific score was discussed until consensus 

was reached (this was rare, and in all cases ended with the recorded score reflecting the 

assessment made by the evaluation or Youthprise team member). The same Youthprise staff 

member was present at all administrations of the Mapping Tool (across partners and over time) 

helping to ensure consistency in the assessments and providing expertise on the model 

strategies and concepts. 

At Time 1 and Time 2, an average of four staff participated at each community partner site, 

including at least one member of the organization’s senior leadership team (e.g., executive 

director, program director, program manager, assistant principal, school administrator) and one 

staff engaged in direct service with the opportunity youth served by the program (e.g., 

employment counselor, teacher, wraparound services team, case manager). While some 

community partners decided to include a larger group of staff for the Time 1 assessment, a core 

group of those staff remained consistent across all administrations, outside of a couple of staff 

who left their respective organizations during the data collection period. A total of 28 

community partner leaders and staff participated at Time 1 and 22 participated at Time 2.

Time 3 Administration

Time 3 administration. For the final administration (Time 3), the Mapping Tool was again 

completed by program staff, an evaluation team member, and Youthprise. However, at this time 

point the tool was completed separately by each reporter in order to independently assess and 

quantify fidelity to the Opportunity Reboot model. Each reporter was blind to the assessments 

of the other reporters; allowing potential discrepancies across reporters to be identified. 

Upon completion, the independent fidelity scores assigned by the evaluation team and 

Youthprise were compared first, before looking at the community partners’ scores. 

Positive Disruption: Opportunity Reboot Model 39
The results show 91% perfect agreement at the strategy level across partners between the two 

reporters; that is, the evaluation team and Youthprise were in perfect agreement on 142 out of 

156 (26 strategies x 6 partners) strategy-level scores. Further broken down by feature: 

§ Feature 1 – Positive Mentoring Relationships: 77% perfect agreement, 23 of 30 scores;

§ Feature 2 – Individualized Goal Supports: 98% perfect agreement, 59 of 60 scores;

§ Feature 3 – Coordinated Career Pathways Supports: 91% perfect agreement, 46 of 48 scores; 

and, 

§ Feature 4 – Impactful Cross-Sector Partnerships: 78% perfect agreement, 14 of 18 scores.

The evaluation team and Youthprise discussed discrepant scores to reach consensus on a final 

fidelity score. This consensus score was deemed the final Time 3 fidelity assessment.

Next, this final consensus fidelity score was compared to each community partner’s final scores 

on the Mapping Tool. The results revealed 94% perfect alignment (147 of 156 scores) at the 

strategy level across partners between reporters. Table 5 describes the 9 discrepant fidelity 

scores; for purposes of brevity the aligned scores were excluded. A majority of the discrepancies 

were due to Youthprise assessing stronger alignment than the GAP program team did in five 

strategies (i.e., S 1Bi-iii, S 2Ciii, S 3Bi). An additional four discrepancies were due to Youthprise 

assessing weaker alignment than three community partners (i.e., Compass, GAP, MIGIZI) on 

strategies in the Impactful Cross-Sector Partnerships feature. An examination of fidelity by 

partner shows strong agreement in assessments across partners: 92% with Compass (24 of 26 

strategies perfectly aligned), 77% with GAP (20 of 26), 96% with MIGIZI (25 of 26), 100% with 

OYOD, Prior Crossing, and Tri-City Bridges (26 of 26). 

Staff-Reported Questionnaire on Program Outputs

Working with community partners, Youthprise set quantitative targets for outputs tied to each 

of the four features of the Opportunity Reboot model (see Figure 2). These outputs are the 

direct products of the model-specified activities. A questionnaire was completed by a 

designated staff member at each of the community partner sites in order to collect data on two 

outputs: (1) the percentage of direct service staff who have participated in the Maximize Your 

Impact training, and (2) the number of new and beneficial collaborations that have been 

established to support opportunity youth locally. The staff member charged with reporting 

these data was typically a program leader, such as the executive director or program director. 

Staff completed these assessments retrospectively in Summer 2018. 

Partners reported on the number of direct service program staff, employers, and/or other 

supportive adults who attended the Maximize Your Impact training made available by MENTOR 

Minnesota as part of the Opportunity Reboot model’s emphasis on supporting the intentional 

and systematic integration of the model features. This training focuses on developing a 

“mentoring mindset” in everyday life. Focusing on some of the fundamentals of positive youth 

development, participants learn how to be strengths-based, youth-centered, help young people 

overcome challenges, and identify and nurture individual goals. Attendance logs that were 

systematically tracked, and verified, by MENTOR Minnesota were used by staff to report on 

training participation

In addition to reporting on participation in these mentoring trainings, staff also reported on the 

number of new partnerships their organization formed as part of their efforts to integrate the 

Opportunity Reboot model. Staff reported these data by listing each of the newly formed 

partnerships.

Positive Disruption: Opportunity Reboot Model 40
Table 5 Discrepancies in Opportunity Reboot Mapping Tool Final (Time 

3) Fidelity Assessments

Program Strategy

Time 3 Opportunity Reboot

Mapping Tool Alignment Assessments

Community 

Partner

Consensus 

Final Reason for Adjustment

Compass S 4Ai. Cross-sector partners are 

identified and establish a clear vision 

and goals for collaboration.

Strongly Mostly Partnerships are transactional, 

less of a cohesive strategic 

network with aligned goals.

S 4Aii. Cross-sector partners agree on 

specific roles and responsibilities as it 

relates to serving youth.

Strongly Mostly

GAP S 1Bi. Youth understand their rights, 

responsibilities and have appropriate 

expectations of a mentoring 

relationship.

Mostly Strongly Mentoring trainings were 

provided to program participants 

and staff, which was the core 

activity for these strategies of the 

model. GAP is going beyond 

connecting participants to others 

by emphasizing a more formalized 

mentoring role for these 

relationships.

S 1Bii. Youth learn how to identify 

informal mentors.

Mostly Strongly

S 1Biii. Youth receive support from 

program staff in order to maximize the 

impact of informal mentors.

Mostly Strongly

S 2Ciii. Program staff provide informal 

and formal assistance, including 

helping youth reflect on the overall 

goal-setting process and address 

barriers to achieving goals.

Mostly Strongly This process is implemented more 

informally than formally at GAP, 

but is still effective in helping 

participants reflect on their 

progress toward individualized 

goals.

S 3Bi. Program staff engage youth in 

occupational aptitude and basic skill 

assessments to develop and 

implement an agreed upon education 

and work plan that aligns with their 

career goals.

Mostly Strongly These assessments are used by 

GAP, albeit somewhat informally. 

These informal assessments still 

track the indicators suggested by 

this strategy, thus alignment was 

increased.

S 4Aiii. Cross-sector partners maintain 

and strengthen collaboration through 

regular interactions and share 

successes and lessons learned.

Strongly Mostly Partnerships do not have a 

formalized structure in which they 

collaborate.

MIGIZI S 4Aii. Cross-sector partners agree on 

specific roles and responsibilities as it 

relates to serving youth.

Strongly Mostly Partnerships did not develop a 

plan of aligned goals.

Positive Disruption: Opportunity Reboot Model 41
Staff-Reported Implementation 

Factors and Youth Focused Outputs

Designated staff members at each of the community partner sites were asked to complete a 

brief survey on each of the young people who participated in either the baseline (Time 1) or 

endline (Time 2) Opportunity Reboot Youth Survey (see the Preliminary Impact section for more 

detail on this survey). Program leadership assigned staff to report on youth with whom they had 

direct experience working with in the program. Seven pieces of data were gathered via these 

surveys: two youth-level implementation factors, and five youth outputs named in the 

Opportunity Reboot logic model (see Figure 2), including the Maximize Your Opportunity 

training provided by MENTOR Minnesota. Maximize Your Opportunity prepares youth to build a 

relationship with a mentor and make the most of their mentoring experience. It uses engaging 

activities to explore the roles and boundaries of a mentor, set realistic expectations, practice 

communication skills, set personal goals, and learn how to engage with natural/informal 

mentors throughout life. 

As described below in Research Question 2, the evaluation team worked closely with each 

community partner to operationalize the two youth-level implementation factors (i.e., dosage 

and engagement). Using the site-specific operational definitions, staff provided a program 

dosage and participant engagement designation for each young person. It was also determined 

during the Design and Planning activities, that program staff were best positioned to provide 

data on five of the youth-focused outputs. Thus, for each participant, staff also indicated 

whether the young person had: (a) participated in the Maximize Your Opportunity mentoring 

training (drawing on attendance logs tracked and verified by MENTOR Minnesota); (b) set goals 

and, if so, whether they had created a formal plan to reach these goals; (c) demonstrated 

marked growth towards the individual goals they had established with the program; (d) enrolled 

in a credential program, training, or coursework aimed at preparing them for a viable career 

path; (e) participated in an apprenticeship, internships, or other paid or unpaid work 

experience.

Program staff completed these youth-specific surveys in Spring 2018. The online survey was 

designed to be accessed by multiple staff, typically case managers, who had access to case 

notes, program databases, and knowledge of the outputs named above. Most staff completed 

the survey either concurrently with the endline (i.e., Time 2 youth survey used in the 

preliminary impact evaluation) or retrospectively at a single time point for participants who 

were no longer in the program. Some community partners elected to add detail on a rolling 

basis, updating fields as information changed or progress was made by participants. The survey 

platform was designed to reflect the program’s roster, allowing staff to select a participant 

based on their participant ID. The evaluation team frequently corresponded with staff to 

respond to questions and gauge their current status, paying close attention to youth who had 

participated in the youth survey at both baseline and endline. Eighty-six percent of the 194 

youth surveys that were matched between baseline and endline had corresponding staff 

surveys. 

Positive Disruption: Opportunity Reboot Model 42
Opportunity Reboot Youth Survey

The Opportunity Reboot Youth Survey was primarily used for the preliminary impact study with 

the exception of a few items tied to the implementation evaluation about program outputs and 

program quality and satisfaction. A description of these implementation-focused items is 

provided below. For a full description of the sample, data collection, and development of the 

Opportunity Reboot Youth Survey, please refer to the Preliminary Impact section of this report. 

Youth were asked to report on two outputs named in the Opportunity Reboot logic model at 

endline: (a) whether they can identify 2-3 program staff who support their growth; and (b) 

whether they can identify at least one new adult outside of the program who supports their 

growth. Responses were scored on a 5-point agreement scale ranging from Strongly Disagree to 

Strongly Agree. An output was considered achieved for an individual youth if they indicated they 

agreed or strongly agreed with each statement.

In addition to knowing how participants experienced the specific features of the Opportunity 

Reboot model, community partners were eager to better understand how youth participants 

more generally experienced the quality of their programming. The endline youth survey was 

extended to include nine questions assessing how youth experienced various aspects of their 

program — e.g., having an interest in program activities, feeling safe, feeling like their needs 

were met. All items were assessed on a 5-point agreement scale ranging from Strongly Disagree 

to Strongly Agree. An indicator of quality was considered achieved for an individual youth if they 

indicated they agreed or strongly agreed with each statement.

Positive Disruption: Opportunity Reboot Model 43
Research Question 1

Was the Opportunity Reboot model implemented with 

fidelity?

Tables 6 and 7 provide graphic summaries of the data collected with the Opportunity Reboot 

Mapping Tool by site aggregated at the feature level (Table 6) and overall (Table 7) at Time 1 (initial) 

and Time 3 (final). These tables collectively illustrate how fidelity to the Opportunity Reboot model 

was strengthened over time. 

Overall, all community partners made significant progress in their alignment to the Opportunity 

Reboot model between the initial and the final administration of the Mapping Tool: There was a 

notable shift from fewer “not or minimally aligned” scores in the initial assessment to (nearly) all 

“mostly” or “strongly” aligned scores in the final assessment. This progress was notable in all four 

features of the model and for all community partners. 

The only exception was the Positive Mentoring Relationships feature for Opportunity Youth of 

Duluth (OYOD). At the final assessment, 60% of the strategies in this feature for this partner were 

still “not or minimally” aligned (however, this is down from 100% at the initial assessment). As 

discussed below, this designation was largely due to program leadership’s resistance to providing 

their participants with formalized mentoring training. Program leadership believe that a structured 

mentoring training delivered by an external training – with whom they have no relationship would 

not be well received by their population of participants. This perception stemmed from their 

experiences with other formalized training and classroom-type settings. Program leadership 

believed that this content would not resonate with their participants in this form, especially when 

they were more concerned, at that time, with meeting participants’ housing and food needs. 

All of the other model features were mostly or strongly aligned across the six community partners. 

Given the role of shared technical assistance as well as the formation of a collective learning 

community, much of the program-specific actions tended to look more similar at the final 

administration of the Mapping Tool, when compared to the first administration. This was an added 

benefit of highlighting and discussing the exemplary features of each community partner at the 

cohort’s convenings. 

Appendix C provides a detailed graphic summary of the data collected by partner and by strategy at 

Time 1 and Time 3. The following is a narrative analysis of how alignment to the Opportunity 

Reboot model shifted over the course of the project.

Positive Disruption: Opportunity Reboot Model 44

IMPLEMENTATION EVALUATION 

FINDINGS BY RESEARCH QUESTION

Four research questions — answered using the data from the instruments described above — guided 

the implementation evaluation. The following section presents each of the research questions, in turn, 

and summarizes the core findings.
Feature 1. Positive Mentoring 

Relationships

Feature 2. Individualized Goal 

Supports

Not or 

Minimally 

Aligned

Mostly 

Aligned

Strongly 

Aligned

Not or 

Minimally 

Aligned

Mostly 

Aligned

Strongly 

Aligned

Compass

Initial 0% 20% 80% 0% 0% 100%

Final 0% 0% 100% 0% 0% 100%

GAP

Initial 80% 20% 0% 50% 0% 50%

Final 0% 60% 40% 0% 10% 90%

MIGIZI

Initial 20% 60% 20% 10% 20% 70%

Final 0% 0% 100% 0% 20% 80%

OYOD

Initial 100% 0% 0% 20% 10% 70%

Final 60% 40% 0% 0% 10% 90%

Prior 

Crossing

Initial 100% 0% 0% 0% 60% 40%

Final 0% 60% 40% 0% 30% 70%

Tri-City 

Bridges

Initial 60% 40% 0% 0% 20% 80%

Final 0% 60% 40% 0% 0% 100%

Table 6 Summary of Model Alignment Assessments: Percent of 

Alignment by Feature

Positive Disruption: Opportunity Reboot Model 45

Feature 3. Coordinated Career 

Pathways Support

Feature 4. Impactful Cross-Sector 

Partnerships

Not or 

Minimally 

Aligned

Mostly 

Aligned

Strongly 

Aligned

Not or 

Minimally 

Aligned

Mostly 

Aligned

Strongly 

Aligned

Compass

Initial 63% 13% 25% 67% 33% 0%

Final 0% 0% 100% 0% 100% 0%

GAP

Initial 25% 13% 63% 67% 33% 0%

Final 0% 0% 100% 0% 100% 0%

MIGIZI

Initial 25% 50% 25% 33% 67% 0%

Final 0% 0% 100% 0% 67% 33%

OYOD

Initial 0% 25% 75% 33% 33% 33%

Final 0% 0% 100% 0% 0% 100%

Prior 

Crossing

Initial 75% 25% 0% 67% 0% 33%

Final 0% 75% 25% 0% 0% 100%

Tri-City 

Bridges

Initial 0% 25% 75% 0% 33% 67%

Final 0% 0% 100% 0% 0% 100%
Not or Minimally 

Aligned

Mostly 

Aligned

Strongly 

Aligned

Compass

Initial 63% 13% 25%

Final 0% 0% 100%

GAP

Initial 25% 13% 63%

Final 0% 0% 100%

MIGIZI

Initial 25% 50% 25%

Final 0% 0% 100%

OYOD

Initial 0% 25% 75%

Final 0% 0% 100%

Prior Crossing

Initial 75% 25% 0%

Final 0% 75% 25%

Tri-City Bridges

Initial 0% 25% 75%

Final 0% 0% 100%

Table 7 Summary of Overall Model Alignment

Positive Disruption: Opportunity Reboot Model 46
Positive Disruption: Opportunity Reboot Model 47

Positive mentoring relationships were largely outside of the purview of community partners prior to 

the launch of Opportunity Reboot (as indicated by the low level of alignment with the mentoringfocused concepts and strategies). In completing the first Opportunity Reboot Mapping Tool, the 

partners had — promisingly — already begun identifying and exploring ways to get more intentional 

about integrating mentoring into their program models. These efforts were further reinforced by the 

mentoring-focused technical assistance provided by MENTOR Minnesota and Youthprise. 

At the time of the final mapping tool, most programs (n = 5) were primarily aligned with this feature 

of the Opportunity Reboot model. At the concept-level, all community partners were, at minimum, 

mostly aligned with Concept 1A and all but one community partner was aligned with Concept 1B. 

Opportunity Youth of Duluth was the only partner that reported, with Youthprise agreement, that they 

were minimally aligned with Concept 1B (Strategies 1Bi, 1Bii, and 1Biii) and not preparing program 

participants for mentoring experiences and helping them develop skills to identify and engage informal 

mentors. As described above, Opportunity Youth of Duluth staff felt their program participants would 

not participate in any kind of formalized training or workshop focused on this content. 

Feature 1

Positive Mentoring Relationships

While they agree that youth need support in making 

healthy decisions related to informal relationships, this 

largely happened in an unstructured manner through 

regular programming and services. Two other community 

partners (Tri-City Bridges and Prior Crossing) were 

assigned scores of mostly aligned for all strategies within 

Concept 1B. These strategies reflected the current state of 

training at those particular sites, given that not all of their 

youth had participated in the Maximize Your Opportunity 

Training.

As a result of the technical assistance provided by core 

partners, community partners revised their case 

management approach to include more access to staff 

and new relationship-building practices, such as 

motivational interviewing, trauma-informed practices, 

restorative justice practices, culturally-responsive 

practices, and elements of the Developmental 

Relationships Framework. A few of the community 

partners employed strategies that included expanding the 

numbers and types of mentors they had available to 

participants to include internal staff beyond a participant’s

case manager and, in some instances, included external employers or other 

community partners. Included in this expansion were additional systematic support

structures for the adults who served in these mentoring roles. This included more 

check-ins with the community partners and reflection activities on their mentoring relationship. 

Much of the positive mentoring relationships feature, in practice, was built on the concept that young 

people benefit from understanding the roles of mentors, developing skills to identify formal and informal 

mentors, and reflecting on their mentoring relationships. These were critical elements of the workshops 

that all community partners participated in (Maximize Your Impact and Maximize Your Opportunity). 

Although, as noted above in the Opportunity Youth of Duluth example, how broadly these workshops 

were incorporated into each community partner varied based on their own level of understanding of 

mentoring practices as well as the unique needs of their populations.
Of the four Opportunity Reboot model features, Individualized Goal Supports had the 

strongest alignment across all six community partners; all of the community partners were 

aligned with this feature of the Opportunity Reboot model over the tenure of the project. This 

included strong alignment at the more nuanced concept- and strategy-levels. Only three 

strategies received scores of less than strong alignment across all community partners 

(Strategies 2Aii, 2Biii, 3Cii). These were largely due to their goal-reflection activities being less 

formalized outside of critical programmatic milestones such as enrollment (as in the case of 

MIGIZI) or differing across career pathways (as in the case of GAP). The lower alignment score 

did not suggest that these activities did not occur, only that they hoped to more systematically 

incorporate reflection activities in the case management process outside of when barriers arose 

that needed to be addressed.

Overall, partners were confident with their established goal setting processes. Drawing on 

highly-relational approaches, program staff work closely with opportunity youth to establish 

open lines of communication and identify, monitor, and adjust goals that are responsive to 

youths’ individual needs. Goal-setting strategies employed across community partners included 

motivational interviewing, creating SMART goals, and facilitating visioning processes. In addition 

to helping participants set goals, a key part of supporting youth with goals involves staff helping 

youth navigate complex educational, financial, health, and legal systems. In some instances, this 

required working with other organizations in the community to meet those goals before a 

participant could fully engage with the program. Some examples of these basic priorities 

included meeting housing needs, addressing mental health concerns, and increasing English 

literacy. 

Program staff acknowledged that youths’ goals often change as they learn more about the 

requirements and industry standards for pursuing specific personal and career interests. 

Motivating, encouraging, and supporting young people to achieve their goals was a 

particularly strong section in the model with moderate or strong alignment from the 

beginning of the project. Working with youth to identify resources and draw on their strengths 

to set and reach goals is the foundation of the partners’ programs, and they — appropriately —

claimed this work proudly. Case management was the method most sites employ to establish, 

communicate, and process goals with opportunity youth. Youth are expected to learn 

professional and healthy relationship boundaries and the skills to set goals through the process 

of working with program staff. 

Program staff leveraged community connections to help youth figure out what they want to do. 

Most programs exposed youth to different career trajectories through guest speakers, field trips, 

and conversations about what professions youth want to explore. Sites used various tactics for 

setting short- and long-term goals, all of which included processes for breaking large goals into 

manageable pieces.

Feature 2

Individualized Goal Supports

Positive Disruption: Opportunity Reboot Model 48
The community partners saw significant growth in their coordinated career pathways supports 

through their integration of the Opportunity Reboot model. This was largely due to 

strengthening their intentionality in providing resources and partnerships that can support this 

work, along with additional technical assistance from Youthprise and the Minnesota 

Department of Employment and Economic Development. The initial Opportunity Reboot 

Mapping Tool showed that only two of the partners were “mostly aligned” with this model 

feature. By the end, all six partners had achieved full alignment. This alignment existed at the 

concept-level and in all but one of the strategies (S 3Aiii). Prior Crossing achieved a mostly 

aligned score for this particular strategy due to the need to scale up their existing credentialing 

and certification opportunities. Many opportunities existed, but long-term plans for this partner 

included additional options that would meet the diverse occupational interests of their 

participants.

Using a pathways approach to support opportunity youths’ career development serves as an 

effective process not only for introducing youth to a job and formal employment, but also 

creating opportunities for youth to explore career trajectories and align themselves with a job 

that is a good fit for their skills and interests. Partners use a wide range of tactics to support 

experiential learning, such as site visits to local colleges and universities, guest speakers from 

various industries, short-term job training such as “ride alongs” and CDL (Commercial Driver’s 

License), and workplace tours. 

At its core, this feature included similar activities across all community partners but some of 

these varied out of necessity due to the different types of credentialing and certification 

pathways offered (e.g., nursing, information technology, hospitality, green jobs, and others). 

Consistent practices across program partners included the administration of career interest 

inventories and aptitude tests, incorporating financial literacy into curricula, conducting mock 

interviews, and supporting the development of application materials (such as resumes and 

cover letters). 

The core activities of each community partner involved building specific skills related to the 

respective pathways they offered as well as supporting the attainment of outputs related to 

certifications and credentialing (such as internship hours and examinations). In addition to these 

activities, other critical tactics included forming and sustaining employment and educational 

partnerships that provided opportunities in relevant industries for their participants. Building 

and maintaining these organizational relationships were viewed as critical to creating and 

sustaining their participants’ pathways to these careers. To facilitate this, community partners 

coached employers and participants on how to build, strengthen, and maintain professional 

relationships. Partners also helped program participants build social capital with potential local 

employers and higher education entities by organizing introductions, meetings, and job fairs, 

and, when necessary, served in a mediation capacity with those entities when issues arose 

around a participant’s performance, including behavior, etiquette, punctuality, or attendance.

Feature 3

Coordinated Career Pathways Supports

Positive Disruption: Opportunity Reboot Model 49
Each of the community partners relied heavily on people and organizations outside of their 

programs to fully implement all model features and to achieve Opportunity Reboot goals that 

they could not achieve on their own. These partnerships range from informal to more structured 

partnerships; and from something more like a vendor relationship to true collaboration and 

integration. The most common partnerships included employers, education institutions, 

governmental agencies, and other community organizations that addressed basic needs. At the 

time the initial Opportunity Reboot Mapping Tool was completed, most partners were actively 

working to expand and deepen their network of partnerships, through outreach and ongoing 

relationship management. By the final administration of the tool, all partners had made 

progress across the three named strategies in this feature and three partners (Tri-City Bridges, 

Prior Crossing, and Opportunity Youth of Duluth) achieved strong model alignment across all 

named strategies (S 4Ai, S 4Aii, S 4Aiii). The other three partners (GAP, Compass, and MIGIZI) 

were mostly aligned by the final administration of the tool in all strategies. While these 

community partners assembled an impressive number of impactful partnerships, these 

relationships were not formalized with a Memorandum of Understanding and collective goals 

were not systematically set. In these instances, community partners did not establish a 

formalized collaborative that codified the roles and responsibilities beyond their own individual 

relationships with the network of partnerships. This led to a slight reduction in their fidelity 

designation to “mostly aligned” from most reporters. 

Most of the cross-sector partnerships leveraged by the partners were aimed at providing 

Opportunity Reboot participants with trainings, services, opportunities, and other resources 

that sites were unable to provide on their own. For example, partners offered training for 

English language learners, and on topics like socioemotional learning, financial literacy, sexual 

health, environmental justice, and jobs in the Green economy. These partnerships also create 

opportunities for college visits, scholarships, internship or job placements, and service-learning. 

They provide mental health services, GED support, and financial counselling. Partnerships were 

also forged with the explicit purpose of removing barriers to program participation and success 

(e.g., access to food, bus passes, childcare). In a few cases, partners provide support to program 

staff, rather than to program participants (e.g., around recruiting or evaluation). Sometimes 

partners purchase services from program sites, providing additional income to support site 

organizational goals. Staff of all sites reported connecting with partners to talk about progress 

being made by youth they serve together, and to work together to identify ways to better help 

youth achieve goals. Initiating and nurturing these partnerships was a significant task for staff at 

each community partner. Once partnerships were formed, staff were tasked with managing the 

relationship and setting expectations for collective success. This most frequently occurred 

through in-person formal meetings and informal check-ins with key individuals at each 

partnering organization.

Feature 4

Impactful Cross-sector Partnerships

Positive Disruption: Opportunity Reboot Model 50
The evaluation team worked closely with each community partner to better understand what 

participation and engagement looked like for opportunity youth in the Opportunity Reboot 

partner programs. This led to the development and operationalization of measures of critical 

implementation factors (i.e., program dosage, participant engagement).

Research Question 2

What does participation and engagement in the 

Opportunity Reboot model look like for opportunity 

youth?

In accordance with criteria pre-specified by the partners in consultation with the evaluation 

team (as described in Tables 8 and 9), participants’ program dosage and engagement were 

scored on a 3-point scale (high, medium, and low). Among the overall sample of participants, 

64% were assessed by program staff to have received high dosage, 28% with medium dosage, 

and 8% with low dosage. A little more than half (59%) of the overall sample of participants 

were assessed by program staff to have high levels of program engagement, 30% with 

medium engagement, and 11% with low engagement. Details of how the dosage and 

engagement criteria were developed follow.

The plan to measure how often young people participated (i.e., program dosage) in programs 

using the Opportunity Reboot model and how engaged they were when they showed up (i.e., 

participant engagement) emerged through ongoing dialogue between the evaluation team and 

community partners.

The Design and Planning activities, described above, illuminated the variability in range that 

existed across community partners in program dosage. For example, Compass — the program 

embedded within Sauk Rapids-Rice School District — has a program model that involves daily 

interactions between program staff and students every day school is in session. This is also true 

of MIGIZI. In contrast, dosage is much lower in a program like Prior Crossing where participants 

are typically older and are only required to meet with program staff once a month (although, in 

reality, interactions often occurred much more frequently but fully at the discretion of the 

participant). This contextual variation in programmatic settings and populations served across 

community partners presented the need for a more nuanced metric that quantified multiple 

objective data points for each program participant. These metrics went beyond a single 

assessment of attendance and included explicit thresholds on multiple variables that 

determined whether a participant was to be codified as having high, medium, or low program 

doses. For the full detail on each community partner’s objective and quantifiable dosage 

metrics, taking into account their program context and target beneficiaries, see Table 8.

Data on participants’ level of program dosage were already being routinely collected, 

monitored, and logged by each partner for other reporting purposes; each being reflective of 

their program model and the populations served. To establish a common metric across the 

community partners, each partner was asked to work with the evaluation team to review and 

strengthen existing metrics and subsequently, to quantify what high, medium, and low program 

dosage looked like using those dosage metrics, thereby essentially creating a formula for use in 

determining the designation. 

Dosage and Engagement

Positive Disruption: Opportunity Reboot Model 51
Community Partner High Dosage Medium Dosage Low Dosage

Compass

• Enrollment: All Compass 

classes

• Attendance: <8 absences

• Contacts: Attend a minimum 

of two face to face meetings 

per month and a weekly 

connect

• Event Attendance: Miss <2 

field trips or guest speakers

• Circle Attendance: Misses <2 

Circle activities

• Enrollment: >half of 

Compass classes

• Attendance: Between 8-14 

absences

• Contacts: Attend a face to 

face and connect with a 

mentor >half of the 

maximum meeting times

• Event Attendance: At least 

half of field trips and guest 

speakers

• Circle Attendance: Attend 

>half of Circle activities

• Enrollment: <half of 

Compass classes

• Attendance: >14 absences

• Contacts: Attend a face to 

face and connect with a 

mentor <half of the 

maximum meeting times

• Event Attendance: Attend 

<half of field trips or guest 

speakers

• Circle Attendance: Attend 

<half of the Circle activities

GAP

• Attendance: 91% to 100%

• Volunteer Hours: >300 hours

• Credits: >8 credits

• Attendance: 80% to 90%

• Volunteer Hours: 150-299 

hours

• Credits: 4 to 7.5 credits

• Attendance: <80%

• Volunteer Hours: <150 hours

• Credits: <4 credits

MIGIZI

• Total Hours: >150 hours

• Completed Projects: 75%-

100%

• Total Hours: 100 to 150 

hours

• Completed Projects: 50%-

74%

• Total Hours: <100 hours

• Completed Projects: <50%

OYOD

• Payroll Hours: >150 hours 

(Life House)

• Attendance: 75%-100% 

(SOAR)

• Payroll Hours: 50-149 hours 

(Life House)

• Attendance: 50%-74% 

(SOAR)

• Payroll Hours: <50 hours 

(Life House)

• Attendance: <50% (SOAR)

Prior Crossing

• Case Manager Interaction: 

>3 meetings per month

• Employment Coach 

Interaction: >3 meetings per 

month

• Length of Stay: >12 months

• Case Manager Interaction: 2-

3 meetings per month

• Employment Coach 

Interaction: 2-3 meetings per 

month

• Length of Stay: 4-11 months

• Case Manager Interaction: 0-

1 meetings per month

• Employment Coach 

Interaction: 0-1 meetings per 

month

• Length of Stay: 0-3 months

Tri-City Bridges

• Contacts: >6 contacts with 

staff

• Tenure: >12 weeks

• Contacts: >2 contacts with 

staff

• Tenure: 6-12 weeks

• Contacts: >1 contacts with 

staff

• Tenure: <6 weeks

Table 8 Description of Quantitative Program Dosage Metrics by 

Community Partner

Positive Disruption: Opportunity Reboot Model 52
As an example, GAP used a combined metric of on-site attendance, volunteer hours, and out-ofschool time credits. The dual benefit of this approach is that it allows for variability in dosage 

that is reflective of each partner’s program model while still allowing the aggregation of data 

across partners for the preliminary quantitative impact evaluation analyses.

A similar process was used to determine the partner-specific definitions of what high, medium, 

and low participant engagement looked like. To start, each community partner was asked to 

describe, in writing, the kinds of behaviors they observe and to record in case notes that signify 

when participants are fully engaged and when they are disengaged in their programs. Drawing 

directly from these descriptions, the evaluation team provided partners with partner-specific 

guidance on how to track participant engagement. It was important for these metrics to be 

easily understood and observable by program staff so that case managers (or, their equivalents 

— depending on the program model) would be able to accurately report on their respective 

metrics in their case notes, and subsequently, in the Staff-Reported Questionnaire. As an 

example, Prior Crossing used a combined metric of participants attending volunteer events, 

consistently attending meetings, and frequency in engaging staff. For the full detail on each 

community partner’s engagement metrics, see Table 9.

Program dosage and participant engagement was collected on every program participant who 

completed either the baseline or endline youth survey as part of the preliminary impact 

evaluation. These data were collected via the Staff-Reported Implementation Factors and Youth 

Outcomes survey (see above for additional detail). Designated staff who worked directly with 

the program participants were charged with reporting this data. 

Positive Disruption: Opportunity Reboot Model 53

Youthprise set quantitative targets for 

implementation outputs tied to each of the 

four features of the Opportunity Reboot model 

(see Figure 2). These outputs are the direct 

products of model-specified activities. The 

diverse nature of these outputs required us to 

gather data from various sources (described in 

the Implementation Evaluation Instruments 

section) including the staff-reported 

questionnaire on program outputs (which 

drew on attendance logs tracked and verified 

by MENTOR Minnesota), the staff-reported 

survey on implementation factors and youthfocused outputs, and the Opportunity Reboot 

Youth Survey.

Research Question 3

Did Opportunity Reboot program partners achieve output 

targets?

Table 10 summarizes the targeted and observed levels of each of the outputs for the overall 

sample and, for youth-focused outputs, by age (14-17 year-olds and 18+ year-olds). Half of the 

outputs tied to the Positive Mentoring Relationships feature of the Opportunity Reboot 

model were achieved across community partners. Staff and youth at five partner sites had high 

levels of participation in both of the mentor trainings. 
Community 

Partner

Engagement Data 

Points High Engagement Medium Engagement Low Engagement

Compass

• Achieved passing 

grades in classes

• Schoolwork 

completion

• High marks in 

classroom 

participation

• Frequent 

communication with 

Compass Program 

Manager

Meets all criteria Meets 75% of the 

criteria

Meets less than 75% of 

the criteria

GAP

• Earning a credential 

or certification

• Improved literacy

• Earning a diploma

Obtained all 3 

milestones

Obtained any 2 of the 3 

milestones

Obtained 1 or less of the 

milestones

MIGIZI

• Participation in 

community projects

• Participation in class 

discussions

Demonstrates the 

named behaviors “a lot” 

of the time

Demonstrates the 

named behaviors 

“sometimes”

Demonstrates the 

named behaviors 

“rarely”

OYOD

• Participation when 

present

Attending regularly and 

participating at a high 

level

Actively involved, but 

not always participating

Not attending regularly 

and not actively 

participating

Prior Crossing

• Attending volunteer 

events

• Attending meetings

• Engagements with 

staff

Actively participating, 

progressing toward 

goals, and showing up 

for meetings and 

volunteer events

Visible at Prior Crossing 

but typically nonresponsive or inactive in 

events

Non-responsive, 

inactive, and rarely 

visible at Prior Crossing

Tri-City Bridges

• Observations made 

when youth are 

accessing the 

program services

Actively pursuing next 

steps in ISS plan and 

responds to Bridge staff 

outreach attempts most 

or all of the time

More passive than 

active participation and 

responds to Bridge staff 

outreach attempts 

some of the time

Passive participation 

and responds to Bridge 

staff outreach attempts 

none of the time or very 

rarely

Table 9 Description of Program Engagement Metrics by Community 

Partner

Positive Disruption: Opportunity Reboot Model 54
Table 10 Targeted and Observed Opportunity Reboot Logic Model 

Outputs

Output Target

Observed

All Ages 14-17 Ages 18+

Feature 1. Positive Mentoring Relationships

O1. Staff who work directly with youth have 

participated in the Maximize Your Impact

training.2

90% 91%

100%, if OYOD is 

excluded

— —

O2. Youth have participated in the Maximize 

Your Opportunity training.3

90% 85%

92%, if OYOD is 

excluded

97%

100%

82%

90%

O3. Youth can identify 2-3 staff at their program 

who support their growth.1

95% 83% 69% 88%

O4. Youth can identify at least one new adult 

outside of the program who supports their 

growth.1

90% 75% 58% 82%

Feature 2. Individualized Goal Supports

O5. Youth have set goals and created a plan to 

reach these goals.3

95% 98% set goals

99% created 

formal goal plan 

100%

98%

97%

99%

O6. Youth have demonstrated marked growth 

towards their individualized goals.3

75% 82% 76% 84%

Feature 3. Coordinated Career Pathways Supports 

O7. Youth have enrolled in credential programs, 

trainings, or coursework aimed at preparing 

them for viable career paths.3

95% 96% 98% 95%

O8. Youth have participated in an 

apprenticeship, internship, or other paid or 

unpaid work experience.3

75% 76% 48% 86%

Feature 4. Impactful Cross-Sector Partnerships 

O9. New, beneficial collaborations have been 

established by the organization to support 

opportunity youth locally.2

100% 100% 100% 100%

Notes. Data on the outputs were gathered from three sources: 1youth-report from the endline Opportunity Reboot Youth Survey; 2staffreport outputs questionnaire; 3staff-report of implementation factors for each individual participant. One partner, Opportunity Youth of 

Duluth (OYOD), determined, in consultation with MENTOR Minnesota and Youthprise, that their program staff and youth participants would 

not participate in either of the mentoring trainings offered as supports for Opportunity Reboot. — is used to denote cells that do not have 

youth data.

Positive Disruption: Opportunity Reboot Model 55
One partner, Opportunity Youth of Duluth (OYOD), determined, in consultation with MENTOR 

Minnesota and Youthprise, that their program staff and youth participants would not participate 

in either of the mentor trainings offered as part of the Opportunity Reboot model. OYOD 

leadership felt their current professional development for staff sufficiently covered issues 

related to mentoring and mentoring mindsets, and that a youth-focused mentor training was 

not an urgent priority for the homeless youth they serve. Across partner sites, high numbers of 

participants reported they could identify 2-3 program staff and one new adult outside of the 

program who supported their growth; although, neither output target was met. Younger 

opportunity youth were also lower on both of these outputs than their older peers. 

All remaining outputs focused on the Individualized Goal Supports, Career Pathways 

Supports, and Impactful Cross-Sector Partnership elements of the Opportunity Reboot model 

were met or exceeded. As expected because of their age, fewer 14-17 year-old opportunity 

youth had participated in an apprenticeship, internship, or other paid or unpaid work 

experience. Noteworthy, all community partners indicated that their organization had 

established new, beneficial collaborations to support opportunity youth in their community. The 

number of new collaborations ranged from 5 to 74. Collaborations included for-profit 

companies, secondary and higher education institutions, adult basic education, juvenile 

corrections facilities, city and government offices, and mental health organizations.

Positive Disruption: Opportunity Reboot Model 56

Research Question 4

How satisfied were Opportunity Reboot participants with 

their experiences and the overall quality of their 

programs?

In addition to focused quantitative and qualitative analysis of opportunity youths’ experiences of 

the Opportunity Reboot model and its impact, youth were also asked to complete a brief set of 

program quality and satisfaction items at the endline administration of the Opportunity Reboot 

Youth Survey. As shown in Table 11, most youth participants rated the quality of their programs 

high; at least 72% of participants answered “agree” or “strongly agree” to each of the program 

quality and satisfaction items. It is particularly noteworthy that four out of five youth in programs 

using the Opportunity Reboot model were interested in program activities, felt physically and 

emotionally safe, felt respected, and also believed their teachers and counselors were trying to 

help them.
If you want your life 

better, if you want to 

achieve your 

goal...come here!

Table 11 Abbreviated Summary of Program Quality and Satisfaction 

Item Frequencies

Items Agreement

When I am at [program], I am interested in the activities, and what is going on. 82%

When I am at [program], people make me feel like I am important or special. 69%

When I am at [program], I feel physically and emotionally safe. 82%

When I am at [program], staff work hard to get to know me. 77%

When I am at [program], people are interested in, and respect my culture. 83%

My teachers and counselors work with key adults in my life who are trying to help me. For example, 

counselors not at my program, parenting adults, employers, teachers, probation officers, or others. 79%

I have someone at [program], who acted like a mentor to me. 75%

I know how to find people to mentor me in my life. 76%

On average, how interested or not interested are you in the activities you participate in at [program]? 72%

Note. [program] is used here as a placeholder. In the Opportunity Reboot Youth Survey, the actual name of the program a participant was 

enrolled in was used.

Positive Disruption: Opportunity Reboot Model 57
Section 3

Preliminary Impact 

Evaluation
The preliminary impact evaluation of Opportunity Reboot built directly and iteratively on the 

implementation evaluation. Its goal was to strengthen the level of preliminary evidence and 

establish emerging moderate evidence that the Opportunity Reboot model had a positive 

impact on the lives of opportunity youth, using the standards set forth by the Corporation for 

National and Community Service. The focus of the preliminary impact evaluation was twofold:

1. To empirically test the association between young people’s experiences in the programs 

using the Opportunity Reboot enhancements and key short-term youth development 

outcomes; and 

2. To test whether participants in Opportunity Reboot programs fare better than nonparticipants on select intermediate outcomes.

As shown in Figure 3, the preliminary impact evaluation consisted of three major research 

activities: (1) two youth surveys (baseline and endline; i.e., a single group non-experimental 

outcome evaluation design); (2) qualitative youth impact narrative activities; 

(3) a quasi-experimental design study of 

employment and wage attainment that 

leveraged data collected by the Minnesota 

Department of Employment and Economic 

Development (DEED) to compare program 

participants and non-participants. As described 

in the Implementation Evaluation section, staffreported dosage and engagement variables 

were created as part of the implementation 

evaluation. These variables were also included 

as part of the preliminary impact evaluation 

This section of the report begins by providing a 

methodological overview of the data collection 

methods used for these three major research 

activities, with the remainder of the section 

structured around the preliminary impact 

evaluation questions, including the specific 

analytic methods used and the associated 

results. 

All research materials, protocols, and 

procedures were reviewed and approved by an 

Institutional Review Board. Prior to 

administering the Opportunity Reboot Youth 

Survey and initiating the Opportunity Reboot 

Youth Impact Narrative activities, the evaluation 

team — in conjunction with community 

partners — obtained parental consent for each

youth to participate in the survey; this was a separate process from obtaining the youths’ 

assent, which was done in the survey itself. Youth who were 18 or older on the survey 

administration dates consented to their own participation. With IRB approval, the requirement 

of parental consent was waived for youth in circumstances where community partner staff 

determined that (a) it was not possible to secure parent or guardian consent as many 

participants were homeless and/or had severed ties with their parents or guardians because of 

neglect or abuse; or, (b) it could not be reasonably obtained or attempting to do so was not in 

the best interests of the participants.

Positive Disruption: Opportunity Reboot Model 59
In these situations, an appointed staff member at the community partner organization (e.g., 

warden, caseworker, school superintendent) who had an established relationship with the 

youth and a background in social welfare or case management was asked to approve (or not) 

the parent consent form as an authorized representative. Additional detail on the human 

participant research protections that guided the evaluation data collection activities can be 

found in Appendix D.

Positive Disruption: Opportunity Reboot Model 60
Opportunity Reboot 

Youth Survey

Positive Disruption: Opportunity Reboot Model 61

Short-Term Youth Outcomes

Survey Development 

Process

In preparation for the forthcoming 

Opportunity Reboot impact 

evaluation activities that were to 

follow, a full-day workshop was 

convened to establish shared 

agreement about the core proximal 

youth outcomes that would be used in the Opportunity Reboot Youth Survey and become the 

cornerstone of the cross-partner impact evaluation. More specifically, this workshop aimed to: (a) 

identify the core short-term youth outcomes of the Opportunity Reboot model; (b) build consensus on 

the operational definition of each outcome; (c) engage participants in vetting possible measures and 

methodologies that might be used to capture these data. The workshop provided an opportunity to 

bring multiple perspectives into the same space to shape the preliminary impact evaluation. Those 

present included program leadership from each community partner, direct service and support staff, 

evaluation team members, and Opportunity Reboot core partners, including Youthprise.

There was widespread buy-in among workshop participants that the following criteria would be 

used to evaluate whether a short-term outcome warranted inclusion in the youth impact 

survey:

§ Young people, the community partners, core Opportunity Reboot partners, and funders 

care about, and value, the short-term outcome.

§ There must be a reasonable expectation that the outcome is malleable, and that 

quantitative change can be captured over the course of a 9-12 month period across 

diverse populations of young people.

§ There is a strong theoretical argument that the short-term outcome could be a direct 

result of the features, concepts, and strategies named in the Opportunity Reboot model 

(see Table 4 ).

§ The outcome has been empirically established by extant scientific research to be a leading 

indicator of the named intermediate and longer-term outcomes of the Opportunity 

Reboot model (see Figure 2).

§ There is shared agreement that the short-term outcome is critical to shifting the life 

trajectories of opportunity youth.

In addition to these criteria, the group agreed it was critical that the short-term outcomes be 

limited to only those deemed essential for the evaluation with the intent of keeping the survey 

brief, the level of youth participation high, and the data quality strong. 
Positive Disruption: Opportunity Reboot Model 62

Further, the metrics used to assess the final outcomes must be responsive to the language, 

reading, and traumatic and contextual experiences of the opportunity youth served by these 

programs. And, these data must be collected in a way that is fair, respectful, and inviting. Many of 

these opportunity youth view research as inherently oppressive and find it difficult to trust 

“outsiders.” Thus, it was important that data were collected in a safe and open relational context 

and that it was clear to youth why these data were being collected and how the data were going to 

be used. 

The work of identifying core short-term youth outcomes began by leveraging data the evaluation 

team collected as part of the early Design and Planning workshops and focus groups. This included 

partner-specific logic models, as well as themes from the youth and staff focus groups about 

program impact. Search Institute led a large-group process in which each named outcome from 

these materials was reviewed based on the five inclusion criteria identified above. This process led 

to reorganization, renaming, and removal of some short-term outcomes. 

Using the refined list of short-term outcomes, all workshop attendees participated in an activity to 

prioritize the list. This process led to quick agreement across community partners on the core 

outcomes that would be measured in the impact study. It also provided a list of contextualized 

outcomes of high importance and relevance to individual community partners. The evaluation 

team worked with these individual community partners to define and measure these outcomes. 

One such example was at MIGIZI, where developing an appreciation of Native American identity 

and culture is an important aspect of their work with opportunity youth from the Native 

community.

With the refined list of short-term youth outcomes identified, small groups of workshop 

participants worked to draft operational definitions of each outcome. These draft definitions were 

then reviewed and commented on by other participants until shared consensus was generally 

achieved. Co-constructing these operational definitions enabled participants to move beyond 

broad outcomes (e.g., social-emotional competencies) and, additionally, articulate the specific 

skills or outcomes of interest (e.g., self-awareness, relationship skills) that comprise these broader 

outcomes. 

Using these draft operational definitions, the process of measurement identification and review 

began. Using the outcomes named in these resources, the evaluation team conducted some prework and identified a pool of possible metrics to be considered for use in the final phase of the 

implementation evaluation and the forthcoming impact evaluation. Then, equipped with a 

prioritized list of outcomes, small groups scored the available metrics based on simplicity of 

language, length, appropriateness, and relevance. Participants were asked to provide specific 

recommendations on all metrics that they felt required modification in order to work for their 

program. 

The evaluation team used the products of this working meeting to further refine the targeted 

short-term youth outcomes, and their associated metrics. The final list of short-term outcomes can 

be found in Figure 2. 
Prior to administration, the Opportunity Reboot Youth Survey was subjected to several rounds 

of careful review to test readability, interpretation, translation needs, contextual and cultural 

appropriateness, and identification of possible traumatic triggers. Vetting procedures included 

expert reviews by core Opportunity Reboot partners, staff reviews by each community partner; 

and cognitive interviews with youth participants.

Expert Reviews. The Opportunity Reboot core partner team — consisting of 10 staff with 

expertise in diversity, equity, inclusion, education, youth development, evaluation, mentoring, 

public policy, and workforce development — conducted expert reviews of the initial youth 

survey draft. Based on this feedback, items were revised to sharpen the conceptual clarity and 

dimensionality of each construct being measured. 

Staff Reviews. Staff from each program (total n = 12) reviewed the individual survey items and 

provided feedback on the appropriateness of the language, if used with their population of 

program participants, as well as item relevance and potential for items to trigger traumatic 

events. Using these recommendations, the evaluation team refined the content and created a 

version for participant review.

Youth Cognitive Interviews. Cognitive interviews (sometimes called “think-aloud interviews”) 

were conducted with 3-4 youth participants at each program location (total n = 24). The purpose 

of these interviews was to evaluate, from the youth perspective, the utility of the Opportunity 

Reboot Youth Survey as an assessment for determining program effectiveness. The evaluation 

team gathered participants’ qualitative evaluations of the survey and facilitated a process 

through which young people from the community partners’ programs helped refine the phrasing 

and appropriateness of items.

In these cognitive interviews, the evaluation team made it clear to participants that the team 

was interested in how youth came to an answer or response to a survey item rather than their 

actual answer. For example, participants were told that evaluators want to know what they were 

thinking about when they heard the question, what pictures or examples they saw in their head, 

and what the words meant to them. Participants were also monitored for any issues in reading 

the survey aloud and asked to circle words that they believed were difficult to read for them or 

their peers.

In addition to adjusting the language of the survey items, broader questions about concepts 

were posed as well. To gauge whether items might be sensitive to change, participants were 

asked if they thought their answers to particular questions would have been different a few 

months ago. Other definitional questions were asked of participants to ensure broad 

understanding of key phrases. These were typically asked due to being flagged by program staff 

as potentially problematic. An example included: “What do phrases like “goals” or “goal setting” 

mean to you? Do you refer to this in a different way? Can you give me an example of how you 

use, or might use, goal setting in your life?” These data were analyzed across sites to determine 

revision needs. All edits were made prior to baseline implementation of the Opportunity Reboot 

Youth Survey. 

The cognitive interviews also made it apparent that a number of Karen students (many refugees 

from Burma) who participated in programming at the GAP community partner site would be 

unable to complete the survey in English. As such, a Sgaw Karen version of the tool was created 

to ensure these youth could participate. This survey underwent a series of translations and 

back-translations to ensure that the wording accurately reflected the intent of the item on the 

English version of the survey.

Validating the Opportunity Reboot Youth Survey

Positive Disruption: Opportunity Reboot Model 63
The Opportunity Reboot Youth Survey was administered twice: baseline was established in Fall 

2017 (Time 1) and again at endline in Spring 2018 (Time 2). While for some youth these baseline 

and endline survey administrations aligned with the beginning and end of their program, this 

was not true for the overwhelming majority of participants; many of whom have or will cycle in 

and out of the programs offered by community partners for years as they work to get their lives 

on track. Thus, it is inappropriate to refer to — or to interpret the results of the data from —

these survey administrations as pre- and post-survey data. The nature of the wraparound 

supports provided by community partners and the (sometimes urgent, sometimes ongoing) 

needs of the youth served called for more responsive research design shaping than is 

traditional, and did not lend themselves to using a strict pre/post research design. The 

accelerated timeline prompted by funding changes also made such a design unrealistic. 

The final versions of the Opportunity Reboot Youth Survey consisted of 86 items at baseline, and 

112 items at endline (a couple of the community partners elected to include additional sitespecific items). The baseline survey took participants 22 minutes, on average, to complete; the 

longer endline survey took participants 34 minutes, on average. Most youth were surveyed on 

computers or tablets using a web-based survey that was hosted and administered via the 

evaluation team’s secure data collection platform. A small subset of youth completed the survey 

on paper; the most common reason for doing so was because one of the community partners 

served incarcerated youth who were unable to access personal electronic devices. Staff at the 

community partner sites helped facilitate youths’ access to the survey and were available to 

participants to answer any clarifying questions. 

Data Collection

The evaluation team, in collaboration with community partner staff, invited all opportunity 

youth ages 14 to 24 who were participating in partner programs in Fall 2017 and Spring 2018 to 

complete the Opportunity Reboot Youth Survey. In total, 298 youth participated at baseline, and 

236 youth participated at endline; 194 of these youth participated at both time points. 

Demographic characteristics of participating youth are reported in Table 12 (note that the sums 

of the demographic subgroups in the table may not add up to the total sample size, as some 

youth chose not to provide responses to these demographic questions).

Two-thirds of baseline (Time 1) participants were retained at endline (Time 2). If known, 

program staff were asked to account for each baseline participant who attempted prior to the 

endline survey administration. The explanations generally fit into four broad categories: (a) 49% 

of attrited participants could not be contacted for the endline survey; (b) 40% were unavailable 

due to no longer being a participant in their programs for reasons tied to mobility, early 

graduation, or disciplinary termination from their program site; (c) 7% were newly incarcerated 

when the endline survey was administered; and, (d) 4% were unavailable due to work or school 

conflicts, or experienced a significant life event when the endline survey was administered. The 

demographic characteristics of these attrited youth were generally similar to that of the youth 

who participated at both time points (see Table 12); however, there were more Black and White 

participants and fewer Asian participants in the attrited group. Additionally, there were more 

US-born participants and participants of non-Twin Cities metropolitan area programs among the 

attrited youth compared to the retained sample of youth. 

Sampling

Positive Disruption: Opportunity Reboot Model 64
Table 12 Demographic Characteristics of Opportunity Reboot Youth 

Participants

Characteristic Matched Sample Attrited Sample

n Valid % n Valid %

Age M = 19.4 M = 19.3

Gender

Male 93 51.1% 48 49.0%

Female 89 48.9% 50 51.0%

Race/Ethnicity

African, African American, or Black 26 17.0% 24 24.5%

Asian 54 35.3% 18 18.4%

Native American 10 6.5% 8 8.2%

White 24 15.7% 22 22.4%

Mixed Race 13 8.5% 11 11.2%

Some Other Race 3 2.0% 4 4.1%

Hispanic or Latinx 23 15.0% 11 11.2%

Sexual Orientation

Heterosexual 78 63.4% 59 62.8%

Not Heterosexual 45 36.6% 35 37.2%

Nativity

Born in the United States 101 55.2% 64 66.0%

Born outside the United States 82 44.8% 33 34.0%

Geographic Location

Participated in Twin Cities Metro Sites 107 57.5% 46 46.9%

Participated in Greater Minnesota Sites 79 42.5% 52 53.1%

Notes. The attrited sample is comprised of youth who only participated in the baseline administration of the Opportunity Reboot Youth Survey. 

The sums of the demographic subgroups may not add up to the total sample size, as some youth chose not to provide responses to these 

demographic questions.

Positive Disruption: Opportunity Reboot Model 65
The Opportunity Reboot Youth Survey included metrics on the following major sections: (a) 

youths’ experiences of the Opportunity Reboot model features; (b) youth short-term outcomes; 

and as mentioned in the implementation evaluation; (c) program quality and satisfaction 

indicators; and, (d) selected youth-focused outputs named in the Opportunity Reboot logic 

model (Figure 2). Appendix E summarizes the measures’ psychometric properties

Preliminary Impact Measures and Instruments

Independent variables: Opportunity Reboot Model features. The Opportunity Reboot model 

has four core features: positive mentoring relationships, individualized goal supports, 

coordinated career pathways supports, and impactful cross-sector partnerships. In the 

Opportunity Reboot Youth Survey, the positive mentoring relationships feature was parsed into 

two measures in order to discreetly capture both youths’ experience of within-program 

mentoring relationships and outside-of-program mentoring relationships. Youths’ experiences of 

the cross-sector partnerships were excluded from the survey for reasons of both brevity and this 

feature’s more direct focus on the six partner organizations. The study (and subsequent 

analyses) included the following four measures of the Opportunity Reboot model features 

assessed only at endline, with the exception of within-program positive mentoring relationships, 

which was assessed at both time points:

Within-program positive mentoring relationships were measured using Search 

Institute’s Developmental Relationships Framework (Roehlkepartain et al., 2017). The 

scale uses five items (αT1=.94; αT2=.92) aligned with each of the major elements of the 

framework. To help youth focus on a specific relationship within the program they 

participate in, the evaluation team worked with each community partner to determine 

who the case manager or case manager-equivalent was at each site. These items were 

coded such that the appropriate site-specific relational target was explicitly named for 

youth in the respective programs. Items asked youth to assess how much their case 

manager(s) at the specific program showed them that they mattered (Express Care), 

pushed them to be their best (Challenge Growth), helped them get things done (Provide 

Support), took their ideas seriously (Share Power), and connected them with new people 

or places (Expand Possibilities). All five items were assessed on a 5-point frequency scale 

ranging from Never to Very Often. 

Outside-of-program positive mentoring relationships were assessed with a single item: 

“During my time at [name of program inserted], I formed a relationship with an adult 

outside of the program who supports my growth. This might include, for example, a 

boss, mentor, neighbor, church members.” This question was asked only at endline 2, 

and was assessed on a 5-point agreement scale ranging from Strongly Disagree to 

Strongly Agree. 

Positive Disruption: Opportunity Reboot Model 66
Individualized goal supports were measured with four items (α= .88): “When I am at [name 

of program inserted], people make me feel like I am important or special,” “[Name of 

program inserted] helped me identify what I am good at and areas where I may need help,” 

“The staff at [name of program inserted] checked in with me

regularly about my goals,” and “The staff at [name of 

program inserted] want me to reach the goals I set.” 

All four items were assessed on a 5-point agreement 

scale ranging from Strongly Disagree to Strongly 

Agree. 

Coordinated career pathways supports were 

measured with two items: “At [name of program 

inserted], I had opportunities to explore education 

and career options” and “[Name of program 

inserted] helped me make a plan to reach my career 

or education goals.” These items were highly 

correlated 

(r = .69***). Both items were assessed on a 5-point 

agreement scale ranging from Strongly Disagree to 

Strongly Agree. 

The means and standard deviations of these 

independent variables can be found on Table 13; 

individual item frequencies can be found in Tables 14 

and 15.

Dependent Variables: Short-term Youth Outcomes 

As described in the Short-Term Youth Outcomes section above, the key stakeholders in the Opportunity 

Reboot project collectively narrowed the targeted youth outcomes to 11 variables, which fit in three 

overarching categories: positive identity, social-emotional competencies, and skills for systems 

navigations. Unless otherwise noted, all of these survey items were asked in both the baseline and 

endline youth surveys, and were assessed on a 5-point agreement scale ranging from Strongly Disagree 

to Strongly Agree. Scales were created by calculating the mean of available items. The short-term youth 

outcomes targeted in this evaluation include: 

Positive Identity

§ Positive self-worth, a 3-item scale (α= .93), assesses youths’ sense of self-worth and belief in 

themselves.

§ Positivity in the face of challenge, a 3-item scale (α= .84), asks youth about their ability to stay positive 

— or perceive positive opportunities and takeaways — when faced with failure, challenges, or other 

difficult situations. 

§ Goal-setting skills, a 6-item scale (α= .88), gauges youths’ ability to set goals and then figure out 

different ways to reach those goals (e.g., breaking the goals down into more manageable steps; 

acquiring specific skills; adapting to setbacks) to reach these goals. 

§ Future orientation, a 3-item scale (α= .82), measures youths’ orientation towards the future, and 

whether they consider the ramifications of decisions on their future. 

§ Civic efficacy, a 3-item scale (α= .78), measures youths’ belief in doing things that have a positive 

impact on other individuals as well as one’s community. 

Positive Disruption: Opportunity Reboot Model 67
Social-Emotional Competencies

§ Self-awareness, a 3-item scale (α= .80), asks youth whether they can identify their 

strengths and weaknesses, if they know what matters most to them, and if they know 

who they are and what they believe in. 

§ Responsible decision making, a single item measure, is represented by this statement: 

“I think about what might happen before making a decision.” 

§ Relationship skills, a 5-item scale (α= .71), assesses a range of interpersonal skills 

including active listening, empathy, sociability, and communication skills. 

Systems Navigation and Skills

§ Financial literacy, a 5-item index, assesses youths’ ability to understand and manage 

their personal finances. For this index, youth were presented with a set of questions on 

money management and budgeting, as well as knowledge of financial institutions and 

products, and asked to assess their familiarity with each. Index scores are calculated by 

summing the scores on the five items. Possible values on this index range from 0 to 20.

§ Job-seeking skills, a 5-item index, consists of questions that ask youth about their 

familiarity with various aspects of the job-seeking process (e.g., where to find postings 

for jobs or paid internships; how to complete an application, prepare a résumé and 

cover letter, and how to prepare for an interview). Although originally scored on a 5-

point agreement scale, for purposes of analysis, items were dichotomized such that 

youth who responded that they Agree or Strongly Agree were recoded as 1. All other 

responses were recoded as 0. These dichotomized versions of these items were 

summed to create an index score, ranging from 0 to 5.

§ Resource identification skills, a 3-item 

scale (α= .85), asks youth if they can find 

the services they need and, if not, if they 

know someone they can talk to who can 

point them toward the needed health, 

education, employment, financial, and 

housing services they may be seeking to 

access.

Positive Disruption: Opportunity Reboot Model 68
Table 13 Means of Opportunity Reboot Model Feature Measures

Measures Mean

(SD)

Within Program Mentoring Relationship at Baseline 3.94

(0.95)

Within Program Mentoring Relationship at Endline 4.11

(0.80)

Outside of Program Mentoring Relationships 3.96

(0.95)

Individualized Goal Supports 4.18

(0.69)

Coordinated Career Pathways Supports 4.09

(0.71)

Notes. All items are on a 5-point scale (response values range between 1 and 5). Baseline refers to the Time 1 administration of the Opportunity 

Reboot Youth Survey; endline refers to the Time 2 administration. Δ represents the change in means from baseline to endline.

Positive Disruption: Opportunity Reboot Model 69

Table 14 Frequencies of Within Program Mentoring Relationship Items

Items

Never Rarely Sometimes Often Very Often

Baseline Endline Baseline Endline Baseline Endline Baseline Endline Baseline Endline

How often does your [case manager] ...

...show you that you 

matter to them? 3.8% 3.3% 7.0% 1.6% 25.9% 21.9% 27.6% 35.0% 35.7% 38.3%

...push you to be your 

best? 3.3% 2.2% 4.9% 0.5% 17.4% 15.3% 35.3% 37.7% 39.1% 44.3%

...help you get things 

done? 1.6% 1.1% 5.5% 4.3% 25.7% 12.0% 27.3% 39.7% 39.9% 42.9%

...listen to your ideas and 

take them seriously? 2.2% 1.6% 4.9% 2.2% 21.9% 18.6% 32.2% 37.2% 38.8% 40.4%

...connect you with new 

people or services that 

help you?

2.7% 2.2% 9.3% 4.3% 23.5% 22.3% 27.9% 33.2% 36.6% 38.0%

Notes. [case manager] is used here as a placeholder for information that was tailored for each partner. The evaluation team worked with each 

partner to tailor the wording of this item such that youth were directed to think about the program staff who most directly oversaw the execution 

of the Opportunity Reboot model at their site. Some partners used terms like “case manager,” or “program mentor” while others opted to refer to 

staff explicitly by name. Baseline refers to the Time 1 administration of the Opportunity Reboot Youth Survey; endline refers to the Time 2 

administration.
Table 15 Frequencies of Outside of Program Mentoring, Individualized 

Goal Supports, and Career Pathways Supports Items

Items Strongly 

Disagree Disagree

Somewhat 

Disagree, 

Somewhat 

Agree

Agree Strongly 

Agree

Outside of Program Mentoring

During my time at [program], I formed a 

relationship with an adult outside of the 

program who supports my growth. This 

might include, for example, a boss, mentor, 

neighbor, church members.

3.0% 3.6% 18.6% 44.3% 30.5%

Individualized Goal Supports

When I am at [program], staff work to meet 

my personal needs. That is, staff figure out 

what your specific needs and interests are 

and then help you.

0.6% 4.8% 15.5% 50.6% 28.6%

[Program] helped me identify what I am 

good at and areas where I may need help. — 4.7% 14.7% 45.9% 34.7%

The staff at [program] checked in with me 

regularly about my goals. — 5.3% 18.1% 48.5% 28.1%

The staff at [program] want me to reach the 

goals I set. — 2.9% 11.1% 47.4% 38.6%

Coordinated Career Pathways Supports

At [program], I had opportunities to explore 

education and career options. — 1.8% 12.3% 49.1% 36.8%

[Program] helped me make a plan to reach 

my career or education goals. 0.6% 2.9% 14.0% 45.6% 36.8%

Note. .[program] is used here as a placeholder. In the Opportunity Reboot Youth Survey, the actual name of the program a participant was 

enrolled in was used. — is used to denote cells in which no youth used this response option.

Positive Disruption: Opportunity Reboot Model 70
Longitudinal impact narrative activities were incorporated in the preliminary impact evaluation of 

Opportunity Reboot to deepen, and provide nuance for understanding how opportunity youth 

experienced the Opportunity Reboot model at community partner sites. The evaluation team 

worked closely with program staff to coordinate these activities, and to ensure the data collection 

procedures were carefully and intentionally designed in ways to make participating youth feel safe 

and comfortable.

The evaluation team employed a collective set of qualitative photovoice methods that consisted 

of four distinct parts: (a) a group orientation with 2-4 participants at each partner site; (b) 

independent youth-initiated activities designed to help youth capture their program experiences 

(i.e., taking photos or writing/drawing in a journal); (c) a one-on-one interview with each 

participant to complete a journey map using their pictures or journal entries; and (d) reconvening 

the participants from the initial orientation to participate in a focus group. 

Qualitative Impact Narrative Activities

At most partner sites, the first two activities occurred in early Spring 2018 and the last two 

activities occurred in late Spring 2018 or early Summer 2018. The timing between the first and 

second group of activities was typically six to eight weeks. Activities were primarily conducted at 

the program site. Participants were invited to take pictures or write in their journals at any 

location that was reflective of their program experiences, including outside the program site 

during field trips.

Data Collection

A small subsample of Opportunity Reboot participants (n = 29; 2-7 from each community partner 

program) took part in the qualitative impact narrative activities. Participants were recruited by 

program staff, who identified a sample of young people who were available and willing to 

participate, and who reflected some of the diversity of youth typically served by the program. As 

part of the impact narrative activities, participants were asked to voluntarily self-identify their 

age, gender, and race or ethnicity by writing it on their journey map. Of the 29 participants, 21 

chose to provide some or all of this information. Participants’ ages ranged from 14 to 24. Eleven 

participants identified as male, ten as female. Two participants identified as African-American, 

three as White, two as mixed, four as Asian, one as Karen, two as Somali, three as Hispanic or 

Latinx, and one as White, Black, and Native American. 

All 29 youth participated in the group orientation and the photo-taking or journaling activities. A 

small number of youth (n = 4) were unable to participate in the focus group due to scheduling 

conflicts or, at one location, there was only one participant who showed up. For these youth, the 

questions from the focus group were discussed using an interview format.

Sampling

The use of photovoice as a research method has been utilized since the early 1990s, blending 

qualitative narratives with the medium of photography in order to explore participants’ 

experiences in their communities (Wang & Burris, 1994). Early photovoice methods included a 

number of distinct steps, such as researcher-guided focus group discussions about key 

photographs selected by participants and participant-led discussions.

Impact Narrative Protocols

Positive Disruption: Opportunity Reboot Model 71
on how to share this information with stakeholders (Wang & Burris, 1994; Wang, 1999). 

Photovoice has since gained popularity as a qualitative research method that allows researchers 

a nuanced insight into the visualized perceptions of individuals’ everyday realities (Close, 2007; 

Foster-Fishman et al., 2005). The use of photovoice data can be effectively used as a tool to 

better understand and evaluate participants’ experiences in a program (Kramer et al., 2013). 

The Opportunity Reboot qualitative impact narratives utilized three protocols that integrated 

several of the cornerstone aspects of photovoice methods. These included (a) a group 

orientation protocol; (b) an interview protocol for participants to share how their photos and 

journals were reflective of their experiences in the program; and (c) a focus group protocol that 

explored how participants perceived change in themselves in their relationship to school and 

work as a result of the program.

Group Orientation Protocol. Activities began with a researcher leading an orientation with 

participants in a group setting. The purpose of the orientation was to build community 

among participants, introduce the impact narrative activities, and provide training on the 

use of disposable cameras or journals. Participants were provided a Fujifilm Instax camera to 

practice taking pictures that were centered and clear. If participants utilized the journal 

option, they were provided a practice journal with a prompt asking them to choose one 

milestone or barrier from the program and make an entry in the journal about that 

experience.

Interview Protocol. Participants used disposable cameras for the photo journaling activity to 

take pictures of places, people (with permission), and experiences throughout their time in 

the program that they identified as important to share. Some partner sites opted-out of the 

photo option, due to privacy concerns. For these sites, a journaling option was used instead, 

with participants writing about or drawing their experiences in a journal provided by the 

evaluation team. During the one-on-one interviews, participants were prompted to select 

several photos, writings, or drawings that

stood out to them and place them on a “journey 

map” generally in chronological order. They 

discussed how these artifacts were representative 

of their experiences in the program and what was 

important about those experiences to their 

growth. Youth were allowed to keep the photos 

(or journal) at the end of the interview. Interviews 

were approximately 90 minutes long 

Focus Group Protocol. In a larger focus group, 

participants were asked to complete a “Three-Act 

Play” of how their relationship to school and work 

had changed as a result of the program. 

Participants drew pictures or wrote

words on an 11×17 handout with three large boxes to create three “scenes” depicting their 

relationship with work and school before the program, their experience now and what they 

think their relationship with school and work will be in the future. Participants described the 

scenes to the focus group facilitator and talked about how they had changed over time. 

Participants took turns describing the scenes and talked about how they had changed over time. 

These focus groups were approximately one and a half to two hours long.

Positive Disruption: Opportunity Reboot Model 72

2Copies of these protocols are available by request from the authors.
Employment and Wage Attainment Study

The impact evaluation also included a quasi-experimental study using data collected by the 

Minnesota Department of Employment and Economic Development (DEED) to track evaluation 

participants’ employment and wage attainment, and to compare how participants who were 

exposed to programs using the Opportunity Reboot model compared to youth who shared 

similar demographic characteristics but were not involved in these programs or other similar 

programs.

In order to assess participants’ success at securing living wage employment and attaining stable 

or increasing wages, the evaluation team partnered with DEED, which is responsible for tracking 

employment and wage data on all Minnesota residents. A contract was established with DEED 

to examine the employment and wage data of our study participants and use propensity score 

matching to compare them with youth who shared similar demographic characteristics over 

four quarters: Q3 of 2018, Q4 of 2018, Q1 of 2019, and Q2 of 2019. Data from Q2 of 2018 (i.e., the 

same quarter when the Time 2 Opportunity Reboot Youth Survey was administered) was 

established as the reference point (or T0) for DEED’s analyses. For some youth (estimated to be 

59% based on program exit data provided by staff), this was also approximately the time of their 

exit from community partner programs.

Data Collection

The wage and employment data that were analyzed in this portion of the evaluation were 

collected by DEED; the collection of these data is one of the many duties the agency is charged 

with in the Minnesota Statutes. The only additional data collected for this piece of the 

preliminary impact evaluation were detailed socio-demographic data on each Opportunity 

Reboot participant. These data were collected using the Common Participant Profile (completed 

by program staff using their internal participant tracking database). The questions in the 

Common Participant Profile were designed to mirror the socio-demographic indicators tracked 

by DEED; allowing for higher quality propensity score matching. The list of socio-demographic 

indicators drawn from the Common Participant Profiles are listed below.

Positive Disruption: Opportunity Reboot Model 73
2 CareerForce is a public-private partnership that provides career training, services, and information to job seekers, and an applicant matching service to employers 

(among other things). CareerForce has physical locations where job seekers can seek career advice (like having someone look over their resumes, help them 

prepare applications, etc.).

3 MinnesotaWorks is a state-run database where job seekers can upload their resumes - employers can then search this database to find potential applicants who 

have skills that they are looking for in positions they would like to fill.

Positive Disruption: Opportunity Reboot Model 74

There were two samples for this portion of the study: the sample of young people who 

participated in community partner programs using the Opportunity Reboot model, and a 

comparison sample that was generated by DEED using propensity score matching.

Sampling

DEED used propensity score matching techniques to generate a comparison sample and estimate 

the effect of the Opportunity Reboot model. The evaluation team provided DEED with detailed 

socio-demographic data from the Common Participant Profile on each of the Opportunity Reboot 

participants who met inclusion criteria. 

The socio-demographic characteristics used to create 

a matched comparison sample were: 

§ Gender (from the Common Participant Profile); 

§ Latinx ethnicity (from the Common Participant Profile); 

§ Race (from the Common Participant Profile); 

§ Geography (specifically, Twin Cities metropolitan area

vs. Greater Minnesota; this item was created using the

program sites’ locations); 

§ Wages of study youth at program enrollment (DEED

was able to locate wage data using the Social Security

numbers and birthdates); and, 

§ Educational attainment of study youth at program 

enrollment (this was provided by the program sites provided).

Overall, 209 youth from Opportunity Reboot participated in the employment and wage portion of 

the impact evaluation, along with 241 youth drawn by DEED as a comparison sample, using 

propensity score matching techniques. The two samples differed on race and educational 

attainment distributions, with the comparison sample having a higher proportion of White youth 

and youth with some college. These differences were then mitigated by employing a number of 

statistical adjustments to strengthen comparability (see Mitigation of Differences between 

Opportunity Reboot and Comparison Samples section below). Demographics and comparisons of 

demographics are found in Tables 16 and 17, and details on each sample follow.

Opportunity Reboot Sample

Inclusion criteria included providing the evaluation team with a Social Security number and 

birthdate and being between the ages of 18 and 24. Social Security numbers and birthdates were 

required in order to identify youth from Opportunity Reboot sites in the state’s employment and 

wage databases. This identifying information was collected from participants as part of the IRBapproved informed consent and assent processes. 

For comparison purposes, the Opportunity Reboot sample for this portion of the study was 

limited to youth ages 18 to 24, as DEED does not collect data from youth 17 years and younger. 

Additionally, Opportunity Reboot participants were excluded by DEED from propensity score 

matching if they had any missing data on any of the matching socio-demographic characteristics 

listed above (9 youth, or 4% of the study sample, met this criteria). Following these inclusion 

criteria, the evaluation team was able to gather Social Security numbers and birthdates from 209 

of the opportunity youth participating in Opportunity Reboot sites. 
Table 16 Descriptive Statistics of Comparison and Opportunity Reboot 

Samples

Characteristic Comparison Opportunity Reboot

Total Sample 241 209

Mean Age (years) 20.6 21.2

Female 47% 49%

Race/Ethnicity

American Indian 5% 11%

Asian 3% 44%

Black / African American 19% 26%

Hispanic / Latinx 6% 7%

White 63% 16%

Multiracial 7% 2%

Educational Attainment

HS Diploma/GED or Less 12% 75%

Some College 45% 2%

College Degree 0% 0%

Prior Employment 67% 56%

Note. These descriptive statistics were provided by the Minnesota Department of Employment and Economic Development (DEED).

Positive Disruption: Opportunity Reboot Model 75
Table 17 t-Tests of Select Sociodemographic Characteristics between 

Comparison and Opportunity Reboot Groups

Characteristic Means △ SD t

Comparison Opportunity Reboot 

Gender .57 .54 0.10 0.48

Person of Color (POC) .87 .87 0.04 0.16

Educational Attainment

Less than HS Diploma/GED .74 .73 0.06 0.16

HS Diploma/GED or Less .23 .23 0.06 0.04

Some College .02 .03 0.01 0.50

Prior Employment .60 .59 0.10 0.26

Age 21.16 21.09 0.35 0.34

Age × Gender 33.12 32.44 2.15 0.56

Prior Employment × POC .52 .51 0.10 0.06

Notes. The results of the t-tests confirmed there were no statistically significant differences between the comparison and Opportunity Reboot 

groups on these variables. Gender was coded as 0=Male, 1=Female; Person of Color, the Educational Attainment categories, and Prior 

Employment were dichotomously coded such that 0=Characteristic absent and 1=Characteristic present. Age is a continuous variable. Results 

were provided by the Minnesota Department of Employment and Economic Development (DEED).

Positive Disruption: Opportunity Reboot Model 76
Approximately half of the Opportunity Reboot sample for the employment and wage attainment 

study were GAP participants (n = 107); the other half of the sample consisted of 36 youth from 

Tri-City Bridges, 33 youth from Prior Crossing, 27 youth from Opportunity Youth of Duluth, and 

12 youth from MIGIZI. Compass, one of the community partners serving high school students, 

was completely excluded from these analyses, since very few of these youth were age-eligible 

for employment under Minnesota labor statutes. 

No additional inclusion criteria such as participation/dosage minimums, or being an active 

program participant at the time were imposed. But, the great majority of youth had participated 

for a relatively long period in the programs, and relatively frequently as well. Using enrollment 

and exit dates from the Common Participant Profile, it was calculated that 89% of study youth 

had participated in their programs for 4 or more quarters (only 4.4% had participated in their 

programs for less than 2 quarters, and fewer than 11% had participated for less than 3 quarters). 

Additionally, 67% of participants were rated by their staff mentors as having high dosage, and 

24% were rated as having medium dosage (i.e., less than 10% of participants had low dosage). 

Exits from Opportunity Reboot partner programs 

As mentioned previously, by the time the analyses for the employment and wage attainment 

study were conducted, 59% of participants had exited their programs; however, other than GAP 

— which has an explicit graduation date, it is unknown whether these exits were due to 

program completion or dropout. Based on the previous experience of the programs, it is 

reasonable to expect that many of these youth will cycle in-and-out of these programs over 

several years as they need additional — or, different — supports. The deep relationships and 

connection opportunity youth have with these programs and staff often make them safe spaces 

for youth to turn to when they need help. Programs, like those we partnered with for 

Opportunity Reboot, are intentional in designing seamless on- and off-ramps that allow 

opportunity youth to re-engaged with the program as needed.

When disaggregated by program, all GAP youth exited the program at endline, while all MIGIZI 

and Tri-City Bridges youth remained. Nearly all Prior Crossing youth (except for two individuals) 

remained as well. Notably, Opportunity Youth of Duluth (OYOD) had a 50% exit rate, with no 

discernable demographic or background differences between those who exited and those who 

remained. These differences are tied to the programs’ designs: 

§ GAP is a 1-year program with an explicit graduation date; 

§ MIGIZI and TCB both provide a year of intensive programming, followed by lighter contact for 

as long as needed by participants;

§ Prior Crossing is a residential program, with no end date;

§ Opportunity Youth of Duluth provides drop-in career planning and job seeking services, as 

well as a year-long course in entrepreneurship and small business.

Positive Disruption: Opportunity Reboot Model 77
Retained vs. Excluded Opportunity Reboot Sample

As noted above, a small total of 9 youth (4% of the sample) were excluded by DEED from the 

employment and wage part of the study. When comparing the Opportunity Reboot participants 

who were retained in DEED’s analyses to those who were excluded or dropped (mostly due to 

study staff’s inability to obtain Social Security numbers and birthdates):

§ There were disproportionately more Asian participants the DEED analysis sample (44.4% vs. 

8.4% in the sample of excluded Opportunity Reboot participants);

§ There were disproportionately fewer US-born participants in the DEED analysis sample 

(43.2% vs. 76.9% of excluded Opportunity Reboot participants);

§ There were disproportionately fewer participants from Greater Minnesota in the DEED 

analysis sample (27.7% vs. 60.0% of excluded Opportunity Reboot participants).

These comparisons overstate the differences between the excluded and retained youth because 

the percentages for the excluded youth are based on a very small group of 9 individuals. 

Nevertheless, to the extent that participants’ inclusion into (or exclusion from) the DEED sample 

was not random, it is therefore a source of potential response bias, albeit quite small. As a 

result, findings from DEED’s analyses are likely biased against more-transient individuals; 

specifically US-born individuals living in Greater Minnesota, whose whereabouts are harder for 

program sites to keep track of (nearly all of Opportunity Reboot’s Asian youth were Karen 

refugees in the GAP program, and residents of the Twin Cities metropolitan area). 

Comparison Sample

Three criteria were used to identify the pool of eligible individuals for the comparison sample: 

(a) the individual needed to be between the ages of 18 and 24, as DEED does not collect data 

from individuals under the age of 17; (b) the individual had to have sought services at a 

CareerForce location, and/or posted their resume on Minnesota Works (a possible indication 

they were disconnected from school and/or work); and, (c) the individual was not currently —

and had not previously been — a participant in an eligibility-based program similar to the 

Opportunity Reboot partner sites. (DEED tracks enrollment using Workforce One; all state and 

federally funded workforce development programs are required to use Workforce One for case 

management and data entry as a condition of receiving funding.)

DEED then used the sociodemographic data noted above to assign a propensity score to each 

member of the comparison group; the higher their score, the more similar they are to 

Opportunity Reboot participants on these demographic characteristics. Thus, each Opportunity 

Reboot participant was matched to an individual (or multiple individuals) whose demographic 

characteristics suggested they were very similar; the only difference was these “matched” 

individuals did not participate in programs using the Opportunity Reboot model. Therefore, it 

can be inferred that any differences DEED observes between the Opportunity Reboot study 

Positive Disruption: Opportunity Reboot Model 78

3 CareerForce is a public-private partnership that provides career training, services, and information to job seekers, and an 

applicant matching service to employers (among other things). CareerForce has physical locations where job seekers can 

seek career advice (e.g., having someone look over their resumes, help them prepare applications).

4 MinnesotaWorks is a state-run database where job seekers can upload their resumes. Employers can then search this 

database to find potential applicants who have skills that they are looking for in positions they would like to fill.
participants and the comparison group can reasonably be attributed to study youths’ 

participation in Opportunity Reboot. 

The comparison sample consisted of 241 individuals. Descriptive statistics on the comparison 

sample, and on how they compare to the Opportunity Reboot sample, can be found in Table 16. 

The unique circumstances and backgrounds of Opportunity Reboot participants resulted in 

marked discrepancies in race and educational attainment when compared to individuals in the 

comparison sample. 

To mitigate this, DEED performed kernel density matching (see Bendewald et al., 2016) using the 

diff Stata package obtained from the Boston College Statistical Software Components (SSC) 

archive of user-contributed programs. The propensity scores are used as weights for comparison 

group data; comparison group individuals who are ‘better’ matches with individuals in the 

Opportunity Reboot sample are assigned higher weights, and are consequently more influential. 

This approach decreases the variance between the control and treatment groups, at the cost of 

having a higher probability of having less-than-ideal matches in the comparison group. Thus, the 

impact of any subpar matches will be assigned very small weights, mitigating their influence 

(Caliendo & Kopeinig, 2005). Additionally, the analyses were restricted to observations within 

the area of common support (i.e., the range of propensity scores where the probability of 

observing treatment and control cases overlap (Heckman et al., 1999), which also weeds out 

suboptimal matches. 

DEED ran two-sample t-tests, which are summarized in Table 17, to verify that the comparison 

and Opportunity Reboot samples were comparable (see Bryson et al., 2002). As expected, none 

of the t-scores were statistically significant. Furthermore, the differences between the 

comparison and Opportunity Reboot samples’ standard deviations were within .25 except for 

age at SD = .34, and the age x gender interaction term indicating that the two samples were 

statistically comparable. Note that interactions between age x gender and persons of color x 

prior employment were included because Hosmer-Lemeshow tests (Lunt, 2014) run by DEED 

indicated that these factors impacted inclusion into the Opportunity Reboot treatment group in 

a non-linear way.

Mitigation of differences between Opportunity Reboot and Comparison 

Samples

Positive Disruption: Opportunity Reboot Model 79

Post hoc power analyses were specified in G*Power (Faul et al., 2007) to estimate the power 

achieved based on small effect sizes (Cohen’s d = .25 – .40). Small effect sizes were selected 

because a previous intervention targeting employment outcomes for youth with similar 

characteristics as the current study sample found similar effect sizes between treatment and 

comparison groups (e.g., Ferguson, 2018). Similarly, a meta-analysis of programs similar to 

Opportunity Reboot found that program effects tend to be small in the year immediately 

following the program (Card et al., 2011; 2015). Based on these parameters, the estimated 

power for the quasi-experimental sample is .845 – .996 (.607 – .923 for the Persons of Color 

subsample). These findings show that the study had enough power (.80 or above) to detect 

statistically significant differences between the Opportunity Reboot sample and the comparison 

group for effect sizes above .25. The study was slightly underpowered to be able to detect 

statistically significant differences between Opportunity Reboot Persons of Color and the 

comparison group subsample for smaller effect sizes. 

Power Analyses
The preliminary impact evaluation was designed to minimize several critical threats to internal 

and external validity in pursuit of strengthening the level of evidence supporting the 

Opportunity Reboot model. The approach to minimizing these threats was informed by the 

evaluation teams’ learning from, and alongside, community partners during the implementation 

evaluation. 

Internal and External Validity

Internal Validity

The Opportunity Reboot Youth Survey was administered at baseline and endline to establish 

associations between opportunity youths’ experiences of the four core features of the 

Opportunity Reboot model and growth in their positive identity, social-emotional competencies, 

and ability to navigate systems. The use of a single-group non-experimental outcome evaluation 

design was selected given that this was the first evaluation of the Opportunity Reboot model 

and community partners had not collected data of this rigor before. 

While this design limits claims that the observed change in outcomes is caused by the 

Opportunity Reboot model, efforts were made to preliminarily strengthen the cause-effect 

relationship between the Opportunity Reboot model and youth outcomes by integrating a rich 

series of qualitative impact narrative activities that elicited organic responses from participants 

about their experience of the Opportunity Reboot model features and their impact (if any). The 

employment and wage employment study was also added to further strengthen the preliminary 

evidence (and establish emerging moderate evidence) targeted in the single group nonexperimental component of the preliminary impact evaluation. Descriptions of how internal 

validity threats were addressed is provided below:

Selection Bias

A propensity score matching procedure (described above) was used to reduce potential 

selection bias in drawing a comparison group. The results of the matching procedures showed 

that the Opportunity Reboot sample and the comparison group are statistically similar across all 

of the matching covariates. Furthermore, the differences between the samples were less than 

0.25 standard deviations for all of the matching covariates with the exception of age (SD = .34). 

These findings suggest that propensity score matching was able to minimize the likelihood that 

characteristics of the sample were responsible for the differences that were found between the 

Opportunity Reboot sample and the comparison sample. It is, however, important to note that 

potential threats to internal validity remain despite the use of propensity score matching. Since 

the matching was limited to a finite number of demographic variables, there were undoubtedly 

other participant characteristics that were excluded and may have been associated with the 

outcomes of interest.

Positive Disruption: Opportunity Reboot Model 80

Although propensity score matching is a powerful method that makes the quasi-experimental 

portion of the present study possible, it is important to also note its limitations. For instance, 

the quality of the comparison group matches is only as good as the matching variables used. For 

example, participants in Opportunity Reboot programs have experienced substantial barriers 

such as homelessness and interactions with the criminal justice system, but since these factors 

were not quantified in the comparison data set, it was not possible to match the Opportunity 

Reboot participants in the treatment sample with fully comparable people in the comparison 

pool. It is therefore possible for these ‘omitted variables’ to adversely impact the statistical 

significance of the models.
Selection Additive Effects 

Important demographic characteristics that may have accounted for differential treatment effects 

were accounted for in regression models (e.g., age, gender, sexual orientation, race/ethnicity) to 

help reduce the likelihood of selection additive effects. As described above, the propensity score 

matching procedure used a number of important demographic characteristics that may be related 

to employment and/or wage attainment. This procedure resulted in a statistically similar 

Opportunity Reboot and comparison group sample, reducing the likelihood that participant 

characteristics accounted for differences that were observed between these two samples. 

Instrumentation

To ensure observed changes were not due to the testing procedure, every effort was made to 

maintain consistency in the survey instrument, the administrators, and the method of 

administration across community partner sites and time points. The only changes in the survey 

between administrations were the introduction of an additional set of metrics that were only 

appropriate to ask at endline. Administrators were provided with an administration guide aimed at 

standardizing the protocol. Lastly, at both time points the survey was administered via the same 

medium.

Experimental Mortality

Many opportunity youth struggle with mental and chemical health issues, homelessness, and 

disconnection from family and, as a result, tend to be highly mobile. This makes study retention 

challenging. However, even given these challenges, two-thirds of the baseline sample were retained 

at the endline survey. Program staff made every effort to invite all youth who were being served by 

their programs at either time point to participate in the evaluation, and made an extra concerted 

effort to invite the endline participation of all baseline participants. Using available data, 

demographic comparisons were made between the retained and attrited samples. Findings showed 

that the retained and attrited samples were similar across most demographic characteristics with a 

few exceptions (i.e., race/ethnicity, nativity). These demographic characteristics were accounted for 

in all of the regression analyses to account for any impact they may have on outcome variables. Yet, 

it is still likely that attrited youth were different from retained youth in other important ways that 

are not accounted for in the regression models, which may limit internal validity. 

Regression to the Mean

In the Design and Planning workshops, program staff explained the phenomenon of creaming that 

sometimes occurs when their programs are asked to provide data upon which high-stakes funding 

decisions are made. Creaming refers to hand selecting the participants from whom data is shared in 

order to maximize empirical evidence of impact. To minimize this phenomenon, the evaluation 

team talked at length with all programs about the need to recruit all youth to participate, not just 

youth who are likely to have extreme baseline scores. Building on the strong relationship with the 

evaluation team, community partners trusted that this approach would yield the most valuable data 

for continuous program improvement and, even if the findings were null or small, there would still 

be a compelling story to tell. Findings also showed few differences between the retained and 

attrited samples however, it is possible that important demographic characteristics were missing in 

the analyses.

Maturation

The time between the two administrations of the Opportunity Reboot Youth Survey was about 9 

months across community partners. Limiting the amount of time between administrations had the 

benefits of reducing the amount of mental and physical changes youth may have experienced. To 

control for the effect of changes that might have been brought about by improved access to mental 

health services through participating programs, mental health status was included as a covariate in 

the models. 

Positive Disruption: Opportunity Reboot Model 81
Expectancy Effects

The informed consent and assent forms distributed before each data collection — and reviewed 

verbally — clearly communicated that the purpose of this study was to understand participants' 

program experiences. At the outset of each data collection, it was also clearly communicated to 

participants that their responses (and choice about whether or not to participate) would not affect 

their access to services or their ability to participate in the program. The evaluation team, and the 

program staff, simply wanted to better understand their program experience.

External Validity

Many of the factors that jeopardized external validity could not be addressed due to study design 

limitations. This evaluation only captured data from a single, multisite implementation of the 

Opportunity Reboot model, with subsample sizes by site being too small to test for differences. 

Additionally, the study sample is quite idiosyncratic in some ways (see Box 2 under Research 

Question 9 below), especially when compared to similarly aged youth across the state of 

Minnesota. Furthermore, with the single group non-experimental component of the preliminary 

impact evaluation, selection into treatment was not random, and the study did not include a control 

group.

Subsequent evaluations that seek to attain higher levels of evidence will need to employ more 

complex quasi-experimental evaluation designs that can rule out other explanations for change in 

the outcomes, test for generalizability, and account for the multiple treatment threats inherent in 

the Opportunity Reboot model approach (i.e., program enhancement model vs. standalone 

program).

Positive Disruption: Opportunity Reboot Model 82
Building on the implementation evaluation research questions, the Opportunity Reboot 

preliminary impact evaluation focused on six additional research questions. The sections below 

provide: (a) detailed methodological information about how each of these questions were 

answered drawing on the quantitative and qualitative data described above; and (b) a summary 

of the core findings.

Preliminary Impact 

Findings by Research Question

To answer this research question, participants’ scores on each of the short-term youth 

outcomes assessed in the Opportunity Reboot Youth Survey at baseline (Time 1) and endline

(Time 2) were compared using a series of paired t-tests. These tests allowed the evaluation 

team to establish whether change in these outcomes did occur; and, if it did, the direction, 

magnitude, and statistical significance of this change. The means on each of the short-term 

youth outcome scales were used to make these comparisons; additional information on the 

measures can be found in the Preliminary Impact Measures and Instruments section above. The 

paired t-test analyses discussed in this section were run using StataSE 13.

The results of the paired t-tests are summarized in Table 18, including a Cohen’s d measure of 

effect size. In general, an effect size of less than d = .20 is considered small but meaningful, d = 

.50 is considered medium, and d = .80 or greater is considered large (Cohen, 1992; Durlak, 

2009). 

§ The analyses showed that youth positively changed between baseline (Time 1) and endline

(Time 2) across all short-term outcomes, and that these changes were statistically significant 

with the exception of positive self-worth, future orientation, civic efficacy, and responsible 

decision making. 

§ Effect sizes for most of these changes were small.

§ These findings show that, on average, youth made significant and measurable gains on a 

myriad of important short-term outcomes including in areas of positive identity, socialemotional competencies, and skills for systems navigations, during the course of their 

participation in the Opportunity Reboot programs. 

To determine whether these changes were experienced across different demographic subgroups 

of youth, changes in short-term youth outcomes were examined by gender, sexual orientation, 

and race/ethnicity. Separate paired t-test models for male and female participants showed that 

both groups experienced positive change on all of the outcomes (see Table 19). 

Research Question 5

Did youth who participated in programs using the 

Opportunity Reboot model experience measurable and 

significant gains in short-term outcomes over the 

program year? Did all youth experience the benefits of 

program participation equally, or were program impacts 

experienced differentially by diverse demographic groups 

of youth?

Positive Disruption: Opportunity Reboot Model 83
Table 18 Paired t-Tests of Short-Term Youth Outcomes

Outcomes n

Means Paired 

t Cohen’s d

Baseline Endline

Positive Identity

Positive Self-Worth 181 4.10 4.15 -0.78 .06

Positivity in the Face of

Challenge 181 3.92 4.10 -3.06** .23

Goal-Setting Skills 183 4.02 4.18 -3.47*** .26

Future Orientation 172 4.19 4.23 -0.81 .06

Civic Efficacy 181 4.00 4.08 -1.35 .10

Social-Emotional Competencies

Self-Awareness 179 3.89 4.08 -3.41*** .25

Responsible Decision Making 180 3.92 4.04 -1.91 .14

Relationship Skills 181 3.83 3.97 -2.97** .22

Skills for Systems Navigation

Financial Literacy 182 13.03 13.87 -2.93** .22

Job-Seeking Skills 182 3.18 3.51 -2.71** .20

Resource Identification Skills 180 3.79 3.95 -2.59* .19

Notes. *p ≤ .05. **p ≤ .01. ***p ≤ .001. Baseline refers to the Time 1 administration of the Opportunity Reboot Youth Survey; endline refers to 

the Time 2 administration. All but two outcomes were scored using a 5-point scale ranging from 1 to 5, with higher scores indicating higher 

levels of the named outcomes. The Financial Literacy Index was created by summing participants’ scores on five items each scored on a 5-

point scale; thus, scores range from 0 to 20. Similarly, the Job-Seeking Skills Index was created by dichotomizing (0/1) responses to five items 

then summing across the five items. The Job-Seeking Skills Index has a possible range of 0 to 5. Paired t-tests employ listwise deletion; as such, 

youth whose surveys contained missing information on the data needed for these analyses were excluded.

Positive Disruption: Opportunity Reboot Model 84
Table 19 Paired t-Tests of Short-Term Youth Outcomes by Gender

Outcomes n

Means Paired 

t Cohen’s d

Baseline Endline

Positive Identity

Positive Self-Worth Male 88 4.20 4.21 -0.07 .01

Female 89 4.00 4.07 -0.82 .09

Positivity in the Face of 

Challenge

Male 90 3.97 4.11 -1.53 .16

Female 87 3.86 4.06 -2.67** .29

Goal-Setting Skills Male 91 4.06 4.19 -1.80 .19

Female 88 3.98 4.16 -2.88** .31

Future Orientation Male 87 4.18 4.24 -0.72 .08

Female 81 4.19 4.20 -0.09 .01

Civic Efficacy Male 89 4.01 4.05 -0.46 .05

Female 88 3.99 4.09 -1.19 .13

Social-Emotional Competencies

Self-Awareness Male 88 3.88 4.04 -2.00 .21

Female 87 3.90 4.10 -2.44* .26

Responsible Decision Making Male 88 4.00 4.06 -0.58 .06

Female 88 3.83 4.03 -2.30* .25

Relationship Skills Male 88 3.92 4.01 -1.29 .14

Female 89 3.75 3.90 -2.60* .28

Skills for Systems Navigation

Financial Literacy Male 90 13.44 13.72 -0.75 .08

Female 88 12.66 13.90 -2.86** .30

Job-Seeking Skills Male 90 3.03 3.17 -0.78 .08

Female 88 3.38 3.83 -2.71** .29

Resource Identification Skills Male 88 3.82 3.95 -1.41 .15

Female 88 3.76 3.92 -1.93 .21

Notes. *p ≤ .05. **p ≤ .01. ***p ≤ .001. Baseline refers to the Time 1 administration of the Opportunity Reboot Youth Survey; endline refers to 

the Time 2 administration. The Other race category includes youth who self-identified as Native Americans, as well as multiracial. These 

categories were combined due to insufficient sample size (which we defined as <15 individuals). All but two outcomes were scored using a 5-

point scale ranging from 1 to 5, with higher scores indicating higher levels of the named outcomes. The Financial Literacy Index was created by 

summing participants’ scores on five items each scored on a 5-point scale; thus, scores range from 0 to 20. Similarly, the Job-Seeking Skills 

Index was created by dichotomizing (0/1) responses to five items then summing across the five items. The Job-Seeking Skills Index has a 

possible range of 0 to 5. Paired t-tests employ listwise deletion; as such, youth whose surveys contained missing information on the data 

needed for these analyses (i.e., demographic variable, short-term youth outcome) were excluded.

Positive Disruption: Opportunity Reboot Model 85
However, these changes were statistically significant only for females (with the exception of 

positive self-worth, future orientation, and civic efficacy — where the changes were noted 

previously to not be statistically significant for the full sample as well; additionally, resource 

identification skills were was also not significant).

On average, participants who identified as heterosexual (or, “straight”) reported stronger goalsetting skills at endline; all other differences in short-term youth outcomes were not statistically 

significant for this group (see Table 20). Participants who identified as non-heterosexual (i.e., 

asexual, bisexual, gay or lesbian, pansexual, queer, questioning, or other) reported statistically 

significant gains in positivity in the face of challenge and self-awareness between baseline and 

endline.

There were four race/ethnicity groups in the sample with subsample sizes sufficient for 

analyses: Asian, Black, Hispanic/Latinx, and White. A fifth group — referred to as Other — was 

comprised of the remaining participants (more specifically, 10 of these Other group members 

identified as Native American; 13 as multiracial; and 3 declined to select any of the race options 

provided in the survey). The paired t-tests by race/ethnicity groups are summarized in Table 21. 

The difference in participants’ means at baseline and endline were not statistically significant for 

youth who identified as White or Hispanic/Latinx. Black youth reported statistically higher levels 

of responsible decision making, while Asian youth reported statistically higher levels of positivity 

in the face of challenge and financial literacy at endline. Youth who identified as another 

race/ethnicity (coded as Other) reported higher levels on seven short-term outcomes at endline: 

positive self-worth, positivity in the face of challenge, goal-setting skills, responsible decision 

making, self-awareness, relationship skills, and resource identification skills.

Youth who participated in programs using the Opportunity Reboot model did experience 

measurable and significant gains in critical short-term outcomes, however these gains were not 

experienced equally by all youth. The findings suggest that female participants may be 

benefiting more from their participation in Opportunity Reboot programs than their male 

counterparts. It also appears that youth with diverse racial and ethnic identities may experience 

the Opportunity Reboot programs differently, with youth in the Other race/ethnicity group 

experiencing the most significant positive growth between baseline (Time 1) and endline

(Time 2). It is unclear why certain subgroups of youth tended to benefit more in terms of the 

short-term outcomes than other youth. More research is needed to understand how youth with 

different identities experience the program features at each site. 

Positive Disruption: Opportunity Reboot Model 86
Table 20 Paired t-Tests of Short-Term Youth Outcomes by Sexual 

Orientation

Outcomes n

Means Paired 

t Cohen’s d

Baseline Endline

Positive Identity

Positive Self-Worth Heterosexual 75 4.22 4.19 0.29 .03

Non-Heterosexual 45 3.96 4.03 -0.57 .09

Positivity in the Face 

of Challenge

Heterosexual 76 4.00 4.01 -0.05 .01

Non-Heterosexual 45 3.84 4.16 -2.85** .42

Goal-Setting Skills Heterosexual 77 4.09 4.22 -2.22* .25

Non-Heterosexual 45 4.02 4.19 -1.59 .24

Future Orientation Heterosexual 73 4.21 4.25 -0.44 .05

Non-Heterosexual 41 4.25 4.27 -0.15 .02

Civic Efficacy Heterosexual 75 4.03 4.11 -0.83 .10

Non-Heterosexual 45 4.01 4.07 -0.43 .06

Social-Emotional Competencies

Self-Awareness Heterosexual 74 3.98 4.10 -1.61 .19

Non-Heterosexual 45 3.79 4.08 -2.24* .33

Responsible Decision 

Making

Heterosexual 76 3.99 4.11 -1.17 .13

Non-Heterosexual 45 3.82 3.96 -0.97 .15

Relationship Skills Heterosexual 75 3.88 4.01 -1.85 .21

Non-Heterosexual 45 3.78 3.88 -1.13 .17

Skills for Systems Navigation

Financial Literacy Heterosexual 76 13.84 14.22 -0.88 .10

Non-Heterosexual 45 12.87 13.89 -1.81 .27

Job-Seeking Skills Heterosexual 76 3.37 3.58 -1.41 .16

Non-Heterosexual 45 3.07 3.51 -1.61 .24

Resource 

Identification Skills

Heterosexual 75 3.87 4.04 -1.84 .21

Non-Heterosexual 45 3.73 3.93 -1.57 .23

Notes. *p ≤ .05. **p ≤ .01. ***p ≤ .001. Baseline refers to the Time 1 administration of the Opportunity Reboot Youth Survey; endline refers to 

the Time 2 administration. The Other race category includes youth who self-identified as Native Americans, as well as multiracial. These 

categories were combined due to insufficient sample size (which we defined as <15 individuals). All but two outcomes were scored using a 5-

point scale ranging from 1 to 5, with higher scores indicating higher levels of the named outcomes. The Financial Literacy Index was created by 

summing participants’ scores on five items each scored on a 5-point scale; thus, scores range from 0 to 20. Similarly, the Job-Seeking Skills Index 

was created by dichotomizing (0/1) responses to five items then summing across the five items. The Job-Seeking Skills Index has a possible 

range of 0 to 5. Paired t-tests employ listwise deletion; as such, youth whose surveys contained missing information on the data needed for 

these analyses (i.e., demographic variable, short-term youth outcome) were excluded.

Positive Disruption: Opportunity Reboot Model 87
Table 21 Paired t-Tests of Short-Term Youth Outcomes by Race/Ethnicity

Outcomes n

Means Paired 

t Cohen’s d

Baseline Endline

Positive Identity

Positive Self-Worth Asian 52 4.33 4.30 0.34 .05

Black 24 4.28 4.60 -2.03 .41

Hispanic 22 3.95 4.11 -0.98 .21

White 24 3.68 3.43 1.47 .30

Other 26 3.71 4.13 -2.32* .46

Positivity in the Face of 

Challenge

Asian 53 4.01 4.21 -2.03* .28

Black 25 4.15 4.27 -0.77 .15

Hispanic 23 3.94 4.29 -2.04 .42

White 23 3.52 3.65 -0.76 .16

Other 25 3.47 4.04 -3.05** .61

Goal-Setting Skills Asian 53 4.07 4.18 -1.52 .21

Black 25 4.31 4.35 -0.45 .09

Hispanic 23 4.07 4.30 -1.39 .29

White 24 3.59 3.77 -1.59 .32

Other 26 3.68 4.17 -2.42* .48

Future Orientation Asian 49 4.22 4.31 -1.09 .16

Black 20 4.47 4.38 0.55 .12

Hispanic 22 4.08 4.24 -1.29 .27

White 24 3.90 3.79 0.70 .14

Other 25 4.21 4.40 -1.13 .23

Civic Efficacy Asian 53 3.91 3.93 -0.18 .03

Black 24 4.04 4.35 -1.59 .32

Hispanic 22 4.11 4.14 -0.12 .03

White 24 3.90 3.84 0.31 .06

Other 22 4.01 4.36 -1.60 .32

Social-Emotional Competencies

Self-Awareness Asian 51 3.71 3.89 -1.80 .25

Black 25 3.98 4.13 -0.91 .18

Hispanic 23 4.03 4.20 -0.99 .21

White 23 4.01 3.99 0.23 .05

Other 25 3.81 4.24 -2.19* .44

Responsible Decision Making Asian 52 3.87 3.98 -1.23 .17

Black 24 3.83 4.38 -2.72* .55

Hispanic 22 4.14 4.00 0.65 .14

White 24 3.88 3.75 0.90 .18

Other 25 3.92 4.20 -1.10 .22

Relationship Skills Asian 53 3.78 3.90 -1.78 .24

Black 23 3.67 3.89 -1.75 .37

Hispanic 22 4.05 4.06 -0.16 .03

White 24 3.72 3.80 -0.70 .14

Other 26 3.75 4.16 -2.19* .43

Positive Disruption: Opportunity Reboot Model 88
Table 21 Continued Paired t-Tests of Short-Term Youth Outcomes by 

Race/Ethnicity

Outcomes n

Means Paired

t Cohen’s d

Baseline Endline

Skills for Systems Navigation

Financial Literacy Asian 53 12.85 13.83 -2.01* .28

Black 24 12.58 13.13 -0.88 .18

Hispanic 23 14.65 15.78 -1.42 .30

White 24 12.08 12.54 -0.60 .12

Other 25 12.24 13.72 -1.38 .28

Job-Seeking Skills Asian 53 3.19 3.47 -1.18 .16

Black 25 3.48 4.00 -1.91 .38

Hispanic 22 3.05 3.68 -2.13 .45

White 24 3.13 3.50 -1.09 .22

Other 25 2.68 3.16 -1.44 .29

Resource Identification Skills Asian 53 3.72 3.79 -0.71 .10

Black 23 3.74 3.99 -1.38 .29

Hispanic 23 4.09 3.94 0.81 .17

White 23 3.87 3.99 -0.79 .16

Other 25 3.48 4.07 -2.38* .48

Notes. *p ≤ .05. **p ≤ .01. ***p ≤ .001. Baseline refers to the Time 1 administration of the Opportunity Reboot Youth Survey; endline refers to 

the Time 2 administration. The Other race category includes youth who self-identified as Native Americans, as well as multiracial. These 

categories were combined due to insufficient sample size (which we defined as <15 individuals). All but two outcomes were scored using a 5-

point scale ranging from 1 to 5, with higher scores indicating higher levels of the named outcomes. The Financial Literacy Index was created by 

summing participants’ scores on five items each scored on a 5-point scale; thus, scores range from 0 to 20. Similarly, the Job-Seeking Skills 

Index was created by dichotomizing (0/1) responses to five items then summing across the five items. The Job-Seeking Skills Index has a 

possible range of 0 to 5. Paired t-tests employ listwise deletion; as such, youth whose surveys contained missing information on the data 

needed for these analyses (i.e., demographic variable, short-term youth outcome) were excluded.

Positive Disruption: Opportunity Reboot Model 89

Research Question 6

Controlling for demographic, background, and 

implementation factors, what are the predictive 

associations between the youths’ experiences of the 

Opportunity Reboot model features and their short-term 

outcomes? 

Building on the paired t-tests, multiple regression models were used to examine how youth’s 

experiences of the Opportunity Reboot model features — positive relationships with mentors 

within their programs, positive relationships with mentors outside their programs, and 

experience of receiving individualized goal supports and career pathways supports — impacted 

change in the targeted short-term youth outcomes between baseline and endline. 
A number of covariates were included in the regression models that were hypothesized to be 

related to the short-term youth outcomes. Informed by the paired t-test findings (see above), 

gender and race/ethnicity were included as control variables. Sexual orientation was excluded 

because the results of the paired t-tests suggested that the change between baseline and 

endline were similar for both groups. In addition, the following controls were also included:

Age was controlled to account for the variation in short-term outcomes that may stem 

from the large age range of participants (14 to 24 years). This range encompasses several 

developmental stages spanning from early and middle adolescence to young adulthood; 

thus it is reasonable to infer that youth may have very different program experiences and 

needs in their Opportunity Reboot programs. Post-hoc analyses were also completed 

with age specified as a categorical variable. Age was dummy coded into three groups: 

14-17, 18-20, and 21-24. The age group 18-20 served as the reference category. 

A single-item mental health indicator was also included, as mental health issues were 

directly and inversely associated with many of the Opportunity Reboot short-term youth 

outcomes. As part of the survey, youth participants were asked: “Are you experiencing 

mental health issues (such as anxiety, depression, or other issues)?” The response option 

was Yes or No. 

Program dosage, as described in the Implementation Evaluation Instruments section, 

was reported on for each youth participant by a staff member from the community 

partner organization. Participants’ program dosage was scored as low, medium, or high 

in accordance with criteria pre-specified by the partners in consultation with the 

evaluation team. Among the overall sample of participants, 64% were assessed by 

program staff to have received high dosage, 28% with medium dosage, and 8% with low 

dosage. It was hypothesized that participants who showed up more often, and/or 

participated in a higher volume of activities, would also experience stronger effects from 

their time at program sites.

As with program dosage, program staff were asked to assess participant engagement 

levels on a 3-point scale (high, medium, and low). Among the overall sample of 

participants, 59% were assessed by program staff to have high levels of engagement, 

30% with medium engagement, and 11% with low engagement. Like program dosage, it 

was hypothesized that participants who were more engaged in their program would 

experience stronger effects — that is, reap more benefits — from their time at their 

program sites.

To understand the relationship between Opportunity Reboot features and youth short-term 

outcomes, the evaluation team specified several multiple regression models. Two models were 

run for each outcome: first a model with only covariates, and then a second model with both 

covariates and the model features. By comparing the two steps’ R2s (i.e., the percentage of 

observed variance that can be explained by the model’s independent variables), the explanatory 

power of the model features could be calculated. Due to the small sample sizes of these models, 

a threshold for statistical significance at p < .10 was used. The regression models are 

summarized in Table 22. 

Positive Disruption: Opportunity Reboot Model 90
As previously discussed in the Measures and Instruments subsection of the Opportunity Reboot 

Youth Survey methodological section, only within program mentoring relationships items were 

asked at both baseline and endline surveys; for this reason, baseline-endline change (Δ), in 

within program mentoring relationships were used in the regression models (the other three 

model features were assessed only in the endline youth survey). These regression models were 

run using StataSE 13. 

Covariates 

§ Of the covariates included in these models, age was significantly associated with positive 

change in positivity in the face of challenge, goal-setting skills, and responsible decision 

making. 

§ Females were more likely than males to experience positive change in financial literacy.

§ Asian youth were less likely than White participants to experience positive change in jobseeking skills. 

§ Black youth were less likely than White participants to experience positive change in goalsetting skills; and youth in the Other race category were more likely than White participants 

to experience positive changes in positivity in the face of challenge, self-awareness, 

responsible decision making, and relationship skills. 

§ Youth with greater engagement during the Opportunity Reboot program (i.e., engagement) 

experienced greater positive change in positivity in the face of challenge.

§ Youth who participated in the Opportunity Reboot program more often (i.e., dosage) 

experienced greater positive changes in goal-setting skills, future orientation, and 

relationships skills. 

Opportunity Reboot Model Features

§ As for the Opportunity Reboot model features, both positive mentoring relationships within

and outside of the program appear to be critical to youths’ growth on the short-term 

outcomes. 

• Participants who strengthened their mentoring relationships with staff within the 

program reported significant gains on all three of the skills for systems navigation 

outcomes (i.e., financial literacy, job-seeking skills, and resource identification skills).

• Youth who forged new mentoring relationships with adults outside of the program 

saw positive gains in their orientation towards the future, civic efficacy, selfawareness, responsible decision making, relationship skills, financial literacy, jobseeking skills, and resource identification skills.

§ The career pathways supports feature was significantly associated with positive changes in 

youth’s job-seeking skills, suggesting that this addition to Opportunity Reboot programs had 

a positive effect on youth’s job-seeking skills. 

Positive Disruption: Opportunity Reboot Model 91
Table 22 Multiple Regression Models on the Impact of Opportunity 

Reboot Model Features on Change in Short-Term Youth Outcomes

Positive Identity

Variables Δ Positive 

Self-Worth

Δ Positivity in 

Face of 

Challenge

Δ GoalSetting 

Skills

Δ Future 

Orientation

Δ Civic 

Efficacy

Covariates

Age 0.07

(.00)

0.27+

(.04)

0.38*

(.08)

0.13

(.01)

0.02

(.00)

Female 

(REFERENCE: MALE)

0.05

(.00)

-0.00

(.00)

0.01

(.00)

-0.06

(.00)

0.08

(.01)

Race/Ethnicity 

(REFERENCE: WHITE)

Asian 0.12

(.00)

-0.19

(.01)

-0.33

(.03)

-0.04

(.00)

-0.11

(.00)

Black 0.20

(.03)

-0.14

(.01)

-0.32*

(.06)

-0.10

(.01)

0.00

(.00)

Hispanic 0.23

(.03)

0.03

(.00)

0.05

(.00)

0.16

(.02)

-0.05

(.00)

Other 0.22

(.03)

-0.00

(.00)

0.01

(.00)

0.03

(.00)

0.22+

(.04)

Mental Health 0.13

(.01)

0.20

(.03)

0.08

(.01)

-0.18

(.03)

-0.01

(.00)

Engagement -0.18

(.02)

0.26+

(.04)

0.05

(.00)

-0.06

(.00)

-0.10

(.01)

Participant Dosage 0.29+

(.04)

-0.02

(.00)

0.29+

(.04)

0.41**

(.09)

0.05

(.00)

Independent Variables

Δ Within Program Mentoring 0.19

(.03)

0.09

(.01)

0.12

(.01)

0.08

(.01)

0.16

(.02)

Outside of Program Mentoring -0.13

(.01)

0.09

(.01)

0.16

(.02)

0.25*

(.06)

0.45***

(.16)

Individualized Goal Supports -0.07

(.00)

-0.12

(.00)

-0.11

(.00)

0.04

(.00)

-0.29

(.02)

Career Pathways Supports 0.28

(.02)

0.18

(.01)

0.14

(.01)

0.13

(.01)

0.13

(.01)

Adj. R2

CONTROLS .089 .049 .128 .075 -.019

Δ Adj. R2

with IVs .031 -.015 .012 .107 .185

Partial η2

IVs .072 .026 .045 .067 .215

Model n 91 91 92 89 92

Notes. Standardized coefficients (β) are reported; effect size (partial η2) in parentheses. +p < .10, *p ≤ .05. **p ≤ .01. ***p ≤ .001. The Other 

race category includes youth who self-identified as Native Americans, as well as multiracial. These categories were combined due to 

insufficient sample size (which we defined as <15 individuals). Multiple regression employs listwise deletion; as such, youth whose surveys 

contained missing information on the data needed for these analyses were excluded.

Positive Disruption: Opportunity Reboot Model 92
Table 22 Continued Multiple Regression Models on the Impact of 

Opportunity Reboot Model Features on Change in Short-Term Youth 

Outcomes

Social-Emotional Competencies Skills for Systems Navigation

Variables Δ SelfAwareness

Δ Responsible 

Decision 

Making

Δ Relationship 

Skills

Δ Financial 

Literacy

Δ Job-Seeking 

Skills

Δ Resource Id 

Skills

Covariates

Age 0.16

(.01)

0.28+

(.04)

-0.02

(.00)

-0.10

(.01)

0.05

(.00)

0.14

(.01)

Female

(REFERENCE: MALE)

0.03

(.00)

0.03

(.00)

-0.06

(.00)

0.21*

(.05)

0.09

(.01)

0.04

(.00)

Race/Ethnicity 

(REFERENCE: WHITE)

Asian -0.02

(.00)

-0.08

(.00)

0.06

(.00)

0.04

(.00)

-0.39*

(.04)

-0.24

(.02)

Black -0.10

(.01)

0.09

(.00)

0.14

(.01)

-0.01

(.00)

-0.21

(.03)

-0.11

(.01)

Hispanic 0.08

(.00)

-0.01

(.00)

0.08

(.00)

-0.01

(.00)

-0.13

(.01)

-0.24

(.03)

Other 0.26+

(.05)

0.24+

(.04)

0.22+

(.04)

0.04

(.00)

-0.06

(.00)

0.04

(.00)

Mental Health 0.11

(.01)

0.10

(.01)

-0.01

(.00)

-0.14

(.02)

-0.03

(.00)

0.08

(.01)

Engagement 0.06

(.00)

-0.02

(.00)

-0.01

(.00)

-0.01

(.00)

0.08

(.00)

0.12

(.01)

Dosage 0.08

(.00)

-0.01

(.00)

0.35*

(.06)

0.11

(.01)

0.06

(.00)

-0.00

(.00)

Independent Variables

Δ Within Program 

Mentoring

0.04

(.00)

0.01

(.00)

-0.05

(.00)

0.28*

(.07)

0.26*

(.06)

0.24+

(.05)

Outside of Program 

Mentoring

0.27*

(.06)

0.24+

(.04)

0.36**

(.10)

0.24*

(.05)

0.21+

(.04)

0.23+

(.04)

Individualized Goal 

Supports

-0.09

(.00)

-0.30

(.02)

-0.20

(.01)

-0.02

(.00)

-0.55**

(.09)

-0.25

(.02)

Career Pathways 

Supports

0.06

(.00)

0.15

(.01)

0.15

(.01)

-0.18

(.01)

0.66***

(.13)

0.21

(.01)

Adj. R2

CONTROLS .043 .036 .055 .001 .087 -.010

Δ Adj. R2

with IVs .029 .008 .063 .123 .098 .081

Partial η2

IVs .064 .076 .122 .128 .320 .118

Model n 88 93 92 93 93 91

Notes. Standardized coefficients (β) are reported; effect size (partial η2) in parentheses. +p < .10, *p ≤ .05. **p ≤ .01. ***p ≤ .001. The 

Other race category includes youth who self-identified as Native Americans, as well as multiracial. These categories were combined due 

to insufficient sample size (which we defined as <15 individuals). Multiple regression employs listwise deletion; as such, youth whose 

surveys contained missing information on the data needed for these analyses were excluded.

Positive Disruption: Opportunity Reboot Model 93
§ Individualized goal supports were negatively associated with positive changes in youth’s 

job-seeking skills. This negative association may simply be a reflection of a core principle 

adopted by all partner sites: the need to meet opportunity youth where they are at. Youth 

who require the most intensive individualized goal supports are often the youth who have 

more urgent needs that must be met before focusing on job-seeking skills. This includes 

addressing chemical and mental health issues, securing stable housing, and — perhaps —

even focusing on education before focusing on employment.

§ The four model features collectively had moderate to large effect sizes on at least one 

outcome in each overarching outcome category: positive identity (civic efficacy, partial η2 = 

.215); social-emotional competencies (relationship skills, partial η2 = .122); and skills for 

systems navigation (financial literacy, partial η2 = .128; job-seeking skills, partial η2 = .320; 

resource identification skills, partial η2 = .118). η2s or partial η2s of .02 are considered small; 

η2 ≅ .13 are considered moderate, and η2 ≅ .26 or greater are considered large (Cohen et al., 

Cohen, West, & Aiken, 2003). 

The regression model results yielded two main takeaways: (1) program sites excel at delivering 

on their core services: namely, providing participants with tangible resources as well as teaching 

them financial literacy and job-seeking skills; and, (2) the models highlight just how essential 

outside-of-program mentors were to participants’ success in their respective programs — and 

potentially to their success in their longer-term post-program lives. Programs that are seeking 

ideas for how to improve their programming may consider providing participants with more 

targeted guidance on how to locate outside-of-program mentors, and how to cultivate these 

relationships.

§ The evaluation team also ran the regression models with age as a categorical variable (14-17, 

18-20, and 21-24) to determine whether there were differential program impacts among 

various age groups. Model results show that program impacts did not differ for youth of 

different age groups for most outcomes with the exception of positivity in the face of 

challenge and responsible decision-making.

§ Opportunity youth (ages 14-17) reported fewer changes in positivity in the face of challenge 

and responsible decision-making relative to youth ages 18-20 and ages 21-24. 

Thus, there were no meaningful differential age impacts for most of the study’s outcomes. It 

makes intuitive sense that younger youth demonstrated much smaller changes than older youth 

in terms of positivity in the face of challenge and responsible decision-making as these 

outcomes are highly influenced by one’s lived experiences.

Positive Disruption: Opportunity Reboot Model 94
Positive Disruption: Opportunity Reboot Model 95

Research Question 7

In their own words, how did participants describe experiencing the 

features of the Opportunity Reboot model at their program sites?

The impact narratives were designed to complement the broader research and evaluation design by 

deepening understanding of a small group of Opportunity Reboot participants, their experience in the 

program, and how their experiences impacted their lives. The qualitative impact narrative interviews 

and focus groups were recorded and transcribed. The analysis of this qualitative data used a grounded 

theory approach, facilitated by NVivo software (Braun & Clarke, 2013; Charmaz, 2008). Themes were 

compiled through the use of an open coding structure, guided by the Opportunity Reboot model 

features and short-term outcomes identified in the Opportunity Reboot logic model. Two or more 

researchers participated in data analysis of each site and across sites to determine a consensus on the 

coding structure. 

To answer this research question, the major qualitative themes tied to each of the four features of the 

Opportunity Reboot model are presented. These are paired with quotes from participants to illuminate 

the core ideas.

When participants talked about mentoring relationships they generally referred to program staff 

including case managers, teachers, and youth workers. These mentoring relationships were 

integral to participant success in the program. Participants shared 

numerous accounts of what these relationships meant to them, 

what they looked like, and how they helped

Positive Mentoring Relationships: 

Relationships Laid the Foundation for Participant Success

She actually gets 

us...instead of most 

teachers pushing us 

away if we’re having a 

bad day.

“ them grow. In a close examination of the impact narrative data, it 

became clear that participants talked about all five critical 

components of what are being described in youth development 

literature as developmental relationships (see Pekel et al., 2015). 

Developmental relationships are defined as close connections 

through which young people discover who they are, gain abilities 

to shape their own lives and learn how to interact with and 

contribute to the world around them. These connections 

between youth and adults or youth and their peers powerfully 

and positively shape young people’s growth and development. 

Past research has highlighted relationships as the critical active ingredient of strong youth 

development programs (Li & Julian, 2012; Liang & Rhodes, 2007; Pekel et al., 2015). Research 

has linked developmental relationships with positive outcomes in families and schools, including 

motivation to learn, emotional competence, prosocial behavior, effortful control, and personal 

responsibility (Pekel et al., 2015).The five core elements of a developmental relationship are: 

express care, challenge growth, provide support, share power, and expand possibilities (see 

Appendix F for an overview of Developmental Relationships Framework).

After analyzing data and noting the emphasis participants placed on relationships as key to 

growth and success, data specifically regarding mentoring relationships were re-analyzed using 

the Developmental Relationships Framework as a lens. What youth experienced through 

mentoring relationships is explored in each of the five elements below. 
Past research has highlighted relationships as the critical active 

ingredient of strong youth development programs (Li & Julian, 

2012; Liang & Rhodes, 2007; Pekel et al., 2015). Research has 

linked developmental relationships with positive outcomes When I first came to 

GAP, I didn’t speak 

English very well, she 

asked me, “What do you 

want to do in your 

future?” and I said, “I 

want to be English 

teacher, but I don’t 

think I can do that.” And 

she said, “No, I believe 

you can do, because I 

will be your mentor."

in families and schools, including motivation to learn, emotional 

“

competence, prosocial behavior, effortful control, and personal 

responsibility (Pekel et al., 2015).The five core elements of a 

developmental relationship are: express care, challenge growth, 

provide support, share power, and expand possibilities (see 

Appendix F for an overview of Developmental Relationships 

Framework).

After analyzing data and noting the emphasis participants placed on 

relationships as key to growth and success, data specifically 

regarding mentoring relationships were re-analyzed using the 

Developmental Relationships Framework as a lens. What youth 

experienced through mentoring relationships is explored in each of 

the five elements below. 

Express Care. For participants in difficult circumstances, being able 

to trust program staff was critical. Mentors build relationships

with young people by providing space for them to share important information, or by providing 

information they could trust. One participant described their trusting relationship this way: “[You can] 

talk to her about something you can’t talk to your family about, you can tell her and trust that she 

won’t tell your family, unless it becomes a big problem.”

Youth described how they appreciated when mentors encouraged or praised them. One participant 

quoted their mentor, who would tell them, “You got this.” While another participant discussed how his 

mentor noticed and praised him for his efforts: “He said I was his best worker...I work harder...and I 

don’t complain about what I do.” Others shared that their mentors give them pep talks, and 

encouraged them by reinforcing their belief that they could reach their goals. 

It was important for participants that their mentors were kind to them and fun to be around. In 

interviews, participants described how their mentors joked around with them, but the jokes were 

mutually fun. In general, mentors were also described as “showing compassion,” “so nice and kind,” 

and “patient.” 

In addition, mentors made efforts to get to know participants. “They have an ability to connect with 

people in our circumstances,” noted one participant. Mentors got to know participants through 

intentionally spending time with them, even when busy, or asking participants about their lives, 

aspirations, and hobbies. Participants appreciated when mentors expressed care by checking in with 

them and making sure they had what they needed.

Challenge Growth. Participants discussed how mentors expected their best, and pushed them to try 

harder to reach their goals. One participant described it this way: “They always pretty much encourage 

students to do their work and get it done, keep pushing forward.” Another said, “She was always trying 

to help them through and trying to get them to realize what they need to do and help them push 

through that.” Some of the ways that mentors pushed them included encouragement to go to college, 

complete their goals, and being brave when faced with challenges. 

Mentors also shared their own stories of overcoming challenges for the purpose of motivating young 

people. One participant said, “They share and talk about their experiences and they keep motivating 

me, that you have to have a goal.”

Positive Disruption: Opportunity Reboot Model 96
Lastly, mentors challenged young people’s growth by holding them accountable for their 

progress and actions. Some participants who were a part of a program’s housing initiative 

shared that mentors would hold them accountable for “showing up” and “being a good tenant.” 

Additional expectations of mentors included being respectful, following the rules, and working 

cooperatively with other young people and staff. 

Provide Support. Participants described various ways that mentors had provided them help and 

support through their program. Mentors did this through academic support, guiding young 

people through problems, and goal setting. “She’s kinda like everybody’s therapist,” noted one 

participant. It was also impactful for participants when mentors helped them with instrumental 

needs outside of the program, such as rent, food, and gas: “She’ll help you with stuff that 

doesn’t involve [the program].” 

Many staff members in Opportunity Reboot programs taught or guided young people in 

academic pursuits. Participants shared several examples of new concepts or skills they learned 

from mentors. “[Staff member] has just helped me learn how to do homework better and be 

able to stay on task,” said one participant. Many participants described feeling stuck when faced 

with a skill they were not yet proficient in, and how a mentor stepped in to help: “[Staff 

member’s] teaching was important for completing projects.” 

Share Power. While this element in the Developmental Relationships framework was expressed 

the least, participants shared some salient examples of how staff members shared power with 

them. In particular, participants appreciated mentors that would “treat them like adults” and 

work collaboratively with them: “She’s very flexible with our schedule...she works with us.” 

Another participant described how being part of the program felt differently than being in 

school as a child; mainly, this was due to how staff members treated them as adults with 

autonomy and respect. 

Other ways that staff members shared power was through providing participants choices on 

activities in the program, or listening to young people’s ideas and valuing their decisions: “She’ll 

be like, ‘Well, here’s a list,’ but if you don’t want to do anything on the list, you can always do 

something else.”

Positive Disruption: Opportunity Reboot Model 97
Expand Possibilities. Most of how staff expanded possibilities for participants is described in the 

following sections about career pathways and individualized goal supports. However, a few 

participants shared how mentors expanded their possibilities through their mentorship and 

teaching, or by connecting participants to resources beyond their teaching. One participant 

noted that they found a mentor’s teaching to be “inspirational” when thinking about the future, 

while another noted that their teacher had helped learn about new supports. “She gives me a 

lot of websites you can go to [for academic help].” Others talked about the adults they viewed 

as mentors broadened their knowledge. One participant gave an example of how they had 

learned about an energy bill that allowed the county to help pay for installing solar panels on 

houses. This was exciting for this participant who was interested in green energy solutions. 

Participants noted that their success depended, in part, on staff “making things happen” or 

helping plan and execute their goals and connect them to new ideas. “[They] even give us a 

calendar for what we’re gonna do,” said one participant. Mentors’ careful planning of events 

and activities facilitated the type of learning and growth that participants experienced in their 

program. One participant noted that “I think they put their students first. No matter what, they 

are always putting their students first.”

Tracking Goals and Habits. As noted in participants’ relationships with mentors, staff provided 

support of goal setting and achievement for young people. Participants named specific ways 

that staff facilitated goal setting by the use of setting goals and tracking strategies. “We go over 

our [goals] and make sure we’re doing our checklists and stuff like that,” said one participant. In 

addition to tracking goals, participants also reflected on their goals and how their progress 

would impact their future.

One strategy utilized by staff and participants included habit tracking, where participants 

learned to track their habits and how they contributed to their goals each day. “[Habit trackers] 

helped me realize what habits I should probably do more or which habits I should totally cut out 

of my life,” noted one participant. 

Individualized Goal Supports:

Finding the Right Resources to Support Participants’ Unique Objectives 

“

...I told [staff] that I wanted 

to be an architect. They told 

me, “I can help you out, I 

know this college,” or “I 

know this teacher.”

Connecting goals to college or future careers. Youth discussed how 

staff helped them see how work experience, coursework, and 

training could help them achieve their goals. For youth who are 

incarcerated, this allowed them to envision a future beyond release 

from detention. One participant said, “[It helps] prepare us for once 

we get released from here, and help us have, you know, the skills to 

obtain jobs in the community and to obtain enrollment in colleges 

and just prepare us for what’s ahead. Figure out what’s ahead if we 

don’t have a clear picture of what’s ahead, too.” Some participants 

had created a goal of attending a certain college or entering 

a particular career field. Staff were able to tailor their support to these participants by looking up 

information on colleges of their choice, explaining what would need to be accomplished to be 

successful in a career, or encouraging participants to stick to their goal, even if it is difficult. This 

individualized support was often unique to the program: “...some places, you don’t get one-on-one 

time with other people [such as in this program], they just work with everybody as a whole.”

Positive Disruption: Opportunity Reboot Model 98
Facilities and resources. For many, the reality of working towards a goal of entering a college or 

career field required resources and financial support. Several participants shared that it was the 

free facilities, staff time, and resources that helped them be successful in achieving their goals: 

“The facility is good, it’s free, you can use the computer after school. But, in high school, you 

don’t even have the opportunity to use the computer after school, you just gotta leave.” 

Another said, “If you want to apply for a job, you can call the teacher and they will help you.”

Participants also described the need for financial support. As one participant noted, it was 

necessary to “make sure I have enough financial support that I’m not gonna go into debt.” Staff 

helped to “reach out to the community and get those resources that we need to be a successful 

human being.” Some other financial support that was provided to participants included 

information on scholarships and savings programs. 

Career Pathways Supports:

Opportunities to Build Skills Today 

and Dream About the Future

Hands-on work experience. Many participants 

worked in a job through their program that 

helped them build workplace skills or specialized 

training. Some participants shared that it felt like 

they were “getting paid to learn” through these 

experiences, with the added benefit of adding job 

experience to their resumes. Many participants 

were also provided specialized training and 

certifications, such as OSHA (Occupational Safety and Health Administration) compliance, CPR 

(Cardiopulmonary Resuscitation) training, and CNA (Certified Nursing Assistant) certification. 

Participants were eager to finish these trainings, due to their direct link with potential job 

opportunities. One participant said, “We have to learn to fix a computer and if we finish, we get the 

certificate, and they said that if we have it, they can help find a job for us.” 

Some participants described these work experiences as an opportunity to “get ahead,” especially 

when others in their age group may not have had these opportunities. “I can only name four 18-

year-olds that I know who have gotten the qualifications I’ve gotten...I’m already a step above my 

age group.” 

Other participants may not have had a paid job through their program, but volunteered instead. 

They described how their volunteer experiences had also been a way to prepare for future careers: 

“We get to work with the people...I have a chance to volunteer in long-term care, so I feel like [I had] 

more experience.”

Leadership and life skills. Participants also described the benefit of learning leadership and life skills, 

beyond job-specific training. One participant shared that they felt they solved “real-life problems” 

through their program, such as how to read and manage an electric bill or utilize environmentallyfriendly and efficient tools in their home. Another said, “We work on different skills, and our main 

skill was teamwork.” Others echoed the sentiment that collaboration with others helped them grow 

in their leadership abilities: “...they just prepare you for the leadership skills...so you feel confident to 

work with other people...you have more confidence and more experience about that.”

Positive Disruption: Opportunity Reboot Model 99
College and career preparation. Participants described several ways that their programs offered 

preparation for college. These included college visits over the course of their participation in a 

program. “It’s a good option, because I look at different colleges and see my options of where I 

want to go,” noted one participant. Programs also provided information on how to apply for 

college, including notification of deadlines, and how they could academically prepare for college. 

One participant shared, “They help you with college opportunities as well, and they explain stuff 

there, too.” Another participant shared that even though they were not interested in colleges 

early in their high school career, they appreciated that staff exposed them to ideas for colleges. 

If I weren’t in [Tri-City 

Bridges], I’d be, like, in hell 

right now. I wouldn’t have 

any idea what to do since 

my parents aren’t really a 

help, but thanks to the 

[Tri-City Bridges] office, I 

have more opportunities. 

They know what to do, 

while I don’t.

“

For some participants, college credits earned through their 

program allowed them to begin their college careers with some 

credits already completed. Participants were able to articulate 

the link they saw to their college coursework and beginning their 

college experience with an advantage: “...with the amount of 

credits I already have...instead of me having to wait at the back 

of the line to get accepted before anyone, I’d probably be the 

first one they’d accept, because I already have some college 

credits. My transcript for college looks good so far.”

In addition to college preparation, participants also experienced 

a variety of opportunities to hone skills for obtaining a job. 

Interviews and job application help were discussed by a large 

number of participants, with some noting that they got a job with 

the help of a staff member who helped them to apply. Other 

activities that helped included mock interviews, resume and 

application workshops, and guidance on how to find jobs. One 

participant said, “It’s important for me, like before I interview, I 

have more experience. If I go to the real interview, it will make me less nervous, like I can’t 

speak. I have experience already. I can do a good job.” Many young people also participated in 

activities through their programs to learn about different career paths and what they could do to 

prepare for them. Others shared what it was like to participate in a career exploration class: 

“[It’s] making me think about how my future could be, like with different jobs, especially through 

the career exploration class I have at the end of the day, ‘cause we’ve done a lot of field trips to 

these different job sites.” 

Positive Disruption: Opportunity Reboot Model 100
Field trips. Several of the cross-sector partnerships that 

participants experienced were in the form of interactive field 

trips. Many of these field trips included a career component by 

exposing participants to the work of a variety of fields that they 

learned about in school. Others went on college visits to 

provide participants the opportunity to see what college is like 

and learn about what is necessary to be accepted at various 

colleges. Some experiences allowed participants the 

opportunity to meet professionals in different careers and 

watch them work. Other field trips provided cultural education 

and enrichment for participants. One such experience was a 

trip to a buffalo farm, where participants learned about 

Indigenous peoples’ “sacred relationships to nature, animals,

Impactful Cross-Sector Partnerships

Connecting with New People, Places, and Skill-Building Opportunities

and the creator.” One participant described this as “such an important field trip to the whole 

experience of the program.” Another field trip was to a camp, where participants interacted 

with people from different countries, learning together for two weeks.

The field trips are so 

inspirational, cause it 

gives me a bigger vantage 

point of where I could go 

in the future for jobs.

“

For reasons of length and site-level variability, participants were not asked about their 

interactions with organizations that partnered with their program site. However, these 

experiences surfaced naturally in the qualitative impact narratives as youth highlighted the 

opportunities to experience new places, people, and skills as a particularly memorable aspect of 

their time at their programs.

Speakers. Some programs invited speakers as part of their relationship to the broader 

community: “[The program] tries to get involved with the community to bring in speakers.” 

Other participants shared that the speakers provided information on career pathways, cultural 

knowledge, or inspiration for the future. They enjoyed having these speakers expand their 

worldview. One participant said they enjoyed a particular speaker who talked to them about 

“staying in touch with your culture and not losing it.”

Work sites and volunteering. For some programs, placement in 

a school allowed them to tap into the deep well of cross-sector 

partnerships the district has established over the years. Other 

programs were able to successfully partner with businesses 

and organizations that gave young people the opportunity to 

work or volunteer. The variety of volunteer opportunities was 

noted by one participant, who described “so many kinds” of 

ways they could volunteer through participation with their 

program. 

“

It’s always good to have 

somebody that’s been 

through it and that has 

that job or has been 

through this experience, 

so they can relay their 

story on to us, so we 

know more about that 

concept.

Positive Disruption: Opportunity Reboot Model 101
Research Question 8

In their own words, how did youth describe the impact of 

Opportunity Reboot on their lives?

The impact narrative protocols flowed from a discussion of the four Opportunity Reboot model 

features (see Research Question 7) into a conversation about how opportunity youths’ 

participation in programs using the Opportunity Reboot model have impacted their lives. These 

data were gathered both through the semi-structured interview protocols and the “Three-Act 

Play” activity used as part of the focus group protocols.

The impact portion of these interviews and focus groups was coded using grounded theory, and 

thus was not limited to focusing only on the same short-term youth outcomes targeted in the 

Opportunity Reboot Youth Survey. That said, there was strong alignment between many of the 

qualitative impact themes that emerged and the three overarching categories used to organize 

the short-term youth outcomes named in the logic model (see Figure 2); thus, the findings for 

this research question are organized around these same three categories as well as one 

additional category that emerged: Commitment to Learning. Many of the specific qualitative 

impact themes align with the quantitative outcome metrics, adding a rich layer of nuance 

embedded in youths’ own words. 

The major themes for each of the four overarching qualitative impact categories are presented 

first. This is followed by a summary of how participants explicitly represented how their 

relationships with school and work had (or had not) changed over the course of the program via 

their drawings and supporting narrative around the “Three-Act Play” activity. 

Positive Disruption: Opportunity Reboot Model 102
Confidence about the future. For many participants, feeling 

that they would be able to be successful in the future was a 

positive outcome of their programs. One participant said, “It 

helps me realize how far I’ve made it and how far I can go, or 

how more far I can go.” Another participant described that they 

felt confident in what they needed to do for success in the 

future: “I know my responsibility. I know my goal...I know what 

I gotta do for the future.” 

More comfortable with themselves. One participant shared 

that they felt “more confident in myself now.” Others cited a 

higher level of confidence overall. In particular, they felt 

confident interacting with others and being less shy. One 

participant discussed that they are now “more focused and in 

tune with myself.” Being confident in their own skin led to 

greater confidence when talking and working with others.

Positive Identity

Ability to speak in front of others. Several participants shared that their program had pushed 

them to try speaking in front of groups. At first, many participants were hesitant or scared. 

However, after trying it a few times with support, they felt excited to continue. One participant 

said, “You feel like, more confident for you to talk.” Another said, “After I’m done talking, I feel 

like, ‘Oh, I wanna go one more time! Give me one more choice, one more opportunity...I have 

lots to tell the audience!’”

Pride in accomplishments and identity. The activities in Opportunity Reboot programs provided 

opportunities for participants to take ownership and feel a sense of pride for their contributions. 

One participant vowed to make their contributions permanent: “[I’m] gonna get a tattoo of the 

candles logo I helped create.” Besides their work or volunteer efforts, several participants were 

proud of how far they had come in their education and who they were as an educated person: 

“Never thought that I could have a better education. That’s how I’m proud of myself.” For 

others, working through adversity strengthened their pride in their ability to persevere: “I’m 

most proud of, you know, even though I had many problems to come through...I can stand by 

myself. I don’t have to stay by other people, so I’m proud of myself.”

Developing and living by values. Some participants talked about outcomes using value-laden 

language. “[The program] helped what values I have and stuff like that. It taught me better 

values,” said one participant. Some attributed these values to who they will be in the future: 

“Now that I’ve developed these other values, it’s like, now I can actually move on in life and be 

able to be a very positive and very helpful person in the community.”

I’m most proud of, you 

know, even though I had 

many problems to come 

through, I have to change 

and I can stand by myself. 

I’m proud of myself..

“

Positive Disruption: Opportunity Reboot Model 103
Social-Emotional Competencies

Improved communication. Several participants discussed how their communication with peers 

or other program members has improved greatly. As an example, many participants described 

how they were explicitly taught to be active listeners, and how this is “actually following what 

they were saying, showing commitment to what they were saying.” Others shared that their 

communication allowed them to “think about how the other person will feel about what I have 

to say.” Communicating across cultural differences was another example. 

Ability to better manage emotions. Some participants noticed that they had changed in the 

ways they could now control outbursts, calm themselves down, or move on when someone had 

irritated them. One participant said, “I still get really annoyed, do you ever get that feeling, ‘I 

just wanna slap ‘em?’ but you can’t, you won’t. It just helps you keep that thought more in your 

head.” Another said, “I got to meet really cool people who helped me cope with talking to 

people.” Others noted that they feel more patient, or are better with managing their anger.

Helped us learn that if you 

work as a team and you’re 

respectful and honest with 

your teammates or your 

coworkers, you all can 

achieve some great things 

in life. The more time you 

work as a team, the more 

you can get stuff done.

“

Working as a team. Effective teamwork was a skill that several 

participants discussed during interviews. One of the ways this 

was achieved was through social competence, understanding 

how honesty and respect can help a team achieve mutual goals. 

One participant said, “Before this, we couldn’t work as a team, 

we wouldn’t be positive to one another.” Another shared this 

story: “We wouldn’t care about the other person’s feelings. So, 

the soft skills class helped us out...Our teacher was like, ‘You 

know, you all really worked good as a team, you helped each 

other out. When someone was down, you all were supportive 

and you helped ‘em out.’”

Soft skills. Many participants also reflected on “soft skills” that 

they developed through their work experience in Opportunity 

Reboot programs. People skills, such as “patience with a 

customer” or “how to work and ask people questions” was an 

important soft skill outcome. Critical thinking was a component 

of these skills, with one participant noting that “when a 

challenge arose, [I am able] to overcome that challenge and find 

a different solution.” 

Positive Disruption: Opportunity Reboot Model 104
Skills for Systems Navigation

Financial literacy. Several participants referenced financial literacy as an outcome of 

participation in Opportunity Reboot programs. Of these participants, a greater.”

understanding of how businesses manage money and 

how they could pay for basic needs were the main 

themes. One participant said, “I’m more mindful at my 

house about keeping the lights on.” Another said, “I 

realize how much money goes into [a business].”

Technical skills. The majority of workplace skills that 

participants discussed in interviews were related to jobspecific technical skills. Many participants connected 

their experience of learning these skills to an advantage 

they would have when applying to jobs in the future. 

Some of these technical skills and training that 

participants experienced included building solar panels, 

OSHA training, certification in Minnesota Career Digital 

Literacy, and CPR. In addition to job-specific skills, 

participants also cited exposure and learning of 

management skills, time management, and learning 

“how a job actually works...what the future would be like 

and how I’m gonna succeed.”

Accessing needed resources. Several of the examples 

that participants shared were ways the Opportunity 

Reboot staff or programs had facilitated access to 

resources that would fulfill basic needs. For instance, 

[Staff] can help you look 

for colleges, or if you’re 

stuck, they help...If you 

have questions about a 

certain type of job after 

high school, they can 

help you with the 

requirements.

“

participants shared that they had “got my own apartment” or learned how to take a bus where 

they needed to go. One participant described how a staff member taught them how to use a 

map so they would be better able to go home from the program or go to school. 

Asking for help. Participants appreciated that they could count on staff members to guide them 

or help them when they needed to navigate systems. 

“[The program] really cares where you come from and what 

you’re doing, just tell them you need help and they’ll help 

you,” said one participant. Another said, “If I weren’t in 

[the program]...I wouldn’t have any idea what to do, since my 

parents aren’t really a help, but thanks to [the program], I have 

more opportunities. They know what to do, while I don’t.”

Learn about future jobs, how to apply for them. As detailed 

previously, several participants described various activities in their 

programs that taught them how to create materials for applying to 

jobs, find jobs, and interview for jobs. In general, “[projects in 

Opportunity Reboot] inspired me to think about my career choice,” 

as noted by one participant. Others described what they learned, 

such as “if you interview, you have to make eye contact,” and “how 

to find a job, how to figure out applications.”

Positive Disruption: Opportunity Reboot Model 105
Academic improvement. Several participants noted that their participation in programs using 

the Opportunity Reboot model had helped them improve their grades. Some youth

Commitment to Learning

Greater motivation to work hard. Participants were able to 

identify that they were “back on track” after participation with 

Opportunity Reboot programs. Many attributed this to feeling 

more motivated. “Yes, it does help me work harder,” said one 

participant. Another said, “This internship has just made me 

wake up and remember what I actually care about and my 

career goals.” Some noted that they felt motivated because 

they knew that an upcoming test could be tied to their grades 

or a future job: “This is about the test...if I don’t try or if I don’t 

motivate myself to do the test good, as well, I’m not gonna

pass.”

Desire to graduate, go to college. For many of the opportunity 

youth interviewed, the goal of graduating from high school or 

entering college was an exciting one: “When I turn that tassel, 

it’ll be a very, very fun experience.” Their belief that they could 

graduate was bolstered by their program. As some participants 

said, “[Dual credits] helped me out a lot. I wouldn’t be where I 

am today without those...I didn’t even see it as possible, of 

graduating with my class.” Others noted that they were looking 

forward to now being able to earn a GED or apply for colleges.

As noted above, in addition to sharing pictures or journal 

entries demonstrating their experiences in Opportunity Reboot 

programs, participants also showed how they had grown through a storyboard activity where 

they looked at their life as a “Three-Act Play.” Through this activity participants thought about 

where they were before the program and how things had changed for them as a result of their 

participation and how that played into their future goals. In the following section, we explicate 

several of those themes beyond the outcomes already shared. Each section begins with an 

illustrative participant quote. 

The past: “I wasn’t really making any forward progress.” Participants talked about feeling 

“depressed” or “lost” before the programs they participated in. Some described themselves as 

“not very social.” Others said they often got into trouble in their schools or communities and 

generally felt “unmotivated” to do well in school or work. Several participants talked about the 

jobs that they did have as “meaningless” or “not worth the time.” 

Representing Change in Three Acts

. . . I actually care about 

my grades. I used to not 

care. I guess I didn’t 

really care about what I 

was gonna do in the 

future, as long as I was 

alive.

“

She’s been able to just 

push past things and be 

able to more or less 

make us learn, even 

when we don’t want to. 

That’s just a really good 

motivational thing for a 

lot of us students....She 

never gave up.

“

enthusiastically noted in interviews, “My grades are better,” or 

“my grades are very good.” One participant said, “I don’t 

wanna say I didn’t care [before], but now, I care more about 

my grades.” Participants also expressed how they were able to 

improve in academic skills, in tandem with receiving better 

grades. “I got better in math,” said one participant. 

Positive Disruption: Opportunity Reboot Model 106
End of the program: “Now I’m focused on both school and life.” Participants talked about 

feeling more comfortable being around others, making friends and feeling less alone. They felt 

“accomplished,” “proud,” and “successful” because of their time in Opportunity Reboot 

programs. Some said it was what helped them get sober and find meaning in their work. Others 

noted that they were excited to have received college credit or landed their first job.

Looking ahead: “I can see myself in the future as a strong bird who can fly through the storm.” 

Youth who had completed programs saw a bright future for themselves. Many saw themselves 

as college graduates, something they had never imagined before. Living in a community and 

owning a house and a car felt realistic. Others talked about specific goals such as getting a 

degree in environmental studies or joining a nursing program. They saw themselves as more 

resilient and able to take on whatever came their way and they knew where to access 

supportive resources.

Positive Disruption: Opportunity Reboot Model 107
DEED attempted to use race as a matching variable when identifying the comparison group using 

propensity score matching (PSM) techniques. However, their analysis revealed the youth in the 

Opportunity Reboot group were distinct from same-race youth that were in the comparison 

group. For example, youth who identified as Asian in the comparison group had much higher 

educational attainment, on average, than Opportunity Reboot’s Asian (mostly Karen refugees) 

participants. In consultation with the evaluation team, DEED elected to combine all non-white 

individuals into a single person of color (POC) category in their analyses.

Research Question 9

What percent of youth in the Opportunity Reboot group 

able to secure employment? How is this similar or 

different from the comparison group?

Box 2: Use of “Persons of Color vs. Race Categories in DEED Analyses

Positive Disruption: Opportunity Reboot Model 108

Working in partnership with the Minnesota Department of Employment and Economic Development 

(DEED), the evaluation team conducted the employment and wage attainment study, which provide 

access to data sources maintained by the state of Minnesota. The comparison group for this study 

were Minnesota youth who were similar on select sociodemographic factors to youth in programs 

using the Opportunity Reboot model; however, youth in the comparison group were not receiving 

services and supports from eligibility-based programs. The data focus on five time points: 2018 

Quarter 3 (which approximately aligned with endline administration of the Opportunity Reboot Youth 

Survey; July to September), 2018 Quarter 4 (October to December), 2019 Quarter 1 (January to 

March) and 2019 Quarter 2 (April to June).

As illustrated in Figure 5, the overall employment rate of youth in the Opportunity Reboot group 

generally increased or remained stable during the four quarters after the endline survey was 

administered (from 59% to 67%). A similar, albeit less linear, trend was observed among Opportunity 

Reboot Youth of Color (see Box 2) from 56% to 70% (see Figure 6). When the employment data were 

disaggregated by employment type (i.e., full-time vs. part-time; see Figure 7), some encouraging 

trends were noted. First, Opportunity Reboot youths’ full-time employment rate increased linearly 

across all four quarters (from 3% at the endline youth survey, to 27% at the end of the first quarter 

following the endline survey, and 36% the following summer). Similarly, their part-time employment 

rate also declined, from 50% at endline to 37% across all four quarters. Given that the overall trend of 

increasing employment over time, it is reasonable to infer that most of the youth were successful at 

converting their part-time positions to full-time positions. These data also suggest that Opportunity 

Reboot youth were, on average, finding jobs in sectors that were generally insulated from seasonal 

fluctuations. According to DEED, the drop in 2019 Quarter 1 employment rates seen in the 

comparison group, compared to the quarter immediately before it, is an expected and recurring 

annual trend.
Figure 5 Percent of Youth Employed Over Time

 

Figure 6 Percent of Youth of Color Employed Over Time

Positive Disruption: Opportunity Reboot Model 109
Youth in the Opportunity Reboot group, as well as just the POC subgroup, tended to have 

better overall employment outcomes compared to comparison youth; furthermore, these 

better outcomes were consistent over time. When disaggregating by employment type, 

Opportunity Reboot youths’ full-time employment rate exceeded that of the comparison group 

from the first post-endline quarter onward, providing evidence that program sites’ activities 

were in fact efficacious at helping youth secure employment. Results are somewhat mixed for 

youth in the Opportunity Reboot group who were employed part-time: despite starting out with 

lower part-time employment rates when compared to the comparison group, the proportion of 

youth employed part-time were fairly similar between both groups.

In addition to reporting the youths’ employment rates, DEED also ran difference-in-differences 

(DiD) analyses to test for statistically significant differences between the Opportunity Reboot 

sample and the comparison sample on four outcomes: (1) any employment, (2) any 

employment (with person of color sub-sample), (3) full-time employment, and (4) part-time 

employment (Table 23; see Imbens & Wooldridge, 2009). DiD analyses were conducted using 

the diff Stata package obtained from the Boston College Statistical Software Components (SSC) 

archive of user-contributed programs. DEED specified logit-estimated Kernel-based propensity 

score difference-in-difference analyses on the area of common support (i.e., the range of 

propensity scores where the probability of observing treatment and control cases overlap). The 

following covariates were included in each of the four models: age, gender, educational 

attainment, and the interaction between age and gender. The covariates of persons of color and 

the interaction term of persons of color by prior employment were also included in the three 

models that were not limited to the person of color sub-sample. 

Positive Disruption: Opportunity Reboot Model 110

 

Figure 7 Percent of Youth Employed, by Full-Time (FT) and Part-Time 

(PT) Work Status, Over Time
While propensity score matching (PSM) seeks to match control and treatment individuals with 

similar attributes, there is always the concern that there are unobserved differences between 

the two that could affect outcomes (such as wages). The added benefit of the DiD approach is 

that as long as the observed differences between control and treatment participants are 

consistent, changes in their differences (i.e., the difference in the difference between the groups 

being compared) can be reasonably attributed to the intervention (i.e., Opportunity Reboot). 

DiD controls for both time and group effects, thereby isolating the program effect (Bendewald, 

Maryns, & Robertson, 2016). 

Across the four models, there were significant differences between the Opportunity Reboot and 

comparison sample for any employment, full-time employment, and full-time employment 

among persons of color sub-sample during the first quarter of 2019, such that a greater 

percentage of Opportunity Reboot participants, on average, were employed relative to the 

comparison sample. There was also a greater percentage of Opportunity Reboot participants 

employed full-time at endline and during the second quarter of 2019 relative to the comparison 

sample. It is unsurprising that most of the differences and DiD analyses are were not statistically 

significant, given the short time frame of the follow-up periods. Findings from previous studies 

suggest that observed effect sizes tend to be small in the year immediately following an 

intervention, and that these program effects tend to rise over time (Card et al., 2011; 2015). 

This is consistent with the current evaluation, as the effect sizes were generally negligible across 

the employment outcomes. It was, however, encouraging to observe small to moderate effect 

sizes (Cohen’s d = .299 - .519) for participants’ full-time employment rate this quickly after 

endline (Cohen’s d = .20 is considered small but meaningful; d = .50 is considered moderate; 

Cohen, 1992; Durlak, 2009). These effect sizes suggest there is indeed a practical and significant 

difference in the percentage of Opportunity Reboot participants who were able to secure fulltime employment in Q1 and Q2 of 2019 relative to the comparison sample. This data supports 

emerging moderate evidence of the Opportunity Reboot model.

Although the DiD analyses revealed few statistically significant differences between Opportunity 

Reboot and comparison group participants, Opportunity Reboot participants’ employment 

numbers exhibited encouraging trends. Opportunity Reboot youth appear to be successful at 

securing employment – notably in jobs that seem to be insulated from seasonal fluctuations. 

Encouragingly, employment trends for Youth of Color tracks the overall trend and compares 

favorably to the trends observed among the Youth of Color in the comparison group. 

Furthermore, Opportunity Reboot youths’ full-time employment rate increased 

consistently across the four quarters after endline survey administration.

Positive Disruption: Opportunity Reboot Model 111
Table 23 Difference-in-Difference (DiD) Results between Comparison 

and Opportunity Reboot Samples’ Quarterly Employment Rates

Follow-Up Periods

Endline 2018 Q3 2018 Q4 2019 Q1 2019 Q2

Any Employment 

Opportunity Reboot .585 .636 .667 .692 .672

Comparison .594 .616 .606 .469 .618

Difference

(SE)

-.009

(.114)

.019

(.088)

.061

(.094)

.229*

(.093)

-.009

(.095)

Difference-in-Difference

(SE)

.029

(.147)

.070

(.133)

.238+

(.133)

.064

(.133)

Cohen’s d .012 .029 .096 .026

Any Employment - Persons of Color Only

Opportunity Reboot .563 .651 .692 .728 .698

Comparison .559 .554 .544 .401 .567

Difference

(SE)

.004

(.107)

.097

(.115)

.149

(.106)

.327**

(.091)

.132

(.112)

Difference-in-Difference

(SE)

.092

(.166)

.144

(.153)

.322*

(.126)

.127

(.146)

Cohen’s d .044 .068 .153 .060

Full-Time Employment

Opportunity Reboot .025 .267 .297 .318 .359

Comparison .087 .163 .232 .160 .206

Difference

(SE)

.062**

(.020)

.103

(.072)

.065

(.343)

.158**

(.059)

.153**

(.065)

Difference-in-Difference

(SE)

.041

(.075)

.003

(.074)

.096

(.065)

.092

(.069)

Cohen’s d .391 .299 .519 .507

Part-Time Employment

Opportunity Reboot .497 .369 .369 .374 .369

Comparison .569 .453 .374 .304 .374

Difference

(SE)

-.071

(.082)

-.084

(.095)

-.004

(.093)

-.071

(.081)

-.004

(.093)

Difference-in-Difference

(SE)

-.012

(.122)

.067

(.120)

.142

(.131)

.067

(.120)

Cohen’s d .007 .039 .082 .039

Notes. +p ≤ .10. *p ≤ .05. **p ≤ .01. SE = Standard Error. The reported effect sizes (Cohen’s d) compares the mean of each follow-up period with the 

endline mean. 

Positive Disruption: Opportunity Reboot Model 112
Drawing on data from the employment and wage attainment study, the findings to Research 

Question 9 suggest promising employment trends for youth in the Opportunity Reboot sample. 

However, while securing any kind of employment is an important step to economic 

independence, skill development, and opportunity, it is also important to consider wage 

attainment. Towards this end, living wage employment (see Box 3) — and the patterns in this 

threshold of employment — were examined. Figure 8 graphically illustrates the trends in living 

wage employment for the Opportunity Reboot and comparison group. 

Research Question 10

What percent of youth in the Opportunity Reboot group able to 

secure living wage employment? How is this similar or different 

from the comparison group?

If you want your life 

better, if you want to 

achieve your 

goal...come here!

Positive Disruption: Opportunity Reboot Model 113

The Minnesota Department of Employment and Economic Development (DEED) defines 

living wage employment as employment that results in $30,900 a year (or $7,725 a 

quarter) in earnings assuming full-time employment, or an hourly wage of $14.86 or 

greater. This figure is DEED’s yearly estimate of the basic-needs cost of living in the state of 

Minnesota, assuming that the individual is single and has no children — which is the case 

for nearly all study youth.

Box 3: Definition of Living Wage Employment

The percent of youth in the Opportunity Reboot group who were able to secure living wage 

employment was relatively small, and — while the percentage generally increased over time (+9%) 

— at the last data point (2019 Quarter 2) only about 1 in 7 Opportunity Reboot youth were 

making a living wage.

Across all five time points, the data show that youth in the Opportunity Reboot sample are less likely 

to have secured living wage employment (see Box 3) than their peers in the comparison group. 

However, the trend for the Opportunity Reboot group generally increased across the first four time 

points (+9%), while the trend for the comparison group was flatter (+3%). Interestingly, the percent 

of youth in the comparison group earning a living wage increased sharply in 2019 Quarter 2 (16% to 

22%) while the Opportunity Reboot group stayed about the same (15% to 14%). 

In light of the low rates of living wage employment, median quarterly earnings were also examined 

to understand the wages opportunity youth were actually earning. Figure 9 graphically illustrates the 

median quarterly earnings for the Opportunity Reboot and comparison groups. The data show that 

youth in the Opportunity Reboot group were, on average, successful at increasing their wages 

across the four quarters (save for a dip during the 2019 Quarter 1) immediately following the 

endline administration of the Opportunity Reboot Youth Survey. Further, the rate of increase appears 

sharper than that experienced by youth in the comparison group. However, median wages for both 

groups still fall far below the threshold for living wage employment ($7,725 per quarter) in 

Minnesota. 
Figure 9 Youths’ Median Quarterly Earnings

 

Positive Disruption: Opportunity Reboot Model 114

Figure 8 Percent of Youth Earning a Living Wage
Collectively, these data show that the number of youth in the Opportunity Reboot sample 

who secured living wage employment increased over time; however, these data do not show 

that this increase is greater than that experienced by the comparison group. That said, while 

below the living wage threshold, Opportunity Reboot youth did experience an increase in their 

median quarterly wages. 

There are a number of reasons that might explain this trend. For some of these youth, they 

might still be in the Opportunity Reboot programs and have limited time to work, or they might 

be pursuing other educational opportunities that appropriately limit their ability to earn more. 

Those would be good reasons. However, this trend might also signal a larger systemic problem 

in Minnesota of having insufficient opportunities for youth — like those in these groups — to 

secure jobs that provide the financial resources required to meet their most basic needs. Future 

research should explore these trends and group differences over a longer time frame.

If you want your life 

better, if you want to 

achieve your 

goal...come here!

Positive Disruption: Opportunity Reboot Model 115
Section 4

Conclusions And 

Contributions
The implementation and impact evaluations sought to elevate the level of preliminary evidence 

of the Opportunity Reboot model. To do this, the evaluation was intentionally designed in close 

collaboration with community partners to ensure a deep understanding of: (a) how this program 

enhancement model was being integrated into existing programming, (b) how to gather high 

quality data from opportunity youth; and (c) the impact of the Opportunity Reboot model 

features on the lives of opportunity youth. In pursuit of these goals, the evaluation studies 

yielded a number of key findings that led both to real-time and future suggested refinements to 

the Opportunity Reboot model as well as preliminary evidence of model impact.

The implementation and preliminary impact evaluations collectively point to the promise of 

models, like Opportunity Reboot, that can be infused across a variety of programs and 

populations as a way to enhance (rather than replace) existing programs. Youthprise merged a 

set of features, concepts, and strategies that have surfaced as empirically-supported practice to 

boost program effectiveness into a cohesive Opportunity Reboot model, then worked closely 

with community partners to integrate these enhancements into their existing programs. 

Inherent in this process was the acknowledgement and celebration of the expertise and 

innovative approaches partners were already utilizing to forge deep, transformative 

relationships with the opportunity youth in their communities. The success of these kinds of 

program enhancement models hinges on honoring community partners’ existing work, and 

finding points where the enhancement model goes deeper with existing program principles. 

When this happens, it offers a way forward through an improvement model to utilize new 

research and practices, rather than putting program leaders in a position where they feel they 

have to abandon an existing program or approach for new ones, retraining staff and investing in 

totally new materials in the process. 

Key Finding # 1

Program enhancement models like Opportunity Reboot 

hold promise.

The innovation of the Opportunity Reboot model does not lie in any one of the model features, 

each of which, alone, already has preliminary or moderate evidence of impact on important 

youth development outcomes. Rather, the innovation comes from integrating these features in a 

coherent model and providing expert technical assistance to guide infusing the model into 

existing programming and supporting the intentional implementation of each feature.

Key Finding # 2

The multi-pronged system of support built around the 

Opportunity Reboot model is critical to implementation. 

Positive Disruption: Opportunity Reboot Model 117

The multi-pronged system of technical assistance led by Youthprise relied on biweekly technical 

assistance calls, quarterly in-person convenings for partners to learn and share with one 

another, annual site visits, and access to mentoring trainings to keep community partners 

focused on the critical aspects of the Opportunity Reboot model. Over time, the by-products of 

these activities included an active learning community among partners where resource-sharing 

and collective problem-solving commonly took place, and a shared vision – bigger than any one 

partner alone had ever imagined possible – of how, collectively, their organizations could 

positively disrupt the life trajectories of opportunity youth in Minnesota. 
Data from the initial administration of the Opportunity Reboot Mapping Tool at the outset of 

this partnership showed that only one of the community partners was “mostly aligned” with the 

Opportunity Reboot model. However, by the final administration, all six community partners 

were determined to be “mostly aligned.” The Implementation Evaluation section describes – in 

great detail – partners’ alignment by model feature. The mapping tool was used as a multipurpose and engaging approach to measuring fidelity and as a mechanism for capturing the 

varied tactics community partners used to operationalize the features in ways that made sense 

for their community and the youth they served. 

Increased fidelity can be attributed to the willingness of community partners to be thorough and 

transparent in sharing their organizational tactics with Opportunity Reboot core partners, and, 

subsequently, other community partners within the cohort. The data were used to identify 

technical assistance and capacity-building needs that, when fulfilled, would enhance alignment. 

Partners with particular strength in implementation of specific features of the Opportunity 

Reboot model were encouraged — and provided with a platform — to share and consult with 

other community partners about their practices. The practice of assessing needs and monitoring 

progress in implementation strategies became a concrete roadmap for community and core 

partners to adhere to and realize.

Key Finding # 3

Community partners implemented the Opportunity 

Reboot model with increasing fidelity over time. 

Positive Disruption: Opportunity Reboot Model 118
The fact that community partners in this evaluation were integrating the Opportunity Reboot 

model into their existing programs was often completely unknown to program participants. And 

yet, participants overwhelmingly self-reported on the survey that they were — indeed —

experiencing the features of the Opportunity Reboot model. Youth also reported high levels of 

satisfaction with the overall program quality. Youth reported feeling physically and emotionally 

safe, respected, and believed that program staff were trying to help them. 

Key Finding # 4

Youth reported experiencing the Opportunity Reboot 

model features and were generally satisfied with the 

program.

Positive changes were made across the outcomes in all three developmental domains, these 

included: positivity in the face of challenge, goal-setting skills, self-awareness, responsible 

decision making, relationship skills, financial literacy, job-seeking skills, and resource 

identification skills. Female participants and youth who identified as non-heterosexual made 

gains across more short-term outcomes than their counterparts. Youth with diverse racial and 

ethnic identities (i.e., those in the Other race/ethnicity group) also experienced markedly more 

positive growth across short-term outcomes.

Key Finding # 5

Youth participants experienced measurable and 

significant gains across a number of critical short-term 

outcomes; however, these gains were not experienced 

equally by all youth.

Qualitatively, youth described the positive impact that the Opportunity Reboot program had on 

their lives in several key developmental and skill-based areas. 

— Positive identity: Youth mentioned having greater confidence about the future, 

they felt more comfortable with themselves, were more confident speaking in 

front of others, had more pride in accomplishments and identity, and felt that they 

were developing and living by their values. 

— Social-emotional competencies: Youth felt that they improved their 

communication, were better at managing emotions, working as a team, and had 

greater intrapersonal skills. 

— Commitment to learning: Youth felt that through the program they made 

academic improvements, had greater motivation to work hard, and a desire to 

graduate or go to college. 

— Systems navigation: Youth developed financial literacy skills, greater workplace 

skills (technical and soft), were able to access resources, ask for help, and learn 

about future jobs and how to apply for them. 

Positive Disruption: Opportunity Reboot Model 119
We all need relationships in our lives that support us, challenge us, and help us grow. Opportunity 

youth are no different. Yet, strong relationships may be even more important when other systems 

and supports have failed these young people. There is truth in the cliché “it’s who you know,” and 

having limited access to social capital can limit all kinds of opportunities. 

Key Finding # 6

Strong relationships with program staff and new 

mentoring relationships with adults outside of the 

program were core to the improved positive values, selfperceptions, and skills youth experience, and had a 

transformative impact on the lives of opportunity youth. 

skills. And when they build strong relationships with other adults outside of the program that support 

their growth, they are more likely to show positive gains – over and above all of the other model 

features – on ALL of the social-emotional competencies and systems navigation and skills outcomes 

targeted in this evaluation. New mentoring relationships with adults outside of the program are also 

significant linked to gains in their orientation towards the future and civic efficacy. 

These quantitative findings are reinforced in the qualitative interview and focus group data in which 

youth revealed through their personal stories and experiences the subtleties of what makes these 

relationships so powerful. For many of these opportunity youth, the relationships they formed with 

mentors (within and outside of the program) were qualitatively different from others in their lives. 

Prior relational trauma could make it difficult for opportunity youth to build trusting, healthy 

relationships. Mentors, particularly those interviewed at the partner sites, took seriously the 

responsibility of teaching (and showing!) program participants that the relationship they formed 

together could – and would – be different: it would be safe and healthy; there would be no quid pro 

quo; they would be challenged to reflect, grow, and learn; and the relationship would be dependable. 

Evidence of mentors’ success in achieving these kinds of relationships is resoundingly strong in the 

narratives youth shared.

Finding that relationships were so predictive of outcomes is great news, because relationships are 

malleable and, while they are hard work, they are cost-effective. Relationship-building does not 

require an expensive program or equipment, it simply requires time and attention.

Add unjust systems and under-resourced 

communities to the list of barriers, and you 

quickly see that those relationships become 

paramount to ensuring success for these youth 

who clearly have the odds stacked against 

them.

The strongest and most consistent empirical 

finding to emerge from this evaluation is that 

relationships matter. The survey data show 

that opportunity youth who build strong 

relationships with program staff are more 

likely to show positive gains in the skills 

required to navigate systems: financial literacy, 

job-seeking skills, and resource identification

Positive Disruption: Opportunity Reboot Model 120
Many of the pictures and journal entries youth provided in the 

impact narrative activities depicted key relationships with 

mentors and peers. Opportunity youth rarely talked about 

ways they had changed or grown without ascribing that 

change to an important relationship. Yet, mentoring 

relationships are not the cure-all.

Most of the opportunity youth being served by Opportunity 

Reboot community partners are facing complex issues that 

require equally complex solutions and interventions. 

Mentoring relationships are vital for accessing many of these. 

The qualitative data from this study illustrate the variety of 

ways this plays out.

Key Finding # 7

Mentoring relationships are gateways 

to other important supports.

When I first came to 

GAP, I didn't speak 

English very well, she 

asked me, "What do 

you want to do in your 

future?" and I said, "I 

want to be English 

teacher, but I don't 

think I can do that." 

And she said, "No, I 

believe you 

can...because I will be 

your mentor."

The teacher support. . . 

makes you wanna come 

back the next day, it 

makes you feel like you 

can do it. It’s 

empowering, you know, 

to know that there’s at 

least one adult in the 

school that will care 

about you and help you. 

“

Mentoring relationships:

— Motivate opportunity youth to come back to safe spaces 

where they can get the help and support they need 

when they are ready, and where they can access critical 

health, education, and career resources.

— Teach opportunity youth what a healthy relationship looks and feels like, and how 

youth can positively contribute to maintaining and strengthening these kinds of 

relationships both within and outside of the program.

— Open doors to new educational and career opportunities and resources.

— Build the foundational trust required for opportunity youth to engage in productive 

and sustained goal-setting conversations about their well-being, educational goals, 

and career aspirations.

Relationships are necessary, but not sufficient. The added-value of relationships is reaped when 

relationships open doors to other opportunities and resources, with that resulting relationshipsplus-resources combination representing strong social capital that promotes educational and 

occupational mobility.

Positive Disruption: Opportunity Reboot Model 121
Key Finding # 8

Individualized goal-setting supports and career pathways 

supports did not predict short-term youth outcomes in the 

quantitative data; yet, they are fundamentally important in 

the lives of opportunity youth. 

The regression models showed evidence of the association between the individualized goalssetting supports and the career pathways supports features of the Opportunity Reboot model 

on only one outcome (over and above the variance predicted by change in within program 

mentoring and outside of program mentoring): growth in job-seeking skills. This association was 

negative for the individualized goal-setting supports feature, and positive for career pathways 

supports feature. As argued above, the negative association – which, although not significant, is 

seen across the regression models – may be an artifact of the phenomenon that youth who 

require the most support setting goals also have more urgent short-term needs than those 

captured by the targeted outcomes in this evaluation. Reassuringly, the experience of being in a 

program that provides career pathways supports was a strong positive predictor of job-seeking 

skills.

The absence of significant empirical associations between individualized goal-setting supports, 

career pathways supports, and short-term youth outcomes should not be interpreted as 

evidence that these features of the Opportunity Reboot model are unimportant. In fact, the 

impact narratives give voice to how critically important these features were to helping youth see 

new possibilities for themselves and propel their lives forward. Relationships are the entry point 

for many of these goal and career supports. Through the impact narrative activities, youth 

talked at length about the many ways their mentors helped to connect them to: (a) people who 

were in careers that they were interested in learning more about; (b) resources to meet basic 

physical and mental health needs; and, (c) financial assistance programs to pursue their 

education and career goals. More nuanced goal-setting and career pathways supports measures 

should be explored in future research.

Key Finding # 9

Youth in programs using the Opportunity Reboot model 

secured employment at higher rates than 

demographically similar youth who were not in these 

programs, with a particularly noteworthy advantage to 

opportunity Youth of Color. 

Opportunity Reboot participants were more likely than similar peers not participating in these 

programs to secure full-time employment over the period of a year, and to avoid the seasonal 

dip in employment often seen during Minnesota’s winter months (2019 Quarter 1). This was 

particularly true for Youth of Color in the Opportunity Reboot group. While this finding holds 

promise, the data suggest very few opportunity youth (whether in the Opportunity Reboot or 

comparison groups) secured living wage employment and — although their wages did increase 

over time — they still fell below the state-defined threshold to support the costs of stable 

housing, food, and other basic necessities in the state of Minnesota.

Positive Disruption: Opportunity Reboot Model 122
Extant research has established preliminary or moderate evidence of the impact of each of the 

Opportunity Reboot model features on important youth development outcomes. Yet, evidence 

had not previously been established of how these features impact youth when: (a) implemented 

in concert via an enhancement model to existing programs; (b) with a multi-pronged system of 

technical assistance supports; and (c) with opportunity youth residing in urban and rural regions 

of Minnesota. The data collection activities in the preliminary impact phase of the evaluation 

sought to attain this foundational level of evidence for the Opportunity Reboot model.

The results of the Opportunity Reboot Youth Survey administered at baseline and endline (i.e., a 

single group non-experimental outcome evaluation design) established associations between 

opportunity youths’ experiences of the four core features of the Opportunity Reboot model and 

growth in their positive identity, social-emotional competencies, and ability to navigate systems. 

As described above, the within and outside-of-program mentoring features of the model were 

the strongest and most consistent predictors of these short-term youth outcomes. Although the 

individualized goal supports and career pathways supports features were not shown to be 

consistent predictors of change (over and above the impact of mentoring), the qualitative 

impact narratives demonstrate the value and benefits opportunity youth reaped by accessing 

these supports in their respective programs.

Building on this evidence, the employment and wage attainment study employed a quasiexperimental design to empirically demonstrate that youth in the Opportunity Reboot group 

were more likely, on average, than youth in the comparison group to be employed in the four 

quarters after the endline survey: Q3 +2%; Q4 +6%; Q1 +22%; Q2 +5% (favoring the Opportunity 

Reboot group). This finding is even stronger when comparing Youth of Color in the Opportunity 

Reboot group to those in the comparison group: Q3 +10%; Q4 +15%; Q1 +33%; Q2 +13% (again, 

favoring the Opportunity Reboot group). Although findings were only statistically significant for 

Q1 and Q2, the Opportunity Reboot participants’ employment numbers show encouraging 

trends that demonstrate emerging moderate evidence. Opportunity Reboot youths’ full-time 

employment rate increased consistently across the four quarters. Youth in the Opportunity 

Reboot group were also more likely to be employed in full-time, rather than part-time, positions 

and over this same period, more youth in the Opportunity Reboot group were earning a living 

wage (although the absolute percentage was still low).

Collectively, these findings demonstrate that the Opportunity Reboot model has strong 

preliminary evidence and emerging moderate evidence of impact on key developmental and 

employment outcomes for opportunity youth residing in Minnesota.

Elevating the 

Level of Evidence

Positive Disruption: Opportunity Reboot Model 123
Youthprise continues to support and make improvements to the Opportunity Reboot model and 

is actively seeking sustainable funding streams to pay for the multi-pronged system of support 

(see Figure 2, Inputs) required to implement the full model and expand it to other community 

partners.

The evaluation findings have prompted several refinements and improvements to the 

Opportunity Reboot model. Forthcoming changes include:

§ The Opportunity Reboot Mapping Tool, an instrument originally developed to support 

evaluation efforts aimed at tracking fidelity and model alignment, will be routinely 

administered by future partners – as part of the multipronged system of technical assistance 

undergirding the implementation of Opportunity Reboot – with future partners to promote 

ongoing dialogue about integration of the model into existing programming.

§ The development of a larger suite of mentoring-focused trainings for community partners. 

Possibilities in discussion include new trauma-informed mentoring training and tapping into 

the learning opportunities available through the National MENTOR network.

§ Ongoing quantitative and qualitative data collection remain an important part of the 

Opportunity Reboot model beyond the CNCS-funded evaluation; however, pieces of the 

preliminary impact evaluation described in this report will be retained but the process and 

approach will be revised to more fully meet the desires of community partners for real-time, 

local data that can be used to drive continuous program improvement.

§ The development of an Opportunity Reboot toolkit that describes the model in detail and 

provides example strategies, resources, tools, ideas, and reflections from partners. This 

toolkit will serve as a guide for program leaders, implementers, and coordinating team 

members of workforce development programs on how to infuse the model features and 

guiding principles into their programs.

In sum, there was considerable evidence in this mixed methods implementation and preliminary 

impact evaluation that the Opportunity Reboot model could make a positive disruption in the 

lives of opportunity youth. The evaluation was not an experimental design, but the combination 

of single group longitudinal surveys, qualitative impact narratives, and a quasi-experimental 

study of employment and wage attainment nevertheless provided a robust portrait of both 

positive effects and the mechanisms by which those effects were achieved. Youth in 

Opportunity Reboot were found to improve in positive identity, social-emotional skills, skills for 

systems navigation, employment, and wages among a demographically diverse sample of 

program participants, largely because of the quality of the positive mentoring that was at the 

center of the partners’ programs. The developmental relationships that opportunity youth 

experienced through the program built on the strengths individual youth already had, in order to 

add confidence, technical and emotional skills, system savvy, and connections to create 

educational and career possibilities that otherwise would have been unavailable to such 

historically marginalized young people.

The Future of the 

Opportunity Reboot Model

Positive Disruption: Opportunity Reboot Model 124
Moreover, the evaluation demonstrated that these enhanced outcomes could be realized 

through a program enhancement model that marshals the best of what individual youth-serving 

organizations are doing currently, with training and other partnership supports to help the 

organizational partners get even stronger in their leveraging of key relationships for youth and in 

learning from each other. The uniqueness of the combined implementation and preliminary 

impact evaluations has been that it was made clear that dedication to fidelity of model 

implementation mattered: It is doubtful that the level of relationship quality youth experienced, 

that was key to enjoying the positive outcomes, could have been achieved absent the 

commitment to and ongoing assessment of, the partners’ attentiveness to aligning their 

program activities with the key features of the Opportunity Reboot model.

Finally, a meta-lesson for future research of this kind is apparent throughout this evaluation 

report: Just as the effects documented among Opportunity Reboot youth required trust to be 

built and honored daily among the organizational partners, and among youth and their case 

managers and others at the individual programs, so a deep well of trust among the researchers 

and all the other organizational, staff, and youth partners in the evaluation needed to be 

constructed and nurtured. The level of early and ongoing involvement of staff and youth in 

helping to shape the evaluation designs and construct the items and protocols used in the 

evaluation, and not simply to respond to questions and prompts, was considerable and 

enduring. Throughout this report, there is displayed a high level of mutuality of respect, 

cooperation, and commitment to opportunity youth among all parties to the evaluation, that 

was maintained from beginning to end. This feature does not show up in regression models or 

even necessarily in focus group quotes, but there is little doubt that it was, like the impact of 

relationships on youth outcomes, another dimension of relationship quality writ large that made 

the evaluation possible to conduct with rigor, even with its complexity and challenges.

In the end, too, it is well to emphasize the meaning of the observed impact. Most of the 

quantitative effect sizes, while statistically significant, were in the range conventionally 

described as “small.” Yet the qualitative narrative impact quotations from youth themselves 

made clear how “large” those so-called “small” effects really were, in the lived experience of 

those opportunity youth. Even a small improvement in confidence, technical or emotional skills, 

system savvy, or connections can make the difference in imagining oneself in a particular job or 

career or not, in knowing how to apply for school admission or for employment, in 

understanding the unwritten rules and social norms of educational and workplace culture which 

often remain systematically hidden from opportunity youth. Life-changing decisions are made in 

moments when confidence, emotional maturity, technical skills, street smarts, and connections 

combine to create opportunities, and then allow young people to believe they can do it. The 

biggest positive disruption Opportunity Reboot made was in this hard-to-quantify contribution, 

to help opportunity youth to allow themselves to dream bigger, and to know they now have a 

newly-realistic chance of one day realizing those dreams.

For up-to-date information on the Opportunity Reboot model or to inquire about potential 

collaborations, please contact Youthprise (https://youthprise.org/contact-us). 

Positive Disruption: Opportunity Reboot Model 125
References
Aspen Institute. (2019). From a nation at risk to a nation at hope: Recommendations from the National 

Commission on Social, Emotional, & Academic Development. Washington, DC: Author.

Bendewald, E., Maryns, N., & Robertson, R. (2016). An evaluation of the Workforce Investment Act adult 

program in Minnesota: Lessons from the financial crisis (IZA Discussion Papers No. 10107). Bonn, 

Germany: IZA Institute of Labor Economics. Retrieved from http://ftp.iza.org/dp10107.pdf

Bouffard, J., & Bergseth, K. (2008). The impact of reentry services on juvenile offenders' recidivism. 

Youth Violence and Juvenile Justice, 6. 295-318. http://dx.doi.org/10.1177/1541204007313384

Braga, A. A., Piehl, A. M., & Hureau, D. (2009). Controlling violent offenders released to the community: 

An evaluation of the Boston Reentry Initiative. Journal of Research in Crime and Delinquency, 

46(4), 411-436. http://dx.doi.org/10.1177/0022427809341935

Braun, V., & Clarke, V. (2013). Successful qualitative research: A practical guide for beginners. Thousand 

Oaks, CA: Sage Publications.

Bridgeland, J., & Mason-Elder, T. (2012). National roadmap for opportunity youth. Washington, DC: Civic 

Enterprises.

Bridgeland, J. M., & Milano, J. A. (2012). Opportunity Road: The promise and challenge of America's 

forgotten youth. Washington, DC: Civic Enterprises.

Bruce, M., & Bridgeland, J. (2014). The mentoring effect: Young people’s perspectives on the outcomes 

and availability of mentoring. Washington, DC: Civic Enterprises.

Bryson, A., Dorsett, R., & Purdon, S. (2002). The use of propensity score matching in the evaluation of 

labour market policies (Working Paper No. 4). London, UK: Department for Work and Pensions. 

Retrieved from 

http://eprints.lse.ac.uk/4993/1/The_use_of_propensity_score_matching_in_the_evaluation_of_

active_labour_market_policies.pdf

Burd-Sharps, S., & Lewis, K. (2018). More than a million reasons for hope: Youth disconnection in America 

today. Brooklyn, NY: Measure of America of the Social Science Research Council.

Caliendo, M., & Kopeinig, S. (2005). Some practical guidance for the implementation of propensity score 

matching (IZA Discussion Papers No. 1588). Bonn, Germany: IZA Institute of Labor Economics. 

Retrieved from http://ftp.iza.org/dp1588.pdf

Positive Disruption: Opportunity Reboot Model 127
Positive Disruption: Opportunity Reboot Model 128

Card, D., Ibarrarán, P., & Villa, J. M. (2011). Building in an evaluation component for active labor market 

programs: A practitioner’s guide (IZA Discussion Papers No. 6085). Bonn, Germany: IZA Institute of 

Labor Economics. Retrieved from http://ftp.iza.org/dp6085.pdf

Card, D., Kluve, J., & Weber, A. (2015). A meta-analysis of recent active labor market program evaluations (IZA 

Discussion Papers No. 9236). Bonn, Germany: IZA Institute of Labor Economics. Retrieved from 

http://ftp.iza.org/dp9236.pdf

Center for Promise (2015). Don't quit on me: what young people who left school say about the power of 

relationships. Washington, DC: America’s Promise.

Centers for Disease Control and Prevention (2020). COVID-19 in racial and ethnic minority groups. Retrieved 

from https://www.cdc.gov/coronavirus/2019-ncov/need-extra-precautions/racial-ethnicminorities.html

Charmaz, K. (2008). Grounded theory as an emergent method. In S. N. Hesse-Biber & P. Leavy (Eds.),

Handbook of emergent methods (pp. 155-172). New York, NY: The Guilford Press.

Close, H. (2007). The use of photography as a qualitative research tool. Nurse Researcher, 15(1), 27-36. 

http://dx.doi.org/10.7748/nr2007.10.15.1.27.c6052

Cohen, J. (1992). Statistical power analysis for the behavioral sciences (2nd ed.). New York, NY: Routledge 

Academic.

Cohen, J., Cohen, P., West, S. G., & Aiken, L.S. (2003). Applied multiple regression/correlation analysis for the 

behavioral sciences (3rd ed.). New York, NY: Routledge.

Crenshaw, K. Cho, S. & McCall, L. (2013). Toward a field of intersectionality studies: Theory, applications, and 

praxis. Signs, 38(4), 785-810.

Delgado, M. (2012). Prisoner reentry at work: adding business to the mix. Boulder, CO: Lynne Reinner

Publishers. 

Drake, E. K., & Barnoski, R. P. (2006). Recidivism findings for the juvenile rehabilitation administration's 

mentoring program: Final report. Olympia, WA: Washington State Institute for Public Policy.

DuBois, D. L., Portillo, N., Rhodes, J. E., Silverthorn, N., & Valentine, J. C. (2011). How effective are mentoring 

programs for youth? A systematic assessment of the evidence. Psychological Science in the Public 

Interest, 12, 57-91. http://dx.doi.org/10.1177/1529100611414806
Positive Disruption: Opportunity Reboot Model 129

DuBois, D. L., & Silverthorn, N. (2005). Characteristics of natural mentoring relationships and adolescent 

adjustment: Evidence from a national study. Journal of Primary Prevention, 26(2), 69-92. 

http://dx.doi.org/10.1007/s10935-005-1832-4

Duckworth, A. L., Kirby, T. A., Gollwitzer, A., Oettingen, G., Gollwitzer, P. M., & Oettingena, G. (2013). From 

fantasy to action: Mental Contrasting with Implementation Intentions (MCII) improves academic 

performance in children. Social Psychological and Personality Science, 4(6), 745–753. 

http://dx.doi.org/10.1177/1948550613476307

Duncan-Andrade, J. (2010). What a coach can teach a teacher: Lessons urban schools can learn from a 

successful sports program. New York, NY: Peter Lang.

Elsey, B., Lanier, L., & Stadd, J. (2015). Career pathways toolkit: A guide for system development. Bethesda, 

MD: Manhattan Strategy Group.

Erickson, L. D., McDonald, S., & Elder, G. H., Jr (2009). Informal mentors and education: complementary or 

compensatory resources?. Sociology of Education, 82(4), 344–367. 

http://dx.doi.org/10.1177/003804070908200403

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis 

program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-

191.

Ferguson, K. M. (2018). Employment outcomes from a randomized controlled trial of two employment 

interventions with homeless youth. Journal of the Society for Social Work and Research, 9(1), 1-21.

Foster-Fishman, P., Nowell, B., Deacon, Z., Nievar, M., & McCann, P. (2005). Using methods that matter: The 

impact of reflection, dialogue, and voice. American Journal of Community Psychology, 36(3-4), 275-

291. http://dx.doi.org/10.1007/s10464-005-8626-y

Ginwright, S., & Cammarota, J. (2007). Youth activism in the urban community: Learning critical civic praxis 

within community organizations. International Journal of Qualitative Studies in Education, 20(6), 693-

710.

Grobe, T., Martin, N., & Steinberg, A. (2015). Creating pathways to employment: The role of industry 

partnerships in preparing low-income youth and young adults for careers in high-demand industries. 

Boston, MA: Jobs For the Future.

Grunewald, R., & Nash. A. (2019). A statewide crisis: Minnesota’s educational achievement gaps. Minneapolis, 

MN: Federal Reserve Bank of Minneapolis.
Positive Disruption: Opportunity Reboot Model 130

Heckman, J., LaLonde, R., & Smith, J. (1999). The economics and econometrics of active labor market 

programs. In O. Ashenfelter & D. Card (Eds.), Handbook of Labor Economics Vol. III (pp. 1865-2097). 

Amsterdam, Netherlands: Elsevier.

Herold, B. (2020). The disparities in remote learning under coronavirus (in charts). Education Week. Retrieved 

from https://www.edweek.org/ew/articles/2020/04/10/the-disparities-in-remote-learning-undercoronavirus.html

Hill, N. E., Castellino, D. R., Lansford, J. E., Nowlin, P., Dodge, K. A., Bates, J. E., & Pettit, G. S. (2004). Parent 

academic involvement as related to school behavior, achievement, and aspirations: Demographic 

variations across adolescence. Child Development, 75(5), 1491-1509. 

http://dx.doi.org/10.1111/j.1467-8624.2004.00753.x

Hooper, M. W., Nápoles, A. M., & Pérez-Stable, E. J. (2020). COVID-19 and racial/ethnic disparities. JAMA. 

http://dx.doi.org/10.1001/jama.2020.8598

Hulleman, C. S., & Harackiewicz, J. M. (2009). Making education relevant: Increasing interest and performance 

in high school science classes. Science, 326, 1410-1412.

Imbens, G. W., & Wooldridge, J. M. (2009). Recent developments in the econometrics of program evaluation. 

Journal of Economic Literature, 47(1): 5-86.

Keating, L. M., Tomishima, M. A., Foster, S., & Alessandri, M. (2002). The effects of a mentoring program on 

at-risk youth. Adolescence, 37(148), 717.

Konopka, G. (1973). Requirements for healthy development of adolescent youth. Adolescence, 8(31), 2-25.

Kramer, L., Schwartz, P., Cheadle, A., & Rauzon, S. (2013). Using photovoice as a participatory evaluation tool 

in Kaiser Permanente’s Community Health Initiative. Health Promotion Practice, 14(5), 686-694.

Leap, J. & Franke, T. (2008). Homeboy Industries: Pathway for successfully leaving gang affiliation. Los 

Angeles, CA: UCLA Luskin School of Public Affairs. 

Lewis, K. (2019). Making the connection: Transportation and youth disconnection. New York, NY: Measure of 

America.

Li, J., & Julian, M. M. (2012). Developmental relationships as the active ingredient: A unifying working 

hypothesis of “what works” across intervention settings. American Journal of Orthopsychiatry, 82, 

157–166.

Liang, B. & Rhodes, J. (2007). Guest editorial: Cultivating the vital element of youth mentoring. Applied 

Developmental Science, 11, 104-107.
Positive Disruption: Opportunity Reboot Model 131

Lunt, M. (2014). Propensity Analysis in Stata, Revision: 1.1. Retrieved from 

http://personalpages.manchester.ac.uk/staff/mark.lunt/propensity_guide.pdf

Mattsson, T. (2014). Intersectionality as a useful tool: Anti-oppressive social work and critical reflection. 

Affilia, 29(1), 8–17. https://doi.org/10.1177/0886109913510659

McBride, D., & Mazur, A. (2008). Women’s movements, feminism, and feminist movements. In G. Goertz & A. 

G. Mazur (Eds.), Politics, gender and concepts: Theory and methodology (pp. 219–243). Cambridge, 

UK: Cambridge University Press. https://doi.org/10.1017/CBO9780511755910.010

Mertens, D. M., & Hesse-Biber, S. (2012). Triangulation and mixed methods research: Provocative positions. 

Journal of Mixed Methods Research, 6(2), 75-79. https://doi.org/10.1177/1558689812437100

Minnesota Compass (2017). High school graduation. St. Paul, MN: Wilder Research. Retrieved from 

https://www.mncompass.org/education/high-school-graduation#1-12655-g

Minnesota Compass (2018). Proportion of adults working. St. Paul, MN: Wilder Research. Retrieved from 

https://www.mncompass.org/workforce/proportion-of-adults-working#1-10678-d

Minnesota Compass (2020). Education overview. St. Paul, MN: Wilder Research. Retrieved from 

https://www.mncompass.org/education/overview

Minnesota Department of Health. (2017). 2017 Minnesota statewide health assessment.

Produced in collaboration with the Healthy Minnesota Partnership. St. Paul, MN: Author.

Nitardy, C. M., Duke, N. N., Pettingell, S. L., & Borowsky, I. W. (2015). Racial and ethnic disparities in 

educational achievement and aspirations: Findings from a statewide survey from 1998 to 2010. 

Maternal and Child Health Journal, 19(1), 58-66.

Oettingen, G. (2012). Future thought and behavior change. European Review of Social Psychology, 23, 1-63.

Oransky, M., Hahn, H., & Stover, C. S. (2013). Caregiver and youth agreement regarding youths’ trauma 

histories: implications for youths’ functioning after exposure to trauma. Journal of Youth and 

Adolescence, 42(10), 1528-1542.

Oyserman, D. (2013). Not just any path: Implication of identity-based motivation for disparities in school 

outcomes. Economics of Education Review, 33, 179-190.
Positive Disruption: Opportunity Reboot Model 132

Patton, M. Q. (2002). Qualitative research and evaluation methods. Thousand Oaks, CA: Sage Publications.

Pekel, K., Roehlkepartain, E. C., Syvertsen, A. K., & Scales, P. C. (2015). Don’t forget the families: The missing 

piece in America’s effort to help all children succeed. Minneapolis, MN: Search Institute. 

Pekel, K., Roehlkepartain, E. C., Syvertsen, A. K., Scales, P. C., Sullivan, T. K.,& Sethi, J. (2018). Finding the 

fluoride: Examining how and why developmental relationships are the active ingredient in 

interventions that work. American Journal of Orthopsychiatry, 88, 493-502.

Putnam, R. D. (2015). Our kids: The American dream in crisis. New York, NY: Simon and Schuster.

Report of the Minnesota Advisory Committee to the U.S. Commission on Civil Rights. (2013). Unemployment 

disparity in Minnesota. St. Paul, MN: Minnesota Advisory Committee.

Roehlkepartain, E. C., Pekel, K., Syvertsen, A. K., Sethi, J., Sullivan, T. K., & Scales, P. C. (2017). Relationships 

first: Creating connections that help young people thrive. Minneapolis, MN: Search Institute.

Scales, P. C., Boat, A., & Pekel, K. (2020). Defining and measuring social capital for young people: A practical 

review of the literature on resource-full relationships. Minneapolis, MN: Search Institute. Report to 

the Bill & Melinda Gates Foundation.

Scales, P. C., Pekel, K., Sethi, J. Chamberlain, R., & Van Boekel, M. (2019). Academic year changes in studentteacher developmental relationships and their linkage to middle and high school students’ 

motivation: A mixed methods study. Journal of Early Adolescence. DOI: 10.1177/0272431619858414

Scales, P. C., Van Boekel, M., Pekel, K., Syvertsen, A. K., & Roehlkepartain, E. C. (2020). Effects of 

developmental relationships on middle-school students’ motivation and performance. Psychology in 

the Schools. Doi.org/10.1002/pits.22350

Schwartz, S. E., Rhodes, J. E., Chan, C. S., & Herrera, C. (2011). The impact of school-based mentoring on 

youths with different relational profiles. Developmental Psychology, 47, 450-462.

Shah, S., & Mediratta, K. (2008). Negotiating reform: Young people's leadership in the educational arena. New 

Directions for Youth Development, (117), 43-59. 
Positive Disruption: Opportunity Reboot Model 133

Sinclair, M. F., Christenson, S. L., Evelo, D. L., & Hurley, C. M. (1998). Dropout prevention for youth with 

disabilities: Efficacy of a sustained school engagement procedure. Exceptional Children, 65(1), 7-21. 

Sinclair, M. F., Christenson, S. L., & Thurlow, M. L. (2005). Promoting school completion of urban secondary 

youth with emotional or behavioral disabilities. Exceptional Children, 71(4), 465-482.

Somers, C. L., Owens, D., & Piliawsky, M. (2008). Individual and social factors related to urban African 

American adolescents' school performance. The High School Journal, 1-11.

Stadtler, L., & Van Wassenhove, L. N. (2016). Coopetition as a paradox: Integrative approaches in a multicompany, cross-sector partnership. Organization Studies, 37(5), 655–685. 

https://doi.org/10.1177/01708406156220

Symonds, C., Schwartz, R., & Ferguson, R. F. (2011). Pathways to prosperity: Meeting the challenge of 

preparing young Americans for the 21st century. Cambridge, MA: Pathways to Prosperity Project. 

Retrieved from http://nrs.harvard.edu/urn-3:HUL.InstRepos:4740480

Syvertsen, A. K., Wu, C.-Y., & Sullivan, T. K. (2018). Unpacking the black box of youth development programs: 

An example from the field of youth conservation. Minneapolis: Search Institute. Manuscript 

submitted for publication. 

The White House Council for Community Solutions. (2012). Community solutions for opportunity youth. 

Washington, DC: Corporation for National & Community Service.

Thompson, L. A., & Kelly-Vance, L. (2001). The impact of mentoring on academic achievement of at-risk youth. 

Children and Youth Services Review, 23(3), 227-242.

VanDenBerg, J. E., & Grealish, E. M. (1996). Individualized services and supports through the wraparound 

process: Philosophy and procedures. Journal of Child and Family Studies, 5(1), 7-21. 

https://doi.org/10.1007/BF02234675

Walter, U. M., & Petr, C. G. (2011). Best practices in wraparound: a multidimensional view of the evidence. 

Social Work, 56(1), 73-80. https://doi.org/10.1093/sw/56.1.73

Wang, C. C. (1999). Photovoice: A participatory action research strategy applied to women's health. Journal of 

Women's Health, 8(2), 185-192.

Wang, C., & Burris, M. (1994). Empowerment through photo novella: Portraits of participation. Health 

Education Quarterly, 21(2), 171-186.
Positive Disruption: Opportunity Reboot Model 134

Watson, W., Sealey-Ruiz, Y., & Jackson, I. (2016). Daring to care: The role of culturally relevant care in 

mentoring Black and Latino male high school students. Race Ethnicity and Education, 19(5), 980–

1002. https://doi.org/10.1080/13613324.2014.911169

Werner, E. E., & Smith, R. S. (2001). Journeys from childhood to midlife: Risk, resilience, and recovery. Ithaca, 

NY: Cornell University Press.

Weigand, A., Manno, M. S., Leshnick, S., Treskon, L., Geckeler, C., Lewis-Charp, H., Sinicrope, C., Clark, M., & 

Nicholson, B. (2015). Adapting to local context: Findings from the YouthBuild Evaluation 

Implementation Study. New York, NY: MDRC. 
Appendices
In March 2017, the Corporation for National and Community Service Social Innovation Fund 

(SIF) approved Youthprise’s SIF Evaluation Plan for an implementation evaluation. The original 

plan included three major phases: (a) developmental evaluation; (b) capacity building; and, (c) 

pilot testing for the forthcoming impact evaluation. This work was due to be completed in July 

2018. The original plan was to incorporate and use the findings from the implementation 

evaluation to prepare a high-quality, contextually-informed Impact SIF Evaluation Plan, which 

was due to be submitted in July 2017. When SIF was defunded by the United States Congress, 

Youthprise worked closely with their SIF program officer to reimagine ways existing funds could 

be leveraged for maximum benefit. This resulted in three significant changes to the evaluation 

design. These changes, and the justification for each, are articulated below.

Change 1. Truncating the capacity-building phase of the implementation evaluation. The 

capacity-building phase of the evaluation was adapted to reflect the newly accelerated timeline 

of the preliminary impact data collection activities. While initially envisioned as a phase to build 

data capacity and streamline data management and collection systems within each community 

partner, the activities in the reimagined phase, by necessity, were instead refocused to highlight 

the current state of data collection and detail the supports that were needed from the 

evaluation team to create and implement the preliminary impact evaluation. These activities 

were critical to the success of our community partners in collecting all of the needed evaluation 

data in a manner that limited research participant attrition and promoted data quality and 

standards.

Change 2. Streamlining research questions. The revised evaluation plan converted the pilot test 

of the youth measures to a longitudinal assessment of those measures across two time points, 

now serving as the basis for the preliminary impact evaluation. The research questions were 

adapted to reflect the change in objectives. Youthprise’s approved SIF Evaluation Plan for the 

implementation evaluation contained four research questions, none that posed questions of 

impact on participants. Given the insertion of a preliminary impact evaluation, new research 

questions were required that addressed impact and attainment of outcomes named in the logic 

model. These questions serve as the organizing structure for Section 3 in this report.

Change 3. Adding more robust methods to test for preliminary impact. The approved SIF 

evaluation plan focused almost-exclusively on implementation, with some activities focused on 

pilot testing and troubleshooting: (a) the processes needed to track and collect survey data from 

opportunity youth, and (b) the psychometric properties of the survey metrics across diverse 

populations and over time. The accelerated timeline for measuring impact required us to forgo 

these pilot testing activities and, instead, roll out these data collection processes and metrics 

untested. Additional data collection support was provided by Search Institute to ensure the 

highest quality data was collected. As described in the Measures and Instruments subsection of 

the Quantitative Data Collection section in the Preliminary Impact section, the youth-level 

survey metrics were largely found to have high reliability.

In addition, two other changes were made to maximize the level of evidence generated with the 

existing funds. First, a qualitative impact narrative piece was added (see discussion of 

Qualitative Impact Narrative Activities subsection under Section 3). As a complement to the 

youth-level survey data, the qualitative impact piece were added to draw out rich, descriptive 

narratives from participants about their experience in programs utilizing the Opportunity 

Reboot model and to explore — more open-endedly — the impact their

Appendix A

Changes to the Approved Evaluation Design

Positive Disruption: Opportunity Reboot Model 136
participation has had on their lives. Second, the Minnesota Department of Employment and 

Economic Development (DEED) was contracted to provide (1) employment and wage data for 

study participants for four quarters following youths’ program participation, and (2) a 

comparison group from state-level data using propensity score matching (see Employment and 

Wage Attainment Study subsection under Section 3). The intent behind contracting these two 

services was to measure impact: the employment and wage data provides some evidence that 

program participation translated to success in the workforce; similarly, the comparison group 

provides some insight into whether program participants achieved better wage and 

employment outcomes than similar youth who did not participate in similar programs.

Positive Disruption: Opportunity Reboot Model 137
Below is an excerpt from the Opportunity Reboot Mapping Tool. This excerpt includes the 

instructions, as well as the portion of the tool focused on assessing Feature 1 (Positive 

Mentoring Relationships), Concept 1A (Mentoring Mindset) of the Opportunity Reboot model. 

The full version of the tool applied the same questions and format to the other model features 

and concepts.

Appendix B

Opportunity Reboot Mapping Tool 

The goal of this mapping tool is twofold: (1) Help Youthprise and the Opportunity Reboot 

evaluation team understand more deeply how features of this model are actually “lived out” in 

each of the community partners; (2) Help partners think through their implementation of the 

Opportunity Reboot model and set goals to improve programming. 

Each of the worksheet columns aims to solicit different information:

Column 1 à Provides an outline of each of the Opportunity Reboot strategies.

Column 2 à Answer: “What are you doing in your program to implement this strategy?” Here 

you list the specific activities or tactics being used by your program.

Column 3 à Reflect on Column 2, then answer: “How would you describe your program’s 

current alignment with this strategy?” Use this alignment scale: 0 = Not Aligned; 1 = Minimally 

Aligned; 2 = Mostly Aligned; 3 = Strongly Aligned

Column 4 à For strategies scored 0-2 in Column 3, answer: “What plans, if any, do you have to 

more fully implement this strategy? By when? What resources or assistance do you need, if any, 

from Youthprise to do this?”

Column 5 à Answer: “Once tactics identified in Column 4 are in place, how well aligned will the 

program be with this strategy?” Use the same alignment scale: 0 = Not Aligned; 1 = Minimally 

Aligned; 2 = Mostly Aligned; 3 = Strongly Aligned

Additional Activities à At the bottom of the worksheet for each feature, answer: “What 

additional activities in your program support this feature that are beyond what is captured in 

the Opportunity Reboot strategies?” 

Program Mapping Process to Measure Community Partner Fidelity to 

the Opportunity Reboot Model

Positive Disruption: Opportunity Reboot Model 138
Feature 1. Positive Mentoring Relationships

Concept 1A. Program staff engaged with youth adopt a mentoring mindset grounded in best practices of culturally responsive 

informal mentoring and relationship building.

Column 1

Opportunity Reboot

Strategies

Column 2

What are you doing in your 

program to implement this 

strategy? List your specific 

activities (or, tactics).

Column 3

Current Alignment*

Column 4

What plans, if any, do you 

have to more fully implement 

this strategy? By when? What 

do you need to do that?

Column 5 Targeted 

Alignment**

Strategy 1Ai. Program staff receive 

customized, ongoing technical 

assistance in informal mentoring 

best practices that supports 

continuous professional 

development.

Strategy 1Aii. Program staff develop 

and implement a plan for 

incorporating relevant informal 

mentoring best practices to 

maximize positive relationships 

between youth and adults (includes 

employers, when applicable).

What additional activities at this site support this feature but are beyond what is captured in the OR strategies?

We encourage you to get input from other staff, as needed, and type responses into this 

document.

Once the worksheet is complete, your evaluator and a representative from Youthprise will meet 

with you in-person or via conference call to discuss the completed worksheet. That conversation 

will seek to answer the following:

§ For Opportunity Reboot strategies identified in Column 4 on the worksheet, what plans are 

currently in place to move towards that goal?

§ How can Youthprise and your program work together to further strengthen this area and 

document the process for others?

§ What additional activities in your program are critical to this work but are outside of the 

current Opportunity Reboot features, concepts, and strategies? 

* How would you describe your program’s current alignment with this strategy? Use this alignment scale: 0 = Not Aligned; 1 = Minimally 

Aligned; 2 = Mostly Aligned; 3 = Strongly Aligned

** Once tactics identified in Column 4 are in place, rate your anticipated level of alignment with the strategy.

Opportunity Reboot 

Mapping Tool Excerpt

Positive Disruption: Opportunity Reboot Model 139
Feature 1. Positive Mentoring Relationships

Concept 1A. Program staff engaged with youth adopt a mentoring mindset grounded in best practices of culturally 

responsive informal mentoring and relationship building.

Partners

Strategies

S 1Ai. Program staff receive customized, 

ongoing technical assistance in informal 

mentoring best practices that supports 

continuous professional development.

S 1Aii. Program staff develop and implement a 

plan for incorporating relevant informal 

mentoring best practices to maximize positive 

relationships between youth and adults 

(includes employers, when applicable).

Initial Final Initial Final

Compass ● ● ● ●

GAP ● ● ● ●

MIGIZI ● ● ● ●

OYOD ● ● ● ●

Prior Crossing ● ● ● ●

Tri-City Bridges ● ● ● ●

Notes. GAP is short for the Guadalupe Alternative Program; OYOD is shorthand for Opportunity Youth of Duluth.

Appendix C

Opportunity Reboot Model Alignment Assessments

●

●

●

Key:

0 (Not Aligned) to 1 (Minimally Aligned)

1.5 to 2 (Mostly Aligned)

2+ to 3 (Strongly Aligned) 

Positive Disruption: Opportunity Reboot Model 140
Feature 1. Positive Mentoring Relationships

Concept 1B. Youth are prepared for mentoring experiences and develop skills to identify and engage informal mentors

Partners

Strategies

S 1Bi. Youth understand their 

rights, responsibilities and have 

appropriate expectations of a 

mentoring relationship.

S 1Bii. Youth learn how to 

identify informal mentors.

S 1Biii. Youth receive support 

from program staff in order to 

maximize the impact of 

informal mentors.

Initial Final Initial Final Initial Final

Compass ● ● ● ● ● ●

GAP ● ● ● ● ● ●

MIGIZI ● ● ● ● ● ●

OYOD ● ● ● ● ● ●

Prior Crossing ● ● ● ● ● ●

Tri-City Bridges ● ● ● ● ● ●

Notes. GAP is short for the Guadalupe Alternative Program; OYOD is shorthand for Opportunity Youth of Duluth.

●

●

●

Key:

0 (Not Aligned) to 1 (Minimally Aligned)

1.5 to 2 (Mostly Aligned)

2+ to 3 (Strongly Aligned) 

Positive Disruption: Opportunity Reboot Model 141
Feature 2. Individualized Goal Supports

Concept 2A. Programs assess youth’s needs and strengths

Partners

Strategies

S 2Ai. Program staff leverage 

internal and external resources 

to meet youth’s needs, such as 

healthcare, housing, and 

transportation.

S 2Aii. Program staff and youth 

engage in a visioning process 

about current and future goals 

that is strengths-based.

2 Aiii. Youth learn to identify 

their strengths, potential 

support networks, and 

resources.

Initial Final Initial Final Initial Final

Compass ● ● ● ● ● ●

GAP ● ● ● ● ● ●

MIGIZI ● ● ● ● ● ●

OYOD ● ● ● ● ● ●

Prior Crossing ● ● ● ● ● ●

Tri-City Bridges ● ● ● ● ● ●

Notes. GAP is short for the Guadalupe Alternative Program; OYOD is shorthand for Opportunity Youth of Duluth.

●

●

●

Key:

0 (Not Aligned) to 1 (Minimally Aligned)

1.5 to 2 (Mostly Aligned)

2+ to 3 (Strongly Aligned) 

Positive Disruption: Opportunity Reboot Model 142
Feature 2. Individualized Goal Supports

Concept 2B. Youth develop an individualized education and work plan in achieving their goals with assistance from 

program staff

Partners

Strategies

S 2Bi. Youth learn to set 

attainable and measurable goals 

within the plan that have shortterm, intermediate, and longterm milestones.

S 2Bii. Youth and program staff 

identify their roles and 

responsibilities within the plan.

S 2Biii.Opportunity youth learn 

to identify and address barriers, 

including modifying their plan if 

necessary.

Initial Final Initial Final Initial Final

Compass ● ● ● ● ● ●

GAP ● ● ● ● ● ●

MIGIZI ● ● ● ● ● ●

OYOD ● ● ● ● ● ●

Prior Crossing ● ● ● ● ● ●

Tri-City Bridges ● ● ● ● ● ●

Notes. GAP is short for the Guadalupe Alternative Program; OYOD is shorthand for Opportunity Youth of Duluth.

●

●

●

Key:

0 (Not Aligned) to 1 (Minimally Aligned)

1.5 to 2 (Mostly Aligned)

2+ to 3 (Strongly Aligned) 

Positive Disruption: Opportunity Reboot Model 143
Feature 2. Individualized Goal Supports

Concept 2C. Programs support youth in tracking progress, recognizing and learning from setbacks, and reaching their 

goals

Partners

Strategies

S 2Ci. Program staff 

motivate and encourage 

youth to attain shortterm, intermediate and 

long-term milestones.

S 2Cii. Program staff 

assist youth in revising 

and expanding goals 

when appropriate.

S 2Ciii. Program staff 

provide informal and 

formal assistance, 

including helping youth 

reflect on the overall 

goal-setting process and 

address barriers to 

achieving goals.

S 2Civ. Program staff 

and youth celebrate 

successes in attaining 

milestones.

Initial Final Initial Final Initial Final Initial Final

Compass ● ● ● ● ● ● ● ●

GAP ● ● ● ● ● ● ● ●

MIGIZI ● ● ● ● ● ● ● ●

OYOD ● ● ● ● ● ● ● ●

Prior Crossing ● ● ● ● ● ● ● ●

Tri-City 

Bridges ● ● ● ● ● ● ● ●

Notes. GAP is short for the Guadalupe Alternative Program; OYOD is shorthand for Opportunity Youth of Duluth.

●

●

●

Key:

0 (Not Aligned) to 1 (Minimally Aligned)

1.5 to 2 (Mostly Aligned)

2+ to 3 (Strongly Aligned) 

Positive Disruption: Opportunity Reboot Model 144
Feature 3. Coordinated Career Pathways Supports

Concept 3A. Programs create opportunities for career exploration that are grounded in connecting youth with employers 

in high growth and high demand local industries

Partners

Strategies

S 3Ai. Program staff facilitate 

exploration of career interest 

inventories and labor market 

information related to high 

growth- high demand 

occupations. information about 

occupations in demand.

S 3Aii. Program staff create 

linkages with specific employers 

in high demand industries to 

deepen youth participants’ 

understanding of career options 

including hands-on work 

experiences with employers.

S 3Aiii. Program staff arrange 

opportunities for youth to 

participate in on-site secondary, 

post-secondary and/or on-thejob training leading to 

industry/employer recognized 

credentials.

Initial Final Initial Final Initial Final

Compass ● ● ● ● ● ●

GAP ● ● ● ● ● ●

MIGIZI ● ● ● ● ● ●

OYOD ● ● ● ● ● ●

Prior Crossing ● ● ● ● ● ●

Tri-City Bridges ● ● ● ● ● ●

Notes. GAP is short for the Guadalupe Alternative Program; OYOD is shorthand for Opportunity Youth of Duluth.

●

●

●

Key:

0 (Not Aligned) to 1 (Minimally Aligned)

1.5 to 2 (Mostly Aligned)

2+ to 3 (Strongly Aligned) 

Positive Disruption: Opportunity Reboot Model 145
Feature 3. Coordinated Career Pathways Supports

Concept 3B. Programs assist youth with career preparation and planning

Partners

Strategies

S 3Bi. Program staff engage 

youth in occupational aptitude 

and basic skill assessments to 

develop and implement an 

agreed upon education and 

work plan that aligns with their 

career goals.

S 3Bii. Program staff provide 

youth with programming aimed 

at improving financial literacy 

and money management.

S 3Biii. Program staff prepare 

youth to successfully navigate a 

job search, application process, 

interviews, and other processes 

associated with their identified 

career path.

Initial Final Initial Final Initial Final

Compass ● ● ● ● ● ●

GAP ● ● ● ● ● ●

MIGIZI ● ● ● ● ● ●

OYOD ● ● ● ● ● ●

Prior Crossing ● ● ● ● ● ●

Tri-City Bridges ● ● ● ● ● ●

Notes. GAP is short for the Guadalupe Alternative Program; OYOD is shorthand for Opportunity Youth of Duluth.

●

●

●

Key:

0 (Not Aligned) to 1 (Minimally Aligned)

1.5 to 2 (Mostly Aligned)

2+ to 3 (Strongly Aligned) 

Positive Disruption: Opportunity Reboot Model 146
Feature 3. Coordinated Career Pathways Supports

Concept 3C. Programs provide culturally appropriate wraparound services that include multiple support structures pre- and 

post-employment to promote retention and encourage continuing education

Partners

Strategies

S 3Ci. Program staff provide training aimed at 

promoting critical job retention and academic 

success, including social-emotional 

competencies, work readiness and life skills.

S 3Cii. Program staff have regular contact with 

youth to discuss barriers to maintaining 

employment and academic success and identify 

specific resources to address those barriers.

Initial Final Initial Final

Compass ● ● ● ●

GAP ● ● ● ●

MIGIZI ● ● ● ●

OYOD ● ● ● ●

Prior Crossing ● ● ● ●

Tri-City Bridges ● ● ● ●

Notes. GAP is short for the Guadalupe Alternative Program; OYOD is shorthand for Opportunity Youth of Duluth.

●

●

●

Key:

0 (Not Aligned) to 1 (Minimally Aligned)

1.5 to 2 (Mostly Aligned)

2+ to 3 (Strongly Aligned) 

Positive Disruption: Opportunity Reboot Model 147
Feature 4. Impactful Cross-Sector Partnerships

Concept 4A. Programs regularly interact with diverse stakeholders to implement strategies and increase capacity to serve 

youth.

Partners

Strategies

S 4Ai. Cross-sector partners are 

identified and establish a clear 

vision and goals for 

collaboration.

S 4Aii. Cross-sector partners 

agree on specific roles and 

responsibilities as it relates to 

serving youth.

S 4Aiii. Cross-sector partners 

maintain and strengthen 

collaboration through regular 

interactions and share 

successes and lessons learned.

Initial Final Initial Final Initial Final

Compass ● ● ● ● ● ●

GAP ● ● ● ● ● ●

MIGIZI ● ● ● ● ● ●

OYOD ● ● ● ● ● ●

Prior Crossing ● ● ● ● ● ●

Tri-City Bridges ● ● ● ● ● ●

Notes. GAP is short for the Guadalupe Alternative Program; OYOD is shorthand for Opportunity Youth of Duluth.

●

●

●

Key:

0 (Not Aligned) to 1 (Minimally Aligned)

1.5 to 2 (Mostly Aligned)

2+ to 3 (Strongly Aligned) 

Positive Disruption: Opportunity Reboot Model 148
Recognizing that our youth participants are individuals who are drawn from vulnerable 

populations by virtue of their youth, socioeconomic disadvantage, and/or juvenile records, the 

evaluation team took several steps to ensure that participants’ identities and their data were 

protected.

1. All researchers who either worked directly with youth participants or performed analysis on 

data collected from them were required to undergo background checks - consisting of FBI 

criminal (with fingerprinting), Child Protection Background, and Predatory Offender checks.

2. Youth participants were assigned project IDs at the earliest opportunity after they enrolled 

in their respective programs; site partners were encouraged to use these project IDs in any 

communications regarding the youth, instead of their names.

3. We assigned our qualitative researchers to specific partner sites, so that youth only 

interacted with the same individuals during their time in their programs. This limited the 

number of research staff who interacted with (and could identify) participating youth, as 

well as provided an opportunity for the researchers and youth to establish familiarity, 

rapport, and trust. All qualitative staff were trained to attend to the emotional state of 

adults and youth participating in the research, to inquire about adverse reactions, and to 

provide opportunity for recovery.

4. A single individual on Search Institute’s evaluation staff was designated to ensure 

compliance with the study’s consent and assent requirements - in other words, only one 

person had access to the files linking all project IDs to individuals’ names. To safeguard 

against inadvertent or malicious unauthorized access to these files, the files were stored on 

a secure server, in a directory that could only be accessed by this individual. Emails 

containing files with personally identifying information were deleted (and the Trash directory 

purged) immediately after being downloaded and moved to this secure directory; USB flash 

drives were immediately wiped and formatted after the data they contained were saved to 

this directory.

5. Our partnership with DEED required the collection of birthdates and Social Security numbers 

from study youth, so that DEED could track their post-program exit wage and employment 

outcomes. The individual mentioned previously in (4) was also responsible for collecting and 

securely storing the birthdate and Social Security data. This individual was Search Institute’s 

primary contact with DEED, and was responsible for securely transmitting these data (via 

encrypted message sent in a secure environment-based client) to the department so that 

they could work on the analyses needed for their report.

Search Institute also retained the services of Chesapeake IRB (which became Advarra in 

November 2017), a commercial IRB accredited by the Association for the Accreditation of 

Human Research Protection Programs (AAHRPP), to conduct a thorough review of our research 

materials and protocols.

Appendix D

Human Research Participant Protections

Positive Disruption: Opportunity Reboot Model 149
A series of psychometric assessments were made on each of the metrics measuring the model 

features and the short-term youth outcomes in the Opportunity Reboot Youth Survey. The 

following were performed on all metrics that were assessed at both baseline and endline youth 

surveys, with three exceptions: Responsible Decision Making, which is a single-item: “I think 

about what might happen before making a decision”; Outside of Program Positive Mentoring 

Relationships, which is also a single item: “During my time at [name of program inserted], I 

formed a relationship with an adult outside of the program who supports my growth. This might 

include, for example, a boss, mentor, neighbor, church members”; and Coordinated Career 

Pathways Supports, which is made up of two items (a minimum of three items is required for 

these psychometric assessments): “At [program], I had opportunities to explore education and 

career options” and “[Program] helped me make a plan to reach my career or education goals.”

Cronbach’s alpha (α) was calculated to determine each metric’s internal reliability – how closely 

related the items making up the measures are, or how well the items are at measuring the same 

construct. Cronbach’s alpha ranges from 0 to 1; α coefficients that are greater than or equal to 

.70 and generally deemed acceptable.

Confirmatory Factor Analysis (CFA) models were then run to determine the measurement 

properties of these constructs. CFA models are particularly useful for showing each item’s 

relative contribution to the construct, and whether certain items are more influential than 

others. Standardized factor loadings are reported in this document, which range from 0 to 1; 

factor loadings greater than .40 are deemed acceptable.

All measures were specified as two-factor CFA models, where each factor represents each time 

point (T1 = baseline Opportunity Reboot Youth Survey; T2 = endline Opportunity Reboot Youth 

Survey); with the one exception of Individualized Goal Supports, which was only assessed at T2. 

The r column reports T1-T2 Pearson’s correlation coefficients. Each model comes with a set of 

model fit indices, which provides an indication of how good the overall construct is, based on 

the collected data:

χ2 (Chi-square): Lower values (and higher p values) indicate better fit. Non-significant p

values are ideal, although rarely seen; hence it is rarely helpful for making decisions about 

model fit. It is typically reported due to convention. As this is antithetical to conventional 

statistical rules-of-thumb, it may be helpful for some to think of the χ2 test of model fit as a 

“badness-of-fit” test (where p ≤ .05 is undesirable). The model’s number of degrees of 

freedom (df) are reported, also due to convention.

RMSEA (Root Mean Square Error of Approximation): Lower values indicate better fit. Values 

lower than or equal to .08 are acceptable; values lower than or equal to .05 are ideal. 

CFI (Comparative Fit Index): Larger values indicate better fit – ideally, .90 or greater.

TLI (Tucker Lewis Index): Larger values indicate better fit – ideally, .90 or greater.

SRMR (Standardized Root Mean Square Residual): Smaller values indicate better fit – ideally 

lower than or equal to .05.

Appendix E

Psychometric Properties of Opportunity Reboot Metrics

Positive Disruption: Opportunity Reboot Model 150
Each of the model fit indices discussed above are based on a unique sets of assumptions –

therefore, each index has different strengths and weaknesses. Consequently, any given CFA

model’s fit cannot be properly assessed by evaluating just one or two of the indices – overall fit 

assessment requires a holistic approach. Please note that the determination of overall fit 

assessment entails some subjectivity: (a) it is sometimes the case that some of a model’s indices 

fall very close to the rule-of-thumb thresholds; and, (b) when comparing two or more models 

(which is what we do in this document), the models are often assessed on their fit relative to 

each other.

Positive Disruption: Opportunity Reboot Model 151

Table E1

CFA Model for Within Program Mentoring

Items

Standardized

Factor Loadings rT1-T2

T1 T2

How often do these people show you that you matter to them? .83 .84 .12

How often do these people push you to be your best? .88 .77 .09

How often do these people help you get things done? .91 .85 .11

How often do these people listen to your ideas and take them 

seriously? .89 .87 .28**

How often do these people connect you with new people or services 

that help you? .87 .78 .09

α .94 .92 .50***

Model Fit Indices: χ2=65.69, df=29, p=0.000 ; RMSEA=0.08; CFI=.98 ; TLI=.96 ; SRMR=0.03

Note. *p ≤ .05. **p ≤ .01. ***p ≤ .001.

Table E2

CFA Model for Individualized Goal Supports

Items

Standardized

Factor Loadings rT1-T2

T1 T2

When I am at [program], staff work to meet my personal needs That 

is, staff figure out what your specific needs and interests are and then 

help you.

— .80 —

[Program] helped me identify what I am good at and areas where I 

may need help. — 74 —

The staff at [program] checked in with me regularly about my goals. — .80 —

The staff at [program] want me to reach the goals I set. — .89 —

α — .88 —

Model Fit Indices: χ2=9.29, df=2, p=0.010 ; RMSEA=0.14; CFI=.98 ; TLI=.94 ; SRMR=0.02

Note. *p ≤ .05. **p ≤ .01. ***p ≤ .001.
Table E3

CFA Model for Positive Self-Worth

Items

Standardized

Factor Loadings rT1-T2

T1 T2

I like myself. .92 .90 .07

I am glad I am me. .92 .92 .32**

I believe in myself. .80 .90 .07

α .91 .93 .65***

Model Fit Indices χ2=10.23, df=5, p=0.069 ; RMSEA=0.07 ; CFI=1.00 ; TLI=.98 ; SRMR=0.02

Note. *p ≤ .05. **p ≤ .01. ***p ≤ .001.

Table E4

CFA Model for Positivity in the Face of Challenge

Items

Standardized

Factor Loadings rT1-T2

T1 T2

I try to stay positive even when I am facing challenges. .73 .73 .17*

When I fail at something, I tell myself there is something positive I can 

learn from it. .83 .82 -.10

When something is hard to learn, I tell myself I can get better at it. .79 .85 .34***

Α .82 .84 .55***

Model Fit Indices χ2=5.00, df=5, p=0.416 ; RMSEA=0.00 ; CFI=1.00 ; TLI=1.00 ; SRMR=0.02

Note. *p ≤ .05. **p ≤ .01. ***p ≤ .001.

Table E5

CFA Model for Goal-Setting Skills

Items

Standardized

Factor Loadings rT1-T2

T1 T2

I set goals for myself. .70 .73 .15

I stay focused on reaching my top goals. .79 .81 .10

Once I set a goal for myself, I try to find out how to reach my goal. .79 .81 .11

I try to learn skills that can help me reach my goal. .76 .75 .15

If something goes wrong, I try to figure out another way to reach my 

goal. .78 .76 .27**

If I find it hard to reach a goal, I break that goal down into smaller steps. .65 .62 .36***

Α .88 .88 .56***

Model Fit Indices χ2=61.93, df=47, p=0.071 ; RMSEA=0.04 ; CFI=.99 ; TLI=.98 ; SRMR=0.03

Note. *p ≤ .05. **p ≤ .01. ***p ≤ .001.

Positive Disruption: Opportunity Reboot Model 152
Table E6

CFA Model for Future Orientation

Items

Standardized

Factor Loadings rT1-T2

T1 T2

I have hope for success in my future. .79 .70 .41***

When I make a decision, I consider the effect it will have on my future. .88 .91 -.45

I think about who I will be in 5 years. .73 .71 .12

α .84 .82 .59***

Model Fit Indices χ2=1.93, df=5, p=0.859 ; RMSEA=0.00 ; CFI=1.00 ; TLI=1.02 ; SRMR=0.02

Note. *p ≤ .05. **p ≤ .01. ***p ≤ .001.

Table E7

CFA Model for Civic Efficacy

Items

Standardized

Factor Loadings rT1-T2

T1 T2

I believe I can have a positive impact on someone else's life. .64 .75 .35***

I believe I can improve my community by doing things like volunteering, or 

helping a neighbor. .82 .89 -.11

I can help others by involving myself in political activities (examples may 

include voting, participating in marches or protests, or writing to public 

officials.) 

.71 .60 .31***

α .77 .78 .30***

Model Fit Indices χ2=8.18, df=5, p=0.147 ; RMSEA=0.06 ; CFI=.99 ; TLI=.97 ; SRMR=0.04

Note. *p ≤ .05. **p ≤ .01. ***p ≤ .001.

Table E8

CFA Model for Self-Awareness

Items

Standardized

Factor Loadings rT1-T2

T1 T2

I know what matters most to me. .72 .79 .04

I can list my strengths and weaknesses. .77 .67 .25**

I know who I am and what I believe in. .85 .81 .14

α .83 .80 .37***

Model Fit Indices χ2=1.32, df=5, p=0.932 ; RMSEA=0.00 ; CFI=1.00 ; TLI=1.03 ; SRMR=0.01

Note. *p ≤ .05. **p ≤ .01. ***p ≤ .001.

Positive Disruption: Opportunity Reboot Model 153
Table E9

CFA Model for Relationship Skills

Items

Standardized

Factor Loadings rT1-T2

T1 T2

I listen to other people's opinions, even if I disagree. .60 .47 .37***

When I talk to someone, I think about how I would feel if I were in their 

place. .52 .43 .28***

I get along with other people. .63 .63 .39***

I can share what I am feeling with other people. .56 .69 .41***

I am good at making friends who have a positive influence on me. .74 .67 .14

α .75 .71 .49***

Model Fit Indices χ2=35.26, df=29, p=0.196 ; RMSEA=0.03 ; CFI=.99 ; TLI=.98 ; SRMR=0.05

Note. *p ≤ .05. **p ≤ .01. ***p ≤ .001.

Table E10

CFA Model for Resource Identification Skills

Items

Standardized

Factor Loadings rT1-T2

T1 T2

I can find the answers to questions I have about services available. .89 .81 .38***

I can find services that can help me. .84 .82 .12

When I have a problem finding the services I need, I know who to talk 

to. .83 .82 .15

α .89 .85 .32***

Model Fit Indices χ2=3.57, df=5, p=0.612 ; RMSEA=0.00 ; CFI=1.00 ; TLI=1.01 ; SRMR=0.02

Note. *p ≤ .05. **p ≤ .01. ***p ≤ .001.

Positive Disruption: Opportunity Reboot Model 154
Young people are more likely to grow up successfully when they experience developmental 

relationships with important people in their lives. Developmental relationships are close 

connections through which young people discover who they are, cultivate abilities to shape 

their own lives, and learn how to engage with and contribute to the world around them. Search 

Institute has identified five elements — expressed in 20 specific actions — that make 

relationships powerful in young people’s lives.

Appendix F

Developmental Relationships Framework

Elements Actions Definitions

Express Care

Show me that I 

matter to you.

Be dependable Be someone I can trust.

Listen Really pay attention when we are together.

Believe in me Make me feel known and valued.

Be warm Show me you enjoy being with me.

Encourage Praise me for my efforts and achievements.

Challenge Growth

Push me to keep 

getting better.

Expect my best Expect me to live up to my potential.

Stretch Push me to go further.

Hold me accountable Insist I take responsibility for my actions.

Reflect on failures Help me learn from mistakes and setbacks.

Provide Support

Help me complete 

tasks and achieve 

goals.

Navigate Guide me through hard situations and systems.

Empower Build my confidence to take charge of my life.

Advocate Stand up for me when I need it.

Set boundaries Put in place limits that keep me on track.

Share Power

Treat me with 

respect and give 

me a say.

Respect me Take me seriously and treat me fairly.

Include me Involve me in decisions that affect me.

Collaborate Work with me to solve problems.

Let me lead Create opportunities for me to take action.

Expand Possibilities

Connect me with 

people and places 

that broaden my 

world.

Inspire Inspire me to see possibilities for my future.

Broaden horizons Expose me to new ideas, experiences, places. 

Connect Introduce me to people who can help me grow.

Note. Relationships are, by definition, bidirectional, with each person giving and receiving. So each person in a strong relationship both engages 

in and experiences each of these actions. However, for the purpose of clarity, this framework is expressed from the perspective of one young 

person.

Copyright © 2017 Search Institute, Minneapolis, MN. www.search-institute.org. May be reproduced for nonprofit, educational use

Positive Disruption: Opportunity Reboot Model 155
© Copyright 2020 | Search Institute